TW202344051A - Temporal initialization points for context-based arithmetic coding - Google Patents

Abstract

A method includes determining one or more context values for at least one context used for encoding or decoding a current slice or picture, determining that a buffer for storing sets of temporal initialization points from two or more slices or pictures for context-based arithmetic coding is full, determining a first set of temporal initialization points associated with a slice or picture, from among the two or more slices or pictures, based on at least one of a slice type, a temporal identification value, or a quantization parameter (QP) value of the slice or picture, removing the first set of temporal initialization points that is associated with the slice or picture, and storing a second set of temporal initialization points associated with the current slice or picture, wherein the second set of temporal initialization points are based on the determined one or more context values.

Description

Temporal initialization point for context-based arithmetic decoding

This patent application claims the priority of U.S. Provisional Application No. 63/268,844 filed on March 3, 2022 and U.S. Provisional Application No. 63/362,118 filed on March 29, 2022, and is therefore incorporated by reference. The entire contents of each of these applications are incorporated herein by reference.

The content of this case is about video encoding and video decoding.

Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), notebook or desktop computers, tablets, e-book readers, digital Cameras, digital recording equipment, digital media players, video game devices, video game consoles, cellular or satellite wireless phones, so-called "smartphones", video conferencing equipment, video streaming equipment, etc. Digital video equipment implements video decoding technologies such as via MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), Video coding techniques described in the standards defined by ITU-T H.265/High Efficiency Video Coding (HEVC), ITU-T H.265/Versatile Video Coding (VVC), and extensions to such standards , and proprietary video transcoders/formats such as AOMedio Video 1 (AV1) developed by the Alliance for Open Media. Video equipment can more efficiently send, receive, encode, decode and/or store digital video information by implementing such video decoding technology.

Video coding techniques include spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove inherent redundancy in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be divided into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and/or coding units. code node. Video blocks in a slice of intra-coded (I) pictures are coded using spatial prediction relative to reference samples in neighboring blocks in the same picture. Video blocks in an inter-frame decoded (P or B) slice of a picture may use spatial prediction relative to reference samples in neighboring blocks in the same picture, or temporal prediction relative to reference samples in other reference pictures. The picture may be called a frame, and the reference picture may be called a reference frame.

Generally, this document describes techniques for determining initialization points for one or more contexts used in context-based arithmetic coding (eg, context self-adjusting binary arithmetic coding (CABAC)). The initialization point may be considered the starting point of one or more contexts, and may include one or more context states, windowing or rate self-adjustment parameters, and other parameters used in arithmetic decoding operations.

In some examples, the initialization point for one or more contexts of the video material in the current slice or current picture may be based on the initialization point of one or more contexts of the video material in the previous picture. Such initialization points are called temporal initialization points.

This article describes an example technique for determining the temporal initialization point of the current slice or picture. In some examples, a video coder (eg, a video transcoder or a video decoder) may utilize a time identifier (ID) value and/or a quantization parameter (QP) value to determine the time initialization point. In some examples, the video decoder may determine the initialization point for the current slice based on the initialization point of the previous slice at the corresponding location.

Due to memory size limitations, there may be a limit to how many time initialization points can be stored. When a new time initialization point set is to be stored, the already stored time initialization point set can be removed. This document describes example techniques for memory management of inserting and removing time initialization points based on time stamp values and/or QP values that balance memory size constraints while ensuring timely access The decoded time initialization point is kept in the buffer.

In one example, the content of this case describes a method of processing video data. The method includes: determining one or more context values for at least one context used to encode or decode the current slice or picture; A buffer is full for a set of temporal initialization points of one or more slices or pictures for context-based arithmetic decoding, where each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures. Associated, and including one or more temporal initialization points; determined from the two or more slices or pictures based on at least one of the slice type, temporal marker value, or quantization parameter (QP) value of the slice or picture The first time initialization point set associated with the slice or picture; remove the first time initialization point set associated with the slice or picture from the buffer; and store the current slice or picture in the buffer. A second set of temporal initialization points associated with the picture, wherein the second set of temporal initialization points is based on the determined one or more contextual values.

In one example, this case content describes a device for processing video data, the device includes: a buffer configured to store a set of temporal initialization points from two or more slices or pictures for use based on Arithmetic decoding of context; and processing circuitry coupled to the buffer, the processing circuitry configured to: determine one or more context values for at least one context for encoding or decoding a current slice or picture; determine for A buffer that stores sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full, where each set of temporal initialization points is associated with a set of temporal initialization points in the two or more slices or pictures. The slices or pictures are associated and include one or more temporal initialization points; based on at least one of the slice type, the temporal identification value or the quantization parameter (QP) value of the slice or picture, from the two or more slices or Determine the first time initialization point set associated with the slice or picture in the picture; remove the first time initialization point set associated with the slice or picture from the buffer; and store the first time initialization point set associated with the slice or picture in the buffer. A second set of temporal initialization points associated with the current slice or picture, wherein the second set of temporal initialization points is based on the determined one or more context values.

In one example, the present disclosure describes a computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to: determine whether to encode the current slice or picture or One or more context values of at least one context being decoded; deciding that a buffer used to store a set of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full, where each temporal The set of initialization points is associated with a slice or picture of the two or more slices or pictures and includes one or more temporal initialization points; based on a slice type, a time stamp value or a quantization parameter (QP) value of the slice or picture At least one of: determining the first time initialization point set associated with the slice or picture from the two or more slices or pictures; removing the first time initialization point set associated with the slice or picture from the buffer A first set of temporal initialization points; and in the buffer, store a second set of temporal initialization points associated with the current slice or picture, wherein the second set of temporal initialization points is based on the determined one or more context values.

In one example, the content of this case describes a device for processing video data, which device includes: a unit for determining one or more context values of at least one context used to encode or decode the current slice or picture; Determining which buffer is full for storing sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding, where each set of temporal initialization points is associated with the two or more Slices or pictures of more than one slice or picture are associated and include one or more temporal initialization points; for based on at least one of the slice type, time stamp value or quantization parameter (QP) value of the slice or picture, from The unit that determines the first time initialization point set associated with the slice or picture among the two or more slices or pictures; used to remove the first time initialization point associated with the slice or picture from the buffer a unit for a set of initialization points; a unit for storing in the buffer a second set of temporal initialization points associated with the current slice or picture, wherein the second set of temporal initialization points is based on the determined one or more context values .

The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, features and advantages will become apparent from the description, drawings and claims.

In video coding (eg, context-based arithmetic coding), a video transcoding decoder (eg, a video transcoder or a video decoder) utilizes an initialization point to initialize the current picture or slice. For example, for context-based arithmetic coding, which includes context self-adjusting binary arithmetic coding (CABAC), an initialization point may be used for each context. The initialization point may include one or more context states, window or rate self-adjustment parameters, and other parameters required for arithmetic decoding operations.

Initialization points can be predefined. However, in some instances, the video decoder may utilize temporal initialization points in addition to or instead of using predefined initialization points. The temporal initialization point may represent an initialization point of a previous picture in the decoding order that was used to determine (eg, select) the initialization point of the current picture. For example, the temporal initialization point may be one or more context values of at least one context used to encode or decode the current picture, or derived based on the one or more context values that are subsequently used to initialize subsequent pictures. The context value of at least one context.

There may be some issues with using time initialization points. For example, a buffer storing temporal initialization points may have a limited size, and the video decoder may remove a set of temporal initialization points (eg, one or more temporal initialization points) to allow storage of another set of temporal initialization points. This document describes an example technique for deciding which set of temporal initialization points to remove in a manner that ensures that the retained set of temporal initialization points facilitates efficient encoding and decoding.

In one or more examples, the video coder stores a corresponding set of temporal initialization points for a slice or picture in a buffer (eg, after decoding the slice or picture). The set of temporal initialization points may include one or more initialization points as described above, and the initialization points may be context values of the context, or derived according to context values of the context used to decode the current slice or picture.

For example, the video decoder stores in a buffer a first set of temporal initialization points associated with a first slice or picture (e.g., after decoding the first slice or picture), and stores in the buffer a set of initialization points associated with the first slice or picture. A second set of temporal initialization points associated with the second slice or picture (eg, after decoding the second slice or picture), and so on. In this way, the buffer stores a set of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding. Each set of temporal initialization points is associated with a slice or picture within two or more slices or pictures.

Each slice or picture may have a temporal identification (ID) value and/or a quantization parameter (QP) value. Each slice can also be associated with a slice type. The temporal ID value of the previous picture may indicate whether the previous picture can be used for inter-frame prediction of the current picture. For example, only pictures with the same or lower temporal ID value as the current picture can be used for inter-frame prediction of the current picture. In this way, pictures with temporal ID values higher than a certain threshold can be dropped or not decoded from the bitstream based on bandwidth availability or processing power, without affecting the processing of temporal IDs with temporal IDs smaller than the threshold. The ability to decode images. The QP value indicates the amount of quantization applied in the encoding process.

In one or more instances, if a buffer used to store a set of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full, the video decoder may based on the slice type , at least one of a timestamp value or a quantization parameter (QP) value of a slice or picture, determining (e.g., identifying) from the buffer the slice or picture associated with the slice or picture of the two or more slices or pictures. Initialize the point collection at the first time. As an example, the video decoder may determine (e.g., identify) from the buffer a first time associated with a slice or picture of two or more slices or pictures that has at least one of the following: Initialization point set: minimum time stamp value or quantization parameter (QP) value.

In some examples, the first time initialization point set may be associated with a slice whose slice type is different from the slice type of the current slice to be encoded or decoded. In some instances, multiple sets of time initialization points may be stored for a slice type. In such instances, the video decoder may determine packets for a set of temporal initialization points associated with slices of the same slice type as the current slice. The video decoder may determine the first temporal initialization point set from packets of temporal initialization point sets associated with slices of the same slice type as the current slice (e.g., associated with the slice or picture with the smallest temporal stamp value or QP value those time initialization points of the connection).

The video decoder may remove the first set of initialization points associated with the slice or picture from the buffer. The video decoder may store a second set of temporal initialization points associated with the current slice or picture in the buffer based on the determined one or more context values of the current slice or picture.

In one or more examples, the video decoder may determine (eg, select) a set of temporal initialization points stored in a buffer and initialize subsequent slices or pictures based on the determined set of temporal initialization points. One or more context values of at least one context encoded or decoded. In order to determine the set of temporal initialization points, the video decoder may determine the slice or picture whose time stamp value and/or QP value is closest to the time stamp value and/or QP value of the subsequent slice or picture, and in some cases, has the same Slice type. For example, if the QP value of a subsequent picture is X, the video decoder can determine which set of temporal initialization points is associated with the picture with a QP value of The video decoder can select a determined set of time initialization points. The video decoder can perform context-based arithmetic encoding or decoding of subsequent slices or pictures.

There may be a coding gain by removing the set of temporal initialization points associated with the slice or picture with the smallest temporal stamp value or QP value. In one or more instances, pictures with smaller temporal stamp values and/or QP values tend to have fewer transform coefficients than pictures with higher temporal stamp values and/or QP values tend to have fewer transform coefficients. due to having more transform coefficients (e.g. due to less quantization). If there is a relatively high number of transform coefficients (e.g., due to a low QP value), the context value of the context can be updated relatively quickly and used for the remainder of the slice (e.g., it can be at the beginning of slice decoding The context is adjusted and the rest of the slice will be decoded efficiently).

However, if there is a relatively small number of transform coefficients (eg due to a high QP value), the context value of the context is updated relatively slowly. That is, slices with higher QP values have fewer transform coefficients and context self-adjustment will be slower, so fewer blocks of the slice will be coded efficiently.

In one or more instances, the set of temporal initialization points stored in the buffer may be closer (eg, after some mapping or scaling) to the context values used to code subsequent slices or pictures. Therefore, if the set of time initialization points stored in the buffer is associated with a picture with a higher time stamp value or QP value, then these time initialization point sets will be more likely to be used for a picture with a higher time stamp value or QP value. Pictures to follow. In other words, in some examples, temporal initialization points may be used to achieve higher coding efficiency gains for slices or pictures with higher temporal stamp values or QP values. Therefore, by storing the temporal initialization points of pictures with higher temporal stamp values or QP values, and removing the temporal initialization points of pictures with lower temporal stamp values and QP values (due to buffer size limitations), the example technique can Improve decoding efficiency without requiring large size buffers.

There may be other issues with using time initialization points. As mentioned before, each picture can be associated with an ID value. However, if the temporal initialization point of the picture with the higher temporal ID value would be used to determine the initialization point of the current picture, there may be an error because the temporal initialization point of the picture with the higher temporal ID value may not be available.

As described in more detail, this document describes an example technique that uses a picture's temporal ID value to determine whether subsequent pictures can utilize the picture's temporal initialization point. In this way, the likelihood that the video decoder depends on an unavailable time initialization point is reduced.

In some instances, the temporal initialization point of a picture may differ from slice to slice. In this case, when the temporal initialization point of a slice of the previous picture is available, the temporal initialization point of one slice of the current picture may overwrite the temporal initialization point of one slice of the previous picture. This case describes example techniques to minimize the negative impact of rewriting time initialization point information and ensure that the correct time initialization point is stored.

FIG. 1 is a block diagram illustrating an example video encoding and decoding system 100 that may perform the techniques described herein. The technology in this case is generally directed at decoding (encoding and/or decoding) video data. Generally speaking, video data includes any data used to process videos. Accordingly, video data may include original, unencoded video, encoded video, decoded (eg, reconstructed) video, and video relay data (such as signaling data).

As shown in FIG. 1 , in this example, system 100 includes source device 102 that provides encoded video material to be decoded and displayed by target device 116 . In particular, source device 102 provides video material to target device 116 via computer-readable media 110 . Source device 102 and target device 116 may include any of a wide range of devices, including desktop computers, notebook computers (i.e., laptops), mobile devices, tablets, set-top boxes, phones Mobile phones (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, etc. In some cases, source device 102 and target device 116 may be equipped for wireless communications, and thus may be referred to as wireless communications devices.

In the example of FIG. 1, source device 102 includes video source 104, memory 106, video transcoder 200, and output interface 108. Target device 116 includes input interface 122, video decoder 300, memory 120, and display device 118. According to the present case, the video transcoder 200 of the source device 102 and the video decoder 300 of the target device 116 may be configured to apply one or more of the context-based arithmetic decoding methods used for the video material of the current picture or slice. Techniques for context time initialization points. Thus, source device 102 represents an instance of a video encoding device, and target device 116 represents an instance of a video decoding device. In other examples, the source device and target device may include other components or arrangements. For example, source device 102 may receive video material from an external video source, such as an external camera. Likewise, target device 116 may be connected to an external display device rather than including an integrated display device.

The system 100 shown in Figure 1 is only one example. In general, any digital video encoding and/or decoding device can perform one or more context techniques used in context-based arithmetic decoding of the video data of the current picture or slice. Source device 102 and target device 116 are merely examples of the type of decoding device in which source device 102 generates decoded video material for transmission to target device 116 . This case refers to a "decoding" device as a device that performs the decoding (encoding and/or decoding) of data. Therefore, video transcoder 200 and video decoder 300 represent examples of decoding devices, particularly a video transcoder and a video decoder, respectively. In some examples, source device 102 and target device 116 may operate in a substantially symmetrical manner such that each of source device 102 and target device 116 includes components for video encoding and video decoding. Accordingly, the system 100 may support one-way video transmission or two-way video transmission between the source device 102 and the target device 116, for example, for video streaming, video replay, video broadcasting, or video telephony.

Generally speaking, video source 104 represents a source of video data (i.e., raw, undecoded video data) and a sequential series of pictures (also referred to as "frames") that provide the video data to video transcoder 200 ), the video transcoder 200 encodes the data for the picture. Video source 104 of source device 102 may include a video capture device (such as a video camera, a video archive containing previously captured raw video, and/or a video feed interface) to receive video from a video content provider. As a further alternative, video source 104 may generate computer graphics-based material as the source video, or a combination of live video, archived video, and computer-generated video. In each case, video transcoder 200 encodes captured, pre-captured, or computer-generated video material. Video transcoder 200 may rearrange pictures from the order in which they are received (sometimes referred to as "display order") to the decoding order for decoding. Video transcoder 200 may generate a bit stream including encoded video data. Source device 102 may then output the encoded video data onto computer-readable media 110 via output interface 108 for reception and/or retrieval by, for example, input interface 122 of target device 116 .

Memory 106 of source device 102 and memory 120 of target device 116 represent common memories. In some examples, the memory 106 and the memory 120 may store original video data, for example, the original video from the video source 104 and the original, decoded video data from the video decoder 300 . Additionally or alternatively, memory 106, memory 120 may store software instructions executable by, for example, video transcoder 200 and video decoder 300, respectively. Although memory 106 and memory 120 are shown separately from video transcoder 200 and video decoder 300 in this example, it should be understood that video transcoder 200 and video decoder 300 may also include Internal memory that serves a functionally similar or equivalent purpose. In addition, the memory 106 and the memory 120 may store encoded video data, such as the output from the video transcoder 200 and the input to the video decoder 300 . In some examples, a portion of memory 106, memory 120 may be allocated as one or more video buffers, for example, to store raw, decoded and/or encoded video data.

Computer-readable media 110 may represent any type of media or device capable of transmitting encoded video material from source device 102 to target device 116 . In one example, computer-readable media 110 represents a communication medium that enables source device 102 to instantly send encoded video data directly to target device 116 , such as via a radio frequency network or a computer-based network. The output interface 108 can modulate the transmission signal including the encoded video data, and the input interface 122 can demodulate the received transmission signal according to a communication standard (such as a wireless communication protocol). The communication media may include any wireless or wired communication media, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network (such as the Internet). Communication media may include routers, switches, base stations, or any other equipment that may be used to facilitate communication from source device 102 to target device 116 .

In some examples, source device 102 may output encoded data from output interface 108 to storage device 112 . Similarly, target device 116 may access encoded data from storage device 112 via input interface 122 . Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray Disc, DVD, CD-ROM, flash memory, volatile memory, or non-volatile memory. memory, or any other suitable digital storage medium for storing encoded video data.

In some examples, source device 102 may output the encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102 . Target device 116 may access stored video data from file server 114 via streaming or downloading.

File server 114 may be any type of server device capable of storing encoded video data and sending the encoded video data to target device 116 . File server 114 may represent a network server (e.g., for a website), a server configured to provide file transfer protocol services such as File Transfer Protocol (FTP) or One-Way File Transfer (FLUTE) protocol, content delivery Network (CDN) devices, Hypertext Transfer Protocol (HTTP) servers, Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) servers, and/or Network Attached Storage (NAS) devices. File server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as Dynamic Self-Adapting Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real-Time Streaming Protocol (RTSP), HTTP Dynamic Streaming etc.

Target device 116 may access encoded video data from file server 114 via any standard data connection, including an Internet connection. This may include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., Digital Subscriber Line (DSL), cable data) suitable for accessing the encoded video data stored on file server 114 machine, etc.) or a combination of both. Input interface 122 may be configured to operate in accordance with any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114 or other such protocols for retrieving media data.

Output interface 108 and input interface 122 may represent a wireless transmitter/receiver, a modem, a wired networking component (e.g., an Ethernet network card), a wireless communications component operating in accordance with any of the various IEEE 802.11 standards, or other Solid parts. In instances where the output interface 108 and the input interface 122 include wireless components, the output interface 108 and the input interface 122 may be configured to communicate in accordance with cellular communications standards such as 4G, 4G-LTE (Long Term Evolution), LTE-Advanced, 5G, etc. Transmit data (such as encoded video data). In some instances where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to operate in accordance with other wireless standards, such as the IEEE 802.11 specification, the IEEE 802.15 specification (eg, Zigbee ^™ ), ^{Blue BudTM} standard, etc.) to transmit data (such as encoded video data). In some examples, source device 102 and/or target device 116 may include respective system-on-chip (SoC) devices. For example, source device 102 may include an SoC device to perform functions attributed to video transcoder 200 and/or output interface 108 and target device 116 may include an SoC device to perform functions attributed to video decoder 300 and/or input interface 122 functions.

The technology described in this case can be applied to support video decoding in any of a variety of multimedia applications, such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, and Internet streaming video transmissions, such as dynamic self-adjusting streaming via HTTP (DASH), digital video encoded on data storage media, decoding of digital video stored on data storage media, or other applications.

The input interface 122 of the target device 116 receives the encoded video bit stream from the computer-readable medium 110 (eg, communication media, storage device 112, file server 114, etc.). The encoded video bitstream may include signaling information defined by video transcoder 200 and also used by video decoder 300, such as information describing video blocks or other coded units (e.g., slices, pictures, Syntax elements that represent properties of a group of pictures, sequence, etc.) and/or values for the processing of video blocks or other decoded units. The display device 118 displays decoded pictures of the decoded video data to the user. Display device 118 may represent any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

Although not shown in Figure 1, in some examples, video transcoder 200 and video decoder 300 may be integrated with an audio encoder and/or audio decoder, respectively, and may include appropriate MUX-DEMUX units. or other hardware and/or software to handle multiplexing of both audio and video included in a common data stream.

Video transcoder 200 and video decoder 300 may each be implemented as any of a variety of suitable encoder and/or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific Integrated circuit (ASIC), field programmable gate array (FPGA), individual logic, software, hardware, firmware, or any combination thereof. When technology is implemented partially in software, a device may store instructions for the software in a suitable, non-transitory computer-readable medium, and execute the instructions in hardware using one or more processors for execution technology in this case. Each of the video transcoder 200 and the video decoder 300 may be included in one or more encoders or decoders, either of which may be integrated into a combination of respective devices. part of the encoder/decoder (CODEC). Devices including video transcoder 200 and/or video decoder 300 may include integrated circuits, microprocessors, and/or wireless communication devices, such as cellular phones.

Video transcoder 200 and video decoder 300 may perform coding according to video coding standards such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC) or extensions thereof such as multi-viewpoint and/or scalable video coding. extension)) to operate. Alternatively, video transcoder 200 and video decoder 300 may operate according to other proprietary or industry standards such as ITU-T H.266, also known as Versatile Video Coding (VVC). In other examples, video transcoder 200 and video decoder 300 may be based on proprietary video transcoders/formats such as AOMedia Video 1 (AV1), extensions to AV1, and/or successful versions of AV1 (eg, AV2) ) to operate. In other examples, video transcoder 200 and video decoder 300 may operate according to other proprietary formats or industry standards. However, the technology in this case is not limited to any particular decoding standard or format.

In general, video transcoder 200 and video decoder 300 may be configured to perform the techniques described herein in conjunction with any video decoding technique that uses context-based arithmetic decoding of the video material of the current picture or slice. The temporal initialization point of one or more contexts to use. For example, the plurality of temporal initialization points are included in the video data of one or more previous slices or pictures preceding the current slice or picture in decoding order.

Temporal initialization points can be associated with slices or pictures. Therefore, for each slice or picture, there may be a plurality of sets of temporal initialization points. For example, a first set of temporal initialization points may be associated with a first slice or picture, a second set of temporal initialization points may be associated with a second slice or picture, and so on.

Each set of temporal initialization points may be a context value of at least one context of a slice or picture, or may be a value derived from a context value of at least one context of a slice or picture. If temporal initialization is enabled for context-based arithmetic coding, video transcoder 200 and video decoder 300 may use a stored set of temporal initialization points (e.g., for previously encoded or decoded slices or pictures) to initialize for encoding or decode the context value of at least one context for a subsequent slice or picture. When the video transcoder 200 and the video decoder 300 encode or decode subsequent slices or blocks of pictures, the video transcoder 200 or the video decoder 300 may update the context value after initialization using the initialization point set.

As described in more detail, in one or more instances, the buffer stores a set of temporal initialization points. However, the number of possible sets of temporal initialization points may be relatively large, and there may be size limitations on the buffer. Accordingly, the video transcoder 200 and the video decoder 300 may be configured to perform buffer management, wherein the video transcoder 200 and the video decoder 300 selectively determine which set of initialization points to remove for the purpose of inserting into the buffer. The new initialization point set in frees up memory space.

This article describes example techniques for such buffer management. For example, if the buffer is full, the video transcoder 200 and the video decoder 300 may evaluate the time identification (ID) value and/or QP value of the slice or picture associated with the time initialization point stored in the buffer. , and possibly along with the slice type. For example, in addition to storing a set of temporal initialization points, the buffer may also store information indicating temporal ID values and/or QP values for slices or pictures, and possibly a slice type associated with each set of temporal initialization points (e.g., I slice, B slice, P slice).

The video transcoder 200 and the video decoder 300 may compare the temporal ID values and/or QP values of slices or pictures associated with the set of temporal initialization points stored in the buffer. Based on the comparison, the video transcoder 200 and the video decoder 300 may determine a set of temporal initialization points associated with the slice or picture based on the temporal ID value or QP value of the slice or picture. For example, the video transcoder 200 and the video decoder 300 may determine a set of temporal initialization points associated with a slice or picture having at least one of a minimum temporal ID value or a QP value. Video transcoder 200 and video decoder 300 may remove the first set of temporal initialization points to make room for a second set of temporal initialization points associated with the current slice or picture (eg, the slice or picture that was just encoded or decoded). out of memory space. The second set of temporal initialization points may be based on one or more context values determined for the current slice or picture.

In one or more examples, the video transcoder 200 and the video decoder 300 may also determine the set of temporal initialization points to be removed based on the slice type. In some examples, if the buffer stores a set of temporal initialization points associated with a slice of the same slice type as the current slice, the video transcoder 200 and the video decoder 300 may remove the set of temporal initialization points. In some cases, video transcoder 200 and video decoder 300 may remove a set of temporal initialization points even if there is a set of temporal initialization points associated with a slice or picture with a lower temporal stamp value and/or QP value.

For example, to remove a temporal initialization point set, the video transcoder 200 and the video decoder 300 may first determine whether there is a temporal initialization point set associated with a slice having the same slice type as the current slice. If present, the video transcoder 200 and the video decoder 300 can remove the time initialization point set. If it does not exist, the video transcoder 200 and the video decoder 300 can add the set of time initialization points of the current slice to the buffer (assuming the buffer is not full). In the above example, the video transcoder 200 and the video decoder 300 give priority to the slice type, but the example technology is not limited by this. In some examples, in order to remove the set of temporal initialization points, the video transcoder 200 and the video decoder 300 may first determine whether there is a slice associated with the same time stamp value or QP value as the current slice's time stamp value or QP value. time to initialize the point collection, regardless of the slice type. If present, the video transcoder 200 and the video decoder 300 can remove the time initialization point set.

However, in the event that the buffer is full and the set of temporal initialization points has been removed, the video transcoder 200 and the video decoder 300 may perform the example techniques described in this context, such as deciding to match the time with the smallest temporal ID value or QP value. A set of temporal initialization points associated with at least one of the slices or pictures, and subsequently removing the set of temporal initialization points.

As described in more detail, video transcoder 200 and video decoder 300 may be based on corresponding time identification (ID) values and/or quantization parameter (QP) values of slices or pictures associated with a plurality of sets of time initialization points and the current The time ID value or QP value of the picture or slice is used to determine (for example, select) at least one time initialization point set among the plurality of time initialization point sets. In this way, the probability of a temporal initialization point that is not available for the current picture can be reduced (eg when a picture with a higher temporal ID value than the current picture is removed from the bitstream or not processed).

Video transcoder 200 and video decoder 300 may perform block-based coding of pictures. The term "chunk" generally refers to a structure containing data to be processed (eg, encoded, decoded, or otherwise used in an encoding and/or decoding process). For example, a block may include a two-dimensional matrix of samples of luma and/or chroma data. Generally speaking, the video transcoder 200 and the video decoder 300 can decode video data represented in YUV (eg, Y, Cb, Cr) format. That is, the video transcoder 200 and the video decoder 300 may decode the luma and chroma components instead of decoding the sampled red, green, and blue (RGB) data for the picture, where the chroma components may Includes both a red hue chroma component and a blue hue chroma component. In some examples, the video transcoder 200 converts the received RGB format data into a YUV representation before encoding, and the video decoder 300 converts the YUV representation into an RGB format. Alternatively, pre- and post-processing units (not shown) can perform these transformations.

This context may generally refer to the decoding (eg, encoding and decoding) of pictures to include procedures for encoding or decoding the material of the pictures. Similarly, this context may refer to the coding of blocks of a picture to include procedures for encoding or decoding data for the blocks, such as prediction and/or residual coding. An encoded video bitstream typically includes a series of values for syntax elements that represent coding decisions (eg, coding modes) and partitioning of the picture into blocks. Therefore, a reference to coding a picture or block should generally be understood to be a coding value for the syntax elements forming the picture or block.

HEVC defines various blocks, including coding units (CU), prediction units (PU), and transform units (TU). According to HEVC, a video coder (such as video transcoder 200) divides coding tree units (CTUs) into CUs according to a quad-tree structure. That is, the video decoder divides the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node," and the CU of such a leaf node may include one or more PUs and/or one or more TUs. Video decoders can further divide PU and TU. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of TUs. In HEVC, PU represents inter-frame prediction data, and TU represents residual data. Intra-predicted CUs include intra-prediction information, such as intra-mode indication.

As another example, video transcoder 200 and video decoder 300 may be configured to operate according to VVC. According to VVC, a video coder (such as video transcoder 200) divides a picture into a plurality of coding tree units (CTUs). The video transcoder 200 may divide the CTU according to a tree structure, such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure removes the concept of multiple partition types, such as the separation between CU, PU and TU for HEVC. The QTBT structure consists of two levels: a first level divided according to a quadtree partition, and a second level divided according to a binary tree partition. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

In the MTT partitioning structure, blocks can be divided using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) (also called ternary tree (TT)) partitioning. . A ternary tree or ternary tree-type partition is a partition that splits a block into three sub-blocks. In some instances, ternary tree or ternary tree-type partitioning divides a block into three sub-blocks without dividing the original block through the center. The types of partitioning in MTT (for example, QT, BT, and TT) can be symmetric or asymmetric.

When operating in accordance with the AV1 transcoder, video transcoder 200 and video decoder 300 may be configured to decode video material in blocks. In AV1, the largest decoding block that can be processed is called a superblock. In AV1, a superblock can be 128x128 luma samples or 64x64 luma samples. However, in subsequent video coding formats (eg, AV2), super-blocks may be defined via different (eg, larger) luma sample sizes. In some instances, the superblock is the top level of a block quadtree. Video transcoder 200 may further partition the super-block into smaller decoding blocks. Video transcoder 200 may partition super-blocks and other decoding blocks into smaller blocks using square or non-square partitioning. Non-square blocks may include N/2xN, NxN/2, N/4xN, and NxN/4 blocks. Video transcoder 200 and video decoder 300 may perform separate prediction and transformation procedures for each coding block.

AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that can be decoded independently of other tiles. That is, the video transcoder 200 and the video decoder 300 can respectively encode and decode decoding blocks within a tile without using video data from other tiles. However, video transcoder 200 and video decoder 300 may perform filtering across block boundaries. The size of the tiles can be uniform or non-uniform. Tile-based decoding enables parallel processing and/or multi-threading of encoder and decoder implementations.

In some examples, video transcoder 200 and video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chrominance components, while in other examples, video transcoder 200 and video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT/MTT structure for the luma component and another QTBT/MTT structure for the two chroma components (or for each of the chroma components). Two QTBT/MTT structures).

The video transcoder 200 and the video decoder 300 may be configured to use quad-tree partitioning, QTBT partitioning, MTT partitioning, super-block partitioning, or other partitioning structures.

In some examples, a CTU includes a interpretation tree block (CTB) of luma samples, two corresponding CTBs of chroma samples for a picture with three sample arrays, or a monochrome picture or using three independent color planes and Syntax structure (used to decode the sample) CTB of the sample of the picture being decoded. The CTB can be an NxN sample block of some N value, such that the partitioning of the constituent elements into the CTB is a partition. A constituent element is an array or a single sample of one of the three arrays (luma and two chroma) that make up a picture in a 4:2:0, 4:2:2, or 4:4:4 color format, or a single sample that makes up a single color This array or a single sample in the array of format pictures. In some examples, the coding block is an MxN block of samples for some M and N values, such that the partitioning of CTBs into coding blocks is a partition.

Blocks (eg, CTUs or CUs) can be grouped in a picture in various ways. As an example, a brick can represent the CTU rows of a rectangular area within a specific tile in the image. A tile can be a rectangular area of CTU within a specific tile column and a specific tile row in the picture. A tile column refers to a CTU of a rectangular area with a height equal to the height of the picture and a width specified by a syntax element (eg, such as in a picture parameter set). A tile row refers to a CTU of a rectangular area with a height specified by a syntax element (eg, such as provided in the picture parameter set), and a width equal to the width of the picture.

In some examples, a tile may be divided into multiple tiles, and each tile may include one or more CTU rows within the tile. Tiles that are not divided into multiple bricks can also be called bricks. However, bricks that are a true subset of tiles cannot be called tiles. You can also arrange the bricks in the picture into slices. A slice can be an integer number of bricks in a picture that can be specifically contained within a single Network Abstraction Layer (NAL) unit. In some instances, a slice includes multiple complete tiles, or only a contiguous sequence of tiles of a tile.

This document may use "NxN" and "N by N" interchangeably to refer to the sampling dimensions of a block (such as a CU or other video block) in terms of vertical and horizontal dimensions, e.g., 16x16 samples or 16 by 16 samples. Generally speaking, a 16x16 CU will have 16 samples in the vertical direction (y=16), and 16 samples in the horizontal direction (x=16). Likewise, a NxN CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in rows and columns. Furthermore, a CU does not necessarily need to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include NxM samples, where M is not necessarily equal to N.

Video transcoder 200 encodes video data representing prediction and/or residual information and other information for a CU. The prediction information indicates how to predict the CU in order to form a prediction block for the CU. Residual information usually represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

To predict a CU, video transcoder 200 may typically form a prediction block for the CU via inter prediction or intra prediction. Inter-frame prediction generally refers to predicting a CU from previously coded data of a picture, while intra-frame prediction generally refers to predicting a CU from previously coded data of the same picture. To perform inter-frame prediction, video transcoder 200 may use one or more motion vectors to generate prediction blocks. Video transcoder 200 may typically perform a motion search to identify reference blocks that closely match the CU, eg, in terms of differences between the CU and the reference blocks. Video transcoder 200 may calculate the difference metric using sum of absolute differences (SAD), sum of square errors (SSD), mean absolute difference (MAD), mean square error (MSD), or other such difference calculations to determine Whether the reference block closely matches the current CU. In some examples, video transcoder 200 may use unidirectional prediction or bidirectional prediction to predict the current CU.

Some examples of VVC also provide an affine motion compensation mode, which can be considered an inter-frame prediction mode. In affine motion compensation mode, video transcoder 200 may determine two or more motion vectors that represent non-translational motion, such as zoom-in or zoom-out, rotation, perspective motion, or other irregular motion types.

To perform intra prediction, video transcoder 200 may select an intra prediction mode to generate prediction blocks. Some instances of VVC offer sixty-seven intra-frame prediction modes, including modes in various directions as well as planar and DC modes. Generally speaking, video transcoder 200 determines an intra prediction mode that describes, for a current block (eg, a block of a CU), neighboring samples from which samples of the current block are predicted. Assuming that video transcoder 200 decodes CTUs and CUs in raster scan order (left to right, top to bottom), such samples may typically be above, above, and above the current block in the same picture as the current block. left, or left.

Video transcoder 200 encodes data representing the prediction mode for the current block. For example, for inter-frame prediction modes, video transcoder 200 may encode data indicating which inter-frame prediction mode among various available inter-frame prediction modes is used and motion information for the corresponding mode. For unidirectional inter-frame prediction or bi-directional inter-frame prediction, for example, the video transcoder 200 may use improved motion vector prediction (AMVP) or merge mode to encode motion vectors. Video transcoder 200 may use a similar pattern to encode motion vectors for affine motion compensation mode.

AV1 includes two common technologies for encoding and decoding coding blocks of video material. The two common techniques are intra-frame prediction (eg, intra-frame prediction or spatial prediction) and inter-frame prediction (eg, inter-frame prediction or temporal prediction). In the context of AV1, when using intra-frame prediction mode to predict blocks of the current frame of video data, video transcoder 200 and video decoder 300 do not use video data from other frames of video data. For most intra-frame prediction modes, video transcoder 200 encodes blocks of the current frame based on differences between sample values in the current block and predicted values generated from reference samples in the same frame. Video transcoder 200 determines prediction values generated from reference samples based on the intra prediction mode.

After prediction (such as intra prediction or inter prediction for the block), video transcoder 200 may calculate residual data for the block. Residual information, such as a residual block, represents the sample-by-sample difference between a block and a prediction block formed using a corresponding prediction mode for the block. Video transcoder 200 may apply one or more transforms to the residual block to produce transformed data in the transform domain rather than the sample domain. For example, video transcoder 200 may apply a discrete cosine transform (DCT), integer transform, wavelet transform, or conceptually similar transform to the residual video data. Alternatively, the video transcoder 200 may apply a secondary transform after the first transform, such as mode-dependent non-separable quadratic transform (MDNSST), signal-dependent transform, Karhunen-Loeve Transform (KLT) etc. Video transcoder 200 generates transform coefficients after applying one or more transforms.

As mentioned above, after any transformation to generate transform coefficients, the video transcoder 200 may perform quantization of the transform coefficients. Quantization generally refers to a procedure in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, providing further compression. By performing a quantization procedure, video transcoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, video transcoder 200 may round down an n- bit value to an m -bit value during quantization, where n is greater than m . In some examples, to perform quantization, video transcoder 200 may perform a bit-wise right shift of the value to be quantized.

After quantization, video transcoder 200 may scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place higher energy (and therefore lower frequency) transform coefficients in front of the vector, and lower energy (and therefore higher frequency) transform coefficients in the back of the vector. In some examples, video transcoder 200 may scan the quantized transform coefficients using a predefined scan order to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, video transcoder 200 may perform self-adjusting scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video transcoder 200 may entropy encode the one-dimensional vector, such as context-based self-adaptive binary arithmetic coding (CABAC). Video transcoder 200 may also entropy encode values for syntax elements describing relay data associated with encoded video data for use by video decoder 300 in decoding the video data.

To perform CABAC, video transcoder 200 may assign one or more context values of the context within the context model to the symbols to be sent. The context value may relate to, for example, whether the symbol's neighboring values are zero values. Probabilistic decisions (eg, context values) may be based on the context assigned to the symbol.

Video transcoder 200 may further provide, for example, a picture header, a block header, a slice header, or other syntax data such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS). The video decoder 300 generates syntax data (such as block-based syntax data, picture-based syntax data, and sequence-based syntax data). The video decoder 300 can also decode the syntax data to determine how to decode the corresponding video data.

In this manner, video transcoder 200 may generate a bitstream that includes encoded video data, such as syntax elements describing the partitioning of pictures into blocks (eg, CUs) and block-specific predictions and/or residuals. Poor information. Finally, the video decoder 300 can receive the bit stream and decode the encoded video data.

Generally speaking, video decoder 300 performs a process identical to that performed by video transcoder 200 to decode encoded video data of a bitstream. For example, video decoder 300 may use CABAC to decode values for syntax elements of the bit stream in a manner substantially similar (albeit reciprocally) to the CABAC encoding procedure of video transcoder 200 . Syntax elements may define partitioning information for partitioning pictures into CTUs and partitioning each CTU according to a corresponding partitioning structure (such as a QTBT structure) to define CUs of CTUs. Syntax elements may further define prediction information and residual information for blocks (eg, CUs) of video data.

The residual information may be represented via, for example, quantized transform coefficients. Video decoder 300 may inverse-quantize and inverse-transform the quantized transform coefficients of a block to reproduce a residual block for the block. Video decoder 300 uses the signaled prediction mode (intra prediction or inter prediction) and the associated prediction information (eg, motion information for inter prediction) to form a prediction block for the block. Video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the initial block. Video decoder 300 may perform additional processing, such as performing deblocking procedures to reduce visual artifacts along block boundaries.

Content can generally refer to "signaling" certain information (such as grammatical elements). The term "signaling" may generally refer to the communication of values for syntax elements and/or other data used to decode encoded video data. That is, the video transcoder 200 may signal values for syntax elements in the bit stream. Generally speaking, signaling means producing a value in a stream of bits. As previously described, source device 102 may stream bits to target device 116 substantially instantaneously or non-instantly, such as possibly while storing syntax elements to storage 112 for later retrieval by target device 116 occurs.

The method of CABAC initialization of the current picture or slice using previous encoding/decoding sequential CABAC (Context Self-Adjusting Binary Arithmetic Coding) initialization points is described in the following document: JVET-Y0181: "AHG12: Based on the previous inter-slice "CABAC Initialization", Seregin et al., Joint Videoconferencing Expert Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 25th Telephone Conference, January 12-21, 2022. Although described with respect to CABAC, these example techniques are also applicable to context-based arithmetic decoding. For example, CABAC arithmetic decoding may require a starting point for each context, and this starting point is called the initialization point. The starting point (eg, initialization point) may include one or more context states, window or rate self-adjustment parameters, and other parameters required for arithmetic decoding operations. For example, there may be one or more initialization points for one or more context values of the context, and the one or more initialization points may be values (eg, initial values) for the one or more context values.

In a video transcoder, usually these initialization points are predefined and known to both the video transcoder 200 and the video decoder 300. For example, in HEVC and VVC, each slice type, I-slice, P Initialization points are defined in the initialization tables for slices and B slices.

In temporal CABAC initialization, in addition to or in addition to using predefined initialization points, initialization points can be stored at specific CTUs of pictures or slices, and these stored initialization points can be used to initialize the next picture or slice, instead of or in addition to outside of predefined initialization points. That is, in some examples, the temporal initialization point is included in the video material of one or more previous slices or pictures preceding the current slice or picture in coding order, and may be specific to the video material of the current slice or picture. One or more contexts used in context-based arithmetic decoding.

The video transcoder 200 or the video decoder 300 may store information indicating that the CTU of the initialization point may change, and may signal the CTU. As an example, the video transcoder 200 and the video decoder 300 can store the initialization point of the slice in the middle of the picture. That is, when the video transcoder 200 encodes a slice or the video decoder 300 decodes a slice, when the video transcoder 200 encodes the CTU in the middle of the slice or the video decoder 300 decodes the CTU in the middle of the slice. , the video transcoder 200 or the video decoder 300 may store the current one or more context values of the context as initialization points, and these initialization points are used to encode or decode syntax elements in subsequent slices or pictures. For example, when the video transcoder 200 or the video decoder 300 starts encoding or decoding subsequent slices or pictures, the video transcoder 200 or the video decoder 300 may set the initial value of the context value of the context to be equal to or based on the stored value. Initialization points (e.g., using mapping, scaling, weighting, etc.). Subsequently, when the video transcoder 200 and the video decoder 300 are encoding or decoding subsequent slices or pictures, the video transcoder 200 or the video decoder 300 can update the context from the initial value based on the most recently encoded or decoded video data. value.

In some examples, video transcoder 200 and video decoder 300 may store initialization points after decoding CABAC status, windows, and other parameters for a specific CTU. For example, for a certain CTU, the storage unit is used to store CABAC status, message windows, and other parameters for initialization after decoding. For example, parameters applicable to the decoded content in the previous picture may better represent the starting initialization point of the video material of the current picture than a predefined initialization point.

The slice quantization parameters (QP) can be stored separately for each slice type. In some instances, initialization with the same slice type and the same QP as the current slice can be used for the current slice CABAC initialization.

There may be some problems with the temporal initialization point of context-based arithmetic decoding. For example, since slices of the current picture can be decoded independently, and pictures with temporal identification (ID) values above a threshold can be removed as part of temporal scalability, techniques that exploit temporal initialization points can be refined to allow Time is scalable while ensuring that available time initialization points are available.

In some examples, such as in a FIFO buffer (where the top initialization point is from the previous picture closest to the current picture), multiple initialization points can be stored from previous pictures in decoding order. For example, the buffer may store a first set of initialization points for one or more context values of the context of a first slice or picture, and a second set of initialization points for one or more context values of the context of a second slice or picture, And so on.

An index can be introduced to indicate which initialization point to use from the FIFO buffer, and the initialization index is signaled in the bitstream (eg, in the picture or slice header). For example, video transcoder 200 may signal and video decoder 300 may receive an index in a buffer that indicates which set of initialization points to use to initialize one or more contexts for the context of the slice or picture currently being decoded. value. The video decoder 300 may retrieve the set of initialization points and one or more context values of the initialization context based on the index. For example, video decoder 300 may set an initial value of one or more context values equal to the retrieved set of initialization points. As another example, the video decoder 300 may map, scale, weight, or perform some other operation on the retrieved set of initialization points to determine the initial value of one or more context values.

Temporal scalability and the use of temporal initialization points to achieve temporal scalability are described below. Each coded picture may have a temporal ID value assigned by video transcoder 200. Video transcoder 200 may signal the time ID value in a Network Abstraction Layer Unit (NALU) header. Temporal ID values are used for temporal scalability, where certain pictures can be ignored (e.g., removed from the bitstream or not processed), while other pictures can be processed without removing or not processing pictures. Decode.

In one example, temporal scalability may be achieved by setting a restriction that pictures with lower temporal ID values cannot be used for inter-frame prediction with higher temporal ID values. In this case, the video decoder 300 is able to decode pictures with lower temporal ID values without using pictures with higher temporal ID values because pictures with higher temporal ID values cannot be used with lower temporal ID values. Inter-frame prediction for pictures with temporal ID values. Therefore, pictures with higher temporal ID values may be removed from the bitstream or not processed (eg, ignored). When applying temporal initialization of context-based arithmetic decoding, the video transcoder 200 and the video decoder 300 may utilize the temporal ID value to identify available storage initialization points for current slice/picture initialization.

In one or more examples, video transcoder 200 and video decoder 30 may store temporal ID values with initialization points. For example, each set of initialization points (where a set includes one or more initialization points) can be associated with a slice or picture. Video transcoder 200 and video decoder 300 may store a set of initialization points and information indicating temporal ID values of slices or pictures associated with the set of initialization points. As one example, a first set of initialization points may be associated with a first slice or picture having a first temporal ID value, and a second set of initialization points may be associated with a second slice or picture having a second temporal ID value. The video transcoder 200 and the video decoder 300 may store the first initialization point set and the first time ID value, and the first time ID value for the first slice or picture information associated with the first initialization point set. (For example, associate the first time ID with the first set of initialization points). The video transcoder 200 and the video decoder 300 may store the second initialization point set and the second time ID value, and the second time ID value for information of the second slice or picture associated with the second initialization point set (eg, , associating the second time ID value with the second set of initialization points).

To determine (eg, select) an initialization point based on the temporal ID value of the stored initialization point set, the video transcoder 200 and the video decoder 300 may compare the temporal ID value with the current slice/picture temporal ID. For example, the video transcoder 200 and the video decoder 300 may select at least one of the plurality of temporal initialization point sets based on the corresponding temporal ID value associated with the plurality of temporal initialization point sets and the temporal ID value of the current picture or slice. A collection of time initialization points.

Select the set of initialization points whose time ID is equal to or smaller than the current time ID as available points. For example, in order to determine at least one time initialization point set, the video transcoder 200 and the video decoder 300 can determine a time initialization point set among the plurality of time initialization point sets, wherein the corresponding time ID value of the time initialization point set is less than or Equal to the time ID value of the current picture or slice. The video transcoder 200 and the video decoder 300 can select a time initialization point set from the time initialization point set.

In one instance, only the set of initialization points with the same time ID as the current slice/picture time ID is selected as available initialization points. For example, since smaller temporal IDs generally have lower QP, initialization points from previous pictures with lower temporal ID values may not correctly represent the initialization points of slices of the current picture. In some examples, in order to determine the at least one time initialization point set, the video transcoder 200 and the video decoder 300 can determine a set of time initialization points in the plurality of time initialization point sets, wherein the time initialization points in the set of time initialization points The corresponding time ID value of the time initialization point is equal to the time ID value of the current picture or slice. The video transcoder 200 and the video decoder 300 may select at least one set of time initialization points from the set of time initialization points.

In another example, search for an initialization point with the same time ID. If the initialization point is not available, then check a smaller time ID (for example, the current time ID value minus 1). If not available, then check the current time ID value. Subtract 2, and so on. Initialize using the found set of initialization points.

For example, in order to determine at least one time initialization point set, the video transcoder 200 and the video decoder 300 may determine that no time initialization point in the plurality of time initialization point sets has a time ID equal to the time ID value of the current picture or slice. Based on determining that no time initialization point in the plurality of time initialization point sets has a time ID value equal to the time ID value of the current picture or slice, the video transcoder 200 and the video decoder 300 may determine that no time initialization point in the plurality of time initialization point sets has a time ID value equal to the time ID value of the current picture or slice. Whether any time initialization point has a time ID that is one less than the current picture or slice's time ID value. Based on determining that one or more temporal initialization points have a temporal ID less than one than the temporal ID value of the current picture or slice, the video transcoder 200 and the video decoder 300 may start from a temporal ID value smaller than the temporal ID value of the current picture or slice. Select at least one time initialization point set from one or more time initialization point sets.

Similarly, instead of using the same QP initialization, when not available, search for QP-1 or QP+1, if stored, use it, if not available, check QP-2 or QP+2, and so on. Use the found results for initialization. Time ID and QP searches can be combined when the same time ID and QP initialization point are not available.

For example, in one or more instances, video transcoder 200 and video decoder 30 may store QP values with initialization points. For example, each set of initialization points (where a set includes one or more initialization points) may be associated with a slice or picture. Video transcoder 200 and video decoder 300 may store a set of initialization points and information indicating a QP value of a slice or picture associated with the set of initialization points. As an example, a first set of initialization points may be associated with a first slice or picture having a first QP value, and a second set of initialization points may be associated with a second slice or picture having a second QP value. The video transcoder 200 and the video decoder 300 may store the first initialization point set and the first QP value, and the first QP value for information of the first slice or picture associated with the first initialization point set (eg, Associating a first QP value with a first set of initialization points). The video transcoder 200 and the video decoder 300 may store the second initialization point set and the second QP value, and the second QP value for information of the second slice or picture associated with the second initialization point set (e.g., The second QP is associated with the second set of initialization points).

In one instance, only the set of initialization points with the same QP value as the current slice/picture QP value is selected as available initialization points. In some examples, in order to determine the at least one time initialization point set, the video transcoder 200 and the video decoder 300 can determine a set of time initialization points in the plurality of time initialization point sets, wherein the time initialization points in the set of time initialization points The corresponding QP value of the temporal initialization point is equal to the QP value of the current picture or slice. The video transcoder 200 and the video decoder 300 may select at least one time initialization point set from the set of time initialization points.

In another example, search for an initialization point with the same QP value. If the initialization point is not available, then check the latest QP value, such as the current QP value plus 1 or minus 1. If it is not available, then check the current QP value plus 2. Or minus 2, and so on. Use the found set of initialization points for initialization.

For example, in order to determine at least one temporal initialization point set, the video transcoder 200 and the video decoder 300 may determine that no temporal initialization point in the plurality of temporal initialization point sets has a QP value equal to the QP value of the current picture or slice. Based on determining that none of the plurality of temporal initialization point sets has a QP value equal to the QP value of the current picture or slice, the video transcoder 200 and the video decoder 300 may determine that any of the plurality of temporal initialization point sets has a QP value equal to the QP value of the current picture or slice. Whether the temporal initialization point has a QP value less than one or greater than the QP value of the current picture or slice. Based on determining that there are one or more temporal initialization points having a QP value that is less than one or greater than one than the QP value of the current picture or slice, the video transcoder 200 and the video decoder 300 may start from a QP value that is greater than the QP value of the current picture or slice. Select at least one time initialization point set from one or more time initialization point sets that are less than one or greater than one.

Accordingly, in one or more examples, video transcoder 200 and video decoder 300 may be configured to determine (eg, select) a set of temporal initialization points stored in a buffer, and based on the selected set of temporal initialization points , to initialize one or more context values of at least one context used for encoding or decoding subsequent slices or pictures. The video transcoder 200 and the video decoder 300 may perform context-based arithmetic encoding or decoding on subsequent slices or pictures. For example, video transcoder 200 and video decoder 300 may set initial values for context values based on the initialization point and use the initial values to encode or decode one or more syntax values of subsequent slices or pictures. The video transcoder 200 and the video decoder 300 can update the context value from the initial value.

An example way of selecting a set of temporal initialization points may include: the video transcoder 200 and the video decoder 300 determine a time stamp value for subsequent slices or pictures. Video transcoder 200 and video decoder 300 may determine that none of two or more slices or pictures having an associated set of temporal initialization points stored in the buffer has a time stamp value equal to that of a subsequent slice or picture. Timestamp value. In this case, the video transcoder 200 and the video decoder 300 may determine, from the two or more slices or pictures, the slice or picture whose time stamp value is closest to and smaller than the time stamp value of the subsequent slice or picture. . To select a set of temporal initialization points, the video transcoder 200 and the video decoder 300 may select a set of temporal initialization points associated with the determined slice or picture that has the closest and smaller time to the subsequent slice or picture. The timestamp value of the identification value.

Another example way of selecting a set of temporal initialization points may include the video transcoder 200 and the video decoder 300 determining QP values for subsequent slices or pictures. Video transcoder 200 and video decoder 300 may determine that none of two or more slices or pictures having an associated set of temporal initialization points stored in the buffer has a QP equal to the QP value of a subsequent slice or picture. value. In this case, the video transcoder 200 and the video decoder 300 can determine the slice or picture with the closest QP value to the subsequent slice or picture from the two or more slices or pictures. In this case, the closest QP value can be greater than the QP value of the current slice or picture. To select a set of temporal initialization points, video transcoder 200 and video decoder 300 may select a set of initialization points associated with a QP value that is closest to the QP value of a subsequent slice or picture.

In the above example, the video transcoder 200 and the video decoder 300 may determine the set of initialization points associated with the slice or picture having the time stamp value and/or the QP with the QP value closest to the subsequent slice or picture A set of initialization points associated with a slice or picture: the time stamp value is closest to the time stamp value of the subsequent slice or picture, and not greater than the time stamp value of the current slice. In some examples, the video transcoder 200 and the video decoder 300 may also consider the slice type when deciding which set of time initialization points to use. For example, the set of temporal initialization points may be associated with the same slice type as that of subsequent slices.

The storage of initialization points is described below. As mentioned before, slices of the same picture can be decoded independently (ie, the next slice of the same picture cannot depend on the previous slice of the same picture). Therefore, initialization points stored in previous slices of the same picture cannot be used for any slice in the same picture.

To achieve this, in one example, initialization points (e.g., a set of initialization points, where the set of initialization points may include one or more initialization points) are temporarily stored in a temporary buffer and are only processed (encoding, decoding, After parsing) all slices of the same image are added from the temporary buffer to the storage buffer. In this way, the initialization points in the temporary buffer can be updated until all slices of the same picture have been processed, and then the storage buffer receives the initialization points from the temporary buffer.

When adding initialization points, previously stored initialization points can be deleted/replaced. This is because of buffer limitations. If there is an update while processing a slice of the same image, the appropriate slice of the previous image can be replaced with the previous slice of the same image. Initialization point. That is, if a temporary buffer is not used, initialization points that should remain in the storage buffer may be overwritten. By using a temporary buffer, you can avoid overwriting initialization points in the storage buffer that should be retained, and only overwrite the storage buffer with initialization points in the temporary buffer after processing a slice of the same picture.

Therefore, there may be some benefit to storing the initialization point separately in a temporary buffer, and only using the temporary buffer to update the storage buffer after all slices of the picture have been processed. To check the end of the picture, you can check if the CTU address is equal to the last CTU or if the slice index is the last slice of the picture.

For example, video transcoder 200 and video decoder 300 may store in a first buffer one or more of one or more contexts used in context-based arithmetic decoding of video data for one or more previous pictures. Time initialization point. Video transcoder 200 and video decoder 300 may store in the second buffer temporal initialization points of one or more contexts used in context-based arithmetic coding of video material for a slice of the current picture (e.g., Temporal initialization point set), wherein storing in the second buffer includes: storing in the second buffer during decoding of the video data of the current picture. The video transcoder 200 and the video decoder 300 may store the temporal initialization point stored in the second buffer in the first buffer after processing the last coding tree unit (CTU) or slice of the current picture.

In another example, the initialization point may be signaled in an self-tuning parameter set (APS), e.g., similar to a self-tuning loop filter parameter set carrying filter coefficients. The APS may already have limitations defined for temporal ID value processing, and the APS exported from the current picture is not used to apply to the same picture decoding.

In one or more instances, the initialization point may be stored at various picture locations (eg, the center or end of the picture). Other locations in the image may also be used.

The choice of which storage location to use can be signaled in the bit stream. In one example, video transcoder 200 may signal such indications, and video decoder 300 may parse such indications in the picture or slice header, or in any other parameter set or elsewhere. The image header can be the signaling location for an instance, since the image header is shared by all slices of the image, and there may not be a need to signal storage selections in every slice for image level self-adjustment.

There can be multiple syntax elements in the picture or slice header for temporal CABAC, such as whether temporal initialization is applied to the picture or slice, storage bit setting signaling, or how to remove entries from the storage initialization buffer, etc. . Such syntax elements signaled in the picture or slice header may be adjusted via a higher level indication, for example, via syntax indicating whether to use temporary CABAC initialization in the SPS or PPS.

Syntax element signaling in the image or slice header may also depend on whether there is an initialization entry in the buffer available for the slice. If no such entry exists in the buffer, temporal initialization may not be applied, and syntax elements associated with temporal initialization may be signaled in the slice or image header.

Entry identification may be performed by comparing the temporal ID and/or QP value of the slice to be coded with the entry value in the buffer. That is, in one or more examples, the video decoder 300 may combine the time identification value and/or QP value of the slice or picture being decoded with the time identification value and/or QP value associated with each set of time initialization points. The values are compared to select a set of initialization points that initialize the context values used to encode or decode the context of a slice or picture.

In some instances, for implementation purposes, the initialization store may be limited to a certain number of entries because it is possible to store the initialization storage for each time ID and QP (e.g., the initialization point, but other storage information is also possible). Can be expensive, e.g. requiring larger buffer sizes. For example, the buffer size per slice type can be limited to N. In this case, when the buffer is full (that is, when all N entries have been added to the buffer), one entry should be deleted before adding the next one. In one example, the number N can be set equal to 5 since it represents typical GOP 32 decoding.

Entry removal may be accomplished by the video transcoder 200 and the video decoder 300 according to specific rules based on the temporal ID value and/or QP value of the buffer entry. For example, the rule may be as follows: remove the entry with the smallest time ID, and/or remove the entry with the smallest time ID and the smallest QP.

In some examples, the video transcoder 200 and the video decoder 300 may remove the entry of the temporal initialization point set if there is an entry of the temporal initialization point set associated with a slice of the same slice type as the subsequent slice, even if the buffering There is an entry in the controller for a set of initialization points associated with a slice or picture with a smaller temporal ID or smaller QP. In some examples, the entry with the smallest temporal ID and/or smallest QP may be used in a set of temporal initialization points associated with a slice whose slice type is different from that of subsequent slices.

In some instances, there may be multiple entries for a slice type. For example, for slice type I slice, the buffer can store up to 5 temporal initialization point sets, for slice type P slice, the buffer can store up to 5 temporal initialization point sets, and for slice type B slice, the buffer can store Store up to 5 time initialization point sets. In such instances, video transcoder 200 and video decoder 300 may first determine a set of temporal initialization points associated with the same slice type as the slice type of the subsequent slice. Subsequently, the video transcoder 200 and the video decoder 300 may determine a set of time initialization points with a minimum time stamp value and/or a QP value within the determined set of time initialization points, and determine that the set of time initialization points should start from the same slice as the subsequent slice. The time initialization point set with the smallest time identification value and/or QP value is removed from the time initialization point set of the type.

Removing the entry with minimum QP can be because the minimum QP slice has more transform coefficients (less quantization), so the context can be adjusted at the beginning of slice decoding, and the rest of the slice will be decoded efficiently. Slices with higher QP have fewer transform coefficients and context self-adjustment may be slower, so fewer blocks of the slice will be decoded efficiently.

In other words, having a temporal initialization point allows context values to be initialized which can subsequently be adjusted as part of encoding or decoding. Having a temporal initialization point may have some benefit if the context value of the slice can be adjusted relatively quickly (for example, a slice with a lower QP value). However, such benefits may be reduced because context values for slices with lower QP values can be adjusted relatively quickly even without proper initialization. For slices where the context values do not adapt relatively quickly (e.g. slices with higher QP values), there may be more benefit from initializing the context values appropriately.

As an example, assume that the context value of a first slice with a lower QP value tends to adapt quickly, while the context value of a second slice with a higher QP value tends not to adapt quickly. If the temporal initialization point of the first slice is available, there may be some benefit in accelerating the adjustment of context values. However, such benefits may be modest, as the context value of the first slice tends to adapt quickly even if it is not initialized correctly.

If a temporal initialization point for the second slice is available, there may be greater benefit in accelerating the adjustment of context values than for the first slice. For example, by initializing the context value of the second slice, the self-adjustment of the context value starts from a value closer to the final value, so there is better compression of the blocks in the second slice than if the temporal initialization point was not available.

As mentioned earlier, slices with higher QP values tend to be slices whose context values adapt more slowly. Therefore, if the temporal initialization points of slices with higher QP values are stored in the buffer, such temporal initialization points will be available for future slices where the benefits of having temporal initialization points are more obvious. As a result, keeping initialized entries with higher QP can provide efficient decoding for future pictures.

Similarly, lower temporal ID entries can be removed since typically lower temporal ID slices are decoded with smaller QPs. For example, similar to above, having a temporal initialization point allows context values to be initialized which can then be adjusted as part of encoding or decoding. Having a temporal initialization point may have some benefit if the context value of the slice can be adapted relatively quickly (for example, a slice with a lower timestamp value). However, this benefit may be reduced because the context values of slices with lower timestamp values can adapt relatively quickly even if they are not initialized correctly. For slices whose context values do not adapt relatively quickly (for example, slices with high timestamp values), there may be more benefit in initializing the context values appropriately.

As an example, assume that the context value of a first slice with a lower time stamp value tends to adapt quickly, while the context value of a second slice with a higher time stamp value tends not to adapt quickly. If the temporal initialization point of the first slice is available, there may be some benefit in accelerating the adjustment of context values. However, such benefits may be modest, as the context value of the first slice tends to adapt quickly even if it is not initialized correctly.

If a temporal initialization point for the second slice is available, there may be greater benefit in adjusting the acceleration context value than for the first slice. For example, by initializing the context values of the second slice, the self-adjustment of the context values starts from a value closer to the final value, so the blocks in the second slice have better compression than if the temporal initialization point was not available.

As mentioned earlier, slices with higher timestamp values tend to be slices whose context values adapt more slowly. Therefore, if the temporal initialization points of slices with higher time stamp values are stored in the buffer, such temporal initialization points will be available for future slices where the benefits of having temporal initialization points are more obvious. As a result, keeping initialized entries with higher temporal IDs can provide efficient decoding of future pictures.

Other rules based on time ID values, QP values, or slice types are possible, and the content of this case is not limited to instance rules for time ID values and QP values or slice types. The selection of this rule may be signaled in the bitstream in the picture or slice header, or in any other parameter set, or elsewhere.

Accordingly, in one or more examples, video transcoder 200 and video decoder 300 may be configured to process video data. To process video data, the video transcoder 200 and the video decoder 300 may be configured to determine one or more context values of at least one context for encoding or decoding the current slice or picture.

Video transcoder 200 and video decoder 300 may decide that a buffer used to store sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full (e.g., may store N There are N entries in the buffer of entries). As mentioned above, each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures, and includes one or more temporal initialization points.

The video transcoder 200 and the video decoder 300 may determine from the two or more slices or pictures based on at least one of a slice type, a time stamp value, or a quantization parameter (QP) value of the slice or picture (eg, , identify) the first set of initialization points associated with the slice or picture. The video transcoder 200 and the video decoder 300 may remove the first temporal initialization point set associated with the slice or picture from the buffer, and store the second temporal initialization point associated with the current slice or picture in the buffer. Point collection. The second set of time initialization points may be based on the determined one or more context values (for example, the second set of time initialization points is equal to the one or more context values of the context, or based on the one or more context values of the context generated).

As an example, in order to determine the first set of temporal initialization points (for example, the set of temporal initialization points to be removed), the video transcoder 200 and the video decoder 300 may determine from two or more slices or pictures with the smallest A first set of temporal initialization points associated with a slice or picture of at least one of a time stamp value or a quantization parameter (QP) value. For example, the video transcoder 200 and the video decoder 300 may determine the slice or picture with the smallest time stamp value from the time stamp values of two or more slices or pictures, or may determine the slice or picture with the smallest QP value. . In some cases, the slice associated with the first set of temporal initialization points may have a different slice type than the slice type of the current slice in the QP values of two or more slices or pictures.

In some examples, if the buffer is full, or perhaps even if the buffer is not full, video transcoder 200 and video decoder 300 may remove the initialization associated with the slice or picture with the smallest time stamp value or QP value. Before clicking the collection, remove duplicate entries. For example, if the buffer is full, or perhaps even if the buffer is not full, video transcoder 200 and video decoder 300 may decide to buffer before storing the set of initialization points in the buffer associated with the current slice or picture. Whether there is any set of initialization points in the controller associated with a slice or picture that has the same timestamp value or QP value as the current slice or picture, and/or has the same slice type.

If there is a set of initialization points in the buffer associated with a slice or picture that has the same timestamp value, QP value, or slice type as the current slice or picture, the video transcoder 200 and the video decoder 300 may remove the initialization point. collection, even if there are other initialization point collections associated with slices or pictures with lower timestamp values or QP values. If there is no set of initialization points in the buffer associated with a slice or picture that has the same timestamp value, QP value, or slice type as the current slice or picture, the video transcoder 200 and the video decoder 300 may remove the fragment with the minimum time A collection of initialization points for identity values or QP values.

In some instances, the removed set of initialization points with the smallest time stamp value or QP value may have a different slice type than the current slice. In some examples, such as where multiple sets of temporal initialization points may be stored for a slice type, the video transcoder 200 and the video decoder 300 may determine a set of temporal initialization points associated with a slice of the same slice type as the current slice. of packets. Subsequently, the video transcoder 200 and the video decoder 300 may determine the time initialization point set with the minimum time stamp value or QP value in the packet as the time initialization point set to be removed.

Therefore, the video transcoder 200 and the video decoder 300 may determine that at least one of the time stamp value or the QP value of the current slice or picture is different from the two or the associated set of time initialization points stored in the buffer. The timestamp value or QP value of each slice or picture of two or more slices or pictures. In such instances, the video transcoder 200 and the video decoder 300 may determine that at least one of the time stamp value or the QP value of the current slice or picture is different from each of the two or more slices or pictures. Or the time stamp value or QP value of the slice or picture to remove the first time initialization point set. For example, the time stamp value or QP value of the slice or picture associated with the first set of temporal initialization points is different from the time stamp value or QP value in the current slice or picture.

For example, assume the current slice or picture is the first slice or picture. In this example, video transcoder 200 and video decoder 300 may determine one or more context values of at least one context for encoding or decoding the second slice or picture. The video transcoder 200 and the video decoder 300 may determine a third slice or picture from two or more slices or pictures, and the third slice or picture has the same time stamp value or QP value as the second slice or picture. At least one of the timestamp value or QP value. In this example, video transcoder 200 and video decoder 300 may remove the third set of temporal initialization points associated with the third slice or picture from the buffer, based on the decision to use for the second slice or picture. One or more context values of the at least one context are encoded or decoded, and a fourth set of temporal initialization points associated with the second slice or picture is stored in the buffer.

The initialization point may include multiple parameters, such as multiple context states and multiple adaptation rates or adaptation windows (indicating how quickly the context state can be adapted after each binary decoding). The initial storage buffer can be reduced by storing quantized values of these parameters to reduce the dynamic range of possible values, which may require fewer bits to store these parameters. In one example, the video transcoder 200 and the video decoder 300 may store the sum of states and the sum of adaptation rates. The video transcoder 200 and the video decoder 300 may divide the sum of states by the number of states (average states) to assign state values. The video transcoder 200 and the video decoder 300 may assign an adaptation rate (adaptation window) value as the sum of the adaptation rates divided by the number of adaptation rates (the average adaptation rate).

In another example, the video transcoder 200 and the video decoder 300 may only store certain parameters (eg, only store some initialization points). In such instances, the video transcoder 200 and the video decoder 300 may initialize other parameters according to preset values. For example, only one state and one adaptation rate are stored. When initialization is performed, the stored values are assigned to the first state and the first adaptation rate respectively, and from the presets that may have been stored in the transcoder (eg, the video transcoder 200 and the video decoder 300 ) The second state and the second adaptation rate are assigned to the initial value (for example, the initialization point stored in each I slice, P slice, and B slice described above). Other value encapsulation mechanisms applied to status and adaptation rate values may also be applied.

Multiple initialization points are described below. Each image can store multiple initialization points. That is, for a picture or slice, there may be a set of initialization points, where the set of initialization points will include one initialization point or multiple initialization points. In one instance, multiple initialization points can be used when using more than one slice in the picture. For example, one initialization point can be stored for each slice.

Slices have relative positions within the picture, so the initialization point is stored from the previous picture corresponding to the current slice position. In one example, the initialization point is stored in the middle of each slice of the previous picture and used to initialize the corresponding current slice.

The slice boundaries of the current picture and the previous picture do not have to be aligned. In one example, the location of the initialization point is driven via the current image slice boundary.

In another example, initialization points are stored at specific locations for each slice of the current picture, and subsequent pictures can decide which initialization point to use based on specific rules. For example, such a rule could be to check the coordinates of the stored initialization and compare it to whether the position belongs to the current slice, and if so, then such an initialization can be used. In another example, the initialization point for each slice can also be stored in a buffer, and the index of the current slice is signaled to identify which initialization point to use.

For example, the video transcoder 200 and the video decoder 300 may store previous temporal initialization points corresponding to one or more contexts used in context-based arithmetic decoding of video data of previous slices in the previous picture. For the current slice, the video transcoder 200 and the video decoder 300 may determine that the current slice in the current picture has a position in the current picture that corresponds to the position of the previous slice in the previous picture. Based on the current slice having a position in the current picture corresponding to the position of the previous slice in the previous picture, the video transcoder 200 and the video decoder 300 may decide a current temporal initialization point for the current slice based on the previous temporal initialization point.

FIG. 2 is a block diagram illustrating an example video transcoder 200 that may perform the techniques of this disclosure. Figure 2 is provided for purposes of explanation and should not be considered a limitation on the techniques broadly illustrated and described in this context. For ease of explanation, the content of this case describes the video transcoder 200 based on VCC technology (ITU-T H.266 under development) and HEVC (ITU-T H.265) technology. However, the techniques described in this case may be performed by video encoding devices configured to implement other video decoding standards and video decoding formats (eg, AV1 and the successor to the AV1 video decoding format).

In the example of FIG. 2, the video transcoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transformation processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transformation processing unit 212, Reconstruction unit 214, filter unit 216, decoded picture buffer (DPB) 218, and entropy encoding unit 220. Video data memory 230, mode selection unit 202, residual generation unit 204, transformation processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transformation processing unit 212, reconstruction unit 214, filter unit 216, DPB 218 and Any or all of the entropy encoding units 220 may be implemented in one or more processors or in processing circuitry. For example, units of the video transcoder 200 may be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Additionally, video transcoder 200 may include supplemental or alternative processors or processing circuitry to perform these and other functions.

Video data memory 230 may store video data to be encoded by components of video transcoder 200 . Video transcoder 200 may receive video data stored in video data memory 230 from, for example, video source 104 (FIG. 1). The DPB 218 may serve as a reference picture memory that stores reference video data for use by the video transcoder 200 when predicting subsequent video data. Video data memory 230 and DPB 218 may be composed of a variety of memory devices such as dynamic random access memory (DRAM) (which includes synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or any other type of storage device). Video data memory 230 and DPB 218 may be provided by the same storage device or different storage devices. In various examples, video data memory 230 may be on-chip with other components of video transcoder 200, as shown, or off-chip relative to those components.

In the context of this case, references to video data memory 230 should not be construed as being limited to memory internal to video transcoder 200 (unless so specifically described), nor should it be construed as being limited to memory outside video transcoder 200 of memory (unless so specifically described). Rather, a reference to the video data memory 230 should be understood as a reference memory that stores the video data that the video transcoder 200 receives for encoding (eg, the current block of video data to be encoded). Memory 106 of FIG. 1 may also provide temporary storage of outputs from various units of video transcoder 200.

The various units of FIG. 2 are illustrated to illustrate understanding the operations performed by the video transcoder 200. These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Fixed function circuits represent circuits that provide a specific function and are preset in terms of the operations they can perform. Programmable circuits represent circuits that can be programmed to perform a variety of tasks and provide flexible functionality in the operations that can be performed. For example, the programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, for receiving parameters or outputting parameters), but the types of operations performed by fixed-function circuits are generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed function or programmable), and in some examples, the one or more units may be integrated circuits.

The video transcoder 200 may include an arithmetic logic unit (ALU), an elementary functional unit (EFU), a digital circuit, an analog circuit, and/or a programmable core formed of programmable circuits. In examples where software executed by programmable circuitry is used to perform operations of video transcoder 200 , memory 106 ( FIG. 1 ) may store instructions (eg, object code) for the software that video transcoder 200 receives and executes. , or another memory (not shown) in the video transcoder 200 can store such instructions.

The video data memory 230 is configured to store received video data. The video transcoder 200 may retrieve pictures of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be original video data to be encoded.

Mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes. For example, mode selection unit 202 may include palette units, intra-block replication units (which may be part of motion estimation unit 222 and/or motion compensation unit 224), affine units, linear model (LM) units, and the like. .

Mode selection unit 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and resulting rate-distortion values for such combinations. Coding parameters may include: CTU to CU partitioning, prediction mode for CU, transform type for residual data of CU, quantization parameters for residual data of CU, etc. The mode selection unit 202 may ultimately select a combination of encoding parameters that has a better rate-distortion value than other tested combinations.

Video transcoder 200 may divide the picture retrieved from video data memory 230 into a series of CTUs and encapsulate one or more CTUs in fragments. The mode selection unit 202 may divide the CTU of the picture according to a tree structure (eg, the above-described MTT structure, QTBT structure, super-block structure, or quad-tree structure). As mentioned above, the video transcoder 200 can form one or more CUs by dividing the CTU according to the tree structure. Such CUs may also be commonly referred to as "video blocks" or "blocks".

Typically, mode select unit 202 also controls its components (eg, motion estimation unit 222, motion compensation unit 224, and intra prediction unit 226) to generate overlapping PUs and TUs for the current block (eg, the current CU, or in HEVC, PU and TU). part) prediction block. For inter-frame prediction of the current block, motion estimation unit 222 may perform a motion search to identify one or more reference pictures (e.g., one or more previously coded pictures stored in DPB 218) a closely matching reference block. Specifically, the motion estimation unit 222 may calculate, for example, based on the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean square error (MSD), etc., to represent the difference between the potential reference block and the current How similar values do blocks have. Motion estimation unit 222 may typically perform these calculations using sample-by-sample differences between the current block and the reference block under consideration. Motion estimation unit 222 may identify the reference block with the minimum value produced by these calculations, which reference block indicates the reference block that most closely matches the current block.

Motion estimation unit 222 may form one or more motion vectors (MVs) that define the location of the reference block in the reference picture relative to the current block in the current picture. Motion estimation unit 222 may then provide the motion vector to motion compensation unit 224. For example, for unidirectional inter-frame prediction, motion estimation unit 222 may provide a single motion vector, and for bi-directional inter-frame prediction, motion estimation unit 222 may provide two motion vectors. Motion compensation unit 224 may then use the motion vectors to generate prediction blocks. For example, motion compensation unit 224 may use motion vectors to retrieve reference block information. As another example, if the motion vector has fractional sampling precision, motion compensation unit 224 may interpolate the value for the prediction block according to one or more interpolation filters. In addition, for bidirectional inter-frame prediction, the motion compensation unit 224 may retrieve the information of two reference blocks identified by the corresponding motion vectors, such as through sample-by-sample averaging or weighted averaging, and combine the retrieved information.

When operating in accordance with the AV1 video coding format, motion estimation unit 222 and motion compensation unit 224 may be configured to use translational motion compensation, affine motion compensation, overlapping block motion compensation (OBMC), and/or composite intra-frame prediction, Encodes coding blocks of video data (eg, luma and chroma coding blocks).

As another example, for intra prediction or intra prediction coding, intra prediction unit 226 may generate a prediction block based on samples adjacent to the current block. For example, for directional mode, intra prediction unit 226 may typically mathematically combine values from adjacent samples and pad these calculated values in a defined direction across the current block to produce a prediction block. As another example, for DC mode, intra prediction unit 226 may calculate an average of adjacent samples to the current block and generate a prediction block to include the resulting average for each sample of the prediction block.

When operating according to the AV1 video coding format, intra prediction unit 226 may be configured to use directional intra prediction, non-directional intra prediction, recursive filtered intra prediction, chroma from luma (CFL) Prediction, intra-block copy (IBC), and/or palette modes encode coding blocks of video data (eg, luma and chroma coding blocks). The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes.

The mode selection unit 202 supplies the prediction block to the residual generation unit 204. The residual generation unit 204 receives the original unencoded version of the current block from the video data memory 230 and the prediction block from the mode selection unit 202 . Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block for the current block. In some examples, the residual generation unit 204 may also determine the difference between the sample values in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

In examples where mode selection unit 202 partitions a CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video transcoder 200 and the video decoder 300 can support PUs of various sizes. As mentioned above, the size of the CU may represent the size of the luma coding block of the CU, and the size of the PU may represent the size of the luma prediction unit of the PU. Assuming that the size of a specific CU is 2Nx2N, the video transcoder 200 can support a PU size of 2Nx2N or NxN for intra-frame prediction, and support a symmetric PU size of 2Nx2N, 2NxN, Nx2N, NxN, etc. for intra-frame prediction. prediction. The video transcoder 200 and the video decoder 300 may also support asymmetric partitioning of PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-frame prediction.

In instances where mode selection unit 202 does not further partition the CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As mentioned above, the size of the CU may represent the size of the luma decoding block of the CU. The video transcoder 200 and the video decoder 300 may support a CU size of 2Nx2N, 2NxN or Nx2N.

For other video coding techniques (eg, intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, to name some examples), mode selection unit 202 is associated with the coding technology via Each unit of , generates a prediction block for the current block being encoded. In some examples (eg, palette mode coding), mode selection unit 202 may not generate predictive blocks but instead generate syntax elements that indicate how the blocks are reconstructed based on the selected palette. In such modes, mode selection unit 202 may provide these syntax elements to entropy encoding unit 220 for encoding.

As mentioned above, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. Subsequently, the residual generation unit 204 generates a residual block for the current block. To generate a residual block, residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

Transform processing unit 206 applies one or more transforms to the residual block to produce a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form a block of transform coefficients. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, transform processing unit 206 may perform multiple transforms (eg, a primary transform and a secondary transform (eg, a rotation transform)) on the residual block. In some examples, transform processing unit 206 does not apply transforms to the residual blocks.

When operating in accordance with AV1, transform processing unit 206 may apply one or more transforms to the residual block to produce a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form a block of transform coefficients. For example, transform processing unit 206 may apply a horizontal/vertical transform combination, which may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), a flipped ADST (eg, inverse ADST), and an identity transform (IDTX). When using an identity transformation, the transformation is skipped in one of the vertical or horizontal directions. In some instances, transformation processing may be skipped.

Quantization unit 208 may quantize the transform coefficients in the block of transform coefficients to produce a quantized block of transform coefficients. Quantization unit 208 may quantize the transform coefficients of the block of transform coefficients based on a quantization parameter (QP) value associated with the current block. Video transcoder 200 (eg, via mode selection unit 202) may adjust the degree of quantization applied to the block of transform coefficients associated with the current block by adjusting the QP value associated with the CU. Quantization may result in loss of information, and therefore, the accuracy of the quantized transform coefficients may be lower than the accuracy of the original transform coefficients generated by the transform processing unit 206 .

The inverse quantization unit 210 and the inverse transform processing unit 212 may respectively apply inverse quantization and inverse transform to the quantized transform coefficient block to reconstruct the residual block from the transform coefficient block. Reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although possibly with a certain degree of distortion) based on the reconstructed residual block and the prediction block generated by mode selection unit 202 . For example, reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples in the prediction block generated by mode selection unit 202 to produce a reconstructed block.

Filter unit 216 may perform one or more filtering operations on the reconstructed blocks. For example, filter unit 216 may perform deblocking operations to reduce blocking artifacts along edges of CUs. In some examples, operation of filter unit 216 may be skipped.

When operating according to AV1, filter unit 216 may perform one or more filtering operations on the reconstructed block. For example, filter unit 216 may perform deblocking operations to reduce blocking artifacts along CU edges. In other examples, filter unit 216 may apply constrained direction enhancement filtering (CDEF), which may be applied after deblocking, and may include applying non-separable, non-linear low-pass directional filtering based on the estimated edge directions. The filter unit 216 may also include a loop recovery filter applied after the CDEF, and may include a separable symmetric normalized Wiener filter or a dual self-steering filter.

Video transcoder 200 stores the reconstructed blocks in DPB 218. For example, in instances where the operations of filter unit 216 are not performed, reconstruction unit 214 may store the reconstructed blocks into DPB 218 . In an example in which the operations of filter unit 216 are performed, filter unit 216 may store the filtered reconstruction block into DPB 218 . Motion estimation unit 222 and motion compensation unit 224 may retrieve reference pictures from DPB 218 that are formed from the reconstructed (and possibly filtered) blocks for inter-frame prediction of subsequently encoded pictures. In addition, the intra prediction unit 226 may use the reconstructed blocks in the DPB 218 of the current picture to perform intra prediction on other blocks in the current picture.

Generally, the entropy encoding unit 220 may entropy encode syntax elements received from other functional elements of the video transcoder 200 . For example, entropy encoding unit 220 may entropy encode the quantized transform coefficient block from quantization unit 208 . As another example, entropy encoding unit 220 may entropy encode prediction syntax elements from mode selection unit 202 (eg, motion information for inter prediction or intra mode information for intra prediction). Entropy encoding unit 220 may perform one or more entropy encoding operations on a syntax element that is another instance of video data to generate entropy encoded data. For example, entropy encoding unit 220 may perform context self-adjusting variable length coding (CAVLC) operations, CABAC operations, variable-to-variable (V2V) length coding operations, syntax-based context self-adjusting binary arithmetic coding (SBAC) operations , a Probabilistic Interval Partitioning Entropy (PIPE) decoding operation, an exponential gorilla coding operation, or another type of entropy coding operation on the data. In some examples, entropy encoding unit 220 may operate in a bypass mode that does not entropy encode syntax elements.

Video transcoder 200 may output a bitstream that includes entropy-encoded syntax elements required for reconstructing blocks of a slice or picture. Specifically, the entropy encoding unit 220 may output a bit stream.

According to AV1, the entropy encoding unit 220 may be configured as a symbol-to-symbol self-adjusting multi-symbol arithmetic decoder. Grammar elements in AV1 consist of an alphabet of N elements, and contexts (e.g., probabilistic models) consist of sets of N probabilities. Entropy encoding unit 220 may store the probabilities as n-bit (eg, 15-bit) cumulative distribution functions (CDFs). Entropy encoding unit 22 may perform recursive scaling using an update factor based on the letter size to update the context.

The operations described above are described with respect to blocks. Such descriptions should be understood to be for the operation of the luma coding block and/or the chroma coding block. As mentioned previously, in some examples, the luma coding block and the chroma coding block are the luma and chroma components of the CU. In some examples, the luma and chroma coding blocks are the luma and chroma components of the PU.

In some examples, operations performed for the luma coding block need not be repeated for the chroma coding block. As an example, the operation of identifying motion vectors (MVs) and reference pictures for luma coding blocks does not need to be repeated to identify MVs and reference pictures for chroma blocks. Instead, the MV for the luma coding block can be scaled to determine the MV for the chroma block, and the reference pictures can be the same. As another example, the intra prediction process may be the same for luma coding blocks and chroma coding blocks.

In one or more examples, for entropy coding such as context-based arithmetic coding, entropy coding unit 220 may determine context values for one or more contexts. During encoding of a slice or picture, entropy encoding unit 220 may update context values (eg, probability values). However, at the beginning of a slice or picture, the context value may be undefined. Entropy encoding unit 220 may initialize one or more context values using a predefined initialization point (e.g., an initial value stored in DPB 218, video data memory 230, or some other memory), rather than undefined. Context value starts.

In one or more examples, instead of or in addition to using predefined initialization points, entropy encoding unit 220 may also use one or more context values of previously encoded slices or pictures, or a mapping of one or more context values. , scaled, weighted, etc. versions as initialization points. For example, entropy encoding unit 220 may store one or more context values (e.g., for the context of the encoded slice or picture) in a buffer (e.g., DPB 218, video data memory 230, or some other memory). After the last CTU of the slice or picture is encoded), it is used as a set of initialization points. Entropy encoding unit 220 may utilize the set of initialization points (eg, context values or values based on context values of previously encoded slices or pictures) to initialize context values for context used to encode subsequent slices or pictures.

Entropy encoding unit 220 may store a plurality of initialization point sets of previously encoded slices or pictures. For example, the buffer may store a first set of initialization points associated with a first slice or picture, a second set of initialization points associated with a second slice or picture, and so on. Additionally, in order to determine (eg, select) which set of initialization points that entropy coding unit 220 may use for subsequent slices or pictures, entropy coding unit 220 may also store slices or pictures associated with each initialization point in the respective sets of initialization points. Information about the timestamp value and/or QP value of the image.

Video transcoder 200 represents an example of a device configured to encode video data. The device includes a memory configured to store video data, and one or more processing units implemented using circuitry. The one or more processing units The processing unit is configured to perform the example techniques described in this case content. For example, in one or more instances, when the buffer is full, entropy encoding unit 220 may determine which set of initialization points to remove to make room for the just determined set of initialization points. For example, entropy encoding unit 220 may determine one or more context values for at least one context used to encode the current slice or picture. Entropy encoding unit 220 may decide that a buffer used to store sets of temporal initialization points from two or more slices or pictures for context-based arithmetic coding is full. As mentioned above, each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures, and includes one or more temporal initialization points.

The entropy encoding unit 220 may determine a first set of temporal initialization points associated with a slice or picture of the two or more slices or pictures based on a temporal identification value or a quantization parameter (QP) value of the slice or picture. The entropy encoding unit 220 may remove the first set of temporal initialization points associated with the slice or picture from the buffer, and store the second set of temporal initialization points associated with the current slice or picture in the buffer. The second set of temporal initialization points is based on the determined one or more context values (e.g., one or more context values equal to the determined one or more context values of at least one context of the current slice or picture, or from at least one context value of the current slice or picture). exported from one or more context values).

Therefore, entropy encoding unit 220 may utilize time stamp values and/or QP values to decide which set of initialization points (including possible slice types) to remove, rather than when to encode a slice or picture as a decision (eg, selection) The only factor that determines which set of initialization points should be removed from the buffer. For example, in order to determine the first temporal initialization point set, the entropy encoding unit 220 may determine the first temporal initialization point set associated with the slice or picture from the two or more slices or pictures, wherein the slice or picture is in The time stamp values or quantization parameter (QP) values of the two or more slices or pictures have at least one of the minimum time stamp value or QP value. For example, the entropy encoding unit 220 may determine the slice or picture with the smallest time identification value from the time identification values of two or more slices or pictures, or determine the QP from the QP values of two or more slices or pictures. The slice or image with the smallest value. Subsequently, entropy encoding unit 220 may remove from the buffer the set of initialization points associated with the determined slice or picture (eg, the initial point with the smallest time stamp value or QP value).

In one or more examples, if no initialization point in the set of initialization points is associated with the following slice or picture, entropy encoding unit 220 may remove the initialization associated with the slice or picture with the smallest time stamp value or QP value. Point set: This slice or picture has the same timestamp value or QP value as the timestamp value or QP value of the current slice or picture. For example, entropy encoding unit 220 may determine that at least one of the temporal identification value or QP value of the current slice or picture is different from the temporal identification value or QP value of each of the two or more slices or pictures. . In this example, entropy encoding unit 220 may determine that at least one of the time identification value or the QP value of the current slice or picture is different from the time identification of each of the two or more slices or pictures. value or QP value to remove the first initialization point set. For example, the time stamp value or QP value of the slice or picture associated with the first set of temporal initialization points is different from the time stamp value or QP value of the current slice or picture.

As an example, assume that the current slice or picture is the first slice or picture. In this example, entropy encoding unit 220 may determine one or more context values for at least one context used to encode a second slice or picture, and determine from the two or more slices or pictures the context value associated with the second slice or picture. A third slice or picture whose time stamp value or QP value has the same time stamp value or at least one of the QP value. In this example, entropy encoding unit 220 may remove a third set of temporal initialization points associated with the third slice or picture from the buffer, and based on the determined at least one parameter for encoding the second slice or picture One or more context values of the context store in the buffer a fourth set of temporal initialization points associated with the second slice or picture.

Subsequently, to encode subsequent pictures, entropy encoding unit 220 may determine (eg, select) a set of temporal initialization points stored in the buffer and initialize the encoding for subsequent slices or pictures based on the selected set of temporal initialization points. One or more context values encoding at least one context. For example, entropy encoding unit 220 may assign an initial value of one or more context values to be equal to, or based on (eg, via mapping, scaling, weighting, etc.) a selected set of temporal initialization points. Initializes one or more context values. Entropy encoding unit 220 may perform context-based arithmetic encoding on subsequent slices or pictures. For example, entropy encoding unit 220 may utilize the initial value of the context value to encode syntax elements of subsequent slices or pictures, and update the context value during encoding of subsequent slices or pictures.

In one or more examples, video transcoder 200 may also be configured to store a plurality of temporal initialization points ( wherein the plurality of time initialization points are included in the video data of one or more previous pictures before the current picture in decoding order), and a corresponding time associated with each of the plurality of time initialization points is stored. An identification (ID) value, selecting at least one time initialization point among the plurality of time initialization points based on each time ID value associated with the plurality of time initialization points and the time ID value of the current picture or slice, and based on the At least one time initialization point is selected to perform context-based arithmetic coding on the video data of the current picture or slice.

Video transcoder 200 may be configured to store, in a first buffer, one or more temporal initializations for one or more contexts used in context-based arithmetic decoding of video data for one or more previous pictures. point, and in the second buffer, store one or more temporal initialization points of the context used in the context-based arithmetic decoding of the video data of the slice of the current picture, wherein the storage in the second buffer includes: Video data for the current picture is stored in the second buffer during decoding and is stored in the first buffer after processing of the last coding tree unit (CTU) or slice of the current picture. time initialization point.

Video transcoder 200 may be configured to store previous temporal initialization points corresponding to one or more contexts used in context-based arithmetic decoding of video material for previous slices (for the current slice) in previous pictures, based on The current slice in the current picture has a position corresponding to the position of the previous slice in the previous picture, determines the current slice in the current picture has a position in the current picture corresponding to the position of the previous slice in the previous picture, based on the previous time Initialization point to determine the current time initialization point of the current slice.

FIG. 3 is a block diagram illustrating an example video decoder 300 that may perform the techniques of this disclosure. Figure 3 is provided for purposes of explanation and should not be considered a limitation on the techniques broadly illustrated and described in this context. For ease of explanation, the content of this case describes the video decoder 300 based on VVC technology (ITU-T H.266 under development) and HEVC technology (ITU-T H.265). However, the techniques of this disclosure may be performed by video decoding devices configured to implement other video decoding standards.

In the example of FIG. 3, video decoder 300 includes coded picture buffer (CPB) memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310. Filter unit 312 and decoded picture buffer (DPB) 314. Any or all of CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 314 may be configured in one or Implemented in multiple processors or in processing circuitry. For example, the units of the video decoder 300 may be implemented as one or more circuits or logic components, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Additionally, video decoder 300 may include supplemental or alternative processors or processing circuitry to perform these and other functions.

Prediction processing unit 304 includes motion compensation unit 316 and intra prediction unit 318. Prediction processing unit 304 may include additional units for performing predictions according to other prediction modes. For example, prediction processing unit 304 may include palette units, intra-block copy units (which may form part of motion compensation unit 316), affine units, linear model (LM) units, and the like. In other examples, video decoder 300 may include more, fewer, or different functional components.

When operating in accordance with AV1, motion compensation unit 316 may be configured to use translational motion compensation, affine motion compensation, OBMC, and/or composite inter-intra prediction to decode blocks of video data (e.g., luma decoding code blocks and chroma decoding blocks) are decoded as described above. Intra-prediction unit 318 may be configured to use directional intra-prediction, non-directional intra-prediction, recursive filtered intra-prediction, CFL, intra-block copy (IBC), and/or palette mode, Decoding blocks of video data (eg, both luma coding blocks and chroma coding blocks) are decoded as described above.

CPB memory 320 may store video data (eg, an encoded video bit stream) to be decoded by components of video decoder 300 . For example, the video data stored in the CPB memory 320 can be obtained from the computer readable medium 110 (FIG. 1). CPB memory 320 may include a CPB that stores encoded video data (eg, syntax elements) from an encoded video bit stream. Furthermore, CPB memory 320 may store video data in addition to the syntax elements of the decoded picture, for example, temporary data representing outputs from various units of video decoder 300 . DPB 314 typically stores decoded pictures, which video decoder 300 may output and/or use as reference video data when decoding subsequent data or pictures in the encoded video bit stream. CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices such as DRAM (including SDRAM), MRAM, RRAM, or other types of storage devices. CPB memory 320 and DPB 314 may be provided by the same memory device or different memory devices. In various examples, CPB memory 320 may be on-die with other components of video decoder 300 or off-die relative to those components.

Additionally or alternatively, in some examples, video decoder 300 may retrieve the decoded video material from memory 120 (FIG. 1). That is, memory 120 may store data, as discussed above with reference to CPB memory 320 . Likewise, memory 120 may store instructions to be executed by video decoder 300 when some or all of the functions of video decoder 300 are implemented using software executed by the processing circuitry of video decoder 300 .

The various units of FIG. 3 are illustrated to illustrate understanding the operations performed by the video decoder 300. These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Similar to Figure 2, a fixed-function circuit represents a circuit that provides a specific function and is preset in terms of the operations it can perform. Programmable circuits represent circuits that can be programmed to perform a variety of tasks and provide flexible functionality in the operations that can be performed. For example, the programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, for receiving parameters or outputting parameters), but the types of operations performed by fixed-function circuits are generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed function or programmable), and in some examples, the one or more units may be integrated circuits.

Video decoder 300 may include ALU, EFU, digital circuits, analog circuits, and/or a programmable core formed of programmable circuits. In instances where operations of video decoder 300 are performed via software executing on programmable circuitry, on-chip or off-chip memory may store instructions (eg, object code) for the software that video decoder 300 receives and executes.

Entropy decoding unit 302 may receive encoded video data from the CPB and entropy decode the video data to reproduce syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310 and filter unit 312 may generate decoded video material based on syntax elements extracted from the bit stream.

Typically, video decoder 300 reconstructs pictures on a block-by-block basis. Video decoder 300 may perform reconstruction operations on each block individually (where the block currently being reconstructed (ie, decoded) may be referred to as the "current block").

Entropy decoding unit 302 may entropy decode syntax elements that define quantized transform coefficients of a block of quantized transform coefficients, and transform information such as quantization parameters (QPs) and/or transform mode indications. Inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine the degree of quantization and, likewise, determine the degree of inverse quantization for inverse quantization unit 306 to apply. For example, inverse quantization unit 306 may perform a bitwise left shift operation to inversely quantize the quantized transform coefficients. Inverse quantization unit 306 may thereby form a transform coefficient block including transform coefficients.

After inverse quantization unit 306 forms the block of transform coefficients, inverse transform processing unit 308 may apply one or more inverse transforms to the block of transform coefficients to produce a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse direction transform, or another inverse transform to the transform coefficient block.

In addition, the prediction processing unit 304 generates prediction blocks based on the prediction information syntax elements entropy-decoded by the entropy decoding unit 302 . For example, if the prediction information syntax element indicates that the current block is inter-predicted, motion compensation unit 316 may generate a prediction block. In this case, the prediction information syntax element may indicate a reference picture in DPB 314 from which the reference block is retrieved, and a motion vector identifying the position of the reference block in the reference picture relative to the current block in the current picture. Motion compensation unit 316 may generally perform inter-frame prediction processing in a manner substantially similar to that described with respect to motion compensation unit 224 (FIG. 2).

As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra prediction unit 318 may generate the prediction block according to the intra prediction mode indicated by the prediction information syntax element. Again, intra-prediction unit 318 may generally perform the intra-prediction process in a manner substantially similar to that described with respect to intra-prediction unit 226 (FIG. 2). Intra-frame prediction unit 318 may retrieve data for adjacent samples of the current block from DPB 314 .

Reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

Filter unit 312 may perform one or more filtering operations on the reconstructed blocks. For example, filter unit 312 may perform deblocking operations to reduce blocking artifacts along the edges of reconstructed blocks. The operation of filter unit 312 may not necessarily be performed in all instances.

Video decoder 300 may store the reconstructed blocks in DPB 314. For example, in an example where the operations of filter unit 312 are not performed, reconstruction unit 310 may store the reconstructed blocks into DPB 314 . In an example in which the operations of filter unit 312 are performed, filter unit 312 may store the filtered reconstruction block in DPB 314 . As mentioned above, DPB 314 may provide reference information to prediction processing unit 304, such as samples of the current picture for intra-frame prediction and previously decoded pictures for subsequent motion compensation. Additionally, video decoder 300 may output decoded pictures (eg, decoded video) from DPB 314 for subsequent presentation on a display device, such as display device 118 of FIG. 1 .

In this manner, video decoder 300 represents an example of a video decoding device that includes a memory configured to store video data, and one or more processing units implemented in circuitry, wherein the one or more processing units The processing unit is configured to perform the example techniques described in this case. For example, for entropy decoding such as context-based arithmetic decoding, entropy decoding unit 302 may determine context values for one or more contexts. During decoding of a slice or picture, entropy decoding unit 302 may update context values (eg, probability values). However, at the beginning of a slice or picture, the context value may be undefined. Entropy decoding unit 302 may initialize one or more context values using a predefined initialization point (e.g., an initial value stored in DPB 314, CPB memory 320, or some other memory), rather than from an undefined context. value starts.

In one or more examples, in addition to or instead of using predefined initialization points, entropy decoding unit 302 may also use one or more context values from previously decoded slices or pictures, or a mapping of one or more context values. , scaled, weighted, etc. versions as initialization points. For example, entropy decoding unit 302 may store one or more context values for the context of the decoded slice or picture (e.g., in a buffer (e.g., DPB 314, CPB memory 320, or some other memory)) Or after decoding the last CTU of the picture), as a set of initialization points. Entropy decoding unit 302 may utilize the set of initialization points (eg, context values or values based on context values of previously decoded slices or pictures) to initialize context values for context used to decode subsequent slices or pictures.

Entropy decoding unit 302 may store a set of initialization points for multiple previously decoded slices or pictures. For example, the buffer may store a first set of initialization points associated with a first slice or picture, a second set of initialization points associated with a second slice or picture, and so on. Additionally, in order to determine (eg, select) which set of initialization points that entropy decoding unit 302 may use for subsequent slices or pictures, entropy decoding unit 302 may also store slices or pictures associated with each initialization point in the respective sets of initialization points. time stamp value and/or QP value information.

In one or more instances, when the buffer is full, entropy decoding unit 302 may determine which set of initialization points to remove to make room for the just decided set of initialization points. For example, entropy decoding unit 302 may determine one or more context values for at least one context used to decode the current slice or picture. Entropy decoding unit 302 may decide that a buffer used to store sets of temporal initialization points from two or more slices or pictures for context-based arithmetic coding is full. As mentioned above, each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures, and includes one or more temporal initialization points.

The entropy decoding unit 302 may determine a first set of temporal initialization points associated with a slice or picture of the two or more slices or pictures based on a temporal identification value or a quantization parameter (QP) value of the slice or picture. The entropy decoding unit 302 may remove the first set of temporal initialization points associated with the slice or picture from the buffer, and store the second set of temporal initialization points associated with the current slice or picture in the buffer. The second set of temporal initialization points is based on the determined one or more context values (e.g., one or more context values equal to the determined one or more context values of the current slice or picture, or based on at least one determined context value of the current slice or picture). exported from one or more context values of the context).

Therefore, entropy decoding unit 302 may utilize the time stamp value and/or the QP value to decide which set of initialization points to remove, rather than when the slice or picture is decoded as the basis for deciding (eg, selecting) which set of initialization points should be removed from the buffer. The only factor that initializes the point collection. For example, in order to determine the first temporal initialization point set, the entropy decoding unit 302 may determine the first temporal initialization point set associated with the slice or picture from the two or more slices or pictures, wherein the slice or picture is in The time stamp values or quantization parameter (QP) values of the two or more slices or pictures have at least one of the minimum time stamp value or QP value. For example, the entropy decoding unit 302 may determine the slice or picture with the smallest time identification value from the time identification values of two or more slices or pictures, or determine the QP from the QP values of two or more slices or pictures. The slice or image with the smallest value. Subsequently, entropy decoding unit 302 may remove from the buffer the set of initialization points associated with the determined slice or picture (eg, the one with the smallest time stamp value or QP value).

In one or more examples, if no initialization point in the set of initialization points is associated with the following slice or picture, entropy decoding unit 302 may remove the initialization associated with the slice or picture with the smallest time stamp value or QP value. Point set: This slice or picture has the same timestamp value or QP value as the timestamp value or QP value of the current slice or picture. For example, entropy decoding unit 302 may determine that at least one of the temporal identification value or QP value of the current slice or picture is different from the temporal identification value or QP value of each of the two or more slices or pictures. . In this example, entropy decoding unit 302 may determine that at least one of the time identification value or the QP value of the current slice or picture is different from the time identification of each of the two or more slices or pictures. value or QP value to remove the first initialization point set. For example, the time stamp value or QP value of the slice or picture associated with the first set of temporal initialization points is different from the time stamp value or QP value of the current slice or picture.

As an example, assume that the current slice or picture is the first slice or picture. In this example, entropy decoding unit 302 may determine one or more context values for at least one context used to decode a second slice or picture, and determine from the two or more slices or pictures the context value associated with the second slice or picture. A third slice or picture whose time stamp value or QP value has the same time stamp value or at least one of the QP value. In this example, entropy decoding unit 302 may remove a third set of temporal initialization points associated with the third slice or picture from the buffer, and based on the determined at least one parameter for decoding the second slice or picture One or more context values of the context store in the buffer a fourth set of temporal initialization points associated with the second slice or picture.

Subsequently, to decode subsequent pictures, entropy decoding unit 302 may determine (eg, select) a set of temporal initialization points stored in the buffer and initialize the decoding of subsequent slices or pictures based on the selected set of temporal initialization points. One or more context values of the decoded at least one context. For example, entropy decoding unit 302 may assign an initial value of one or more context values to be equal to a selected set of temporal initialization points, or may be based on (eg, via mapping, scaling, weighting, etc.) a selected set of temporal initialization points. Initializes one or more context values. Entropy decoding unit 302 may perform context-based arithmetic decoding on subsequent slices or pictures. For example, entropy decoding unit 302 may utilize the initial value of the context value to decode syntax elements of subsequent slices or pictures, and update the context value during decoding of subsequent slices or pictures.

In one or more examples, video decoder 300 may also be configured to store a plurality of temporal initialization points for one or more contexts used in context-based arithmetic coding of video data for the current picture or slice (where The plurality of time initialization points are included in the video data of one or more previous pictures before the current picture in decoding order), and a corresponding time identifier associated with each of the plurality of time initialization points is stored. (ID) value, selecting at least one of the plurality of time initialization points based on each time ID value associated with the plurality of time initialization points and the time ID value of the current picture or slice, and based on the selected At least one time initialization point, context-based arithmetic decoding is performed on the video data of the current picture or slice.

Video decoder 300 may be configured to store, in a first buffer, one or more temporal initialization points for one or more contexts used in context-based arithmetic decoding of video material of one or more previous pictures. , and in the second buffer, store one or more context temporal initialization points used in the context-based arithmetic decoding of the video data of the slice of the current picture, wherein the storage in the second buffer includes: in the current The video data of the picture is stored in the second buffer during decoding, and after processing the last coding tree unit (CTU) or slice of the current picture, the temporal initialization point stored in the second buffer is stored in the second buffer. in a buffer.

Video decoder 300 may be configured to store previous temporal initialization points for one or more contexts used in context-based arithmetic coding of video material for previous slices in previous pictures (for the current slice), based on the current picture. The current slice in the current picture has a position corresponding to the position of the previous slice in the previous picture, determines that the current slice in the current picture has a position in the current picture corresponding to the position of the previous slice in the previous picture, based on the previous time initialization point to determine the current time initialization point of the current slice.

4 is a flowchart illustrating an example method for encoding a current block in accordance with the techniques of this disclosure. The current block may include the current CU. Although described with respect to the video transcoder 200 (Figs. 1 and 2), it should be understood that other devices may also be configured to perform a method similar to that of Fig. 4.

In this example, video transcoder 200 initially predicts the current block (400). For example, the video transcoder 200 may form a prediction block for the current block. Video transcoder 200 may then calculate a residual block for the current block (402). To calculate the residual block, video transcoder 200 may calculate the difference between the original uncoded block and the predicted block of the current block. Subsequently, the video transcoder 200 may transform the residual block and quantize the transform coefficients of the residual block (404). Next, video transcoder 200 may scan the quantized coefficients of the residual block (406). During or after scanning, video transcoder 200 may entropy encode the transform coefficients (408). For example, the video transcoder 200 may use CAVLC or CABAC to encode the transform coefficients. According to one or more examples, video transcoder 200 may encode transform coefficients using context values determined using techniques described in this document. Video transcoder 200 may then output the entropy-encoded data of the block (410).

5 is a flowchart illustrating an example method for decoding a current block of video material in accordance with the techniques of this disclosure. The current block may include the current CU. Although described with respect to video decoder 300 (Figures 1 and 3), it should be understood that other devices may be configured to perform methods similar to Figure 5.

Video decoder 300 may receive entropy-encoded information for the current block (eg, entropy-encoded prediction information and entropy-encoded information for transform coefficients of a residual block corresponding to the current block) (500). The video decoder 300 may perform entropy decoding on the entropy-encoded data to determine prediction information of the current block and reproduce transform coefficients of the residual block (502). According to one or more examples, video decoder 300 may decode encoded material using context values determined using techniques described in this document. Video decoder 300 may predict the current block (504), for example using an intra or inter prediction mode as indicated by prediction information for the current block, to calculate a predicted block for the current block. Video decoder 300 may then inverse scan (506) the rendered transform coefficients to create a block of quantized transform coefficients. Video decoder 300 may then inversely quantize the transform coefficients and apply the inverse transform to the transform coefficients to produce a residual block (508). Video decoder 300 may finally decode the current block by combining the prediction block and the residual block (510).

Figure 6 is a flowchart illustrating an example method for processing video data. For ease of explanation, the example of FIG. 6 is described with respect to processing circuits and buffers. Examples of processing circuits include the processing circuits of video transcoder 200 and video decoder 300 . Examples of buffers include memory 106 , memory 120 , video data memory 230, DPB 218, CPB memory 320, DPB 314, or any other memory for the video transcoder 200 or the video decoder 300.

The processing circuitry may be configured to determine one or more context values for at least one context used to encode or decode the current slice or picture (600). For example, when the video transcoder 200 and the video decoder 300 are encoding or decoding the current slice or picture, the video transcoder 200 or the video decoder 300 may update the most recently encoded or decoded value for the context-based slice or picture. The context value of the video data.

The processing circuitry may determine that a buffer used to store sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full (602). As mentioned above, each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures, and includes one or more temporal initialization points. For example, there may be a limit on the number of sets of temporal initialization points that a buffer can store to keep the size of the buffer practical. As an example, the buffer can store up to five sets of time initialization points.

The processing circuit may determine (eg, identify) a slice or picture association from the two or more slices or pictures based on at least one of a slice type, a time stamp value, or a quantization parameter (QP) value of the slice or picture The first time initialization point set (604). For example, the buffer may also store timestamp values, QP values, and/or slice type information of slices and pictures associated with each set of initialization points stored in the buffer.

As an example, in order to determine the first temporal initialization point set, the processing circuit may be configured to determine, from the two or more slices or pictures, the first temporal initialization point set associated with the slice or picture, which is in the first temporal initialization point set. The time stamp values or quantization parameter (QP) values of two or more slices or pictures have at least one of the smallest time stamp value or QP value. For example, the processing circuit may determine the slice or picture with the smallest time stamp value from the time stamp values of the two or more slices or pictures, and/or from the QP values of the two or more slices or pictures Determine the slice or image with the smallest QP value.

In some examples, the first set of initialization points with the smallest time stamp value or QP value may have a different slice type than the current slice. In some examples, such as where multiple sets of temporal initialization points are stored for a slice type, video transcoder 200 and video decoder 300 may determine the set of temporal initialization points associated with a slice having the same slice type as the current slice. of packets. Subsequently, the video transcoder 200 and the video decoder 300 may determine the time initialization point set with the minimum time stamp value or QP value in the packet as the first time initialization point set.

The processing circuit may remove the first set of initialization points associated with the slice or picture from the buffer (606). The processing circuit may store a second set of temporal initialization points associated with the current slice or picture in the buffer (608). The second set of temporal initialization points is based on the determined context value(s) (eg, equal to or derived from the determined context value(s) for the current slice or picture). In some examples, the processing circuitry may store a second set of temporal initialization points after processing the last coding tree unit (CTU) of the current slice or picture.

In one or more examples, processing circuitry may determine (eg, select) a set of temporal initialization points to be stored in a buffer for subsequent slices or pictures in coding order. The processing circuitry may initialize one or more context values for at least one context used to encode or decode subsequent slices or pictures based on the selected set of temporal initialization points (e.g., set an initial value of the context value equal to the selected a set of time initialization points, or export the initial value of the context value based on a selected set of time initialization points). Processing circuitry can perform context-based arithmetic encoding or decoding of subsequent slices or pictures.

Figure 7 is a flowchart illustrating another example method for processing video data. For ease of explanation, the example of FIG. 7 is described with respect to processing circuits and buffers. Examples of processing circuits include the processing circuits of video transcoder 200 and video decoder 300 . Examples of buffers include memory 106 , memory 120 , video data memory 230, DPB 218, CPB memory 320, DPB 314, or any other memory used in the video transcoder 200 or the video decoder 300.

As previously described, the processing circuit may remove the set of temporal initialization points associated with the slice or picture having the smallest temporal stamp value or QP value. However, in some examples, the processing circuitry may perform such removal if there is no set of temporal initialization points associated with a slice or picture that has the same time stamp value, QP value, or slice type as the current slice or picture.

For example, assume that the current slice or picture of Figure 6 is the first slice or picture. In this example, the processing circuitry may determine one or more context values for at least one context used to encode or decode the second slice or picture (700). That is, the processing circuitry may perform similar encoding or decoding operations on the second slice or picture and update the context value as previously described.

The processing circuit may determine a third slice or picture from the two or more slices or pictures, the third slice or picture having the same time stamp value, QP value or slice type as the second slice or picture. At least one of QP value and slice type (702). In this example, the processing circuitry may remove the third set of temporal initialization points associated with the third slice or picture from the buffer, rather than removing the initialization point associated with the slice or picture with the smallest temporal stamp value or QP value. Collection(704). The processing circuit may store in the buffer a fourth temporal initialization point associated with the second slice or picture based on the determined one or more context values of the at least one context used to encode or decode the second slice or picture. Collection(706).

The example of Figure 7 is for illustrative purposes only and should not be considered limiting. In some examples, the processing circuitry may not perform the method of Figure 7. Instead, the processing circuit may remove the set of initialization points based on the time stamp value or QP value (eg, the set of time initialization points associated with the slice or picture with the smallest time stamp value or QP value). Additionally, slice type may not be considered one of these factors. That is, the processing circuit may remove entries in the temporal initialization point set that have the same slice type and/or the same time identifier value and/or the same QP value as the current slice or picture.

Figure 8 is a flowchart illustrating another example method of processing video data. For ease of description, the example of FIG. 8 is described with respect to processing circuits and buffers. Examples of the processing circuits include the processing circuits of the video transcoder 200 and the video decoder 300 . Examples of the buffers include the memory 106 , the memory 120 , video data memory 230, DPB 218, CPB memory 320, DPB 314, or any other memory used in the video transcoder 200 or the video decoder 300.

Similar to Figures 6 and 7, processing circuitry may determine one or more context values for at least one context used to encode or decode the current slice or picture (800). In one or more examples, the processing circuitry may determine whether the buffer has stored a set of temporal initialization points for a slice or picture that has the same time stamp value or QP value as the current slice or picture (802).

If the buffer already stores a set of temporal initialization points for a slice or picture that has the same timestamp value or QP value as the current slice or picture ("Yes" at 802), then the processing circuitry may use the temporal initialization of the current slice or picture. Points to overwrite (804) a stored set of time initialization points (eg, one or more context values determined or exported based on the one or more context values). The processing circuitry may then set the next slice or picture to be the current slice or picture and return one or more context values that determine at least one context used to encode or decode the current slice or picture (800).

If the buffer has not yet stored a set of temporal initialization points for a slice or picture with the same timestamp value or QP value as the current slice or picture (No at 802), the processing circuitry may determine whether the buffer is full (806) . If the buffer is not full (NO at 806), the processing circuitry may store a temporal initialization point (e.g., the context value of the current slice or picture, or a value exported from the context value of the current slice or picture) in in buffer (808). The processing circuitry may then set the next slice or picture to be the current slice or picture and return one or more context values that determine at least one context used to encode or decode the current slice or picture (800).

If the buffer is full (YES at 806), the processing circuitry may determine (eg, identify) a first set of initialization points associated with the slices or pictures from two or more slices or pictures. , which has at least one of the smallest time signature value or QP value among the time signature values or quantization parameter (QP) values of the two or more slices or pictures (810). Similar to Figure 6, the processing circuit may remove the first time initialization point set associated with the slice or picture from the buffer (812) and store the second time initialization point associated with the current slice or picture in the buffer. A set of initialization points, wherein a second set of temporal initialization points is based on the determined one or more context values (814).

The example sequence of operations shown in Figure 8 and performed by the processing circuitry should not be considered limiting. For example, the processing circuit may first determine whether the buffer is full (806), and if the buffer is not full, the processing circuit may store the initialization point (808) even if the buffer already stores the same timestamp value as the current slice or picture Or the temporal initialization point of a slice or picture of QP values. As another example, if the buffer is full, the processing circuitry may remove the set of time initialization points associated with the slice or picture with the smallest time stamp value or QP value, even if the buffer already stores the same set of time initialization points as the current slice or picture. The time initialization point of the slice or picture of the timestamp value and QP value. Other modifications to the sequence of operations are possible.

Techniques shown by reference numerals 804 and 812 include instances of removing a set of initialization points. Generally, the processing circuit can first determine whether the buffer stores a time initialization point with the same time ID or QP value as the current slice or picture. If so, remove the time initialization point set and write the time initialization point of the current slice or picture. . Otherwise, the processing circuitry may remove the determined (eg, identified) set of initialization points associated with the slice or picture having the smallest time stamp value or QP value.

Example techniques that may be performed together or separately are described below.

Clause 1. A method of decoding video data, the method comprising: storing a plurality of temporal initialization points for one or more contexts used in context-based arithmetic decoding of video data of a current picture or slice, wherein the plurality of The time initialization point is included in the video data of one or more previous pictures before the current picture in decoding order; storing each time identification (ID) associated with each of the plurality of time initialization points. value; selecting at least one of the plurality of time initialization points (e.g., if available) a time initialization point based on the respective time ID value associated with the plurality of time initialization points and the time ID value of the current picture or slice; And based on the selected at least one time initialization point, context-based arithmetic decoding is performed on the video data of the current picture or slice.

Clause 2. The method according to clause 1, wherein selecting the at least one time initialization point includes: determining a time initialization point set among the plurality of time initialization points, wherein each time ID value of the time initialization point in the time initialization point set is less than or equal to the The time ID value of the current picture or slice; and select at least one time initialization point from the time initialization point set.

Clause 3. The method according to clause 1, wherein selecting the at least one time initialization point includes: determining a time initialization point set among the plurality of time initialization points, wherein each time ID value of the time initialization point in the time initialization point set is equal to the current picture or the time ID value of the slice; and select at least one time initialization point from the time initialization point set.

Clause 4. The method according to clause 1, wherein selecting the at least one temporal initialization point includes: determining that no temporal initialization point among the plurality of temporal initialization points has a temporal ID value equal to the temporal ID value of the current picture or slice; based on the plurality of temporal The determination that no time initialization point among the initialization points has a time ID value equal to the time ID value of the current picture or slice determines whether any of the plurality of time initialization points has a time ID value greater than that of the current picture or slice. a temporal ID value with an ID value less than one; and selecting the one or more temporal initialization points from the one or more temporal initialization points based on determining that there are one or more temporal initialization points with a temporal ID value that is one smaller than the temporal ID value of the current picture or slice. At least one temporal initialization point, which will have a temporal ID value less than one than the temporal ID value of the current picture or slice.

Clause 5. A method of decoding video data, the method comprising: storing in a first buffer one or more contexts used in context-based arithmetic decoding of the video data for one or more previous pictures. one or more temporal initialization points; in the second buffer, store temporal initialization points for one or more contexts used in context-based arithmetic decoding of the video material of the slice of the current picture, wherein in the second buffer Storing in the second buffer includes: storing in the second buffer during decoding of the video data of the current picture; and storing after processing the last coding tree unit (CTU) or slice of the current picture. The time initialization point in the second buffer is stored in the first buffer.

Clause 6. The method according to any one of clauses 1-4, wherein storing the plurality of time initialization points includes: according to the method of clause 5, storing the plurality of time initialization points.

Clause 7. A method of decoding video data, the method comprising: storing one or more context-corresponding previous time initialization points used in context-based arithmetic decoding of the video data of previous slices in previous pictures; For the current slice, it is determined that the current slice in the current picture has a position in the current picture corresponding to the position of the previous slice in the previous picture; based on the current slice having in the current picture a position corresponding to the position in the previous picture The current time initialization point for the current slice is determined based on the previous time initialization point.

Clause 8. A method of decoding video material, including in accordance with any combination of clauses 1-7.

Clause 9. A method according to any one of clauses 1-8, wherein the context-based arithmetic coding includes context self-adjusting binary arithmetic coding (CABAC).

Clause 10. A method as in any one of clauses 1-9, wherein context-based arithmetic decoding includes context-based arithmetic decoding.

Clause 11. A method as in any one of clauses 1-9, wherein context-based arithmetic decoding includes context-based arithmetic encoding.

Clause 12. A device for decoding video data, the device comprising: a memory configured to store the video data; and a processing circuit configured to perform any one or combination of clauses 1-11 method.

Clause 13. Equipment under clause 12, wherein the equipment includes a video decoder.

Clause 14. Equipment according to any of clauses 12 and 13, wherein the equipment includes a video transcoder.

Clause 15. Apparatus according to any of clauses 12-14 also includes a display configured to display decoded video data.

Clause 16. Equipment according to any of clauses 12-15, wherein the equipment includes one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device or a set-top box.

Clause 17. A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform the method described in any of Clauses 1-11.

Clause 18. A device for decoding video material, the device comprising: a unit for performing the method according to any one or combination of clauses 1-11.

Clause 1A. A method for processing video data, the method includes: determining one or more context values of at least one context used to encode or decode the current slice or picture; deciding to store data from two or more slices or A buffer is full for a set of temporal initialization points of a picture for context-based arithmetic decoding, where each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures and includes one or Multiple temporal initialization points; determining the slice or picture to be associated with from the two or more slices or pictures based on at least one of the slice type, time stamp value, or quantization parameter (QP) value of the slice or picture the first time initialization point set; remove the first time initialization point set associated with the slice or picture from the buffer; and store the second time initialization point set associated with the current slice or picture in the buffer A set of initialization points, wherein the second set of temporal initialization points is based on the determined one or more context values.

Clause 2A. The method according to clause 1A, wherein determining the first temporal initialization point set includes: determining the first temporal initialization point set associated with the slice or picture from the two or more slices or pictures, wherein There is at least one of the smallest time stamp value or QP value among the time stamp values or quantization parameter (QP) values of the two or more slices or pictures.

Clause 3A. The method according to clause 2A, wherein determining the slice or picture having at least one of the minimum time stamp value or the QP value from the two or more slices or pictures includes: from the two or more Among the time stamp values of the above slices or pictures, the slice or picture with the smallest time stamp value is determined.

Clause 4A. The method according to clause 2A, wherein determining the slice or picture with at least one of the minimum time stamp value or the QP value from the two or more slices or pictures includes: from the two or more Among the QP values of a slice or picture, determine the slice or picture with the smallest QP value.

Clause 5A. The method according to any one of clauses 1A-4A, also comprising: determining that at least one of the time stamp value or QP value of the current slice or picture is different from each of the two or more slices or pictures. a time stamp value or QP value of a slice or picture, wherein the time stamp value or QP value of the slice or picture associated with the first set of time initialization points is different from the time stamp value or QP value of the current slice or picture or QP value, and wherein removing the first temporal initialization point set includes: determining that at least one of the temporal identification value or the QP value of the current slice or picture is different from each of the two or more slices or pictures. or the time stamp value and QP value of the slice or picture to remove the first time initialization point set.

Clause 6A. A method according to any one of clauses 1A-5A, wherein the current slice or picture is a first slice or picture, the method further comprising: determining at least one of the parameters for encoding or decoding the second slice or picture. One or more context values of the context; from the two or more slices or pictures, determine that the time stamp value, QP value or slice type has the same time stamp value, QP value or slice type as the second slice or picture a third slice or picture of at least one of the slice types; removing from the buffer a third set of temporal initialization points associated with the third slice or picture; and based on the determined value for the second slice or one or more context values of at least one context in which a picture is encoded or decoded, and a fourth set of temporal initialization points associated with the second slice or picture is stored in the buffer.

Clause 7A. The method according to any one of clauses 1A-6A, further comprising: determining a set of temporal initialization points stored in the buffer; and, based on the determined set of temporal initialization points, encoding or decoding subsequent slices or pictures. Initialize one or more context values of at least one context; and perform context-based arithmetic encoding or decoding on the subsequent slice or picture.

Clause 8A. The method according to clause 7A, further comprising: determining a time stamp value of the subsequent slice or picture; determining two or more slices or pictures having an associated set of time initialization points stored in the buffer. No slice or picture has a time stamp value equal to the time stamp value of the subsequent slice or picture; and from the two or more slices or pictures, determine the time stamp value that is closest to and smaller than that of the subsequent slice or picture The second slice or picture of the time identification value, wherein determining the set of time initialization points includes: selecting the set of time initialization points associated with the second slice or picture.

Clause 9A. The method of clause 7A, further comprising: determining a QP value for the subsequent slice or picture; determining that none of the two or more slices or pictures has an associated set of temporal initialization points stored in the buffer a slice or picture having a QP value equal to the QP value of the subsequent slice or picture; and from the two or more slices or pictures, determine the second slice whose QP value is closest to the QP value of the subsequent slice or picture or picture, wherein determining the set of temporal initialization points includes: selecting the set of initialization points associated with the second slice or picture.

Clause 10A. The method of clauses 1A-9A, wherein storing the second set of initialization points includes storing the second set of temporal initialization points after processing the last coding tree unit (CTU) of the current slice or picture.

Clause 11A. An apparatus for processing video data, the apparatus comprising: a buffer configured to store a set of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding; and Processing circuitry coupled to the buffer, the processing circuitry configured to: determine one or more context values for at least one context for encoding or decoding the current slice or picture; determine for use in storing the context values from two or both The buffer for the above set of temporal initialization points for context-based arithmetic decoding of slices or pictures, where each set of temporal initialization points is associated with a slice or picture of the two or more slices or pictures, is full, and Including one or more time initialization points; based on at least one of the slice type, time stamp value or quantization parameter (QP) value of the slice or picture, determine from the two or more slices or pictures the same as the slice or picture The set of first-time initialization points associated with the picture; remove the set of first-time initialization points associated with the slice or picture from the buffer; and store the set of first-time initialization points associated with the current slice or picture in the buffer. A second set of temporal initialization points, wherein the second set of temporal initialization points is based on the determined one or more context values.

Clause 12A. The device according to clause 11A, wherein in order to determine the first set of temporal initialization points, the processing circuit is configured to: from the two or more slices or pictures, determine the third slice or picture associated with the slice or picture. A set of temporal initialization points that has at least one of the smallest temporal identifier value or QP value among the temporal identifier values or quantization parameter (QP) values of the two or more slices or pictures.

Clause 13A. The apparatus according to clause 12A, wherein in order to determine the slice or picture having at least one of the minimum time stamp value or the QP value from the two or more slices or pictures, the processing circuit is configured to: From the time stamp values of the two or more slices or pictures, the slice or picture with the smallest time stamp value is determined.

Clause 14A. The apparatus according to clause 12A, wherein in order to determine the slice or picture having at least one of the minimum time stamp value or the QP value from the two or more slices or pictures, the processing circuit is configured to: from Among the QP values of the two or more slices or pictures, the slice or picture with the smallest QP value is determined.

Clause 15A. The device according to any one of clauses 11A-14A, wherein the processing circuit is configured to: determine that at least one of the time stamp value or QP value of the current slice or picture is different from the two or more A time stamp value or QP value for each of the slices or pictures, wherein the time stamp value or QP value of the slice or picture associated with the first set of time initialization points is different from the time stamp value of the current slice or picture value or QP value, and wherein in order to remove the first time initialization point set, the processing circuit is configured to: determine that at least one of the time identification value or QP value of the current slice or picture is different from the two or The time stamp value and the QP value of each of the two or more slices or pictures are used to remove the first time initialization point set.

Clause 16A. Apparatus according to any of clauses 11A-15A, wherein the current slice or picture is a first slice or picture, and wherein the processing circuit is configured to: decide to encode a second slice or picture or One or more context values of the decoded at least one context; from the two or more slices or pictures, it is determined that the time stamp value, QP value or slice type of the second slice or picture has the same time stamp value a third slice or picture of at least one of a , QP value, or slice type; removing from the buffer a third set of temporal initialization points associated with the third slice or picture; and based on the determination, a third slice or picture for One or more context values of at least one context in which the second slice or picture is encoded or decoded, and a fourth temporal initialization point set associated with the second slice or picture is stored in the buffer.

Clause 17A. The device according to any one of clauses 11A-16A, wherein the processing circuit is configured to: determine a set of time initialization points stored in the buffer; based on the determined set of time initialization points, for encoding or decoding one or more context values of at least one context of a subsequent slice or picture to initialize; and performing context-based arithmetic encoding or decoding of the subsequent slice or picture.

Clause 18A. Apparatus according to clause 17A, wherein the processing circuit is configured to: determine a time stamp value of the subsequent slice or picture; determine two or more sets of associated time initialization points stored in the buffer. None of the above slices or pictures has a time stamp value equal to the time stamp value of the subsequent slice or picture; and from the two or more slices or pictures, it is decided that the time stamp value is closest to and smaller than the A second slice or picture of the time identification value of a subsequent slice or picture, wherein in order to determine the set of time initialization points, the processing circuit is configured to: select the set of time initialization points associated with the second slice or picture.

Clause 19A. Apparatus according to clause 17A, wherein the processing circuit is configured to: determine a QP value for the subsequent slice or picture; determine two or more slices having an associated set of temporal initialization points stored in the buffer or none of the slices or pictures in the picture has a QP value equal to the QP value of the subsequent slice or picture; and from the two or more slices or pictures, determine the QP value closest to the QP of the subsequent slice or picture A second slice or picture of values, wherein in order to determine the set of temporal initialization points, the processing circuit is configured to: select the set of initialization points associated with the second slice or picture.

Clause 20A. An apparatus according to any one of clauses 11A-19A, wherein to store the second set of initialization points, the processing circuit is configured to: after processing the last coding tree unit (CTU) of the current slice or picture, Store the second time initialization point set.

Clause 21A. Equipment under any of clauses 11A-20A, wherein the equipment includes one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device or a set-top box.

Clause 22A. A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform the following operations: determine at least one context for encoding or decoding the current slice or picture. One or more context values; determines that the buffer used to store sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full, where each set of temporal initialization points is associated with the two Slices or pictures of one or more slices or pictures are associated and include one or more temporal initialization points; based on at least one of a slice type, a time stamp value or a quantization parameter (QP) value of the slice or picture, Determine a first time initialization point set associated with the slice or picture from the two or more slices or pictures; remove the first time initialization point set associated with the slice or picture from the buffer; And in the buffer, a second set of temporal initialization points associated with the current slice or picture is stored, wherein the second set of temporal initialization points is based on the determined one or more context values.

Clause 23A. The computer-readable storage medium described in Clause 22A also includes instructions causing the one or more processors to perform the method described in any of Clauses 1A-10A.

Clause 24A. A device for processing video data, the device includes: a unit for determining one or more context values of at least one context used to encode or decode the current slice or picture; A set of temporal initialization points in two or more slices or pictures for buffer-full units for context-based arithmetic decoding, wherein each set of temporal initialization points is associated with a set of temporal initialization points in the two or more slices or pictures Slices or pictures are associated and include one or more temporal initialization points; for based on at least one of the slice type, temporal identification value or quantization parameter (QP) value of the slice or picture, from the two or more A unit in a slice or picture that determines the first time initialization point set associated with the slice or picture; a unit used to remove the first time initialization point set associated with the slice or picture from the buffer; with A unit storing a second set of temporal initialization points associated with the current slice or picture in the buffer, wherein the second set of temporal initialization points is based on the determined one or more context values.

Clause 25A. An apparatus according to clause 24A, also comprising instructions for causing the one or more processors to perform a method according to any of clauses 1A-10A.

It is understood that, depending on the instance, certain acts or events of any of the techniques described herein may be performed in a different sequence, may be added together, combined, or omitted (e.g., not all The actions or events described are necessary for the practice of the technology). Furthermore, in some instances, actions or events may be performed concurrently, for example, via multi-thread processing, interrupt processing, or multiple processors, rather than sequentially.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or sent over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media that correspond to tangible media (such as data storage media), or may include, for example, computers that facilitate communication from one place to another in accordance with a communications protocol Any medium of communication over which the program is transmitted. In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data used to implement the technology described in this case. structure. Computer program products may include computer-readable media.

By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or may be used for Store the desired program code in the form of instructions or data structures and any other medium that can be accessed by the computer. Also, any connection is properly termed a computer-readable medium. For example, if coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave) are used to send instructions from a website, server, or other remote source, coaxial cable, Fiber optic cable, twisted pair, DSL or wireless technologies (such as infrared, radio and microwave) are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead refer to non-transitory, tangible storage media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs use lasers to optically reproduce data. Reproduce data. The above combinations should also be included in the category of computer-readable media.

Instructions may be executed by one or more processors, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated logic circuits or individual logic circuits. Accordingly, the terms "processor" and "processing circuitry" as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware modules and/or software modules configured for encoding and decoding, or incorporated into a combined transcoder. Additionally, such techniques may be substantially implemented in one or more circuits or logic elements.

The technology at issue may be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or collections of ICs (e.g., chip sets). Various components, modules, or units are described in this context to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by distinct hardware units. Rather, as described above, the various units may be combined in a transcoder hardware unit, or via a set of hardware units that interact with each other in conjunction with appropriate software and/or firmware (including as described above) one or more processors as described in this article).

Various examples have been described. These and other examples are within the scope of the following claims.

100: Video encoding and decoding system 102: Source device 104:Video source 106:Memory 108:Output interface 110: Computer readable media 112:Storage device 114:File server 116:Target device 118:Display device 120:Memory 122:Input interface 200:Video transcoder 202: Mode selection unit 204: Residual generation unit 206: Transformation processing unit 208: Quantization unit 210: Inverse quantization unit 212: Inverse transformation processing unit 214: Reconstruction unit 216: Filter unit 218: Decoded Picture Buffer (DPB) 220: Entropy coding unit 222: Motion estimation unit 224: Motion compensation unit 226: In-frame prediction unit 230: Video data memory 300:Video decoder 302: Entropy decoding unit 304: Prediction processing unit 306: Inverse quantization unit 308: Inverse transformation processing unit 310: Reconstruction unit 312: Filter unit 314: Decoded Picture Buffer (DPB) 316: Motion compensation unit 318: In-frame prediction unit 320:CPB memory 400:block 402:Block 404:Block 406:Block 408:Block 410:block 500:block 502: Block 504:Block 506:Block 508:Block 510:block 600:block 602: Block 604: Block 606:Block 608:Block 700:block 702:Block 704:Block 706:Block 800:block 802: Block 804: Block 806: Block 808: Block 810:block 812:block 814:block

FIG. 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure.

FIG. 2 is a block diagram illustrating an example video transcoder that may perform the techniques of this disclosure.

3 is a block diagram illustrating an example video decoder that may perform the techniques of this disclosure.

4 is a flowchart illustrating an example method for encoding a current block in accordance with the techniques of this disclosure.

5 is a flowchart illustrating an example method for decoding a current block in accordance with the techniques of this disclosure.

Figure 6 is a flowchart illustrating an example method for processing video data.

Figure 7 is a flowchart illustrating another example method for processing video data.

8 is a flowchart illustrating another example method for processing video data.

Domestic storage information (please note in order of storage institution, date and number) without Overseas storage information (please note in order of storage country, institution, date, and number) without

800:block

802: Block

804: Block

806: Block

808: Block

810:block

812:block

814:block

Claims (23)

A method for processing video data, the method includes the following steps: determining one or more context values for at least one context used to encode or decode a current slice or picture; Determining that a buffer for storing sets of temporal initialization points from two or more slices or pictures, each associated with the two or more slices, for context-based arithmetic decoding is full or all slices or pictures in the picture are associated and include one or more time initialization points; Determine a slice or picture associated with the slice or picture from the two or more slices or pictures based on at least one of a slice type, a time stamp value, or a quantization parameter (QP) value. Initialize the point collection at the first time; Remove the first set of initialization points associated with the slice or picture from the buffer; and In the buffer, a second set of temporal initialization points associated with the current slice or picture is stored, wherein the second set of temporal initialization points is based on the determined one or more context values. The method according to claim 1, wherein determining the first time initialization point set includes: determining from the two or more slices or pictures, a time stamp value or quantization corresponding to the two or more slices or pictures. The set of first time initialization points associated with the slice or picture having at least one of a minimum time identification value or a QP value in the parameter (QP) value. The method according to claim 2, wherein determining the slice or picture having at least one of the minimum time stamp value or the QP value from the two or more slices or pictures includes: selecting from the two or more than two slices or pictures Among the timestamp values of slices or pictures, determine the slice or picture with the smallest timestamp value. The method according to claim 2, wherein determining the slice or picture with at least one of the minimum time stamp value or the QP value from the two or more slices or pictures includes: from the two or more slices Or the QP value of the picture, determine the slice or picture with the smallest QP value. The method according to claim 1 also includes: determining that at least one of a time stamp value or QP value for the current slice or picture is different from a time stamp value or QP value for each of the two or more slices or pictures, wherein the time stamp value or QP value of the slice or picture associated with the first set of time initialization points is different from the time stamp value or QP value of the current slice or picture, and Wherein removing the first time initialization point set includes: based on determining that at least one of the time identification value or QP value for the current slice or picture is different from the slice for each of the two or more slices or pictures. Or the time stamp value and QP value of the picture to remove the first time initialization point set. According to the method of claim 1, wherein the current slice or picture is a first slice or picture, the method also includes: determining one or more context values for at least one context used to encode or decode a second slice or picture; From the two or more slices or pictures, determine at least one of a time identifier value, QP value or slice type that is the same as a time identifier value, QP value or slice type of the second slice or picture. a third slice or picture; removing a third set of temporal initialization points associated with the third slice or picture from the buffer; and A fourth time associated with the second slice or picture is stored in the buffer based on the determined one or more context values of the at least one context used to encode or decode the second slice or picture. Initialize the point collection. The method according to claim 1 also includes: Determine a set of time initialization points stored in the buffer; Initializing one or more context values for at least one context used to encode or decode a subsequent slice or picture based on the determined set of temporal initialization points; and Context-based arithmetic encoding or decoding is performed on the subsequent slice or picture. The method according to claim 7 also includes: Determine a time stamp value for the subsequent slice or picture; Determining that no one of the two or more slices or pictures having an associated set of temporal initialization points stored in the buffer has a time stamp value equal to the time stamp value for the subsequent slice or picture ;and from the two or more slices or pictures, determine a second slice or picture having a time stamp value that is closest to and smaller than the time stamp value of the subsequent slice or picture, Determining the time initialization point set includes: selecting the time initialization point set associated with the second slice or picture. The method according to claim 7 also includes: Determine a QP value for the subsequent slice or picture; determining that no one of the two or more slices or pictures having an associated set of temporal initialization points stored in the buffer has a QP value equal to the QP value for the subsequent slice or picture; and from the two or more slices or pictures, determine a second slice or picture having a QP value closest to the QP value of the subsequent slice or picture, Determining the temporal initialization point set includes: selecting the initialization point set associated with the second slice or picture. The method of claim 1, wherein storing the second set of initialization points includes: storing the second set of temporal initialization points after processing a last coding tree unit (CTU) of the current slice or picture. A device used to process video data. The device includes: a buffer configured to store a set of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding; and Processing circuitry coupled to the buffer, the processing circuitry being configured to: determining one or more context values for at least one context used to encode or decode a current slice or picture; It is decided that the buffer used to store sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding is full, where each set of temporal initialization points is associated with the two or more slices or pictures. All slices or pictures in the picture are associated and include one or more time initialization points; Determine a slice or picture associated with the slice or picture from the two or more slices or pictures based on at least one of a slice type, a time stamp value, or a quantization parameter (QP) value. Initialize the point collection at the first time; Remove the first set of initialization points associated with the slice or picture from the buffer; and In the buffer, a second set of temporal initialization points associated with the current slice or picture is stored, wherein the second set of temporal initialization points is based on the determined one or more context values. The device according to claim 11, wherein in order to determine the first time initialization point set, the processing circuit is configured to: from the two or more slices or pictures, determine the relationship between the two or more slices or pictures. The first set of temporal initialization points associated with the slice or picture having at least one of the time stamp value or the quantization parameter (QP) value of the picture and a minimum time stamp value or QP value. The device according to claim 12, wherein in order to determine the slice or picture having at least one of the minimum time stamp value or the QP value from the two or more slices or pictures, the processing circuit is configured to: from Among the time stamp values of the two or more slices or pictures, the slice or picture with the smallest time mark value is determined. The device according to claim 12, wherein in order to determine the slice or picture with at least one of the minimum time stamp value or the QP value from the two or more slices or pictures, the processing circuit is configured to: from the Among the QP values of two or more slices or pictures, the slice or picture with the smallest QP value is determined. The device according to claim 11, wherein the processing circuit is configured as: determining that at least one of a time stamp value or QP value for the current slice or picture is different from a time stamp value or QP value for each of the two or more slices or pictures, wherein the time stamp value or QP value of the slice or picture associated with the first set of time initialization points is different from the time stamp value or QP value of the current slice or picture, and In order to remove the first time initialization point set, the processing circuit is configured to: determine that at least one of the time identification value or the QP value for the current slice or picture is different from that for the two or more slices. Or the time stamp value and QP value of each slice or picture of the picture to remove the first time initialization point set. The apparatus according to claim 11, wherein the current slice or picture is a first slice or picture, and wherein the processing circuit is configured to: determining one or more context values for at least one context used to encode or decode a second slice or picture; From the two or more slices or pictures, determine at least one of a time identifier value, QP value or slice type that is the same as a time identifier value, QP value or slice type of the second slice or picture. a third slice or picture; removing a third set of temporal initialization points associated with the third slice or picture from the buffer; and A fourth time associated with the second slice or picture is stored in the buffer based on the determined one or more context values of the at least one context used to encode or decode the second slice or picture. Initialize the point collection. The device according to claim 11, wherein the processing circuit is configured as: Determine a set of time initialization points stored in the buffer; Initializing one or more context values for at least one context used to encode or decode a subsequent slice or picture based on the determined set of temporal initialization points; and Context-based arithmetic encoding or decoding is performed on the subsequent slice or picture. The device according to claim 17, wherein the processing circuit is configured to: Determine a time stamp value for the subsequent slice or picture; Determining that no one of the two or more slices or pictures having an associated set of temporal initialization points stored in the buffer has a time stamp value equal to the time stamp value for the subsequent slice or picture ;and from the two or more slices or pictures, determine a second slice or picture having a time stamp value that is closest to and smaller than the time stamp value of the subsequent slice or picture, In order to determine the temporal initialization point set, the processing circuit is configured to: select the temporal initialization point set associated with the second slice or picture. The device according to claim 17, wherein the processing circuit is configured to: Determine a QP value for the subsequent slice or picture; determining that no one of the two or more slices or pictures having associated sets of temporal initialization points stored in the buffer has a QP value equal to the QP value for the subsequent slice or picture; and from the two or more slices or pictures, determine a second slice or picture having a QP value closest to the QP value of the subsequent slice or picture, In order to determine the set of temporal initialization points, the processing circuit is configured to: select the set of initialization points associated with the second slice or picture. The apparatus according to claim 11, wherein in order to store the second set of initialization points, the processing circuit is configured to: store the second temporal initialization point after processing a last coding tree unit (CTU) of the current slice or picture. gather. The device according to claim 11, wherein the device includes one or more of a camera, a computer, a mobile device, a broadcast receiver device or a set-top box. A computer-readable storage medium on which instructions are stored that, when executed, cause one or more processors to: determining one or more context values for at least one context used to encode or decode a current slice or picture; Determining that a buffer for storing sets of temporal initialization points from two or more slices or pictures, each associated with the two or more slices, for context-based arithmetic decoding is full or all slices or pictures in the picture are associated and include one or more time initialization points; Determine a slice or picture associated with the slice or picture from the two or more slices or pictures based on at least one of a slice type, a time stamp value, or a quantization parameter (QP) value. Initialize the point collection at the first time; Remove the first set of initialization points associated with the slice or picture from the buffer; and In the buffer, a second set of temporal initialization points associated with the current slice or picture is stored, wherein the second set of temporal initialization points is based on the determined one or more context values. A device used to process video data. The device includes: means for determining one or more context values for at least one context used to encode or decode a current slice or picture; Determining a buffer full unit for storing sets of temporal initialization points from two or more slices or pictures for context-based arithmetic decoding, where each set of temporal initialization points is associated with the two or more All of the two or more slices or pictures are associated and include one or more temporal initialization points; For determining the association with the slice or picture from the two or more slices or pictures based on at least one of a slice type, a time stamp value or a quantization parameter (QP) value of the slice or picture A unit that initializes the point set at the first time; A unit for removing the first set of initialization points associated with the slice or picture from the buffer; and A unit for storing a second set of temporal initialization points associated with the current slice or picture in the buffer, wherein the second set of temporal initialization points is based on the determined one or more context values.