RU2789520C1 - Method for signalling a picture header in a stream of encoded video data - Google Patents

Abstract

FIELD: video data processing.

SUBSTANCE: invention relates to video encoding tools. A network abstraction layer (NAL) unit of the video data encoding layer (VCL) is obtained. It is determined whether the VCL NAL unit is the first VCL NAL unit of the picture unit (PU) containing the VCL NAL unit. The VCL NAL unit is herein defined as the first VCL NAL unit of said PU based on determining that the VCL NAL unit is the first VCL NAL unit following the NAL unit of the picture header. Based on determining that the VCL NAL unit is the first VCL NAL unit of said PU and determining that the least significant bit of the VCL NAL unit picture sequence counter differs from the least significant bit of the picture sequence counter for the previous image, it is determined whether the VCL NAL unit is the first VCL NAL unit of the access unit (AU) containing the PU. Based on determining that the VCL NAL unit is the first VCL NAL unit of said AU, the AU is decoded based on the VCL NAL unit.

EFFECT: increased efficiency of video encoding.

7 cl, 9 dwg

Description

Cross-reference to related application

[1] This application claims priority under U.S. Provisional Application No. 63/005,640, filed April 6, 2020, and U.S. Patent Application No. 17/096,168, filed November 12, 2020, the contents of which are incorporated herein in their entirety. through a link.

Technical field

[2] The invention relates to encoding and decoding video data, and more specifically, signaling a picture header in a coded video stream.

State of the art

[3] ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11) published the H.265/HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3) and 2016 (version 4). In 2015, the two standards organizations jointly formed the JVET (Joint Video Exploration Team) to explore the possibilities of developing the next video coding standard after HEVC. In October 2017, they launched a joint call for proposals for video data compression with capabilities beyond HEVC (CfP, Call for Proposals). By February 15, 2018, 22 standard dynamic range (SDR) CfP responses, 12 high dynamic range (HDR) CfP responses, and 12 360-degree video category CfP responses had been submitted. In April 2018, all CfP responses received were evaluated at the 122nd MPEG / 10th JVET meeting. As a result of this meeting, JVET formally launched the next generation video coding standardization process after HEVC. The new standard was named VVC (Versatile Video Coding), and JVET was renamed the Joint Video Expert Team.

The essence of the invention

[4] In an embodiment of the invention, a method is provided for decoding a bitstream of encoded video data using at least one processor, including obtaining a block of the network abstraction layer (NAL, network abstraction layer) of the video coding layer (VCL, video coding layer); determining whether the VCL NAL unit is the first VCL NAL unit of the picture unit (PU, picture unit) containing the VCL NAL unit; based on determining that the VCL NAL is the first VCL NAL of the PU, determining whether the VCL NAL is the first VCL NAL of the access unit containing the PU; and based on determining that the VCL NAL is the first VCL NAL of the AU, decoding the AU based on the VCL NAL.

[5] In an embodiment of the invention, there is provided an apparatus for decoding an encoded video bitstream, comprising at least one memory configured to store program code and at least one processor configured to read the program code and operate in accordance with the instructions of the program code, wherein the program code includes: a first acquisition code configured to allow at least one processor to obtain a network abstraction layer (NAL) block of a video coding layer (VCL); a first determination code configured to allow at least one processor to determine whether the VCL NAL unit is the first VCL NAL unit of a picture unit (PU) containing the VCL NAL unit; a second determination code configured to provide at least one processor, based on determining that the VCL NAL unit is the first VCL NAL unit of said PU, determining whether the VCL NAL unit is the first VCL NAL unit of the access unit (AU) containing the PU ; and a decoding code configured to cause the at least one processor, based on determining that the VCL NAL unit is the first VCL NAL unit of said AU, to decode the AU based on the VCL NAL unit.

[6] In one embodiment of the invention, a computer-readable medium is provided on which instructions are stored, including one or more instructions, which, when executed by one or more processors of a device for decoding a coded video bitstream, cause one or more processors to perform: (NAL) video coding level (VCL); determining whether the VCL NAL unit is the first VCL NAL unit of the picture unit (PU) containing the VCL NAL unit; based on determining that the VCL NAL is the first VCL NAL of the PU, determining whether the VCL NAL is the first VCL NAL of the access unit (AU) containing the PU; and based on determining that the VCL NAL is the first VCL NAL of the AU, decoding the AU based on the VCL NAL.

Brief description of the drawings

[7] Additional features, nature and various advantages of the present invention will be apparent from the following detailed description and the accompanying drawings.

[8] FIG. 1 is a schematic illustration of a simplified block diagram of a communication system in accordance with an embodiment of the invention.

[9] FIG. 2 is a schematic illustration of a simplified block diagram of a communication system in accordance with an embodiment of the invention.

[10] FIG. 3 is a schematic illustration of a simplified block diagram of a decoder according to an embodiment of the invention.

[11] FIG. 4 is a schematic illustration of a simplified block diagram of an encoder in accordance with an embodiment of the invention.

[12] FIG. 5 is a schematic illustration of syntax table examples according to an embodiment of the invention.

[13] FIG. 6A-6C are flowcharts of examples of methods for decoding an encoded video bitstream in accordance with an embodiment of the invention.

[14] FIG. 7 is a schematic illustration of a computer system in accordance with an embodiment of the invention.

Detailed description of the invention

[15] FIG. 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the present invention. The system (100) may include at least two terminals (110-120) interconnected via a network (150). For unidirectional data transmission, the first terminal (110) may encode video data at the local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive another terminal's encoded video data from the network (150), decode the encoded data, and display the recovered video data. One-way communication may be inherent in media service applications and the like.

[16] FIG. 1 illustrates a second pair of terminals (130, 140) supporting bidirectional transmission of coded video data, which may occur during video conferences, for example. For bi-directional data transmission, each terminal (130, 140) may encode video data captured at a local location for transmission to another terminal over a network (150). Each terminal (130, 140) can also receive the encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[17] In the example shown in FIG. 1, terminals (110-140) can be illustrated as servers, personal computers, and smartphones, but this does not limit the principles of the present invention. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or equipment used for video conferencing. Network (150) represents any number of networks that carry encoded video data between terminals (110-140), including, for example, wired and/or wireless networks. The communication network (150) allows data to be exchanged in a circuit-switched and/or packet-switched mode. Examples of networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may not be relevant to the present invention unless specifically noted below.

[18] FIG. 2 shows, as an example of the application of the present invention, the placement of a video encoder and a video decoder in a streaming environment. The present invention may be equally applicable to other video processing applications including, for example, video conferencing, digital television, storage of compressed video on digital media such as CD, DVD, memory card, and the like.

[19] The streaming system may include a capture subsystem (213), which may include a source (201) of video data, such as a digital camera, creating, for example, an uncompressed stream (202) of video samples. This stream (202) of samples, depicted with a thick line to emphasize the large amount of data compared to the bitstreams of encoded video data, can be processed by an encoder (203) connected to a camera (201). The encoder (203) may include hardware, software, or a combination thereof to provide or implement aspects of the present invention, as described in more detail below. The encoded video data bitstream (204), depicted as a thin line to emphasize the smaller amount of data compared to the sample stream, may be stored on the streaming server (205) for later use. One or more streaming clients (206, 208) can access the streaming server (205) to extract copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210) that decodes an incoming copy of the encoded video bitstream (207) and produces an outgoing stream (211) of video samples that can be displayed on a display (212) or other visualization device (not shown) . In some streaming systems, bitstreams (204, 207, 209) of video data may be encoded according to certain video coding/compression standards. Examples of these standards include ITU-T Rec. H.265. A video coding standard is under development, informally known as Universal Video Coding or VVC (Versatile Video Coding). The present invention can be used in the context of VVC.

[20] FIG. 3 is a functional diagram of a video decoder (210) according to an embodiment of the present invention.

[21] The receiver (310) may receive one or more encoded video sequences for decoding by video decoder (210), in the same or another embodiment of the invention, one encoded video sequence at a time, with the decoding of each encoded video sequence independent of other encoded video sequences. The encoded video sequence may be received from a channel (312), which may be a hardware/software link to a storage device where the encoded video data is stored. The receiver (310) may receive encoded video data with other data, such as encoded audio data and/or auxiliary data streams, which may be redirected to their respective utilizing entities (not shown). The receiver (310) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (315) can be connected between the receiver (310) and the entropy decoder/analyzer (320) (hereinafter "analyzer"). When the receiver (310) receives data from a storage/redirector with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (315) may not be needed or may be small. For use in best effort packet networks such as the Internet, a buffer (315) may be required, which may be relatively large and may preferably have an adaptive size.

[22] The video decoder (210) may include an analyzer (320) for reconstructing the symbols (321) from the entropy-coded video sequence. The categories of these symbols include information used to control the operation of the decoder (210) and possibly information to control a visualization device such as a display (212) that is not an integral part of the decoder but may be connected to it, as shown in FIG. 3. The control information for the rendering device(s) may be Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The analyzer (320) may analyze/entropy decode the received encoded video sequence. The encoding of the video sequence may be in accordance with a video coding technology or standard and may follow principles known to those skilled in the art, including variable run length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The analyzer (320) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOP, Groups of Pictures), pictures, sub-images, tiles, slices, modules, macroblocks, coding tree units (CTU, Coding Tree Unit), coding units (CU, Coding Unit), blocks, transformation units ( TU, Transform Unit), prediction units (PU, Prediction Unit), etc. A tile can point to a rectangular area of a CU/CTU in a specific column and row of a tile in an image. A module can point to a rectangular area of CU/CTU rows within a specific tile. A slice may point to one or more image units that are contained in a NAL unit. A sub-image may point to a rectangular area of one or more slices in an image. The entropy decoder/analyzer can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[23] The analyzer (320) may perform an entropy decoding/analysis operation of the video sequence obtained from the buffer (315) to create symbols (321).

[24] Several different modules may be used for symbol reconstruction (321) depending on the type of encoded video image or parts thereof (eg, intra- and inter-coded images, intra- and inter-coded blocks) and other factors. Which modules are used and how can be determined by the subgroup management information obtained from the encoded video sequence by the analyzer (320). The flow of such subgroup control information between the analyzer (320) and the modules is not shown hereinafter for simplicity.

[25] In addition to the previously mentioned functional blocks, video decoder 210 can be fundamentally divided into several functional modules, as described below. In practical implementation under commercial constraints, many of these modules interact closely with each other and can, at least partially, be built into one another. However, for the purpose of describing the present invention, a fundamental division into the following functional modules is appropriate.

[26] The first module is the scaling/inverse transform module (351). The scaling/inverse transform module (351) receives the quantized transform coefficient as well as control information including the transform used, block size, quantization factor, quantization scaling matrices, etc. as character(s) (321) from the analyzer (320). It can output blocks containing sample values that can be entered into the aggregator (355).

[27] In some cases, the output samples of the scaling / inverse transform module (351) may refer to an intra-coded block, that is, a block that does not use predicted information from previously reconstructed images, but can use predicted information from previously reconstructed parts of the current image . Such predicted information may be provided by the intra image prediction module (352). In some cases, the intra image prediction module (352) generates a block of the same size and shape as the block to be reconstructed using the already reconstructed environment information extracted from the current (partially reconstructed) image (358). The aggregator (355) in some cases adds, for each sample, the prediction information generated by the intra prediction module (352) to the output sample information provided by the scaling/inverse transform module (351).

[28] In other cases, the output samples of the scaling/inverse transform module (351) may refer to an intra-coded block, possibly with motion compensation. In such a case, the motion compensation prediction module (353) may access the reference picture memory (357) to retrieve samples used for prediction. After motion compensation has been applied to the extracted samples according to the symbols (321) associated with the block, these samples can be added by an aggregator (355) to the output of the scaler/demapper (in this case referred to as residual samples or residual signal) to generate output information. counts. Reference picture memory addresses from where the motion-compensated prediction module retrieves prediction samples may be driven by motion vectors available to the motion-compensated prediction module in the form of symbols (321), which may have, for example, X, Y, and reference picture components. . Motion compensation may also include interpolation of sample values retrieved from reference picture memory when accurate subsample motion vectors, motion vector prediction mechanisms, and so on are used.

[29] Various loop filtering methods can be applied to the aggregator (355) output samples in the loop filter module (356). The video data compression technologies may include loop filter technologies controlled by parameters included in the encoded video bitstream and available to the loop filter module (356) as symbols (321) from the analyzer (320), but may also be responsive to metainformation obtained during decoding previous (in decoding order) parts of the encoded image or encoded video sequence, as well as responding to previously reconstructed and contour filtered sample values.

[30] The loop filter module (356) can output a stream of samples that can be fed to the renderer (212) as well as stored in the reference picture memory 557 for use in future inter-image prediction.

[31] Some encoded pictures, when fully reconstructed, can be used as reference pictures for future prediction. For example, when the encoded picture is fully reconstructed and identified as a reference picture (e.g., by analyzer (320)), the current reference picture (358) may become part of the reference picture buffer memory (357), and fresh memory for the current picture may be re-allocated prior to reconstruction. next encoded picture.

[32] Video decoder 210 may perform decoding operations according to a predetermined video data compression technology documented in a standard such as ITU-T Rec. H.265. The encoded video sequence may follow the syntax given by the video compression technology or standard used, in the sense that the encoded video sequence may follow the syntax of the video compression technology or standard specified in the video compression technology document or standard, and in particular in the profile document. . It may also be necessary for consistency that the complexity of the encoded video sequence remains within the limits set by the level of technology or video compression standard. In some cases, the levels limit the maximum image size, the maximum frame rate, the maximum sampling rate for reconstruction (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the levels, in some cases, may be further limited by the specifications of the Hypothetical Reference Decoder (HRD) and the metadata for managing the HRD buffer signaled in the encoded video sequence.

[33] According to an embodiment of the invention, the receiver (310) may receive additional (redundant) data with encoded video data. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (210) to properly decode the data and/or more accurately reconstruct the original video data. The side data may be, for example, temporal enhancement levels, spatial enhancement levels or signal noise ratio (SNR) enhancement levels, redundant slices, redundant images, forward error correction codes, and so on.

[34] FIG. 4 is a functional diagram of a video encoder (203) according to an embodiment of the present invention.

[35] The encoder (203) may receive video data samples from a video data source (201) (which is not part of the encoder) that may capture the video image(s) to be encoded by the video encoder (203).

[36] The video data source (201) may provide the source video sequence to be encoded by the video encoder (203) in the form of a digital video data sample stream of any suitable bit depth (eg: 8 bits, 10 bits, 12 bits, ...), any color space (eg , BT.601 Y CrCB, RGB,…) and any suitable sampling structure (eg Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the source (201) of video data may be a storage device where previously prepared video is stored. In a videoconferencing system, the video data source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that give a sense of movement when viewed in sequence. The images themselves may be organized as a spatial array of pixels, where each pixel may contain one or more samples depending on the sampling structure used, color space, and so on. It is not difficult for a person skilled in the art to understand the relationship between pixels and samples. The following description is devoted to readings.

[37] According to an embodiment of the invention, the encoder (203) may encode and compress images of the source video sequence into the encoded video sequence (443) in real time or subject to any other time constraints imposed by the application. Setting the proper coding rate is one of the functions of the controller (450). The controller controls other functional modules as described below and is functionally connected to them. The connection is not shown for simplicity. The parameters set by the controller may include parameters related to frequency adjustment (picture skip, quantizer, lambda value used in rate-distortion optimization methods, etc.), image size, group of images (GOP, group of pictures), maximum motion vector search area, etc. One skilled in the art will appreciate that the controller (450) may have other suitable functions related to the video encoder (203) optimized for a particular system design.

[38] Some video encoders may operate in a "coding loop" as understood by one skilled in the art. As a very simplified description, for example, an encoding loop may include an encoder (430) encoding part (herein referred to as a "source encoder") (responsible for creating symbols, for example, a stream of symbols, based on the input image to be encoded and the reference image(s). (s)) and a (local) decoder (433) built into the encoder (203) that reconstructs the symbols to produce sample data as a (remote) decoder would (because any symbol compression into the encoded video bitstream is lossless in video compression technologies discussed in this description). The reconstructed stream of samples enters the memory (434) reference images. Since decoding a stream of symbols produces bit-accurate results that are independent of the position (local or remote) of the decoder, the contents of the reference picture buffer will also be the same bit-accuracy for the local encoder and the remote encoder. In other words, the prediction part of the encoder "sees" as reference picture samples exactly the same sample values as the decoder would "see" if using prediction during decoding. This fundamental principle of reference picture synchronism (and eventually drift if synchronism cannot be maintained due to, for example, channel errors) is well known to those skilled in the art.

[39] The "local" decoder (433) may operate in the same manner as the "remote" decoder (210) detailed above with reference to FIG. However, again, as shown in FIG. 4, since the symbols are available and encoding/decoding of the symbols into the encoded video sequence by the entropy encoder (445) and parser (320) can be lossless, the entropy decoding portions of the decoder (210) including the channel (312) , receiver (310), buffer (315), and parser (320) may not be fully implemented in the local decoder (433).

[40] In doing so, it can be concluded that any decoding technology present in a decoder, with the exception of analysis/entropy decoding, must also necessarily be present in essentially identical functional form in the corresponding encoder. For this reason, the present invention is focused on the operation of the decoder. The description of the encoding technologies can be abbreviated as they are the inverse of the decoding technologies described in detail. Only in some areas a more detailed description is required, which is given below.

[41] In operation, the source encoder (430) may perform motion-compensated predictive coding in which an input frame is predictively encoded based on one or more previously encoded frames from the video sequence, referred to as "reference frames". Thus, the coding engine (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame(s) that can be selected as the prediction reference(s) to the input frame.

[42] The local video decoder (433) may decode the encoded video data of the frames, which may be indicated as reference frames, based on the symbols generated by the source encoder (430). The operations of the encoding engine (432) may be predominantly lossy methods. When the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (433) duplicates the decoding processes that may be performed by the video decoder on the reference frames and may store the reconstructed reference frames in the reference picture cache (434). Thus, the encoder (203) can locally store copies of the reconstructed key frames having the same content as the reconstructed key frames that would be received by the remote video decoder (in the absence of transmission errors).

[43] The predictor (435) may perform prediction searches for the encoding engine (432). Thus, for a new frame to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as pixel reference block candidates) or specific metadata such as reference picture motion vectors, block shapes, etc. .d., which can serve as a proper reference for predicting new images. The predictor (435) may operate on a "blocks of samples×blocks of pixels" basis to find proper references for prediction. In some cases, according to the search results obtained by the predictor (435), the input picture may have prediction references extracted from the plurality of reference pictures stored in the reference picture memory (434).

[44] The controller (450) may control the encoding operations of the video encoder (430), including, for example, setting parameters as well as subgroup parameters used to encode video data.

[45] The output signal of all the above functional modules can be entropy encoded in an entropy encoder (445). The entropy encoder translates symbols generated by various functional units into an encoded video sequence by lossless symbol compression according to, for example, techniques known to a person skilled in the art, for example, according to Huffman coding, variable run length coding, arithmetic coding, etc.

[46] The transmitter (440) may buffer the encoded video sequence(s) created by the entropy encoder (445) in preparation for transmission over a communication channel (460), which may be a hardware/software memory link. where the encoded video data is stored. The transmitter (440) may combine the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and/or auxiliary data streams (sources not shown).

[47] The controller (450) may control the operation of the encoder (203). During encoding, the controller (450) may assign to each encoded picture one or another encoded picture type, which may determine the encoding methods applicable to the corresponding picture. For example, the following types of frames can often be assigned to images.

[48] An intra-coded picture (I-picture) that can be encoded and decoded without using any other frame in the sequence as a source for prediction. Some video codecs allow different types of intra-coded images, including, for example, images in the Independent Decoder Refresh (IDR) format. One skilled in the art will be aware of the I-image varieties and their respective uses and features.

[49] A predictive picture (P-picture) that can be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index to predict sample values of each block.

[50] A bidirectional predictive picture (B-picture) that can be encoded and decoded using intra prediction or inter prediction using no more than two motion vectors and reference indices to predict sample values of each block. Likewise, multi-prediction pictures may use more than two reference pictures and associated metadata to reconstruct a single block.

[51] Source images typically allow for spatial division into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coding on a block-by-block basis. The blocks may be predictively encoded with reference to other (previously coded) blocks defined by the coding assignment applied to the respective images of those blocks. For example, blocks of I-pictures may be encoded without prediction or predictively with reference to previously encoded blocks of the same picture (with spatial prediction or intra prediction). The pixel blocks of P-pictures may be encoded without prediction, either by spatial prediction or temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be encoded without prediction, by spatial prediction or temporal prediction with reference to one or two previously encoded reference pictures.

[52] The video encoder (203) may perform encoding operations according to a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data may follow the syntax given by the technology used or the video coding standard used.

[53] According to an embodiment of the invention, the transmitter (440) may transmit additional data with encoded video data. The source video encoder (430) may include such data as part of the encoded video sequence. The additional data may comprise temporal/spatial/SNR enhancement levels, other forms of redundant data such as redundant images and slices, additional enhancement information (SEI) messages, video usability information (VUI) parameter set fragments, and so on.

[54] Embodiments of the invention may relate to modifications to the syntax and semantics related to image headers. At least one embodiment of the invention may relate to signaling all_pic_coding_info_present_in_ph_flag in a picture parameter set (PPS) as a gateway flag to store some bits to indicate whether picture level coding tool information is present in a picture header (PH). ) or slice header (SH, slice header). At least one embodiment of the invention may relate to correcting the identification of the first VCL NAL of the PU or AU. At least one embodiment of the invention may relate to modifications to the semantics of gdr_or_irap_pic_flag to deal with the case where the value of mixed_nalu_types_flag is 1.

[55] In embodiments of the invention, a Picture Parameter Set (PPS) may refer to a syntax structure containing syntax elements that do not apply or apply to complete encoded pictures as defined by the syntax element found in each slice header.

[56] In embodiments of the invention, a picture header (PH) may refer to a syntax structure containing syntax elements that apply to all slices of an encoded picture.

[57] In embodiments, a slice header (SH) may refer to a portion of an encoded slice containing data elements relating to all tiles or lines of a coding tree unit (CTU) in a tile represented in the slice.

[58] Embodiments of the invention may relate to a video coding layer (VCL, Video Coding Layer).

[59] In embodiments of the invention, a network abstraction layer (NAL) block may refer to a syntax structure containing an indication of the type of data that follows, and bytes containing this data in the form of a payload of a raw byte sequence payload (RBSP, raw byte sequence payload) , interspersed with emulation prevention bytes if necessary.

[60] In embodiments, a VCL NAL unit may refer to a collection term for coded slice NAL units and a subset of NAL units that have reserved nal_unit_type values and are classified herein as VCL NAL units.

[61] In embodiments of the invention, a picture unit (PU) may refer to a set of NAL units that are related to each other according to a given classification rule, are sequential in decoding order, and contain exactly one encoded picture.

[62] In embodiments of the invention, an access unit (AU) may refer to a set of PUs that belong to different layers and contain encoded pictures associated with the same output time from the decoded picture buffer (DPB).

[63] Embodiments of the invention may relate to sample adaptive offset (SAO).

[64] In embodiments of the invention, an adaptive loop filter (ALF) may refer to a filtering process that is applied as part of the decoding process and is controlled by parameters transmitted in an adaptation parameter set (APS).

[65] Embodiments of the invention may relate to a quantization parameter (QP, quantization parameter).

[66] Embodiments of the invention may relate to an internal random access point (IRAP, intra random access point).

[67] Embodiments of the invention may relate to gradual decoding refresh (GDR).

[68] In embodiments of the invention, a GDR image may refer to an image for which each VCL NAL unit has a nal_unit_type value equal to GDR NUT.

[69] According to the latest version of the VVC draft (JVET-Q2001-vE), six flags can be used to indicate whether image level encoding information is present in the image header or slice header in the PPS syntax. For example, rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, etc. In most cases, these values can have the same value - 0 or 1. It is unlikely that each xxx_info_in_p>h flag will have a different value.

[70] Therefore, in embodiments of the invention, the gateway flag all_pic__coding_info_present_in_ph_flag may indicate the presence of these flags in the PPS to store bits in the PPS. If the value of all_pic_coding_info_present_in_ph_flag is 1, these xxx_info_in_ph flags are not signaled, and it can be inferred that the values of these flags are 1 because the signaling of picture layer coding information in the picture header may occur more frequently than the signaling of slice header information for slice level control. . An example of a syntax table corresponding to embodiments of the invention is shown in FIG. 5.

[71] В вариантах осуществления изобретения значение all_pic_coding_info_present_in_ph_flag, равное 1, может указывать на то, что rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, alf_info_in_ph_flag, wp_info_in_ph_flag и qp_ delta_info_in_ph_flag не присутствуют в PPS, значение all_pic_ coding_info_present_in_ph_flag, равное 0, может указывать на то, что rpl_info_in_ph_flag , dbf__info_in_ph_flag, sao_info_in_ph_flag, alf_info_in_ph_flag, wp_info_in_ph_flag and qp_delta_info_in_ph_flag are present in PPS.

[72] In embodiments, an rpl_info_in_ph_flag value of 1 may indicate that reference picture list information is present in the PH syntax structure and not present in PPS-related slice headers that do not contain the PH syntax structure. An rpl_info_in_ph_flag value of 0 may indicate that the reference picture list information is not present in the PH syntax structure and may be present in PPS-related slice headers that do not contain the PH syntax structure. When the value of rpl_info_in_ph_flag is missing, it can be inferred that it is equal to 1.

[73] In embodiments, a dbf_info_in_ph_flag value of 1 may indicate that deblocking filter information is present in the PH syntax and not present in PPS-related slice headers that do not contain the PH syntax. A dbf_info_in_ph_flag value of 0 may indicate that deblocking filter information is not present in the PH syntax and may be present in PPS-related slice headers that do not contain the PH syntax. When the value of dbf_info_in_ph_flag is absent, it can be inferred that it is 0. When the value of dbf_info_in_ph_flag is absent, it can be inferred that it is equal to 1.

[74] A sao_info_in_ph_flag value of 1 may indicate that SAO filter information is present in the PH syntax structure and not present in PPS-related slice headers that do not contain the PH syntax structure. A sao_info_in_ph_flag value of 0 may indicate that the SAO filter information is not present in the PH syntax and may be present in PPS-related slice headers that do not contain the PH syntax. When the value of sao_info_in_ph_flag is missing, it can be inferred that it is equal to 1.

[75] An alf_info_in_ph_flag value of 1 may indicate that ALF information is present in the PH syntax structure and not present in PPS-related slice headers that do not contain the PH syntax structure. An alf_info_in_ph_flag value of 0 may indicate that ALF information is not present in the PH syntax and may be present in PPS-related slice headers that do not contain the PH syntax. When alf_info_in_ph_flag is missing, it can be inferred that it is 1.

[76] A wp_info_in_ph_flag value of 1 may indicate that weighted prediction information may be present in the PH syntax and not present in PPS-related slice headers that do not contain the PH syntax. A wp_info_in_ph_flag value of 0 may indicate that the weighted prediction information is not present in the PH syntax and may be present in PPS-related slice headers that do not contain the PH syntax. When the value of wp_info_in_ph_flag is absent, it can be inferred that it is 0. When the value of wp_info_in_ph_flag is absent, it can be inferred that it is equal to 1.

[77] A qp_delta_info_in_ph_flag value of 1 may indicate that delta QP information is present in the PH syntax and is not present in PPS slice headers that do not contain the PH syntax. A qp_delta_info_in_ph_flag value of 0 may indicate that delta QP information is not present in the PH syntax and may be present in PPS-related slice headers that do not contain the PH syntax. When the qp_delta_info_in_ph_flag value is missing, it can be inferred that it is equal to 1.

[78] It is not clear from the latest draft of the VVC specification how to identify the first VCL NAL of a PU or AU. Embodiments of the invention may relate to the following modification of the NAL unit order description.

[79] In embodiments, the VCL NAL is the first VCL NAL of the AU (and therefore the PU containing the VCL NAL is the first PU of said AU) when the VCL NAL is the first VCL NAL that follows the VCL NAL. PH header NAL or has picture_header_in_slice_header_flag equal to 1 and one or more of the following conditions are met:

- The nuh_layer_id value of the VCL NAL unit is less than the nuh_layer_id of the previous picture in decoding order.

- The ph_pic_order_cnt_lsb value of the VCL NAL block is different from the ph_pic_order_cnt_lsb of the previous picture in decoding order.

- The PicOrderCntVal value obtained for the VCL NAL block is different from the PicOrderCntVal value of the previous picture in the decoding order.

[80] In embodiments, the gdr_or_irap_pic_flag flag in the picture header indicates whether the current picture is an IRAP or GDR picture. When the value of gdr_or_irap_pic_flag is 1, the no_output_of_prior_pics_flag flag may also be present in the image header. When concatenating sub-picture bitstreams, the no_output_of_prior_pics_flag value of the IRAP sub-picture needs to be stored to retrieve the sub-picture. To solve this problem, embodiments of the invention may refer to the following modification of the semantics of gdr_or_irap_pic_flag:

[81] In embodiments, a gdr_or_irap_pic_flag value of 1 may indicate that the current picture is a GDR or IRAP picture, or a picture with a VCL NAL unit equal to IDR_W_RADL, IDR_N_LP, or CRA_NUT and a mixed_nalu_types_in_pic_flag value of 1. A gdr_or_irap_pic_flag value of 0 may indicate that the current image may or may not be a GDR or IRAP image.

[82] In embodiments, a gdr_or_irap_pic_flag value of 1 may indicate that the current image is a GDR or IRAP image, or an image containing an IRAP sub-image with a mixed_nalu_types_in_pic_flag value of 1. A gdr_or_irap_pic_flag value of 0 may indicate that that the current image may or may not be a GDR or IRAP image.

[83] In embodiments of the invention, a gdr_or_irap_pic_flag value of 1 may indicate that the current picture is a GDR or IRAP picture. A gdr_or_irap_pic_flag value of 0 may indicate that the current picture may or may not be a GDR or IRAP picture.

[84] A bitstream matching requirement may be used, which is that the value of gdr_or_irap_pic_flag is 0 when the value of mix_nalu_types_in_pic_flag is 1.

[85] FIG. 6A-6C are flowcharts of examples of methods 600A, 600B, and 600C for decoding an encoded video bitstream. In some implementations, one or more steps of the methods shown in FIG. 6A-6C may be performed by decoder 210. In some implementations, one or more steps of the methods shown in FIG. 6A-6C may be performed by another device or group of devices separate from or including decoder 210, such as encoder 203.

[86] In embodiments of the invention, one or more of the steps shown in FIG. 6A may correspond to one or more of the steps shown in FIG. 6B and 6C, or may be performed in conjunction with them.

[87] As shown in FIG. 6A, method 600A may include obtaining a network abstraction layer (NAL) block of a video coding layer (VCL) (block 611).

[88] As also shown in FIG. 6A, method 600A may include determining that the VCL NAL unit is the first VCL NAL unit of a picture unit (PU) containing the VCL NAL unit (block 612).

[89] As also shown in FIG. 6A, method 600A may include, based on determining that the VCL NAL is the first VCL NAL of the PU, determining that the VCL NAL is the first VCL NAL of the AU containing the PU (step 613).

[90] As also shown in FIG. 6A, method 600A may include, based on determining that the VCL NAL unit is the first VCL NAL unit of the AU, decoding the AU based on the VCL NAL unit (block 614).

[91] In embodiments of the invention, one or more of the steps shown in FIG. 6B may correspond to one or more of the steps shown in FIG. 6A and 6B, or may be performed in conjunction with them.

[92] As shown in FIG. 6B, method 600B may include obtaining a VCL NAL unit (block 621).

[93] As also shown in FIG. 6B, method 600B may include determining if the VCL NAL unit is the first VCL NAL unit following the picture header NAL unit (block 622).

[94] As also shown in FIG. 6B, method 600B may include, based on determining that the VCL NAL unit is the first VCL NAL unit following the picture header NAL unit ("Yes" in step 622), proceeding to step 623.

[95] As also shown in FIG. 6B, method 600B may include, based on a determination that a VCL NAL unit is not the first VCL NAL unit following a picture header NAL unit ("No" in step 623), proceeding to step 624. In embodiments of the invention, method 600 B may instead proceed to block 625.

[96] As also shown in FIG. 6B, method 600B may include determining if a flag is set in the VCL NAL unit to indicate that a picture header is included in a slice header included in the VCL NAL unit (block 624). In embodiments of the invention, the flag may correspond to picture_header_in_slice_header_flag.

[97] As also shown in FIG. 6B, method 600B may include, based on determining that a flag is set in the VCL NAL unit to indicate that a picture header is included in a slice header included in the VCL NAL unit ("Yes" in step 624), proceeding to step 623.

[98] As also shown in FIG. 6B, method 600B may include, based on determining that a flag in the VCL NAL unit is not set to indicate that a picture header is included in a slice header included in the VCL NAL unit ("No" at 624), proceeding to step 625.

[99] As also shown in FIG. 6B, method 600B may include determining that the VCL NAL unit is the first VCL NAL unit of the PU containing the VCL NAL unit (block 623).

[100] As also shown in FIG. 6B, method 600B may include determining that the VCL NAL unit is not the first VCL NAL unit of the PU containing the VCL NAL unit (block 625).

[101] In embodiments of the invention, one or more of the steps shown in FIG. 6C may correspond to one or more of the steps shown in FIG. 6A and 6C, or may be performed in conjunction with them.

[102] As shown in FIG. 6C, method 600C may include determining that the VCL NAL is the first VCL NAL of the PU (block 631).

[103] As also shown in FIG. 6C, method 600C may include determining if the VCL NAL unit's layer identifier is less than the previous picture's layer identifier (block 632).

[104] As also shown in FIG. 6C, method 600C may include, based on determining that the VCL NAL unit level identifier is less than the previous picture level identifier ("Yes" in step 632), proceeding to step 633.

[105] As also shown in FIG. 6C, method 600C may include, based on determining that the VCL NAL unit level identifier is not less than the previous picture level identifier ("No" in step 633), proceeding to step 634. In embodiments of the invention, method 600C may instead proceed to step 635.

[106] As also shown in FIG. 6C, method 600C may include determining if the VCL NAL block picture sequence counter is different from the picture sequence counter for the previous picture (block 634). In embodiments of the invention, this can be determined based on the least significant bits (LSB, least significant bit) of the image sequence counters.

[107] As also shown in FIG. 6C, method 600C may include, based on determining that the VCL NAL unit's picture sequence count is different from the picture sequence count for the previous picture ("Yes" in step 634), proceeding to step 633.

[108] As also shown in FIG. 6C, method 600C may include, based on determining that the VCL NAL unit's picture sequence counter is the same as the previous picture's picture sequence counter ("No" at 634), proceeding to 635.

[109] As also shown in FIG. 6C, method 600C may include determining that the VCL NAL unit is the first VCL NAL unit of the AU containing the VCL NAL unit (block 633).

[110] As also shown in FIG. 6C, method 600C may include determining that the VCL NAL unit is not the first VCL NAL unit of the AU containing the VCL NAL unit (block 635).

[111] In embodiments of the invention, based on a flag indicating that all of the image encoding information is present in the image header, a plurality of flags corresponding to the image encoding information may not be signaled. In embodiments of the invention, the flag may correspond to all_pic_coding_info_present_in_ph_flag.

[112] Although FIG. 6A-6C show examples of steps in methods 600A-600C, in some implementations, methods 600A-600C may include additional steps, fewer steps, different steps, or steps arranged differently than shown in FIG. 6A-6C. Additionally or alternatively, two or more steps of methods 600A-600C may be performed in parallel.

[113] In addition, the proposed methods can be implemented using processing circuits (eg, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program that is stored on a computer-readable medium to implement one or more of the proposed methods.

[114] The above methods may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 7 shows a computer system 700 suitable for practicing some embodiments of the invention.

[115] Computer software may be coded using any suitable machine code or computer language that may be assembled, compiled, linked, or similarly processed to produce code containing instructions that can be executed directly or through interpretation, microcode execution, etc. etc., computer central processing units (CPU, central processing units), graphic processors (GPU, Graphics Processing Units), etc.

[116] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[117] The components of computer system 700 shown in FIG. 7 are illustrative in nature and are not intended to impose any limitation on the scope of application or functionality of computer software implementing embodiments of the present invention. The component configuration should also not be interpreted as having any dependency or requirement in connection with any component or combination of components shown in the exemplary computer system 700.

[118] Computer system 700 may include some human-machine interface input devices. Such an input device may be responsible for input by one or more users through, for example, tactile input (e.g., keystrokes, waving, glove movements), audio input (e.g., voice, claps), visual input (e.g., gestures), olfactory input (not shown). Interface devices may also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, external sound), images (e.g. scanned images, photographic images received from a still camera), video (eg 2D video, 3D video including stereoscopic video).

[119] The input human-machine interface of the device may include one or more of the following (shown one at a time): keyboard 701, mouse 702, touch pad 703, touch screen 710 and associated graphics adapter 750, data glove, joystick 705, microphone 706, scanner 707 and camera 708.

[120] Computer system 700 may also include some human interface output devices. Such output devices may stimulate the senses of one or more users through, for example, tactile output, sound, light, and smell/taste. Such output devices may include tactile output devices (for example, haptic feedback via the 710 touch screen, information glove, or 705 joystick, as well as haptic feedback devices that do not serve as input devices), audio output devices (for example: speakers 709, headphones ( not shown)), visual output devices (e.g., 710 screens, including cathode ray tube (CRT) screens, liquid-crystal display (LCD), plasma screens, organic light-emitting diode screens ( OLED, organic light-emitting diode), each with or without touch screen input capabilities, each with or without tactile feedback capabilities, some of them are capable of two-dimensional visual output or more than three-dimensional output through, for example, stereographic output, glasses virtual reality (not shown), holographic displays, smoke tanks (not shown) and printers (not shown). cauldrons).

[121] Computer system 700 may also include human accessible storage devices and associated media, such as optical media including CD/DVD ROM/RW 720 with CD/DVD media 721, etc., flash memory cards 722, removable hard drive or solid state drive 723, conventional magnetic media such as tape and magnetic disk (not shown), dedicated ROM/ASIC/PLD based devices such as security dongles (not shown), and the like.

[122] Those skilled in the art should also understand that the term "computer-readable media" as used in connection with the present invention does not encompass transmission media, carrier waves, or other time signals.

[123] Computer system 700 may also include interface(s) to one or more communication networks (1155). Networks can be, for example, wireless, wired, optical. Networks can also be local, global, urban, transport and industrial, real-time, latency, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks, including global systems for mobile communications (GSM), 3G, 4G, and 5G generation networks, Long Term Evolution (LTE, Long Term Evolution) -Term Evolution), etc., global digital networks of wired or wireless television, including cable television, satellite television and terrestrial television broadcasting, transport and industrial networks, including CANBus, etc. Some networks typically require external network interface adapters (1154) that are connected to some general purpose data ports or peripheral buses (1149) (eg, the USB universal serial bus ports of the computer system 700); others are typically built into the core of computer system 700 by connecting to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). For example, network 755 may be connected to peripheral bus 749 using network interface 754. Using any of these networks, computer system 700 may communicate with other entities. Such communication may be unidirectional with receive-only capability (eg broadcast), unidirectional with transmit-only capability (eg CANbus to some CANbus devices), or bidirectional, eg to other computer systems using a local area network or a wide area digital network. Several protocols and protocol stacks may be used on each of these networks and the network interfaces described above (1154).

[124] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces can be connected to the core 740 of the computer system 700.

[125] The core 740 may include one or more central processing units (CPUs) 741, graphics processing units (GPUs) 742, field programmable gate arrays (FPGAs) 743, hardware accelerators 744 for some tasks, etc. These devices, together with Read Only Memory (ROM) 745, Random Access Memory (RAM) 746, high capacity internal storage such as internal hard drives that are not user accessible, solid state drives (SSDs, solid-state drives), etc.747 , may be connected via the system bus 748. In some computer systems, the system bus 748 may be available in the form of one or more physical connectors to allow for expansions with additional CPUs, GPUs, etc. Peripheral devices may be connected either directly to the system bus 748 core, or via a peripheral bus 749. Peripheral bus architectures include a peripheral component interconnect (PCI) bus, USB, and the like.

[126] The CPU 741, GPU 742, FPGA 743, and accelerators 744 may execute some instructions that together may constitute the aforementioned computer code. This computer code may be stored in ROM 745 or RAM 746. Temporary data may also be stored in RAM 746, while permanent data may be stored in, for example, internal high capacity data storage 747. Fast storage and retrieval from any storage device can be achieved through the use of a cache memory that can be tightly coupled to one or more CPU 741, GPU 742, mass storage 747, ROM 745, RAM 746, and the like.

[127] Computer-readable media may store computer code for performing various computer-executable operations. The media and computer code may be specially created for the purposes of the present invention or may be well known and available to those skilled in the art of computer software.

[128] By way of example, and not limitation of the invention, a computer system having architecture 700, and in particular core 740, can provide functionality by being executed by processor(s) (including(and) including CPUs, GPUs, FPGAs, accelerators etc.) software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with the user-accessible mass storage described above as well as some persistent storage of the core 740, such as internal mass storage 747 or ROM 745. Software implementing various embodiments of the present invention may stored in such devices and executed by the core 740. The computer-readable medium may include one or more storage devices or chips, as appropriate. The software may direct the core 740, and in particular its processors (including CPUs, GPUs, FPGAs, and the like), to perform specific methods or specific portions of the specific methods described herein, including setting data structures stored in RAM 746 and modifying such data structures according to the methods specified by the software. Additionally or alternatively, the computer system may provide functionality through logic hardwired or otherwise embodied in circuitry (e.g., accelerator 744) that may operate in place of or in conjunction with software to perform specific methods or specific portions of the specific methods described herein. A reference to software may cover logic, and vice versa when appropriate. Reference to computer-readable media may cover a circuit (eg, an integrated circuit (IC, integrated circuit)), where software is stored for execution, a circuit embodying logic for execution, or both circuits, when appropriate. The present invention encompasses any suitable combination of hardware and software.

[129] Although some embodiments of the invention have been described here, variations, permutations, and various equivalent substitutions are possible within the scope of the invention. Thus, those skilled in the art can provide numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the invention and thus fall within its spirit and scope.

Claims (11)

1. A method for decoding a bitstream of encoded video data using at least one processor, including: obtaining a network abstraction layer (NAL) block of a video coding layer (VCL); determining whether the VCL NAL unit is the first VCL NAL unit of the picture unit (PU) containing the VCL NAL unit; based on determining that the VCL NAL is the first VCL NAL of the PU and determining that the least significant bit of the picture sequence counter of the VCL NAL is different from the least significant bit of the picture sequence counter for the previous picture, determining whether the VCL is NAL by the first VCL NAL of the access unit (AU) containing the PU; And based on determining that the VCL NAL is the first VCL NAL of the AU, decoding the AU based on the VCL NAL. 2. The method of claim 1, wherein the VCL NAL unit is determined to be the first VCL NAL unit of said PU based on determining that the VCL NAL unit is the first VCL NAL unit following the picture header NAL unit. 3. The method of claim 1, wherein the VCL NAL unit is determined to be the first VCL NAL unit of said PU based on determining that a flag is set in the VCL NAL unit to indicate that a picture header is included in a slice header included in the VCL unit. NAL. 4. The method of claim 1, wherein the VCL NAL unit is determined to be the first VCL NAL unit of said AU also based on determining that the VCL NAL unit level ID is less than the previous picture level ID. 5. The method of claim 1, wherein, based on a flag indicating that all of the image encoding information is present in the image header, a plurality of flags corresponding to the image encoding information are not signaled. 6. A device for decoding a bit stream of encoded video data, comprising at least one memory and at least one processor, while said at least one memory stores a plurality of computer programs that, when executed by said at least one processor, ensure that the device executes the method according to any one of claims 1 to 5 through the use of computer programs. 7. A computer-readable medium storing instructions, including one or more instructions, which, when executed by one or more processors of the device for decoding the encoded video bitstream, causes the one or more processors to execute the method according to any one of claims 1-5.

Images