RU2787850C1 - Method for signaling of set of output layers in scalable video stream - Google Patents

Abstract

FIELD: video encoding/decoding.

SUBSTANCE: decoding of an encoded video bitstream, using at least one processor, includes obtainment of an encoded video sequence including multiple sets of output layers from the encoded video bitstream; obtainment of the first flag indicating, whether each set of output layers of multiple sets of output layers includes more than one layer; based on the first flag indicating that each set of output layers includes more than one layer, obtainment of the first syntax element indicating a mode of the set of output layers; selection of at least one layer from a number of layers included in multiple sets of output layers as at least one output layer based on at least one of the first flag and the first syntax element; and output of at least one output layer.

EFFECT: provision of a mechanism of signaling of a set of output layers for a scalable video stream.

17 cl, 7 dwg

Description

Cross-reference to related application

[1] This application claims priority of U.S. Provisional Application No. 62/912,275, filed October 8, 2019, and U.S. Patent Application No. 17/030,950, filed September 24, 2020, which are incorporated herein in their entirety.

The field of technology to which the invention belongs

[2] The disclosed subject matter relates to video encoding and decoding, and more specifically, to a mechanism for signaling a set of output layers for a scalable video stream.

State of the art

[3] ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11) published the H.265/HEVC (High Efficiency Video Coding) standard in 2013 (version 1) 2014. (version 2) 2015 (version 3) and 2016 (version 4). In 2015, these two standards organizations jointly formed the JVET (Joint Video Exploration Team) to explore the potential of developing the next video coding standard beyond HEVC. In October 2017, they launched a joint call for proposals for video compression with capabilities beyond HEVC (CfP). By February 15, 2018, a total of 22 standard dynamic range (SDR) CfP responses, 12 high dynamic range (HDR) CfP responses, and 12 360-degree video category CfP responses had been submitted, respectively. In April 2018, all CfP responses received were evaluated at the 122nd MPEG/10th JVET meeting. As a result of this meeting, JVET officially launched the next generation video coding standardization process beyond HEVC. The new standard was named Versatile Video Coding (VVC), and JVET was renamed the Joint Video Expert Team.

Disclosure of the essence of the invention

[4] In one embodiment, a method for decoding an encoded video bitstream using at least one processor is provided, including: deriving an encoded video sequence including a plurality of output layer sets from an encoded video bitstream; obtaining a first flag indicating whether each output layer set of the plurality of output layer sets includes more than one layer; based on the first flag indicating that each output layer set includes more than one layer, obtaining a first syntax element indicating the mode of the output layer set; selecting at least one layer from among the layers included in the plurality of output layer sets as at least one output layer based on at least one of the first flag and the first syntax element; and deriving at least one output layer.

[5] In one embodiment, an apparatus for decoding an encoded video bitstream is provided, comprising at least one memory element configured to store program code; and at least one processor configured to read the program code and operate in accordance with the instructions of the program code, the program code including: a first acquisition code configured to cause at least one processor to obtain an encoded video sequence including a plurality sets of output layers, from the encoded video bitstream; a second obtaining code configured to cause at least one processor to obtain a first flag indicating whether each output layer set of the plurality of output layer sets includes more than one layer; a third obtaining code configured to cause at least one processor, based on the first flag indicating that each output layer set includes more than one layer, to obtain the first syntax element indicating the mode of the output layer set; selection code configured to cause at least one processor to select a layer from among the layers included in the plurality of output layer sets as at least one output layer based on at least one of the first flag and the first syntax element; and output code configured to cause at least one processor to output at least one output layer.

[6] In one embodiment, a non-transitory computer-readable medium is provided that stores instructions containing one or more instructions that, when executed by one or more processors of a device for decoding an encoded video bitstream, cause one or more processors to: obtain an encoded video sequence including a plurality of sets of output layers, from the encoded video bitstream; receive a first flag indicating whether each output layer set of the plurality of output layer sets includes more than one layer; based on the first flag indicating that each output layer set includes more than one layer, obtain a first syntax element indicating the mode of the output layer set; select at least one layer from among the layers included in the plurality of output layer sets as at least one output layer based on at least one of the first flag and the first syntax element; output at least one output layer.

Brief description of the drawings

[7] Additional features, essence and various advantages of the disclosed subject matter will be better understood from the following detailed description and the accompanying drawings, in which:

[8] FIG. 1 is a schematic illustration of a simplified block diagram of a communication system in accordance with an embodiment.

[9] FIG. 2 is a schematic illustration of a simplified block diagram of a communication system in accordance with an embodiment.

[10] FIG. 3 is a schematic illustration of a simplified block diagram of a decoder according to an embodiment.

[11] FIG. 4 is a schematic illustration of a simplified block diagram of an encoder according to an embodiment.

[12] FIG. 5 is a schematic illustration of examples of a syntax table according to an embodiment.

[13] FIG. 6 shows a flowchart of an exemplary process for decoding an encoded video bitstream in accordance with an embodiment.

[14] FIG. 7 is a schematic illustration of a computer system in accordance with an embodiment.

Implementation of the invention

[15] FIG. 1 illustrates a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. System (100) may include at least two terminals (110 and 120) interconnected via network (150). For unidirectional data transmission, the first terminal (110) may encode video data at the local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the encoded video data of another terminal from the network (150), decode the encoded data, and display the recovered video data. Unidirectional data transfer may be common in media serving applications and the like.

[16] FIG. 1 shows a second pair of terminals (130, 140) providing support for bi-directional encoded video transmission, such as may occur during a video conference call. For bi-directional data transmission, each terminal (130, 140) may encode video data captured at a local location for transmission to another terminal over a network (150). Each terminal (130, 140) may also receive the encoded video data transmitted by the other terminal, may decode the encoded data, and may display the recovered video data on a local display device.

[17] FIG. 1, terminals (110-140) may be depicted as servers, personal computers, and smartphones, but the principles of the present disclosure cannot be limited to this. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or specialized video conferencing equipment. Network (150) is any number of networks that transmit encoded video data between terminals (110-140), including, for example, wired and/or wireless networks. The communication network (150) may communicate on circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may not be essential to the operation of this disclosure, unless explained herein below.

[18] FIG. 2 illustrates, as an example of application for the disclosed subject matter, the placement of a video encoder and a video decoder in a streaming environment. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, videoconferencing, digital television, storage of compressed video on digital media including CD, DVD, memory card, etc., and so on. .

[19] The streaming system may include a capture subsystem (213), which may include a video source (201), such as a digital camera, producing, for example, a stream (202) of uncompressed video samples. This stream (202) of samples, depicted with a thick line to emphasize the large amount of data compared to encoded video bitstreams, can be processed by an encoder (203) connected to a camera (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204), drawn with a thin line to emphasize the smaller amount of data compared to the sample stream, may be stored on the streaming server (205) for future use. One or more streaming clients (206, 208) can access the streaming server (205) to extract copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210) that decodes an incoming copy of the encoded video bitstream (207) and produces an outgoing stream (211) of video samples that can be displayed on a display (212) or other visualization device (not shown). In some streaming systems, video bitstreams (204, 207, 209) may be encoded according to certain video coding/compression standards. Examples of these standards include ITU-T Recommendation H.265. A video coding standard is under development, informally known as Universal Video Coding or VVC. The disclosed subject matter may be used in the context of VVC.

[20] FIG. 3 may be a functional block diagram of a video decoder (210) according to an embodiment of the present disclosure.

[21] The receiver (310) may receive one or more encoded video sequences to be decoded by the decoder (210); in the same or another embodiment, one coded video sequence at a time, where the decoding of each coded video sequence is independent of other coded video sequences. An encoded video sequence may be received from a channel (312), which may be a hardware/software connection to a storage device that stores the encoded video data. The receiver (310) may receive encoded video data with other data, such as encoded audio data and/or ancillary data streams, which may be sent to their respective using entities (not shown). The receiver (310) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (315) can be connected between the receiver (310) and the entropy decoder/parser (320) (hereinafter "parser"). When the receiver (310) receives data from a storage/transfer device with sufficient bandwidth and controllability, or from an isosynchronous network, buffer memory (315) may not be needed or may be small. For use in best quality packet networks such as the Internet, a buffer (315) may be required, which may be relatively large and may preferably be of an adaptive size.

[22] The video decoder (210) may include a parser (320) for recovering symbols (321) from the entropy encoded video sequence. The categories of these symbols include information used to control the operation of the decoder (210) and potentially information to control a visualization device such as a display (212) which is not an integral part of the decoder but can be connected to it as shown. in fig. 3. The control information for the rendering device(s) may be in the form of incremental enhancement information (SEI messages) or video usability information (VUI) parameter set fragments (not shown). The parser (320) may parse/entropy decode the received encoded video sequence. Encoding of the encoded video sequence may be in accordance with a video coding technology or standard, and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The parser (320) may extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOPs), pictures, tiles, slices, bricks, macroblocks, coding tree units (CTUs), coding units (CUs), blocks, transform units (TUs), prediction units (PUs), and so on. . A tile may indicate a rectangular area of a CU/CTU in a particular column and row of tiles in an image. A brick may point to a rectangular area of CU/CTU rows within a particular tile. A slice may point to one or more image bricks that are contained in a NAL unit. A sub-image may indicate a rectangular region of one or more slices in an image. The entropy decoder/parser may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[23] The parser (320) may perform an entropy decoding/parsing operation of the video sequence received from the buffer (315) to create symbols (321).

[24] Symbol recovery (321) may include many different modules depending on the type of encoded video image or parts thereof (such as: external and internal image, external and internal block) and other factors. Which modules are involved and how can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (320). The flow of such subgroup control information between the parser (320) and the plurality of modules is not shown below for clarity.

[25] In addition to the functional blocks already mentioned, the decoder 210 can be conceptually subdivided into a number of functional modules, as described below. In a practical implementation operating under commercial constraints, many of these modules interact closely with each other and can be at least partially integrated into each other. However, for purposes of describing the disclosed subject matter, the conceptual division into functional modules below is appropriate.

[26] The first module is the scaling/inverse transform module (351). The scaling/inverse transform module (351) receives the quantized transform coefficient, as well as control information, including which transform to use, block size, quantization factor, quantization scaling matrices, and so on, in the form of symbol(s) (321) from the parser ( 320). It can output blocks containing sample values that can be entered into the aggregator (355).

[27] In some cases, the output samples of the scaling/inverse transform module (351) may refer to an intra-coded block; that is, to a block that does not use prediction information from previously restored images, but can use prediction information from previously restored portions of the current image. Such prediction information may be provided by the intra prediction module (352). In some cases, the intra-prediction module (352) generates a block of the same size and shape as the reconstructed block using the already reconstructed surrounding information extracted from the current (partially reconstructed) image (358). The aggregator (355), in some cases, adds for each sample the prediction information generated by the intra-prediction module (352) to the output sample information provided by the scaling/inverse transform module (351).

[28] In other cases, the output samples of the scaling/inverse transform module (351) may refer to a block with intercoding and potentially with motion compensation. In such a case, the motion compensation prediction module (353) may access the reference picture memory (357) to retrieve samples used for prediction. After compensating for the movement of the selected samples according to the symbols (321) associated with the block, these samples can be added by the aggregator (355) to the output of the scaler/demap module (in this case called residual samples or residual signal) to generate output information. readings. Reference picture memory addresses from where the motion compensation prediction module selects prediction samples may be driven by motion vectors available to the motion compensation prediction module in the form of symbols (321), which may have X, Y and reference picture components, for example. Motion compensation may also include interpolation of sample values retrieved from reference picture memory when accurate sub-sample motion vectors are used, motion vector prediction mechanisms, and so on.

[29] The output samples of the aggregator (355) may be subjected to various loop filtering methods in the loop filter module (356). The video compression technologies may include in-loop filtering technologies that are controlled by parameters included in the encoded video bitstream and provided to the loop filtering module (356) as symbols (321) from the parser (320), but may also respond to metainformation received during decoding the previous (in the order of decoding) parts of the encoded image or encoded video sequence, as well as responding to previously restored and filtered by loop filtering sample values.

[30] The output of the loop filter module (356) may be a stream of samples that can be output to the renderer (212) and also stored in reference picture memory for use in future inter-picture prediction.

[31] Certain encoded pictures after full recovery can be used as reference pictures for future prediction. Once the encoded picture has been completely reconstructed and the encoded picture has been identified as a reference picture (eg, by the parser (320)), the current reference picture (358) may become part of the reference picture buffer (357), and fresh current picture memory may be reallocated before the beginning of the reconstruction of the next encoded picture.

[32] Video decoder 210 may perform decoding operations according to a predetermined video compression technology, which may be documented in a standard such as ITU-T Rec. H.265. The encoded video sequence may conform to the syntax given by the video compression technology or standard used, in the sense that it conforms to the syntax of the video compression technology or standard as specified in the video compression technology document or standard, and in particular their profile document. It may also be necessary for compliance that the complexity of the encoded video sequence be within limits defined by the level of technology or video compression standard. In some cases, layers limit the maximum image size, the maximum frame rate, the maximum recovery sampling rate (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the layers may, in some cases, be further constrained by hypothetical reference decoder (HRD) specifications and metadata for managing the HRD buffer transmitted in the encoded video sequence.

[33] In an embodiment, receiver (310) may receive additional (redundant) data with encoded video. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (210) to correctly decode the data and/or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, and so on.

[34] FIG. 4 may be a functional block diagram of a video encoder (203) according to an embodiment of the present disclosure.

[35] The encoder (203) may receive video samples from a video source (201) (which is not part of the encoder) that may capture video image(s) for encoding by the encoder (203).

[36] The video source (201) may provide the source video sequence for encoding by the encoder (203) in the form of a digital stream of video samples, which may have any suitable bit depth (eg: 8 bit, 10 bit, 12 bit, ...), any color space (eg, BT.601 Y CrCB, RGB, ...) and any suitable sample structure (eg, Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (201) may be a storage device storing pre-prepared video. In a videoconferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that, when viewed sequentially, convey motion. The images themselves can be organized as a spatial array of pixels, in which each pixel can contain one or more samples depending on the sample structure used, color space, and so on. One skilled in the art can easily understand the relationship between pixels and samples. The description below is based on readings.

[37] According to an embodiment, the encoder (203) may encode and compress images of the source video sequence into the encoded video sequence (443) in real time or with any other time constraints as required by the application. Providing an appropriate coding rate is one of the functions of the controller (450). The controller controls other functional modules, as described below, and is functionally associated with these modules. The connection is not shown for clarity. The parameters set by the controller may include parameters related to speed control (picture skip, quantizer, speed-distortion optimization methods lambda value, ...), image size, group of pictures (GOP) layout, maximum motion vector search range, and so on. . A person skilled in the art can easily determine other functions of the controller (450) as they may relate to a video encoder (203) optimized for a particular system design.

[38] Some video encoders operate in what one of ordinary skill in the art would readily recognize as a "coding loop". As a simplified description, an encoding loop may consist of an encoder coding part (430) (hereinafter "source encoder") (responsible for creating symbols based on the input picture to be encoded and the reference picture(s)), and a (local) decoder (433) built into the encoder (203) that decompresses the symbols to create sample data that the (remote) decoder can also create (because any compression between the symbols and the encoded video bitstream is lossless in the video compression technologies considered in the disclosed subject matter) . This reconstructed sample stream is entered into the reference picture memory (434). Because decoding a stream of symbols results in bit-accurate results, regardless of the location of the decoder (local or remote), the contents of the reference picture buffer are also bit-accurate between the local encoder and the remote encoder. In other words, the prediction portion of the encoder "sees" as reference picture samples exactly the same sample values that the decoder would "see" using prediction during decoding. This fundamental principle of reference picture synchronism (and the resulting drift if synchronism cannot be maintained due to, for example, channel errors) is well known to those skilled in the art.

[39] The operation of the "local" decoder (433) may be the same as that of the "remote" decoder (210), which has already been described in detail above in the description of FIG. 3. Briefly referring also to FIG. 4, however, since the symbols are available and encoding/decoding of the symbols into the encoded video sequence by the entropy encoder (445) and the parser (320) can be lossless, the entropy decoding parts of the decoder (210), including the channel (312), the receiver (310), buffer (315) and parser (320) cannot be fully implemented in the local decoder (433).

[40] At this point, it can be observed that any decoding technology other than parsing/entropy decoding that is present in a decoder must also necessarily be present in a substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on the operation of the decoder. The description of encoding technologies can be abbreviated as they are the inverse of fully described decoding technologies. Only in some areas a more detailed description is required, which is given below.

[41] As part of its operation, the source encoder (430) may perform motion-compensated predictive encoding that predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence, which have been designated "reference frames". Thus, the coding subsystem (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame(s) that can be selected as the prediction reference(s) for the input frame.

[42] The local video decoder (433) may decode the encoded video data of the frames, which may be designated as reference frames, based on the symbols generated by the source encoder (430). The operations of the encoding subsystem (432) may preferably be lossy processes. When the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (433) replicates the decoding processes that may be performed by the video decoder on the reference frames and may cause the recovered reference frames to be stored in the reference picture cache (434). Thus, the encoder (203) can locally store copies of the reconstructed reference frames that share content as the reconstructed reference frames to be received by the far-end video decoder (in the absence of transmission errors).

[43] The predictor (435) may perform predictive searches for the coding subsystem (432). That is, for a new picture to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as reference pixel block candidates) or certain metadata such as reference picture motion vectors, block shapes, and so on. further, which can serve as a suitable prediction reference for new images. The predictor (435) may operate on a block-by-pixel basis to find appropriate prediction references. In some cases, as determined by the search results obtained by the predictor (435), the input image may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (434).

[44] The controller (450) may control the encoding operations of the video encoder (430), including, for example, setting parameters and subgroup parameters used to encode video data.

[45] The output signals of all the above functional modules can be entropy encoded in an entropy encoder (445). The entropy encoder translates symbols generated by various functional modules into an encoded video sequence by lossless symbol compression according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, and so on.

[46] The transmitter (440) may buffer the encoded video sequence(s) generated by the entropy encoder (445) to prepare it for transmission over a communication channel (460), which may be a hardware/software connection to a storage device that will store the encoded video data. . The transmitter (440) may merge the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and/or ancillary data streams (sources not shown).

[47] The controller (450) may control the operation of the encoder (203). During encoding, the controller (450) may assign to each encoded picture a certain type of encoded picture, which may affect the encoding methods that can be applied to the corresponding picture. For example, images can often be assigned to one of the following frame types:

[48] An intra-picture (I-picture) may be one that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow various types of internal images, including, for example, images with independent decoder update. The person skilled in the art will be aware of these variants of I-images and their respective uses and features.

[49] A predictive picture (P-picture) may be one that can be encoded and decoded using intra-prediction or inter-prediction using at most one motion vector and a reference index to predict sample values of each block.

[50] A bidirectional predictive picture (B-picture) may be one that can be encoded and decoded using intra-prediction or inter-prediction using at most two motion vectors and reference indices to predict the sample values of each block. Similarly, multi-prediction images may use more than two reference images and associated metadata to reconstruct a single block.

[51] Source images can typically be spatially divided into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and encoded on a block-by-block basis. The blocks may be predictively encoded with reference to other (already encoded) blocks, as determined by the coding assignment applied to the corresponding block images. For example, I-picture blocks may be predictively encoded or they may be predictively encoded with reference to already encoded blocks of the same picture (spatial prediction or intra prediction). P-picture pixel blocks may be encoded without prediction, by spatial prediction, or by temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be encoded without prediction, by spatial prediction, or by temporal prediction with reference to one or two previously encoded reference pictures.

[52] The video encoder (203) may perform encoding operations in accordance with a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The encoded video data may therefore follow the syntax given by the video coding technology or standard used.

[53] In an embodiment, transmitter (440) may transmit additional data with encoded video. Video encoder (430) may include such data as part of an encoded video sequence. The additional data may comprise temporal/spatial/SNR enhancement layers, other forms of redundant data such as redundant images and slices, incremental enhancement information (SEI) messages, video usability information (VUI) parameter set fragments, and so on.

[54] Embodiments may relate to a mechanism for signaling a set of output layers for a scalable video stream. Embodiments may relate to a method for deriving the respective (forward dependent/independent) layers of each set of output layers from the direct support layer information when output layers are signaled for each set of output layers.

[55] In embodiments, an output layer may refer to a layer from a set of output layers that is output. In embodiments, an output layer set may refer to a given layer set, where one or more layers in the layer set are specified as output layers. In embodiments, the layer index in the output layer set may refer to the layer index in the OUTPUTLAYERSET, to the list of layers in the OUTPUTLAYERSET.

[56] FIG. 5 shows an example of a syntax table related to the Video Parameter Set (VPS) raw byte sequence (RBSP) payload syntax, according to an embodiment.

[57] In embodiments, the VPS RBSP may be available to the decoding process prior to being a reference included in at least one access unit with TemporalId equal to 0 or provided through external means, and the VPS NAL unit containing the VPS RBSP , may have nuh_layer_id equal to vps_layer_id[0].

[58] All VPS NAL units with a certain vps_video_parameter_set_id value in CVS may have the same content.

[59] vps_video_parameter_set_id provides a VPS identifier for reference by other syntax elements.

[60] The value of vps_max_layers_minus1 plus 1 may indicate the maximum allowed number of layers in each CVS referencing the VPS.

[61] A vps_all_independent_layers_flag value of 1 may indicate that all layers in CVS are independently encoded without using interlayer prediction. A vps_all_independent_layers_flag value of 0 may indicate that one or more layers in CVS may use interlayer prediction. If not present, the value of vps_all_independent_layers_flag may be inferred to be 1. When vps_all_independent_layers_flag is equal to 1, the value of vps_independent_layer_flag[i] may be inferred to be 1.

[62] vps_layer_id [i] may indicate the value of the nuh_layer_id of the ith layer. For any two non-negative integer values m and n, when m is less than n, vps_layer_id[m] may be less than vps_layer_id[n].

[63] In this case, vps_independent_layer_flag [i] equal to 1 may indicate that the layer with index i does not use interlayer prediction. In this case, vps_independent_layer_flag [i], equal to 0, may indicate that the layer with index i can use interlayer prediction, and vps_layer_dependency_flag [i] is present in the VPS. If absent, the vps_independent_layer_flag[i] value can be inferred to be 1.

[64] vps_direct_ref_layer_flag [i] [j] equal to 0 may indicate that layer index j is not a direct reference layer for layer index i. Wherein vps_direct_ref_layer_flag [i] [j] equal to 1 may indicate that the layer with index j is a direct reference layer for the layer with index i. When vps_direct_ref_layer_flag[i][j] is not present for i and j in the range 0 to vps_max_layers_minus 1 inclusive, it may be output as 0.

[65] The variables NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], and RefLayerIdx[i][r] can be retrieved as follows:

[66] The variable GeneralLayerIdx[i], which defines the layer index for a layer with nuh_layer_id equal to vps_layer_id[i], can be obtained as follows:

[67] each_layer_is_an_outputLayerSet_flag equal to 1 may indicate that each output layer set contains only one layer, and each layer in the bitstream is itself an output layer set with the only included layer being the output layer. With each_layer_is_an_outputLayerSet_flag equal to 0, it can indicate that the output layer set can contain more than one layer. If vps_max_layers_minus1 is 0, the vlaue value for each_layer_is_an_outputLayerSet_flag may be inferred to be 1. Otherwise, when vps_all_independent_layers_flag is 0, the value of each_layer_is_an_outputLayerSet_flag may be inferred to be 0.

[68] outputLayerSet_mode_idc equal to 0 may indicate that the total number of OUTPUTLAYERSET set by the VPS is vps_max_layers_minus1+1, the i-th OUTPUTLAYERSET includes layers with layer indices from 0 to i inclusive, and only the highest OUTPUTLAYERSET layer is output for each OUTPUTLAYERSET.

[69] outputLayerSet_mode_idc equal to 1 may indicate that the total number of OUTPUTLAYERSET given by the VPS is vps_max_layers_minus1+1, the i-th OUTPUTLAYERSET includes layers with layer indices from 0 to i inclusive, and for each OUTPUTLAYERSETUTLAYERSET, all layers in the OUTPUTLAYERSETLAYERSET are output.

[70] outputLayerSet_mode_idc equal to 2 may indicate that the total number of OUTPUTLAYERSET set by the VPS is explicitly signaled, and for each OUTPUTLAYERSET, output layers are explicitly signaled, and the other layers are layers that are layers that directly or indirectly support output OUTPUTLAYERSET layers.

[71] The value of outputLayerSet_mode_idc can range from 0 to 2 inclusive. The value 3 outputLayerSet_mode_idc is reserved for future use by ITU-T | ISO/IEC.

[72] When vps_all_independent_layers_flag is equal to 1 and each_layer_is_an_outputLayerSet_flag is equal to 0, the value of outputLayerSet_mode_idc may be output as 2.

[73] num_output_layer_sets_minus1 plus 1 can indicate the total number of OUTPUTLAYERSETs specified by the VPS when outputLayer_Set_mode_idc is 2.

[74] The TotalNumOutputLayerSets variable, which specifies the total number of OUTPUTLAYERSETs set by the VPS, can be obtained as follows:

[75] outputLayerSet_output_layer_flag [i] [j] equal to 1 may indicate that the layer with nuh_layer_id equal to vps_layer_id [j] is the i-th OUTPUTLAYERSET output layer when outputLayerSet_mode_idc is equal to 2. In this case, outputLayerSet_output_layer_flag[i][j] equal to 0 may indicate that the layer with nuh_layer_id equal to vps_layer_id[j] is not the output layer of the i-th OUTPUTLAYERSET when outputLayerSet_mode_idc is equal to 2.

[76] The NumOutputLayersInOutputLayerSet [i] variable indicating the number of output layers in the i-th OUTPUTLAYERSET and the OutputLayerldlnOutputLayerSet [i] [j] variable specifying the nuh_layer_id value for the j-th output layer in the i-th OUTPUTLAYERSET are obtained as follows:

[77] The NumLayersInOutputLayerSet [i] variable indicating the number of layers in the i-th OUTPUTLAYERSET and the LayerIdInOutputLayerSet [i] [j] variable specifying the nuh_layer_id value for the j-th layer in the i-th OUTPUTLAYERSET are obtained as follows:

[78] An OutputLayerSetLayeIdx[i][j] variable indicating the layer index OUTPUTLAYERSET of a layer with nuh_layer_id equal to LayerIdInOutputLayerSet[i][j] can be obtained as follows:

[79] The bottommost layer in each OUTPUTLAYERSET may be an independent layer. In other words, for every i in the range 0 to TotalNumOutputLayerSets-1 inclusive, vps_independent_layer_flag[GeneralLayerIdx[LayerIdInOutputLayerSet[i][0]]] may be equal to 1.

[80] Each layer may be included in at least one OUTPUTLAYERSET specified by the VPS. In other words, for each layer with a particular nuh_layer_id_nuhLayerId value equal to one of vps_layer_id[k] for k in the range 0 to vps_max_layers_minus1 inclusive, there can be at least one pair of i and j values, where i is in the range 0 to TotalNumOutputLayerSets - 1 inclusive, a j is in the range NumLayersInOutputLayerSet[i] - 1 inclusive, so the value of LayerIdInOutputLayerSet[i][j] is nuhLayerId.

[81] A vps_constraint_info_present_flag of 1 may indicate that a general_constraint_info() syntax structure is present in the VPS. However, a vps_constraint_info_present_flag of 0 may indicate that the general_constraint_info() syntax structure is not present in the VPS.

[82] vps_reserved_zero_7bits may be 0 in bitstreams conforming to this version of this specification. Other vps_reserved_zero_7bits values are reserved for future use by ITU-T | ISO/IEC. Decoders MAY ignore the vps_reserved_zero_7bits value.

[83] A vps_extension_flag of 0 may indicate that there are no vps_extension_data_flag syntax elements in the VPS RBSP syntax structure. However, a vps_extension_flag of 1 may indicate that vps_extension_data_flag syntax elements are present in the VPS RBSP syntax structure.

[84] vps_extension_data_flag can be any value. Its presence and value do not affect the compliance of the decoder with the profiles specified in this version of this Specification. Decoders conforming to this version of this specification MAY ignore all elements of the vps_extension_data_flag syntax.

[85] FIG. 6 shows an exemplary flowchart of a process 600 for decoding an encoded video bitstream. In some implementations, one or more of the process blocks of FIG. 6 may be performed by decoder 210. In some implementations, one or more of the process blocks of FIG. 6 may be performed by another device or group of devices separate from or including decoder 210, such as encoder 203.

[86] As shown in FIG. 6, process 600 may include deriving an encoded video sequence including a plurality of output layer sets from an encoded video bitstream (block 601).

[87] As further shown in FIG. 6, process 600 may include obtaining a first flag (block 602).

[88] As further shown in FIG. 6, process 600 may include determining, based on a first flag, whether each output layer set of the multiple output layer sets includes more than one layer (block 603).

[89] As further shown in FIG. 6, process 600 may include, based on a first flag indicating that each output layer set includes only one layer (NO at block 603), selecting one layer of each output layer set as at least one output layer ( block 604) and drawing at least one output layer (block 608).

[90] As further shown in FIG. 6, based on the first flag indicating that each output layer set includes more than one layer (YES at block 603), process 600 may proceed to block 605, block 606, block 607, and block 608.

[91] As further shown in FIG. 6, process 600 may include obtaining a first syntax element indicating the mode of the output layer set (block 605).

[92] As further shown in FIG. 6, process 600 may include determining an output layer setting mode based on the first syntax element (block 606).

[93] As further shown in FIG. 6, process 600 may include selecting at least one layer from among the layers included in the output layer set set as at least one output layer based on the output layer set mode (block 607).

[94] As further shown in FIG. 6, process 600 may include drawing at least one output layer (block 608).

[95] In an embodiment, the first flag and the first syntax element may be signaled in a video parameter set (VPS).

[96] In an embodiment, one layer may be selected as at least one output layer based on a first flag indicating that each output layer set includes only one layer.

[97] In an embodiment, the topmost layer of each output layer set may be selected as at least one output layer based on the first syntax element indicating that the mode of the output layer set is the first mode.

[98] In an embodiment, all layers included in the output layer set plurality may be selected as at least one output layer based on the first syntax element indicating that the output layer set mode is the second mode.

[99] In an embodiment, based on the first syntax element indicating that the output layer set mode is the third mode, at least one output layer can be selected from among the layers included in the output layer set plurality based on the second syntax element, which is signaled in the encoded video bitstream.

[100] In an embodiment, based on the first syntax element indicating that the output layer set mode is the third mode, unselected layers from among the layers included in the multiple output layer sets can be used as reference layers for at least one output layer. .

[101] In an embodiment, the output layer set mode may be derived as a third mode, based on a first flag indicating that each output layer set includes more than one layer, and a second flag indicating that all layers included in the set of output layers output layers are encoded independently.

[102] Although FIG. 6 shows exemplary blocks of process 600, in some implementations process 600 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently from those depicted in FIG. 6. Additionally or alternatively, two or more blocks of process 600 may be executed in parallel.

[103] In addition, the proposed methods can be implemented by processing circuits (eg, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program that is stored on a non-transitory computer-readable medium to implement one or more of the proposed methods.

[104] The methods described above may be implemented in computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 7 shows a computer system 700 suitable for implementing certain embodiments of the disclosed subject matter.

[105] Computer software may be coded using any suitable machine code or computer language, which may be the object of assembly, compilation, linking, or similar mechanisms to produce code containing instructions that can be executed directly or through interpretation, execution of microcode, and and the like, computer central processing units (CPUs), graphics processing units (GPUs), and the like.

[106] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[107] The components shown in FIG. 7 for computer system 700 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of computer software implementing embodiments of the present disclosure. The component configuration is also not to be interpreted as having any dependency or requirement relating to any one or combination of the components illustrated in the exemplary embodiment of computer system 700.

[108] Computer system 700 may include certain human-machine interface input devices. Such a human-machine interface input device may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, movement of a control glove), audio input (e.g., voice, handclaps), visual input (for example: gestures), olfactory input (not shown). HMI devices can also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, ambient sound), images (e.g. scanned images, photographic images taken from a camera for still images), video (eg 2D video, 3D video including stereoscopic video).

[109] The human-machine interface input devices may include one or more of (only one of each depicted): a keyboard 701, a mouse 702, a trackpad 703, a touch screen 710 and an associated graphics adapter 750, a control glove, a joystick 705, microphone 706, scanner 707, camera 708.

[110] Computer system 700 may also include certain human interface output devices. Such human interface output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell/taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback from a 710 touch screen, 704 control glove, or 705 joystick, but there may also be haptic feedback devices that do not serve as input devices) , audio output devices (for example: 709 speakers, headphones (not shown)), image output devices (for example, 710 screens, including CRT screens, LCD screens, plasma screens, OLED screens, each of which has or does not have the ability to touchscreen inputs, each with or without haptic feedback capability - some of which may be capable of outputting 2D visual output or more than 3D output by means such as stereographic output; virtual reality goggles (not shown), holographic displays and smoke tanks (not shown)) and printers (not shown).

[111] Computer system 700 may also include human accessible storage devices and associated media such as optical media including CD/DVD ROM/RW 720 with CD/DVD or similar media 721, flash drive 722, removable hard disk or solid state drive 723, previously developed magnetic media such as tape and floppy disk (not shown), dedicated ROM/ASIC/PLD based devices such as dongles (not shown), etc.

[112] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other time signals.

[113] Computer system 700 may also include interface(s) to one or more communication networks (1155). Networks can be, for example, wireless, wired, optical. In addition, networks can be local, wide area, metropolitan, automotive, industrial, real-time, latency tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless local area networks (LANs), cellular networks including global systems for mobile communications (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), Long Term Evolution (LTE) networks, etc., wired television or wireless wide area digital networks, including cable television, satellite television and terrestrial broadcast television, automotive and industrial networks, including CAN bus, and so on. Some networks typically require external network interface adapters (1154) that are connected to certain general purpose data ports or peripheral buses (1149) (for example, the universal serial bus (USB) ports of the 700 computer system); others are typically integrated into the core of computer system 700 by being attached to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). For example, network 755 may be connected to peripheral bus 749 using network interface 754. Using any of these networks, computer system 700 may communicate with other entities. Such communication can be unidirectional, receive-only (for example, broadcast television), unidirectional, send-only (for example, CAN bus to certain CAN-bus devices), or bidirectional, for example, to other computer systems using a local or wide area digital network. . Each of these networks and network interfaces (1154) may use specific protocols and protocol stacks as described above.

[114] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces may be attached to the core 740 of the computer system 700.

[115] The core 740 may include one or more central processing units (CPUs) 741, graphics processing units (GPUs) 742, dedicated field programmable gate array (FPGA) processors 743, hardware accelerators 744 for certain tasks, etc. . These devices, along with Read Only Memory (ROM) 745, Random Access Memory (RAM) 746, internal mass storage such as internal non-user hard drives, solid state drives (SSDs), and the like. 747 may be connected via system bus 748. In some computer systems, system bus 748 may be available as one or more physical connectors to allow expansion with additional CPUs, GPUs, and the like. Peripherals may be connected either directly to the core system bus 748 or via a peripheral bus 749. Peripheral bus architecture includes Peripheral Component Interconnect (PCI), USB, and the like.

[116] CPU 741, GPU 742, FPGA 743, and accelerators 744 may execute certain instructions, which in combination may constitute the aforementioned computer code. This computer code may be stored in ROM 745 or RAM 746. Transient data may also be stored in RAM 746, while persistent data may be stored in, for example, internal mass storage 747. Rapid storage and retrieval to any of the storage devices may be accessed by the use of cache memory, which may be closely associated with one or more of the CPU 741, GPU 742, mass storage 747, ROM 745, RAM 746, and the like.

[117] The computer-readable medium may contain computer code for performing various computer-implemented operations. The media and computer code may be specially designed and constructed for the purposes of this disclosure, or they may be well known and available to those skilled in the art of computer software.

[118] By way of example, and not limitation, a computer system having architecture 700, and in particular core 740, may provide functionality as a result of having processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied on one or more tangible, computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, as well as certain core storage devices 740 that are of a non-transitory nature, such as internal mass storage 747 or ROM 745. implementation of the present disclosure may be stored in such devices and executed by the core 740. The computer-readable medium may include one or more memories or chips in accordance with specific needs. The software may cause the core 740, and in particular the processors therein (including the CPU, GPU, FPGA, etc.) to execute certain processes or certain parts of the specific processes described herein, including the definition of data structures stored in RAM 746 and modifying such data structures in accordance with processes defined by the software. In addition or alternatively, a computer system may provide functionality as a result of logic embedded in hardware circuitry or otherwise embodied in circuitry (e.g., an accelerator 744) that can operate in place of or in conjunction with software to perform certain processes or parts of specific processes described in this document. The link to the software may include logic and vice versa where applicable. A reference to a computer-readable medium may include circuitry (eg, an integrated circuit (IC)) storing software to execute, circuitry embodying logic to execute, or both, where applicable. The present disclosure includes any suitable combination of hardware and software.

[119] While this disclosure describes several exemplary embodiments, there are variations, permutations, and various replacement equivalents that fall within the scope of the disclosure. Thus, it will be appreciated that those skilled in the art will be able to develop numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the disclosure and thus fall within its spirit and scope.

Claims (17)

1. A method for decoding an encoded video bitstream using at least one processor, comprising: obtaining an encoded video sequence including a plurality of sets of output layers from an encoded video bitstream; obtaining a first flag indicating whether each output layer set of the plurality of output layer sets includes more than one layer; based on the first flag indicating that each output layer set includes more than one layer, obtaining a first syntax element indicating the mode of the output layer set; selecting at least one layer from among the layers included in the plurality of output layer sets as at least one output layer based on at least one of the first flag and the first syntax element; and deriving at least one output layer for decoding the encoded video bitstream, wherein based on the first syntax element indicating that the output layer set mode is the third mode, at least one output layer set is selected from among the layers included in the plurality of output layer sets. layer based on the second syntax element that is signaled in the encoded video bitstream. 2. The method of claim 1, wherein the first flag and the first syntax element are signaled in a video parameter set (VPS). 3. The method of claim 1, wherein one layer is selected as at least one output layer based on a first flag indicating that each output layer set includes only one layer. 4. The method of claim 1, wherein the topmost layer of each output layer set is selected as the at least one output layer based on a first syntax element indicating that the mode of the output layer set is the first mode. 5. The method of claim 1, wherein all layers included in the plurality of output layer sets are selected as the at least one output layer based on the first syntax element indicating that the mode of the output layer set is the second mode. 6. The method of claim 1, wherein based on the first syntax element indicating that the output layer set mode is the third mode, unselected layers from among the layers included in the output layer set plurality are used as reference layers for at least one output layer. 7. The method of claim 1, wherein the output layer set mode is output as the third mode based on the first flag indicating that each output layer set includes more than one layer and the second flag indicating that all layers included in the set sets of output layers are encoded independently. 8. A device for decoding an encoded video bitstream, the device comprising: at least one memory element configured to store program code; and at least one processor configured to read the program code and operate in accordance with the instructions of the program code, the program code including: a first acquisition code configured to cause at least one processor to obtain an encoded video sequence including a plurality sets of output layers, from the encoded video bitstream; a second obtaining code configured to cause at least one processor to obtain a first flag indicating whether each output layer set of the plurality of output layer sets includes more than one layer; a third obtaining code configured to cause at least one processor, based on the first flag indicating that each output layer set includes more than one layer, to obtain the first syntax element indicating the mode of the output layer set; selection code configured to cause at least one processor to select at least one layer from among the layers included in the plurality of output layer sets as at least one output layer based on at least one of the first flag and the first element syntax; and a derivation code configured to cause at least one processor to output at least one output layer for decoding the encoded video bitstream, moreover, based on the first syntax element indicating that the output layer set mode is the third mode, among the layers, included in the plurality of output layer sets, at least one output layer is selected based on a second syntax element that is signaled in the encoded video bitstream. 9. The apparatus of claim 8, wherein the first flag and the first syntax element are signaled in a video parameter set (VPS). 10. The apparatus of claim 8, wherein one layer is selected as at least one output layer based on a first flag indicating that each output layer set includes only one layer. 11. The apparatus of claim 8, wherein the topmost layer of each output layer set is selected as the at least one output layer based on a first syntax element indicating that the mode of the output layer set is the first mode. 12. The apparatus of claim 8, wherein all layers included in the plurality of output layer sets are selected as the at least one output layer based on the first syntax element indicating that the output layer set mode is the second mode. 13. The apparatus of claim 8, wherein based on the first syntax element indicating that the output layer set mode is the third mode, unselected layers from among the layers included in the output layer set plurality are used as reference layers for at least one output layer. 14. The apparatus of claim 8, wherein the output layer set mode is output as the third mode based on a first flag indicating that each output layer set includes more than one layer and a second flag indicating that all layers included in the set sets of output layers are encoded independently. 15. A non-transitory computer-readable medium storing instructions containing: one or more instructions that, when executed by one or more processors of a device for decoding an encoded video bitstream, instruct one or more processors to: obtain an encoded video sequence including a plurality of sets of output layers from an encoded video bitstream; receive a first flag indicating whether each output layer set of the plurality of output layer sets includes more than one layer; based on the first flag indicating that each output layer set includes more than one layer, obtain a first syntax element indicating the mode of the output layer set; select at least one layer from among the layers included in the plurality of output layer sets as at least one output layer based on at least one of the first flag and the first syntax element; output at least one output layer, and based on the first syntax element indicating that the output layer set mode is the third mode, at least one output layer is selected from among the layers included in the plurality of output layer sets based on the second syntax element, which is signaled in the encoded video bitstream. 16. The non-transitory computer-readable medium of claim 15, wherein the topmost layer of each output layer set is selected as the at least one output layer based on a first syntax element indicating that the mode of the output layer set is the first mode. 17. The non-transitory computer-readable medium of claim 15, wherein all layers included in the output layer set plurality are selected as the at least one output layer based on the first syntax element indicating that the output layer set mode is the second mode.

Images