RU2783961C1 - Method for indicating the number of sublevels in a multilevel video stream - Google Patents

Abstract

FIELD: video decoding.

SUBSTANCE: group of inventions relates to video encoding and decoding technologies, more specifically, to extracting the output level in a multilevel encoded video stream. Proposed is a method for video decoding executed by at least one processor. The method includes a stage of identifying the maximum number of time sublevels permitted in the encoded video sequence referring to a video parameter set in a video parameter set (VPS) field and in a sequence parameter set (SPS) field, respectively. The encoded video sequence is then decoded based on the identified maximum number of time sublevels permitted in the encoded video sequence referring to a video parameter set.

EFFECT: reduced redundancy in the input video signal by means of compression.

9 cl, 23 dwg, 1 tbl

Description

Cross-references to related applications

[0001] The present invention claims priority of U.S. Patent Application No. 16/994,004, filed Aug. 14, 2020, which, pursuant to 35 U.S.C. § 119, claims priority of U.S. Provisional Application No. 63/000,936, filed March 27, 2020, at the US Patent and Trademark Office. The contents of previous applications are hereby incorporated by reference in their entirety.

Technical field

[0002] The disclosed invention relates to video encoding and decoding, and more specifically, to extracting an output level in an encoded video stream with multiple layers.

State of the art

[0003] Video encoding and decoding using motion-compensated inter-picture prediction has been known for several decades. Uncompressed digital video may include a sequence of images, each of which has a spatial size of, for example, 1920×1080 luma samples and their associated chrominance samples. An image sequence may have a fixed or variable image rate (informally also referred to as a "frame rate"), such as 60 images per second, or 60 Hz. Uncompressed video data has significant bit rate requirements. For example, a 1080p60 4:2:0 video with 8 bits per sample (resolution of 1920×1080 luminance samples at 60Hz frame rate) requires about 1.5 Gbps of bandwidth. An hour of such video requires more than 600 GB of memory.

[0004] One goal of video encoding and decoding may be to reduce redundancy in the input video signal by compressing. Compression can help to alleviate the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. For this, both lossless and lossy compression can be used. Lossless compression refers to techniques for reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the discrepancy between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used for video. The amount of distortion allowed depends on the application; for example, users of some custom streaming applications may tolerate higher distortion than users of broadcast television applications. The achievable compression ratio may reflect that a higher allowed/allowable distortion may result in a higher compression ratio.

[0005] The video encoder and decoder may use techniques from several broad categories including, for example, motion compensation, transformation, quantization, and entropy coding, some of which are presented below.

[0006] Historically, video encoders and video decoders were designed to work with a given image size, which in most cases for a coded video sequence (CVS, Coded Video Sequence), a group of pictures (GOP, Group of Pictures) or a similar multi-picture time frame was defined and remained constant. For example, with respect to MPEG-2, system designs are known to change the horizontal resolution (and hence image size) depending on factors such as scene activity, but only for 1-pictures, which is typically the case for GOP. Resampling of reference pictures to use different resolutions within CVS is known, for example, from ITU-T H.263 Annex P. However, in this case, the image size does not change, resampling is performed only for reference pictures, as a result of which only parts of the image are potentially used. (in the case of downsampling) or only part of the captured scene (in the case of upsampling). In addition, the H.263 standard, Annex Q allows single macroblock resampling by a factor of two (in each dimension), up or down. Again, the image is not resized. The macroblock size is fixed in H.263 and therefore does not need to be transmitted.

[0007] Image resizing in predicted images has become mainstream in current video coding techniques. For example, VP9 provides the ability to resample reference pictures and change resolution for the entire picture. Similarly, some solutions for VVC (including, for example, the solution by Hendry et al., "About Adaptive Resolution Resizing (ARC) for VVC", Joint Video Team document JVET-M0135-vl, January 9-19, 2019. , incorporated herein in its entirety) provide the possibility of resampling entire reference pictures with a different, higher or lower resolution. This document proposes to encode the different resolutions in a sequence parameter set and refer to them using the syntax elements for each image in the image parameter set.

The essence of the invention

[0008] Techniques for signaling adaptive image size in a video bitstream are disclosed.

[0009] The method may include determining the maximum number of temporal sublayers allowed in the encoded video sequence referring to the parameter set; and decoding the coded video sequence based on the identified maximum number of temporal sublayers allowed in the coded video sequence referring to the parameter set.

[0010] The device may include at least one memory module configured to store computer program code; and at least one processor configured to read the program code and operating under the control of the computer program code, wherein the program code may include an identification code configured such that the processor identifies the maximum number of temporal sublevels allowed in the encoded video sequence, referring to a set of parameters; and decoding code configured such that the processor decodes the encoded video sequence based on the identified maximum number of temporal sublayers allowed in the encoded video sequence referring to the parameter set.

[0011] The computer-readable medium may comprise instructions that, when executed by one or more device processors, cause the one or more processors to identify the maximum number of temporal sublayers allowed in the encoded video sequence referring to the parameter set; and decoding the encoded video sequence based on the identified maximum number of temporal sublayers allowed in the encoded video sequence referring to the parameter set.

Brief description of the drawings

[0012] Additional features, nature and various advantages of the disclosed invention will be apparent from the following detailed description and the accompanying drawings, in which:

[0013] FIG. 1 is a schematic representation of a simplified block diagram of a communication system according to an embodiment;

[0014] FIG. 2 is a schematic representation of a simplified block diagram of a communication system according to an embodiment;

[0015] FIG. 3 is a schematic representation of a simplified block diagram of a decoder according to an embodiment;

[0016] FIG. 4 is a schematic representation of a simplified block diagram of an encoder according to an embodiment;

[0017] FIG. 5 is a schematic representation of ARC parameter signaling options according to the prior art or embodiment, as shown;

[0018] FIG. 6 is an example of a syntax table according to an embodiment;

[0019] FIG. 7 is a schematic representation of a computer system according to an embodiment;

[0020] FIG. 8 is an example of a prediction structure for adaptive resolution scalability;

[0021] FIG. 9 is an example of a syntax table according to an embodiment;

[0022] FIG. 10 is a schematic representation of a simplified flowchart of parsing and decoding a DOC cycle according to an access block and an access block sequence number;

[0023] FIG. 11 is a schematic representation of the structure of a video bitstream containing layered sub-images;

[0024] FIG. 12 is a schematic representation of an enhanced resolution sub-image display;

[0025] FIG. 13 is a schematic representation of a process for decoding and displaying a video bitstream containing layered sub-images;

[0026] FIG. 14 is a schematic representation of a 360 degree video display with a sub-image enhancement level;

[0027] FIG. 15 is an example of sub-image structure information and corresponding level structure and image prediction;

[0028] FIG. 16 shows an example of sub-image structure information and corresponding layer structure and image prediction with local area spatial scalability;

[0029] FIG. 17 is an example of a syntax table for a sub-image structure;

[0030] FIG. 18 is an example of an SEI message syntax table for sub-picture structure information;

[0031] FIG. 19 is an example of a syntax table for specifying output levels and profile/tier/level information for each set of output levels;

[0032] FIG. 20 is an example of a syntax table for specifying an output level mode for each set of output levels;

[0033] FIG. 21 is an example of a syntax table for specifying the current sub-image of each level for each set of output levels;

[0034] FIG. 22 is an example of a video parameter set RBSP;

[0035] FIG. 23 is an example of a sequence parameter set RBSP.

Detailed description

[0036] When images are encoded into a bitstream that includes multiple levels of different quality, the bitstream may have syntax elements that determine which levels can be output on the decoder side. The set of levels to be output is defined as the set of output levels. In the latest video codec supporting multiple levels and scalability, one or more output level sets are carried in a video parameter set. These syntax elements defining output level sets and their dependencies, profile/tier/level parameters and a hypothetical decoder reference model must be efficiently conveyed in a parameter set.

[0037] FIG. 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the present invention. System (100) may include at least two terminals (110 and 120) connected to each other via network (150). In the case of unidirectional data transmission, the first terminal (110) may encode video data at the local location for transmission to another terminal (120) via the network (150). The second terminal device (120) may receive the encoded video data of another terminal device from the network (150), decode the encoded data, and display the reconstructed video data. One-way communication may be inherent in media service applications and the like.

[0038] FIG. 1 shows a second pair of terminals (130, 140) designed to support bi-directional transmission of encoded video data, which may occur, for example, during a video conference. In the case of bi-directional data transmission, each terminal device (130, 140) may encode the video data captured at the local location for transmission to another terminal device via the network (150). Each terminal device (130, 140) may also receive the encoded video data transmitted by the other terminal device, may decode the encoded data, and may display the reconstructed video data on a local display device.

[0039] FIG. 1, terminal devices (110-140) can be illustrated as servers, personal computers, and smartphones, but this does not limit the principles of the present invention. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or dedicated video conferencing equipment. Network (150) represents any number of networks that carry encoded video data between terminals (110-140), including, for example, wired and/or wireless networks. The communication network (150) allows data to be exchanged in a circuit-switched and/or packet-switched mode. Exemplary networks include telecommunications networks, local area networks, wide area networks, and/or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may not be relevant to the present invention unless specifically noted below.

[0040] In FIG. 2 shows, as an example of the application of the disclosed invention, the placement of a video encoder and a video decoder in a streaming environment. The disclosed invention may be equally applicable to other video processing applications including, for example, videoconferencing, digital television, storage of compressed video on digital media, including CD, DVD, memory card, and the like.

[0041] The streaming system may include a capture subsystem (213), which may include a video source (201), such as a digital camera, producing, for example, a stream (202) of uncompressed video samples. This stream (202) of samples, shown with a thick line to emphasize the large amount of data compared to the bitstreams of the encoded video, can be processed by the encoder (203) connected to the camera (201). The encoder (203) may include hardware, software, or a combination thereof for providing or implementing aspects of the disclosed invention, as described in more detail below. The encoded video bitstream (204), shown as a thin line to emphasize the smaller amount of data compared to the sample stream, may be stored on the streaming server (205) for future use. One or more streaming clients (206, 208) may access the streaming server (205) to retrieve copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210) that decodes an incoming copy of the encoded video bitstream (207) and produces an outgoing stream (211) of video samples that can be displayed on a display (212) or other rendering device (not shown). ). In some streaming systems, the video bitstreams (204, 207, 209) may be encoded according to one video coding/compression standard or another. Examples of these standards include ITU-T Recommendation H.265. A video coding standard is being developed, officially called Versatile Video Coding or VVC. The disclosed invention may be used in the context of VVC.

[0042] FIG. 3 may be a functional block diagram of a video decoder (210) according to an embodiment of the present invention.

[0043] The receiver (310) may receive one or more codec video sequences for decoding by the decoder (210); in the same or another embodiment, one coded video sequence at a time, where the decoding of each coded video sequence is independent of other coded video sequences. The encoded video sequence may be received from a channel (312), which may be a hardware/software link to a storage device where the encoded video data is stored. The receiver (310) may receive encoded video data with other data, such as encoded audio data and/or auxiliary data streams, which may be relayed to their respective usable objects (not shown). The receiver (310) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (315) may be connected between the receiver (310) and the entropy decoder/analyzer (320) (hereinafter "analyzer"). If the receiver (310) receives data from a storage/forwarder with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer (315) may not be used or may be small. For use in best effort packet networks, such as the Internet, a buffer memory (315) may be required, which may be relatively large and may have a predominantly adaptive size.

[0044] The video decoder (210) may include an analyzer (320) for reconstructing symbols (321) from an entropy coded video sequence. The categories of these symbols include information used to control the operation of the video decoder (210) and possibly information to control a visualization device such as a display (212) that is not an integral part of the decoder but may be connected to it, as shown in FIG. .2. The control information for the rendering device(s) may be Supplementary Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The analyzer (320) may analyze/entropy decode the received encoded video sequence. Encoding of the encoded video sequence may be performed according to a video coding method or standard, and may follow principles well known to one skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The analyzer (320) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOP, Groups of Pictures), images, tiles, slices, macroblocks, coding units (CU, Coding Units), blocks (blocks), transformation units (TU, Transform Units), prediction units (PU , Prediction Units), etc. The entropy decoder/analyzer can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0045] The analyzer (320) may perform an entropy decoding/analysis operation of the video sequence received from the buffer (315) to create symbols (321).

[0046] Several different modules may be used for symbol reconstruction (321) depending on the type of encoded video image or parts thereof (eg, inter- and intra-image, inter- and intra-block) and other factors. Which modules are used, and how, may be determined by the subgroup management information extracted from the encoded video sequence by the analyzer (320). The flow of such subgroup management information between the analyzer (320) and the multiple modules is not shown hereinafter for simplicity.

[0047] In addition to the previously mentioned functional blocks, the decoder (210) can be fundamentally divided into several functional modules, as described below. In a practical implementation operating under commercial constraints, many of these modules interact closely with one another and can, at least partially, be nested within one another. However, for the purposes of describing the disclosed invention, a fundamental subdivision into the following functional modules is appropriate.

[0048] The first module is the scaling/inverse transform module (351). The scaling/inverse transform module (351) receives the quantized transform coefficient as well as control information including the transform used, block size, quantization factor, quantization scaling matrices, etc. as character(s) (321) from the analyzer (320). This module can output blocks containing sample values that can be entered into the aggregator (355).

[0049] In some cases, the output samples of the scaling / inverse transform block (351) may refer to an intra-coded block; that is, a block that does not use predicted information from previously reconstructed images, but may use predicted information from previously reconstructed portions of the current image. Such predicted information may be provided by an intra-picture prediction module (352). In some cases, the intra-image prediction module (352) generates a block of the same size and shape as the block to be reconstructed using previously reconstructed environment information extracted from the current (partially reconstructed) image (356). The aggregator (355) in some cases adds, for each sample, the prediction information generated by the intra prediction module (352) to the output sample information provided by the scaling/inverse transform module (351).

[0050] In other cases, the output samples of the scaler/inverse transform module (351) may refer to an inter-prediction encoded block, possibly with motion compensation. In such a case, the motion compensation prediction module (353) may access the reference picture memory (357) to retrieve samples used for prediction. After motion compensation has been applied to the extracted samples according to the symbols (321) associated with the block, these samples can be added by the aggregator (355) to the output of the scaler/demap module (in this case referred to as "residual samples" or "residual signal") to generate output sample information. The reference picture memory addresses from where the motion compensation prediction module retrieves the predicted samples may be governed by the motion vectors available to the motion compensation prediction module in the form of symbols (321), which may have X, Y and reference picture components, for example. Motion compensation may also include interpolation of sample values retrieved from reference picture memory when accurate subsample motion vectors, motion vector prediction mechanisms, and so on are used.

[0051] Various loop filtering techniques can be applied to the aggregator (355) output samples in the loop filter module (356). The video compression techniques may include deblocking filter techniques driven by parameters included in the encoded video bitstream and made available to the loop filter module (356) as symbols (321) from the analyzer (320), but may also respond to metainformation received during decoding of the previous (in the order of decoding) parts of the encoded image or encoded video sequence, as well as to respond to previously reconstructed and contour filtered sample values.

[0052] The loop filter module (356) may provide a stream of samples that may be fed to the renderer (212) as well as stored in the reference picture memory (356) for use in future inter-picture prediction.

[0053] Some encoded pictures, when fully reconstructed, can be used as reference pictures for future prediction. When the encoded picture is fully reconstructed and the encoded picture is identified as a reference picture (e.g., by the analyzer (320)), the current picture buffer (356) may become part of the reference picture buffer (357), and a fresh current picture buffer may be re-allocated before reconstruction of the next picture begins. encoded image.

[0054] Video decoder 320 may perform decoding operations in accordance with a predefined video data compression method, which may be documented in a standard such as ITU-T recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression method or standard used, in the sense that it adheres to the syntax of the video compression method or standard specified in a video compression technology document or standard, and in particular in the related documents of those standards. It may also be necessary for consistency that the complexity of the encoded video sequence remain within the boundaries set by the video compression technology level or standard. In some cases, the levels limit the maximum image size, the maximum frame rate, the maximum sampling rate for reconstruction (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the levels, in some cases, may be further limited by the specifications of the hypothetical reference decoder (HRD, Hypothetical Reference Decoder) and metadata for managing the HRD buffer signaled in the encoded video sequence.

[0055] According to embodiments, the receiver (310) may receive additional (redundant) data with encoded video. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (320) to correctly decode the data and/or more accurately reconstruct the original video data. Additional data may be provided in the form of, for example, temporal, spatial or SNR enhancement levels, redundant slices, redundant pictures, forward error correction codes, and so on.

[0056] FIG. 4 may be a functional block diagram of a video encoder (203) according to an embodiment of the present invention.

[0057] The encoder (203) may receive video samples from a video source (201) (which is not part of the encoder) that may capture the video image(s) to be encoded by the encoder (203).

[0058] The video source (201) may provide the source video sequence to be encoded by the encoder (203) in the form of a digital video sample stream of any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits, etc.), any color space (eg, BT.601 Y CrCB, RGB, etc.), and any suitable sampling structure (eg, Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (201) may be a storage device where previously prepared video is stored. In a videoconferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that give a sense of movement when viewed in sequence. The images themselves can be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc. used. It is not difficult for a person skilled in the art to understand the relationship between pixels and samples. In the following description, emphasis is placed on sampling.

[0059] According to an embodiment, the encoder (203) may encode and compress images of the source video sequence into the encoded video sequence (443) in real time or subject to any other time constraints imposed by the application. Setting the proper coding rate is one of the functions of the controller (450). The controller controls other function modules as described below and is operatively connected to these function modules. The connection is not shown for simplicity. The parameters set by the controller may include parameters related to frequency adjustment (picture skip, quantizer, lambda value applied in rate-distortion optimization, etc.), picture size, group of pictures scheme (GOP, Group Of Pictures), maximum motion vector search area, etc. A person skilled in the art can readily identify other functions of the controller (450) as they may relate to a video encoder (203) optimized for a particular system design.

[0060] Some video encoders operate in a mode known to those skilled in the art as "coding loop". As a very simplified description, an encoding loop may include an encoding component of an encoder (430) (hereinafter "source encoder") (responsible for creating symbols based on the input image to be encoded and the reference image(s)), and ( local) decoder (433) built into the encoder (203) that reconstructs the symbols to produce sample data that the (remote) decoder must also produce (because any compression between the symbols and the encoded video bitstream is lossless in the video compression technologies considered in the disclosed invention). This reconstructed sample stream enters the reference picture memory (434). Since decoding a stream of symbols produces bit-accurate results independent of the position of the decoder (local or remote), the contents of the reference picture buffer will also be the same bit-accuracy for the local encoder and the remote encoder. In other words, the predicted portion of the encoder "sees" as reference picture samples exactly the same sample values as the decoder would "see" if using prediction during decoding. This fundamental principle of reference picture synchronism (and eventually drift if synchronism cannot be maintained due to, for example, channel errors) is well known to those skilled in the art.

[0061] The "local" decoder (433) may operate in the same manner as the "remote" decoder (210) detailed above with reference to FIG. 3. However, again, referring to FIG. 3, since symbols are available and encoding/decoding of symbols in the encoded video sequence by the entropy encoder (445) and analyzer (320) can be lossless, the entropy decoding parts of the decoder (210), including the channel (312), the receiver (310), buffer (315) and parser (320) may not be fully implemented in the local decoder (433).

[0062] In doing so, it can be concluded that any decoding technology present in a decoder, with the exception of analysis/entropy decoding, must also be present, in essentially identical functional form, in the corresponding encoder. For this reason, the disclosed invention is focused on the operation of the decoder. The description of the encoding technologies can be abbreviated as they are the inverse of the decoding technologies described in detail. Only in some areas a more detailed description is required, which is given below.

[0063] In operation, the source encoder (430) may perform motion-compensated predictive coding, in which an input frame is predictively encoded based on one or more previously encoded frames from the video sequence, referred to as "reference frames". Thus, the encoding engine (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame(s) that can be selected as the predicted reference(s) to the input frame.

[0064] The local video decoder (433) may decode the encoded video data of the frames, which may be specified as reference frames, based on the symbols generated by the source encoder (430). The operations of the encoding engine (432) may be predominantly lossy processes. When the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (433) duplicates the decoding processes that may be performed by the video decoder on the reference frames and may cause the reconstructed reference frames to be stored in the reference picture cache (434). Thus, the encoder (203) can locally store copies of the reconstructed key frames having the same content as the reconstructed key frames that will be received by the video decoder at the receiver side (in the absence of transmission errors).

[0065] The predictor (435) may perform prediction searches for the encoding engine (432). Thus, for a new frame to be encoded, the predictor (435) can search the reference picture memory (434) for sample data (as candidates for the role of reference pixel blocks) or some metadata, for example, motion vectors of the reference picture, block shapes etc., which can serve as a proper reference for predicting new images. The predictor (435) may operate based on "sample blocks □ pixel blocks" to find proper references for prediction. In some cases, according to the search results obtained by the predictor (435), the input picture may have prediction references extracted from the plurality of reference pictures stored in the reference picture memory (434).

[0066] The controller (450) may control the encoding operations of the video encoder (430), including, for example, setting parameters and subgroup parameters used to encode video data.

[0067] The output of all of the above functional modules may be entropy encoded in an entropy encoder (445). The entropy encoder translates symbols generated by various functional modules into an encoded video sequence by lossless symbol compression according to techniques known to one skilled in the art, such as Huffman coding, variable run length coding, arithmetic coding, and so on.

[0068] The transmitter (440) may buffer the encoded video sequence(s) created by the entropy encoder (445) in preparation for transmission over a communication channel (460), which may be a hardware/software link to a storage device where encoded video data. The transmitter (440) may combine the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and/or auxiliary data streams (sources not shown).

[0069] The controller (450) may control the operation of the encoder (203). During encoding, the controller (450) may assign to each encoded picture one or another encoded picture type, which may determine the encoding methods applicable to the corresponding picture. For example, images can often be assigned the following frame types:

[0070] An intra-coded (intra-) picture (I-picture) may be a picture that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra-pictures, including, for example, pictures in the Independent Decoder Refresh (IDR) format. One skilled in the art will be aware of the 1-image varieties and their respective uses and features.

[0071] A predicted picture (P-picture) that can be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index to predict sample values of each block.

[0072] A bi-directionally predicted picture (B-picture) that can be encoded and decoded using intra prediction or inter prediction using no more than two motion vectors and reference indices to predict sample values of each block. Likewise, multi-predicted pictures may use more than two reference pictures and associated metadata to reconstruct a single block.

[0073] Source images typically allow for spatial division into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coding on a block-by-block basis. Blocks may be predictively encoded with reference to other (previously encoded) blocks determined by the coding assignment applied to the respective images of those blocks. For example, I-picture blocks may be encoded without prediction or predictively with reference to previously encoded blocks of the same picture (spatial prediction or intra prediction). Blocks of pixels of P-pictures may be encoded without prediction, using spatial prediction or temporal prediction with reference to one pre-coded reference picture. Blocks of B-pictures may be encoded without prediction by spatial prediction or temporal prediction with reference to one or two previously encoded reference pictures.

[0074] The video encoder (203) may perform encoding operations according to a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (203) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancies in the input video sequence. Therefore, the encoded video data may conform to the syntax given by the video coding technology or standard used.

[0075] According to an embodiment, the transmitter (440) may transmit additional data with encoded video. Video encoder (430) may include such data as part of an encoded video sequence. Supplementary data may include temporal/spatial/SNR enhancement levels, other types of redundant data such as redundant images and slices, Supplementary Enhancement Information (SEI) messages, visual information applicability parameter set (VUI, Visual Usability Information), etc.

[0076] Before a more detailed description of certain aspects of the disclosed invention, it is necessary to introduce several terms that will be referred to in the remainder of this description.

[0077] Further, a sub-image refers, in some cases, to a rectangular arrangement of samples, blocks, macroblocks, coding blocks, or the like, which are semantically grouped and which can be independently encoded with a changed resolution. One or more sub-images may form an image. One or more encoded sub-images may form an encoded image. One or more sub-images may be combined into an image, and one or more sub-images may be extracted from the image. In certain environments, one or more encoded sub-images may be combined in the compressed region without being re-coded to the sample level in the encoded picture, and in the same or in certain other cases, one or more encoded sub-images may be extracted from the encoded image in the compressed region.

[0078] Further, adaptive resolution change (ARC, Adaptive Resolution Change) refers to mechanisms that allow changing the resolution of an image or sub-image in an encoded video sequence, for example, by resampling a reference image. In the following, ARC parameters refer to control information required to perform adaptive resolution change, which may include, for example, filter parameters, scaling factors, output and/or reference picture resolutions, various control flags, and so on.

[0079] The description above is focused on encoding and decoding a single semantically independent encoded video image. Before describing the implementation of encoding/decoding of multiple sub-images with independent ARC parameters and the implied additional complication of the workflow, options for signaling ARC parameters should be described.

[0080] FIG. Figure 5 shows several new ARC parameter signaling options. It should be noted that each of the options has certain advantages and disadvantages in terms of coding efficiency, complexity and architecture. In a video coding standard or technology, one or more of these options, or options known in the art, may be selected to signal the ARC parameters. The parameters may not be mutually exclusive and are expected to be interchanged depending on application requirements, standards used, or choice of encoder.

[0081] ARC parameter classes may include:

[0082] - upsampling and/or downsampling coefficients, separate or combined in the X and Y dimensions;

[0083] - upsampling and/or downsampling coefficients with the addition of a temporal dimension indicating zooming in/out at a constant rate for a given number of images;

[0084] - any of the above two items may trigger the encoding of one or more supposedly short syntax elements that may point to a table containing coefficients);

[0085] is the resolution, in terms of X or Y, in units of samples, blocks, macroblocks, CUs, or any other suitable granularity of the input picture, output picture, reference picture, coded picture, merged or separate. If there is more than one resolution (eg, one for the input image and one for the reference image), in some cases one set of values may be derived from another set of values. This can be controlled, for example, using flags. See below for a more detailed example.

[0086] - "Warp" coordinates similar to those used in Annex P of the H.263 standard, again with the appropriate granularity as described above. Annex P of the H.263 standard defines one efficient way to encode such warp coordinates, but other potentially more efficient ways may also be developed. For example, the reverse Huffman-style variable-length encoding of Annex P warp coordinates may be replaced by a suitable length binary encoding, where the length of the binary codeword may, for example, be determined based on the maximum image size, possibly multiplied by a certain factor and offset by a certain value to account for "warping" beyond the limits of the maximum image size.

[0087] - Upsampling or downsampling filter parameters. In the simplest case, only one upsampling and/or downsampling filter can be used. However, in some cases it may be preferable to provide more flexibility in the design of the filter, this may require the signaling of the filter parameters. Such parameters may be selected by an index into a list of possible filter designs, a filter may be fully specified (e.g., by a list of filter coefficients using suitable entropy coding techniques), a filter may be implicitly selected by upsampling and/or downsampling relationships in which, in turn, are signaled according to any of the mechanisms mentioned above, and so on.

[0088] The following description assumes encoding a finite set of upsampling and/or downsampling coefficients (the same coefficient to be used in both the X dimension and the Y dimension) indicated by a codeword. For this codeword, variable length encoding is advantageously used, such as using exponential Golomb code common to certain syntax elements in video coding specifications such as H.264 and H.265. One suitable mapping of values to upsampling and/or downsampling coefficients may, for example, correspond to the following table:

[0089] Many similar mappings can be designed in accordance with the requirements of the application and the capabilities of the upsampling and downsampling mechanisms available in the video compression technology or standard. This table can contain more values. Values may also be represented by entropy encoding mechanisms other than Exponential Golomb codes, such as using binary encoding. This can have certain advantages when the oversampling factors are of interest outside of the video processing engines themselves (primarily the encoder and decoder), such as with MANE. It should be noted that for the (presumably) most common case where no resolution change is required, a short exponential Golomb code can be chosen; in the table above it is only one bit. This may have an advantage in coding efficiency over using binary codes for the most common case.

[0090] The number of entries in the table, as well as their semantics, can be fully or partially configurable. For example, the basic table structure may be conveyed in a "high" level parameter set, such as a sequence or a decoder parameter set. Alternatively, or additionally, one or more such tables may be defined in a video coding method or standard and may be selected, for example, by a decoder or sequence parameter set.

[0091] The following describes how the upsampling and/or downsampling ratio (ARC information) encoded as described above can be included in the syntax of a video coding method or standard. Similar considerations may apply to one or more codewords driving upsampling and/or downsampling filters. See below for a discussion of the case where a filter or other data structure requires relatively large amounts of data.

[0092] H.263 Annex P includes ARC information (502) in the form of four warp coordinates in the image header (501), specifically in the H.263 PLUSPTYPE header extension (503). This may be a reasonable design choice when a) there is an image header available and b) frequent changes to the ARC information are expected. However, the amount of data when using H.263-style signaling can be quite large, and the scaling factors may not refer to picture boundaries because the picture header may be temporary.

[0093] The JVCET-M135-vl cited above includes the ARC (index) reference information (505) found in the image parameter set (504), table indexing (506) including target resolutions, which in turn is within the set (507) of sequence parameters. Placing a possible resolution in a table (506) in a set (507) of sequence parameters can, according to the oral statements of the authors, be justified by using the SPS as a compatibility negotiation point during capability exchange. The resolution can be changed within the limits set by the values in the table (506) from image to image by referring to the corresponding set (504) of image parameters.

[0094] Referring to FIG. 5, the following additional options may exist for transmitting ARC information in the video bitstream. Each of these options has certain advantages over the prior art, as described above. These options may be simultaneously present in the same video coding method or standard.

[0095] According to an embodiment, ARC information (509) such as a resampling (scaling) factor may be represented in a slice header, a GOB header, a tile header, or a tile group header (hereinafter referred to as "tile group header) (508) . This may be sufficient for small size ARC information such as a single variable length codeword ue(v) or a fixed length codeword of several bits, such as shown above. The presence of ARC information in the header of a tile group is in itself an additional advantage, since the ARC information can be applied to one or another sub-image represented, for example, by this tile group, and not to the entire image. See also below. In addition, even if a video compression method or standard provides for adaptive resizing of only the entire image (as opposed to, for example, adaptive resizing based on a group of tiles), placing the ARC information in the tile group header instead of placing it in the H. 263 has certain advantages in terms of error tolerance.

[0096] According to the same or another embodiment, the ARC information (512) itself may be represented in a corresponding parameter set (511), such as an image parameter set, a header parameter set, a tile parameter set, an adaptation parameter set, etc. (a set of adaptation parameters is shown). The volume of this set of parameters cannot be larger than the image, for example, a group of tiles. The use of ARC information in an implicit form is carried out by activating the corresponding set of parameters. For example, when only image-based ARC is considered in a video coding method or standard, an image parameter set or equivalent may be used.

[0097] According to the same or another embodiment, the ARC reference information (513) may be represented in a tile group header (514) or a similar data structure. This reference information (513) may refer to a subset of the ARC information (515) available in a parameter set (516) with a scope larger than a single picture, such as a sequence parameter set or a decoder parameter set.

[0098] The additional level of indirection implying the activation of PPS from the tile group header, PPS, SPS, as used in JVET-M0135-vl, is unnecessary, since image parameter sets, like sequence parameter sets, can (and are available in some standards such as RFC3984) be used to negotiate capabilities or notifications. However, if the ARC information is to be applied to a sub-image represented, for example, also by tile groups, then a parameter set with an activation area limited by the tile group, such as an adaptation parameter set or a header parameter set, may be the best choice. Also, if the ARC information is of very small size - for example, contains filter management information such as different filter coefficients - then the parameter may be a better choice than using the (508) header directly from a coding gain point of view, since these settings can be reused. used in future images or subimages by referring to the same set of parameters.

[0099] When using a sequence parameter set or other higher level parameter set with a scope spanning multiple images, certain considerations may apply:

[0100] 1. The parameter set for storing the ARC information table (516) may be a sequence parameter set in some cases, but preferably a decoder parameter set in other cases. A decoder parameter set may have an activation area of several CVSs, namely the encoded video stream - that is, all bits of the encoded video from start to session interruption. This amount may be more appropriate because the possible ARC coefficients may be a function of the decoder, possibly implemented in hardware, and the hardware functions are generally not changed when encoding a video sequence (which, at least in some entertainment systems, is a group of pictures of duration one second or less). However, the inclusion of a table in a sequence parameter set is explicitly included in the placement options described here, specifically in conjunction with item 2 below.

[0101] 2. The ARC reference information (513) may preferably be placed directly in the slice/GOB/rpynnbi image/tile header (514) of tiles (hereinafter "tile group header") rather than in the image parameter set as in JVCET- M0135-vl. The reason is that when an encoder needs to change one value in a set of image parameters, such as ARC reference information, it must create a new PPS and refer to this new PPS. Assume that only the ARC reference information is changed and other information, such as the quantization matrix information in PPS, is kept. Such information may be of considerable size and will need to be retransmitted to form a new PPS. Since a single codeword, such as an index in a table (513), may be used as the ARC reference information, and may be the only changeable value, it would be redundant and uneconomical to retransmit all, such as the quantization matrix information. With this in mind, from a coding efficiency point of view, it may be much better to avoid indirection via PPS, as proposed in JVET-M0135-vl. In the same way, inputting the ARC reference information in the PPS has the additional disadvantage that the ARC information pointed to by the ARC reference information (513) must necessarily be applied to the entire picture, and not to a particular sub-picture, because the set of parameters image is activated for the entire image.

[0102] According to the same or another embodiment, ARC parameter signaling may be performed according to a detailed example as shown in FIG. 6. In FIG. 6 shows syntax diagrams in the representation used in video coding standards since at least 1993. The designations of such syntax diagrams roughly correspond to C-style programming. "Bold" lines indicate syntax elements in the bitstream, while "non-bold" lines generally indicate control flow or variable setting.

[0103] A tile group header (601) as an example of a header syntax structure applied to a (possibly rectangular) portion of an image may, under certain conditions, contain an exponential Golomb encoded variable length syntax element dec_pic_size_idx (602) (shown in bold) . The presence of this syntax element in a tilegroup header can be limited by using adaptive resolution (603) - here the flag value is not in bold, which means that this flag is present in the bitstream at the point where it occurs in the syntax diagram. Information about whether adaptive resolution is used for this image or parts of it can be conveyed in any high-level syntactic structure inside or outside the bitstream. In the example shown, this is passed in a sequence parameter set, as shown below.

[0104] In FIG. 6 also shows a selection from the set (610) of sequence parameters. The first syntax element shown is adaptive_pic_resolution_change_flag (611). This flag may indicate the use of an adaptive resolution, which in turn may require certain control information. In the example, the indication of such control information is determined by the value of this flag based on the if () [if ()] statement in the parameter set (612) and in the tile group header (601).

[0105] When adaptive resolution is used, in this example, the output resolution is encoded in units of samples (613). The numeric value (613) refers to the output_pic_width_in_luma_samples syntax element, and the output_pic_height_in_luma_samples syntax element, which together can define the resolution of the output image. Elsewhere in a video coding method or standard, certain restrictions may be defined for any value. For example, the level definition may limit the number of total output samples that can be the product of the values of these two syntax elements. In addition, certain video coding methods or standards, or external methods or standards such as, for example, system standards, may restrict the numbering range (for example, one or both dimensions must be divisible by a power of 2) or image format (for example, width and height must have a ratio of 4:3 or 16:9). Such restrictions may be introduced to facilitate hardware implementations, or for other reasons that will be understood by a person skilled in the art.

[0106] In some applications, it may be desirable for the encoder to instruct the decoder to use a specific reference picture size, rather than assuming that this size is the size of the output picture. In this example, the reference_pic_size_presentflag (614) syntax element disables the conditional presence of the reference picture sizes (615) (again, the numeric value refers to both width and height).

[0107] Finally, a table of possible image width and height for decoding is shown. Such a table may be expressed, for example, by specifying a table (num_dec_pic_size_in_luma_samples_minus1) (616). The "minus1" operator can refer to the interpretation of the meaning of this syntax element. For example, if the encoded value is zero, then there is one entry in the table. If the value is five, then there are six entries in the table. For each "row" in the table, the width and height of the decoded picture are included in the syntax (617).

[0108] Represented entries (617) in the table can be indexed using the dec_pic_size_idx (602) syntax element in the tile group header, which allows for different decoded sizes - essentially scaling factors - for each tile group.

[0109] Certain video coding techniques or standards, such as VP9, support spatial scalability by implementing certain forms of reference picture resampling (implemented quite differently from the disclosed invention) in combination with temporal scalability to provide spatial scalability. In particular, some reference pictures may be upsampled using higher resolution ARC-style techniques to form the basis of the spatial enhancement layer. These upsampled images can be enhanced using conventional high resolution prediction engines to add detail.

[IT] The disclosed invention can be used in such an environment. In some cases, in the same or in another embodiment, a value in the NAL unit header, such as a temporal identifier field, may be used to indicate not only the temporal but also the spatial level. This provides certain advantages for certain systems; for example, existing Selected Forwarding Units (SFUs) created and optimized for forwarding selected at the temporal level based on the NAL Header Temporal Identifier value can be used without modification for scalable environments. To make this possible, there may be a requirement for a match between the encoded picture size and the temporal level indicated by the temporal identifier field in the NAL unit header.

[0111] In some video coding techniques, an Access Unit (AU) may refer to the encoded image(s), slice(s), tile(s), NAL unit(s), etc. that have been captured and inserted into the corresponding bitstream of images/slices/tiles/NAL units at the specified point in time. Such point in time can be composition time.

[0112] According to HEVC and some other video coding techniques, a Picture Order Count (POC) value can be used to indicate a selected reference picture from a plurality of reference pictures stored in a Decoded Picture Buffer (DPB). When an access unit (AU, Access Unit) contains one or more images, slices or tiles, each image, slice or tile belonging to the same AU can carry the same POC value, from which it can be concluded that they were created from content with the same composition time. In other words, in a scenario where two images/slices/tiles carry the same specified POC value, this may indicate that the two images/slices/tiles belong to the same AU and have the same time. compositions. Conversely, two images/slices/tiles with different POC values may indicate images/slices/tiles belonging to different AUs and having different composition times.

[0113] According to an embodiment of the disclosed invention, the aforementioned rigid relationship can be mitigated because the access block can contain pictures, slices, and tiles with different POC values. By allowing different POC values within the same AU, it becomes possible to use the POC value to identify potentially independently decodable pictures. /slices/tiles with identical presentation times. This, in turn, can support multiple scalable levels without changing the reference picture selection signaling (eg, reference picture set signaling or reference picture list signaling), as described in more detail below.

[0114] However, it is still desirable to be able to identify the AU to which an image/slice/tile belongs relative to other images/slices/tiles with a different POC value based on the POC value alone. This can be achieved as described.

[0115] According to the same or other embodiments, an access unit number (AUC, Access Unit Count) may be conveyed in a high-level syntax structure such as a NAL unit header, slice header, tile group header, SEI message, parameter set, or AU delimiter. The AUC value may be used to identify which NAL units, pictures, slices, or tiles belong to a given AU. The AUC value may correspond to a specific point in time of the composition. The AUC value can be a multiple of the POC value. The AUC value can be calculated by dividing the POC value by an integer. In some cases, division operations can create some overhead in decoder implementations. In such cases, slight restrictions on the numbering space of the AUC values may allow shift operations to replace division. For example, the AUC value may be equal to the value of the most significant bit (MSB, Most Significant Bit) from the range of POC values.

[0116] According to the same embodiment, a POC value for an AU (poc_cycle_au) may be conveyed in a high-level syntax structure such as a NAL unit header, slice header, tilegroup header, SEI message, parameter set, or AU delimiter. The poc_cycle_au value may indicate how many different and consecutive POC values may be associated with the same AU. For example, if the value of poc_cycle_au is 4, then images, slices, or tiles with a POC value of 0-3 inclusive are associated with an AU with an AUC value of 0, and images, slices, or tiles with a POC value between 4 and 7, inclusive, are associated with AU with an AUC value of 1. Thus, the AUC value can be obtained by dividing the ROS value by the value of loc_cycle_au.

[0117] According to the same or another embodiment, the value of po_cyle_au can be derived from information found in, for example, a Video Parameter Set (VPS) that specifies the number of spatial or SNR levels in a coded video sequence. Such a possible relationship is briefly described below. Although the derivation as described above may save a few bits in the VPS and therefore may improve coding efficiency, it may be advantageous to explicitly encode pac_cycle_au in an appropriate high-level syntax structure hierarchically below the video parameter set to be able to minimize pac_cycle_au for a specified small parts of a bitstream, such as an image. This optimization can save more bits than can be saved by the above process because POC values (and/or POC implicitly related syntax element values) can be encoded in low level syntax structures.

в VPS (или в SPS), который указывает рос_cycle_au, используемый для всех изображений/слайсов в кодированной видеопоследовательности, и элемент синтаксиса

который указывает

текущего слайса в заголовке слайса. Если значение РОС увеличивается одинаковым образом для каждой AU, то для

в VPS устанавливается значение 1, и в VPS передается

В этом случае

в явном виде не указывается, и значение AUC для каждого AU вычисляется путем деления значения РОС на

Если значение РОС не увеличивается одинаковым образом для каждой AU, то для

в VPS устанавливается значение 0. В этом случае

не передается, а

передается в заголовке слайса для каждого слайса или изображения. У каждого слайса или изображения может быть свое значение

Значение AUC для каждого AU вычисляется путем деления значения РОС на

На фиг. 10 показана блок-схема, иллюстрирующая соответствующий рабочий процесс.[0118] According to the same or another embodiment, in FIG. 9 shows an example of syntax tables for passing a syntax element

in the VPS (or in the SPS) which specifies the growth_cycle_au used for all images/slices in the encoded video sequence and the syntax element

which indicates

current slice in the slice header. If the POC value increases in the same way for each AU, then for

the VPS is set to 1, and the VPS is sent

In this case

not explicitly stated and the AUC value for each AU is calculated by dividing the POC value by

If the POC value does not increase in the same way for each AU, then for

VPS is set to 0. In this case

not transmitted, but

passed in the slice header for each slice or image. Each slice or image can have its own meaning

The AUC value for each AU is calculated by dividing the POC value by

In FIG. 10 is a flowchart illustrating the corresponding workflow.

[0119] According to the same or other embodiments, even though the POC value of an image, slice, or tile may be different, an image, slice, or tile corresponding to an AU with the same AUC value may be associated with the same decoding or output time. Thus, in the absence of any analysis/decoding dependency between images, slices or tiles in the same AU, all or a subset of the images, slices or tiles associated with the same AU can be decoded simultaneously and can be output at the same point in time.

[0120] According to the same or other embodiments, even though the POC value of an image, slice, or tile may be different, an image, slice, or tile corresponding to an AU with the same AUC value may be associated with the same composition/display time. When the composition time is contained in the container format, even if the images correspond to different AUs, if the images have the same composition time, the images may be displayed at the same point in time.

[0121] According to the same or other embodiments, each image, slice, or tile may have the same temporal identifier (temporal_id) in the same AU. All or a subset of the images, slices or tiles corresponding to a particular point in time may be associated with the same temporal sublayer. According to the same or other embodiments, each image, slice or tile may have the same or different spatial layer identifiers (layer_id) in the same AU. All or a subset of the images, slices or tiles corresponding to a particular point in time may be associated with the same or different spatial levels.

и

или 1, а изображение, слайс или тайл во втором AU с

и

или 1 соответственно. Значение РОС увеличивается на единицу для каждого изображения независимо от значений

В этом примере значение

может быть равно 2. Предпочтительнее, чтобы в качестве значения

было установлено число уровней (пространственной масштабируемости). Таким образом, в этом примере значение РОС увеличивается на 2, а значение AUC увеличивается на 1.[0122] In FIG. 8 shows an example of a video sequence structure with a combination of temporal_id, layer_id, POC, and AUC values with adaptive resolution change. In this example, an image, slice or tile in the first AU with AUC=0 might have

and

or 1, and the image, slice, or tile in the second AU with

and

or 1 respectively. The ROS value increases by one for each image, regardless of the values

In this example, the value

can be equal to 2. Preferably, as a value

the number of levels (spatial scalability) has been established. Thus, in this example, the ROS value is increased by 2 and the AUC value is increased by 1.

[0123] According to the above embodiments, all or a subset of the structure of inter-picture prediction or inter-layer prediction and reference picture indication can be supported by using existing reference picture set (RPS) signaling in HEVC or reference picture list signaling ( RPL, Reference Picture List). In RPS or RPL, the selected reference picture is indicated by signaling a POC value or by changing the POC value between the current picture and the selected reference picture. In the disclosed invention, RPS and RPL can be used to indicate an inter-picture prediction or inter-layer prediction structure without signaling change, but with the following limitations. If the temporal id value of the reference picture is greater than the temporal_id value of the current picture, then the current picture may not use the reference picture for motion compensation or other predictions. If the layer_id value of the reference picture is greater than the layer_id value of the current picture, then the current picture may not use the reference picture for motion compensation or other predictions.

[0124] According to the same or other embodiments, motion vector scaling based on POC difference for temporal motion vector prediction may be disabled for multiple pictures in an access block. Thus, although each picture may have a different access block POC value, the motion vector is not scaled and is used for temporal prediction of the access block motion vector. This is because a reference picture with a different ROS in the same AU is considered a reference picture with the same point in time. Thus, according to an embodiment, the motion vector scaling function may return a value of 1 when the reference picture refers to an AU associated with the current picture.

[0125] According to the same or other embodiments, motion vector scaling based on the POC difference for temporal motion vector prediction can be optionally disabled for multiple images when the spatial resolution of the reference image is different from the spatial resolution of the current image. When motion vector scaling is enabled, the motion vector is scaled based on both the POC difference and the spatial resolution ratio between the current picture and the reference picture.

имеет неоднородное значение (когда

В противном случае (когда

масштабирование вектора движения на основе разности AUC может быть идентично масштабированию вектора движения на основе разности РОС.[0126] According to the same or another embodiment, the motion vector may be scaled based on the AUC difference instead of the POC difference for temporal motion vector prediction, especially when

has a non-uniform value (when

Otherwise (when

the motion vector scaling based on the AUC difference may be identical to the motion vector scaling based on the POC difference.

[0127] According to the same or another embodiment, when the motion vector is scaled based on the AUC difference, the reference motion vector in the same AU (with this AUC value) with the current picture is not scaled based on the AUC difference and is used for motion vector prediction without scaling or with scaling based on the ratio of spatial resolution between the current picture and the reference picture.

[0128] According to the same or other embodiments, the AUC value is used to identify the AU boundary and is used for Hypothetical Reference Decoder (HRD) operation that requires input and output synchronization with AU granularity. In most cases, the decoded picture with the highest level in the AU can be output for display. The AUC value and the layer_id value may be used to identify the output image.

[0129] According to an embodiment, an image may consist of one or more sub-images. Each subimage may cover a local area or an entire area of the image. The area supported by one sub-image may or may not overlap with the area supported by another sub-image. An area composed of one or more sub-images may or may not cover the entire image area. If the image consists of one sub-image, then the area supported by this sub-image is identical to the area supported by the entire image.

[0130] According to the same embodiment, the sub-image may be encoded by a coding method similar to the coding method used for the encoded image. A sub-picture may be encoded independently or dependent on another sub-picture or encoded picture. A sub-image may be analyzed independently or in dependence on another sub-image or encoded image.

[0131] According to the same embodiment, the encoded sub-image may be contained in one or more layers. Encoded sub-images in different layers may have different spatial resolutions. The original sub-image can be spatially resampled (up or down), encoded with different spatial resolution parameters, and it can also be contained in a bit stream corresponding to one or another level.

[0132] According to the same or another embodiment, a sub-image with (W, H, where W indicates the width of the sub-image and H indicates the height of the sub-image, respectively) may be encoded and contained in the encoded bitstream corresponding to layer 0, and the sub-image with upsampling (or downsampling) from the sub-image with the original spatial resolution, with (W*S _w,k , H* S _h,k ), can be encoded and contained in the encoded bitstream corresponding to layer k, where S _w,k , S _h,k indicate the horizontal and vertical resampling factors. If the values of S _w,k , S _h,k are greater than one, then upsampling is performed. If the values of S _w,k , S _h,k are less than one, then downsampling is performed.

[0133] According to the same or another embodiment, an encoded sub-image in one layer may have a different visual quality than the quality of an encoded sub-image in another layer in the same sub-image or in a different sub-image. For example, sub-image i at level n is encoded using a quantization parameter Q _i,n , and sub-image j at level m is encoded using a quantization parameter O _j,m .

[0134] According to the same or another embodiment, an encoded sub-image in one layer or another can be decoded independently, without any dependence on the parsing or decoding of the encoded sub-image in another layer in the same local area. A sub-image layer that can be independently decoded without reference to another sub-image layer in the same local area is an independent sub-image layer. An encoded sub-image in an independent sub-image layer may or may not have a decoding or analysis dependency on a previously encoded sub-image in that same sub-image layer, but an encoded sub-image may not have any dependency on a coded image in another sub-image layer.

[0135] According to the same or another embodiment, an encoded sub-image in one layer or another can be decoded in a dependent manner, depending on the parsing or decoding of the encoded sub-image in another layer in the same local area. A sub-image layer that can be decoded in a dependent manner with reference to another sub-image layer in the same local area is a dependent sub-image layer. An encoded sub-image in a dependent sub-image may refer to an encoded sub-image related to the same sub-image, to a previously encoded sub-image in the same sub-image layer, or to both reference sub-images.

[0136] According to the same or another embodiment, an encoded sub-image consists of one or more independent sub-image layers and one or more dependent sub-image layers. However, at least one independent sub-picture layer may be represented for the encoded sub-picture. An independent sub-image layer may have a layer identifier (layer_id) value, which may be represented in a NAL unit header or other high-level syntax structure, equal to 0. A sub-image layer with layer_id equal to 0 is a base sub-image layer.

[0137] According to the same or another embodiment, an image may consist of one or more foreground sub-images and one background sub-image. The area supported by the background sub-image may be equal to the image area. The area supported by the foreground sub-image may overlap with the area supported by the background sub-image. The background sub-image may be the base sub-image level, and the foreground sub-image may be the non-base (enhanced) sub-image level. One or more non-base sub-image layers may refer to the same base layer for decoding. Each non-base subimage layer with layer_id equal to a may refer to a non-base subimage layer with layer_id equal to b, where a is greater than b.

[0138] According to the same or another embodiment, the image may consist of one or more foreground sub-images with or without a background sub-image. Each subimage may have its own baseline subimage level and one or more non-baseline (enhanced) levels. Each base sub-image level may be referenced by one or more non-base sub-image levels. Each non-base subimage layer with layer_id equal to a may refer to a non-base subimage layer with layer_id equal to b, where a is greater than b.

[0139] According to the same or another embodiment, the image may consist of one or more foreground sub-images with or without a background sub-image. Each encoded sub-image at the (base or non-base) sub-image level may be referenced by one or more non-base-level sub-images that belong to the same sub-image and one or more non-base-level sub-images that do not belong to the same sub-image.

[0140] According to the same or another embodiment, the image may consist of one or more foreground sub-images with or without a background sub-image. The sub-image in level a may be further divided into multiple sub-images in one level. One or more encoded sub-images in layer b may refer to a split sub-image in layer a.

[0141] According to the same or another embodiment, a coded video sequence (CVS, Coded Video Sequence) may be a group of coded pictures. The CVS may consist of one or more Coded Sub-Picture Sequences (CSPS), where the CSPS may be a group of coded sub-pictures covering the same local area of an image. The CSPS may have the same or different temporal resolution as the encoded video sequence.

[0142] According to the same or another embodiment, the CSPS may be encoded and contained in one or more layers. The CSPS may consist of one or more levels of CSPS. Decoding one or more CSPS levels corresponding to the CSPS may recover a sequence of sub-images corresponding to the same local area.

[0143] According to the same or another embodiment, the number of CSPS levels corresponding to one CSPS may be identical or different from the number of CSPS levels corresponding to another CSPS.

[0144] According to the same or another embodiment, a CSPS layer may have a different temporal resolution (eg, frame rate) than another CSPS layer. The original (uncompressed) sequence of sub-images can be temporally resampled (up or down), encoded with different temporal resolution parameters, and it can also be contained in a bitstream corresponding to one or another level.

[0145] According to the same or another embodiment, the sub-image sequence with frame rate F may be encoded and contained in the encoded bitstream corresponding to level 0, and the time-upsampled (or down-sampled) sub-image sequence from the original sub-image sequence with F* S _t,k may be encoded and contained in the encoded bitstream corresponding to layer k, where S _t,k indicates the temporal sampling factor for layer k. If the value of S _t,k is greater than one, then the temporal resampling process is identical to frame rate upconversion . If the value of S _t,k is less than one, then the temporal resampling process is identical to frame rate downconversion.

[0146] According to the same or another embodiment, when the sub-image with CSPS layer a is a reference to the sub-image with CSPS layer b for motion compensation or any inter-layer prediction, if the spatial resolution of CSPS layer a is different from the spatial resolution of CSPS layer b , the decoded pixels at CSPS layer a are resampled and used for reference. The resampling process may require upsampling or downsampling filtering.

[0147] FIG. 11 shows an example of a video stream containing a background video CSPS with layer_id equal to 0 and multiple foreground CSPS layers. Although an encoded sub-image may be composed of one or more CSPS layers, a background region that does not belong to any of the foreground CSPS may be composed of a base layer. The base layer may contain a background region and foreground regions, and the CSPS enhancement layer contains a foreground region. The CSPS enhancement layer may have better visual quality than the base layer in the same area. The CSPS enhancement layer may refer to reconstructed pixels and base layer motion vectors corresponding to the same area.

[0148] According to the same or another embodiment, the video bitstream corresponding to the base layer is contained in some track, and the CSPS levels corresponding to each sub-image are contained in a separate track in the video file.

[0149] According to the same or another embodiment, the video bitstream corresponding to the base layer is contained in some track, and the CSPS layers with the same layer_id are contained in a separate track in the video file. In this example, the track corresponding to level k contains only the CSPS levels corresponding to level k.

[0150] According to the same or another embodiment, each CSPS level of each sub-image is stored in a separate track. Each track may or may not have an analysis or decoding dependency on one or more other tracks.

[0151] According to the same or another embodiment, each track may contain bitstreams corresponding to the CSPS level (from level i to level j) of all or a subset of sub-images, where 0<i=<j=<k and k is the highest CSPS level.

[0152] According to the same or another embodiment, an image consists of one or more associated blocks of media data, including a depth map, an alpha map, 3D geometry data, an occupancy map, and so on. Such associated synchronized media data may be divided into one or more sub-data streams, each corresponding to one sub-image.

[0153] According to the same or another embodiment, FIG. 12 shows an example of a video conference based on a layered sub-image generation method. The video stream contains one base layer video bitstream corresponding to the background image and one or more enhancement layer video bitstreams corresponding to the foreground sub-images. Each enhancement layer video bitstream corresponds to a CSPS layer. The default display shows the image corresponding to the baseline. It contains one or more picture-in-picture (PIP) user pictures. When a particular user is selected at the command of the client, the CSPS enhancement level corresponding to the selected user is decoded and displayed with enhanced quality or spatial resolution. In FIG. 13 shows a diagram of the workflow.

[0154] According to the same or another embodiment, an intermediate network unit (eg, a router) may select a subset of layers for transmitting data to a user depending on its bandwidth. Organization of information within pictures/sub-pictures can be used for bandwidth adaptation. For example, if the user has no bandwidth, then the router splits the layers or selects a portion of the sub-images because of their importance or based on the configuration used; this can be done dynamically to adapt to the bandwidth.

[0155] In FIG. 14 shows an example of using 360 video. When a 360 degree spherical image is projected onto a flat image, the 360 degree projection image can be divided into several sub-images as a base layer. The enhancement level of a particular sub-image may be encoded and transmitted to the client. The decoder may decode both the base layer containing all sub-pictures and the enhancement layer of the selected sub-picture. When the current viewport is identical to the selected sub-picture, the displayed image can be of higher quality when using the decoded sub-picture by applying the enhancement layer. Otherwise, a decoded image with a low quality base layer may be displayed.

[0156] According to the same or another embodiment, any information about the structure to be displayed can be provided in the file as additional information (eg, in the form of SEI messages or metadata). One or more decoded sub-pictures may be moved and displayed depending on the transmitted structure information. The structure information may be transmitted by a streaming server or translator, or may be retrieved by an appropriate network entity or cloud server, or may be determined by a user's individual setting.

[0157] According to an embodiment, when an input image is divided into one or more (rectangular) sub-regions, each sub-region may be encoded as an independent layer. Each independent layer corresponding to a local area can have a unique layer_id value. For each independent layer, information about the size and location of the sub-image can be transmitted. For example, image size (width, height), information about the offset of the left-upper corner (x_offset, y_offset). In FIG. 15 shows an example of a split sub-image structure, size and location information, and a corresponding image prediction structure. Structure information, including the size(s) of the sub-image(s) and the position(s) of the sub-image(s), may be conveyed as a high-level syntactic structure such as parameter set(s), a slice or tile group header, or an SEI message. .

[0158] According to the same embodiment, each sub-image corresponding to the independent layer may have its own unique POC value within the AU. When any reference picture among the pictures stored in the DPB is specified using the syntax element(s) in the RPS or RPL structure, the POC value(s) of each sub-picture corresponding to the particular layer may be used.

[0159] According to the same or another embodiment, the layer_id may not be used to indicate the structure of the (inter-layer) prediction, but a POC (delta) value may be used.

[0160] According to the same embodiment, a sub-image with a POC value of N corresponding to a certain level (or local area) may or may not be used as a reference image of a sub-image with a POC value of N+K corresponding to the same level ( or the same local area) for motion-compensated prediction. In most cases, the value of K may be equal to the maximum number of (independent) levels, which may be identical to the number of subregions.

[0161] According to the same or another embodiment, in FIG. 16 shows an expanded version of the example of FIG. 15. When the input image is divided into multiple (eg, four) sub-regions, each local region may be encoded with one or more levels. In this case, the number of independent levels may be equal to the number of sub-regions, and one or more levels may correspond to one sub-region. Thus, each sub-region may be encoded with one or more independent layers and no or more dependent layers.

[0162] According to the same embodiment, in FIG. 16, the input image can be divided into four sub-regions. The right-upper sub-region may be coded as two layers, layer 1 and layer 4, and the right-bottom sub-region may be coded as two other layers, layer 3 and layer 5. In this case, layer 4 may refer to layer 1 for compensation prediction. motion, and layer 5 can refer to layer 3 for motion compensation.

[0163] According to the same or another embodiment, in-loop filtering (e.g., deblocking filtering, adaptive in-loop filtering, shape recovery, two-way filtering, or any deep learning-based filtering) across the layer boundary can be (optionally) disabled.

[0164] According to the same or another embodiment, motion-compensated prediction or intra-block copying across a layer boundary can be (optionally) disabled.

[0165] According to the same or another embodiment, padding at boundaries for motion-compensated prediction or in-loop filtering at a sub-image boundary can be optionally processed. A flag indicating whether padding at boundaries is handled or not may be conveyed in a high-level syntax structure such as parameter set(s) (VPS, SPS, PPS, or APS), a slice or tile group header, or an SEI message.

передается в VPS. Этот флаг может указывать, разделено ли входное(ые) изображение(я) на несколько субобластей или нет.Когда значение

равно 0, входное изображение(я) в кодированной(ых) видеопоследовательности(ях), соответствующее текущему VPS, не может быть разделено на несколько субобластей. В этом случае размер входного изображения может быть равен размеру кодированного изображения

который передается в SPS. Когда значение

равно 1, входное изображение(я) в кодированной(ых) видеопоследовательности(ях), соответствующее текущему VPS, не может быть разделено на несколько субобластей. В этом случае элементы синтаксиса

и

передаются в VPS. Значения

и

могут быть равны ширине и высоте входного изображения(й) соответственно.[0166] According to the same or another embodiment, information about the structure of the sub-area(s) (or sub-image(s)) may be transmitted to the VPS or SPS. 17 shows an example of syntax elements in VPS and SPS. In this example

transferred to the VPS. This flag may indicate whether the input image(s) is split into multiple sub-regions or not. When the value

is 0, the input image(s) in the encoded video sequence(s) corresponding to the current VPS cannot be divided into multiple sub-regions. In this case, the size of the input image can be equal to the size of the encoded image

which is transmitted to the SPS. When the value

is 1, the input image(s) in the encoded video sequence(s) corresponding to the current VPS cannot be divided into multiple sub-regions. In this case, the syntax elements

and

transferred to the VPS. Values

and

may be equal to the width and height of the input image(s), respectively.

и

могут не использоваться для декодирования, но могут использоваться для композиции и отображения.[0167] According to the same embodiment, the values

and

may not be used for decoding, but may be used for composition and display.

равно 1, элементы синтаксиса

могут передаваться в SPS, который соответствует конкретному(ым) уровню(ям). В этом случае размер кодированного изображения

переданный в SPS, может быть равен ширине и высоте субобласти, соответствующей конкретному уровню. Кроме того, позиция

левого-верхнего угла субобласти может передаваться в SPS.[0168] According to the same embodiment, when the value

equals 1, syntax elements

may be transmitted in the SPS that corresponds to the specific layer(s). In this case, the size of the encoded image

transmitted in the SPS may be equal to the width and height of the sub-region corresponding to the particular layer. In addition, the position

the left-upper corner of the sub-area may be transmitted to the SPS.

левого-верхнего угла субобласти может не использоваться для декодирования, но может использоваться для композиции и отображения.[0169] According to the same embodiment, position information

the top-left corner of the sub-region may not be used for decoding, but may be used for composition and display.

указывает число (прямоугольных) субобластей в текущей кодированной видеопоследовательности, а элемент синтаксиса

указывает число уровней в текущей кодированной видеопоследовательности. Значение numlayers может быть равно или больше значения

Когда какая-либо субобласть кодируется как один уровень, значение num_layers может быть равно значению

Когда одна или более субобластей кодируются как несколько уровней, значение num_layers может быть больше значения

Элемент синтаксиса

указывает зависимость от j-го уровня до i-го уровня,

указывает число уровней, ассоциированных с i-ой субобластью.

указывает layer_id j-го уровня, ассоциированного с i-ой субобластью.

и

указывают горизонтальное и вертикальное положение левого-верхнего угла i-ой субобласти соответственно.

и

указывают ширину и высоту i-ой субобласти, соответственно.[0170] According to the same or another embodiment, information about the structure (size and position) of all or a subset of sub-areas of the input(s) image(s), information about the relationship between levels can be transmitted in a parameter set or in an SEI message. In FIG. 18 shows an example of syntax elements for specifying information about the structure of subdomains, dependencies between levels, and relationships between a subdomain and one or more levels. In this example, the syntax element

indicates the number of (rectangular) subregions in the current encoded video sequence, and the syntax element

indicates the number of levels in the current encoded video sequence. The numlayers value can be equal to or greater than the value

When any sub-region is encoded as one layer, the value of num_layers may be equal to the value

When one or more subregions are encoded as multiple layers, the value of num_layers may be greater than the value

Syntax element

indicates dependence from j-th level to i-th level,

indicates the number of levels associated with the i-th sub-region.

indicates the layer_id of the j-th layer associated with the i-th sub-region.

and

indicate the horizontal and vertical position of the left-upper corner of the i-th sub-region, respectively.

and

indicate the width and height of the i-th subregion, respectively.

указывающий число наборов выходных уровней (OLS) в кодированной видеопоследовательности, ссылающейся на VPS, может передаваться в VPS. Для каждого набора выходных уровней элемент синтаксиса

может передаваться столько раз, сколько имеется выходных уровней.[0171] According to one embodiment, one or more syntax elements that define a set of output levels to specify one or more levels to be output with or without profile/tier/level information may be conveyed in a high-level syntax structure, e.g. in VPS, DPS, SPS, PPS, APS, or in an SEI message. In FIG. 19 syntax element

indicating the number of output level sets (OLS) in the encoded video sequence referring to the VPS may be transmitted to the VPS. For each set of output levels, the syntax element

can be transmitted as many times as there are output levels.

равный 1, указывает, что выводится i-ый уровень.

равный 0, указывает, что i-ый уровень не выводится.[0172] According to the same embodiment

equal to 1 indicates that the i-th level is output.

equal to 0 indicates that the i-th level is not output.

указывающий объем информации о профилях/ярусах/уровнях для OLS в кодированной видеопоследовательности, ссылающейся на VPS, может передаваться в VPS. Для каждого набора выходных уровней набор элементов синтаксиса для информации о профилях/ярусах/уровнях или индекс, указывающий конкретную информации о профилях/ярусах/уровнях среди записей в информации о профилях/ярусах/уровнях, может передаваться столько раз, сколько имеется выходных уровней.[0173] According to the same or another embodiment, one or more syntax elements that define profile/tier/layer information for each set of output layers may be conveyed in a high-level syntax structure, such as in a VPS, DPS, SPS, PPS, APS or in the SEI message. In FIG. 19 syntax element

indicating the amount of profile/tier/layer information for the OLS in the encoded video sequence referring to the VPS may be transmitted to the VPS. For each set of output levels, a set of syntax elements for profile/tier/layer information or an index indicating specific profile/tier/layer information among entries in the profile/tier/layer information may be transmitted as many times as there are output levels.

указывает индекс в списке синтаксических структур

в VPS, синтаксической структуры

которая применяется к j-му уровню i-го OLS.[0174] According to the same embodiment

specifies the index in the list of syntactic structures

in VPS, syntactic structure

which is applied to the j-th level of the i-th OLS.

и/или

могут передаваться, когда число максимальных уровней больше 1

[0175] According to the same or another embodiment, in FIG. 20 syntax elements

and/or

may be transmitted when the number of maximum levels is greater than 1

указывающий режим передачи выходного уровня для i-ro набора выходных уровней, может быть представлен в VPS.[0176] According to the same or another embodiment, in FIG. 20 syntax element

indicating the output layer transfer mode for the i-ro set of output layers, may be represented in the VPS.

равный 0, указывает, что выводится только самый высокий уровень с i-ым набором выходных уровней

равный 1, указывает, что выводятся все уровни с i-ым набором выходных уровней. vps_output_layer_mode[i], равный 2, указывает, что выводятся уровни с

равным 1, с i-ым набором выходных уровней. Могут быть зарезервированы другие значения.[0177] According to the same embodiment,

equal to 0 indicates that only the highest level with the i-th set of output levels is output

equal to 1 indicates that all levels with the i-th set of output levels are output. vps_output_layer_mode[i] equal to 2 indicates that levels with

equal to 1, with the i-th set of output levels. Other values may be reserved.

может или не может передаваться в зависимости от значения

для i-го набора выходных уровней.[0178] According to the same embodiment

may or may not be passed depending on the value

for the i-th set of output levels.

может быть представлен i-го набора выходных уровней. В зависимости от значения

информация о профилях/ярусах/уровнях для i-го набора выходных уровней может передаваться или не передаваться.[0179] According to the same or another embodiment, in FIG. 20 flag

can be represented by the i-th set of output levels. Depending on the value

the profile/tier/level information for the i-th set of output levels may or may not be transmitted.

в текущем CVS может передаваться в синтаксической структуре высокого уровня, например в VPS, DPS, SPS, PPS, APS или в сообщении SEI.[0180] According to the same or another embodiment, in FIG. 21 number of sub-images,

in the current CVS, it may be carried in a high-level syntax, such as VPS, DPS, SPS, PPS, APS, or an SEI message.

для i-го субизображения может передаваться, когда число субизображений больше 1

[0181] According to the same embodiment, in FIG. 21 sub-image ID,

for the i-th sub-picture may be transmitted when the number of sub-pictures is greater than 1

который указывает k-е субизображение, представлен в j-ом уровне i-го набора выходных уровней. С помощью этой информации декодер может распознать, какое субизображение может быть декодировано и выведено для каждого уровня конкретного набора выходных уровней.[0182] According to the same or another embodiment, one or more syntax elements indicating a sub-image identifier related to each level of each set of output levels may be transmitted to the VPS. In FIG. 22

which indicates the k-th sub-image is represented in the j-th layer of the i-th output layer set. With this information, the decoder can recognize which sub-picture can be decoded and output for each level of a particular set of output levels.

[0183] According to an embodiment, a picture header (PH, Picture Header) is a syntactic structure containing syntax elements that apply to all slices of an encoded picture. Picture Unit (PU, Picture Unit) is a set of NAL units that are related to each other in accordance with a given classification rule, sequentially arranged in accordance with the decoding order, and contain an accurately encoded image. The PU may contain a Picture Header (PH) and one or more VCL NAL units constituting the encoded picture.

[0184] According to an embodiment, an SPS (RBSP) contained in at least one AU with Temporalid equal to 0 or provided by external means may be available to the decoding process before it is referenced.

[0185] According to an embodiment, an SPS (RBSP) contained in at least one AU with Temporalid equal to 0 in a CVS that contains one or more SPS-related PPSs or provided by external means may be available to decoding process before it is referenced.

равным наименьшему значению

блоков NAL с PPS, которые относятся к блоку NAL с SPS в CVS, который содержит один или более PPS, относящихся к SPS, или предоставленных с помощью внешних средств.[0186] According to an embodiment, an SPS (RBSP) may be available to the decoding process before it is referenced by one or more PPSs contained in at least one PU with

equal to the smallest value

PPS NAL units that refer to an SPS NAL unit in a CVS that contains one or more SPS-specific or externally provided PPSs.

равным наименьшему значению

блоков NAL с PPS, относящихся к блоку NAL с SPS или предоставленных с помощью внешних средств.[0187] According to an embodiment, an SPS (RBSP) may be available to the decoding process before it is referenced by one or more PPSs contained in at least one PU with Temporalid equal to 0, and

equal to the smallest value

PPS NAL units that are related to the SPS NAL unit or provided by external means.

равным наименьшему значению

блоков NAL с PPS, которые относятся к блоку NAL с SPS в CVS, который содержит один или более PPS, относящихся к SPS, или предоставленных с помощью внешних средств.[0188] According to an embodiment, an SPS (RBSP) may be available to the decoding process before it is referenced by one or more PPSs contained in at least one PU with Temporalid equal to 0, and

equal to the smallest value

PPS NAL units that refer to an SPS NAL unit in a CVS that contains one or more SPS-specific or externally provided PPSs.

указывает значение

для указанного SPS. Значение

может быть одинаковым во всех PPS, на которые ссылаются кодированные изображения в CLVS.[0189] According to the same or another embodiment,

indicates the value

for the specified SPS. Meaning

may be the same in all PPS referenced by encoded pictures in CLVS.

в CVS могут иметь одинаковое содержимое.[0190] According to the same or another embodiment, all NAL units with SPS with a certain value

in CVS can have the same content.

блоки NAL с SPS могут совместно использовать одно и то же пространство значений

[0191] According to the same or another embodiment, regardless of the values

NAL units with SPS can share the same value space

блока NAL с SPS может быть равно наименьшему значению

блоков NAL с PPS, которые относятся к блоку NAL с SPS.[0192] According to the same or another embodiment, the value

NAL unit with SPS may be equal to the smallest value

PPS NAL units that are related to the SPS NAL unit.

равным m, обращается один или более PPS с

равным n, уровень с

равным m, может быть идентичен уровню с

равным n, или (прямым или косвенным) опорным уровнем уровня с

равным m.[0193] According to an embodiment, when to the SPS with

equal to m, one or more PPS with

equal to n, level c

equal to m, may be identical to the level with

equal to n, or a (direct or indirect) level reference level with

equal to m.

[0194] According to an embodiment, a PPS (RBSP) contained in at least one AU with a TemporalId equal to the TemporalId of the PPS NAL unit, or provided by external means, may be available to the decoding process before any work is done on it. link.

[0195] According to an embodiment, a PPS (RBSP) contained in at least one AU with a TemporalId equal to the TemporalId of a PPS NAL unit in a CVS that contains one or more PHs (or slice-coded NAL units) related to the PPS, or provided by external means may be available to the decoding process before it is referenced.

равным наименьшему значению

блоков NAL с кодированными слайсами, которые относятся к блоку NAL с PPS в CVS, который содержит один или более РН (или блоков NAL с кодированными слайсами), относящихся к SPS, или предоставленных с помощью внешних средств.[0196] According to an embodiment, a PPS (RBSP) may be available to the decoding process before it is referenced by one or more PHs (or coded slice NAL units) contained in at least one PU with

equal to the smallest value

coded slice NAL units that refer to a PPS NAL unit in a CVS that contains one or more PHs (or coded slice NAL units) that belong to the SPS or are provided by external means.

равным наименьшему значению

блоков NAL с кодированными слайсами, которые относятся к блоку NAL с PPS в CVS, который содержит один или более РН (или блоков NAL с кодированными слайсами), относящихся к SPS, или предоставленных с помощью внешних средств.[0197] According to an embodiment, a PPS (RBSP) may be available to the decoding process before being referenced by one or more PHs (or coded slice NAL units) contained in at least one PU with a TemporalId equal to TemporalId of the NAL unit with PPS, and with

equal to the smallest value

coded slice NAL units that refer to a PPS NAL unit in a CVS that contains one or more PHs (or coded slice NAL units) that belong to the SPS or are provided by external means.

в РН определяет значение

для используемого PPS, на который выполняется ссылка. Значение

может быть одинаковым во всех PPS, на которые ссылаются кодированные изображения в CLVS.[0198] According to the same or another embodiment,

in pH defines the value

for the PPS in use that is being referenced. Meaning

may be the same in all PPS referenced by encoded pictures in CLVS.

в PU могут иметь одинаковое содержимое.[0199] According to the same or another embodiment, all NAL units with PPS with a certain value

in PU can have the same content.

блоки NAL с SPS могут совместно использовать одно и то же пространство значений

[0200] According to the same or another embodiment, regardless of the values

NAL units with SPS can share the same value space

блока NAL с PPS может быть равно наименьшему значению

блоков NAL с кодированными слайсами, которые относятся к блоку NAL, который относится к блоку NAL с PPS.[0201] According to the same or another embodiment, the value

NAL unit with PPS may be equal to the smallest value

coded slice NAL units that are related to the NAL unit that is related to the PPS NAL unit.

равным m, обращается один или более блоков NAL с кодированными слайсами с

равным m, уровень с

равным m, может быть идентичен уровню с

равным n, или (прямым или косвенным) опорным уровнем уровня с

равным m.[0202] According to an embodiment, when to the PPS with

equal to m, one or more coded slice NAL units are accessed with

equal to m, level c

equal to m, may be identical to the level with

equal to n, or a (direct or indirect) level reference level with

equal to m.

[0203] According to an embodiment, a PPS (RBSP) contained in at least one AU with a TemporalId equal to the TemporalId of the PPS NAL unit, or provided by external means, may be available to the decoding process before it is made link.

[0204] According to an embodiment, a PPS (RBSP) contained in at least one AU with a Temporalid equal to the Temporalid of a PPS NAL unit in a CVS that contains one or more PHs (or slice-coded NAL units) related to the PPS, or provided by external means may be available to the decoding process before it is referenced.

равным наименьшему значению

блоков NAL с кодированными слайсами, которые относятся к блоку NAL с PPS в CVS, который содержит один или более РН (или блоков NAL с кодированными слайсами), относящихся к SPS, или предоставленных с помощью внешних средств.[0205] According to an embodiment, a PPS (RBSP) may be available to the decoding process before it is referenced by one or more PHs (or coded slice NAL units) contained in at least one PU with

equal to the smallest value

coded slice NAL units that refer to a PPS NAL unit in a CVS that contains one or more PHs (or coded slice NAL units) that belong to the SPS or are provided by external means.

равным наименьшему значению

блоков NAL с кодированными слайсами, которые относятся к блоку NAL с PPS в CVS, который содержит один или более РН (или блоков NAL с кодированными слайсами), относящихся к SPS, или предоставленных с помощью внешних средств.[0206] According to an embodiment, a PPS (RBSP) may be available to the decoding process before it is referenced by one or more PHs (or encoded slice NAL units) contained in at least one PU with Temporalid equal to Temporalid NAL unit with PPS, and with

equal to the smallest value

coded slice NAL units that refer to a PPS NAL unit in a CVS that contains one or more PHs (or coded slice NAL units) that belong to the SPS or are provided by external means.

в РН определяет значение

для используемого PPS, на который выполняется ссылка. Значение

может быть одинаковым во всех PPS, на которые ссылаются кодированные изображения в CLVS.[0207] According to the same or another embodiment,

in pH defines the value

for the PPS in use that is being referenced. Meaning

may be the same in all PPS referenced by encoded pictures in CLVS.

в PU могут иметь одинаковое содержимое.[0208] According to the same or another embodiment, all NAL units with PPS with a certain value

in PU can have the same content.

блоки NAL с SPS могут совместно использовать одно и то же пространство значений

[0209] According to the same or another embodiment, regardless of the values

NAL units with SPS can share the same value space

блока NAL с PPS может быть равно наименьшему значению

блоков NAL с кодированными слайсами, которые относятся к блоку NAL, который относится к блоку NAL с PPS.[0210] According to the same or another embodiment, the value

NAL unit with PPS may be equal to the smallest value

coded slice NAL units that are related to the NAL unit that is related to the PPS NAL unit.

равным m, обращается один или более блоков NAL с кодированными слайсами с

равным n, уровень с

равным m, может быть идентичен уровню с

равным n, или (прямым или косвенным) опорным уровнем уровня с

равным m.[0211] According to an embodiment, when to the PPS with

equal to m, one or more coded slice NAL units are accessed with

equal to n, level c

equal to m, may be identical to the level with

equal to n, or a (direct or indirect) level reference level with

equal to m.

[0212] The output level indicates the level from the set of output levels that is output. An Output Layer Set (OLS) indicates a layer set consisting of a given layer set in which one or more levels in the layer set are defined as output layers. The level index from the output level set (OLS) is the index of the level in the OLS in the list of levels in the OLS.

[0213] The sublayer indicates the temporal scaling level of a temporally scalable bitstream containing VCL NAL units with a specific TemporalId variable value and associated non-NAL VCL units. The sublayer representation indicates a subset of the bitstream containing the NAL units of the particular sublayer and lower sublayers.

в CVS могут иметь одинаковое содержимое. На фиг. 22 показан пример RBSP VPS.[0214] A VPS RBSP contained in at least one AU with TemporalId equal to 0, or provided by external means, may be available to the decoding process before it is referenced. All SPS NAL units with a specific value

in CVS can have the same content. In FIG. 22 shows an example of an RBSP VPS.

предоставляет идентификатор VPS для ссылки на него других элементов синтаксиса. Значение

может быть больше 0.[0215]

provides a VPS ID for other syntax elements to refer to. Meaning

may be greater than 0.

плюс 1 указывает максимально допустимое число уровней в каждой CVS, ссылающейся на VPS.[0216]

plus 1 specifies the maximum number of levels allowed in each CVS linking to the VPS.

плюс 1 указывает максимальное число временных подуровней, которые могут быть представлены на одном уровне в каждой CVS, ссылающейся на VPS. Значение

должно находиться в диапазоне от 0 до 6 включительно.[0217]

plus 1 specifies the maximum number of temporary sublevels that can be present at the same level in each CVS referring to the VPS. Meaning

must be between 0 and 6 inclusive.

равный 1, указывает, что число временных подуровней является одинаковым для всех уровней в каждой CVS, ссылающейся на VPS.

равный 0, указывает, что уровни в каждой CVS, ссылающейся на VPS, могут иметь или не иметь одинаковое число временных подуровней. Когда этот элемент синтаксиса не представлен, предполагается, что значение

равно 1.[0218]

equal to 1 indicates that the number of temporary sublevels is the same for all levels in each CVS referring to the VPS.

set to 0 indicates that the levels in each CVS referring to the VPS may or may not have the same number of temporary sublevels. When this syntax element is not present, the value is assumed to be

equals 1.

равный 1, указывает, что все уровни в CVS кодируются независимо без использования межуровневого предсказания.

равный 0, указывает, что на одном или более уровней в CVS может использоваться межуровневое предсказание. Когда этот элемент синтаксиса не представлен, предполагается, что значение

[0219]

equal to 1 indicates that all layers in CVS are encoded independently without using inter-layer prediction.

equal to 0 indicates that one or more levels in CVS can use inter-level prediction. When this syntax element is not present, the value is assumed to be

i-го уровня. В случае любых двух неотрицательных целых значений тип, когда m меньше n, значение

может быть меньше

[0220]

i-th level. In the case of any two non-negative integer values, the type, when m is less than n, the value

maybe less

равный 1, указывает, что на уровне с индексом i межуровневое предсказание не используется,

равный 0, указывает, что на уровне с индексом i может использоваться межуровневое предсказание, а элементы синтаксиса

для] в диапазоне от 0 до «i - 1» включительно представлены в VPS. Когда этот элемент синтаксиса не представлен, предполагается, что значение

равно 1.[0221]

equal to 1 indicates that the level with index i does not use inter-level prediction,

equal to 0 indicates that interlevel prediction can be used at the level with index i, and the syntax elements

for] in the range from 0 to "i - 1" inclusive are represented in the VPS. When this syntax element is not present, the value is assumed to be

equals 1.

равный 0, указывает, что уровень с индексом j не является прямым опорным уровнем для уровня с индексом i.

равный 1, указывает, что уровень с индексом] является прямым опорным уровнем для уровня с индексом i. Когда

отсутствует для i и j в диапазоне от 0 до

включительно, предполагается, что он равен 0. Когда

может быть по меньшей мере одно значение j в диапазоне от 0 до «i - 1» включительно, так что значение

[0222]

equal to 0 indicates that the level with index j is not a direct reference level for the level with index i.

equal to 1 indicates that the level at index] is the direct reference level for the level at index i. When

missing for i and j in the range from 0 to

inclusive, it is assumed to be 0. When

there can be at least one value of j in the range from 0 to "i - 1" inclusive, so that the value

пределяются следующим образом:[0223] Variables

are defined as follows:

равным

определяется следующим образом:[0224] Variable GeneralLayerIdx[i], indicating the index of the layer with

equal

is defined as follows:

и

которые применяются к i-му уровню, могли быть равны значениям

соответственно, которые применяются к j-му уровню.[0225] For any two different values of i and j that are both in the range 0 to vps_max_layers_minus 1 inclusive, when dependencyFlag[i][j] is 1, bitstream matching is required so that the values

and

that apply to the i-th level could be equal to the values

respectively, which apply to the j-th level.

равный 1, указывает, что представлен элемент синтаксиса

равный 0, указывает, что элемент синтаксиса

не представлен.[0226]

equal to 1 indicates that a syntax element is present

equal to 0 indicates that the syntax element

not presented.

равный 0, указывает, что межуровневое предсказание для изображений i-ro уровня, не являющимися IRAP, не используется.

больше 0 указывает, что для декодирования изображений i-го уровня ни одно изображение с Temporalid больше

не используется в качестве ILRP. Когда этот элемент синтаксиса не представлен, предполагается, что значение

[0227]

equal to 0 indicates that inter-layer prediction for non-IRAP layer i-ro images is not used.

greater than 0 indicates that for decoding images of the i-th level, no image with Temporalid is greater than

not used as an ILRP. When this syntax element is not present, the value is assumed to be

равный 1, указывает, что в каждом OLS содержится только один уровень, а сам каждый уровень в CVS, ссылающейся на VPS, является OLS с единственным включенным уровнем, который является единственным выходным уровнем.

равный 0, указывает, что в OLS может содержаться более одного уровня. Когда значение элемента синтаксиса

равно 0, предполагается, что значение элемента синтаксиса

В противном случае, когда значение элемента синтаксиса

предполагается, что значение элемента синтаксиса

[0228]

equal to 1 indicates that each OLS contains only one level, and each level in the CVS referring to the VPS is an OLS with a single included level, which is the only output level.

equal to 0 indicates that more than one layer can be contained in the OLS. When the value of a syntax element

is 0, it is assumed that the value of the syntax element

Otherwise, when the value of the syntax element

it is assumed that the value of the syntax element

указывает, что общее число OLS, указанное VPS, равно

OLS содержит уровни с индексами уровней от 0 до i включительно, и для каждого OLS выводится только самый высокий уровень в OLS.[0229]

indicates that the total number of OLSs specified by the VPS is

The OLS contains levels with level indices from 0 to i inclusive, and for each OLS only the highest level in the OLS is output.

равный 1, указывает, что общее число OLS, указанное VPS, равно

i-ый OLS содержит уровни с индексами уровней от 0 до i включительно, и для каждого OLS выводятся все уровни в OLS.[0230]

equal to 1 indicates that the total number of OLSs specified by the VPS is

The i-th OLS contains levels with level indices from 0 to i inclusive, and for each OLS all levels in the OLS are output.

равный 2, указывает, что общее число OLS, указанных VPS, передается в явном виде, и для каждого OLS в явном виде передаются выходные уровни, и другие уровни являются прямыми или косвенными опорными уровнями выходных уровней OLS.[0231]

equal to 2 indicates that the total number of OLSs specified by the VPS is explicitly transmitted, and for each OLS, output layers are explicitly transmitted, and other layers are direct or indirect reference layers of output OLS layers.

может находиться в диапазоне от 0 до 2 включительно. Значение 3 элемента

зарезервировано для использования в будущем в стандартах

[0232] Value

can be in the range from 0 to 2 inclusive. 3 element value

reserved for future use in standards

а

равен 0, предполагается, что значение

равно 2.[0233] When

a

is 0, it is assumed that the value

equals 2.

плюс 1 указывает общее число OLS, указанных VPS,

равен 2.[0234]

plus 1 indicates the total number of OLSs specified by the VPS,

equals 2.

[0235] The TotalNumOlss variable indicating the total number of OLSs specified by the VPS is defined as follows:

равный 1, указывает, что уровень с

является выходным уровнем i-го OLS, когда

равный 0, указывает, что уровень с

не является выходным слоем i-го OLS, когда

[0236]

equal to 1 indicates that the level with

is the output level of the i-th OLS when

equal to 0 indicates that the level with

is not the output layer of the i-th OLS when

выходного уровня в i-ом OLS, и переменная

указывающая, используется ли k-ый уровень в качестве выходного уровня по меньшей мере в одном OLS, определяются следующим образом:[0237] The NumOutputLayersmOls[i] variable indicating the number of output layers in the i-th OLS, the NumSubLayersInLayerlnOLS[i][j] variable indicating the number of sublayers of the j-th layer in the i-th OLS, the OutputLayerldmOls[i][j] variable, indicating value

output level in the i-th OLS, and the variable

indicating whether the k-th layer is used as an output layer in at least one OLS are defined as follows:

включительно значения LayerUsedAsRefLayerFlag[i] и LayerUsedAsOutputLayerFlag[i] не могут быть равны 0 одновременно. Другими словами, не может существовать уровень, который не является ни выходным уровнем хотя бы одного OLS, ни прямым опорным уровнем любого другого уровня.[0238] For each value of i in the range from 0 to

inclusive, the values of LayerUsedAsRefLayerFlag[i] and LayerUsedAsOutputLayerFlag[i] cannot both be equal to 0. In other words, there cannot be a layer that is neither the output layer of at least one OLS nor the direct reference layer of any other layer.

[0239] For each OLS, there may be at least one layer that is an output layer. In other words, for any value of i between 0 and TotalNumOlss 1 inclusive, NumOutputLayersInOls[i] can be greater than or equal to 1.

j-го уровня в i-ом OLS, определяются следующим образом:[0240] The NumLayersInOls[i] variable indicating the number of layers of the i-th OLS, and the LayerIdInOls[i][j] variable indicating the value

j-th level in the i-th OLS are defined as follows:

равным LayerIdInOls[i][j], определяется следующим образом:[0241] Variable OlsLayerIdx[i][j], indicating the index of the OLS layer with

equal to LayerIdInOls[i][j] is defined as follows:

может быть равно 1.[0242] The lowest layer in each OLS may be an independent layer. In other words, for each i in the range from 0 to TotalNumOlss - 1 inclusive, the value

may be equal to 1.

равным одному из

для к в диапазоне от 0 до vps_max_layers_minus1 включительно, может существовать по меньшей мере одна пара значений из i и j, где i находится в диапазоне от 0 до TotalNumOlss 1 включительно, а j находится в диапазоне значений до NumLayersInOls[i] 1 включительно, так что значение LayerIdInOls [i] [j] равно nuhLayerId.[0243] Each layer can be included in at least one OLS specified by the VPS. In other words, for each level with a certain value

equal to one of

for k in the range 0 to vps_max_layers_minus1 inclusive, there can be at least one pair of i and j where i is in the range 0 to TotalNumOlss 1 inclusive and j is in the range up to and including NumLayersInOls[i] 1, so that the value of LayerIdInOls[i][j] is nuhLayerId.

в VPS, на который ссылается блок NAL с VCL.[0244] According to an embodiment, the temporalId value of any VCL NAL unit in the encoded video sequence cannot be greater than

in the VPS referenced by the NAL unit with the VCL.

имеет следующую семантику:[0245] According to the same or another embodiment,

has the following semantics:

равный 0, указывает, что межуровневое предсказание для изображений i-ro уровня, не являющимися IRAP, не используется.

больше 0 указывает, что для декодирования изображений i-го уровня ни одно изображение с TemporalId больше

-1 не используется в качестве ILRP. Когда этот элемент синтаксиса не представлен, предполагается, что значение

равно

[0246]

equal to 0 indicates that inter-layer prediction for non-IRAP layer i-ro images is not used.

greater than 0 indicates that for decoding images of the i-th level, no image with TemporalId is greater than

-1 is not used as an ILRP. When this syntax element is not present, the value is assumed to be

equals

не может быть больше

[0247] According to the same or another embodiment,

can't be more

и layerIncludedInOlsFlag не могут быть определены для независимых уровней.[0248] According to an embodiment, a set of output levels across sublevels cannot be obtained for independent levels. In this way,

and layerIncludedInOlsFlag cannot be defined for independent layers.

выходного уровня в i-ом OLS, и переменная LayerUsedAsOutputLayerFlag[k], указывающая, используется ли k-ый уровень в качестве выходного уровня по меньшей мере в одном OLS, определяются следующим образом:[0249] According to the same or another embodiment, the NumOutputLayersInOls[i] variable indicating the number of output layers in the i-th OLS, the NumSubLayersInLayerInOLS[i][j] variable indicating the number of sublayers of the j-th layer in the i-th OLS, the variable OutputLayerIdInOls[i][j] indicating the value

output layer in the i-th OLS, and the variable LayerUsedAsOutputLayerFlag[k] indicating whether the k-th layer is used as an output layer in at least one OLS are defined as follows:

плюс 1 указывает максимальное число временных подуровней, которые могут быть представлены в каждой CLVS, ссылающейся на SPS. Значение

должно находиться в диапазоне от 0 до

включительно.[0250] FIG. 23 shows an example RBSP SPS syntax table. In syntactical structure

plus 1 indicates the maximum number of temporal sublayers that can be present in each CLVS referring to the SPS. Meaning

must be in the range from 0 to

inclusive.

[0251] According to an embodiment, when all layers in an encoded video sequence have the same number of sublayers, the number of sublayers of each layer may be equal to the maximum number of sublayers transmitted in the video parameter set.

значение

в SPS, относящемся к VPS, должно быть равно значению

[0252] According to an embodiment, when the value

meaning

in the VPS related SPS must be equal to the value

больше 0 и

должен быть равен

[0253] According to an embodiment, when

greater than 0 and

should be equal

[0254] The adaptive resolution parameter transmission techniques described above may be implemented in computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 7 shows a computer system (700) suitable for implementing some embodiments of the disclosed invention.

[0255] Computer software may be coded using any suitable machine code or programming language that can be assembled, compiled, linked, or similar mechanisms to generate code containing instructions that can be directly executed or interpreted, executed as microinstructions, etc. .p. computer central processing units (CPU, Central Processing Unit), graphic processors (GPU, Graphics Processing Unit), etc.

[0256] Instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[0257] The computer system components (700) shown in FIG. 7 are illustrative in nature and are not intended to impose any limitation on the scope of application or functionality of computer software implementing embodiments of the present invention. The configuration of the components should also not be interpreted as having any dependency or requirement in connection with any one or combination of the components shown in the illustrative embodiment of the computer system (700).

[0258] Computer system 700 may include user interface input devices. Such an input device may be responsible for input by one or more users through, for example, tactile input (e.g., keystrokes, waving, glove movements), audio input (e.g., voice, claps), visual input (e.g., gestures), olfactory input (not shown). Human interface devices can also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, external sound), images (e.g. scanned images, photographic images received from a still image camera). ), video (eg, 2D video, 3D video including stereoscopic video).

[0259] The input human interface of the device may include one or more of the following components (only one of them is shown): keyboard (701), mouse (702), touchpad (703), touch screen (710), information glove ( 740), joystick (705), microphone (706), scanner (707), camera (708).

[0260] The computer system (700) may also include user interface output devices. Such output devices may stimulate the senses of one or more users through, for example, tactile output, sound, light, and smell/taste. Such output devices may include tactile output devices (e.g., haptic feedback via touch screen (710), data glove (704), or joystick (705), but may also be tactile feedback devices that do not serve as input devices), audio output devices (e.g., speakers (709), headphones (not shown)), visual output devices (e.g., screens (710), including CRT screens, LCD screens, plasma screens, OLED screens, each with capabilities touch-screen input, each with or without haptic feedback capabilities, some of which are capable of 2D visual output or more than 3D output via, for example, stereographic output; virtual reality goggles (not shown), holographic displays, and smoke tanks (not shown)) and printers (not shown).

[0261] Computer system 700 may also include user-accessible storage devices and associated media such as optical media, including CD/DVD ROM/RW 720 with CD/DVD or similar media 721, flash drive 722, removable or non-removable hard disk disk 723, conventional magnetic media such as a floppy disk (not shown in the drawing), specialized ROM/ASIC/PLD based devices such as dongles (not shown in the drawing), and the like.

[0262] Those skilled in the art should also understand that the term "computer-readable media" as used in connection with the invention disclosed herein does not encompass transmission media, carrier waves, or other short-term signals.

[0263] Computer system 700 may also include an interface to one or more communication networks. Networks can be, for example, wireless, wired, optical. Networks may additionally be local, wide area, metropolitan, transport and industrial, real-time, latency, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., digital wide area networks for wired or wireless television, including cable television, satellite TV and terrestrial TV, transport and industrial include CANBus, etc. Some networks typically require external network interface adapters that are connected to some general purpose data ports or peripheral buses (749) (for example, computer system USB ports (700)); others are typically built into the core of the computer system (700) by connecting to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). When using any of these networks, computer system 700 can communicate with other entities. Such communications may be unidirectional with receive-only capability (eg, broadcast), unidirectional with transmit-only capability (eg, CANBus to some CANBus devices), or bidirectional, eg, to other computer systems using a local or wide area digital network. Several protocols and protocol stacks may be used on each of these networks and the network interfaces described above.

[0264] The aforementioned human interface devices, human accessible storage devices, and network interfaces can be connected to the core (740) of the computer system (700).

[0265] The core 740 may contain one or more central processing units (CPU, Central Processing Unit) 741, graphics processors (GPU, Graphics Processing Unit) 742, specialized programmable processor units in the form of field programmable gate arrays (FPGA, Field Programmable Gate Area) 743, hardware accelerators 744 for certain tasks, etc. These devices, together with read-only memory (ROM) (745), random access memory (746), high-capacity internal storage such as user-inaccessible internal hard drives, SSDs, etc. (747), can be connected via the system bus (748). In some computer systems, the system bus (748) may be available in the form of one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripherals may be connected either directly to the kernel system bus (748) or through a peripheral. tire (749). Peripheral bus architectures include PCI, USBht.h.

[0266] CPUs (741), GPUs (742), FPGAs (743), and accelerators (744) can execute certain instructions, which together can compensate for the shortcomings of the computer code mentioned above. This computer code may be stored in ROM (745) or RAM (746). Transient data may also be stored in RAM (746), while persistent data may be stored, for example, in an internal mass storage (747). Fast storage and retrieval from any storage device can be achieved through the use of cache memory, which can be closely associated with one or more CPU (741), GPU (742), mass storage (747), ROM (745), RAM (746) etc.

[0267] Computer-readable media may store computer code for performing various computer-implemented operations. The media and computer code may be specially created for the purposes of the present invention or may be well known and available to those skilled in the art of computer software.

[0268] For example, without being limited to the above example, a computer system (700) of a certain architecture, and in particular, a core (740) may provide functionality as a result of execution by the processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc. .) software implemented on one or more computer-readable media. Such computer-readable media can be media associated with the user-accessible bulk data storage described above, as well as some persistent kernel storage (740), such as internal bulk data storage (747) or ROM (745). Software implementing various embodiments of the present invention may be stored on such devices and executed by the core (740). The computer-readable medium may include one or more memory devices or microcircuits, in accordance with specific requirements. The software may initialize the core 740, and in particular the processors located therein (including the CPU, GPU, FPGA, etc.), to execute specific processes or parts of specific processes described in this description, including the definition of data structures in RAM 746 and modifying such data structures in accordance with processes defined by the software. Additionally or alternatively, the computer system may provide functionality through logic hardwired or otherwise embodied in circuitry (e.g., an accelerator (744)) that may operate in place of or in conjunction with software to perform specific processes or specific portions of the specific processes described herein. A reference to software may cover logic, and vice versa when appropriate. Reference to computer-readable media may encompass a circuit (e.g., an integrated circuit (IC) that stores software to execute, a circuit embodying logic to execute, or both, as appropriate. The present invention encompasses any suitable combination of hardware and software. security.

[0269] Although several illustrative embodiments are described here, variations, permutations, and various replacement equivalents are possible and are within the scope of the invention. Thus, those skilled in the art can provide numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the invention and thus fall within its spirit and scope.

Claims (19)

1. Video decoding method performed by at least one processor and including: identifying the maximum number of temporal sublayers allowed in the encoded video sequence referring to the video parameter set in the video parameter set (VPS) field and in the sequence parameter set (SPS) field, respectively; and decoding the encoded video sequence based on the identified maximum number of temporal sublayers allowed in the encoded video sequence referring to the video parameter set, wherein the VPS field contains a syntax element indicating the output layer signaling mode for the i-th output layer set, and a value of this syntax element of 0 indicates that only the highest level is output with the i-th output level set, a value of this syntax element of 1 indicates that all levels are output with the i-th output level set, and the value of this element a syntax equal to 2 indicates that output levels of the i-th set of output levels for which the output level flag is 1; and decoding the coded video sequence based on the identified maximum number of temporal sublayers allowed in the coded video sequence referring to the video parameter set includes decoding the encoded video sequence based on the identified maximum number of temporal sublayers allowed in the encoded video sequence referring to the video parameter set, as well as the value of said syntax element in the VPS field. 2. The method of claim 1, wherein the temporal sublayers indicate the temporal scale levels of the temporally scaled bitstream. 3. The method according to p. 1, also including: identifying the maximum number of allowed levels in the encoded video sequence, and decoding the encoded video sequence based on the identified maximum number of allowed levels. 4. The method of claim 1, wherein the maximum number of temporal layers allowed in the encoded video sequence is specified in a raw byte sequence (RBSP) payload field of a video parameter set (VPS). 5. The method of claim 4, wherein the VPS RBSP comprises a VPS ID field and a field that identifies the maximum number of temporal layers allowed in the encoded video sequence. 6. The method of claim 1, wherein the maximum number of temporal layers allowed in the encoded video sequence is specified in a raw byte sequence (RBSP) payload field of a sequence parameter set (SPS). 7. The method of claim 6, wherein the SPS RBSP comprises an SPS ID field and a field that identifies the maximum number of temporal layers allowed in the encoded video sequence. 8. Device for video decoding, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate under the control of this program code to implement the method according to any one of paragraphs. 1-7. 9. A computer-readable medium that stores instructions containing one or more instructions that, when executed by one or more processors of a video decoding device, cause one or more processors to execute the method according to any one of paragraphs. 1-7.

Images