TWI821923B - Video coding concept allowing for limitation of drift - Google Patents

Abstract

A video decoder for decoding a video from a data stream is configured to decode an indication from the data stream which is valid for a sequence of pictures of the video, and indicates that RASL pictures within the sequence of pictures are coded in a manner excluding a predetermined set of one or more coding tools.

Description

Video coding concept technology that allows drift limits

Field of invention

This application is about video coding and concepts suitable for limiting drift.

Background of the invention

HTTP streaming of coded video has become an important path for video distribution over the past decade, and OTT service providers can now reach hundreds of millions of users over the public Internet. Standard protocols such as Dynamic Adaptive Streaming over HTTP (DASH)[1] enable service providers to stream media to clients by having the server serve the media in a time-sliced fashion at various bitrates . The client device is then able to download consecutive segments for continuous playback by selecting among the provided variants of a particular segment based on available network bandwidth and its decoding capabilities in a dynamic and adaptive manner. In practice, content is provided as multiple so-called representations generated from optimized bitrate ladders, often involving multiple resolutions and fidelity, to optimize the perceptual quality for a specific bitrate, and by This optimizes user experience[2]. Since each segment is typically coded using a so-called closed group of pictures (GOP) coding structure, independent of earlier segments [2], the downloaded and depacketized segment data string can be Connected to a consistent bit stream and fed to the decoder. In contrast to such closed GOP structures, segments using so-called open GOP coding structures contain some images that use inter-frame predictions of images in earlier segments, which is beneficial to coding efficiency. Although images using inter-frame prediction from earlier segments can be skipped from the output without playback issues or visual artifacts when random accessing the first-ordered segment, when A problem occurs when a resolution switch occurs during continuous playback, as the images are skipped in this non-seamless switch. Even in pure bitrate switching, some images may be lost or exhibit severe visual artifacts when segments are not properly encoded for switching.

Proliferating early codecs such as AVC [4] and HEVC [5] did not provide the reference picture resampling (RPR) functionality required to use reference pictures of different resolutions. Therefore, after a resolution switch, when executing under this open GOP structure, some images of a segment cannot be decoded correctly because the reference images from earlier segments are not available at the required resolution, which results in Non-constant frame rate playback under segment switching of self-dropped images. In [6], the authors proposed ways to overcome the Open GOP resolution switching problem by making normative changes to the HEVC decoding procedure or using a less proliferative scalability extension to HEVC (SHVC) that provides RPR functionality. . However, the solutions available to date do not make extensive use of open GOP coding in HTTP streaming.

The recently finalized version 1 of the Versatile Video Coding (VVC) standard [7] is the latest version developed jointly by the ITU-T Video Coding Expert Group and ISO/IEC Subcommittee 29 (also known as the animation professional body) Video coding standards. In addition to providing greatly improved coding efficiency compared with earlier codecs [8], VVC also included many application-driven features, such as RPR, in the original Main 10 configuration file. During the development of VVC, RPR was primarily studied in the context of conversational scenarios with low-latency coding structures [9], where real-world latency and buffer size requirements necessitated the insertion of in-frame coding images for parsing Severe limits are placed on the feasibility of degree switching.

However, RPR in VVC can also provide substantial benefits for coding efficiency in video encoding in the streaming domain.

It would be nice to have a concept at hand that enables the use of open GOP resolution switching in HTTP streaming using codecs such as VVC, where the above problem occurs not only in terms of RPR, but also by concatenating with e.g. different SNR associated segments to form a video bit stream.

It is therefore an object of the present invention to provide a video coding concept that makes it possible to more effectively limit the negative impact of drift on video quality, such as segmented video when switching between different video bit stream representations. Caused by bit stream formation.

This object is achieved by the subject matter of the independent claims of the present application.

Summary of the invention

According to a first aspect of the present invention, a video decoder for decoding a video from a data stream is provided, the video decoder being configured to decode a directive (eg, gci_rasl_pictures_tool_constraint_flag) from the data stream. , the indication is valid for an image sequence of the video and indicates that the RASL images in the image sequence are coded in a manner that does not include a predetermined set of one or more coding tools. For example, this indication may be used as a commitment to make known to the decoder open-GOP switching by concatenating separately coded open-GOP versions of video coded at different spatial resolutions and/or different SNRs. Does not cause excessive drift in RASL images. Other embodiments provide a video encoder for encoding a video into a data stream, the video encoder configured for encoding the indication into the data stream. For example, a RASL image represents an image that follows, in decoding order, an in-box coded image in a sequence of images, such as CRA, but precedes it in presentation order, and which can be preceded in decoding order using The reference image before the coded image in the box. For example, the previous image may belong to a previous image sequence, for example, the coded image within the frame may be the first image in the sequence in coding order. Since the reference is to an image before the inboxed image in decoding order, the resolution is switched at the inboxed image with respect to the previous segment of the video containing the previous inboxed image. , the drift artifacts mentioned above or other types of artifacts may appear. The above-mentioned indication in the signaling data stream assures the decoder that resolution switching at a sequence of images does not involve excessive drift in the RASL image compared to the previous sequence of images. Therefore, the decoder can decide whether resolution switching is advantageous based on the indication.

Other embodiments in accordance with the present invention provide a video decoder for decoding a video from a data stream, the video decoder configured to decode a directive from the data stream (e.g., using sps_extra_ph_bit_present_flag and ph_extra_bit, or using gci_rasl_pictures_tool_contraint_flag), which indicates whether the individual image is configured to exclude one or more coding tools, based on the image in an image sequence of the video, globally for each individual image, or on a per-slice basis. The code is written in a way that a predetermined set contains a prediction tool based on a cross-component linear model (e.g., as a graphical indication that it is possible to view RASL images where the potential drift is sufficiently low). Other embodiments provide a video encoder for encoding a video into a data stream, the video encoder configured for encoding the indication into the data stream.

Detailed description of preferred embodiments

The following description of the figures begins with a description of the encoder and decoder presenting a block-based predictive codec for coding images of video to form buildable scripts. An example of a coding framework according to an embodiment of the invention. The respective encoders and decoders are described with reference to Figures 1 to 3. In the following, a description of embodiments of the concepts of the present invention is presented and a description of how these concepts can be built into the encoder and decoder of Figures 1 and 2 respectively, but using the embodiments of Figure 4 and subsequently described It can also be used to form encoders and decoders that do not operate according to the coding framework underlying the encoders and decoders of Figures 1 and 2.

Figure 1 shows an apparatus for illustratively using transform-based residual coding to predictively code image A12 into data stream A14. Use reference symbol A10 to designate equipment or encoders. Figure 2 shows a corresponding decoder A20, that is, a device A20 configured to predictively decode image A12' from data stream A14 also using transform-based residual decoding, where the apostrophe is used to indicate that by decoding Image A12' reconstructed by device A20 deviates from image A12 originally encoded by device A10 in terms of coding loss introduced by quantization of the prediction residual signal. Although FIGS. 1 and 2 illustrate the use of transform-based predictive residual coding, embodiments of the present application are not limited to such predictive residual coding. The same is true for the other details described with reference to Figures 1 and 2, as will be summarized below.

Encoder A10 is configured to subject the prediction residual signal to a spatial to spectral transformation and encode the prediction residual signal thus obtained into data stream A14. Likewise, decoder A20 is configured to decode the prediction residual signal from data stream A14 and subject the prediction residual signal thus obtained to a spectral to spatial transformation.

Internally, encoder A10 may include a prediction residual signal former A22 that generates prediction residue A24 in order to measure the deviation of prediction signal A26 from the original signal (ie, from image A12). The prediction residual signal former A22 may for example be a subtractor that subtracts the prediction signal from the original signal, ie from the image A12. Encoder A10 then further includes a transformer A28 which subjects the prediction residual signal A24 to a spatial to spectral transformation to obtain a spectral domain prediction residual signal A24', which is then subjected to a spectral domain prediction residual signal A24' by Quantization is performed by quantizer A32. The thus quantized prediction residual signal A24'' is coded into a bit stream A14. To this end, encoder A10 may optionally include an entropy coder A34 that entropy codes the transformed and quantized prediction residual signal into data stream A14. Prediction signal A26 is generated by prediction stage A36 of encoder A10 based on prediction residual signal A24'' encoded into and decodable from data stream A14. To this end, as shown in Figure 1, the prediction stage A36 may internally include: a dequantizer A38, which dequantizes the prediction residual signal A24" to obtain a spectral domain prediction residual signal A24"", which signal is dequantized corresponds to the signal A24' except for the quantizer; an inverse transformer A40 after the dequantizer, which subjects the latter prediction residual signal A24''' to an inverse transform, that is, a spectral to spatial transformation, in order to obtain, except for the quantization loss, an inverse transformer A40 corresponding to the original Prediction residual signal A24'''' of prediction residual signal A24. The combiner A42 of the prediction stage A36 then recombines the prediction signal A26 and the prediction residual signal A24'''', such as by addition, in order to obtain a reconstructed signal A46, which is a reconstruction of the original signal A12. Reconstructed signal A46 may correspond to signal A12'. Prediction module A44 of prediction level A36 then generates prediction signal A26 based on signal A46 by using, for example, spatial prediction (ie, intra-image prediction) and/or temporal prediction (ie, inter-image prediction).

Likewise, as shown in Figure 2, decoder A20 may be internally composed of components that correspond to prediction level A36 and are interconnected in a manner corresponding to the prediction level. In particular, the entropy decoder A50 of the decoder A20 may entropy decode the quantized spectral domain prediction residual signal A24'' from the data stream, and then interconnect and cooperate in the manner described above with reference to the module of the prediction stage A36 The dequantizer A52, the inverse transformer A54, the combiner A56 and the prediction module A58 restore the reconstructed signal based on the prediction residual signal A24'' such that as shown in Figure 2, the output of the combiner A56 produces a reconstructed signal, also That is image A12'.

Although not specifically described above, it is easy to understand that the encoder A10 can set some coding parameters according to a certain optimization scheme, such as optimizing a certain rate and distortion related criteria (ie, coding cost), This includes, for example, prediction modes, motion parameters, etc. For example, the encoder A10 and the decoder A20 and the corresponding modules A44 and A58 may respectively support different prediction modes such as intra-frame coding mode and inter-frame coding mode. The granularity by which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of images A12 and A12' into coding segments or coding blocks, respectively. Taking these coding sections as units, for example, the image can be subdivided into blocks for writing codes within the frames and blocks for writing codes between the frames. In-frame coded blocks are predicted based on the spatial coded/decoded neighborhood of the respective block as outlined in more detail below. Several in-box coding modes may exist and be selected for respective in-box coding sections, including directional or angular in-box coding modes, in which respective sections are encoded according to the directional or angular in-box coding mode by For a certain direction specific to each directional box coding pattern, the sample values of the neighborhood are extrapolated into respective in-box coding sections and filled. The in-box coding mode may, for example, also include one or more other modes, such as a DC coding mode, for assigning DC values to respective in-box coding blocks based on the predictions of the in-box coding modes. After all the samples in the in-frame coding section; and/or the plane in-frame coding pattern, the prediction of each block is estimated or determined based on the plane in-frame coding pattern by relative to the two-dimensional linear function The spatial distribution of sample values is described by a two-dimensional linear function of the sample position of the coded block within the frame, respectively, based on the driving tilt and offset of the plane defined by adjacent samples. In comparison, inter-coded blocks may be predicted temporally, for example. For inter-frame coded blocks, a motion vector may be signaled within the data stream, indicating the spatial displacement of the previously coded image portion of the video to which image A12 belongs, at which spatial displacement for the previously coded image The coded/decoded image is sampled to obtain prediction signals for respective inter-coded blocks. This means that in addition to being encoded by the residual signal contained in the data stream A14 (such as the level of entropy-coded transform coefficients representing the quantized spectral prediction residual signal A24''), the data stream A14 may have been encoded into Coding mode parameters for assigning coding modes to various blocks, prediction parameters for some of the blocks (such as motion parameters for inter-coded segments), and optional other parameters (such as Parameters used to control and signal the subdivision of images A12 and A12' into segments respectively). Decoder A20 uses these parameters to subdivide the image in the same way as the encoder, assigning the same prediction modes to the segments and performing the same predictions to produce the same prediction signals.

Figure 3 illustrates the relationship between the reconstructed signal (i.e. the reconstructed image A12') on the one hand and the combination of the prediction residual signal A24'''' and the prediction signal A26 as signaled in the data stream A14 on the other hand. . As indicated above, the combination may be additive. Prediction signal A26 is illustrated in FIG. 3 as a subdivision of the image area into intra-coded blocks illustratively indicated by hatching and inter-coded blocks illustratively indicated by non-shading. The subdivision can be any subdivision, such as a conventional subdivision of the image area into columns and rows of square blocks or non-square blocks or a polytree of image A12 from the tree root block into multiple leaf blocks with different sizes. Subdivisions, such as quarter-tree subdivision, etc., the mixture of which is illustrated in Figure 3. In Figure 3, the image area is first subdivided into columns and rows of tree root blocks, which are then subdivided according to the recursive polytree subdivision. Additionally subdivided into one or more leaf blocks.

Again, data stream A14 may be directed to in-frame coding block A80 in which an in-frame coding mode is coded, with the in-frame coding mode assigning one of a number of supported in-frame coding modes to Write code block A80 in each frame. For inter-frame coding block A82, data stream A14 may code one or more motion parameters therein. Generally speaking, the inter-frame coding block A82 is not limited to coding in time. Alternatively, the interframe coded block A82 may be a previously coded image beyond the current image A12 (such as the video to which image A12 belongs, or an adjustable encoder and decoder at the encoder and decoder respectively. In this case, the image of another view or the lower layer of the hierarchy) itself is any block predicted from the previously coded part.

The prediction residual signal A24'''' in FIG. 3 is also illustrated as a subdivision of the image area into blocks A84. These blocks may be referred to as transform blocks to distinguish them from coding blocks A80 and A82. In fact, Figure 3 illustrates that the encoder A10 and the decoder A20 can use two different subdivisions of the image A12 and the image A12' into blocks, namely, into one subdivision of the coding blocks A80 and A82, respectively, and into another subdivision of transform block A84. The two subdivisions can be the same, that is, each coding block A80 and A82 can simultaneously form a transformation block A84, but Figure 3 illustrates the following situation: where, for example, the subdivision into a transformation block A84 forms a coding block The extension of the subdivision of A80 and A82 makes any boundary between two blocks in blocks A80 and A82 overlap with the boundary between two blocks A84, or in other words, each block A80, A82 and the transformation One of the blocks A84 coincides with or coincides with a cluster of transformed blocks A84. However, subdivisions may also be determined or selected independently of each other, such that transform block A84 may instead span the block boundary between blocks A80, A82. With respect to the subdivision into transform block A84, similar statements thus hold as to those made with respect to the subdivision into blocks A80, A82, i.e., block A84 may be an image area into blocks (with or without The result of a regular subdivision (column and row configuration), a recursive polytree subdivision of an image area, or a combination thereof or any other category of tiles. By the way, it should be noted that blocks A80, A82 and A84 are not limited to square, rectangular or any other shapes.

FIG. 3 further illustrates that the combination of prediction signal A26 and prediction residual signal A24'''' directly produces reconstructed signal A12'. However, it should be noted that according to alternative embodiments, more than one prediction signal A26 may be combined with the prediction residual signal A24'''' to produce image A12'.

In Figure 3, transform block A84 should have the following importance. The converter A28 and the inverse converter A54 perform their conversion in units of the conversion blocks A84. For example, many codecs use some kind of DST or DCT for all transform blocks A84. Some codecs allow transforms to be skipped such that for some of the transform blocks A84 the prediction residual signal is coded directly in the spatial domain. However, according to embodiments described below, encoder A10 and decoder A20 are configured in such a way that they support several transformations. For example, the transformations supported by encoder A10 and decoder A20 may include: ○ DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform ○ DST-IV, where DST stands for Discrete Sine Transform ○DCT-IV ○DST-VII ○Identity transformation (IT)

Of course, while transformer A28 will support all forward transform versions of these transforms, decoder A20 or inverse transformer A54 will support their corresponding backward or inverse versions: ○Inverse DCT-II (or Inverse DCT-III) ○Reverse DST-IV ○Reverse DCT-IV ○Reverse DST-VII ○Identity transformation (IT)

The subsequent description provides more details on which transformations encoder A10 and decoder A20 can support. In any case, it should be noted that the set of supported transforms may include only one transform, such as a spectral-to-spatial or spatial-to-spectral transform.

As summarized above, Figures 1 to 3 have been presented as examples in which the inventive concepts described further below may be implemented to form specific examples of encoders and decoders according to the present application. In this regard, the encoders and decoders of FIGS. 1 and 2 may represent possible implementations of the encoder and decoder, respectively, described below. However, Figures 1 and 2 are only examples. However, an encoder according to an embodiment of the present application may perform block-based encoding of image A12 using concepts outlined in more detail below, and differs from the encoder of FIG. 1 in that, for example, block A80 The subdivision is performed in a manner different from that illustrated in Figure 3. Likewise, a decoder in accordance with embodiments of the present application may perform block-based decoding of image A12' from data stream A14 using coding concepts outlined further below, but is different from, for example, decoder A20 of Figure 2 This may differ in that it does not support intra-frame prediction, or in that it subdivides image A12' into blocks in a manner different from that described with respect to Figure 3, and/or in that it is not in the transform domain but in, for example, the spatial domain. Export prediction residuals from data stream A14.

As discussed, Figures 1-3 are only intended to provide a rough overview of video codecs upon which the subsequently outlined embodiments of this application may be based. For example, VVC is an example of a video codec based on which the video decoder and video encoder of Figures 1 and 2 can be customized.

The following description is structured as follows. Preliminarily, VVC is used as an example of a video codec environment, and based on this example, the following description provides an experimental report on investigating the general coding efficiency impact of the open GOP coding structure and the image quality impact at zone switches. Again, the embodiments described later are not limited to VVC, and the coding tools discussed with respect to these embodiments are not limited to those discussed with respect to VVC, but these experiments and the presentation of their results provide the basis for generating the results described later. motivation for the implementation. In addition, the ensuing description will provide an overview of the GOP coding structure and segmentation, and then present constrained coding to enable open GOP switching, such as open GOP resolution switching, and effectively limit the drift associated with the switching. In the following, several embodiments of the present application arising from considerations regarding VVC are presented.

The following provides an overview of the structure within the VVC bitstream and media segments used for streaming. Media sections are typically aligned with in-frame random access point (IRAP) images using only in-frame coding tools. IRAP images may appear frequently in encoded video bit streams to allow functionality such as seeking or fast delivery, and also serve as switching points for adaptive HTTP streaming. Systems used for Video on Demand (VoD) streaming typically align segments to the IRAP image period, that is, the IRAP image is typically placed at the beginning of the segment and between the IRAP images as determined by the desired segment duration. time distance. However, there are use cases where not all media segments contain IRAP images, such as very low latency streaming, so that small segments can be used for transmission without waiting for IRAP images and thus reducing latency on the content generation side. Section size may vary in length depending on the target application. For example, VoD services allow players to build larger buffers (e.g., 30 seconds) to overcome throughput fluctuations. For this case, a segment size of up to several seconds (e.g., 5 seconds) can be a reasonable design Select [3]. However, real-time services that require tighter end-to-end latency do not allow such large buffers on the user side, and therefore require more frequent switching points and shorter segments of 1 second or less.

As long as the decoding latency requirements allow, the picture between two IRAP pictures is usually encoded in a bidirectional predicted hierarchical GOP structure, which involves reordering before rendering, since this structure provides as introduced in AVC [10] Substantial coding efficiency benefits. The hierarchical structure of the GOP can be used for temporal scalability, where all images up to a given layer are decoded corresponding to a given frame rate, and each image is assigned a corresponding temporal Id (Tid) value, as in Figure 1 Shown for GOP size of 8 images. The GOP may be defined as all pictures in decoding order from the first Tid 0 picture up to but not including the following Tid 0 picture. Typically, a section includes one or more GOP structures depending on the IRAP time period and GOP size. In HEVC, the number of reference picture slots in the Decoded Picture Buffer (DBP) allows for a typical GOP size of 16 pictures, while in VVC the increased DPR capacity allows for decoding of up to 32 pictures. Layer GOP size.

Pictures that follow the IRAP picture in decoding order but precede it in rendering order are introduced in HEVC and are called front-end pictures. It can be further distinguished into Random Access Skip Preamble (RASL) and Random Access Decodable Preamble (RADL). While a RADL picture may only use reference pictures starting from the IRAP picture in decoding order, a RASL picture may additionally use reference pictures that precede the IRAP. An IRAP image of the Instantaneous Random Access (IDR) type resets the DBP and may only have a preamble image, which is a RADL image resulting in a so-called closed GOP structure. Additionally, on the other hand, Clean Random Access (CRA) type IRAP images do not reset the DPB. Therefore, images reconstructed from before CRA in decoding order can be used as a reference for future images, ie, RASL images that allow so-called open GOP coding structures. RASL images exhibit higher coding efficiency than RADL images, but may appear undecodable when the reference image is not available, such as random access at the associated IRAP at the beginning of the segment. during fetch without decoding the previous segment. A more detailed overview of VVC's high-level syntax can be found in [11].

Figure 4 illustrates, for example, a video data stream formed by the concatenation of two consecutive segments with different resolutions, where the second segment uses an open GOP coding structure with a reference image from the first segment. Specifically, the reference images cited are the rectangles in Figure 4 from which arrows appear. The arrow itself illustrates prediction interdependence, that is, it points from the reference image to the reference image. Each image is associated with a certain time ID Tid, and as can be seen, the coding order deviates from the output/presentation order of the images. As can be seen, the images in output order 9 to 15 are RASL images that refer directly or indirectly to the CRA image of its own segment (Segment 1) to which it belongs, as well as from the previous segment ( The images in section 0) are mainly images with output order sorting 8. For example, a video segment may also be called an image sequence, which may include a GOP, for example.

When the reference picture of a RASL picture is located in the previous section and the streaming client switches representation after the previous section, the client decoder will use at least a partial difference in the reference picture compared to the encoder side Variant to decode RASL images. This situation can result in non-uniform bit streams if the content was not generated properly, or significant mismatch in the reconstructed RASL image, and this drift can be propagated to all RASL images up to but not including the correlation CRA image. In the following, appropriate generation of content is discussed that allows the use of open GOP structures while maintaining bitstream consistency at sector switches and avoiding unwanted drift that would otherwise be detrimental to visual quality during switches.

When open GOP switching is implemented, multiple inter-frame prediction tools in VVC exhibit different potentials to cause drift, and at the same time, tool usage is limited by the consistency constraint. Next, we analyze the drift potential of inter-frame prediction tools in VVC when open-GOP resolution switches, and in this paper propose a constrained coding method that overcomes severe artifacts of open-GOP resolution switches while ensuring VVC consistency.

Regarding the drift potential of VVC coding tools, the first set of coding tools in VVC can be classified as sample-to-sample prediction, such as those introduced from many predecessors of VVC or the most recent introduction in VVC called Affine Motion Compensation (AMC) Inter prediction mode is known as conventional block-based translation motion compensated sample prediction, which decomposes the prediction block into smaller sub-blocks that are individually motion compensated to simulate affine motion compensation [12 ]. Prediction improvements using Optical Flow (PROF) or Bidirectional Optical Flow (BDOF), which are optional components of AMC, are the latest inter-frame prediction tools introduced in VVC, by relying on optical flow-based methods to modify the predicted samples in order to simulate sample-by-frame inter-frame prediction. When a different representation is used as a reference for reconstruction using this sample-to-sample prediction tool, the visual quality of the reconstructed image will be biased towards the visual quality of that representation and away from the visual quality of the original representation. However, this sample-to-sample prediction has a relatively low probability of causing visual interference artifacts, but leads to elegant quality transitions in a given RASL image sequence, because the first visual quality of the predicted source sample is obtained by residual information Gradually updated in second visual quality.

Figure 5 illustrates an optical flow tool 300 and its functionality, for example. Inter-predicted block 10c within image 12 is shown. Inter-predicted block 10c is associated with motion vector 302. That is, motion vector 302 is signaled in the data stream for inter-prediction block 10c. The motion vector 302 indicates the translational displacement through the inter-prediction block 10c at which the reference image 304 is sampled/copied in order to generate the translational inter-prediction signal through the inter-prediction block 10c. If optical flow 300 is to be used for inter-prediction block 10c, optical flow tool 300 improves the translational inter-prediction signal by means of optical flow-based analysis. More precisely, instead of sampling reference image 304 only at the footprint of block 10c displaced according to motion vector 302, optical flow tool 300 uses an area slightly larger than the footprint of image 10c within reference image 304, also That is, the area 306 is used to determine the inter prediction signal for the inter prediction block 10c, that is, the gradient within the area 306 is detected to determine the inter prediction signal. In other words, it is possible by using the optical flow tool 300 , by using the gradient Sensitive FIR filtering is used to determine each sample of the inter-frame prediction signal of block 10c.

It should be noted that although Figure 5 only shows one reference image and one motion vector, the optical flow tool can also perform optical flow analysis on two reference images and two motion vectors, where image 12 is between the two reference images. and the image contains the inter-predicted block 10c.

Subsequently, when describing embodiments of the present application, optical flow tool 300 may form an example for coding tools that undergo self-coding exclusion to avoid drift. Therefore, according to some embodiments described below, the video decoder and/or the video encoder supports this optical flow tool. Since Figures 1 and 2 show possible implementations for video decoders and video encoders, video decoders and encoders according to Figures 1 and 2 that support the optical flow tool according to Figure 5 can be represented for use in this application illustrative basis for the embodiment. Regardless, there are different possibilities as to how to decide whether to apply the optical flow tool 300 to the inter-predicted block 10c at the decoder side and the encoder side respectively. For example, the optical flow tool 300 may be a coding tool native to the application. For example, whether the optical flow tool is applied to block 10c may depend on one or more coding options being signaled in the data stream for block 10c relative to another coding tool other than the optical flow tool. Even alternatively, the optical flow tool 300 may be a native application coding tool, where the decision of whether the optical flow tool 300 should be applied to a block 10c depends on the size of the block 10c. Naturally, two kinds of dependencies apply. Even alternatively, the optical flow tool 300 may be an application-specific coding tool, meaning that the syntax elements are coded into a data stream that specifically signals whether the optical flow tool 300 applies to block 10c. In other words, this syntax element will be specific to block 10c. It should be noted that this syntax element may not be a flag or binary value in the sense of merely being able to switch between non-application and application of tool 350 . Indeed, the syntax element may be an m-gram element, where one of its m states is associated with the application of tool 350, for example. Alternatively, the syntax element may be an m-gram element, where one of its m states is associated with non-application of tool 350, for example.

In an embodiment explained subsequently, the encoder signals to the decoder the exclusion of certain coding tools such as the optical flow tool 300 from the encoding of the RASL image. This signaling, discussed subsequently, need not serve as the actual control of whether these coding tools can be used for a block of an image or a slice of an image. In fact, this signaling or indication presented in the embodiments described further below may actually serve as an additional signaling or commitment from the encoder to the decoder: some of these coding tools (or simply A coding tool) has been excluded from the encoding of certain images such as RASL images. In the latter case, the instructions or signaling discussed subsequently are redundant or supplementary to the assembly signaling within the data stream that activates certain coding tools, rendering them unusable for the application. Image blocks within certain images or image slices. The encoder complies with the guarantees given by setting the assembly signaling and, if applicable, the syntax elements associated with the block-based tool application, thus enabling the tool to be/is not used in e.g. RASL images. Therefore, the optical flow tool can be a deactivated coding tool that, for its application to inter-frame prediction blocks such as block 10c, can be configured in units of images or slices by assembling data within the data stream. Go to boot. This assembly signaling may be contained by a slice header or an image header, for example. Whether the optical flow tool 300 will be applied to a certain inter-predicted block 10c will therefore be decided based on the assembly signaling, and only if the assembly signaling is for image 12 (or a slice of image 12) (inter-predicted Block 10c (part of this image) indicates that the optical flow tool 300 is enabled, ie available, and makes explicit signaling or inherent decisions regarding its application as described above.

The latter scenario is again depicted in Figure 6 because the general possibilities for deciding whether to apply a certain coding tool are similar to those discussed for the coding tools herein. Thus, FIG. 6 illustrates the currently coded image 12 and the block 10 that is part of the image 12 . Image block 10 is depicted illustratively only, and is actually only one image block into which image 12 is divided. The block level to which block 10 belongs may, for example, correspond to the block where the intra/inter prediction mode decision is made, but there are other possibilities, such as a block being smaller than a subsequent block. Possible examples for segmenting the image 12 into blocks such as block 10 have been described with respect to FIG. 3 . A representative coding tool 350 is shown in Figure 6. Whether this coding tool is actually applied to block 10 can be controlled by: the application can depend on the size of the block as indicated by arrow 352 and/or signaling for the predetermined block 10 in the data stream. Coding options, such as whether to use intra-frame block mode or inter-frame block mode to write the code block 10. The latter correlation is depicted in Figure 6 using arrow 354. As an alternative to or in addition to block size dependence, dependence on block aspect ratio or tree partitioning level of the corresponding recursive polytree-based partitioning tree may apply. Coding options may relate to blocks being coded in-frame, blocks being coded in-frame, blocks being bi-predicted, blocks being bi-predicted from equidistantly spaced and oppositely placed reference images, etc. one or more of them. Two information entities, namely block size/aspect ratio/split level and coding options are communicated in the data stream, i.e. by partitioning information about the partitioning of image 12 into blocks including block 10 Block size/aspect ratio/split level, and coding options specific to, for example, block 10. Instead of relying on implicit block-level decisions, explicit signaling in the form of syntax element 353 may be used to control the application of tool 350 to block 10 . The syntax elements will be specific to block 10 and to tool decisions for tool 350. While this application decision 356 is made at the block level, at a larger level, the application decision may additionally depend on an activation decision 358 at a larger level, such as with respect to the entire image 12 or along the image 12 12 is performed by coding the data stream into slices subdivided in the block coding order. The startup decision 358 may be controlled by the aforementioned configuration signaling, such as settings in the slice header of the slice to which block 10 belongs or in the image header of image 12 or parameters associated with image 12 such as an image parameter set. set. Not all decisions 356 and 358 are applicable. None of these are applicable, however the indications discussed below will then not only indicate the activation or deactivation of a certain slice, image or RASL image by a certain coding tool 350 as a commitment or redundant signaling , and also functions similarly to group signaling.

Thus, while the coding tool 350 of FIG. 6 may be the optical flow tool 300 of FIG. 5, it is depicted in FIG. 6 as representing any of the coding tools subject to the encoder constrained decisions discussed further below.

The second set of coding tools in VVC is used to predict the grammar (that is, the model parameters) from the image or sample. Similar to its predecessor, VVC allows motion vector (MV) prediction via Temporal Motion Vector Prediction (TMVP) [13] on a block basis using temporal MV candidates from so-called collocated reference pictures. This feature is extended in VVC by introducing a more fine-grained variant of TMVP (SBTMVP) on a sub-block basis, adding a displacement step to find the corresponding motion information in the collocated reference image.

Figure 7 illustrates a temporal motion vector prediction tool 500. This tool 500 is another example of the coding tool 350 used in FIG. 6 and discussed further below with respect to encoder constrained indications. A temporal motion vector prediction tool is used to predict motion vector 508 for inter-predicted block 10c of image 12 based on motion vector 510 associated with block 506 within reference image 502. Although according to the example of FIG. 7 , tool 500 may use motion vectors of collocated blocks within reference image 502 as predictors 508 , tool 500 first derives displacement vectors 504 for block 10c, such as spatially predicted motion vectors, and This displacement vector 504 is used to locate "collocated block 506" within the reference image 502, and the motion vector 510 of this collocated block 506 is then used to temporally predict motion vector candidates 508. Additionally, FIG. 7 illustrates that the tool 500 is operable to determine the temporal prediction motion vector 508 only for insertion into the motion vector candidate list 512 for block 10c, such as by using the The index is then used to select a motion vector predictor from the list. Alternatively, list 512 is understood in some way that the motion vector candidates within list 512 are generated in some order, and the motion vector candidate with the highest ranking is simply finalized/selected for use in the inter prediction region In block 10c.

It should be noted that the motion vector prediction tool of Figure 6 should be understood as broad enough to also cover the situation of temporal motion vector prediction in a merged sense, that is, providing motion from collocated blocks with other motion prediction settings such as reference picture indexes. Vector predictors into blocks.

All of the options discussed with respect to Figure 6 may be used with tool 500 to determine whether tool 500 actually applies to block 10c. More precisely, the tool 500 may be an example of the coding tool 350 in FIG. 6 , where explicit signaling in the form of syntax element 353 is used to control the application of the block 10 by the tool 500 , that is, in the data string The index in the stream signaled for block 10c selects or deselects TMVP candidate 508 in list 512 . If unselected, tool 500 remains invalid for block 10c, which is interpreted as a non-application of tool 500 for block 10c. However, configuration signaling at a higher level via decision 358 may be used to launch tool 500 more globally, such that for blocks residing in a region (image or slice) where tool 500 is signaled to launch, Manifest 512 is anyway understood to not include TMVP candidates 508, and the manifest index will no longer serve as the aforementioned block-level decision control. The encoder can decide which way to use in order to comply with any commitment given to the decoder in order to avoid drift in RASL images as taught herein.

Other tools in the aforementioned second group may be characterized as sample-to-syntax prediction tools. A completely new inter-frame prediction tool introduced in VVC is Decoder Side Motion Vector Improvement (DMVR), which improves the accuracy of MV in bidirectional prediction based on the mirroring properties of two reference images. Have the same and opposite temporal distance from the current image.

Figure 8 illustrates a decoder side motion vector improvement tool 400. If applied to the inter-frame predicted block 10d of image 12, the tool 400 will do so by improving the motion vector 402 coded/signaled for this block 10d in the data stream by using a best match search. Motion vector 402 is modified for inter prediction of this block 10d from reference image 404. Best match searches can be performed at the highest resolution supported by the decoder and encoder (1/16 pixel resolution). The signaled motion vectors may be of lower resolution and only serve to "substantially" indicate the improved motion vectors ultimately determined by the DMVR tool. There are different possibilities as to what should match the reference image in order to perform a best match search. One possibility would be to use the already decoded portion of the neighboring inter-prediction block 10d. This portion will be subjected to motion vector displacement using motion vector candidates at and around the signaled motion vector, and the candidate that yields the best match will be selected as the improved motion vector. Alternatively, block 10d may be a bi-predicted block where there is a pair of signaled motion vectors to be improved by tool 400. That is, in this situation, the inter-predicted block 10d is a bi-predicted block. The reference image may have the image 12 between it in presentation order, that is, the reference image is placed temporally in front of and behind the image 12 . Optionally, the two reference images are equidistant in time from image 12. The tool 400 may even be specialized for this situation, i.e. the tool 400 will be inherently enabled at the block level depending on block coding options which instructions will be based on equidistant intervals from the current image 10 Block 10d is bidirectionally predicted with a reference picture of picture 10 in between. In that case, the pair of signaled motion vectors will be improved by performing a best match search among motion vector pairing candidates including the pair of signaled motion vectors and the pair of signaled motion vectors, one of which will be vector402. For example, a best match search may be performed by testing the similarity between reference images sampled at portions called motion vector pairing candidates. Motion vector pairing candidates may be limited to motion vector pairing candidates in which a motion vector candidate for one reference image has a corresponding signal motion vector for that reference image and a motion vector candidate for another reference image. The other motion vector candidate deviates in an opposite manner to the deviation of the other signal motion vector for this other reference image. For similarity, SAD or SSD can be used. The best matching motion vector pairing candidate will then be used as a replacement 406 for the signal motion vector, that is, the vector 402 for the reference image 404 will be replaced 406, and the other signaled vector for another reference image will be replaced 406 by The motion vector pairing candidate is replaced by another vector. There will also be other possibilities. For example, bidirectional prediction can be improved individually by performing a best match search of the average of sampled tiles in two reference images at and around the signaled motion vector. Two signaled motion vectors of block 10d.

Regarding the decision of whether MVR tool 400 will be applied to block 10d, all alternatives discussed with respect to Figure 6 may apply. That is, the MVR tool 400 may be the coding tool 350.

Another new tool in VVC is the Cross-Component Linear Model (CCLM), which allows predicting the chrominance component of a block within a respective luma component box using a linear model in which the model parameters are derived from reconstructed luma sample values. The linear model transforms the successively sampled lightness samples rec ' _L into chroma predictions with the help of the following equation: P ( i , j ) = a ⋅ rec ' _L ( i , j ) + b , where the parameters a and b are: Luminance and chroma samples are exported. _In the case _{where X} l _and Y _l - Y _s ) / ( X _l - X _s ) b = Y _s - a ⋅ X _s

Because the parameter derivation procedure only considers extreme values of adjacent sample values, it is prone to widespread drift even in the presence of single-sample drift outliers in adjacent blocks. Also, due to the linear model, if a is larger, the brightness drift can be amplified. For other in-box prediction modes that consider all adjacent sample values, the drift propagation is less obvious and cannot be linearly amplified. Due to this inherent instability, this mode requires special care when used in applications where constrained drift is acceptable, such as open GOP switching in HTTP adaptive streaming. In addition, since in the context of the described application, the drift may only occur in RASL frames, ie motion predicted frames. If the encoder decides to use CCLM, i.e. intra prediction mode, this is usually due to the lack of appropriate motion compensated predictors, meaning regions of high temporal activity. In such regions, the reconstruction drift expected for open GOP switching is expected to be high, contributing even more to the discussed instability effects.

Figure 9 schematically illustrates the mode of operation of the cross-component linear model tool 100. Block 10a is shown relative to which tool 100 is applied. In combining the prediction signal and the residual signal for luma, the luma component of this block 10a is reconstructed using any type of prediction 120 and by decoding 122 the residual signal from the data stream. The purpose of tool 100 is to predict chroma components based on reconstructed luma components 124 . This is done using a linear model or linear mapping 106 . This linear mapping 106 uses a scalar linear function to predict the chroma component of sample 126 on a sample-by-sample basis for each sample 126 of block 10a based on the reconstructed luma component of sample 126. The linear parameters of the scalar linear function, namely a and b represented above, are determined for the block 10a through the analysis of the statistical data of the luminance and chroma components of the reconstructed samples in the neighborhood 112 of the block 10a. Block-wide determination. In detail, the statistical analysis 124 performed by the tool 100 is indicated in FIG. 9 at 128 and can determine for each component the external luminance and chroma values that occur within the reconstructed sample in the neighborhood 112 . For example, you can use the average of the two maximum brightness values, and the two minimum brightness values. The same is true for predicted chrominance components. Based on the four resulting averages, measures for the span of luma values and the span of chroma values within neighborhood 112 are determined, and the ratio therebetween is used as the slope of the scalar linear function for linear mapping 106 . The average of the minimum values for lightness - the slope x, and the average of the minimum values for chroma - are the intercepts used to determine the scalar linear function. The parameter derivation 108 so performed produces a scalar linear function, and each luma sample of the reconstructed luma component 124 is used to predict the corresponding chroma sample value within block 10a, thereby generating the color for block 10a. Inter-component prediction signal. Not shown in Figure 9, but possible, the data stream may have a chroma residual signal coded therein to correct the inter-chroma component prediction signal for chroma components C ₁ and/or C ₂ .

Again, tool 100 is another example of coding tool 350 for use in FIG. 6 . In other words, whether the coding tool is applied to the block 10a of the image may be determined based on any of the options discussed with respect to FIG. 6 . It is worth noting that VVC does not provide any means for launching the tool 100 image-wide or at least slice-wide, but according to an embodiment described later, this signaling is proposed, thereby avoiding errors caused by the tool 100 Harmful drift. More precisely, the tool 100 may be an example of the coding tool 350 used in FIG. 6 , where explicit signaling in the form of syntax element 353 is used to control the application of the block 10 by the tool 100 . The syntax element may be, for example, a flag that opens or closes tool 100 for block 10 . Regarding the subsequent discussion of the coding constrained indication regarding the embodiment discussion, there are two possibilities: it can simply inform the recipient (i.e. the decoder) of the fact that all syntax elements for a certain image 12 353 indicates non-application of the tool 100, or it may alternatively serve as a configuration signal to launch the tool 100 with respect to the image 12, whereby the data stream will not carry any syntax for the block 10 within that image 12 Element 353. Taking VVC as an example, for example, there is no assembly signaling to launch the tool 100 at the granularity of images or slices. In VVC, this configuration signaling is only used to control the launch of the tool 100 for a sequence of images. Therefore, RASL-style deactivation is not feasible.

Yet another new tool was introduced into the loop filtering stage of VVC and is called Luminance Mapping and Chroma Scaling (LMCS), where the chroma sample values are scaled using parameters derived from the luma samples, as illustrated in Figure 10 .

There is also a chroma-to-lightness correlation here, but it is not as obvious as in the case of CCLM. During the chroma scaling portion of the procedure, the transformed and inverse quantized chroma residues are scaled according to model parameters derived from the luma samples of adjacent virtual pipeline data units (VPDUs). For the purpose of pipeline latency reduction, CCLM relies on samples of adjacent VPDUs. However, in LMCS, all adjacent brightness samples are considered, allowing averaging of drifting outliers in adjacent samples of the VPDU. Also, these model parameters are used to scale the residual signal, which does not aggregate drift but is directly signaled. For these reasons, tools are much less likely to amplify drift, but should still be considered when coding for controlled drift applications.

The operating mode for the LMCS tool 200 is depicted in Figure 11. The idea here is to perform the luma tool mapping 212 in order to perform luma component prediction 202 and luma component residual decoding 204 for the predetermined image 12 in a written luma tone scale 208 rather than a rendered luma tone scale 210 . More precisely, while the reconstructed luma values may represent the luma components of the weighted signal for image 12 on a linear scale at a certain bit depth, luma tone mapping 212 may use, for example, per-image linear tone mapping function or some other tone mapping function to map this scale 210 onto the coding scale 208. The tone mapping function may be signaled at the data stream such as the image parameter set of image 12. This function is appropriately determined by the encoder. Therefore, the inter prediction signal obtained by inter prediction 202 for block 10b of image 12 is subjected to luma tone mapping 212 before being combined with residual signal 204 at coding scale 208 to produce the The reconstructed lightness component. For intra-predicted blocks, inter-prediction 206 is used. Intra-box prediction is performed within the coding scale 208 domain. Another goal of the tool 200 is to control the quantization error of the chroma components according to the luma tone map 212 . That is, the chroma component quantization error is controlled individually for each block, and the effects of the luma component are accommodated by luma tone mapping 212 . To this end, chroma residual scaling is determined for block 10b of image 12 from an average 220 of coded luma tone scaled versions of the reconstructed luma components of image 12 within a neighborhood 222 of image block 10b Factor 216. The chroma residual signal 224 decoded from the data stream for image block 10b is scaled 226 according to the chroma residual scaling factor 216 so determined, and this scaling is used to correct 228 the in-frame chroma for image block 10b Prediction signal 230. Compared to the luma component, the in-frame chroma prediction signal 230 may use the same prediction tool or a subset thereof. By using neighborhood 222 for determining mean 220, the luma and chroma components for block 10b can be reconstructed in parallel rather than serially. The reconstructed luma and chroma components of image 12 are then subjected to inverse luma tone mapping 240 to produce the reconstructed final result for image 12 and to produce the basis for subsequent coding/decoding of the image, i.e., used for this process. The reference image for subsequent writing/decoding of images.

Regarding coding tool 200, this comment is valid as with other coding tools discussed with regard to the preceding figures, that is, coding tool 200 is an example of coding tool 350 for FIG. 6, and may not be used for Options that determine the application of this coding tool to the specific blocks discussed in Figure 6. As an example, with respect to coding tool 300, block-by-block decision 356 may be omitted, but assembly signaling may be used to control the application on a per-image or per-slice basis.

Errors in MV predictions from grammar-to-grammar and sample-to-grammar box prediction tools have a relatively high probability of causing serious artifacts in subsequent sample-to-sample prediction tools that use these experience Mispredicted MVs serve as spatial or temporal MV candidates. This is particularly effective for (SB) TMVP and DMVR which exhibit the majority of visible artifacts in open GOP switches, since errors in inappropriate motion vectors may propagate with increasing magnitude in subsequent images. However, this also applies to other prediction models, such as CCLM and/or LMCS, which are implemented based on parameters derived from reconstructed sample values. Figure 12 illustrates the impact of general syntax or parameter prediction errors on the visual and objective quality of RASL images using a GOP size of 32 images in conventional open GOP coding. It is obvious that the RASL image has obvious artifacts in the brightness and chroma components of the reconstructed image.

The third problem in open GOP switching may stem from the use of adaptive parameter sets (APS) in VVC, which carry the filter coefficients of the adaptive loop filter (ALF), the luma map with chroma scaling (LMCS) List of parameters and quantization scaling. Since a RASL picture may refer to APS transmitted in decoding order before the respective CRA, these APS are available during consecutive decoding, but not during random access at the CRA picture since they are discarded in this case associated RASL image. Therefore, open GOP resolution switching may result in references to missing APS, which can cause non-fault-tolerant decoders to crash or produce visual artifacts when using parameters of the wrong APS with coincidentally matching identifier values. Similar to syntax prediction tools, this issue has a high probability of causing visual interference until the decoder fails completely.

In order to avoid the problems described above when performing open GOP resolution switching, the constrained VVC encoding method consisting of three pillars described below can be used.

First, the RASL pictures associated with the CRA are constrained such that pictures preceding the CRA in decoding order are not selected as collocated reference pictures to perform syntax-to-syntax prediction, that is, (SB) TMVP. Thereby, the exact same reference pictures and motion information as on the encoder side are used, and any syntax prediction errors from incorrect source motion information to earlier segment pictures are prevented. In a possible implementation, the first RASL picture in decoding order is restricted to using only its associated CRA picture as a collocated reference picture, which naturally only hosts zero motion vectors, while the other RASL pictures The image has access to non-zero time MV candidates for the first RASL picture and subsequent pictures in decoding order. Regarding the sample-to-syntax prediction tool, DMVR is disabled for all RASL images that have a valid reference image preceding the associated CRA in decoding order. In another alternative, DMVR is disabled for all RASL images regardless of their reference images, and in another alternative, DMVR is disabled only for RASL images used as collocated reference images for is used, and in yet another alternative, DMVR is disabled for all RASL images except those belonging to the highest temporal layer and thereby not used as a reference. Thereby, erroneous sample values from reference pictures in early segments that differ from encoder-side or drift-affected samples in other RASL pictures do not result in sample-to-syntax errors in prediction.

To ensure VVC consistency after section switching, the use of other tools, namely the optical flow related tools BDOF and PROF, must be disabled for all RASL images with a reference image before the associated CRA. In an alternative, for simplicity, BDOF and PROF might be disabled for all RASL images. In addition, the new feature of VVC for independently coded sub-images within a video, such as 360-degree viewport-related video streaming, must be disabled to use RPR. All the above tool constraints are also part of the conformance constraints defined in the VVC specification to enable RPR. In addition to the consistency constraints associated with the use of RPR in VVC, other tool constraints are also required because prediction techniques that use parameter predictions from reconstructed samples can also produce significant artifacts. Therefore, in our implementation, CCLM is disabled by block-by-block constraints of the encoder-side search algorithm. This is because the current VVC syntax only allows sequence-by-sequence disabling, which significantly reduces the overall coding efficiency. This effectively allows ensuring that drift is avoided on the encoder side, but cannot be easily confirmed on the decoder without thorough low-level profiling. Also, because the tool is enabled but not used, unnecessary bits (ie, code unit level flags used by CCLM, such as cclm_mode_flag or cclm_mode_idx) are sent to signal the encoding decision not to use them.

Secondly, also for open GOP coding structures, that is, CRA pictures that precede their associated RASL pictures in decoding order, the necessary APS for all pictures need to be present in this section. It should be noted that for random access open GOP streaming, this constraint is not necessary and a RASL picture is allowed to refer to an APS transmitted in the bit stream that precedes the associated CRA picture in decoding order. Since when such an image is randomly accessed at such a CRA image, the RASL image is discarded and this reference is not problematic. Also, such APS is available in continuous decoding when switching is not performed. However, these APS may not be available in streams with open GOP switching, and therefore need to be prevented from being referenced. In our implementation, the processing related to ALF, LMCS, and quantized scaling lists is reconfigured in a similar manner to the closed GOP coding structure.

Third, from the perspective of VVC high-level syntax, the individual encodings of the variants in the bitstreaming ladder must be implemented in a coordinated manner, keeping in mind that the target of open GOP switching is on the decoder side. Therefore, the sequence parameter sets (SPS) of all segment variants need to be aligned so that segment switching does not trigger the start of a new coded layer video sequence through changes in the SPS. For example, with appropriate coordination, SPS will indicate the maximum resolution within the bitstream ladder, match block size and chroma format, appropriate match level indicators and related constraint flags such as gci_no_res_change_in_clvs_constraint_flag, sps_ref_pic_resampling_enabled_flag and sps_res_change_in_clvs_allowed_flag, It has the appropriate configuration to enable the use of RPR on the decoder side. Devices with capabilities lower than those required for the indicated maximum resolution or level will need to be serviced by system mechanisms using adjusted SPS.

RPR in VVC has been designed in a constrained manner to limit its implementation and runtime complexity as apparent from the discussion of tool constraints above. An important aspect of this complexity consideration is that the memory bandwidth when accessing scaled reference samples in use with RPR is acceptable and not significantly higher than without RPR. Coded images in VVC are accompanied by a so-called scaling window, which is used to determine the scaling factor between two images. In order to set a bound on the memory bandwidth requirements of RPR, the relationship between the zoom window of the image using RPR and the zoom window of its reference image is limited to allow a maximum of eight times magnification and two times reduction. In other words, each zoom window is assumed to match the image size of its representation, allowing RPR to be used when switching to a representation with eight times the height of the image size. However, if the image size is reduced by no less than half in each dimension, down switching may only use RPR.

Typically, in the case of adaptive streaming, upswitching is performed in a progressive manner, that is, gradually increasing the resolution or quality. However, on downswitching, it may happen that when the player's buffer is low, the player switches to the lowest quality to avoid buffer underruns, meaning that downswitching may not occur gradually. One way to mitigate this limitation of RPR in VVC is to use a closed GOP structure to encode the lowest quality representation so that it can be used as a backing when the image size is reduced to less than half during such non-progressive switching events. plan.

In the following, embodiments are described regarding decoders and encoders supporting one, more or all of the coding tools 100, 200, 300, 400 and 500 discussed above. The decoder and encoder described next can be implemented in accordance with the manner of FIG. 1 and FIG. 2 . Although the tools 100, 200, 300, 400 and 500 have been discussed above mainly with respect to the decoder side, it is obvious that the description of the corresponding tools can be transferred to the encoder side, the difference being that the encoder inserts the involved information into The information involved is decoded in the data stream rather than from the data stream. Each supported coding tool represents a coding tool 350 . Any coding tool using explicit syntax element control 353 on a block basis involves the encoder encoding the syntax elements on a block basis and the decoder decoding the syntax elements from the data stream. Coding tool 100 may be the only coding tool that uses this explicit block-based syntax element. Other coding tools 200, 300, 400, and 500 may use native block-based application decisions 356 along with image-based or slice-based assembly signaling for fully deactivating the tool.

The subsequently explained embodiments relate to an indication or signaling to the decoder whether certain coding constraints regarding the use of the just mentioned set of one or more coding tools have been complied with. The encoder signals this indication in the data stream and limits its encoding accordingly by complying with the corresponding encoding constraints. In the case of a sector switch, the decoder in turn uses this indication and interprets it as a drift-limited guarantee or indication. According to alternative embodiments, the instructions/signaling discussed below may also be used to actually launch one or more of the coding tools. For example, in VVC it has not been possible so far to launch the tool 100 on an image or slice basis. In addition to the commitment function, the instructions/signaling discussed below may also assume the function of configuring signaling to launch the tool 100 with respect to certain images/slices. The block-based syntax elements used for block-based decision 356 may then be omitted and not encoded into the data stream and not decoded from the data stream.

Subsequently, a combination is presented that uses presentation-constrained signaling to enable open-GOP resolution switching, i.e., when performing stream switching, the RASL images of the CRA can be decoded with acceptable drift due to certain write Coding tools are not available on RASL images. While current state-of-the-art technology allows for such indication for some of the tools that are part of the presented method (such as TMVP, SBTMVP, BDOF, PROF, and DMVR), there are significant additional constraints in the presented method that require Avoid serious artifacts from samples to grammar prediction tools (i.e., CCLM and/or LMCS). Therefore, there is a need to have encoders that can indicate in the bitstream that such tools are not valid for certain images (i.e. CRA's RASL images).

VVC has an extension mechanism to add bit flags to the image header (PH) and slice header (SH) syntax in a backwards-compatible manner. For this purpose, the respective SPS indicates the number of extra bits in the PH or SH syntax used for this purpose. These extra bits must be parsed when reading the syntax, and the derivation is used to Bits are assigned to flags or variable values. The following table shows the respective SPS and PH syntax along with the respective semantics. SH syntax and semantics are similar to PH syntax and semantics. seq_parameter_set_rbsp( ) { Descriptor […] sps_num_extra_ph_bytes u(2) for( i = 0; i < (sps_num_extra_ph_bytes * 8 ); i++ ) sps_extra_ph_bit_present_flag [i] u(1) […] sps _extra _ph _bit _present _flag [ i ] equal to 1 specifies that the i-th extra bit is present in the PH syntax structure referencing SPS. sps_extra_ph_bit_present_flag[i] equal to 0 specifies that the i-th extra bit does not exist in the PH syntax structure referencing SPS.

The variable NumExtraPhBits is derived as follows: NumExtraPhBits = 0 for( i = 0; i < ( sps_num_extra_ph_bytes * 8 ); i++ ) if( sps_extra_ph_bit_present_flag[ i ] ) (1) NumExtraPhBits++ picture_header_structure( ) { Descriptor […] for( i = 0; i <NumExtraPhBits; i++ ) ph_extra_bit [i ] u(1) […] ph_extra_bit[i] can have any value. Decoders conforming to this version of this specification shall ignore the presence and value of ph_extra_bit[i]. Its value does not affect the decoding procedures specified in this version of this specification.

An unknown decoder can at least parse the bitstream correctly and decode it correctly, while a decoder that knows the meaning of the extra bits can further interpret the extra bits indication and act accordingly, such as suggesting the string to the user. Flow switching is possible without severe drift due to compliance with constraints according to the presented method. Likewise, a file format wrapper, HTTP streaming server, or even an RTP streaming server can consider this bitstream directive when packaging, serving, and serving the content in a manner that utilizes bitstream switching.

Embodiments of the present invention would carry an indication of the rendered method in extra bits of the PH or SH syntax of the RASL image or associated CRA image as follows. In SPS semantics, a specific extra bit flag is identified by the index i of sps_extra_ph_bit_present_flag[i] as indicating the presence of a PH/SH extra bit flag, which indicates the presentation method. For example, the presence of the first extra bit in the PH may be identified by the first SPS PH extra bit (i=0) of the first SPS PH extra byte.

The value of the variable ConstraintMethodFlagPresentFlag is set equal to sps_extra_ph_bit_present_flag[ 0 ]. It should be noted that index 0 is used, but another index could be used instead, namely, the bits in sps_extra_ph_bit_present_flag[i] are chosen to have the following meaning: constrain the RASL image in terms of the tool used.

In PH semantics, the respective variables indicating the characteristics of the rendered constraint method are derived as follows.

The value of the variable ConstrainedRASLFlagEnabledFlag/ConstrainedCRAFlagEnabledFlag is set equal to (ConstraintMethodFlagPresentFlag && ph_extra_bit[ 0 ]). It should be noted that index 0 is used, but depending on the value indicated by sps_extra_ph_bit_present_flag[i] and which index is used for the constraint against the RASL image ph_extra_bit[j], the jth flag among the extra flags in the PH will indicate that. Whether the equality constraints are appropriate for RASL images.

Alternative 1 (carrying signaling in RASL images): When ConstrainedRASLFlagEnabledFlag is equal to 1, encode the current image without using CCLM. PH/SH control flags and sequence-level constraint flags for BDOF, DMVR, PROF, (SB)TMVP and LMCS are already in VVC version 1, while CCLM misses control flags with image or slice scope.

Alternative 2 (carrying signaling in the associated CRA image): When ConstrainedCRAFlagEnabledFlag is equal to 1, encode the RASL image associated with the current image without using TOOLSET, where TOOLSET refers to CCLM and/or Or LMCS and/or BDOF and/or PROF and/or DMVR, and/or RASL images do not use any collocated image used for (sb)TMVP before the current image (ie CRA image).

Alternative embodiments similar to those above may be constructed for SH signaling of slices of RASL or CRA images.

In another alternative embodiment, the above constraints are indicated as CVS, CLVS and /or properties/constraints of bit streams.

Corresponding to alternative 1: gci_rasl_pictures_tool_constraint_flag equal to 1 specifies that the ConstrainedRASLFlagEnabledFlag for all RASL images in OlsInScope should be equal to 1. gci_rasl_pictures_tool_constraint_flag equal to 0 does not impose such constraints.

Corresponding to alternative 2: gci_cra_pictures_tool_constraint_flag equal to 1 specifies that the ConstrainedCRAFlagEnabledFlag for all CRA images in OlsInScope should be equal to 1. gci_cra_pictures_tool_constraint_flag equal to 0 does not impose such constraints.

That is, when this general constraint flag is set, all RASL images associated with CVS, CLVS and/or CRA in the bitstream are encoded without using TOOLSET - where TOOLSET refers to CCLM and/or The LMCS and/or BDOF and/or PROF and/or DMVR, and/or RASL images do not use any collocated image used for (sb)TMVP before the current image (that is, the CRA image).

In another alternative embodiment, the above constraints are indicated in the PPS extension syntax. The RASL image associated with the CRA may refer to a PPS that indicates that the above constraints are in effect, while other images of the bitstream do indeed refer to PPSs that do not indicate the above constraints.

In another alternative embodiment, the above constraint signaling is performed by SEI messages in the CRA image or associated RASL image or the entire coded layer video sequence for the image.

In another alternative embodiment, the above constraint signaling is used to conditionally send the CCLM flag at the code writing unit level. In another alternative embodiment, the above constraint signaling does not apply to all RASL images associated with CRA, but is limited to a subset of RASL images, depending on the actual tool, taking into account the acceptable drift and incurring loss of coding efficiency. :

- DMVR: (Same as described above in this article): DMVR is disabled for all RASL pictures that have a valid reference picture preceding the associated CRA in decoding order. It should be noted that the current picture can have valid reference pictures in its reference picture list (RPL) (actually used for prediction), and non-valid reference pictures, which are not used for prediction of the current picture but subsequent (as per decoding order) image samples or syntax, and it is therefore not ready to be removed from the Decoded Picture Buffer (DPB). In another alternative, DMVR is disabled for all RASL images regardless of their reference images, and in another alternative, DMVR is disabled only for RASL images used as collocated reference images for used, and in yet another alternative, DMVR is disabled for RASL images that do not belong to the highest temporal layer. These alternatives can be combined.

- BDOF and PROF: Disabled for all RASL pictures or only for RASL pictures with a valid reference picture preceding the associated CRA in decoding order.

According to one embodiment, the indication is communicated in the data stream in the form of an SEI message. As mentioned above, the SEI message may be valid for all images in the image sequence (eg, coded video sequence, CVS). For example, SEI messages may be signaled in this sequence. Therefore, the decoder can infer from the presence of the SEI message or from the indication in the SEI message that all RASL images in the sequence are coded in a manner that does not include the predetermined set of coding tools. For example, according to this embodiment, the set of coding tools includes at least a prediction tool 100 based on a cross-component linear model and a decoder-side motion vector improvement tool 400.

In the following, further embodiments of the invention described above are described.

D1.1. A video decoder for decoding a video from a data stream configured to decode a directive from the data stream [e.g., gci_rasl_pictures_tool_constraint_flag] that for a sequence of images of the video is Valid and indicates that the RASL images within the image sequence are coded with a predetermined set that does not include one or more coding tools [e.g., as a commitment to make the decoder known, by concatenating different Open-GOP switching of separately coded Open-GOP versions of this video coded at different spatial resolutions and/or different SNRs does not result in excessive drift in the RASL image].

D1.2. The video decoder of any preceding embodiment D1.#, wherein the set of one or more coding tools includes A prediction tool (100) based on a cross-component linear model.

D1.3. The video decoder of embodiment D1.2, wherein according to the prediction tool based on a cross-component linear model, a chroma component (102) of an image block (10a) uses a linear model (106 ) is predicted from a brightness component (104) of the image block (10a), the parameters of the linear model are derived from the brightness and chroma extrema (112) of one of the decoded neighborhoods (112) of the image block 110) to determine (108).

D1.4. The video decoder of any preceding embodiment D1.#, wherein the set of one or more coding tools includes A luma tone mapping and chroma residual scaling prediction tool (200).

D1.5. The video decoder of embodiment D1.4, wherein based on the luma tone mapping and chroma residual scaling prediction tools, Luminance component prediction (202) [eg, inter-frame prediction] and a luma component residual decoding (204) for a predetermined image (12) are performed with a coded luma tone scale (208), a rendered luma The hue scale (210) is mapped onto the coded value tone scale by a value tone map (212) to obtain a coded value tone scale version of the reconstructed value component of the predetermined image ( 214), A chroma residual scaling factor (216) for an image block (10b) of the predetermined image is derived from the reconstructed luma component of the predetermined image within a neighborhood (222) of the image block It is judged by the average value (220) of the lightness and hue scale version of the code, and A chroma residual signal (224) decoded from the data stream for the image block is scaled (226) according to the chroma residual scaling factor and used to correct (228) a frame for the image block Inner chroma prediction signal (230).

D1.6. The video decoder of any preceding embodiment D1.#, wherein the set of one or more coding tools includes An optical flow tool (300).

D1.7. The video decoder of embodiment D1.6, wherein the optical flow tool is A translational inter prediction signal for improving a predetermined inter prediction block (10c) by means of optical flow based analysis.

D1.8. The video decoder of any preceding embodiment D1.#, wherein the set of one or more coding tools includes A decoder side motion vector improvement tool (400).

D1.9. The video decoder of embodiment D1.8, wherein the decoder-side motion vector improvement tool is for improving a signaled motion vector by performing a best match search among motion vector candidates at and around a signaled motion vector coded in the data stream (402). , for performing inter prediction on a predetermined inter predicted block (10d) from a reference image (404).

D1.9a. The video decoder of embodiment D1.9, wherein the decoder-side motion vector improvement tool is configured with The best match search is performed using one of the decoded neighborhoods of the inter-predicted block relative to the reference image.

D1.9b. The video decoder of embodiment D1.8, wherein the decoder-side motion vector improvement tool is configured with Improving a pair of signaled motion vectors by performing a best match search among motion vector pairing candidates including a pair of signaled motion vectors encoded in the data stream (402) and surrounding the pair of signaled motion vectors Vectors for mapping a predetermined frame from a pair of reference images (404) temporally placed in front of and behind one of the images in the predetermined inter-frame bidirectional prediction block (10d) in presentation order. The inter-frame bidirectional prediction block (10d) performs inter-frame prediction.

D1.10. The video decoder of any of the preceding embodiments D1.#, wherein the set of one or more coding tools includes A temporal motion vector prediction tool (500).

D1.11. The video decoder of embodiment D1.10, wherein according to the temporal motion vector prediction tool, forming a motion vector candidate list for an inter-predicted block includes proceeding from a previously decoded image (502) Motion vector candidate complement.

D1.12. The video decoder of embodiment D1.11, wherein according to the temporal motion vector prediction tool, the motion vector candidate list for the inter prediction block is formed including a free motion vector predictor (504) pointing Motion vector candidate supplementation is performed on a block of the previously decoded image (506).

D1.13. The video decoder of embodiment D1.12, wherein the motion vector predictor includes a temporal motion vector predictor.

D1.14. The video decoder of any preceding embodiment D1.#, wherein the instruction is included in one of the following: a decoder capability information section of the data stream, and one of the video or sequence parameter sets of the data stream, and 1. Supplement and enhance information messages.

D1.15. The video decoder of any preceding embodiment D1.#, wherein the indication includes a bit that collectively indicates the coding relative to the RASL images in the image sequence, excluding a or all coding tools in the predetermined set of multiple coding tools.

D1.16. The video decoder of any preceding embodiment D1.#, wherein the decoder is configured to support reference picture resampling.

D1.17. The video decoder of embodiment D1.16, wherein, based on the reference picture resampling, one of the reference pictures of the inter-frame prediction block is subjected to sample resampling in order to bridge the reference picture and the A scaling window size deviation or sample resolution deviation between images containing the inter-predicted block provides an inter-prediction signal for the inter-predicted block.

D1.18. The video decoder as in any one of the aforementioned embodiments D1.#, wherein the set of one or more coding tools includes [for example, 200, 300, 400, 500] one or more first application-specific coding tools, each of which is applied for a predetermined block depending on one or more coding options communicated in the data stream for the predetermined block, and Relevant to a coding tool other than the respective coding tool, and/or One or more second application-specific coding tools, each of which is applied to a predetermined block depending on a size of the predetermined block.

D1.19. The video decoder as in any one of the aforementioned embodiments D1.#, wherein the set of one or more coding tools includes [for example, 100, 500] One or more explicitly applied coding tools, each of which is applied to a predetermined block in dependence on a syntax element coded into the data stream for specifically transmitting The application of each coding tool to the predetermined block.

D1.20. The video decoder of embodiment D1.19, wherein the decoder is configured for blocks within the RASL images and for blocks for images other than the RASL images from the The data stream decodes this syntax element.

D1.21. The video decoder of embodiment D1.19, wherein the decoder is configured to decode the syntax element from the data stream only for blocks within images other than RASL images [e.g., by This saves bits in these RASL images].

D1.22. The video decoder of any one of the preceding embodiments D1.#, wherein the decoder is configured to support an intra-prediction block decoding mode and an inter-prediction block decoding mode.

D1.23. The video decoder as in any one of the aforementioned embodiments D1.#, wherein the image sequence Beginning with and including a CRA image and including images up to—in coding order—and ending with an image immediately preceding a CRA image, or Contains images that are consecutive in coding order and contain more than one CRA.

D1.24. The video decoder as in any one of the aforementioned embodiments D1.#, wherein the set of one or more coding tools includes One or more enableable coding tools, each of which with respect to its application to an image block, may be enabled on an image or slice basis by assembly signaling within the data stream.

D1.25. A video decoder as in any of the preceding embodiments D1.# configured to use this indication to see whether open GOP switching causes persistent drift.

D1.25a. A video decoder as in any one of the preceding embodiments D1.# configured to use this instruction to see if open GOP switching results in a sample mismatch, but maintaining syntax and parameter settings.

D1.26. The video decoder as in any one of the aforementioned embodiments D1.#, wherein the indication indicates that all RASL images within the sequence of images are coded in such a manner that does not include the predetermined set of one or more coding tools, and/or The indication indicates that all RASL images within the image sequence are coded in the manner excluding the predetermined set of one or more coding tools, the RASL images having reference images, the reference images being The decoding order precedes a CRA picture with which it is associated, and/or The indication indicates that all RASL images within the image sequence are coded in such a manner excluding the predetermined set of one or more coding tools that are used as a temporal motion vector for the image predict reference images, and/or The indication indicates that all RASL images within the image sequence that do not belong to a top temporal layer are coded in such a manner that does not include the predetermined set of one or more coding tools.

D1.27. The video decoder as in any one of the aforementioned embodiments D1.#, wherein The instruction instructs the RASL images within the sequence of images to be coded in a manner that does not include the predetermined set of one or more coding tools such that For a first subset of one or more coding tools in the predetermined set of one or more coding tools, a first characteristic is applied to all RASL images excluding the one or more coding tools the manner of the first subset of one or more coding tools in the predetermined set is coded, and For a second subset of one or more coding tools in the predetermined set of one or more coding tools, a second characteristic applies to all RASL images, or to all RASL images excluding The manner in which the second subset of one or more coding tools in the predetermined set of one or more coding tools is coded is such that the first subset and the second subset are mutually disjoint.

D1.28. As in the video decoder of embodiment D1.27, the first characteristic and/or the second characteristic is selected: causing the reference picture to precede a CRA picture with which it is associated in decoding order, serves as a reference image for temporal motion vector prediction for one of the following images, and Does not belong to a highest time layer.

D1.29. The video decoder of embodiment D1.27 or higher, wherein The first subset includes one or more of a decoder-side motion vector improvement tool and a temporal motion vector prediction tool.

D1.30. The video decoder of embodiment D1.29, wherein the first characteristic is selected from: causing the reference picture to precede a CRA picture with which it is associated in decoding order, serves as a reference image for temporal motion vector prediction for one of the following images, and Does not belong to a highest time layer.

D2.1. A video decoder for decoding a video from a data stream, configured with Decode a directive from the data stream [e.g., using sps_extra_ph_bit_present_flag and ph_extra_bit, or using gci_rasl_pictures_tool_contraint_flag], either per picture in a sequence of pictures in the video, globally for individual pictures, or on a per-slice basis Whether the respective image is coded in a manner that does not include a predetermined set of one or more coding tools, the predetermined set including a prediction tool based on a cross-component linear model [e.g., as an graphical indication, which The potential drift at which it is possible to view RASL images is low enough].

D2.2. The video decoder of any one of the preceding embodiments D2.#, wherein the decoder is configured to support an intra-prediction block decoding mode and an inter-prediction block decoding mode.

D2.3. The video decoder of any of the preceding embodiments D2.#, wherein according to the prediction tool based on the cross-component linear model, The chrominance component of an image block is predicted from the lightness component of the image block using a linear model whose parameters are derived from the lightness and color in one of the decoded neighborhoods of the image block. Judgment by extreme value.

D2.4. The video decoder of any preceding embodiment D2.#, wherein the set of one or more coding tools further includes A luma tone mapping and chroma residual scaling prediction tool.

D2.5. The video decoder of embodiment D2.4, wherein based on the luma tone mapping and chroma residual scaling prediction tools, Luminance component prediction and a luminance component residual decoding for a predetermined image are performed with a coded luma tone scale, and a rendered luma tone scale is mapped to the coded luma tone by a luma tone map on a scale to obtain a coded luma tonal scaled version of one of the reconstructed luma components of the predetermined image, A chroma residual scaling factor for an image block of the predetermined image is derived from the coded luma tone scaled version of the reconstructed luma component of the predetermined image within a neighborhood of the image block to determine the average value of A chroma residual signal decoded from the data stream for the image block is scaled according to the chroma residual scaling factor and used to correct an intra-frame chroma prediction signal for the image block.

D2.6. The video decoder of any preceding embodiment D2.#, wherein the set of one or more coding tools further includes An optical flow tool.

D2.7. The video decoder of embodiment D2.6, wherein the optical flow tool is A translational inter prediction signal for improving a predetermined inter prediction block by means of optical flow based analysis.

D2.8. The video decoder of any preceding embodiment D2.#, wherein the set of one or more coding tools further includes A decoder-side motion vector improvement tool.

D2.9. The video decoder of embodiment D2.8, wherein the decoder-side motion vector improvement tool is for improving a signaled motion vector by performing a best match search among motion vector candidates at and around a signaled motion vector coded in the data stream (402). , for performing inter prediction on a predetermined inter predicted block (10d) from a reference image (404).

D2.9a is the video decoder of embodiment D2.9, wherein the decoder-side motion vector improvement tool is configured with The best match search is performed using one of the decoded neighborhoods of the inter-predicted block relative to the reference image.

D2.9b. The video decoder of embodiment D2.8, wherein the decoder-side motion vector improvement tool is configured with Improving a pair of signaled motion vectors by performing a best match search among motion vector pairing candidates including a pair of signaled motion vectors encoded in the data stream (402) and surrounding the pair of signaled motion vectors Vectors for mapping a predetermined inter-frame bi-predicted block (10d) from a pair of reference pictures (404) temporally placed in front of and behind a picture of the predetermined inter-frame bi-predicted block (10d). (10d) Perform inter-frame prediction.

D2.10. The video decoder of any of the preceding embodiments D2.#, wherein the set of one or more coding tools further includes A temporal motion vector prediction tool.

D2.11. The video decoder of embodiment D2.10, wherein according to the temporal motion vector prediction tool, formation of the motion vector candidate list for the inter-predicted block includes generating motion vector candidates from a previously decoded image. who added.

D2.12. The video decoder of embodiment D2.11, wherein according to the temporal motion vector prediction tool, the motion vector candidate list for the inter prediction block is formed to include the motion vector predictor pointing to the free one. Motion vector candidate supplementation is performed on a block of a previously decoded image.

D2.13. The video decoder of embodiment D2.12, wherein the motion vector predictor includes a temporal motion vector predictor.

D2.14. The video decoder of any of the preceding embodiments D1.#, wherein the instruction is included in one of the following: one or more image parameter sets referenced by the images in the image sequence, one of the image headers of the images in the sequence of images, and A slice header for one of the slices of the images in the image sequence.

D2.15. The video decoder of any of the preceding embodiments D2.#, wherein the instruction is included in the following: A set of image parameters referenced by the images in the sequence of images, wherein the sets of image parameters include: at least one first set of image parameters indicating a picture that references the at least one first set of image parameters The image is coded in a manner that does not include the predetermined set of one or more coding tools; and at least one second set of image parameters indicating that images referencing the at least one second set of image parameters may be used one of the predetermined sets of one or more coding tools is coded, or Image parameter sets referenced by the images in the image sequence, wherein the image parameter sets include: at least a first image parameter set indicating and referencing the at least one first image parameter set a RASL image associated with the image is coded in a manner that does not include one of the predetermined set of one or more coding tools; and at least one second image parameter set indicating and referencing the at least one second image The RASL image associated with the image of the parameter set is coded in one of the predetermined sets, possibly using one or more coding tools.

D2.16. The video decoder of embodiment D2.15, wherein the indication includes a syntax element within an extended syntax portion of the image parameter set [e.g., using sps_extra_ph_bit_present_flag and ph_extra_bit].

D2.17. The video decoder of embodiment D2.16, wherein a length of the extended syntax part of the image parameter set is specified in a sequence of the data stream or a video parameter set.

D2.18. The video decoder of any preceding embodiment D2.#, wherein the instruction is a syntax element in an extension of one of the following: one of the image headers of the images in the sequence of images, and/or A slice header for one of the slices of the images in the sequence of images, One of the lengths of the extension [eg, NumExtraPhBits] is indicated in an image or sequence of the data stream or in a video parameter set.

D2.18a. The video decoder of embodiment D2.18, wherein the syntax element indicates whether an image to which the syntax element belongs [for example, to which the image header or slice header relates] is coded in a manner that does not include one of the predetermined set of one or more coding tools, or Whether the RASL image associated with the image to which the syntax element belongs is coded in a manner that does not include one of the predetermined set of one or more coding tools.

D2.19. The video decoder of any preceding embodiment D2.#, wherein the decoder is configured to support reference picture resampling.

D2.22. The video decoder of embodiment D2.19, wherein, based on the reference picture resampling, one of the reference pictures of the inter-frame prediction block is subjected to sample resampling in order to bridge the reference picture and the A scaling window size deviation or sample resolution deviation between images containing the inter-predicted block provides an inter-prediction signal for the inter-predicted block.

D2.23. The video decoder as in any one of the aforementioned embodiments D2.#, wherein the set of one or more coding tools includes [for example, 200, 300, 400, 500] one or more first application-specific coding tools, each of which is applied for a predetermined block depending on one or more coding options communicated in the data stream for the predetermined block, and Relevant to a coding tool other than the respective coding tool, and/or One or more second application-specific coding tools, each of which is applied to a predetermined block depending on a size of the predetermined block.

D2.24. The video decoder as in any one of the aforementioned embodiments D2.#, wherein the set of one or more coding tools includes [for example, 100, 500] One or more explicitly applied coding tools, each of which is applied to a predetermined block in dependence on a syntax element coded into the data stream for specifically transmitting The application of each coding tool to the predetermined block.

D2.25. The video decoder of embodiment D2.24, wherein the decoder is configured to decode the syntax element from the data stream for blocks within: the one or more coding tools A predetermined set of images or slices that are signaled to be excluded from self-encoding; and the predetermined set of one or more coding tools that are not signaled to be excluded from self-encoding.

D2.26. The video decoder of embodiment D2.24, wherein the decoder is configured to signal regions within images or slices excluded from encoding only for the predetermined set of one or more coding tools Block decodes the syntax element from the data stream.

D2.27. The video decoder of the preceding embodiment D2.24 or any one of the above, wherein the prediction tool based on the cross-component linear model belongs to the one or more one or more explicitly applied coding tools.

D2.28. The video decoder of any one of the preceding embodiments D2.16 or above, wherein the syntax element is a bit that collectively indicates that the predetermined set of one or more coding tools is not included All coding tools.

D2.29. The video decoder as in any one of the aforementioned embodiments D2.#, wherein the set of one or more coding tools includes One or more enableable coding tools, each of which with respect to its application to an image block, may be enabled on an image or slice basis by assembly signaling within the data stream.

D2.30. A video decoder as in any of the preceding embodiments D2.# configured to use this indication to see whether open GOP switching causes persistent drift.

D2.30a. A video decoder as in any of the preceding embodiments D2.# configured to use this instruction to see if open GOP switching results in a sample mismatch, but maintaining syntax and parameter settings.

E1.1. A video encoder for encoding a video into a data stream, configured with Encode into the data stream a directive [e.g., gci_rasl_pictures_tool_constraint_flag] that is valid for an image sequence of the video and indicates that the RASL images within the image sequence do not include one or more coding tools. A predetermined set of coded in one way [e.g. as a commitment made known to the encoder by concatenating individually coded open GOP versions of the video coded at different spatial resolutions and/or different SNRs Open GOP switching does not cause excessive drift in RASL images].

E1.2. The video encoder of any of the preceding embodiments E1.#, wherein the set of one or more coding tools includes A prediction tool (100) based on a cross-component linear model.

E1.3. The video encoder such as E1.2, wherein according to the prediction tool based on the cross-component linear model, A chrominance component (102) of an image block (10a) is predicted from a luminance component (104) of the image block (10a) using a linear model (106) whose parameters are derived from the One of the image blocks has been determined (108) by decoding the brightness and chroma extrema (110) in the neighborhood (112).

E1.4. The video encoder of any of the preceding embodiments E1.#, wherein the set of one or more coding tools includes A luma tone mapping and chroma residual scaling prediction tool (200).

E1.5. The video encoder of embodiment E1.4, wherein based on the luma tone mapping and chroma residual scaling prediction tools, Luminance component prediction (202) [eg, inter-frame prediction] and a luma component residual encoding (204) for a predetermined image (12) are performed with a coded luma tone scale (208), a rendered luma The hue scale (210) is mapped onto the coded value tone scale by a value tone map (212) to obtain a coded value tone scale version of the reconstructed value component of the predetermined image ( 214), A chroma residual scaling factor (216) for an image block (10b) of the predetermined image is derived from the reconstructed luma component of the predetermined image within a neighborhood (222) of the image block It is judged by the average value (220) of the lightness and hue scale version of the code, and A chroma residual signal coded in the data stream for the image block (224) is scaled (226) according to the chroma residual scaling factor and used to correct for the image block (228) In-frame chroma prediction signal (230).

E1.6. The video encoder of any of the preceding embodiments E1.#, wherein the set of one or more coding tools includes An optical flow tool (300).

E1.7. The video encoder of embodiment E1.6, wherein the optical flow tool is A translational inter prediction signal for improving a predetermined inter prediction block (10c) by means of optical flow based analysis.

E1.8. The video encoder of any of the preceding embodiments E1.#, wherein the set of one or more coding tools includes A decoder side motion vector improvement tool (400).

E1.9. The video encoder of embodiment E1.8, wherein the decoder-side motion vector improvement tool is for improving a signaled motion vector by performing a best match search among motion vector candidates at and around a signaled motion vector coded in the data stream (402). , for performing inter prediction on a predetermined inter predicted block (10d) from a reference image (404).

E1.9a. The video encoder of embodiment E1.9, wherein the decoder-side motion vector improvement tool is configured with The best match search is performed using one of the decoded neighborhoods of the inter-predicted block relative to the reference image.

E1.9b. The video encoder of embodiment E1.8, wherein the decoder-side motion vector improvement tool is configured with Improving a pair of signaled motion vectors by performing a best match search among motion vector pairing candidates including a pair of signaled motion vectors encoded in the data stream (402) and surrounding the pair of signaled motion vectors Vectors for mapping a predetermined inter-frame bi-predicted block (10d) from a pair of reference pictures (404) temporally placed in front of and behind a picture of the predetermined inter-frame bi-predicted block (10d). (10d) Perform inter-frame prediction.

E1.10. The video encoder of any preceding embodiment E1.#, wherein the set of one or more coding tools includes A temporal motion vector prediction tool (500).

E1.11. The video encoder of embodiment E1.10, wherein according to the temporal motion vector prediction tool, forming a motion vector candidate list for an inter-predicted block includes performing from a previously decoded image (502) Motion vector candidate complement.

E1.12. The video encoder of embodiment E1.11, wherein according to the temporal motion vector prediction tool, the motion vector candidate list for the inter prediction block is formed to include a free motion vector predictor (504) pointing Motion vector candidate supplementation is performed on a block of the previously encoded image (506).

E1.13. The video encoder of embodiment E1.12, wherein the motion vector predictor includes a temporal motion vector predictor.

E1.14. The video encoder of any preceding embodiment E1.#, wherein the instruction is included in one of the following: a decoder capability information section of the data stream, and one of the video or sequence parameter sets of the data stream, and 1. Supplement and enhance information messages.

E1.15. The video encoder of any preceding embodiment E1.#, wherein the indication includes a bit that collectively indicates the writing relative to the RASL images in the image sequence, excluding All coding tools in the predetermined set of one or more coding tools.

E1.16. The video encoder of any preceding embodiment E1.#, wherein the encoder is configured to support reference picture resampling.

E1.17. The video encoder of embodiment E1.16, wherein based on the reference picture resampling, one of the reference pictures of an inter-frame prediction block is subjected to sample resampling in order to bridge the reference picture with the reference picture therein. A scaling window size deviation or sample resolution deviation between images containing the inter-predicted block provides an inter-prediction signal for the inter-predicted block.

E1.18. The video encoder as in any one of the preceding embodiments E1.#, wherein the set of one or more coding tools includes [for example, 200, 300, 400, 500] one or more first application-specific coding tools, each of which is applied for a predetermined block depending on one or more coding options communicated in the data stream for the predetermined block, and Relevant to a coding tool other than the respective coding tool, and/or One or more second application-specific coding tools, each of which is applied to a predetermined block depending on a size of the predetermined block.

E1.19. The video encoder as in any one of the preceding embodiments E1.#, wherein the set of one or more coding tools includes [for example, 100] One or more explicitly applied coding tools, each of which is applied to a predetermined block in dependence on a syntax element coded into the data stream for specifically transmitting The application of each coding tool to the predetermined block.

E1.20. The video encoder of embodiment E1.19, wherein the encoder is configured to encode blocks within the RASL images and blocks in images other than the RASL images. The syntax element is encoded into the data stream.

E1.21. The video encoder of embodiment E1.19, wherein the encoder is configured to encode the syntax element into the data stream only for blocks within images other than RASL images [e.g., thereby saving bits in these RASL images].

E1.22. The video encoder as in any one of the preceding embodiments E1.#, wherein the encoder is configured to support intra-prediction block coding mode and inter-prediction block coding mode.

E1.23. The video encoder as in any one of the preceding embodiments E1.#, wherein the image sequence Beginning with and including a CRA image and including images up to - in coding order - and ending with an image immediately preceding a CRA image, or Contains images that are consecutive in coding order and contain more than one CRA.

E1.24. The video encoder as in any one of the preceding embodiments E1.#, wherein the set of one or more coding tools includes One or more enableable coding tools, each of which with respect to its application to an image block, may be enabled on an image or slice basis by assembly signaling within the data stream.

E1.25. The video encoder of any one of the preceding embodiments E1.# configured to comply with instructions as an encoding constraint when encoding the video into the data stream.

E1.26. The video encoder as in any one of the aforementioned embodiments E1.#, wherein the indication indicates that all RASL images within the sequence of images are coded in such a manner that does not include the predetermined set of one or more coding tools, and/or The indication indicates that all RASL images within the image sequence are coded in the manner excluding the predetermined set of one or more coding tools, the RASL images having reference images, the reference images being The decoding order precedes a CRA picture with which it is associated, and/or The indication indicates that all RASL images within the image sequence are coded in such a manner excluding the predetermined set of one or more coding tools that are used as a temporal motion vector for the image predict reference images, and/or The indication indicates that all RASL images within the image sequence that do not belong to a top temporal layer are coded in such a manner that does not include the predetermined set of one or more coding tools.

E1.27. The video encoder as in any one of the aforementioned embodiments E1.#, wherein The instruction instructs the RASL images within the sequence of images to be coded in a manner that does not include the predetermined set of one or more coding tools such that For a first subset of one or more coding tools in the predetermined set of one or more coding tools, a first characteristic is applied to all RASL images excluding the one or more coding tools the manner of the first subset of one or more coding tools in the predetermined set is coded, and For a second subset of one or more coding tools in the predetermined set of one or more coding tools, a second characteristic applies to all RASL images, or to all RASL images excluding The manner in which the second subset of one or more coding tools in the predetermined set of one or more coding tools is coded is such that the first subset and the second subset are mutually disjoint.

E1.28. The video encoder of embodiment E1.27, wherein the first characteristic and/or the second characteristic is selected from causing the reference picture to precede a CRA picture with which it is associated in decoding order, serves as a reference image for temporal motion vector prediction for one of the following images, and Does not belong to a highest time layer.

E1.29. The video encoder of embodiment E1.27 or higher, wherein The first subset includes one or more of a decoder-side motion vector improvement tool and a temporal motion vector prediction tool.

E1.30. The video encoder of embodiment E1.29, wherein the first characteristic is selected from causing the reference picture to precede a CRA picture with which it is associated in decoding order, serves as a reference image for temporal motion vector prediction for one of the following images, and Does not belong to a highest time layer.

E2.1. A video encoder for encoding a video into a data stream, configured with Encode an instruction into the data stream [e.g., using sps_extra_ph_bit_present_flag and ph_extra_bit, or using gci_rasl_pictures_tool_contraint_flag], by image in an image sequence of the video, globally for individual images, or on a per-slice basis Indicates whether the respective image is coded in a manner that does not include a predetermined set of one or more coding tools that includes a prediction tool based on a cross-component linear model [e.g., as an graphical indication, This makes it possible to view RASL images where the potential drift is sufficiently low].

E2.2. The video encoder as in any one of the preceding embodiments E2.#, wherein the encoder is configured to support intra prediction block coding mode and inter prediction block coding mode.

E2.3. The video encoder of any of the preceding embodiments E2.#, wherein according to the prediction tool based on the cross-component linear model, The chrominance component of an image block is predicted from the lightness component of the image block using a linear model whose parameters are derived from the lightness and color in one of the encoded neighborhoods of the image block. Judgment by extreme value.

E2.4. The video encoder of any of the preceding embodiments E2.#, wherein the set of one or more coding tools further includes A luma tone mapping and chroma residual scaling prediction tool.

E2.5. The video encoder of embodiment E2.4, wherein based on the luma tone mapping and chroma residual scaling prediction tools, Luminance component prediction and a luminance component residual encoding for a predetermined image are performed with a coded luma tone scale, and a rendered luma tone scale is mapped to the coded luma tone by a luma tone map on a scale to obtain a coded luma tonal scaled version of one of the reconstructed luma components of the predetermined image, A chroma residual scaling factor for an image block of the predetermined image is derived from the coded luma tone scaled version of the reconstructed luma component of the predetermined image within a neighborhood of the image block one average value to determine, and A chroma residual signal encoded from the data stream for the image block is scaled according to the chroma residual scaling factor and used to correct an intra-frame chroma prediction signal for the image block.

E2.6. The video encoder of any of the preceding embodiments E2.#, wherein the set of one or more coding tools further includes An optical flow tool.

E2.7. The video encoder of embodiment E2.6, wherein the optical flow tool is A translational inter prediction signal for improving a predetermined inter prediction block by means of optical flow based analysis.

E2.8. The video encoder of any of the preceding embodiments E2.#, wherein the set of one or more coding tools further includes A decoder-side motion vector improvement tool.

E2.9. The video encoder of embodiment E2.8, wherein the decoder-side motion vector improvement tool is for improving a signaled motion vector by performing a best match search among motion vector candidates at and around a signaled motion vector coded in the data stream (402). , for performing inter prediction on a predetermined inter predicted block (10d) from a reference image (404).

E2.9a is the video encoder of embodiment E2.9, wherein the decoder-side motion vector improvement tool is configured with The best match search is performed using one of the decoded neighborhoods of the inter-predicted block relative to the reference image.

E2.9b. The video encoder of embodiment E2.8, wherein the decoder-side motion vector improvement tool is configured with Improving a pair of signaled motion vectors by performing a best match search among motion vector pairing candidates including a pair of signaled motion vectors encoded in the data stream (402) and surrounding the pair of signaled motion vectors Vectors for mapping a predetermined inter-frame bi-predicted block (10d) from a pair of reference pictures (404) temporally placed in front of and behind a picture of the predetermined inter-frame bi-predicted block (10d). (10d) Perform inter-frame prediction.

E2.10. The video encoder of any preceding embodiment E2.#, wherein the set of one or more coding tools further includes A temporal motion vector prediction tool.

E2.11. The video encoder of embodiment E2.10, wherein according to the temporal motion vector prediction tool, formation of the motion vector candidate list for the inter-predicted block includes generating motion vector candidates from a previously encoded image. who added.

E2.12. The video encoder of embodiment E2.11, wherein according to the temporal motion vector prediction tool, the motion vector candidate list for the inter prediction block is formed to include the motion vector predictor pointing to the free one. Motion vector candidate supplementation is performed on a block of a previously encoded image.

E2.13. The video encoder of embodiment E2.12, wherein the motion vector predictor includes a temporal motion vector predictor.

E2.14. The video encoder of any preceding embodiment D1.#, wherein the instruction is included in one of the following: one or more image parameter sets referenced by the images in the image sequence, one of the image headers of the images in the sequence of images, and A slice header for one of the slices of the images in the image sequence.

E2.15. The video encoder of any preceding embodiment E2.#, wherein the instruction is included in the following: A set of image parameters referenced by the images in the sequence of images, wherein the sets of image parameters include: at least one first set of image parameters indicating a picture that references the at least one first set of image parameters The image is coded in a manner that does not include the predetermined set of one or more coding tools; and at least one second set of image parameters indicating that images referencing the at least one second set of image parameters may be used one of the predetermined sets of one or more coding tools is coded, or Image parameter sets referenced by the images in the image sequence, wherein the image parameter sets include: at least a first image parameter set indicating and referencing the at least one first image parameter set a RASL image associated with the image is coded in a manner that does not include one of the predetermined set of one or more coding tools; and at least one second image parameter set indicating and referencing the at least one second image The RASL image associated with the image of the parameter set is coded in one of the predetermined sets, possibly using one or more coding tools.

E2.16. The video encoder of embodiment E2.15, wherein the indication includes a syntax element within an extended syntax portion of the image parameter set [eg, using sps_extra_ph_bit_present_flag and ph_extra_bit].

E2.17. The video encoder of embodiment E2.16, wherein a length of the extended syntax part of the image parameter set is specified in a sequence of the data stream or a video parameter set.

E2.18. The video encoder of any preceding embodiment E2.#, wherein the indication is a syntax element in an extension of one of the following: one of the image headers of the images in the sequence of images, and/or A slice header for one of the slices of the images in the sequence of images, One of the lengths of the extension [eg, NumExtraPhBits] is indicated in an image or sequence of the data stream or in a video parameter set.

E2.18a. The video encoder of embodiment E2.18, wherein the syntax element indicates whether an image to which the syntax element belongs [for example, to which the image header or slice header relates] is coded in a manner that does not include one of the predetermined set of one or more coding tools, or Whether the RASL image associated with the image to which the syntax element belongs is coded in a manner that does not include one of the predetermined set of one or more coding tools.

E2.19. The video encoder of any preceding embodiment E2.#, wherein the encoder is configured to support reference picture resampling.

E2.22. The video encoder of embodiment E2.19, wherein based on the reference picture resampling, one of the reference pictures of an inter-frame prediction block is subjected to sample resampling in order to bridge the reference picture and therein A scaling window size deviation or sample resolution deviation between images containing the inter-predicted block provides an inter-prediction signal for the inter-predicted block.

E2.23. The video encoder as in any one of the preceding embodiments E2.#, wherein the set of one or more coding tools includes [for example, 200, 300, 400, 500] one or more first application-specific coding tools, each of which is applied for a predetermined block depending on one or more coding options communicated in the data stream for the predetermined block, and Relevant to a coding tool other than the respective coding tool, and/or One or more second application-specific coding tools, each of which is applied to a predetermined block depending on a size of the predetermined block.

E2.24. The video encoder as in any one of the preceding embodiments E2.#, wherein the set of one or more coding tools includes One or more explicitly applied coding tools, each of which is applied to a predetermined block in dependence on a syntax element coded into the data stream for specifically transmitting The application of each coding tool to the predetermined block.

E2.25. The video encoder of embodiment E2.24, wherein the encoder is configured to encode the syntax element into the data stream for blocks within: one or more encoding tools The predetermined set of images or slices that are signaled to exclude from self-encoding; and the predetermined set of one or more coding tools that are not signaled to exclude from self-encoding.

E2.26. The video encoder of embodiment E2.24, wherein the encoder is configured to signal regions within images or slices excluded from encoding only for the predetermined set of one or more encoding tools block encodes the syntax element into the data stream.

E2.27. The video encoder of any one of the preceding embodiments E2.24 or above, wherein the prediction tool based on the cross-component linear model belongs to the one or more one or more explicitly applied coding tools.

E2.28. The video encoder of any one of the preceding embodiments E2.16 or above, wherein the syntax element is a bit that collectively indicates that one of the predetermined sets of one or more coding tools is not included. All coding tools.

E2.29. The video encoder as in any one of the preceding embodiments E2.#, wherein the set of one or more coding tools includes One or more enableable coding tools, each of which with respect to its application to an image block, may be enabled on an image or slice basis by assembly signaling within the data stream.

E2.30. The video encoder of any one of the preceding embodiments E2.#, configured to comply with instructions as an encoding constraint when encoding the video into the data stream.

B1.1. A data stream in which a video is encoded, which contains An indication [e.g., gci_rasl_pictures_tool_constraint_flag] that is valid for an image sequence of the video and indicates that the RASL images within the image sequence were processed in a manner that does not include a predetermined set of one or more coding tools Coding [e.g. as a commitment to make it known to the encoder that open-GOP switching is in the RASL diagram by concatenating separately coded open-GOP versions of the video coded at different spatial resolutions and/or different SNRs does not cause excessive drift in the image].

B1.2 The data stream of Embodiment B1.1 is generated by the encoder of any one of Embodiments E1.#.

B2.1. A data stream in which a video is encoded, which contains An indication [e.g., using sps_extra_ph_bit_present_flag and ph_extra_bit, or using gci_rasl_pictures_tool_contraint_flag] that indicates whether the individual image is present, per image in a sequence of images in the video, globally for individual images, or on a per-slice basis. Coded in a manner that does not include a predetermined set of one or more coding tools, the predetermined set containing a prediction tool based on a cross-component linear model [e.g. as a graphical indication that makes it possible to view a RASL plot The potential drift of the system is low enough].

B2.2 The data stream of Embodiment B2.1 is generated by the encoder of any one of Embodiments E2.#.

M. A method performed by any one of the above decoders and encoders.

P. A computer program having a program code for executing the method of embodiment M when the program is executed on a computer.

Although some aspects have been described in the context of apparatus, it is obvious that these aspects also represent descriptions of corresponding methods, where blocks or means correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of corresponding blocks or items or features of the corresponding apparatus. Some or all of the method steps may be performed by (or using) hardware devices, such as microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important method steps may be performed by such a device.

The data stream of the present invention may be stored on a digital storage medium or may be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium (such as the Internet).

Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. Implementation may be performed using a digital storage medium such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM or flash memory having electronically readable control signals stored thereon, such as The electronically readable control signals cooperate (or can cooperate) with the programmable computer system to enable execution of the respective methods. Therefore, the digital storage medium can be computer readable.

Some embodiments according to the invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system such that one of the methods described herein is performed.

Generally, embodiments of the invention may be implemented as a computer program product having program code operative to perform one of the methods when the computer program product is run on a computer. The program code may, for example, be stored on a machine-readable carrier.

Other embodiments include a computer program stored on a machine-readable carrier for performing one of the methods described herein.

In other words, an embodiment of the method of the invention is therefore a computer program having code for performing one of the methods described herein when the computer program is run on a computer.

Therefore, another embodiment of the method of the invention is a data carrier (or digital storage medium, or computer readable medium) comprising recorded thereon a computer program for performing one of the methods described herein. Data carriers, digital storage media or recording media are usually tangible and/or non-transitory.

Therefore, another embodiment of the method of the invention is a data stream or a signal sequence representing a computer program for performing one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted over a data communications connection (eg, over the Internet).

Another embodiment includes processing means, such as a computer or programmable logic device configured or adapted to perform one of the methods described herein.

Another embodiment includes a computer having installed thereon a computer program for performing one of the methods described herein.

Another embodiment in accordance with the invention includes a device or system configured to transmit (eg electronically or optically) a computer program for performing one of the methods described herein to a receiver. For example, the receiver can be a computer, a mobile device, a memory device, etc. The device or system may, for example, include a file server for transmitting computer programs to the receiver.

In some embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

The devices described herein may be implemented using hardware devices or using computers or using a combination of hardware devices and computers.

The apparatus described herein, or any component of the apparatus described herein, may be implemented, at least in part, in hardware and/or in software.

The methods described herein may be performed using hardware devices or using computers or using a combination of hardware devices and computers.

Any component of a method described herein or an apparatus described herein may be performed, at least in part, by hardware and/or software.

The embodiments described above merely illustrate the principles of the invention. It is understood that modifications and variations to the configurations and details described herein will be apparent to those skilled in the art. Therefore, it is intended that the claims be limited only by the scope of the claims that follow and not by the specific details presented by the description and explanation of the embodiments herein. References

[1] ISO/IEC JTC 1, Information technology — Dynamic adaptive streaming over HTTP (DASH) — Part 1: Media presentation description and segment formats, ISO/IEC 23009-1, 2012 (and subsequent editions).

[2] J. De Cock, Z. Li, M. Manohara, A. Aaron. "Complexity-based consistent-quality encoding in the cloud." 2016 IEEE International Conference on Image Processing (ICIP). IEEE, 2016

[3] DASH Industry Forum Implementation Guidelines. [Online]. Available: https://dashif.org/guidelines/

[4] ITU-T and ISO/IEC JTC 1, Advanced Video Coding for generic audio-visual services, Rec. ITU-T H.264 and ISO/IEC 14496-10 (AVC), May 2003 (and subsequent editions).

[5] ITU-T and ISO/IEC JTC 1, “High Efficiency Video Coding,” Rec. ITU-T H.265 and ISO/IEC 23008-2 (HEVC), April 2013 (and subsequent editions).

[6] Y. Yan, M. Hannuksela, and H. Li. "Seamless switching of H. 265/HEVC-coded dash representations with open GOP prediction structure." 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 2015 .

[7] ITU-T and ISO/IEC JTC 1, “Versatile video coding”, Rec. ITU-T H.266 and ISO/IEC 23090-3 (VVC), August 2020.

[8] V. Baroncini and M. Wien, “VVC verification test report for UHD SDR video content”, doc. JVET-T2020 of ITU-T/ISO/IEC Joint Video Experts Team (JVET), 21th meeting: October 2020.

[9] D. Luo, V. Seregin, W. Wan. “Description of Core Experiment 1 (CE1): Reference picture resampling filters“, doc. JVET-Q2021 of ITU-T/ISO/IEC Joint Video Experts Team (JVET ), 15th meeting: July 2019

[10] H. Schwarz, D. Marpe, and T. Wiegand, “Analysis of hierarchical B pictures and MCTF”, ICME 2006, IEEE International Conference on Multimedia and Expo, Toronto, Ontario, Canada, July 2006.

[11] Y.-K. Wang et al., “The High-Level Syntax of the Versatile Video Coding (VVC) Standard” IEEE Trans. Circuits Syst. Video Technol., in press

[12] H. Yang et al., “Subblock based Motion Derivation and Inter-Prediction Refinement in Versatile Video Coding Standard”, IEEE Trans. Circuits Syst. Video Technol., in press

[13] W.-J. Chien et al., “Motion Vector Coding and Block Merging in Versatile Video Coding Standard”, IEEE Trans. Circuits Syst. Video Technol., in press

10, 10a, 10b: Image block 10c, 10d: Predetermined inter-frame prediction block 12: Predetermined image 100: Cross-component linear model tool 102: Chroma component 104: Luminance component 106: Linear model 108: Parameter calculation Guide 110: Luminance and chroma extremes 112: Decoded neighborhood 120: Any type of prediction 122: Decoding 124: Reconstructed luma component 126: Sample 128: Operation 200: Luminance tone mapping and chroma residual scaling prediction tools 202: Luminance component prediction 204: Luminance component residual decoding 206: Inter-frame prediction 208: Coding luminance tone scale 210: Rendering luma tone scale 212: Luminance tone mapping 214: Writing luma tone scale version 216: Chroma residual scaling factor 220: Average 222: Neighborhood 224: Chroma residual signal 226: Scaling 228: Correction 230: In-box chroma prediction signal 240: Inverse luma tone mapping 300: Optical flow tools 302: Motion vectors 304: Reference image 306: Area 350: Coding Tool 352, 354: Arrow 353: Syntax Element 356: Application Decision 358: Startup Decision 400: Decoder Side Motion Vector Improvement Tool 402: Signaled Motion Vector 404: Reference Image 406: Replacement 500: Temporal Motion Vector Prediction Tool 502: Previously decoded image 504: Motion vector predictor 506, A84: Block 508, 510: Motion vector 512: Motion vector candidate list A10: Device or encoder A12, A12': Image A14: Data stream A20 : Decoder A22: Prediction residual signal former A24: Prediction residual signal A24', A24''': Spectral domain prediction residual signal A24'', A24'''': Prediction residual signal A26: Prediction signal A28: Transformer A32: Quantizer A34: entropy coder A36: prediction stage A38, A52: dequantizer A40, A54: inverse transformer A42, A56: combiner A46: reconstructed signal A50: entropy decoder A58, A44: prediction module A80 : In-frame coding block A82: Between-frame coding block A84: Blocks C ₁ , C ₂ : Chroma component

The advantageous aspects of this application are the subject of the subsidiary claims. Preferred embodiments of the present application are described below with respect to the drawings. Figure 1 illustrates a video encoder according to one embodiment, Figure 2 illustrates a video decoder according to an embodiment, Figure 3 illustrates a block-based residual coding scheme according to one embodiment, Figure 4 illustrates a video data stream including two segments, according to one embodiment, Figure 5 illustrates an operation scheme of an optical flow tool according to an embodiment, Figure 6 illustrates an application solution for a coding tool according to one embodiment, 7 illustrates an operation scheme of a temporal motion vector prediction tool according to one embodiment, Figure 8 illustrates the operation scheme of the decoder side motion vector improvement tool according to one embodiment, Figure 9 illustrates the operation scheme of the cross-component linear model tool according to one embodiment, Figure 10 illustrates the operation scheme of the luma mapping and chroma scaling tools according to one embodiment. Figure 11 illustrates another operation scheme of the luma mapping and chroma scaling tools according to one embodiment. Figure 12 illustrates an example of prediction error in an open GOP scenario.

A12':Image

A14: Data streaming

A20:Decoder

A24": Predict residual signal

A50: Entropy decoder

A52: Dequantizer

A54: Inverse converter

A56: Combiner

A58: Prediction module

Claims (6)

A video decoder configured to: decode a supplemental enhancement information (SEI) message from a data stream, the SEI message indicating that for all random access skips within a sequence of images Coding of RASL images is constrained in a manner that does not include a predetermined set of one or more coding tools, where the predetermined set includes: a prediction tool (100) based on a cross-component linear model, and a Decoder side motion vector improvement tool (400), wherein the image sequence includes at least one random access skip preamble (RASL) picture and a clean random access (CRA) associated with the at least one RASL picture. ) image. The video decoder of claim 1, wherein the supplemental enhancement information (SEI) message further indicates that coding for the at least one RASL image is constrained such that there is no moving image for the at least one RASL image. The collocated reference picture for quantified temporal motion vector prediction (TMVP) or sub-block temporal motion vector prediction (sbTMVP) precedes the CRA picture associated with the RASL picture in decoding order. The video decoder of claim 1 or 2, wherein the video decoder is configured to support reference picture resampling, wherein based on the reference picture resampling, once a reference picture of the inter-frame prediction block is is subjected to resampling to bridge a scaling window size deviation or sample resolution deviation between the reference image and an image containing the inter-prediction block to provide the inter-prediction block An inter-frame prediction signal. A video encoder configured to encode a supplemental enhancement information (SEI) message into a data stream, the SEI message indicating a skip preamble ( RASL) images are constrained in a manner that does not include a predetermined set of one or more coding tools, Wherein, the predetermined set includes: a prediction tool (100) based on a cross-component linear model, and a decoder-side motion vector improvement tool (400), wherein the image sequence includes at least one random access skip preamble ( RASL) image and a clean random access (CRA) image associated with the at least one RASL image. The video encoder of claim 4, wherein the supplemental enhancement information (SEI) further indicates that coding for the at least one RASL image is constrained such that there is no moving image for the at least one RASL image. The collocated reference picture for quantified temporal motion vector prediction (TMVP) or sub-block temporal motion vector prediction (sbTMVP) precedes the CRA picture associated with the RASL picture in decoding order. The video encoder of claim 4 or 5, wherein the video encoder is configured to support reference picture resampling, wherein based on the reference picture resampling, a reference picture of the inter-frame prediction block is subjected to resampling to bridge a scaling window size deviation or sample resolution deviation between the reference image and an image containing the inter-prediction block to provide the inter-prediction block An inter-frame prediction signal.