RU2772795C1 - Reference picture resampling method with offset in video bitstream - Google Patents

Abstract

FIELD: reference image resampling.

SUBSTANCE: invention relates to means for resampling a reference image with an offset in a video bitstream. A flag indicating that the match window is not used for reference picture resampling is obtained. Based on a flag indicating that the match window is not used for resampling the reference image, it is determined whether the size of the resampling image is signaled. Based on the determination that the size of the oversampling image is signaled, an oversampling ratio is determined based on the size of the oversampling image. Based on the determination that the oversampling image size is not signaled, an oversampling ratio is determined based on the output image size. The reference image is resampled for the current image using the resampling factor.

EFFECT: improving the efficiency of video decoding.

10 cl, 11 dwg, 1 tbl

Description

Cross-reference to related application

[1] This application claims priority of U.S. Provisional Application No. 62/905,319, filed September 24, 2019, and U.S. Patent Application No. 17/022,727, filed September 16, 2020, the contents of which are incorporated herein in their entirety by links.

FIELD OF TECHNOLOGY TO WHICH THE INVENTION RELATES

[2] The invention relates to the encoding and decoding of video data, and more particularly to reference picture resampling signaling with an indication of the size of the resampling picture.

BACKGROUND OF THE INVENTION

[3] Encoding and decoding of video data may be performed using motion-compensated inter-picture prediction. Uncompressed digital video may include a sequence of images, each of which has a spatial size of, for example, 1920×1080 luma samples and their associated chrominance samples. The image sequence may have a fixed or variable image rate (informally also referred to as frame rate), such as 60 images per second, or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (resolution of 1920x1080 luma samples at 60Hz frame rate) requires about 1.5Gbps of bandwidth. An hour of such video requires more than 600 GB of storage.

[4] One of the purposes of encoding and decoding video data may be to reduce redundancy in the input video signal by compressing. Compression can help to alleviate the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. You can use both lossless and lossy compression, as well as a combination of both. Lossless compression refers to techniques for reconstructing an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the discrepancy between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used for video. The amount of distortion allowed depends on the application; for example, users of some custom streaming applications may tolerate greater distortion than users of broadcast television applications. Higher allowed/tolerable distortion allows a higher compression ratio to be achieved.

[5] The video encoder and decoder may use techniques from several broad categories including, for example, motion compensation, transformation, quantization, and entropy coding, some of which will be presented below.

[6] Historically, video encoders and decoders have usually worked with a given image size, which in most cases was set and remained constant for a coded video sequence (CVS, coded video sequence), a group of pictures (GOP, Group of Pictures) or similar time frame with multiple images. For example, in MPEG-2, systems are known that change the horizontal resolution (and hence the image size) depending on factors such as activity in the frame, but only in I-pictures, hence typical for GOP. Resampling of reference pictures to use different resolutions in CVS is known, for example, from ITU-T Rec. H.263, Annex P. However, here the image size is not changed, only the reference pictures are resampled, with the result that only parts of the image can potentially be used (in the case of downsampling). sampling) or only parts of the captured frame (in the case of upsampling). In addition, Annex Q of the H.263 standard allows single macroblock resampling by a factor of 2 (in each dimension) in the up or down direction. Again, the image size remains the same. The macroblock size in H.263 is fixed and therefore does not need to be signaled.

[7] Image resizing for predicted images has become widespread in modern video coding. For example, VP9 allows you to resample the reference image and change the resolution for the entire image. Likewise, some proposals made in relation to VVC (including, for example, Hendry, et. al, "On adaptive resolution change (ARC) for VVC", document JVET-M0135-v1, January 9-19, 2019, are in full included herein) allow entire reference pictures to be resampled at various higher or lower resolutions. This document proposes to encode the various possible resolutions in a sequence parameter set and refer to the syntax elements for each picture in the picture parameter set.

SUMMARY OF THE INVENTION

[8] In an embodiment of the invention, there is provided a method for decoding an encoded video bitstream using at least one processor, including: obtaining a flag indicating that a match window is not used for resampling a reference picture; based on a flag indicating that the match window is not used for resampling the reference picture, determining whether the size of the resampling picture is signaled; based on determining that the size of the oversampling image is signaled, determining an oversampling ratio based on the size of the oversampling image; based on determining that the resampling image size is not signaled, determining an oversampling ratio based on the output image size; and performing resampling of the reference picture on the current picture using the resampling factor.

[9] In an embodiment of the invention, there is provided an apparatus for decoding an encoded video bitstream, comprising: at least one memory configured to store a program code and at least one processor configured to read the program code and operate in accordance with the instructions of the program code. wherein the program code includes: an acquisition code configured to cause at least one processor to receive a flag indicating that the match window is not used for the resampling reference picture; a first determination code configured to cause at least one processor to determine, based on a flag indicating that a matching window is not used for the resampling reference image, whether the resampling image size is signaled; a second determination code configured to cause at least one processor to determine, based on the determination that the resampling image size is signaled, an oversampling factor based on the resampling image size; a third determination code configured to allow at least one processor to determine, based on a determination that the oversampling image size is not signaled, an oversampling factor based on the output image size, and an execution code configured to be executed by at least one processor , resampling the reference picture for the current picture using the resampling factor.

[10] In an embodiment, a computer-readable medium is provided that stores instructions comprising one or more instructions that, when executed by one or more processors of a device for decoding an encoded video bitstream, cause one or more processors to obtain a flag indicating that the window is matching is not used for reference picture resampling; based on a flag indicating that the match window is not used for resampling the reference picture, determining whether the size of the resampling picture is signaled; based on determining that the resampling image size is signaled, determining an oversampling ratio based on the resampling image size; based on determining that the resampling image size is not signalled, determining a resampling ratio based on the output image size, and performing resampling of the reference image on the current image using the resampling ratio.

BRIEF DESCRIPTION OF THE DRAWINGS

[11] Additional features, nature and various advantages of the present invention will be apparent from the following detailed description and the accompanying drawings.

[12] FIG. 1 is a schematic illustration of a simplified block diagram of a communication system in accordance with an embodiment of the invention.

[13] FIG. 2 is a schematic illustration of a simplified block diagram of a communication system in accordance with an embodiment of the invention.

[14] FIG. 3 is a schematic illustration of a simplified block diagram of a decoder according to an embodiment of the invention.

[15] FIG. 4 is a schematic illustration of a simplified block diagram of an encoder in accordance with an embodiment of the invention.

[16] FIG. 5 is a schematic illustration of options for ARC parameter signaling in accordance with an embodiment of the invention.

[17] FIG. 6A-6B are schematic illustrations of examples of syntax tables in accordance with an embodiment of the invention.

[18] FIG. 7 is a schematic illustration of image size signaling and matching window in PPS in accordance with an embodiment of the invention.

[19] FIG. 8 is a schematic illustration of image size signaling and matching window in PPS according to an embodiment of the invention.

[20] FIG. 9 is a flowchart of an example of a method for decoding an encoded video bitstream according to an embodiment of the invention.

[21] FIG. 10 is a schematic illustration of a computer system in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

[22] FIG. 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the present invention. The system (100) may include at least two terminals (110-120) interconnected via a network (150). For unidirectional data transmission, the first terminal (110) may encode video data at the local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the encoded video data of another terminal from the network (150), decode the encoded data, and display the recovered video data. One-way communication may be inherent in media service applications and the like.

[23] FIG. 1 illustrates a second pair of terminals (130, 140) supporting bidirectional transmission of coded video data, which may occur during video conferences, for example. For bi-directional data transmission, each terminal (130, 140) may encode video data captured at a local location for transmission to another terminal over a network (150). Each terminal (130, 140) can also receive the encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[24] In the example shown in FIG. 1, terminals (110-140) can be illustrated as servers, personal computers, and smartphones, but this does not limit the principles of the present invention. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or equipment used for video conferencing. Network (150) represents any number of networks that carry encoded video data between terminals (110-140), including, for example, wired and/or wireless networks. The communication network (150) allows data to be exchanged in a circuit-switched and/or packet-switched mode. Examples of networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may not be relevant to the present invention unless specifically noted below.

[25] FIG. 2 shows, as an example of the application of the present invention, the placement of a video encoder and a video decoder in a streaming environment. The present invention may be equally applicable to other video processing applications including, for example, videoconferencing, digital television, storage of compressed video on digital media such as CD, DVD, memory card, and the like.

[26] The streaming system may include a capture subsystem (213), which may include a source (201) of video data, such as a digital camera, creating, for example, an uncompressed stream (202) of video samples. This stream (202) of samples, depicted with a thick line to emphasize the large amount of data compared to the bitstreams of encoded video data, can be processed by an encoder (203) connected to a camera (201). The encoder (303) may include hardware, software, or a combination thereof to provide or implement aspects of the present invention, as described in more detail below. The encoded video data bitstream (204), depicted as a thin line to emphasize the smaller amount of data compared to the sample stream, may be stored on the streaming server (205) for later use. One or more streaming clients (206, 208) can access the streaming server (205) to extract copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210) that decodes an incoming copy of the encoded video bitstream (207) and produces an outgoing stream (211) of video samples that can be displayed on a display (212) or other visualization device (not shown) . In some streaming systems, bitstreams (204, 207, 209) of video data may be encoded according to certain video coding/compression standards. Examples of these standards include ITU-T Rec. H.265. A video coding standard is under development, informally known as Universal Video Coding or VVC (Versatile Video Coding). The present invention can be used in the context of VVC.

[27] FIG. 3 shows a functional diagram of a video decoder (210) according to an embodiment of the present invention.

[28] The receiver (310) may receive one or more encoded video sequences for decoding by video decoder (210), in the same or another embodiment of the invention, one encoded video sequence at a time, with the decoding of each encoded video sequence independent of other encoded video sequences. The encoded video sequence may be received from a channel (312), which may be a hardware/software link to a storage device where the encoded video data is stored. The receiver (310) may receive encoded video data with other data, such as encoded audio data and/or auxiliary data streams, which may be redirected to their respective utilizing entities (not shown). The receiver (310) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (315) can be connected between the receiver (310) and the entropy decoder/analyzer (320) (hereinafter "analyzer"). When the receiver (310) receives data from a storage/redirector with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (315) may not be needed or may be small. For use in best effort packet networks such as the Internet, a buffer (315) may be required, which may be relatively large and may preferably have an adaptive size.

[29] The video decoder (210) may include an analyzer (320) for reconstructing the symbols (321) from the entropy-coded video sequence. The categories of these symbols include information used to control the operation of the decoder (210) and possibly information to control a visualization device such as a display (212) that is not an integral part of the decoder but may be connected to it, as shown in FIG. 3. The control information for the rendering device(s) may be Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The analyzer (320) may analyze/entropy decode the received encoded video sequence. The encoding of the video sequence may be in accordance with a video coding technology or standard, and may follow principles known to those skilled in the art, including variable run length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The analyzer (320) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOP, Groups of Pictures), pictures, sub-images, tiles, slices, modules, macroblocks, coding tree units (CTU, Coding Tree Unit), coding units (CU, Coding Unit), blocks, transformation units ( TU, Transform Unit), prediction units (PU, Prediction Unit), etc. A tile can point to a rectangular area of a CU/CTU in a specific column and row of a tile in an image. A module can point to a rectangular area of CU/CTU rows within a specific tile. A slice may point to one or more image units that are contained in a NAL unit. A sub-image may point to a rectangular area of one or more slices in an image. The entropy decoder/analyzer can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[30] The analyzer (320) may perform an entropy decoding/analysis operation of the video sequence obtained from the buffer (315) to create symbols (321).

[31] Several different modules may be used for symbol reconstruction (321), depending on the type of encoded video image or parts thereof (eg, intra- and inter-coded images, intra- and inter-coded blocks) and other factors. Which modules are used and how can be determined by the subgroup management information obtained from the encoded video sequence by the analyzer (320). The flow of such subgroup control information between the analyzer (320) and the modules is not shown hereinafter for simplicity.

[32] In addition to the previously mentioned functional blocks, video decoder 210 can be fundamentally divided into several functional modules, as described below. In practical implementation under commercial constraints, many of these modules interact closely with each other and can, at least partially, be built into one another. However, for the purpose of describing the present invention, a fundamental division into the following functional modules is appropriate.

[33] The first module is the scaling/inverse transform module (351). The scaling/inverse transform module (351) receives the quantized transform coefficient as well as control information including the transform used, block size, quantization factor, quantization scaling matrices, etc. as character(s) (321) from the analyzer (320). It can output blocks containing sample values that can be entered into the aggregator (355).

[34] In some cases, the output samples of the scaling / inverse transform module (351) may refer to an intra-coded block, that is, a block that does not use predicted information from previously reconstructed images, but can use predicted information from previously reconstructed parts of the current image . Such predicted information may be provided by the intra image prediction module (352). In some cases, the intra image prediction module (352) generates a block of the same size and shape as the block to be reconstructed using the already reconstructed environment information extracted from the current (partially reconstructed) image (358). The aggregator (355) in some cases adds, for each sample, the prediction information generated by the intra prediction module (352) to the output sample information provided by the scaling/inverse transform module (351).

[35] In other cases, the output samples of the scaling/inverse transform module (351) may refer to an intra-coded block, possibly with motion compensation. In such a case, the motion compensation prediction module (353) may access the reference picture memory (357) to retrieve samples used for prediction. After motion compensation has been applied to the extracted samples according to the symbols (321) associated with the block, these samples can be added by an aggregator (355) to the output of the scaler/demap module (in this case referred to as residual samples or residual signal) to generate output information. counts. Reference picture memory addresses from where the motion-compensated prediction module retrieves prediction samples may be driven by motion vectors available to the motion-compensated prediction module in the form of symbols (321), which may have, for example, X, Y, and reference picture components. . Motion compensation may also include interpolation of sample values retrieved from reference picture memory when accurate subsample motion vectors, motion vector prediction mechanisms, and so on are used.

[36] Various loop filtering methods can be applied to the aggregator (355) output samples in the loop filter module (356). The video data compression technologies may include loop filter technologies controlled by parameters included in the encoded video bitstream and available to the loop filter module (356) as symbols (321) from the analyzer (320), but may also be responsive to metainformation obtained during decoding the previous (in decoding order) parts of the encoded image or encoded video sequence, as well as responding to previously reconstructed and contour filtered sample values.

[37] The loop filter module (356) can output a stream of samples that can be fed to the renderer (212) as well as stored in the reference picture memory 557 for use in future inter-image prediction.

[38] Some encoded pictures, when fully reconstructed, can be used as reference pictures for future prediction. For example, when an encoded picture is fully reconstructed and identified as a reference picture (e.g., by analyzer (320)), the current reference picture (358) may become part of the reference picture buffer memory (357), and fresh memory for the current picture may be re-allocated prior to reconstruction. next encoded picture.

[39] Video decoder 210 may perform decoding operations according to a predetermined video data compression technology documented in a standard such as ITU-T Rec. H.265. The encoded video sequence may follow the syntax given by the video compression technology or standard used, in the sense that the encoded video sequence may follow the syntax of the video compression technology or standard specified in the video compression technology document or standard, and in particular in the profile document. . It may also be necessary for consistency that the complexity of the encoded video sequence remains within the boundaries given by the level of technology or video compression standard. In some cases, the levels limit the maximum image size, the maximum frame rate, the maximum sampling rate for reconstruction (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the levels, in some cases, may be further limited by the specifications of the Hypothetical Reference Decoder (HRD) and the metadata for managing the HRD buffer signaled in the encoded video sequence.

[40] According to an embodiment of the invention, the receiver (310) may receive additional (redundant) data with encoded video data. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (210) to correctly decode the data and/or more accurately reconstruct the original video data. The side data may be, for example, temporal enhancement levels, spatial enhancement levels or signal noise ratio (SNR) enhancement levels, redundant slices, redundant images, forward error correction codes, and so on.

[41] FIG. 4 is a block diagram of a video encoder (203) according to an embodiment of the present invention.

[42] The encoder (203) may receive video data samples from a video data source (201) (which is not part of the encoder) that may capture the video image(s) to be encoded by the video encoder (203).

[43] The video data source (201) may provide the source video sequence to be encoded by the video encoder (203) in the form of a digital video data sample stream of any suitable bit depth (eg: 8 bits, 10 bits, 12 bits, ...), any color space (eg , VT.601 Y CrCB, RGB, …) and any suitable sampling structure (eg Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the source (201) of video data may be a storage device where previously prepared video is stored. In a videoconferencing system, the video data source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that give a sense of movement when viewed in sequence. The images themselves may be organized as a spatial array of pixels, where each pixel may contain one or more samples depending on the sampling structure used, color space, and so on. It is not difficult for a person skilled in the art to understand the relationship between pixels and samples. The following description is devoted to readings.

[44] According to an embodiment of the invention, the encoder (203) may encode and compress images of the source video sequence into the encoded video sequence (443) in real time or subject to any other time constraints imposed by the application. Setting the proper coding rate is one of the functions of the controller (450). The controller controls other functional modules as described below and is functionally connected to them. The connection is not shown for simplicity. The parameters set by the controller may include parameters related to frequency adjustment (picture skip, quantizer, lambda value used in rate-distortion optimization techniques, etc.), image size, group of images (GOP, group of pictures), maximum motion vector search area, etc. One skilled in the art will appreciate that the controller (450) may have other suitable functions related to the video encoder (203) optimized for a particular system design.

[45] Some video encoders may operate in a "coding loop" as understood by one skilled in the art. As a very simplified description, for example, an encoding loop may include an encoder (430) coding portion (herein referred to as a "source encoder") (responsible for creating symbols, for example, a stream of symbols, based on the input image to be encoded and the reference image(s). (s)) and a (local) decoder (433) built into the encoder (203) that reconstructs the symbols to produce sample data as a (remote) decoder would (because any compression of symbols into the encoded video bitstream is lossless in video compression technologies discussed in this description). The reconstructed stream of samples enters the memory (434) reference images. Since decoding a stream of symbols produces bit-accurate results that are independent of the position (local or remote) of the decoder, the contents of the reference picture buffer will also be the same bit-accuracy for the local encoder and the remote encoder. In other words, the prediction portion of the encoder "sees" as reference picture samples exactly the same sample values as the decoder would "see" if using prediction during decoding. This fundamental principle of reference picture synchronism (and eventually drift if synchronism cannot be maintained due to, for example, channel errors) is well known to one skilled in the art.

[46] The "local" decoder (433) may operate in the same manner as the "remote" decoder (210) detailed above with reference to FIG. 3. However, again, referring to FIG. 4, since the symbols are available and the entropy encoder (445) and parser (320) can be losslessly encoded/decoded into the encoded video sequence, the entropy decoding parts of the decoder (210) including channel (312), receiver (310), buffer ( 315) and analyzer (320) may not be fully implemented in the local decoder (433).

[47] In doing so, it can be concluded that any decoding technology present in a decoder, with the exception of analysis/entropy decoding, must also necessarily be present in essentially identical functional form in the corresponding encoder. For this reason, the present invention is focused on the operation of the decoder. The description of the encoding technologies can be abbreviated as they are the inverse of the decoding technologies described in detail. Only in some areas a more detailed description is required, which is given below.

[48] In operation, the source encoder (430) may perform motion-compensated predictive coding, in which an input frame is predictively encoded based on one or more previously encoded frames from the video sequence, referred to as "reference frames". Thus, the coding engine (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame(s) that can be selected as the prediction reference(s) to the input frame.

[49] The local video decoder (433) may decode the encoded video data of the frames, which may be indicated as reference frames, based on the symbols generated by the source encoder (430). The operations of the encoding engine (432) may be predominantly lossy methods. When the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (433) duplicates the decoding processes that may be performed by the video decoder on the reference frames and may store the reconstructed reference frames in the reference picture cache (434). Thus, the encoder (203) can locally store copies of the reconstructed key frames having the same content as the reconstructed key frames that would be received by the remote video decoder (in the absence of transmission errors).

[50] The predictor (435) may perform prediction searches for the encoding engine (432). Thus, for a new frame to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as pixel reference block candidates) or specific metadata such as reference picture motion vectors, block shapes, etc. .d., which can serve as a proper reference for predicting new images. The predictor (435) may operate on a "sample x pixel blocks" basis to find the proper prediction references. In some cases, according to the search results obtained by the predictor (435), the input picture may have prediction references extracted from the plurality of reference pictures stored in the reference picture memory (434).

[51] The controller (450) may control the encoding operations of the video encoder (430), including, for example, setting parameters as well as subgroup parameters used to encode video data.

[52] The output signal of all the above functional modules can be entropy encoded in an entropy encoder (445). The entropy encoder translates the symbols generated by the various functional units into an encoded video sequence by lossless symbol compression according to, for example, techniques known to one skilled in the art, such as Huffman coding, variable run length coding, arithmetic coding, and so on.

[53] The transmitter (440) may buffer the encoded video sequence(s) created by the entropy encoder (445) in preparation for transmission over a communication channel (460), which may be a hardware/software memory link. where the encoded video data is stored. The transmitter (440) may combine the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and/or auxiliary data streams (sources not shown).

[54] The controller (450) may control the operation of the encoder (203). During encoding, the controller (450) may assign to each encoded picture one or another encoded picture type, which may determine the encoding methods applicable to the corresponding picture. For example, images can often be assigned the following types of frames.

[55] An intra-coded picture (I-picture) that can be encoded and decoded without using any other frame in the sequence as a source for prediction. Some video codecs allow different types of intra-coded images, including, for example, images in the Independent Decoder Refresh (IDR) format. The person skilled in the art will be aware of the varieties of I-images and their respective applications and features.

[56] A predictive picture (P-picture) that can be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index to predict sample values of each block.

[57] A bidirectional predictive picture (B-picture) that can be encoded and decoded using intra prediction or inter prediction using no more than two motion vectors and reference indices to predict sample values of each block. Similarly, multi-prediction pictures may use more than two reference pictures and associated metadata to reconstruct a single block.

[58] Source images typically allow for spatial division into multiple blocks of samples (eg, blocks of 4×4, 8×8, 4×8, or 16×16 samples each) and coding on a block-by-block basis. The blocks may be predictively encoded with reference to other (previously coded) blocks defined by the coding assignment applied to the respective images of those blocks. For example, blocks of I-pictures may be encoded without prediction or predictively with reference to previously encoded blocks of the same picture (with spatial prediction or intra prediction). P-picture pixel blocks may be encoded without prediction, either by spatial prediction or temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be encoded without prediction, by spatial prediction or temporal prediction with reference to one or two previously encoded reference pictures.

[59] The video encoder (203) may perform encoding operations according to a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data may follow the syntax given by the technology used or the video coding standard used.

[60] According to an embodiment of the invention, the transmitter (440) may transmit additional data with encoded video data. The source video encoder (430) may include such data as part of the encoded video sequence. The additional data may comprise temporal/spatial/SNR enhancement levels, other forms of redundant data such as redundant images and slices, enhancement additional information (SEI) messages, video usability information (VUI) parameter set fragments, and so on.

[61] Recently, some attention has been attracted by combining compressed regions or extracting several semantically independent parts of an image to form a single video image. In particular, in the context of e.g. 360-degree encoding or certain surveillance applications, multiple semantically independent source images (e.g., a six-facet surface of a 360-degree image projected onto a cube, or separate camera inputs in the case of a multi-camera surveillance setup) may require separate adaptive resolution settings to account for different activity for each image at a given time. In other words, encoders at a given time may use different resampling factors for different semantically independent images that make up the entire 360-degree image or scene of observation. When combined into a single picture, this in turn requires performing reference picture resampling and adaptive resolution coding signaling for parts of the encoded picture.

[62] Several terms will be introduced below and will be used in the rest of this description.

[63] A sub-image may refer, in some cases, to a rectangular organization of samples, blocks, macroblocks, coding units, or similar objects that are semantically grouped and can be independently encoded in a modified resolution. One or more sub-images may form an image. One or more encoded sub-images may form an encoded image. One or more sub-images may be assembled into an image, and one or more sub-images may be extracted from the image. In certain environments, one or more encoded sub-images may be assembled into a compression-domain encoded image without transcoding to a sample layer, and in the same or other cases, one or more encoded sub-images may be extracted from the compressed-domain encoded image.

[64] Reference Picture Resampling (RPR) or Adaptive Resolution Change (ARC) may refer to mechanisms that allow changing the resolution of a picture or sub-picture in a coded video sequence, for example, by resampling the reference picture. The RPR/ARC parameters hereinafter refer to control information that is necessary to perform adaptive resolution change and may include, for example, filter parameters, scaling factors, output and/or reference picture resolutions, various control flags, and so on.

[65] In embodiments of the invention, encoding and decoding may be performed on a single semantically independent encoded video image. Before describing the implications of encoding/decoding multiple sub-pictures with independent RPR/ARC parameters and their expected additional complexity, options for signaling RPR/ARC parameters should be described.

[66] FIG. 8 shows several embodiments of the invention for RPR/ARC parameter signaling. As noted, in each of the embodiments of the invention, they may have certain advantages and certain disadvantages in terms of coding efficiency, complexity and architecture. A video coding standard or technology may choose one or more of these embodiments or prior art options for signaling the RPR/ARC parameters. Embodiments of the invention may not be mutually exclusive and may possibly be interchangeable depending on the requirements of the application, the standards technology used, or the choice of encoder.

[67] RPR/ARC parameter classes may include:

[68] - upsampling/downsampling factors, singly or in combination in the X and Y dimensions

[69] - Upsampling/downsampling ratios with added temporal dimension indicating zooming in/out at a constant rate for a given number of images

[70] - any of the above two classes may include the encoding of one or more supposedly short syntax elements that may indicate a table containing coefficients).

[71] is the resolution, along the X or Y axis, in units of samples, blocks, macroblocks, coding units (CUs), or any other suitable granularity of the input picture, output picture, reference picture, coded picture, considered in combination or separately. If there is more than one resolution (eg, one for the input image, one for the reference image), then in some cases one set of values may be derived from another set of values. This can be controlled, for example, with flags. A more detailed example is described below.

[72] - "warp" coordinates are similar to those used in H.263 Annex P, again with the appropriate degree of subdivision as mentioned above. Annex P of the H.263 standard defines an efficient way to encode such strain coordinates, but other, potentially more efficient ways may be devised. For example, the reversible Huffman-style variable-length encoding of warp coordinates from Annex P can be replaced by a suitable length binary encoding, where the length of a binary code word can be, for example, obtained from the maximum image size, perhaps multiplied by a certain factor and offset by a certain value, to allow "warping" beyond the bounds of the maximum image size.

[73] - upsampling/downsampling filter parameters. In embodiments of the invention, only one upsampling/downsampling filter may be present. However, in embodiments of the invention, it may be desirable to provide greater flexibility in filter design, and this may require filter parameter signaling. Such parameters may be selected by an index into a list of possible filter designs, the filter may be fully specified (e.g., by a list of filter coefficients, using suitable entropy coding techniques), the filter may be implicitly selected by upsampling/downsampling coefficients, which , in turn, are signaled according to any of the mechanisms mentioned above, and so on.

[74] The following description assumes encoding a finite set of upsampling/downsampling coefficients (the same coefficient is used in both the X dimension and the Y dimension) that are indicated by a codeword. This codeword may be encoded with variable length codes, for example using the Exponential Golomb code adopted for certain syntax elements in video coding specifications such as H.264 and H.265. A suitable conversion of values to upsampling/downsampling coefficients can, for example, follow Table 1:

[75] Many such transformations can be obtained according to the needs of the application and the capabilities of the upscaling and downscaling mechanisms available in the video compression technology or standard. The table could be expanded to more values. Values may also be represented by entropy encoding mechanisms other than Exponential Golomb codes, such as using binary encoding. This can have certain advantages when the oversampling factors are of interest outside of the video processing engines themselves (primarily the encoder and decoder), such as MANE. It should be noted that in situations where a change in resolution is not required, a short Exponential Golomb code can be chosen; in the table above, it is only one bit. This may have an advantage in coding efficiency over using binary codes in the most common case.

[76] The number of entries in the table, as well as their semantics, can be fully or partially customizable. For example, the underlying table data may be transmitted in a "high" parameter set, such as a sequence or decoder parameter set. In some embodiments of the invention, one or more such tables may be defined in a video coding technology or standard and may be selected, for example, by a set of decoder or sequence parameters.

[77] The following describes how the upsampling/downsampling ratio (ARC information) encoded as described above can be included in the syntax of a video coding technology or standard. A similar consideration may be applied to one or more codewords driving the upsampling/downsampling filters. The following is a discussion of the case where a filter or other data structure requires relatively large amounts of data.

[78] As shown in FIG. 5, H.263 Annex P includes ARC information (502) in the form of four warp coordinates in the image header (501), namely the H.263 PLUSPTYPE header extension (503). This may be a reasonable design choice if a) there is an image header, and b) the ARC information is expected to change frequently. However, the overhead of using H.263 signaling can be quite high, and the scaling factors may not be related to picture boundaries because the picture header may be transient.

[79] In the same or another embodiment of the invention, ARC parameter signaling may follow the detailed example shown in FIG. 6A-6B. In FIG. 6A-6B show syntax diagrams in a representation using a notation that roughly corresponds to C-style programming used, for example, in video coding standards since at least 1993. Lines in bold indicate syntax elements present in the bitstream, lines without bold often indicate a control data stream or variable setting.

[80] As shown in FIG. 6A, a tile group header (601), as an example of a header syntax structure applicable to a (possibly rectangular) part of an image, may conditionally contain a variable length dec_pic_size_idx (602) (bold) syntax element encoded in Exponential Golomb code. The presence of this syntax element in the tile group header may be due to the use of adaptive resolution (603) - here the flag value is not in bold, which means that the flag is present in the bitstream where it occurs in the syntax diagram. Whether adaptive resolution is being used for a given image or part of it can be signaled in any high-level syntactic structure inside or outside the bitstream. In the example shown, this is signaled in the sequence parameter set, as described below.

[81] FIG. 6B also shows a fragment of the set (610) of sequence parameters. The first syntax element shown is adaptive_pic_resolution_change_flag (611). If true, this flag may indicate the use of adaptive resolution, which in turn may require certain control information. In the example, such control information is conditionally present based on the value of the flag based on the if() construct in the parameter set (612) and the tile group header (601).

[82] When using adaptive resolution, this example encodes the output resolution in units of samples (613). The number 613 refers to both output_pic_width_in_luma_samples and output_pic_height_in_luma_samples, which together can set the resolution of the output image. Elsewhere in the video coding technology or standard, some restrictions may be defined for any of these values. For example, setting the level may limit the number of total output samples that can be the product of the value of these two syntax elements. In addition, certain video coding technologies or standards or external technologies or standards, such as systems standards, for example, may restrict the numbering range (for example, one or both dimensions must be a multiple of a power of 2) or aspect ratio (for example, the ratio width and height 4:3 or 16:9). Such restrictions may be introduced to facilitate hardware implementations or for other reasons and are well known in the art.

[83] In some applications, it may be desirable for the encoder to instruct the decoder to use a specific reference picture size instead of implicitly accepting that size as the output picture size. In this example, the reference_pic_size_present_flag syntax element (614) controls the conditional presence of the reference picture dimensions (615) (again, the figure refers to both width and height).

[84] Finally, a table of possible image width and height values during decoding is shown. Such a table may be expressed, for example, by specifying a table (num_dec_pic_size_in_luma_samples_minus1) (616). Here "minus1" may refer to the interpretation of the meaning of this syntax element. For example, if the encoded value is zero, then there is one table entry. If the value is five, then there are six records in the table. For each "row" in the table, the width and height of the decoded picture are then included in the syntax (617).

[85] The rendered table entries (617) can be indexed using the dec_pic_size_idx (602) syntax element in the tile group header, which allows for different decoded sizes—essentially scaling factors—for each tile group.

[86] Some video coding technologies or standards, such as VP9, support spatial scalability by implementing certain forms of reference picture resampling (signaled differently than in the present invention) in combination with temporal scalability to enable spatial scalability. In particular, some reference pictures may be upsampled using ARC technologies to higher resolution to form the basis of the spatial enhancement layer. These upsampled images can be refined using conventional high resolution prediction mechanisms to add detail.

[87] Embodiments of the invention discussed here can be used in such an environment. In some cases, in the same or another embodiment of the invention, a value in the NAL unit header, such as a temporal identifier (Temporal ID) field, can be used to indicate not only the temporal but also the spatial level. This may have certain advantages for some systems; for example, existing Selected Forwarding Units (SFUs) created and optimized for selected forwarding at the temporal level based on the value of the temporal ID in the NAL header can be used without modification for scalable environments. This may require converting the encoded picture size to the temporal level indicated by the temporal identifier field in the NAL unit header.

[88] In one embodiment of the invention, the match window size may be signaled in the PPS. The match window parameters can be used to calculate the resampling factor when the match window size of the reference picture is different from the match window size of the current picture. The decoder may need to recognize the size of the matching window of each picture to determine if a resampling process is needed.

[89] In one embodiment of the invention, the scaling factor for reference picture resampling (RPR) may be calculated based on the width and height of the output picture, which may be obtained from the match window parameters between the current picture and the reference picture. This may allow the scaling factor to be calculated more accurately than using the decoded image size. This approach can work well for most video sequences whose output image size is almost identical to the decoded image size, with small padded areas.

[90] However, this can also cause various problems. For example, when the size of the match window is very different from the size of the decoded image with large offsets, for immersive media applications (such as 360-degree cubemap, stereoscopic images, point clouds), calculating the scaling factor based on the size of the match window may not guarantee the quality of inter prediction. with different permissions. In extreme cases, the matched area of the current CU may not be present in the reference picture. When RPR is used for multilayer scalability, match window offsets may not be used to calculate the link area between layers. It should be noted that the reference area of each directly dependent layer can be explicitly transferred in the PPS extension, in the HEVC scalability extension (SHVC, HEVC Scalability Extension). When a sub-bitstream targeted at a specific area (sub-image) is extracted from the entire bitstream, the size of the match window is not at all the same as the size of the image. It should be noted that the match window parameters cannot be updated until they are used to calculate the scaling after bitstream encoding.

[91] Based on the above potential problems, calculating scaling factors based on match window size may have corner cases that require alternative parameters. As a fallback, when the match window parameters cannot be used to calculate scaling factors, it is proposed to signal reference area parameters that can be used to calculate the scaling for RPR and scalability.

[92] In one embodiment of the invention shown in FIG. 7, the conformance_window_flag flag may be signaled in the PPS. A conformance_window_flag equal to 1 may indicate that the cropping conformance window offset parameters follow next in the PPS. A conformance_window_flag of 0 may indicate that there are no conformance window offset options for cropping.

[93] In the embodiment of the invention shown in FIG. 7, conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset define the samples of images referring to PPS that are output during decoding with respect to a rectangular area specified in image coordinates for output. When the conformance_window_flag is 0, it can be inferred that the values of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset are 0.

[94] In one embodiment, the flag may be present in the PPS or other parameter set and may indicate whether the dimensions (width and height) of the resampling image are explicitly signaled in the PPS or other parameter set or not. If the image size parameters are signaled explicitly, the resampling ratio between the current image and the reference image can be calculated based on the resampling image sizes.

[95] In one of the embodiments of the invention shown in FIG. 8A, a use_conf_win_for_rpr_flag flag of 0 may indicate that resampled_pic_width_in_luma_samples and resampled_pic_height_in_luma samples follow at an appropriate location, such as later in the PPS.

[96] In one embodiment, the use_conf_win_for_rpr_flag flag of 1 may indicate that resampling_pic_width_in_luma_samples and resampling_pic_height_in_luma samples are missing.

[97] In one embodiment, the resampling_pic_width_in_luma_samples parameter may indicate the width of each PPS reference picture in units of luminance samples to be resampled. The value of the resampling_pic_width_in_luma_samples parameter may not be equal to 0, may be an integer multiple of Max(8, MinCbSizeY), and may be less than or equal to the value of pic_width_max_in_luma_samples.

[98] In one embodiment, the resampling_pic_height_in_luma_samples parameter may indicate the height of each PPS-related reference picture, in units of luma samples, for resampling. The value of the resampling_pic_height_in_luma_samples parameter may not be equal to 0, may be an integer multiple of Max(8, MinCbSizeY), and may be less than or equal to the value of pic_height_max_in_luma_samples.

[99] If the resampling_pic_width_in_luma_samples syntax element is absent, it can be concluded that the value of resampling_pic_width_in_luma_samples is equal to PicOutputWidthL.

[100] If the resampling_pic_height_in_luma_samples syntax element is absent, it can be concluded that the value of resampling_pic_height_in_luma_samples is equal to PicOutputHeightL.

[101] In one embodiment of the invention, an example of a fractional interpolation process with reference picture resampling can be performed as follows:

[102] The variable fRefWidth may be set equal to the resampling_pic_width_in_luma_samples value of the reference image in luma samples.

[103] The variable fRefHeight may be set to resampling_pic_height_in_luma_samples of the reference image in luma samples.

[104] The motion vector mvLX may be set to (refMvLX - mvOffset).

[105] If the value of cIdx is 0, the scaling factors and their fixed-point representations can be set according to expressions (1) and (2) below:

[106] In the embodiment of the invention shown in FIG. 8B, a use_conf_win_for_rpr_flag flag of 0 may indicate that resampled_pic_width_in_luma_samples and resampled_pic_height_in_luma_samples follow next in PPS. A use_conf_wid_for_rpr_flag value of 1 indicates that resampling_pic_width_in_luma_samples and resampling_pic_height_in_luma_samples are missing.

[107] In one of the embodiments of the invention, the ref_region_left_offset parameter may specify a horizontal offset of the upper left brightness reference of the reference region in the decoded image. The value of refregion_left_offset must be between -2 ¹⁴ and 2 ¹⁴ - 1 inclusive. If there is no ref_region_left_offset value, it can be inferred that it is equal to conf_win_left_offset.

[108] In one of the embodiments of the invention, the ref_region_top_offset parameter may specify a vertical offset of the top left brightness reference of the reference region in the decoded image. The value of ref_region_top_offset must be between -2 ¹⁴ and 2 ¹⁴ - 1 inclusive. If there is no ref_region_top_offset value, we can infer that it is equal to conf_win_right_offset.

[109] In one of the embodiments of the invention, the ref_region_right_offset parameter may specify a horizontal offset of the lower right brightness reference of the reference region in the decoded image. The value of ref_layer_right_offset must be between 2 ¹⁴ and 2 ¹⁴ 1 inclusive. If there is no ref_region_right_offset value, we can infer that it is equal to conf_win_top_offset.

[110] In one of the embodiments of the invention ref_region_bottom offset may specify a vertical offset of the lower right reference region brightness sample in the decoded image. The value of ref_layer_bottom offset must be between -2 ¹⁴ and 2 ¹⁴ - 1 inclusive. If ref_region_bottom_offset[ ref_loc_offset_layer id[i]] is not present, it can be inferred that it is equal to conf_win_bottom_offset.

[111] The variables PicRefWidthL and PicRefHeightL can be obtained as shown in expressions (3) and (4) below:

[112] The variable fRefWidth may be set equal to the PicRefWidthL value of the reference image in luminance samples.

[113] The variable fReffleight may be set equal to the PicRefHeightL value of the reference image in luminance samples.

[114] The mvLX motion vector may be set to (refMvLX mvOffset).

[115] If cIdx is 0, the scaling factors and their fixed-point representations can be determined as shown in expressions (5) and (6) below:

[116] The upper left coordinate of the bounding box for padding the reference samples (xSbIntL, ySbIntL) may be set to

(xSb+(mvLX[0] >> 4), ySb+(mvLX[1] >> 4)).

[117] For each brightness sample location

(xL=0..sbWidth - 1+brdExtSize, yL=0..sbHeight - 1+brdExtSize)

in an array of prediction luma samples predSamplesLX, the corresponding prediction luma sample value predSamplesLX[xL][yL] can be obtained as follows:

- Let (refxSbL, refySbL) and (refxL, refyL) be the locations for luma pointed to by the motion vector (refMvLX, refMvLX) given in units of 1/16 of a sample. Variables refxSbL, refxL, refySbL and refyL can be obtained as shown in expressions (7)-(10) below:

[118] FIG. 9 is a flowchart illustrating an example of a method 900 for decoding an encoded video bitstream. In some embodiments of the invention, one or more of the steps of the method of FIG. 9 may be performed by decoder 210. In some embodiments, one or more of the method steps in FIG. 9 may be performed by another device or group of devices separate from decoder 210 or including decoder 210, such as encoder 203.

[119] As shown in FIG. 9, method 900 may include obtaining a flag indicating that the match window is not being used for reference picture resampling (block 910).

[120] As shown in FIG. 9, method 900 may include, based on a flag indicating that a match window is not used for reference picture resampling, determining whether a resampling picture size is being signaled (block 920).

[121] If it is determined that the second flag indicates that the resampling image size is being signaled (Yes branch of block 920), method 900 may proceed to block 930 and then to block 950. At block 930, method 900 may include determining a resampling factor based on the resampling image size.

[122] If it is determined that the second flag does not indicate that the match window is being used for reference picture resampling ("No" branch at block 920), method 900 may proceed to block 940 and then to block 950. At block 940, method 900 may include determining a resampling factor based on the output image size.

[123] As shown in FIG. 9, method 900 may include resampling the reference picture for the current picture using a resampling factor (step 950).

[124] In an embodiment, the flag may be signaled in a set of image parameters.

[125] In one embodiment of the invention, the size of the resampling image may be signaled in the encoded video bitstream as the width and/or height of the resampling image.

[126] In an embodiment of the invention, the width and/or height may be signaled in a set of image parameters.

[127] In an embodiment of the invention, the width and/or height may be expressed as the number of luma samples included in the width and/or height.

[128] In an embodiment of the invention, the width and/or height may be determined based on at least one offset distance between the boundary of the current image and a predetermined reference area brightness reference.

[129] In an embodiment of the invention, at least one offset distance may be signaled in a set of image parameters.

[130] In an embodiment of the invention, at least one offset distance may include at least one of the following: a horizontal offset distance between the left border of the current image and the upper left reference area brightness sample, a vertical offset distance between the upper border of the current image and the upper left sample the reference area brightness, a horizontal offset distance between the right edge of the current image and the right bottom brightness sample of the reference area, and a vertical offset distance between the bottom edge of the current image and the bottom right brightness sample of the reference area.

[131] Although FIG. 9 shows examples of the steps of a method 900, in some embodiments of the invention, the method 900 may include additional steps, fewer steps, other steps, or steps arranged differently than shown in FIG. Additionally or alternatively, two or more steps of method 900 may be performed in parallel.

[132] In addition, the proposed methods can be implemented using processing circuits (eg, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program that is stored on a computer-readable medium to perform one or more of the proposed methods.

[133] The above methods may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 10 shows a computer system 1000 suitable for practicing some embodiments of the invention.

[134] Computer software may be coded using any suitable machine code or computer language that may be assembled, compiled, linked, or similarly processed to produce code containing instructions that can be executed directly or through interpretation, microcode execution, etc. etc., computer central processing units (CPU, central processing units), graphic processors (GPU, Graphics Processing Units), etc.

[135] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[136] The components of the computer system 1000 shown in FIG. 10 are illustrative in nature and are not intended to impose any limitation on the scope of application or functionality of computer software implementing embodiments of the present invention. The component configuration should also not be interpreted as having any dependency or requirement in connection with any component or combination of components shown in the exemplary computer system 1000.

[137] Computer system 1000 may include some human-machine interface input devices. Such an input device may be responsible for input by one or more users through, for example, tactile input (e.g., keystrokes, waving, glove movements), audio input (e.g., voice, claps), visual input (e.g., gestures), olfactory input (not shown). Interface devices may also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, external sound), images (e.g. scanned images, photographic images received from a still camera), video (eg 2D video, 3D video including stereoscopic video).

[138] The input human-machine interface of the device may include one or more of the following (shown one at a time): keyboard 1001, mouse 1002, touch pad 1003, touch screen 1010 and associated graphics adapter 1050, data glove, joystick 1005, microphone 1006, scanner 1007 and camera 1008.

[139] Computer system 1000 may also include some human-machine interface output devices. Such output devices may stimulate the senses of one or more users through, for example, tactile output, sound, light, and smell/taste. Such output devices may include tactile output devices (for example, haptic feedback via a 1010 touch screen, data glove, or 1005 joystick, as well as haptic feedback devices that do not serve as input devices), audio output devices (for example: speakers 1009, headphones ( not shown)), visual output devices (e.g., 1010 screens, including cathode ray tube (CRT) screens), liquid-crystal display (LCD), plasma screens, organic light-emitting diode screens ( OLED, organic light-emitting diode), each with or without touch screen input capabilities, each with or without tactile feedback capabilities, some of them are capable of two-dimensional visual output or more than three-dimensional output through, for example, stereographic output, glasses virtual reality (not shown), holographic displays, smoke tanks (not shown) and printers (not shown). e shown).

[140] Computer system 1000 may also include human accessible storage devices and associated media, such as optical media including CD/DVD ROM/RW 1020 with CD/DVD media 1021, etc., flash memory cards 1022, removable hard disk or solid state disk 1023, conventional magnetic media such as tape and magnetic disk (not shown), dedicated ROM/ASIC/PLD based devices such as security dongles (not shown), and the like.

[141] Those skilled in the art should also understand that the term "computer-readable media" as used in connection with the present invention does not encompass transmission media, carrier waves, or other time signals.

[142] Computer system 1000 may also include interface(s) to one or more communication networks (1155). Networks can be, for example, wireless, wired, optical. Networks can also be local, global, urban, transport and industrial, real-time, latency, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks, including global systems for mobile communications (GSM), 3G, 4G, and 5G generation networks, Long Term Evolution (LTE, Long Term Evolution) -Term Evolution), etc., global digital networks of wired or wireless television, including cable television, satellite television and terrestrial television broadcasting, transport and industrial networks, including CANBus, etc. Some networks typically require external network interface adapters (1154) that are connected to some general purpose data ports or peripheral buses (1149) (eg, the USB universal serial bus ports of the computer system 1000); others are typically built into the core of computer system 1000 by connecting to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). For example, network 1055 may be connected to peripheral bus 1049 using network interface 1054. Using any of these networks, computer system 1000 may communicate with other entities. Such communication may be unidirectional with receive-only capability (eg broadcast), unidirectional with transmit-only capability (eg CANbus to some CANbus devices), or bidirectional, eg to other computer systems using a local area network or a wide area digital network. Several protocols and protocol stacks may be used on each of these networks and the network interfaces described above (1154).

[143] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces can be connected to the core 1040 of the computer system 1000.

[144] The core 1040 may include one or more central processing units (CPUs) 1041, graphics processing units (GPUs) 1042, field programmable gate arrays (FPGAs) 1043, hardware accelerators 1044 for some tasks, etc. These devices, together with Read Only Memory (ROM) 1045, Random Access Memory (RAM) 1046, high-capacity internal storage such as user-inaccessible internal hard drives, solid-state drives (SSDs, solid-state drives), and the like. 1047 may be connected via system bus 1048. In some computer systems, system bus 1048 may be available in the form of one or more physical connectors to allow for expansions with additional CPUs, GPUs, and the like. Peripherals can be connected either directly to the core system bus 1048 or via a peripheral bus 1049. Peripheral bus architectures include a peripheral component interconnect (PCI) bus, USB, and the like.

[145] CPU 1041, GPU 1042, FPGA 1043, and accelerators 1044 may execute some instructions that together may constitute the above computer code. This computer code may be stored in ROM 1045 or RAM 1046. Temporary data may also be stored in RAM 1046, while permanent data may be stored in, for example, internal high capacity data storage 1047. Rapid storage and retrieval from any storage device can be achieved through the use of a cache that can be tightly coupled to one or more CPU 1041, GPU 1042, mass storage 1047, ROM 1045, RAM 1046, and the like.

[146] Computer-readable media may store computer code for performing various computer-executable operations. The media and computer code may be specially created for the purposes of the present invention or may be well known and available to those skilled in the art of computer software.

[147] By way of example, and not limitation of the invention, a computer system having architecture 1000, and in particular core 1040, can provide functionality by being executed by processor(s) (including(and) including CPU, GPU, FPGA, accelerators etc.) software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with the user-accessible mass storage described above, as well as some persistent storage of the kernel 1040, such as internal mass storage 1047 or ROM 1045. Software implementing various embodiments of the present invention may stored in such devices and executed by the core 1040. The computer-readable medium may include one or more storage devices or chips, in accordance with particular needs. The software may direct the core 1040, and in particular its processors (including CPUs, GPUs, FPGAs, and the like), to perform specific methods or specific portions of the specific methods described herein, including setting data structures stored in RAM 1046 and modifying such data structures according to the methods specified by the software. Additionally or alternatively, the computer system may provide functionality through logic hardwired or otherwise embodied in circuitry (e.g., accelerator 1044) that may operate in place of or in conjunction with software to perform specific methods or specific portions of the specific methods described herein. A reference to software may cover logic, and vice versa when appropriate. Reference to computer-readable media may cover a circuit (eg, an integrated circuit (IC, integrated circuit)), where software is stored for execution, a circuit embodying logic for execution, or both circuits, when appropriate. The present invention encompasses any suitable combination of hardware and software.

[148] Although some embodiments of the invention have been described here, changes, permutations, and various equivalent substitutions are possible within the scope of the invention. Thus, those skilled in the art can provide numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the invention and thus fall within its spirit and scope.

Claims (21)

1. A method for decoding a bitstream of encoded video data using at least one processor, including: obtaining a flag indicating that the match window is not used for resampling the reference picture; based on a flag indicating that the match window is not used for resampling the reference picture, determining whether the size of the resampling picture is signaled; based on determining that the size of the oversampling image is signaled, determining an oversampling ratio based on the size of the oversampling image; based on determining that the resampling image size is not signaled, determining an oversampling ratio based on the output image size; and performing resampling of the reference picture on the current picture using the resampling factor. 2. The method of claim 1, wherein the flag is signaled in the image parameter set. 3. The method of claim 1, wherein the resampling image size is signaled in the encoded video data bitstream as the width and/or height of the resampling image. 4. The method of claim 3, wherein the width and/or height are signaled in the image parameter set. 5. The method of claim 3, wherein the width and/or height is expressed as the number of luminance samples included in the width and/or height. 6. The method of claim 3, wherein the width and/or height is determined based on at least one offset distance between a current image boundary and a predetermined reference area brightness reference. 7. The method of claim 6, wherein at least one offset distance is signaled in the image parameter set. 8. The method of claim 6, wherein the at least one offset distance comprises at least one of the following: horizontal offset distance between the left border of the current image and the upper left reference area brightness reading, vertical offset distance between the upper border of the current image and the upper left reference area brightness reading, horizontal offset distance between the right border of the current image and the lower right reference area brightness reading, and vertical offset distance between the lower border of the current image and the lower right reference area brightness reading. 9. A device for decoding a bit stream of encoded video data, comprising: at least one memory configured to store program code, and at least one processor configured to read the program code and operate in accordance with the instructions of the program code to implement the method for decoding the encoded video bitstream according to any one of paragraphs. 1-8. 10. A computer-readable medium that stores instructions, including: one or more instructions that, when executed by one or more processors of a device for decoding an encoded video bitstream, cause one or more processors to perform a method for decoding an encoded video bitstream according to any one of paragraphs. 1-8.

Images