JP2022539657A - Adaptive resolution change in video processing - Google Patents

Abstract

The present disclosure provides systems and methods for adaptive resolution change during video encoding and decoding. The method compares resolutions of a target picture and a first reference picture, and a first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions. and resampling the reference picture of the second reference picture, and encoding or decoding the target picture using the second reference picture.

Description

Cross-reference to related applications
[0001] This disclosure is disclosed in U.S. Provisional Patent Application Nos. 62/865,927, filed June 24, 2019, and U.S. Provisional Patent Application No. 62/865,927, filed September 13, 2019, both of which are incorporated herein in their entirety. This application claims the benefit of priority to U.S. Provisional Patent Application No. 62/900,439.

Technical field
[0002] This disclosure relates generally to video processing, and more particularly to methods and systems for adaptive resolution change in video coding.

background
[0003] A video is a series of still pictures (or "frames") that capture visual information. To reduce storage memory and transmission bandwidth, the video may be compressed before storage or transmission and decompressed before display. The compression process is usually called encoding and the decompression process is usually called decoding. There are various video coding formats that use standardized video coding techniques, most commonly based on prediction, transform, quantization, entropy coding, and in-loop filtering. Video coding standards such as HEVC (High Efficiency Video Coding)/H.265 standard, VVC (Versatile Video Coding)/H.266 standard, and AVS standard, which specify specific video coding formats, have been standardized. developed by an institution. As more and more advanced video coding techniques are adopted in video standards, the coding efficiency of new video coding standards is getting higher and higher.

Summary of Disclosure
[0004] Embodiments of the present disclosure provide a method for adaptive resolution change during encoding or decoding of video. In one exemplary embodiment, the method includes comparing resolutions of a target picture and a first reference picture; Re-sampling a first reference picture to generate a picture, and encoding or decoding a target picture using a second reference picture.

[0005] Embodiments of the present disclosure also provide devices for adaptive resolution change during encoding or decoding of video. In an exemplary embodiment, the device comprises one or more memories storing computer instructions and executing the computer instructions to instruct the device to compare resolutions of a target picture and a first reference picture; resampling the first reference picture to generate a second reference picture in response to the picture and the first reference picture having different resolutions; using the second reference picture; and one or more processors configured to perform encoding or decoding of the target picture.

[0006] An embodiment of the present disclosure provides a non-transitory computer-readable medium storing a set of instructions, the set of instructions for causing a computer system to perform a method of adaptive resolution change. can be executed by at least one processor of In one exemplary embodiment, the method includes comparing resolutions of a target picture and a first reference picture; Re-sampling a first reference picture to generate a picture, and encoding or decoding a target picture using a second reference picture.

Brief description of the drawing
[0007] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings. Various features illustrated in the drawings are not drawn to scale.

[0008] Fig. 2 is a schematic diagram illustrating an exemplary video encoder, consistent with disclosed embodiments; [0009] FIG. 4 is a schematic diagram illustrating an exemplary video decoder, consistent with disclosed embodiments; [0010] FIG. 7 illustrates an example where a reference picture has a different resolution than a current picture, consistent with an embodiment of the disclosure. [0011] Fig. 5 is a table showing sub-pel motion compensated interpolation filters used for luma components in Versatile Video Coding (VVC), consistent with disclosed embodiments; [0012] Fig. 4 is a table showing sub-pel motion compensated interpolation filters used for chroma components in VVC, consistent with disclosed embodiments; [0013] FIG. 4 illustrates an exemplary reference picture buffer, consistent with disclosed embodiments. [0014] Fig. 4 is a table showing an example of a supported resolution set including three different resolutions consistent with disclosed embodiments; [0015] FIG. 4 illustrates an exemplary decoded picture buffer (DPB) when both resampled and original reference pictures are preserved, consistent with disclosed embodiments. [0016] Fig. 3 illustrates progressive downsampling consistent with disclosed embodiments; [0017] FIG. 4 illustrates an exemplary video encoding process with resolution change consistent with disclosed embodiments. [0018] Fig. 4 is a table showing exemplary downsampling filters consistent with disclosed embodiments; [0019] FIG. 4 illustrates an exemplary video encoding process using peak signal-to-noise ratio (PSNR) calculation consistent with disclosed embodiments. [0020] FIG. 4 illustrates a frequency response of an exemplary low pass filter consistent with disclosed embodiments; [0021] Fig. 6 is a table showing a 6-tap filter, consistent with the disclosed embodiment; [0022] Fig. 4 is a table showing an 8-tap filter consistent with disclosed embodiments; [0023] Fig. 4 is a table showing a 4-tap filter consistent with disclosed embodiments; [0024] Fig. 4 is a table showing interpolation filter coefficients for 2:1 downsampling, consistent with disclosed embodiments; [0025] Fig. 6 is a table showing interpolation filter coefficients for 1.5:1 downsampling, consistent with disclosed embodiments; [0026] FIG. 4 illustrates an exemplary luma sample interpolation filtering process for reference downsampling, consistent with disclosed embodiments. [0027] FIG. 4 illustrates an exemplary chroma sample interpolation filtering process for reference downsampling, consistent with disclosed embodiments. [0028] FIG. 4 illustrates an exemplary luma sample interpolation filtering process for reference downsampling, consistent with disclosed embodiments. [0029] Fig. 4 illustrates an exemplary chroma sample interpolation filtering process for reference downsampling, consistent with disclosed embodiments; [0030] FIG. 4 illustrates an exemplary luma sample interpolation filtering process for reference downsampling, consistent with disclosed embodiments. [0031] FIG. 4 illustrates an exemplary chroma sample interpolation filtering process for reference downsampling, consistent with disclosed embodiments. [0032] FIG. 4 illustrates an exemplary chroma fractional sample position calculation for reference downsampling, consistent with disclosed embodiments. 1 is a table showing an 8-tap filter for MC interpolation using a 2:1 ratio reference downsampling, consistent with disclosed embodiments; [0002] Fig. 4 is a table showing an 8-tap filter for MC interpolation using a 1.5:1 ratio reference downsampling, consistent with disclosed embodiments; [0003] Fig. 4 is a table showing an 8-tap filter for MC interpolation using a 2:1 ratio reference downsampling, consistent with disclosed embodiments; [0004] Fig. 4 is a table showing an 8-tap filter for MC interpolation using a 1.5:1 ratio reference downsampling, consistent with disclosed embodiments; [0005] Fig. 4 is a table showing 6-tap filter coefficients for luma 4x4 block MC interpolation using a 2:1 ratio reference downsampling, consistent with disclosed embodiments; [0006] Fig. 6 is a table showing 6-tap filter coefficients for luma 4x4 block MC interpolation using a 1.5:1 ratio reference downsampling, consistent with disclosed embodiments;

detailed description
[0007] Reference will now be made in detail to exemplary embodiments illustrated in the accompanying drawings. The following description refers to the accompanying drawings, where like numbers in different drawings represent the same or similar elements, unless otherwise stated. The implementations set forth in the following description of exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are merely examples of apparatus and methods consistent with related aspects of the present invention as recited in the appended claims. Certain aspects of the disclosure are described in greater detail below. In the event of conflict with the incorporated terms and/or definitions, the terms and definitions provided herein control.

[0008] A video is a series of still pictures (or "frames") arranged in time sequence to preserve visual information. A video capture device (e.g., camera) can be used to capture and store these pictures in chronological order, and a video playback device (e.g., television, computer, smart phone, tablet computer, video player, or display capability). Any end-user terminal equipped with a terminal) can be used to display such pictures in chronological order. Also, in some applications, the video capture device can transmit the captured video in real time to a video playback device (e.g., a computer with a monitor) for monitoring, conferencing, live broadcasting, and the like.

[0009] To reduce the storage space and transmission bandwidth required in such applications, the video may be compressed. For example, video may be compressed before storage and transmission and decompressed before display. Compression and decompression may be performed by software executed by a processor (eg, a processor of a general purpose computer) or dedicated hardware. The modules for compression are commonly called "encoders" and the modules for decompression are commonly called "decoders". Encoders and decoders are sometimes collectively referred to as "codecs." Encoders and decoders may be implemented as any of a variety of suitable hardware, software, or combinations thereof. For example, the hardware implementation of the encoder and decoder may be one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, or any of these. may include circuitry such as combinations of A software implementation of the encoder and decoder may include program code, computer-executable instructions, firmware, or any suitable computer-implemented algorithm or process embodied in a computer-readable medium. Video compression and decompression may be performed by various algorithms or standards such as MPEG-1, MPEG-2, MPEG-4, H.26x family. Depending on the application, a codec may recover video from a first encoding standard and recompress the recovered video using a second encoding standard, in which case the codec is a "transcoder." It is sometimes called

[0010] The video encoding process can identify and retain useful information that can be used for picture reconstruction. If the information ignored in the video encoding process cannot be completely reconstructed, the encoding process is sometimes called "lossy." Otherwise it may be called "reversible". Most encoding processes are irreversible, a trade-off to reduce storage space and transmission bandwidth required.

[0011] In many cases, useful information in a picture being encoded (referred to as the "current picture") may include changes relative to a reference picture (eg, a previously encoded or reconstructed picture). . Such changes may include pixel position changes, brightness changes, or color changes, of which position changes are of most concern. A change in position of pixels representing an object may reflect movement of the object between the reference picture and the current picture.

[0012] In order to achieve the same subjective quality as HEVC/H.265 at half the bandwidth, JVET has developed a technology that surpasses HEVC using the joint exploration model ("JEM") reference software. there is JEM achieved significantly higher coding performance than HEVC because the coding technology was embedded in JEM. VCEG and MPEG have also officially started development of a next-generation video compression standard (Versatile Video Coding (VVC/H.266) standard) that surpasses HEVC.

[0013] The VVC standard continues to add additional encoding techniques that provide better compression performance. VVC can be implemented in the same video coding systems that have been used in modern video compression standards such as HEVC, H.264/AVC, MPEG2, H.263. FIG. 1 is a schematic diagram illustrating an exemplary video encoder 100 consistent with disclosed embodiments. For example, video encoder 100 may perform intra- or inter-coding of blocks within video frames, including video blocks or partitions or sub-partitions of video blocks. Intra-coding may rely on spatial prediction to reduce or remove spatial redundancy in video within a given video frame. Inter-coding may rely on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames of a video sequence. Intra-modes may refer to some spatial-based compression modes, and inter-modes (such as uni-prediction or bi-prediction) may refer to some temporal-based compression modes.

[0014] Referring to Figure 1, an input video signal 102 may be processed block by block. For example, a video block unit may be a 16×16 pixel block (eg, macroblock (MB)). In HEVC, extended block sizes (eg, coding units (CUs)) can be used to compress video signals with a resolution of, eg, 1080p or higher. In HEVC, a CU can contain up to 64x64 luma samples and corresponding chroma samples. In VVC, the size of the CU can be further increased to contain 128x128 luma samples and corresponding chroma samples. A CU may be partitioned into prediction units (PUs) to which separate prediction methods may be applied. Each input video block (eg, MB, CU, PU, etc.) may be processed by using spatial prediction unit 160 or temporal prediction unit 162 .

[0015] Spatial prediction unit 160 performs spatial prediction (eg, intra prediction) on the current CU using information about the same picture/slice that contains the current CU. Spatial prediction may use pixels from already coded neighboring blocks within the same video picture/slice to predict the current video block. Spatial prediction can reduce the spatial redundancy inherent in video signals. Temporal prediction (eg, inter prediction or motion compensated prediction) may use samples from already coded video pictures to predict the current video block. Temporal prediction can reduce the temporal redundancy inherent in video signals.

[0016] Temporal prediction unit 162 performs temporal prediction for the current CU using information from one or more pictures/slices that are different from the picture/slice that contains the current CU. (eg, inter-prediction). A temporal prediction for a video block may be signaled by one or more motion vectors. A motion vector may indicate the amount and direction of motion between a current block and one or more of the current block's predictive blocks in a reference frame. If multiple reference pictures are supported, one or more reference picture indices may be sent for a video block. One or more reference indices are used to identify from which one or more reference pictures in a decoded picture buffer (DPB) 164 (also called reference picture storage 164) the temporal prediction signal may come from. can do. After spatial prediction or temporal prediction, mode decision and encoder control unit 180 within the encoder may choose a prediction mode, for example, based on a rate-distortion optimization method. The prediction block may be subtracted from the current video block at summer 116 . The prediction residual may be transformed by transform unit 104 and quantized by quantization unit 106 . The quantized residual coefficients may be inverse quantized at inverse quantization unit 110 and inverse transformed at inverse transform unit 112 to form a reconstructed residual. The reconstructed block may be added to the prediction block at adder 126 to form a reconstructed video block. Before the reconstructed video block is added to the reference picture store 164 and used to encode future video blocks, in-loop filtering such as the deblocking filter and the adaptive loop filter 166 is applied to the reconstructed video block. can be applied to To form the output video bitstream 120, the coding mode (e.g., inter or intra), prediction mode information, motion information, and quantized residual coefficients are sent to entropy encoding unit 108 to convert the bitstream 120 into It can be compressed and packed to form.

[0017] Consistent with the disclosed embodiments, the above-described units of video encoder 100 include software modules (eg, computer programs that perform different functions), hardware components (eg, different circuits for performing respective functions). network block), or as a hybrid of software and hardware.

[0018] Figure 2 is a schematic diagram illustrating an exemplary video decoder 200, consistent with the disclosed embodiments. Referring to FIG. 2, video bitstream 202 may be unpacked or entropy decoded in entropy decoding unit 208 . Coding mode or prediction information may be sent to spatial prediction unit 260 (eg, if intra-coded) or temporal prediction unit 262 (eg, if inter-coded) to form a prediction block. When inter-coded, the prediction information may be the prediction block size, one or more motion vectors (which may indicate, for example, the direction and amount of motion), or (for example, from which reference picture the prediction is taken). ) may include one or more reference indices.

[0019] Motion compensated prediction may be applied by temporal prediction unit 262 to form a temporal prediction block. The residual transform coefficients may be sent to inverse quantization unit 210 and inverse transform unit 212 to reconstruct a residual block. The prediction block and residual block may be summed 226 . The reconstructed block may undergo in-loop filtering (by loop filter 266) before being stored in decoded picture buffer (DPB) 264 (also called reference picture store 264). The reconstructed video of the DPB 264 can be used to drive a display device or predict future video blocks. A decoded video 220 may be displayed on the display.

[0020] Consistent with the disclosed embodiments, the above-described units of the video decoder 200 are comprised of software modules (eg, computer programs that perform different functions), hardware components (eg, different circuits for performing respective functions). network block), or as a hybrid of software and hardware.

[0021] One of the goals of the VVC standard is to provide videoconferencing applications with the ability to tolerate a diversity of networks and devices. In particular, the VVC standard adapts rapidly to changing network environments, including rapidly decreasing the encoding bitrate when network conditions deteriorate and rapidly increasing video quality when network conditions improve. should provide the ability to Additionally, for adaptive streaming services that provide multiple representations of the same content, each of the multiple representations may have different characteristics (e.g., spatial resolution, or sample bit depth), with video quality varying from low to can vary to high. Therefore, the VVC standard needs to support fast representation switching for adaptive streaming services. During switching from one representation to another (such as switching from one resolution to another), the VVC standard allows the use of efficient prediction structures without compromising fast and seamless switching capabilities. need to be

[0022] In some embodiments consistent with this disclosure, to change resolution, an encoder (eg, encoder 100 of FIG. 1) uses reference picture buffers (eg, DPB 164 of FIG. 1 and DPB 264 of FIG. 2). ), send an instantaneous-decoder-refresh (IDR) coded picture. Upon receipt of an IDR encoded picture, the decoder (eg, decoder 200 of FIG. 2) marks all pictures in the reference buffer as "unused for reference." All subsequent transmitted pictures may be decoded without reference to any frames decoded prior to the IDR picture. The first picture in an encoded video sequence is always an IDR picture.

[0023] In some embodiments consistent with the present disclosure, adaptive resolution change (ARC) techniques are used to allow a video stream to be processed without the need for new IDR pictures and with scalable video codecs. It may allow changing the spatial resolution between coded pictures within the same video sequence without requiring multiple layers as in the case. According to the ARC technique, at a resolution switch point, the currently coded picture is either predicted from a reference picture of the same resolution (if available), or a different resolution by resampling the reference picture. is predicted from reference pictures. A diagram of adaptive resolution change is shown in FIG. 3, where the resolution of the reference picture 'Ref0' is the same as the resolution of the currently encoded picture. However, the resolutions of the reference pictures 'Ref1' and 'Ref2' are different from the resolution of the current picture. Both 'Ref1' and 'Ref2' are resampled to the resolution of the current picture to generate the motion compensated prediction signal for the current picture.

[0024] Consistent with this disclosure, if the resolution of the reference frame is different from the resolution of the current frame, one method of generating a motion-compensated prediction signal is to first convert the reference picture to the same resolution as the current picture. It is a picture-based resampling that can be resampled and existing motion compensation processes using motion vectors can be applied. The motion vector may be scaled (if the motion vector is sent in units before resampling is applied) or scaled (if the motion vector is sent in units after resampling is applied). It doesn't have to be. With picture-based resampling, particularly for reference picture downsampling (i.e., when the resolution of the reference picture is greater than the resolution of the current picture), downsampling is usually followed by decimation after low-pass filtering. As achieved by following, information may be lost in the reference resampling step prior to motion compensated interpolation.

[0025] Another method is block-based resampling, where resampling is done at the block level. This is done by looking at one or more reference pictures used by the current block, and if one or both of them have a different resolution than the current picture, then resampling is done with sub-pel motion compensated interpolating. It is done in conjunction with the poration process.

[0026] This disclosure provides both picture-based and block-based resampling methods for use with ARC. The following discussion first addresses the disclosed picture-based resampling method and then the disclosed block-based resampling method.

[0027] The disclosed picture-based resampling method can solve several problems caused by conventional block-based resampling methods. First, in conventional block-level resampling methods, on-the-fly block-level resampling may be performed when the reference picture resolution is determined to be different from the current picture resolution. However, the block-level resampling process can complicate the encoder design because the encoder may have to resample the block at each search point during motion search. Motion search is generally a time consuming process for an encoder, so the on-the-fly requirement during motion search can complicate the motion search process. As an alternative to on-the-fly block-level resampling, the encoder may pre-resample the reference picture such that the resampled reference picture has the same resolution as the current picture. However, this may lead to different prediction signals during motion estimation and motion compensation, thus reducing coding efficiency.

[0028] Second, in conventional block-based resampling methods, the design of block-level resampling includes subblock-based temporal motion vector prediction (SbTMVP), affine motion compensation prediction, decoder side motion vector refinement (DMVR). incompatible with some other useful encoding tools such as When ARC is enabled, these encoding tools can be disabled. However, such invalidation significantly degrades coding performance.

[0029] Third, although any sub-pel motion compensated interpolation filter known in the art can be used for block resampling, it is similarly not applicable for block upsampling and block downsampling. Sometimes. For example, if the reference picture has a lower resolution than the current picture, the interpolation filters used for motion compensation may be used for upsampling. However, if the reference picture has a higher resolution than the current picture, the interpolation filter may not be able to filter the integer positions and thus cause aliasing. is not suitable. For example, Table 1 (FIG. 4) shows exemplary interpolation filter coefficients used for luma component integer positions along with various values for fractional sample positions. Table 2 (FIG. 5) shows exemplary interpolation filter coefficients used for the chroma components along with various values for the fractional sample positions. As shown in Table 1 (FIG. 4) and Table 2 (FIG. 5), at fractional sample position 0 (ie, integer position), no interpolation filter is applied.

[0030] To avoid the above problems associated with conventional block-based resampling methods, this disclosure provides a picture-level resampling method that can be used for ARC. The picture resampling process may involve upsampling or downsampling. Upsampling is increasing the spatial resolution while maintaining a two-dimensional (2D) representation of the image. In the upsampling process, the resolution of a reference picture is increased by interpolating unavailable samples from neighboring available samples. In the downsampling process the resolution of the reference image is reduced.

[0031] According to disclosed embodiments, resampling can be done at the picture level. In picture-level resampling, a reference picture is resampled to the resolution of the current picture if the resolution of the reference picture is different from the resolution of the current picture. Motion estimation and/or compensation for the current picture may be performed based on the resampled reference pictures. In this way, ARC can be implemented in a 'transparent' manner at the encoder and decoder, as the block-level operation does not depend on resolution changes.

[0032] Picture-level resampling may be done on-the-fly, ie, while the current picture is being predicted. In some exemplary embodiments, only original (non-resampled) reference pictures are stored in decoded picture buffers (DPBs), eg, DPB 164 of encoder 100 (FIG. 1) and DBP 264 of decoder 200 (FIG. 2). stored in The DPB can be managed in the same way as the current version of VVC. In the disclosed picture-level resampling method, before the picture is encoded or decoded, the encoder or decoder re-sampling the reference picture if the resolution of the reference picture in the DPB is different from the resolution of the current picture. to sample. In some embodiments, a resampled picture buffer may be used to store all resampled reference pictures for the current picture. A resampled picture of the reference picture is stored in a resampled picture buffer, and motion search/compensation for the current picture is performed using the picture from the resampled picture buffer. After the encoding or decoding is completed, the resampled picture buffer is removed. FIG. 6 shows an example of on-the-fly picture-level resampling in which a low-resolution reference picture is resampled and stored in a resampled picture buffer. As shown in FIG. 6, the encoder or decoder DPB contains three reference pictures. The resolution of the reference pictures 'Ref0' and 'Ref2' is the same as the resolution of the current picture. Therefore, 'Ref0' and 'Ref2' do not need to be resampled. However, the resolution of the reference picture 'Ref1' is different from the resolution of the current picture and therefore needs to be resampled. Thus, only the resampled 'Ref1' is saved in the reference picture buffer, and 'Ref0' and 'Ref2' are not saved in the reference picture buffer.

[0033] Consistent with some exemplary embodiments, with on-the-fly picture-level resampling, the number of resampling reference pictures that a picture can use is constrained. For example, the maximum number of resampled reference pictures for a given current picture may be preset. For example, the maximum number may be set to be one. In this case, the encoder or decoder may allow at most one of the reference pictures to have a resolution different from that of the current picture, and all other reference pictures have the same resolution. Bitstream constraints can be imposed as they must. Since this maximum number indicates the size of the resampled picture buffer and the maximum number of resamplings that can be done for any picture of the current video sequence, the maximum number is the worst case decoder complexity. directly related to Therefore, this maximum number may be signaled as part of the Sequence Parameter Set (SPS) or Picture Parameter Set (PPS) and may be specified as part of the Profile/Level definition.

[0034] In some exemplary embodiments, two versions of a reference picture are stored in the DPB. One version has the original resolution and the other version has the full resolution. If the resolution of the current picture differs from the original or full resolution, the encoder or decoder may perform on-the-fly downsampling from the stored full resolution picture. Regarding picture output, the DPB always outputs the original (not resampled) reference picture.

[0035] In some exemplary embodiments, the resampling ratio can be arbitrarily chosen and the vertical and horizontal scaling ratios can be different. Because picture-level resampling is applied, the block-level operations of the encoder/decoder are made independent of the resolution of the reference picture, thereby without further complicating the block-level design logic of the encoder/decoder. , is allowed to allow arbitrary resampling ratios.

[0036] In some exemplary embodiments, coding resolution and full resolution signaling may be performed as follows. The full resolution of any picture of the video sequence is signaled in the SPS. The coding resolution of a picture can be signaled in the PPS or in the slice header. In both cases, one flag is signaled that indicates whether the encoding resolution is the same as the full resolution. If the encoding resolution is not the same as the maximum resolution, the encoding width and height of the current picture are also signaled. When PPS signalling is used, the signaled coding resolution applies to all pictures that reference this PPS. If slice header signalling is used, the signaled coding resolution applies only to the current picture itself. In some embodiments, the difference between the current encoding resolution and the maximum signaled resolution may be signaled in the SPS.

[0037] In some exemplary embodiments, instead of using an arbitrary resampling ratio, the resolution is a pre-defined set of N supported resolutions, where N is the number of supported resolutions in the sequence. is limited). The values of N and supported resolutions can be signaled in the SPS. The coding resolution of a picture is signaled using the PPS or slice header. In both cases, instead of signaling the actual coding resolution, the corresponding resolution index is signaled. Table 3 (Fig. 7) shows an example of a set of supported resolutions where the encoding sequence allows for three different resolutions. If the coding resolution of the current picture is 1440×816, the corresponding index value (=1) is signaled using the PPS or slice header.

[0038] In some exemplary embodiments, both resampled and non-resampled reference pictures are stored in the DPB. For each reference picture, one original (ie, non-resampled) picture and N−1 resampled copies are saved. If the resolution of the reference picture is different from the resolution of the current picture, the resampled picture of the DPB is used to encode the current picture. Regarding picture output, the DPB outputs the original (not resampled) reference picture. As an example, FIG. 8 shows the DPB occupancy for the supported resolution set shown in Table 3 (FIG. 7). As shown in FIG. 8, the DPB contains N (eg, N=3) copies of each picture that are used as reference pictures.

[0039] The choice of value for N depends on the application. As the value of N increases, flexibility in resolution selection increases, but complexity and memory requirements increase. Small values of N are suitable for low complexity devices, but limit the resolutions that can be selected. Therefore, in some embodiments flexibility may be provided to the encoder to determine the value of N to be signaled by the SPS based on application and device capabilities.

[0040] Consistent with this disclosure, downsampling of reference pictures may be done incrementally (ie, over time). In conventional (ie, direct) downsampling, the downsampling of the input image is done in a single step. For higher downsampling ratios, single step downsampling requires longer tap downsampling filters to avoid severe aliasing problems. However, longer tap filters are computationally expensive. In some exemplary embodiments, downsampling occurs gradually when the downsampling ratio is greater than a threshold (eg, greater than 2:1 downsampling) to maintain the quality of the downsampled image. A progressive downsampling is used. For example, a downsampling filter sufficient for 2:1 downsampling may be repeatedly applied to implement downsampling ratios greater than 2:1. FIG. 9 shows an example of progressive downsampling, where quarter downsampling is performed over two pictures in both horizontal and vertical dimensions. The first picture is downsampled by a factor of two (in both directions) and the second picture is downsampled by a factor of two (in both directions).

[0041] Consistent with this disclosure, the disclosed picture-level resampling methods can be used with other coding tools. For example, in some exemplary embodiments, a scaled motion vector based on a resampled reference picture is a temporal motion vector predictor (TMVP) or an advanced temporal motion vector predictor ( It can be used during ATMVP (advanced temporal motion vector predictor).

[0042] Next, a test methodology for evaluating coding tools for ARC is described. Since adaptive resolution change is mainly used to adapt to network bandwidth, the following test conditions can be considered to compare the coding efficiency of different ARC schemes during network bandwidth change. .

[0043] In some embodiments, the resolution is changed in half (both vertically and horizontally) at a particular instance of time, and then changed back to the original resolution after a period of time. FIG. 10 shows an example of resolution change. At time t ₁ the resolution is changed to half and then back to full resolution at time t ₂ .

[0044] In some embodiments, when the resolution is reduced, if the downsampling ratio is too large (eg, the downsampling ratio for a given dimension is greater than 2:1), incremental downsampling is performed over a period of time. is used. However, when upsampling is used, gradual upsampling is not used.

[0045] In some embodiments, a scalable HEVC test model (SMH) downsampling filter may be used for source resampling. Table 4 (FIG. 11) shows the detailed filter coefficients with different fractional sample positions.

[0046] In some exemplary embodiments, two peak signal-to-noise ratios (PSNR) are calculated to measure video quality. A diagram of the PSNR calculation is shown in FIG. A first PSNR is computed between the resampled source and the decoded picture. A second PSNR is computed between the original source and the upsampled decoded source.

[0047] In some embodiments, the number of encoded pictures is the same as the current test conditions for VVC.

[0048] The disclosed block-based resampling method will now be described. Block-based resampling can reduce the information loss described above by combining resampling and motion compensated interpolation into one filtering operation. Take the following case as an example: the motion vector of the current block has half-pel accuracy in one dimension (e.g. in the horizontal dimension) and the width of the reference picture is twice the width of the current picture. . In this case, compared to picture-level resampling that reduces the width of the reference picture by half to match the width of the current picture, followed by half-pel motion interpolation, the block-based resampling method is , with half-pel precision, directly fetching the odd positions of the reference picture as reference blocks. At the 15th JVET meeting, a block-based resampling method for ARC, in which motion compensated (MC) interpolation and reference resampling are integrated and performed with a single-step filter, was adopted for VVC. In VVC Draft 6, existing filters for MC interpolation without reference resampling are reused for MC interpolation with reference resampling. The same filter is used for both reference upsampling and downsampling. Details of filter selection are described below.

[0049] For the luma component, if the half-pel AMVR mode is selected and the interpolation position is half-pel, a 6-tap filter [3, 9, 20, 20, 9, 3] is used and the motion compensation block size is 4× If 4, then the following 6-tap filter as shown in Table 5 (FIG. 13) is used, otherwise an 8-tap filter as shown in Table 6 (FIG. 14) is used. For the chroma component, the 4-tap filter shown in Table 7 (FIG. 15) is used.

[0050] In VVC, the same filter is used for MC interpolation without reference resampling and for MC interpolation with reference resampling. Although the VVC MCIF is designed based on DCT upsampling, it may not be suitable to use it as a single-step filter integrating reference downsampling and MC interpolation. For example, for phase 0 filtering (i.e., scaled mv is an integer), the VVC 8-tap MCIF coefficients are [0,0,0,64,0,0,0,0], which is the predicted It means that the sample was copied directly from the reference sample. This may not be a problem for MC interpolation without reference downsampling or with reference upsampling, but for the reference downsampling case the lack of a low-pass filter before decimation leads to May cause aliasing artifacts.

[0051] This disclosure provides a method of using a cosine-windowed sinc filter for MC interpolation with reference downsampling.

[0052] A windowed sinc filter is a bandpass filter that separates one band of frequencies from another. A windowed sinc filter is a lowpass filter with a frequency response that passes all frequencies below the cutoff frequency with an amplitude of 1 and cuts all frequencies above the cutoff frequency with a zero amplitude, as shown in FIG. is.

式中、ｆｃは、［０，１］の値を有するカットオフ周波数であり、ｒは、ダウンサンプリング比、すなわち、１．５：１のダウンサンプリングに関して１．５、及び２：１のダウンサンプリングに関して２である。シンク関数は、以下のように定義される。

[0053] The filter kernel, also known as the filter's impulse response, is obtained by taking the inverse Fourier transform of the frequency response of an ideal low-pass filter. The impulse response of a low-pass filter is the general form of a sinc function.

where fc is the cutoff frequency with values of [0,1] and r is the downsampling ratio, i.e. 1.5 for 1.5:1 downsampling and 2:1 downsampling is 2 with respect to A sink function is defined as follows.

[0054] The sink function is infinite. To make the filter kernel of finite length, a window function is applied to truncate the filter kernel to L points. To obtain a smooth tapered curve, a cosine windowing function is used, which is obtained by:

[0055] The kernel of a cosine windowed sinc filter is the product of the ideal response function h(n) and the cosine window function w(n).

[0056] There are two parameters selected for the windowed sinc kernel, the cutoff frequency fc and the kernel length L. For the downsampling filter used in the scalable HEVC test model (SHM), fc=0.9 and L=13.

[0057] The filter coefficients obtained in (4) are real numbers. Applying the filter is equivalent to computing a weighted average of the reference samples, with the weights as the filter coefficients. For efficient computation in a digital computer or hardware, the coefficients are normalized, multiplied by a scaler, and integer rounded to . The filtered samples are divided by 2 to the Nth power (equivalent to a right shift of N bits). For example, in VVC Draft 6, the total number of interpolation filter coefficients is 64.

[0058] In some disclosed embodiments, a downsampling filter is used in SHM for VVC motion compensated interpolation using reference downsampling for both luma and chroma components, and the existing MCIF is used for motion compensated interpolation using reference upsampling. If the kernel length L=13 and the first coefficient is small and rounded to zero, the filter length can be reduced to 12 without affecting filter performance.

[0059] As an example, filter coefficients for 2:1 downsampling and 1.5:1 downsampling are shown in Table 8 (Fig. 17) and Table 9 (Fig. 18), respectively.

[0060] Besides the values of the coefficients, there are some other differences between SHM filter designs and existing MCIF designs.

[0061] As a first difference, SHM filters require filtering at integer and fractional sample positions, whereas MCIF only requires filtering at fractional sample positions. An example of a modification to the luma sample interpolation filtering process for the reference downsampling case of VVC Draft 6 is listed in Table 10 (FIG. 19).

[0062] An example of a change to the chroma sample interpolation filtering process for the reference downsampling case of VVC Draft 6 is listed in Table 11 (FIG. 20).

[0063] As a second difference, for the SHM filter, the sum of filter coefficients is 128, whereas for the existing MCIF, the sum of filter coefficients is 64. In VVC Draft 6, the intermediate prediction signal is kept to a higher precision (represented by a larger bit depth) than the output signal in order to reduce the loss caused by rounding errors. The accuracy of the intermediate signal is called internal accuracy. In one embodiment, to keep the internal precision the same as VVC Draft 6, the output of the SHM filter needs to be right-shifted an additional bit compared to using the existing MCIF. An example of a modification to the luma sample interpolation filtering process for the VVC Draft 6 reference downsampling case is shown in Table 12 (FIG. 21).

[0064] An example of a change to the chroma sample interpolation filtering process for the reference downsampling case of VVC Draft 6 is shown in Table 13 (FIG. 22).

[0065] According to some embodiments, the internal precision can be increased by one bit, and an additional one-bit right shift can be used to convert the internal precision to the output precision. An example of a modification to the luma sample interpolation filtering process for the VVC Draft 6 reference downsampling case is shown in Table 14 (FIG. 23).

[0066] An example of a change to the chroma sample interpolation filtering process for the reference downsampling case of VVC Draft 6 is shown in Table 15 (Fig. 24).

[0067] As a third difference, the SHM filter has 12 taps. Therefore, 11 adjacent samples (5 left, 6 right, or 5 top, 6 bottom) are required to generate an interpolated sample. Additional neighboring samples are fetched compared to MCIF. In VVC draft 6, the chroma mv precision is 1/32. However, SHM filters only have 16 phases. Therefore, the chroma mv can be rounded to 1/16 with respect to the reference downsampling. This can be done by right shifting the last 5 bits of the chroma mv by 1 bit. An example of a change to the chroma fractional sample position calculation for the reference downsampling case of VVC Draft 6 is shown in Table 16 (FIG. 25).

[0068] In some embodiments, to match the existing MCIF design of the VVC draft, we propose the use of an 8-tap cosine-windowed sinc filter. The filter coefficients can be derived by setting L=9 in the cosine windowed sinc function of equation (4). To further match existing MCIF filters, the total filter coefficients can be set to 64. Exemplary filter coefficients for ratios of 2:1 and 1.5:1 are shown in Table 17 (FIG. 26) and Table 18 (FIG. 27), respectively.

[0069] According to some embodiments, a 32-phase cosine windowed sinc filter set is used in chroma motion compensated interpolation using reference downsampling to accommodate 1/32 sample accuracy for the chroma components. can be Examples of filter coefficients for ratios of 2:1 and 1.5:1 are shown in Table 19 (FIG. 28) and Table 20 (FIG. 29), respectively.

[0070] According to some embodiments, for a 4x4 luma block, a 6-tap cosine-windowed sinc filter may be used for MC interpolation with reference downsampling. Examples of filter coefficients for ratios of 2:1 and 1.5:1 are shown below in Tables 21 (FIG. 30) and 22 (FIG. 31), respectively.

[0071] In some embodiments, a non-transitory computer-readable storage medium is also provided that includes instructions, which may be executed by devices (such as the disclosed encoders and decoders) to perform the methods described above. Common forms of non-transitory media include, for example, floppy disks, floppy disks, hard disks, solid state drives, magnetic tapes or other magnetic data storage media, CD-ROMs, and other optical data storage. Media, any physical media with a pattern of holes, RAM, PROM and EPROM, FLASH-EPROM or other flash memory, NVRAM, cache, registers, other memory chips or cartridges, and networking of the above version is included. A device may include one or more processors (CPUs), input/output interfaces, network interfaces, and/or memory.

[0072] The relative terms herein such as "first" and "second" are only used to distinguish one entity or action from another entity or action, and these entities Note that no actual relationship or order between acts is required or implied. Also, the terms "comprising," "having," "containing," and "including," and other similar forms are equivalent in meaning, and The term or terms following any one of these terms are not limited in any way to an exhaustive list of such terms or terms, or to only the listed term or terms. In that respect it is intended to be an open-ended format.

[0073] As used herein, unless stated otherwise, the term "or" encompasses all possible combinations unless impracticable. For example, where it is stated that a database can include A or B, the database can include A or B, or A and B, unless stated otherwise or impracticable. As a second example, when it is stated that a database may contain A, B, or C, unless otherwise stated or impracticable, the database may include A, or B, or C, or It can include A and B, or A and C, or B and C, or A and B and C.

[0074] It will be appreciated that the above embodiments may be implemented in hardware, or software (program code), or a combination of hardware and software. When implemented by software, it can be stored on any of the computer readable media described above. The software, when executed by a processor, can perform the disclosed methods. The computing units and other functional units described in this disclosure may be implemented by hardware or software or a combination of hardware and software. Those skilled in the art will also appreciate that multiple of the above modules/units may be integrated as one module/unit, and that each of the above modules/units may be further divided into multiple sub-modules/sub-units. will understand.

[0075] In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Certain adaptations and modifications of the described embodiments may be made. Other embodiments may be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the above specification and examples be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims. Also, the order of steps shown in the figures is intended for illustration only and is not intended to be limited to any particular order of steps. As such, those skilled in the art will appreciate that these steps may be performed in a different order while performing the same method.

を有し、式中、

であり、ｆｃが、コサイン窓化シンクフィルタのカットオフ周波数であり、Ｌが、カーネル長さであり、ｒが、ダウンサンプリング比である、条項１５に記載の方法。
１７．ｆｃが、０．９に等しく、Ｌが、１３に等しい、条項１６に記載の方法。
１８．コサイン窓化シンクフィルタが、８タップフィルタである、条項１５～１７の何れか一項に記載の方法。
１９．コサイン窓化シンクフィルタが、４タップフィルタである、条項１５～１７の何れか一項に記載の方法。
２０．コサイン窓化シンクフィルタが、３２位相フィルタである、条項１５～１７の何れか一項に記載の方法。
２１．３２位相フィルタが、クロマ動き補償インターポレーションで使用される、条項２０に記載の方法。
２２．帯域通過フィルタを参照ピクチャに適用することが、
小数サンプル位置でルマサンプル又はクロマサンプルを取得することを含む、条項１４～２１の何れか一項に記載の方法。
２３．デバイスであって、
コンピュータ命令を保存する１つ又は複数のメモリと、
コンピュータ命令を実行して、デバイスに、
ターゲットピクチャ及び参照ピクチャが異なる解像度を有することに応答して、動き補償インターポレーションを行うため、及び参照ブロックを生成するために、帯域通過フィルタを参照ピクチャに適用することと、
参照ブロックを使用して、ターゲットピクチャのブロックのエンコーディング又はデコーディングを行うことと、
を行わせるように構成される１つ又は複数のプロセッサと、
を含む、デバイス。
２４．一組の命令を保存する非一時的コンピュータ可読媒体であって、前記一組の命令は、映像コンテンツを処理する方法をコンピュータシステムに行わせるように、コンピュータシステムの少なくとも１つのプロセッサによって実行可能であり、前記方法が含む：
２５．一組の命令を保存する非一時的コンピュータ可読媒体であって、前記一組の命令は、映像コンテンツを処理する方法をコンピュータシステムに行わせるように、コンピュータシステムの少なくとも１つのプロセッサによって実行可能であり、前記方法が、
ターゲットピクチャ及び第１の参照ピクチャの解像度を比較することと、
ターゲットピクチャ及び第１の参照ピクチャが異なる解像度を有することに応答して、第２の参照ピクチャを生成するために第１の参照ピクチャを再サンプリングすることと、
第２の参照ピクチャを使用して、ターゲットピクチャのエンコーディング又はデコーディングを行うことと、
を含む、非一時的コンピュータ可読媒体。 [0076] Embodiments can be further described using the following clauses.
1. A computer-implemented video processing method comprising:
comparing resolutions of a target picture and a first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding a target picture using a second reference picture;
method including.
2. storing a second reference picture in a first buffer, the first buffer being different than a second buffer storing decoded pictures used for prediction of future pictures; ,
removing the second reference picture from the first buffer after completing the encoding or decoding of the target picture;
The method of Clause 1, further comprising:
3. encoding or decoding the target picture,
3. The method of any one of clauses 1 and 2, comprising encoding or decoding the target picture by using no more than a predetermined number of resampled reference pictures.
4. encoding or decoding the target picture,
4. The method of clause 3, comprising signaling the predetermined number as part of a sequence parameter set or a picture parameter set.
5. resampling the first reference picture to generate a second reference picture;
Storing a first version and a second version of a first reference picture, the first version having the original resolution and a second version capable of resampling the reference picture. storing with full resolution;
resampling the maximum version to generate a second reference picture;
The method of clause 1, comprising
6. storing a first version and a second version of the first reference picture in a decoded picture buffer;
outputting the first version for encoding future pictures;
6. The method of clause 5, further comprising
7. resampling the first reference picture to generate a second reference picture;
2. The method of clause 1, comprising generating the second reference picture at the supported resolution.
8. number of supported resolutions and
pixel dimensions corresponding to supported resolutions, and
8. The method of clause 7, further comprising signaling as part of a sequence parameter set or a picture parameter set information indicative of.
9. information is
9. The method of clause 8, including an index corresponding to at least one of the supported resolutions.
10. 10. The method of any one of Clauses 8 and 9, further comprising setting the number of supported resolutions based on the configuration of the video application or video device.
11. resampling the first reference picture to generate a second reference picture;
11. The method of any one of clauses 1-10, wherein progressive downsampling of a first reference picture to generate a second reference picture.
12. a device,
one or more memories storing computer instructions;
execute computer instructions to the device,
comparing the resolution of the target picture to the resolution of the first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding a target picture using a second reference picture;
one or more processors configured to cause
device, including
13. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by at least one processor of a computer system to cause the computer system to perform a method of processing video content. Yes, the method comprising:
comparing resolutions of a target picture and a first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding a target picture using a second reference picture;
A non-transitory computer-readable medium, including
14. applying a bandpass filter to a reference picture to perform motion compensated interpolation and to generate a reference block in response to the target picture and the reference picture having different resolutions;
encoding or decoding a block of a target picture using the reference block;
A computer-implemented video processing method comprising:
15. 15. The method of clause 14, wherein the bandpass filter is a cosine windowed sinc filter.
16. A cosine-windowed sinc filter uses the kernel function

and where

16. The method of clause 15, wherein fc is the cutoff frequency of the cosine windowed sinc filter, L is the kernel length, and r is the downsampling ratio.
17. 17. The method of clause 16, wherein fc equals 0.9 and L equals 13.
18. 18. The method of any one of clauses 15-17, wherein the cosine-windowed sinc filter is an 8-tap filter.
19. 18. The method of any one of clauses 15-17, wherein the cosine windowed sinc filter is a 4-tap filter.
20. 18. A method according to any one of clauses 15-17, wherein the cosine windowed sinc filter is a 32-phase filter.
21. The method of clause 20, wherein a 32 phase filter is used in chroma motion compensated interpolation.
22. Applying a bandpass filter to the reference picture
22. The method of any one of clauses 14-21, comprising obtaining luma or chroma samples at fractional sample positions.
23. a device,
one or more memories storing computer instructions;
execute computer instructions to the device,
applying a bandpass filter to a reference picture to perform motion compensated interpolation and to generate a reference block in response to the target picture and the reference picture having different resolutions;
encoding or decoding a block of a target picture using the reference block;
one or more processors configured to cause
device, including
24. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by at least one processor of a computer system to cause the computer system to perform a method of processing video content. Yes, the method includes:
25. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by at least one processor of a computer system to cause the computer system to perform a method of processing video content. Yes, the method comprising:
comparing resolutions of a target picture and a first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding a target picture using the second reference picture;
A non-transitory computer-readable medium, including

[0077] Exemplary embodiments have been disclosed in the drawings and specification. However, many variations and modifications can be made to these embodiments. Accordingly, although specific terms have been employed, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims (20)

A computer-implemented video processing method comprising:
comparing resolutions of a target picture and a first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding the target picture using the second reference picture;
method including. storing the second reference picture in a first buffer, the first buffer being different than a second buffer storing decoded pictures used for prediction of future pictures; and
removing the second reference picture from the first buffer after completing the encoding or decoding of the target picture;
2. The method of claim 1, further comprising: encoding or decoding the target picture;
2. The method of claim 1, comprising encoding or decoding the target picture by using no more than a predetermined number of resampled reference pictures. encoding or decoding the target picture;
4. The method of claim 3, comprising signaling the predetermined number as part of a sequence parameter set or a picture parameter set. resampling the first reference picture to generate the second reference picture;
storing a first version and a second version of the first reference picture, the first version having an original resolution and the second version being a reproduction of the reference picture; storing, having the maximum resolution at which sampling is possible;
resampling the maximum version to generate the second reference picture;
2. The method of claim 1, comprising: storing the first version and the second version of the first reference picture in a decoded picture buffer;
outputting the first version for encoding future pictures;
6. The method of claim 5, further comprising: resampling the first reference picture to generate the second reference picture;
2. The method of claim 1, comprising generating the second reference picture at a supported resolution. number of supported resolutions and
pixel dimensions corresponding to said supported resolution;
8. The method of claim 7, further comprising signaling as part of a sequence parameter set or a picture parameter set information indicative of . said information is
9. The method of claim 8, comprising an index corresponding to at least one of said supported resolutions. 9. The method of claim 8, further comprising setting the number of supported resolutions based on a video application or video device configuration. resampling the first reference picture to generate the second reference picture;
2. The method of claim 1, comprising progressively downsampling the first reference picture to generate the second reference picture. a device,
one or more memories storing computer instructions;
executing the computer instructions to cause the device to:
comparing the resolution of the target picture to the resolution of the first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding the target picture using the second reference picture;
one or more processors configured to cause
device, including the one or more processors executing the computer instructions to cause the device to:
storing the second reference picture in a first buffer, the first buffer being different than a second buffer storing decoded pictures used for prediction of future pictures; and
removing the second reference picture from the first buffer after completing the encoding or decoding of the target picture;
13. The device of claim 12, further configured to cause: the one or more processors executing the computer instructions to cause the device to:
13. The device of claim 12, further configured to cause encoding or decoding of the target picture by using no more than a predetermined number of resampled reference pictures. the one or more processors executing the computer instructions to cause the device to:
15. The device of claim 14, further configured to cause signaling of the predetermined number as part of a sequence parameter set or a picture parameter set. the one or more processors executing the computer instructions to cause the device to:
storing a first version and a second version of the first reference picture in the one or more memories, the first version having an original resolution and the second version having an original resolution; saving a version having a maximum resolution at which the reference picture can be resampled;
resampling the maximum version to generate the second reference picture;
13. The device of claim 12, further configured to cause: the one or more processors executing the computer instructions to cause the device to:
storing the first version and the second version of the first reference picture in a decoded picture buffer;
outputting the first version for encoding future pictures;
17. The device of claim 16, further configured to cause: the one or more processors executing the computer instructions to cause the device to:
13. The device of Claim 12, further configured to generate the second reference picture at a supported resolution. the one or more processors executing the computer instructions to cause the device to:
number of supported resolutions and
pixel dimensions corresponding to said supported resolution;
19. The device of claim 18, further configured to cause signaling of information indicative of as part of a sequence parameter set or a picture parameter set. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by at least one processor of a computer system to cause the computer system to perform a method of processing video content. and wherein the method is
comparing resolutions of a target picture and a first reference picture;
resampling the first reference picture to generate a second reference picture in response to the target picture and the first reference picture having different resolutions;
encoding or decoding the target picture using the second reference picture;
A non-transitory computer-readable medium, including

Images