WO2023041317A1 - Procédé et appareil de codage et de décodage vidéo avec échantillonnage de résidus de chrominance - Google Patents

Procédé et appareil de codage et de décodage vidéo avec échantillonnage de résidus de chrominance Download PDF

Info

Publication number
WO2023041317A1
WO2023041317A1 PCT/EP2022/073995 EP2022073995W WO2023041317A1 WO 2023041317 A1 WO2023041317 A1 WO 2023041317A1 EP 2022073995 W EP2022073995 W EP 2022073995W WO 2023041317 A1 WO2023041317 A1 WO 2023041317A1
Authority
WO
WIPO (PCT)
Prior art keywords
chroma
block
sampling
residuals
chroma residuals
Prior art date
Application number
PCT/EP2022/073995
Other languages
English (en)
Inventor
Karam NASER
Edouard Francois
Tangi POIRIER
Gaelle Martin-Cocher
Original Assignee
Interdigital Vc Holdings France, Sas
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Interdigital Vc Holdings France, Sas filed Critical Interdigital Vc Holdings France, Sas
Publication of WO2023041317A1 publication Critical patent/WO2023041317A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • At least one of the present embodiments generally relates to a method or an apparatus for video encoding or decoding, and more particularly, to a method or an apparatus comprising sampling chroma residuals.
  • image and video coding schemes usually employ prediction, including motion vector prediction, and transform to leverage spatial and temporal redundancy in the video content.
  • prediction including motion vector prediction, and transform
  • intra or inter prediction is used to exploit the intra or inter frame correlation, then the differences between the original image and the predicted image, often denoted as prediction errors or prediction residuals, are transformed, quantized, and entropy coded.
  • the compressed data are decoded by inverse processes corresponding to the entropy coding, quantization, transform, and prediction.
  • Recent additions to video compression technology include various industry standards, versions of the reference software and/or documentations such as Joint Exploration Model (JEM) and later VTM (Versatile Video Coding (VVC) Test Model) being developed by the JVET (Joint Video Exploration Team) group.
  • JEM Joint Exploration Model
  • VTM Very Video Coding
  • JVET Joint Video Exploration Team
  • the aim is to make further improvements to the existing HEVC (High Efficiency Video Coding) standard.
  • YUV 4:2:0 format a pre-coding format that reduces redundancy of chroma data using horizontal and vertical subsampling.
  • existing methods for coding and decoding also address alternative pre-coding data formats, for instance with unsampled data such as YUV 4:4:4 or RGB 4:4:4. Therefore, there is a need to improve the state of the art by providing encoding and decoding methods that allows more efficient coding of the chroma data.
  • the method comprises video decoding by obtaining coded video data comprising at least one block of inverse transformed chroma residuals; upsampling the at least one block of inverse transformed chroma residuals; and decoding coded video data by adding the at least one block of up-sampled inverse transformed chroma residuals to at least one block of chroma predicted samples.
  • a second method comprises video encoding by obtaining video data to encode, video data comprising at least one block of chroma residuals; subsampling the at least one block of chroma residuals; applying transform to the at least one block of subsampled chroma residuals; and encoding the at least one block of transformed chroma residuals into coded video data.
  • an apparatus comprising one or more processors, wherein the one or more processors are configured to implement the method for video decoding according to any of its variants.
  • the apparatus for video decoding comprises means for obtaining coded video data comprising at least one block of inverse transformed chroma residuals; means for upsampling the at least one block of inverse transformed chroma residuals; and means for decoding coded video data by adding the at least one block of up-sampled inverse transformed chroma residuals to at least one block of chroma predicted samples.
  • the apparatus comprises one or more processors, wherein the one or more processors are configured to implement the method for video decoding according to any of its variants.
  • the apparatus for video decoding comprises means for obtaining video data to encode, the video data comprising at least one block of chroma residuals; means for subsampling the at least one block of chroma residuals; means for applying transform to the at least one block of subsampled chroma residuals; and encoding the at least one block of transformed chroma residuals into coded video data.
  • At least one syntax data element indicating to perform chroma residuals sampling is signaled.
  • the sampling is performed according to at least one direction among a horizontal direction, a vertical direction.
  • a sampling filter is applied the at least one block of chroma residuals/inverse transformed chroma residuals.
  • the sampling is enabled only for coded video data with a chroma sampling format being one of 4:2:2 or 4:4:4
  • at least one syntax data element indicating a direction of the chroma residuals sampling is signaled.
  • the chroma sampling format of the coded video data is 4:2:2 and wherein a direction of the chroma residuals sampling is vertical.
  • the color chroma sampling format of the coded video data is RGB 4:4:4, and the chroma residuals sampling is combined with adaptive color transform.
  • At least one syntax data element indicating a ratio of the chroma residuals sampling.
  • a device comprising an apparatus according to any of the decoding embodiments; and at least one of (i) an antenna configured to receive a signal, the signal including the video block, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the video block, or (iii) a display configured to display an output representative of the video block.
  • a non- transitory computer readable medium containing data content generated according to any of the described encoding embodiments or variants.
  • a signal comprising video data generated according to any of the described encoding embodiments or variants.
  • a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variants.
  • a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the described encoding/decoding embodiments or variants.
  • Figure 1 illustrates a generic decoding method according to a general aspect of at least one embodiment.
  • Figure 2 illustrates a generic encoding method according to a general aspect of at least one embodiment.
  • Figures 3a, 3b and 3c illustrate various examples of chroma blocks and subsampled chroma blocks according to a general aspect of at least one embodiment.
  • Figure 4 illustrates a generic decoding method according to at least one embodiment.
  • Figure 5 illustrates a block diagram of an embodiment of video encoder in which various aspects of the embodiments may be implemented.
  • Figure 6 illustrates a block diagram of an embodiment of video decoder in which various aspects of the embodiments may be implemented.
  • Figure 7 illustrates a block diagram of an example apparatus in which various aspects of the embodiments may be implemented.
  • the various embodiments are described with respect to the encoding/decoding of an image. They may be applied to encode/decode a part of image, such as a slice or a tile, a tile group or a whole sequence of images.
  • each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and/or use of specific steps and/or actions may be modified or combined.
  • At least some embodiments relate to method for encoding or decoding a video wherein subsampling/upsampling of chroma residuals allows to improve the coding gain.
  • Chroma subsampling is the practice of encoding images by implementing less resolution for chroma information than for luma information, taking advantage of the human visual system's lower acuity for color differences than for luminance.
  • Figure 3a illustrates various subsampling chroma format used in encoding or decoding method.
  • the video data is divided into a luma (Y) component and two color difference components (chroma Cb, Cr or U, V).
  • chroma Cb, Cr or U, V color difference components
  • a variety of filtering methods can be used to arrive at the resolution-reduced chroma values.
  • the subsampling scheme is commonly expressed as a three-part ratio J:a:b (e.g. 4:2:2) that describe the number of luminance (Y) and chrominance samples (Cr, Cb or U, V) in a conceptual region that is J pixels wide and 2 pixels high.
  • the parts are (in their respective order):
  • VVC defines other formats as defined in the table below:
  • each of the two chroma arrays has half the height and half the width of the luma array.
  • each of the two chroma arrays has the same height and half the width of the luma array.
  • each of the two chroma arrays has the same height and width as the luma array.
  • VVC further specifies a tool, called adaptive color transform (ACT), that converts the residual samples of RBG block to YCgCo format, similar to YLIV, using the following formula:
  • the encoder may advantageously select the ACT tool at coding unit level based on a rate distortion optimization i.e. in case this conversion applied to residuals of a coding unit provides better coding efficiency.
  • Figure 1 illustrates a generic decoding method (100) according to a general aspect of at least one embodiment.
  • the block diagram of Figure 1 partially represents modules of a decoder or decoding method, for instance implemented in the exemplary decoder of Figure 6.
  • the present principles allow the encoder to subsample the chroma component to improve coding efficiency.
  • the subsampling is performed on residuals before the forward transform, and the up-sampling is performed after the inverse transform (at both encoder and decoder sides) so that the same resolution is retained at the decoder side as well as inside the reference picture buffer for inter prediction.
  • a generic decoding method 100 comprises a step of obtaining 1 10 coded video data comprising at least one block of inverse transformed chroma residuals.
  • Such inverse transformed chroma residuals correspond to an output of the inverse transform module 550 of the exemplary encoder of Figure 5 or to an output of the inverse transform module 650 of the exemplary decoder of Figure 6.
  • the at least one block of inverse transformed chroma residuals is upsampled, at both encoder and decoder sides, so that a same resolution is retained for block of upsampled chroma residuals as for block of predicted samples (inter or intra) as well as inside the reference picture buffer for inter prediction.
  • the further decoding of coded video is performed by adding 130 the at least one block of upsampled inverse transformed chroma residuals to the at least one block of chroma predicted samples.
  • Both upsampled inverse transformed chroma residuals and chroma predicted samples thus have a same resolution as specified in the original pre-coding chroma format.
  • Figure 2 illustrates a generic encoding method (200) according to a general aspect of at least one embodiment.
  • the block diagram of Figure 2 partially represents modules of an encoder or encoding method, for instance implemented in the exemplary encoder of Figure 5.
  • a generic encoding method 200 comprises a step of obtaining 210 coded video data comprising at least one block of chroma residuals. .Such chroma residuals are obtained from the difference 210 between chroma samples and chroma predicted samples (being inter or intra prediction). Such chroma residuals correspond to an input of the transform module 525 of the exemplary encoder of Figure 5. Then, in a step 220, the at least one block of chroma residuals is subsampled for to reduce the coding cost.
  • the further encoding of video data is performed by applying forward transform 230 to the at least one block of subsampled chroma residuals; and applying quantization (not shown on Figure 2) and entropy coding (not shown on Figure 2) to form coded video data as in an exemplary video encoder.
  • a chroma sampling format of the coded video data is one of 4:2:2 or 4:4:4.
  • the subsampling method is preferably adapted to subsample the chroma residuals of non-4:2:0 format such as nonlimiting examples 4:2:2 or 4:4:4.
  • the present principles are also compatible with 4:2:0 format.
  • the sampling comprises filtering chroma residuals.
  • default sampling filters of VVC can be used for the sub-sampling or the up-sampling process. These filters are used for the Reference Picture Resampling (RPR) process which allow different size of reference picture in the reference picture buffer RPB. Luma and chroma data of a picture are sampled before storage in RPB then resampled at the output of the RPB in order to perform regular decoding operations on original picture size.
  • RPR Reference Picture Resampling
  • Luma and chroma data of a picture are sampled before storage in RPB then resampled at the output of the RPB in order to perform regular decoding operations on original picture size.
  • Such sampling filters are defined among the 16 phases interpolation filters used in inter prediction for high precision (1/16 pel) motion vector storage and motion compensation.
  • interpolation filters are 8-tap interpolation filter for luma component, 6-tap interpolation filter for luma component for CUs 4x4, 4-tap interpolation filter for chroma component.
  • upsampling is the inverse process of subsampling, corresponding filter and inverse filter are respectively used for subsampling and upsampling.
  • At least one syntax data element indicating to perform chroma residuals sampling is decoded from the bitstream of coded video data. For instance, at the CU level, a new flag is added to signal the usage of chroma sampling. The flag is name as “cu_chroma_sampling_flag”.
  • An example of the signaling jointly with ACT is proposed in the following (modification of syntax appears as underlined):
  • the flag “cu_chroma_sampling_flag” is received and parsed by the decoder. A test is performed on the flag. In case where the flag indicates to perform chroma residuals sampling, for instance when cu_chroma_sampling_flag is equal to 1 , the sampling (upsampling 120 at decoding or subsampling 220 at encoding) is applied at the chroma residuals at the block level (or coding unit level). Otherwise, the sampling (upsampling 120 at decoding or subsampling 220 at encoding) is skipped for the block level (or coding unit level) and the chroma residuals remains unchanged. According to different variants, the cu_chroma_sampling_flag could be used for any of predictions such as Intra prediction, Inter prediction or Intra prediction with ACT as detailed below.
  • the sampling is performed according to at least one direction among a horizontal direction, a vertical direction. For instance, when cu_chroma_sampling_flag is equal to 1 , horizontal and/or vertical chroma subsampling is performed for chroma residuals before forward transform.
  • U(x,y) and V(x,y) represent the residual buffer of the chroma component (II and V) with dimension NxM, the following is performed:
  • V*(1:N/2,y) Fil H (V(1:2:N,y))
  • 1 :N means all elements from first to nth sample.
  • 1 :2:N is the same with pixel sampling by factor of one.
  • V*(x, 1:M/2) Fil_V(U(x, 1:2:M))
  • the sampling comprises filtering the at least one block of chroma residuals.
  • Fil H and Fil_ V respectively are horizontal sampling filter and vertical sampling filter while L/*and I/* are the subsampled chroma components.
  • the horizontal and vertical filters can be any type of low pass filters like the ones used for subpixel motion compensation or for reference picture resampling.
  • the filtering can be simply ignored in the subsampling for reducing the complexity.
  • Figure 3b and 3c illustrates various examples of chroma blocks and subsampled chroma blocks according to a general aspect of at least one embodiment.
  • Figure 3b shows on the upper line another representation of the 4:4:4 format as previously illustrated with figure 3a.
  • 4:4:4 format there are as may chroma (Cr Cb) samples/residual samples as the luminance samples/residual samples in a block or coding unit, for instance 16 samples.
  • Figure 3b shows on the lower line, luminance and chroma residuals after chroma subsampling with both half width and half height as described above, the number of subsampled chroma residuals is then reduced to 4.
  • Figure 3c shows another example with 4:2:2 format.
  • the block of chroma (Cr Cb) samples/residual samples is subsampled to half width compared to a block of the luminance samples/residual samples.
  • Figure 3c shows on the lower line, luminance and chroma residuals after chroma subsampling with only half height.
  • the number of subsampled chroma residuals is then reduced to 4 as in 4:2:0 format.
  • 4:2:2 chroma format the width of the chroma components is half of luma width. That is, the source format is already horizontally subsampled. In this case, only vertical subsampling is allowed.
  • the chroma sampling format of the coded video data is 4:2:2 and a direction of the chroma residuals sampling is vertical.
  • the subsampling in 4:2:2 format is not limited to this embodiment and the twice subsampling in both width and height is also compatible with 4:2:2 format.
  • the sampling is applied to coded video data in a chroma sampling format being one of 4:2:2 or 4:4:4.
  • this embodiment optimizes the tradeoff between reduction of the chroma data and distortion since is 4:2:0 chroma is already widely subsampled in the source of the video data.
  • the sampling method implemented in the decoder corresponds to the sampling method implemented in the encoder.
  • chroma up- sampling is performed:
  • L/’ and V’ are the up-sampled chroma residuals. Both II’ and V’ undergo filtering process (upsampling filter as previously described) to fill in the missing pixels.
  • the chroma residuals sampling is jointly applied with ACT.
  • Figure 4 illustrates a decoding method according to at least one embodiment related to interaction between ACT and Chroma Residuals Sampling.
  • Adaptive Color Transform ACT is an effective tool for RGB 4:4:4 format. Its objective is to convert the RGB format to luma and chroma components during the coding process.
  • an inverse ACT is applied to the chroma residuals after inverse quantization and inverse transform to recover the residuals in RGB format. Therefore, the encoder select this mode if it provides better coding gain and signal to the decoder to apply inverse ACT using a flag cu_act_enabled_flag.
  • the ACT mode is further improved by allowing the encoder to further subsample the chroma residuals components to approach YUV 4:2:0 format.
  • an embodiment proposes to allow chroma sampling process for RGB 4:4:4 format only if ACT is used.
  • a chroma residuals upsampling is applied at the decoder that corresponds to the chroma residuals subsampling applied at the encoder.
  • the upsampling is before the inverse ACT.
  • both processes (upsampling and ACT) are independent and can be performed in any order.
  • the order at the decoder upsampling then inverse ACT
  • the flag cu_chroma_sampling_flag is conditionally determined based on cu_act_enabled_flag. In other words, the chroma subsampling flag is inferred to zero if ACT is not used. Accordingly in a decoder, the at least one syntax data element indicating to perform adaptive color transform on at least a block of color samples is received and parsed, and the at least one syntax data element indicating to perform chroma residuals upsampling is set to zero thus indicating to skip the chroma residuals subsamples in case the at least one syntax data element indicates not applying adaptive color transform.
  • the value of received and parsed syntax data element indicates whether to perform chroma residuals upsampling or not.
  • chroma sampling is constantly performed when ACT is enabled.
  • the at least one syntax data element indicating to perform adaptive color transform on at least one block of color samples is received and parsed; and on condition that the at least one syntax data element indicates performing adaptive color transform, applying both inverse adaptive color transform and upsampling on the at least one block of color residuals (in any order).
  • This embodiment reduces the signaling overhead (since cu_chroma_sampling_flag is not signaled) but reduces the encoder flexibility.
  • At least one syntax data element indicating a direction of the chroma residuals sampling is signaled from the encoder to the decoder.
  • this embodiment offers a flexible horizontal and vertical sampling scheme by allowing an encoder to indicate to a decoder, to perform either horizontal sampling or vertical sampling or even both horizontal and vertical sampling. Indeed, instead of performing both horizontal and vertical sampling the encoder can select to perform either of them based on ratedistortion decision. That is, instead of signaling a single flag “cu_chroma_sampling_flag” for chroma sampling, 2 flags are signaled to indicate weather horizontal or vertical sampling is performed. The following signaling is proposed (amendments to syntax are underlined):
  • cu_chroma_sampling_direction indicates the direction of sampling (0 means horizontal, 1 means vertical, 2 means horizontal and vertical).
  • the syntax element when chroma residual sampling applies for both directions, the syntax element can be duplicated into two syntax elements, one for each direction.
  • At least one syntax data element indicating a ratio of the chroma residuals sampling is signaled from the encoder to the decoder.
  • the resampling ratio can be different to 2. For instance, resampling ratios of 4, 8 can also be considered, for horizontal and/or vertical dimensions.
  • the resampling ratio can be signaled by an additional syntax element cu_chroma_sampling_ratio for instance specified as follows:
  • At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a bitstream generated or encoded.
  • At least one of the aspects can be implemented as a method, an apparatus, a computer readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the methods described, and/or a computer readable storage medium having stored thereon a bitstream generated according to any of the methods described.
  • the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, the terms “image,” “picture” and “frame” may be used interchangeably.
  • each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and/or use of specific steps and/or actions may be modified or combined. Additionally, terms such as “first”, “second”, etc. may be used in various embodiments to modify an element, component, step, operation, etc., such as, for example, a “first decoding” and a “second decoding”. Use of such terms does not imply an ordering to the modified operations unless specifically required. So, in this example, the first decoding need not be performed before the second decoding, and may occur, for example, before, during, or in an overlapping time period with the second decoding.
  • Various methods and other aspects described in this application can be used to modify modules or provide additional modules to, for example, the transform modules, and/or inverse transform modules (525, 550, 650), of a video encoder 500 and decoder 600 as shown in Figure 5 and Figure 6.
  • the present aspects are not limited to VVC or HEVC, and can be applied, for example, to other standards and recommendations, whether pre-existing or future-developed, and extensions of any such standards and recommendations (including VVC and HEVC). Unless indicated otherwise, or technically precluded, the aspects described in this application can be used individually or in combination.
  • numeric values are used in the present application, for example, the sampling in the chroma formats, the sampling ratio.
  • the specific values are for example purposes and the aspects described are not limited to these specific values.
  • Figure 5 illustrates an encoder 500. Variations of this encoder 500 are contemplated, but the encoder 500 is described below for purposes of clarity without describing all expected variations.
  • the video sequence may go through pre-encoding processing (501 ), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to get a signal distribution more resilient to compression (for instance using a histogram equalization of one of the color components).
  • Metadata can be associated with the preprocessing, and attached to the bitstream.
  • a picture is encoded by the encoder elements as described below.
  • the picture to be encoded is partitioned (502) and processed in units of, for example, CUs.
  • Each unit is encoded using, for example, either an intra or inter mode.
  • intra prediction 560
  • inter mode motion estimation (575) and compensation (570) are performed.
  • the encoder decides (505) which one of the intra mode or inter mode to use for encoding the unit, and indicates the intra/inter decision by, for example, a prediction mode flag.
  • Prediction residuals are calculated, for example, by subtracting (510) the predicted block from the original image block.
  • the prediction residuals are then transformed (525) and quantized (530).
  • the quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (545) to output a bitstream.
  • the encoder can skip the transform and apply quantization directly to the non-transformed residual signal.
  • the encoder can bypass both transform and quantization, i.e., the residual is coded directly without the application of the transform or quantization processes.
  • the encoder decodes an encoded block to provide a reference for further predictions.
  • the quantized transform coefficients are de-quantized (540) and inverse transformed (550) to decode prediction residuals.
  • In-loop filters (565) are applied to the reconstructed picture to perform, for example, deblocking/SAO (Sample Adaptive Offset) filtering to reduce encoding artifacts.
  • the filtered image is stored at a reference picture buffer (580).
  • Figure 6 illustrates a block diagram of a video decoder 600.
  • a bitstream is decoded by the decoder elements as described below.
  • Video decoder 600 generally performs a decoding pass reciprocal to the encoding pass as described in Figure 5.
  • the encoder 500 also generally performs video decoding as part of encoding video data.
  • the input of the decoder includes a video bitstream, which can be generated by video encoder 500.
  • the bitstream is first entropy decoded (630) to obtain transform coefficients, motion vectors, and other coded information.
  • the picture partition information indicates how the picture is partitioned.
  • the decoder may therefore divide (635) the picture according to the decoded picture partitioning information.
  • the transform coefficients are dequantized (640) and inverse transformed (650) to decode the prediction residuals. Combining (655) the decoded prediction residuals and the predicted block, an image block is reconstructed.
  • the predicted block can be obtained (670) from intra prediction (660) or motion-compensated prediction (i.e., inter prediction) (675).
  • In-loop filters (665) are applied to the reconstructed image.
  • the filtered image is stored at a reference picture buffer (680).
  • the decoded picture can further go through post-decoding processing (685), for example, an inverse color transform (e.g. conversion from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping performing the inverse of the remapping process performed in the pre-encoding processing (501 ).
  • post-decoding processing can use metadata derived in the preencoding processing and signaled in the bitstream.
  • FIG. 7 illustrates a block diagram of an example of a system in which various aspects and embodiments are implemented.
  • System 700 can be embodied as a device including the various components described below and is configured to perform one or more of the aspects described in this document. Examples of such devices, include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers.
  • Elements of system 700, singly or in combination can be embodied in a single integrated circuit (IC), multiple ICs, and/or discrete components.
  • the processing and encoder/decoder elements of system 700 are distributed across multiple ICs and/or discrete components.
  • system 700 is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports.
  • system 700 is configured to implement one or more of the aspects described in this document.
  • the system 700 includes at least one processor 710 configured to execute instructions loaded therein for implementing, for example, the various aspects described in this document.
  • Processor 710 can include embedded memory, input output interface, and various other circuitries as known in the art.
  • the system 700 includes at least one memory 720 (e.g., a volatile memory device, and/or a non-volatile memory device).
  • System 700 includes a storage device 740, which can include non-volatile memory and/or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and/or optical disk drive.
  • the storage device 740 can include an internal storage device, an attached storage device (including detachable and non-detachable storage devices), and/or a network accessible storage device, as non-limiting examples.
  • System 700 includes an encoder/decoder module 730 configured, for example, to process data to provide an encoded video or decoded video, and the encoder/decoder module 730 can include its own processor and memory.
  • the encoder/decoder module 730 represents module(s) that can be included in a device to perform the encoding and/or decoding functions. As is known, a device can include one or both of the encoding and decoding modules. Additionally, encoder/decoder module 730 can be implemented as a separate element of system 700 or can be incorporated within processor 710 as a combination of hardware and software as known to those skilled in the art.
  • processor 710 Program code to be loaded onto processor 710 or encoder/decoder 730 to perform the various aspects described in this document can be stored in storage device 740 and subsequently loaded onto memory 720 for execution by processor 710.
  • processor 710, memory 720, storage device 740, and encoder/decoder module 730 can store one or more of various items during the performance of the processes described in this document.
  • Such stored items can include, but are not limited to, the input video, the decoded video or portions of the decoded video, the bitstream, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.
  • memory inside of the processor 710 and/or the encoder/decoder module 730 is used to store instructions and to provide working memory for processing that is needed during encoding or decoding.
  • a memory external to the processing device (for example, the processing device can be either the processor 710 or the encoder/decoder module 730) is used for one or more of these functions.
  • the external memory can be the memory 720 and/or the storage device 740, for example, a dynamic volatile memory and/or a non-volatile flash memory.
  • an external non-volatile flash memory is used to store the operating system of, for example, a television.
  • a fast external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations, such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group, MPEG-2 is also referred to as ISO/IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by JVET, the Joint Video Experts Team).
  • MPEG-2 MPEG refers to the Moving Picture Experts Group
  • MPEG-2 is also referred to as ISO/IEC 13818
  • 13818-1 is also known as H.222
  • 13818-2 is also known as H.262
  • HEVC High Efficiency Video Coding
  • VVC Very Video Coding
  • the input to the elements of system 700 can be provided through various input devices as indicated in block 705.
  • Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a Component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and/or (iv) a High Definition Multimedia Interface (HDMI) input terminal.
  • RF radio frequency
  • COMP Component
  • USB Universal Serial Bus
  • HDMI High Definition Multimedia Interface
  • the input devices of block 705 have associated respective input processing elements as known in the art.
  • the RF portion can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets.
  • the RF portion of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers.
  • the RF portion can include a tuner that performs various of these functions, including, for example, downconverting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband.
  • the RF portion and its associated input processing element receives an RF signal transmitted over a wired (for example, cable) medium, and performs frequency selection by filtering, downconverting, and filtering again to a desired frequency band.
  • Adding elements can include inserting elements in between existing elements, such as, for example, inserting amplifiers and an analog-to-digital converter.
  • the RF portion includes an antenna.
  • USB and/or HDMI terminals can include respective interface processors for connecting system 700 to other electronic devices across USB and/or HDMI connections.
  • various aspects of input processing for example, Reed-Solomon error correction
  • aspects of USB or HDMI interface processing can be implemented within separate interface ICs or within processor 710 as necessary.
  • the demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 710, and encoder/decoder 730 operating in combination with the memory and storage elements to process the data stream as necessary for presentation on an output device.
  • connection arrangement 715 for example, an internal bus as known in the art, including the Inter- IC (I2C) bus, wiring, and printed circuit boards.
  • I2C Inter- IC
  • the system 700 includes communication interface 750 that enables communication with other devices via communication channel 790.
  • the communication interface 750 can include, but is not limited to, a transceiver configured to transmit and to receive data over communication channel 790.
  • the communication interface 750 can include, but is not limited to, a modem or network card and the communication channel 790 can be implemented, for example, within a wired and/or a wireless medium.
  • Wi-Fi Wireless Fidelity
  • IEEE 802.11 IEEE refers to the Institute of Electrical and Electronics Engineers
  • the Wi-Fi signal of these embodiments is received over the communications channel 790 and the communications interface 750 which are adapted for Wi-Fi communications.
  • the communications channel 790 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over- the-top communications.
  • Other embodiments provide streamed data to the system 700 using a set-top box that delivers the data over the HDMI connection of the input block 705.
  • Still other embodiments provide streamed data to the system 700 using the RF connection of the input block 705.
  • various embodiments provide data in a non-streaming manner.
  • various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network.
  • the system 700 can provide an output signal to various output devices, including a display 765, speakers 775, and other peripheral devices 785.
  • the display 765 of various embodiments includes one or more of, for example, a touchscreen display, an organic lightemitting diode (OLED) display, a curved display, and/or a foldable display.
  • the display 765 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or other device.
  • the display 765 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop).
  • the other peripheral devices 785 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and/or a lighting system.
  • Various embodiments use one or more peripheral devices 785 that provide a function based on the output of the system 700. For example, a disk player performs the function of playing the output of the system 700.
  • control signals are communicated between the system 700 and the display 765, speakers 775, or other peripheral devices 785 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communications protocols that enable device- to-device control with or without user intervention.
  • the output devices can be communicatively coupled to system 700 via dedicated connections through respective interfaces 765, 775, and 785. Alternatively, the output devices can be connected to system 700 using the communications channel 790 via the communications interface 750.
  • the display 765 and speakers 775 can be integrated in a single unit with the other components of system 700 in an electronic device such as, for example, a television.
  • the display interface 765 includes a display driver, such as, for example, a timing controller (T Con) chip.
  • T Con timing controller
  • the display 765 and speaker 775 can alternatively be separate from one or more of the other components, for example, if the RF portion of input 705 is part of a separate set-top box.
  • the output signal can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.
  • the embodiments can be carried out by computer software implemented by the processor 710 or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits.
  • the memory 720 can be of any type appropriate to the technical environment and can be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples.
  • the processor 710 can be of any type appropriate to the technical environment, and can encompass one or more of microprocessors, general purpose computers, special purpose computers, digital signal processors (DSPs), and processors based on a multi-core architecture, as non-limiting examples.
  • Decoding can encompass all or part of the processes performed, for example, on a received encoded sequence in order to produce a final output suitable for display.
  • processes include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding.
  • processes also, or alternatively, include processes performed by a decoder of various implementations described in this application, for example, comprising upsampling the at least one block of inverse transformed chroma residuals.
  • decoding refers only to entropy decoding
  • decoding refers only to differential decoding
  • decoding refers to a combination of entropy decoding and differential decoding.
  • encoding can encompass all or part of the processes performed, for example, on an input video sequence in order to produce an encoded bitstream.
  • processes include one or more of the processes typically performed by an encoder, for example, partitioning, differential encoding, transformation, quantization, and entropy encoding.
  • processes also, or alternatively, include processes performed by an encoder of various implementations described in this application, for example, subsampling at least one block of chroma residuals before inverse transform and quantization.
  • encoding refers only to entropy encoding
  • encoding refers only to differential encoding
  • encoding refers to a combination of differential encoding and entropy encoding.
  • cu_chroma_sampling_flag cu_act_enabled_flag
  • cu_chroma_sampling_direction cu_chroma_sampling_ratio
  • This disclosure has described various pieces of information, such as for example syntax, that can be transmitted or stored, for example.
  • This information can be packaged or arranged in a variety of manners, including for example manners common in video standards such as putting the information into an SPS, a PPS, a NAL unit, a header (for example, a NAL unit header, or a slice header), or an SEI message.
  • Other manners are also available, including for example manners common for system level or application level standards such as putting the information into one or more of the following: • SDP (session description protocol), a format for describing multimedia communication sessions for the purposes of session announcement and session invitation, for example as described in RFCs and used in conjunction with RTP (Real-time Transport Protocol) transmission;
  • SDP session description protocol
  • RTP Real-time Transport Protocol
  • DASH MPD Media Presentation Description
  • Descriptors for example as used in DASH and transmitted over HTTP, a Descriptor is associated to a Representation or collection of Representations to provide additional characteristic to the content Representation;
  • RTP header extensions for example as used during RTP streaming
  • HLS HTTP live Streaming
  • a manifest can be associated, for example, to a version or collection of versions of a content to provide characteristics of the version or collection of versions.
  • Various embodiments refer to rate distortion optimization.
  • the rate distortion optimization is usually formulated as minimizing a rate distortion function, which is a weighted sum of the rate and of the distortion.
  • the approaches may be based on an extensive testing of all encoding options, including all considered modes or coding parameters values, with a complete evaluation of their coding cost and related distortion of the reconstructed signal after coding and decoding.
  • Faster approaches may also be used, to save encoding complexity, in particular with computation of an approximated distortion based on the prediction or the prediction residual signal, not the reconstructed one.
  • the implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program).
  • An apparatus can be implemented in, for example, appropriate hardware, software, and firmware.
  • the methods can be implemented in, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs”), and other devices that facilitate communication of information between end-users.
  • PDAs portable/personal digital assistants
  • references to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment.
  • the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment.
  • Determining the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory.
  • Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.
  • this application may refer to “receiving” various pieces of information.
  • Receiving is, as with “accessing”, intended to be a broad term.
  • Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory).
  • “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.
  • any of the following 7”, “and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B).
  • such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C).
  • This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.
  • the word “signal” refers to, among other things, indicating something to a corresponding decoder.
  • the encoder signals a particular one of a plurality of parameters for chroma sampling.
  • the same parameter is used at both the encoder side and the decoder side.
  • an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter.
  • signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter.
  • signaling can be accomplished in a variety of ways. For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun.
  • implementations can produce a variety of signals formatted to carry information that can be, for example, stored or transmitted.
  • the information can include, for example, instructions for performing a method, or data produced by one of the described implementations.
  • a signal can be formatted to carry the bitstream of a described embodiment.
  • Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal.
  • the formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream.
  • the information that the signal carries can be, for example, analog or digital information.
  • the signal can be transmitted over a variety of different wired or wireless links, as is known.
  • the signal can be stored on a processor- readable medium.
  • embodiments can be provided alone or in any combination, across various claim categories and types. Further, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types: • Sampling chroma residual in the decoder and/or encoder (upsampling and subsampling).
  • a TV, set-top box, cell phone, tablet, or other electronic device that performs a chroma residual sampling process according to any of the embodiments described.
  • a TV, set-top box, cell phone, tablet, or other electronic device that performs a chroma residual sampling process according to any of the embodiments described, and that displays (e.g. using a monitor, screen, or other type of display) a resulting image.
  • a TV, set-top box, cell phone, tablet, or other electronic device that selects (e.g. using a tuner) a channel to receive a signal including an encoded image, and performs a chroma residual sampling process according to any of the embodiments described.
  • a TV, set-top box, cell phone, tablet, or other electronic device that receives (e.g. using an antenna) a signal over the air that includes an encoded image, and performs a chroma residual sampling process according to any of the embodiments described.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne au moins un procédé et un appareil pour coder ou décoder efficacement une vidéo. Par exemple, le procédé comprend le sous-échantillonnage d'au moins un bloc de résidus de chrominance au niveau du codage ou du suréchantillonnage d'au moins un bloc de résidus de chrominance transformés inverses au niveau du décodage. En permettant au codeur de sous-échantillonner horizontalement et/ou verticalement les résidus de chrominance d'un bloc de codage, le procédé réduit le débit binaire. Les résidus de chrominance sous-échantillonnés des blocs de codage sont suréchantillonnés, par exemple à l'aide de filtres prédéfinis, côté décodeur et également côté codeur s'ils sont utilisés en tant que blocs de référence pour une inter-prédiction.
PCT/EP2022/073995 2021-09-14 2022-08-30 Procédé et appareil de codage et de décodage vidéo avec échantillonnage de résidus de chrominance WO2023041317A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP21306262.3 2021-09-14
EP21306262 2021-09-14

Publications (1)

Publication Number Publication Date
WO2023041317A1 true WO2023041317A1 (fr) 2023-03-23

Family

ID=77998927

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/073995 WO2023041317A1 (fr) 2021-09-14 2022-08-30 Procédé et appareil de codage et de décodage vidéo avec échantillonnage de résidus de chrominance

Country Status (1)

Country Link
WO (1) WO2023041317A1 (fr)

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KIM: "Adaptive Residue Transform and Sampling", 11. JVT MEETING; 68. MPEG MEETING; 15-03-2004 - 19-03-2004; MUNICH,DE; (JOINT VIDEO TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16), no. JVT-K018r1, 19 March 2004 (2004-03-19), XP030005835 *
SHUHUI WANG ET AL: "4:4:4 screen content coding using Macroblock-Adaptive Mixed Chroma-Sampling-Rate", no. JCTVC-H0073, 6 February 2012 (2012-02-06), XP030232655, Retrieved from the Internet <URL:http://phenix.int-evry.fr/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H0073-v2.zip JCTVC-H0073-r1.doc> [retrieved on 20120206] *

Similar Documents

Publication Publication Date Title
WO2022167322A1 (fr) Compensation spatiale d&#39;éclairage local
US20240275960A1 (en) High-level syntax for picture resampling
EP4360313A1 (fr) Procédés et appareils pour coder/décoder une vidéo
US20230141577A1 (en) Method and apparatus for video encoding and decoding
EP4118823A1 (fr) Procédé et appareil de codage et de décodage vidéo
CN115516858A (zh) 视频编码中的缩放列表控制
WO2023041317A1 (fr) Procédé et appareil de codage et de décodage vidéo avec échantillonnage de résidus de chrominance
US20230262268A1 (en) Chroma format dependent quantization matrices for video encoding and decoding
US20240323367A1 (en) Methods and apparatuses for encoding/decoding a video
US20240195991A1 (en) Adapting luma mapping with chroma scaling to 4:4:4 rgb image content
US20240298011A1 (en) Method and apparatus for video encoding and decoding
US20220368912A1 (en) Derivation of quantization matrices for joint cb-br coding
US20220360781A1 (en) Video encoding and decoding using block area based quantization matrices
US20220224902A1 (en) Quantization matrices selection for separate color plane mode
WO2023099249A1 (fr) Indication de phase de sous-échantillonnage
EP4406224A1 (fr) Procédés et appareils pour coder/décoder une vidéo
KR20230150293A (ko) 비디오를 인코딩/디코딩하기 위한 방법들 및 장치들
WO2023222521A1 (fr) Sei conçues pour de multiples points de conformité
CN117280683A (zh) 用于对视频进行编码/解码的方法和装置
WO2024083500A1 (fr) Procédés et appareils de remplissage d&#39;échantillons de référence
CN117813817A (zh) 用于对视频进行编码/解码的方法和装置
WO2023052141A1 (fr) Procédés et appareils de codage/décodage d&#39;une vidéo
WO2021058408A1 (fr) Signalisation de mode le plus probable avec prédiction intra de ligne de référence multiple

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22769687

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22769687

Country of ref document: EP

Kind code of ref document: A1