WO2025040751A1 - In-loop mapping function update - Google Patents

Abstract

A method and an apparatus for encoding or decoding a video are provided wherein in-loop luma mapping is used for encoding or decoding one or more blocks of the video. In some embodiments, for at least one block of the video, an update of at least one in-loop luma mapping function is determined based neighboring reconstructed samples of the at least one block, and the at least one block is encoded or decoded using the at least one updated mapping function.

Description

IN-LOOP MAPPING FUNCTION UPDATE

This application claims the priority to European Application No. 23306414.6, filed on 24 August 2023, which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present embodiments generally relate to video compression. The present embodiments relate to a method and an apparatus for encoding or decoding an image or a video. More particularly, the present embodiments relate to update inside the coding loop the luma mapping function.

BACKGROUND

To achieve high compression efficiency, image and video coding schemes usually employ prediction and transform to leverage spatial and temporal redundancy in the video content. Generally, intra or inter prediction is used to exploit the intra or inter picture correlation, then the differences between the original block and the predicted block, often denoted as prediction errors or prediction residuals, are transformed, quantized, and entropy coded. In inter prediction, motion vectors used in motion compensation are often predicted from motion vector predictor. To reconstruct the video, the compressed data are decoded by inverse processes corresponding to the entropy coding, quantization, transform, and prediction.

SUMMARY

According to an aspect, a method for encoding or decoding a video is provided. The method comprises determining for at least one block of the video, at least one mapping function using template samples of the at least one block and encoding or decoding the at least one block using the at least one mapping function.

According to another aspect, an apparatus for encoding or decoding a video is provided. The apparatus comprises one or more processors operable to determine for at least one block of the video, at least one mapping function using template samples of the at least one block and encode or decode the at least one block using the at least one mapping function.

In some embodiments, the mapping function is a forward mapping function, and wherein encoding or decoding the at least one block using the mapping function comprises applying the mapping function to a prediction block of the at least one block. In other embodiments, the mapping function is an inverse mapping function and wherein encoding or decoding the at least one block using the mapping function comprises applying the mapping function to a reconstruction of the at least one block. In some embodiments, determining the mapping function corresponds to updating a default mapping function, for instance a mapping function defined by the LMCS tool, for instance a mapping function computed at the encoder side, signaled by the encoder into the bitstream, further decoded and reconstructed by the decoder.

In other embodiments, the mapping function is determined from an update of an identity mapping function.

Further embodiments that can be used alone or in combination are described herein.

One or more embodiments also provide a computer program comprising instructions which when executed by one or more processors cause the one or more processors to perform the method for encoding or decoding a video according to any of the embodiments described herein. One or more of the present embodiments also provide a non-transitory computer readable medium and/or a computer readable storage medium having stored thereon instructions for encoding or decoding a video according to the methods described herein.

One or more embodiments also provide a computer readable storage medium having stored thereon a bitstream generated according to the methods described herein. One or more embodiments also provide a method and apparatus for transmitting or receiving the bitstream generated according to the methods described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a system within which aspects of the present embodiments may be implemented.

FIG. 2 illustrates a block diagram of an embodiment of a video encoder within which aspects of the present embodiments may be implemented.

FIG. 3 illustrates a block diagram of an embodiment of a video decoder within which aspects of the present embodiments may be implemented.

FIG. 4 illustrates a block diagram of another embodiment of a video decoder within which aspects of the present embodiments may be implemented.

FIG. 5 illustrates a block diagram of another embodiment of a video decoder within which aspects of the present embodiments may be implemented.

FIG. 6 illustrates a block diagram of another embodiment of a video decoder within which aspects of the present embodiments may be implemented.

FIG. 7 illustrates a block diagram of another embodiment of a video decoder within which aspects of the present embodiments may be implemented.

FIG. 8 illustrates an example of a mapping update of the forward mapping function according to an embodiment.

FIG. 9 illustrates an example of a mapping update of the inverse mapping function according to an embodiment.

FIG. 10 illustrates examples of reference areas in reference pictures for updating a mapping function according to an embodiment.

FIG. 1 1 illustrates an example of a flowchart of a method for encoding at least one block of a video according to an embodiment.

FIG. 12 illustrates an example of a flowchart of a method for decoding at least one block of a video according to an embodiment.

FIG. 13 illustrates an example of a flowchart of a method for encoding at least one block of a video according to another embodiment.

FIG. 14 illustrates an example of a flowchart of a method for decoding at least one block of a video according to another embodiment.

FIG. 15 illustrates a block diagram of a system within which aspects of the present embodiments may be implemented, according to another embodiment.

FIG. 16 shows two remote devices communicating over a communication network in accordance with an example of the present principles.

FIG. 17 shows the syntax of a signal in accordance with an example of the present principles.

DETAILED DESCRIPTION

This application describes a variety of aspects, including tools, features, embodiments, models, approaches, etc. Many of these aspects are described with specificity and, at least to show the individual characteristics, are often described in a manner that may sound limiting. However, this is for purposes of clarity in description, and does not limit the application or scope of those aspects. Indeed, all of the different aspects can be combined and interchanged to provide further aspects. Moreover, the aspects can be combined and interchanged with aspects described in earlier filings as well.

The aspects described and contemplated in this application can be implemented in many different forms. FIGs. 1 , 2 and 3 below provide some embodiments, but other embodiments are contemplated and the discussion of FIGs. 1 , 2 and 3 does not limit the breadth of the implementations. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a bitstream generated or encoded. These and other aspects can be implemented as a method, an apparatus, a computer readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the methods described, and/or a computer readable storage medium having stored thereon a bitstream generated according to any of the methods described. In the present application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, the terms “image,” “picture” and “frame” may be used interchangeably.

Various methods are described herein, and each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and/or use of specific steps and/or actions may be modified or combined. Additionally, terms such as “first”, “second”, etc. may be used in various embodiments to modify an element, component, step, operation, etc., such as, for example, a “first decoding” and a “second decoding”. Use of such terms does not imply an ordering to the modified operations unless specifically required. So, in this example, the first decoding need not be performed before the second decoding, and may occur, for example, before, during, or in an overlapping time period with the second decoding.

The present aspects are not limited to VVC or HEVC, and can be applied, for example, to other standards and recommendations, whether pre-existing or future-developed, and extensions of any such standards and recommendations (including VVC and HEVC). Unless indicated otherwise, or technically precluded, the aspects described in this application can be used individually or in combination.

FIG. 1 illustrates a block diagram of an example of a system in which various aspects and embodiments can be implemented. System 100 may be embodied as a device including the various components described below and is configured to perform one or more of the aspects described in this application. Examples of such devices, include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 100, singly or in combination, may be embodied in a single integrated circuit, multiple ICs, and/or discrete components. For example, in at least one embodiment, the processing and encoder/decoder elements of system 100 are distributed across multiple ICs and/or discrete components. In various embodiments, the system 100 is communicatively coupled to other systems, or to other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports. In various embodiments, the system 100 is configured to implement one or more of the aspects described in this application.

The system 100 includes at least one processor 110 configured to execute instructions loaded therein for implementing, for example, the various aspects described in this application. Processor 110 may include embedded memory, input output interface, and various other circuitries as known in the art. The system 100 includes at least one memory 120 (e.g., a volatile memory device, and/or a non-volatile memory device). System 100 includes a storage device 140, which may include non-volatile memory and/or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drive, and/or optical disk drive. The storage device 140 may include an internal storage device, an attached storage device, and/or a network accessible storage device, as non-limiting examples.

System 100 includes an encoder/decoder module 130 configured, for example, to process data to provide an encoded video or decoded video, and the encoder/decoder module 130 may include its own processor and memory. The encoder/decoder module 130 represents module(s) that may be included in a device to perform the encoding and/or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Additionally, encoder/decoder module 130 may be implemented as a separate element of system 100 or may be incorporated within processor 1 10 as a combination of hardware and software as known to those skilled in the art.

Program code to be loaded onto processor 1 10 or encoder/decoder 130 to perform the various aspects described in this application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 1 10. In accordance with various embodiments, one or more of processor 1 10, memory 120, storage device 140, and encoder/decoder module 130 may store one or more of various items during the performance of the processes described in this application. Such stored items may include, but are not limited to, the input video, the decoded video or portions of the decoded video, the bitstream, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

In some embodiments, memory inside of the processor 110 and/or the encoder/decoder module 130 is used to store instructions and to provide working memory for processing that is needed during encoding or decoding. In other embodiments, however, a memory external to the processing device (for example, the processing device may be either the processor 1 10 or the encoder/decoder module 130) is used for one or more of these functions. The external memory may be the memory 120 and/or the storage device 140, for example, a dynamic volatile memory and/or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations, such as for MPEG-2, HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding also known as H.266, standard developed by JVET, the Joint Video Experts Team).

The input to the elements of system 100 may be provided through various input devices as indicated in block 105. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a Component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and/or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples, not shown in FIG. 1 , include composite video.

In various embodiments, the input devices of block 105 have associated respective input processing elements as known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) down converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the down converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF portion of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include a tuner that performs various of these functions, including, for example, down converting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. In one set-top box embodiment, the RF portion and its associated input processing element receives an RF signal transmitted over a wired (for example, cable) medium, and performs frequency selection by filtering, down converting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and/or add other elements performing similar or different functions. Adding elements may include inserting elements in between existing elements, for example, inserting amplifiers and an analog-to-digital converter. In various embodiments, the RF portion includes an antenna.

Additionally, the USB and/or HDMI terminals may include respective interface processors for connecting system 100 to other electronic devices across USB and/or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within processor 110 as necessary. Similarly, aspects of USB or HDMI interface processing may be implemented within separate interface ICs or within processor 1 10 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110, and encoder/decoder 130 operating in combination with the memory and storage elements to process the data stream as necessary for presentation on an output device. Various elements of system 100 may be provided within an integrated housing, Within the integrated housing, the various elements may be interconnected and transmit data therebetween using suitable connection arrangement 115, for example, an internal bus as known in the art, including the I2C bus, wiring, and printed circuit boards.

The system 100 includes communication interface 150 that enables communication with other devices via communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and to receive data over communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network card and the communication channel 190 may be implemented, for example, within a wired and/or a wireless medium.

Data is streamed to the system 100, in various embodiments, using a Wi-Fi network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal of these embodiments is received over the communications channel 190 and the communications interface 150 which are adapted for Wi-Fi communications. The communications channel 190 of these embodiments is typically connected to an access point or router that provides access to outside networks including the Internet for allowing streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 100 using a set-top box that delivers the data over the HDMI connection of the input block 105. Still other embodiments provide streamed data to the system 100 using the RF connection of the input block 105. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network.

The system 100 may provide an output signal to various output devices, including a display 165, speakers 175, and other peripheral devices 185. The display 165 of various embodiments includes one or more of, for example, a touchscreen display, an organic lightemitting diode (OLED) display, a curved display, and/or a foldable display. The display 165 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or other device. The display 165 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop). The other peripheral devices 185 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and/or a lighting system. Various embodiments use one or more peripheral devices 185 that provide a function based on the output of the system 100. For example, a disk player performs the function of playing the output of the system 100.

In various embodiments, control signals are communicated between the system 100 and the display 165, speakers 175, or other peripheral devices 185 using signaling such as AV.Link, CEC, or other communications protocols that enable device-to-device control with or without user intervention. The output devices may be communicatively coupled to system 100 via dedicated connections through respective interfaces 160, 170, and 180. Alternatively, the output devices may be connected to system 100 using the communications channel 190 via the communications interface 150. The display 165 and speakers 175 may be integrated in a single unit with the other components of system 100 in an electronic device, for example, a television. In various embodiments, the display interface 160 includes a display driver, for example, a timing controller (T Con) chip.

The display 165 and speaker 175 may alternatively be separate from one or more of the other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments in which the display 165 and speakers 175 are external components, the output signal may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

The embodiments can be carried out by computer software implemented by the processor 1 10 or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. The memory 120 can be of any type appropriate to the technical environment and can be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 1 10 can be of any type appropriate to the technical environment, and can encompass one or more of microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting examples.

FIG. 2 illustrates an example of a block-based hybrid video encoder 200. Variations of this encoder 200 are contemplated, but the encoder 200 is described below for purposes of clarity without describing all expected variations.

In some embodiments, FIG. 2 also illustrate an encoder in which improvements are made to the HEVC standard or a VVC standard Versatile Video Coding, Standard ITU-T H.266, ISO/IEC 23090-3, 2020) or an encoder employing technologies similar to HEVC or VVC, such as an encoder ECM under development by JVET (Joint Video Exploration Team).

FIG. 2 illustrates an example of a block-based hybrid video encoder 200. Before being encoded, the video sequence may go through pre-encoding processing (201 ), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to get a signal distribution more resilient to compression (for instance using a histogram equalization of one of the color components). Metadata can be associated with the pre-processing, and attached to the bitstream.

In the encoder 200, a picture is encoded by the encoder elements as described below. The picture is first analyzed to determine when a luma mapping is applied (LMCS) the function to be applied (295). Step 295 also possibly applies the forward luma mapping to the slice or picture. The picture to be encoded is partitioned (202) and processed in units of, for example, CUs (Coding Units). Each unit is encoded using, for example, either an intra or inter mode. When a unit is encoded in an intra mode, it performs intra prediction (260). In an inter mode, motion estimation (275) and compensation (270) are performed. When LMCS is applied a forward luma mapping of the motion compensated prediction is applied (285). The encoder decides (205) which one of the intra mode or inter mode to use for encoding the unit, and indicates the intra/inter decision by, for example, a prediction mode flag. Prediction residuals are calculated, for example, by subtracting (210) the predicted block from the original image block.

The prediction residuals are then transformed (225) and quantized (230). The quantized transform coefficients, as well as motion vectors and other syntax elements such as the picture partitioning information, are entropy coded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the non-transformed residual signal. The encoder can bypass both transform and quantization, i.e., the residual is coded directly without the application of the transform or quantization processes.

The encoder decodes an encoded block to provide a reference for further predictions. The quantized transform coefficients are de-quantized (240) and inverse transformed (250) to decode prediction residuals. Combining (255) the decoded prediction residuals and the predicted block, an image block is reconstructed. When LMCS is applied an inverse luma mapping is performed on the reconstructed picture (290). In-loop filters (265) are then applied to the reconstructed picture to perform, for example, deblocking/SAO (Sample Adaptive Offset)/ALF (Adaptive Loop Filter) filtering to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (280).

FIG. 3 illustrates a block diagram of an example video decoder 300. In the decoder 300, a bitstream is decoded by the decoder elements as described below. Video decoder 300 generally performs a decoding pass reciprocal to the encoding pass as described in FIG. 2. The encoder 200 also generally performs video decoding as part of encoding video data.

In particular, the input of the decoder includes a video bitstream, which can be generated by video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, prediction modes, motion vectors, and other coded information. The picture partition information indicates how the picture is partitioned. The decoder may therefore divide (335) the picture according to the decoded picture partitioning information. The transform coefficients are de-quantized (340) and inverse transformed (350) to decode the prediction residuals. Combining (355) the decoded prediction residuals and the predicted block, an image block is reconstructed. When LMCS is applied an inverse luma mapping is performed on the reconstructed picture (390). The predicted block can be obtained (370) from intra prediction (360) or motion-compensated prediction (i.e., inter prediction) (375). When LMCS is applied a forward luma mapping of the motion compensated prediction is applied (325). In-loop filters (365) are applied to the reconstructed image. The filtered image is stored at a reference picture buffer (380). Note that, for a given picture, the contents of the reference picture buffer 380 on the decoder 300 side is identical to the contents of the reference picture buffer 280 on the encoder 200 side for the same picture.

The decoded picture can further go through post-decoding processing (385), for example, an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or an inverse luma mapping performing the inverse of the forward luma mapping process performed in the pre-encoding processing (201 ). The post-decoding processing can use metadata derived in the pre-encoding processing and signaled in the bitstream.

Some of the embodiments described herein relates to in-loop luma mapping, and more particularly on updating inside the coding loop the luma mapping function, both at encoder and decoder side, based on samples already reconstructed in the current slice or picture or in other slice or picture.

An aim is to improve the compression efficiency, that is, to reduce the bitrate while maintaining the quality, or equivalently to improve the quality while maintaining the bitrate.

Any one of the embodiments described herein can be implemented for instance in the forward luma mapping module (285, 325) or inverse luma mapping (290, 390) of a video encoder 200 or video decoder 300 or in additional modules placed before these modules in the video encoder or decoder.

Luma Mapping with Chroma Scaling (LMCS)

In most of the SDR sequences especially for captured content ones, data range of the luma signal is not corresponding to 1024 levels as expected for a 10-bit signal but it is corresponding to the limited range of 64-940 (also named standard range). For HDR signal, when they are in PQ format, the actual code range is even more reduced, because the maximum value of 1023 corresponds to a peak luminance of 10000 cd/m2, which is in practice never used. Instead, grading uses more frequently peak luminance up to 1000, 4000 or 5000 cd/m2, which leads to a codeword around 720, 850 and 875, respectively. In VTM and ECM, the LMCS function allows to modify the luma signal of the content to better exploit the 1 O-bit capabilities. A mapping function is applied to luma signal thanks to a pre-calculated function or LUT, computed at the encoder side. Values of the function or LUT are defined by the encoder based on specific characteristic of the content. For this purpose, the 1 O-bit representation (1024 levels) is split into a given number of N pieces (e.g. 16 intervals) of 1024/N levels (e.g. 64 when 16 intervals are used). Therefore, the luma mapping function is defined as a piece-wise linear function, defined in N segments.

Then, for each input value x (in [0,1023] for 10-bit signal), a Forward_Mapping LUT may be built from the piece-wise linear function. In parallel an lnverse_Mapping LUT is also generated based on the same piece-wise linear function. It will be used each time when a conversion from the mapped to the non-mapped domain is required. The Forward_Mapping and lnverse_Mapping may be applied to any picture of the video.

The LMCS parameters are signaled by the encoder, in general once per intra period, in a high-level structure named APS. The table below illustrates an example of a corresponding syntax as implemented in the VVC/ITU-T H.266 standard.

Table 1. LMCS syntax table.

At decoder side, the decoder decodes this information, rebuild the forward and inverse mapping functions, and use these functions during the decoding process (forward mapping is applied to inter prediction samples, and inverse mapping is applied after the reconstruction process and before the in-loop filtering). When using the current implementations of the forward and inverse mapping functions tool, such as in VVC, the functions are pre-computed and do not change or adapt inside the slice or picture. Therefore, the mapping functions do not take into account the distortion brought by the signal quantization. They also do not take into account the local statistical changes of the signal. These drawbacks may impact the coding efficiency.

Some embodiments provide a method and an apparatus wherein a mapping update stage is added before applying the mapping to a coding block being processed. The mapping update stage uses as input signal prediction and reconstructed samples already processed in the slice or picture (typically, samples located in a template area of the block) to determine an update of the mapping function. In an embodiment, the mapping update stage is applied prior to the forward mapping step (285, 325) applied to the inter-prediction block. In another embodiment, the mapping update stage is applied prior to the inverse mapping step (290, 390) applied to the reconstructed block. In other embodiment, the mapping update stage is applied before both the forward mapping step and the inverse mapping step.

FIG. 1 1 and FIG. 12 respectively illustrate an example of a flowchart of a method for encoding (1100) and a method for decoding (1200) at least one block of a video according to an embodiment.

In this embodiment, a default mapping function is updated on a block-basis.

On the encoding side, at 1 110, the default mapping function is obtained. The default mapping function is for example a luma mapping function determined for the LMCS tool and signaled in the bitstream. The current picture to encode references one of the parameter sets signaled in the bitstream to identify the default mapping function.

At 1120, for a current block of the picture to encode, an update of the default mapping function is determined based on template samples of the current block, and at 1130 the current block is encoded using the update of the default mapping function.

On the decoding side, at 1210, the default mapping function is obtained, for instance by decoding the mapping function parameters encoded in the bitstream that are referred by the current picture to decode. At 1220, for a current block, an update of the default mapping function is determined based on template samples of the current block, and at 1230 the current block is decoded using the update of the default mapping function.

FIG. 13 and FIG. 14 respectively illustrate an example of a flowchart of a method for encoding (1300) and a method for decoding (1400) at least one block of a video according to another embodiment. In this embodiment, the mapping function of the LMCS is no more used as a starting point for determining an update of the mapping function on a block-basis, but an identity mapping function is rather used. To do so, on the encoding side, at 1310, for a current block of the picture to encode, the mapping function is determined based on template samples of the current block, and at 1320 the current block is encoded using the determined mapping function. On the decoding side, at 1410, for a current block, the mapping function is determined in a same manner as on the encoder side based on template samples of the current block, and at 1420 the current block is decoded using the determined mapping function.

In some variants of the embodiments described herein, the mapping function is determined as a look-up-table that maps sample values in a range of values of the signal bitdepth (for example, for a 10-bit signal, the considered range may be [0,1023]). The mapping function is an in-loop mapping function and can be a forward mapping function, or an inverse mapping function or both the forward and inverse mapping function, as illustrated on FIG. 2-7.

More detailed variants of these embodiments are provided below.

In the following, let’s consider a current block of a video is being processed. For this current block, the following notations are used:

• pred block as prediction samples block obtained from an intra prediction (460, 560) or an inter-prediction including motion compensation steps (470, 575).

• rec block as reconstructed samples block obtained from residual and prediction addition step (455, 555).

• flit block as filtered samples block obtained from in-loop filtering step (465, 565).

In an embodiment, a stage of mapping update of the forward mapping function is applied prior to the forward mapping of the inter-prediction block. The mapping update takes as input prediction and reconstructed samples from areas of the current slice or picture that have already been processed. For example, the top and left neighboring prediction and reconstructed samples of the inter-prediction block are used.

An example of the process is described in FIG. 4 for the encoder. A new mapping update step 495 is added between the motion compensation step 470 and the forward mapping step 485. The mapping update modifies the forward mapping function based on neighboring prediction and reconstruction samples (“template pred samples" and “template rec samples" in FIG. 4) of the current block. In a variant, neighboring filtered samples (“template flit samples" in FIG. 4) coming from the in-loop filtering step 465 are also used as input of the mapping update step. The forward mapping (485) is then applied to the pred samples of the current block using the updated forward mapping function. Neighboring samples may be top and left samples surrounding the current block, but may also be more distant samples from areas already processed by the encoder / decoder, from the same picture, or from reference pictures. FIG. 5 illustrates the decoder side. A new mapping update step 595 is added between the motion compensation step 575 and the forward mapping step 525. The mapping update modifies the forward mapping function based on the neighboring prediction and reconstruction samples (“template pred samples" and “template rec sampled in FIG. 5) of the current block. In a variant, neighboring filtered samples (“template flit samples" in FIG. 5) coming from the in-loop filtering step 565 are also used as input of the mapping update step. The forward mapping (525) is then applied to the pred samples of the current block using the updated forward mapping function.

The process is also illustrated in FIG. 8. For the current block, the samples of the pred block (800) have already been generated from the motion compensation step. The reconstructed and filtered samples are not yet available for the current block, but they may be available in the causal area surrounding the current block. The mapping update step (495 at encoder, 595 at decoder) takes as input the prediction samples in causal neighborhood (801 ) of the prediction block (800) (gray area around the pred block, named pred template), and the reconstructed samples in causal neighborhood (802) of the current block (gray area around the current block, named rec template). In a variant, the template prediction samples (801 ) used for updating the mapping function are obtained by applying the intra or inter prediction (460, 470) that was used for encoding these samples. In this variant, the prediction of the template samples may have been buffered when these samples were encoded / decoded, to avoid having to recompute the prediction of the template samples a second time for updating the mapping function. In another variant, the template prediction samples (801 ) used for updating the mapping function are obtained by applying the same prediction mode (intra or inter-prediction) with the same parameters as the prediction for the current block.

The template reconstructed samples (802) are obtained at the output of the addition step (455, 555) of the residual and the prediction when reconstructing these samples, before the in-loop filtering is applied to the template reconstructed samples. Here, the prediction of the template samples is the one that was generated for encoding these samples.

In a variant, the filtered samples (803) in causal neighborhood of the current block (gray area around the current block, named flit template) can be used instead of or in complement of the rec template samples. The template filtered samples (803) are obtained as the samples output by the in-loop filters module (465, 565) applied to the template reconstructed samples after inverse mapping.

The output of the mapping update step is an Updated Forward mapping function UFm(.). This function is used in step 485 (encoder) or 525 (decoder) to map the pred block samples, noted Pred(x,y) (800) into the output mapped pred block samples (804), noted MapPred(x,y): MapPred(x,y) = UFm( Pred(x,y) ), for any position (x,y) inside the block.

In another embodiment, a stage of mapping update of the inverse mapping function is applied prior to the inverse mapping of the reconstructed block. The mapping update takes as input reconstructed and filtered samples from areas of the current slice or picture that have already been processed. For example, the top and left neighboring reconstructed and filtered samples of the block are used.

The process is described in FIG. 6 for the encoder. A new mapping update step 496 is added between the residual and prediction addition step 455 and the inverse mapping step 490. The mapping update modifies the inverse mapping function based on the neighboring reconstructed and filtered samples (“template rec samples" and “template flit samples" in FIG. 6) of the current block. The inverse mapping (490) is then applied to the reconstructed samples (rec samples) of the current block using the updated inverse mapping function.

FIG. 7 illustrates the decoder side. A new mapping update step 596 is added between the residual and prediction addition step 555 and the inverse mapping step 590. The mapping update modifies the inverse mapping function based on the neighboring reconstructed and filtered samples (“template rec samples" and “template flit samples" in FIG. 7) of the current block. The inverse mapping (596) is then applied to the reconstructed samples (rec samples) of the current block using the updated inverse mapping function.

The process is also illustrated in FIG. 9. For the current block, the samples of the rec block (900) have already been generated from the residual and prediction addition step. The filtered samples are not yet available for the current block, but they may be available in the causal area surrounding the current block. The mapping update step (496 at encoder, 596 at decoder) takes as input the reconstructed samples in causal neighborhood (902) of the reconstructed block (900) (gray area around the reconstructed block, named rec template), and the filtered samples in causal neighborhood (903) of the current block (gray area around the current block, named flit template). Neighboring samples may be top and left samples surrounding the current block, but may also be more distant samples from areas already processed by the encoder / decoder, from the same picture, or from reference pictures.

The template reconstructed samples (902) are obtained at the output of the addition step (455, 555) of the residual and the prediction when reconstructing these samples. Here, the prediction of the template samples is the one that was generated for encoding these samples. The template filtered samples (903) are obtained as the samples output by the inloop filters module (465, 565) applied to the template reconstructed samples after inverse mapping.

The output of the mapping update step is an Updated Inverse mapping function Ulm(.). This function is used in step 490 (encoder) or 596 (decoder) to inverse map the rec block samples, noted Rec(x,y) (900) into the output inverse mapped rec block samples (904), noted lnvMapRec(x,y): lnvMapRec(x,y) = Ulm( Rec(x,y) ), for any position (x,y) inside the block.

In some embodiments, the use of the updated mapping function instead of the default (non-updated) mapping function is controlled by an indicator. In a variant, the indicator is a flag signaled in the bitstream.

In some variants, the signaling can be done for example at high-level such as at sequence level (SPS signaling), picture level (PPS signaling), slice level (slice header signaling), APS level (APS signaling). In other variants, the signaling can be done for example at local level, such as CTU or CU level.

In a variant, the choice between the default mapping function and the updated mapping function for a given block (CTU or CU) may be inferred at both the encoder and decoder sides based on an analysis of the sanity of the updated mapping function. This advantageously avoids signaling of a control flag. Examples of sanity check of the updated mapping function are described further below.

In a variant, multi-mode updating is used, where multiple updated mapping functions may be derived, and an index is signaled, or inferred, at local level (e.g., CTU or CU) to indicate which mapping functions has to be used. For example, multiple updated mapping functions can be derived using different sets of template samples. For example, one first updated mapping function is derived using the top and left template samples, one second updated mapping function is derived using the top template samples, one third updated mapping function is derived using the left template samples. An index indicates which one of the e.g., default mapping function, first updated mapping function, second updated mapping function, or third updated mapping function is used. When top or left samples in the template are not available, the coding length of the index may be reduced.

In an embodiment, the updated mapping function replaces the default mapping function for the mapping process used for the next coding blocks. In other words, the updated mapping function becomes the default mapping function for the next coding blocks.

In this embodiment, the inverse mapping function may then need to be updated as well, to ensure that applying the forward mapping function fwdLut then the inverse mapping function invLut leads to the identity function (except for rare cases due to rounding errors): invLut[ fwdLut[ x ] ] = x, for any x in [0 , 2^BD-1 ] where BD is the bitdepth of the samples.

Nevertheless, in principle, the updating of the inverse mapping function should not be needed, even if the forward mapping function has been updated. Indeed, the updating of the forward mapping function is done to apply in the forward mapping step a slight correction of the input samples that aims at limiting the correction done by the following steps. For example, when the updating of the forward mapping function is done in step 495, its goal is to make the forward mapped prediction signal closer to the forward mapped original signal and therefore to reduce the energy of the residual. For example, when the updating of the forward mapping function is done in step 496, its goal is to make the inverse mapped reconstructed signal closer to the original signal and therefore to reduce the energy of corrective signal brought by the following in-loop filters.

In another embodiment, the default mapping function remains the reference mapping function for the next coding blocks. Other update of mapping function can be determined for the next coding blocks.

Examples for determining the update of the mapping function are described below.

It is considered here that the default mapping function corresponds to a LUT, def Lut[.], where defLut[ x ] is the mapped version of x.

The input samples of the process are either:

- a set of template samples in 1 (x,y) that are in the non-mapped domain (e.g. for the embodiment relating to the forward mapping, they may be the template prediction samples) and a set of template samples in2(x,y) that are in the mapped domain (e.g. for the embodiment relating to the forward mapping, they may be the template reconstructed samples),

- or a set of template samples in 1 (x,y) that are in the mapped domain (e.g. for the embodiment relating to the inverse mapping, they may be the template reconstructed samples) and a set of template samples in2(x,y) that are in the inverse-mapped domain (e.g. for the embodiment relating to the inverse mapping, they may be the template filtered samples).

The default mapping function def Lut[.] to be updated is either:

- a forward mapping function (e.g. FIG. 4, 5 and 8),

- or an inverse mapping function (e.g. FIG. 6, 7 and 9),

- or both the forward mapping function and inverse mapping function.

The following embodiments provide for deriving an updated mapping function updLut[.] from def Lut[.], in 1 (x,y), in2(x,y), for (x,y) in the template area.

In an embodiment, the updated mapping function updLut[.] is derived by minimizing over the template samples a distortion between the samples in2(x,y) and the samples in 1 (x,y) mapped by updLut. Two examples of distortions are described below:

where abs(x) is the absolute value of x.

A variant for minimizing this distortion is described as follows:

Stepl - initialization

Initialize 3 arrays (named occurrence, average and variance in the following) of length 2^BD - for k=0 to (2^BD - 1 ) (where BD is the bitdepth of the signal in 1 ,in2) o occurrence[ k ] = 0 o average[ k ] = 0 o variance[ k ] = 0

Initialize updLut to be equal to defLut: for k=0 to (2^BD - 1 ) o updLut[ k ] = defLut[ k ]

Step2 - samples occurrence counting

- Scan all the template pixels (x,y) o occurrence[ in 1 (x,y) ] = occurrence[ in 1 (x,y) ] +1 o average[ in 1 (x,y) ] = average[ in 1 (x,y) ] + in2(x,y) o variance[ in 1 (x,y) ] = variance[ in 1 (x,y) ] + ( in2(x,y)*in2(x,y) )

Step3 - samples average derivation

- for k=0 to (2^BD - 1) o if occurrence[ k ] > 0

■ average[ k ] = average[ k ] -? occurrence[ k ]

• variance[ k ] = ( variance[ k ] -? occurrence[ k ] )

- ( average[ k ]*average[ k ] ) o where the symbol is the floating-point division - an integer implementation can be also considered, with some adaptations of these equations.

Step4 - LUT update

- for k=0 to (2^BD - 1) o updLut[ k ] is updated based on average[ k ], and possibly variance[ k ]. For example: updLut[ k ] = a*updLut[ k ] + (1 -a)*average[ k ]

(rounded to closest integer value) with “a” being a parameter in [0,1], either predefined by default, or depending on variance[ k ]. For example, a=0.5, or 0.75. In another example case, a=0. The purpose of parameter “a” is to control the deviation from the default LLIT defLut. When “a” is close to 1 , updLut is constrained to stay close to defLut.

Alternatively, a = variance[ k ] -? ( b + variance[ k ] ), or a = sqrt( variance[ k ] ) -? ( b + sqrt( variance[ k ] ) ), where sqrt is the square root operator, and b a positive parameter, for instance equal to 1 .

In a variant, average[k] is set to 0 when occurrence[ k ] is below a threshold T_0Cc which can be for instance equal to T_0Cc = 4, or size_template/4, where size_template is a number of samples in the template).

In a variant, a sanity check of the updated mapping function is applied. For example, if for one or several k, variance[ k ] is too high (e.g. above a given threshold T_var which can be for instance equal to T_var = (2^BD / 16)*( 2^BD / 16), the LUT is not updated. Or if the maximum value of variance[ k ] is too high, the LUT is not updated.

Indeed, too large values of variance[ k ] indicate that there is too much dispersion of the mapped samples, and that the updated mapping for value k would not be reliable.

In another variant, the sanity check is based on the amplitude of average[ k ]. When for one or several k, average[ k ] is too high (e.g. above a given threshold T_avg which can be for instance equal to T_avg = (2^BD / 16)), the LUT is not updated. Or if the maximum value of average[ k ] is too high, the LUT is not updated. In another variant, the LUT is updated, but all average[ k ] , for any k, are clipped to a maximum value, for instance T_avg.

In another variant, the sanity check is based on occurrence[ k ]. When the maximum value of occurrence[ k ], over all k values, is below a threshold T_0Cc which can be for instance equal to Tocc — 4, or size_template/4), the LUT is not updated.

In the specific case of the embodiment relating to the update of the forward mapping function where the samples in1 corresponds to the template prediction samples (that are in non-mapped domain), and in2 corresponds to the template filtered samples (that are also in non-mapped domain), the goal of the mapping update is to derive updLut such as:

or

because both in1 (the template filtered samples) and in2 (the template prediction samples) are in the non-mapped domain. Therefore, the application of invLut has to be taken into account. Another embodiment for deriving the updated mapping function updLut[.] is provided below. In this alternate approach, a single offset Off is computed and used for all the values identified in the template area:

Stepl - initialization

Initialize an occurrence parameter “occurrence” and an offset parameter “Off” o occurrence = 0 o Off = 0

Step2 - samples occurrence counting

- Scan all the template pixels (x,y) o occurrence = occurrence +1 o Off = Off + (in2(x,y) - defLut[ in 1 (x,y) ] )

Step3 - offset derivation if occurrence > 0 o Off = ( Off + occurrence/2 ) / occurrence

Step4 - LUT update based on offset

- Scan all the template pixels (x,y) o updLut[ in 1 (x,y) ] = defLut[ in 1 (x,y) ] + Off o possibly values around in 1 (x,y) may also be modified:

■ for r= max(0 , in 1 (x,y)-K ) to min( 2^BD-1 , in 1 (x,y)+K )

• updLut[ r ] = defLut[ r ] + Off where K is predefined parameter, for example equal to 2, 4, 8, 16 or 32.

In a variant, several offsets are considered, for instance, the average of in1 is computed over the template, then one first offset is defined for samples that are lower than the average, and one second offset is used for samples that are larger than or equal to the average. In another example, the samples range is divided into N sub-intervals (e.g. N=4), and for each sub-interval of index i (i=0 to N-1 ) an offset Off i is estimated. For each r in interval i, updLut[ r ] is then set to ( defLut[ r ] + Offi ).

In some embodiments, an history list of mapping functions is used to provide multiple mapping functions. For example, after the processing of each CTU, an updated mapping function is derived as described above (using the samples inside the CTU that has been processed, instead of the template samples surrounding the CTU). The derived mapping function is inserted in the history list of mapping functions, e.g. using a first-in-first-out approach. This advantageously allows dynamic adaptation of the used mapping function, to adapt to the local variations of the signal. For the current block, the updated mapping function may be selected among all the mapping functions stored in the history list of mapping functions.

At block level (CU or CTU), an index may be signaled to indicate which mapping function of the list is used for the current block.

In a variant, the default LUT is always included in the list, possibly at the first position in the list. In another variant, the first candidate in the list is the latest derived updated mapping function.

In a variant, one index is used to indicate that the updated mapping function must be computed based on template samples, and not picked from a mapping function previously stored in the list.

This approach may apply to the forward mapping function, as well as to the inverse mapping function. The idea can also be extended to other types of area than CTU, for instance group of CTUs, tiles, subpictures, slices. For instance, a mapping function can be updated using a reference area, such as a group of preceding CTUs, tiles, subpictures or slices. When applied to such a reference area, in a variant, the list can comprise only one updated mapping function, which is the latest updated mapping function from the reference area. In another variant, the list also comprises the default mapping function.

In the embodiments described above, the template samples considered from updating the mapping function are taken from the causal spatial neighborhood of the current block to encode or decode. In a variant, the template samples can also belong to reference areas that comprises temporal blocks or CTUs in the reference frames. For example, the collocated temporal blocks or CTUs in the reference pictures used for predicting the current block are used as reference areas to update the mapping function of the current block. In another example, the temporal blocks or CTUs in the reference pictures pointed by the motion vector(s) MV used for predicting the current block are used as reference areas to update the mapping function of the current block. An illustration of this example is provided in the top part (a) of FIG.10.

Several reference areas may be used when the current block is predicted from several reference pictures (example of bi-predicted block), or several subblocks in the reference pictures (example of sub-block temporal motion vector prediction, or of sub-block affine motion prediction, as in VVC). This is illustrated in the bottom part (b) of FIG.10 where the current block is made of two subblocks, each one being predicted from a different reference picture using distinct motion vectors (MV1 , MV2).

In an embodiment, the updating process from any of the embodiments described above may apply even when LMCS is not activated for the current block, slice or picture. In that case, the default mapping function is considered to be an identity function as the default forward and inverse mapping functions. fwdLut[ x ] = lnvLut[ x ] = x, for any x in [ 0 , (2^BD - 1 ) ]

In that case, even when determining the LMCS is disabled, the use of a mapping function can still be enabled by determining an update mapping function from the identity function and applying the updated mapping function to the current block. The rest of the processes described above can then apply in the same manner as when LMCS is enabled. For determining the update of the mapping function updLut[ k ], the default mapping function defLut[ k ] is set to the identity function, or the updLut[ k ] function is initialized with the identity function.

In this embodiment, the update of the mapping function can be enabled or disabled independently from the enablement of LMCS tool.

An aim of the update of the mapping function is to enable local correction of the block’s prediction or block’s reconstruction (depending on where the mapping is performed) per sample values. When LMCS is disabled, this embodiment provides a new block-based tool that enables the correction of the block’s prediction using a mapping function determined locally for the block. Parameters of the mapping function are determined for the current block based on the template samples as described in any of the embodiments above. Comparing with the LIC (local illumination compensation) tool that provides for a linear correction of the prediction, an advantage of the embodiments provided herein wherein the correction is performed by the mapping function is that the correction is determined locally per sample values. Advantageously, the mapping function is implemented using a LUT that enables providing a correction that is no more linear (or based on a parametric model). The LUT representing the mapping function is determined based on template samples histogram or occurrences. Such embodiments described herein provide for locally adapting the sample values mapping on the fly when encoding or decoding a current block.

In an embodiment, instead of considering look-up-tables as the representations of the forward and inverse mapping functions, the mapping functions are represented as piece-wise linear functions. For example, P pieces are used. One piece of index k, k=0 to P - 1 , is defined by 2 pivot points pivot(k)=(px(k),py(k)) and pivot(k+1 )=(px(k+1 ),py(k+1 )). Samples s in piece of index k are those such that s >= px(k) and s < px(k+1 ). s is mapped to mapk(s): mapk(s) = py(k) + (s - px(k)) * (py(k+1 ) - py(k)) - (px(k+1 ) - px(k)), k being the index such that s >= px(k) and s < px(k+1 ).

The formula is represented in floating-point, but a fixed-point formulation can be easily derived. Using a look-up-table notation, this is equivalent to:

LUT[s] = mapk(s), k being the index such that s >= px(k) and s < px(k+1 ). (py(k+1 ) - py(k)) -? (px(k+1 ) - px(k)) represents the slope of the piece of index k. The updating process of the mapping function may consist in updating the pivot points (or equivalently the slopes), in a same manner as described above. For instance, the updated pivot points may be estimated in order to minimize the distortion:

Or

with k being the index such that jnl(x,y) >= px(k) and jnl(x,y) < px(k+1 ).

FIG. 15 illustrates a block diagram of a system within which aspects of the present embodiments may be implemented, according to another embodiment. FIG. 15 shows one embodiment of an apparatus 1500 for encoding or decoding a video according to any one of the embodiments described herein. The apparatus comprises Processor 1510 and can be interconnected to a memory 1520 through at least one port. Both Processor 1510 and memory 1520 can also have one or more additional interconnections to external connections.

Processor 1510 is also configured to, using any one of the embodiments described herein. For instance, the processor 1510 is configured to determine for at least one block of the video, an update of at least one mapping function using template samples of the at least one block, the mapping function being an in-loop mapping function, and encode or decode the at least one block using the at least one updated mapping function, using any one of the embodiments described herein. For instance, the processor 1510 using a computer program product comprising code instructions that implements any one of embodiments described herein.

In an embodiment, illustrated in FIG. 16, in a transmission context between two remote devices A and B over a communication network NET, the device A comprises a processor in relation with memory RAM and ROM which are configured to implement a method for encoding a video, as described with FIG. 1 -15 and the device B comprises a processor in relation with memory RAM and ROM which are configured to implement a method for decoding a video as described in relation with FIG 1 -15. In accordance with an example, the network is a broadcast network, adapted to broadcast/transmit a coded video from device A to decoding devices including the device B.

FIG. 16 shows an example of the syntax of a signal transmitted over a packet-based transmission protocol. Each transmitted packet P comprises a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may comprise video data encoded according to any one of the embodiments described above. The payload can also comprise any signaling as described above. For example, the payload can comprise one or more indicators. The one or more indicators comprise at least one of an indicator that indicates whether an updated mapping function or a default mapping function is used for the at least one block, or an indicator that indicates which updated mapping function to use for the at least one block when multiple updated mapping functions are available for the at least one block.

Various implementations involve decoding. “Decoding”, as used in this application, can encompass all or part of the processes performed, for example, on a received encoded sequence in order to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by a decoder of various implementations described in this application, for example, entropy decoding a sequence of binary symbols to reconstruct image or video data.

As further examples, in one embodiment “decoding” refers only to entropy decoding, in another embodiment “decoding” refers only to differential decoding, and in another embodiment “decoding” refers to a combination of entropy decoding and differential decoding, and in another embodiment “decoding” refers to the whole reconstructing picture process including entropy decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or generally to the broader decoding process will be clear based on the context of the specific descriptions and is believed to be well understood by those skilled in the art.

Various implementations involve encoding. In an analogous way to the above discussion about “decoding”, “encoding” as used in this application can encompass all or part of the processes performed, for example, on an input video sequence in order to produce an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, for example, partitioning, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, such processes also, or alternatively, include processes performed by an encoder of various implementations described in this application, for example, determining re-sampling filter coefficients, resampling a decoded picture.

As further examples, in one embodiment “encoding” refers only to entropy encoding, in another embodiment “encoding” refers only to differential encoding, and in another embodiment “encoding” refers to a combination of differential encoding and entropy encoding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or generally to the broader encoding process will be clear based on the context of the specific descriptions and is believed to be well understood by those skilled in the art.

Note that the syntax elements as used herein, are descriptive terms. As such, they do not preclude the use of other syntax element names.

This disclosure has described various pieces of information, such as for example syntax, that can be transmitted or stored, for example. This information can be packaged or arranged in a variety of manners, including for example manners common in video standards such as putting the information into an SPS, a PPS, a NAL unit, a header (for example, a NAL unit header, picture header or a slice header), or an SEI message. Other manners are also available, including for example manners common for system level or application level standards such as putting the information into one or more of the following: a. SDP (session description protocol), a format for describing multimedia communication sessions for the purposes of session announcement and session invitation, for example as described in RFCs and used in conjunction with RTP (Real-time Transport Protocol) transmission. b. DASH MPD (Media Presentation Description) Descriptors, for example as used in DASH and transmitted over HTTP, a Descriptor is associated to a Representation or collection of Representations to provide additional characteristic to the content Representation. c. RTP header extensions, for example as used during RTP streaming. d. ISO Base Media File Format, for example as used in OMAF and using boxes which are object-oriented building blocks defined by a unique type identifier and length also known as 'atoms' in some specifications. e. HLS (HTTP live Streaming) manifest transmitted over HTTP. A manifest can be associated, for example, to a version or collection of versions of a content to provide characteristics of the version or collection of versions.

When a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of a corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of a corresponding method/process.

Some embodiments refer to rate distortion optimization. In particular, during the encoding process, the balance or trade-off between the rate and distortion is usually considered, often given the constraints of computational complexity. The rate distortion optimization is usually formulated as minimizing a rate distortion function, which is a weighted sum of the rate and of the distortion. There are different approaches to solve the rate distortion optimization problem. For example, the approaches may be based on an extensive testing of all encoding options, including all considered modes or coding parameters values, with a complete evaluation of their coding cost and related distortion of the reconstructed signal after coding and decoding. Faster approaches may also be used, to save encoding complexity, in particular with computation of an approximated distortion based on the prediction or the prediction residual signal, not the reconstructed one. Mix of these two approaches can also be used, such as by using an approximated distortion for only some of the possible encoding options, and a complete distortion for other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete evaluation of both the coding cost and related distortion.

The implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented in, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users.

Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment.

Additionally, this application may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

Further, this application may refer to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information. Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

It is to be appreciated that the use of any of the following

“and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.

Also, as used herein, the word “signal” refers to, among other things, indicating something to a corresponding decoder. In this way, in an embodiment the same parameter is used at both the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter. Conversely, if the decoder already has the particular parameter as well as others, then signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter. By avoiding transmission of any actual functions, a bit savings is realized in various embodiments. It is to be appreciated that signaling can be accomplished in a variety of ways. For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun.

As will be evident to one of ordinary skill in the art, implementations can produce a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal can be formatted to carry the bitstream of a described embodiment. Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor- readable medium.

A number of embodiments has been described above. Features of these embodiments can be provided alone or in any combination, across various claim categories and types.

Claims

1 . A method, comprising: determining for at least one block of a video, at least one mapping function using template samples of the at least one block, encoding or decoding the at least one block using the at least one mapping function.

2. An apparatus, comprising one or more processors, wherein said one or more processors are operable to: determine for at least one block of a video, at least one mapping function using template samples of the at least one block, encode or decode the at least one block using the at least one mapping function.

3. The method of claim 1 or the apparatus of claim 2, wherein the mapping function is a forward mapping function, and wherein encoding or decoding the at least one block using the mapping function comprises applying the mapping function to a prediction block of the at least one block.

4. The method of claim 1 or 3 or the apparatus of claim 2-3, wherein the mapping function is an inverse mapping function and wherein encoding or decoding the at least one block using the mapping function comprises applying the mapping function to a reconstruction of the at least one block.

5. The method of claim 1 or 3-4 or the apparatus of claim 2-4, wherein determining the mapping function determines a Look-Up-Table.

6. The method of claim 1 or 3-5 or the apparatus of claim 2-5, wherein the mapping function is an in-loop mapping function.

7. The method of claim 1 or 3-6 or the apparatus of claim 2-6, wherein determining the at least one mapping function uses template reconstructed samples of the at least one block and at least one of template prediction samples of a prediction block of the at least one block, or template filtered samples of the at least one block.

8. The method of claim 1 or 3-7 or the apparatus of claim 2-7, wherein determining the mapping function updates a default mapping function.

9. The method or the apparatus of claim 8, wherein the default mapping function is built from mapping function parameters encoded in a bitstream for at least one picture of the video.

10. The method or the apparatus of claim 8, wherein the default mapping function is an identity mapping function.

1 1 . The method or the apparatus of any of claims 8-10, wherein an indicator indicates whether the updated mapping function or the default mapping function is used for the at least one block.

12. The method or the apparatus of claim 1 1 , wherein the indicator is signaled in a bitstream at a sequence level, or a picture level or a slice level or an APS level, or at a block level.

13. The method or the apparatus of claim 1 1 , wherein the indicator is inferred when determining the updated mapping function.

14. The method of claim 1 or 3-13 or the apparatus of claim 2-13, wherein when multiple mapping functions are available for the at least one block, an indicator indicates which mapping function to use for the at least one block.

15. The method or the apparatus of claim 14, wherein the indicator is signaled in a bitstream.

16. The method or the apparatus of claim 14 or 15, wherein multiple mapping functions are obtained using distinct sets of template samples.

17. The method or the apparatus of claim 14 or 15, wherein the multiple mapping functions are obtained from an history list of mapping functions.

18. The method of claim 1 or 3-17 or the apparatus of claim 2-17, wherein the mapping function used for the at least one block is also used for one or more blocks following the at least one block in encoding or decoding order.

19. The method or the apparatus of claim 18, wherein both a forward mapping function and its inverse mapping function are updated.

20. The method of claim 1 or 3-19 or the apparatus of claim 2-19, wherein determining the at least one mapping function comprises at least one of determining an update of a forward mapping function, determining an update of an inverse mapping function, or determining an update of both the forward mapping function and the inverse mapping function.

21. The method or the apparatus of claim 20, wherein determining the at least one mapping function comprises minimizing over the template samples a distortion between a first version of the template samples and a second version of the template samples mapped by the updated mapping function.

22. The method or the apparatus of claim 20, wherein determining the mapping function using template samples of the at least one block is based on histograms or occurrences determined on the template samples.

23. The method or the apparatus of claim 20, wherein determining the at least one mapping function comprises determining one or more offsets based on template samples.

24. The method of claim 1 or 3-23 or the apparatus of claim 2-23, wherein the template samples are neighboring samples of the at least one block and/or reference temporal samples of the at least one block.

25. The method of claim 1 or 3-24 or the apparatus of claim 2-24, wherein determining the at least one mapping function is responsive to a determination that a luma mapping and chroma scaling tool is disabled and wherein the at least one mapping function is an identity mapping function.

26. A computer program product including instructions for causing one or more processors to carry out the method of any of claims 1 , 3-25.

27. A non-transitory computer readable medium storing executable program instructions to cause a computer executing the program instructions to perform a method according to any of claims 1 , 3-25.

28. A bitstream comprising data representative of a video, wherein the bitstream comprises for at least one block of the video, at least one of an indicator that indicates whether an updated mapping function or a default mapping function is used for the at least one block, or an indicator that indicates which mapping function to use for the at least one block when multiple mapping functions are available for the at least one block.

29. A non-transitory computer readable medium storing a bitstream of claim 28.

30. A device comprising: an apparatus according to claim 2; and at least one of (i) an antenna configured to receive or transmit a signal, the signal including data representative of the video, (ii) a band limiter configured to limit the signal to a band of frequencies that includes the data representative of video, or (iii) a display configured to display the video.

31. A device according to claim 30, wherein the device comprises at least one of a television, a cell phone, a tablet, a set-top box.