WO2025149463A1

WO2025149463A1 - Method and apparatus for encoding/decoding

Info

Publication number: WO2025149463A1
Application number: PCT/EP2025/050211
Authority: WO
Inventors: Didier Doyen; Edouard Francois; Ya CHEN; Sylvain Thiebaud
Original assignee: InterDigital CE Patent Holdings SAS
Current assignee: InterDigital CE Patent Holdings SAS
Priority date: 2024-01-09
Filing date: 2025-01-07
Publication date: 2025-07-17
Anticipated expiration: 2026-07-09

Abstract

In various implementations, method and devices are disclosed that determines a luma forward mapping function applied to a prediction of luma samples that maps codewords used to represent luma samples within an input luma range into a coding luma range by reallocating a total number of codewords of coding luma range into a fewer number of pieces than an initial number of pieces. For instance, on condition that a minimum value or a maximum value of the input luma range is not a multiple of an initial average number of codewords per piece, the number of codeword for a first piece comprising the minimum value or for a last piece comprising the maximum value is adjusted such that all or a maximized number of codewords of coding luma range are used to represent luma samples within the input luma range.

Description

METHOD AND APPARATUS FOR ENCODING/DECODING

CROSS REFERENCE TO RELATED APPLICATION

[1] This application claims the benefit of European Patent Application No. 24305053.1, filed on January 9, 2024, which is incorporated herein by reference in their entirety.

TECHNICAL FIELD

[2] The present embodiments generally relate to a method and an apparatus for video encoding or decoding including with luma mapping with chroma scaling function LMCS.

BACKGROUND

[3] To achieve high compression efficiency, image and video coding schemes usually employ prediction and transform to leverage spatial and temporal redundancy in the video content. Generally, intra or inter prediction is used to exploit the intra or inter picture correlation, then the differences between the original block and the predicted block, often denoted as prediction errors or prediction residuals, are transformed, quantized, and entropy coded. To reconstruct the video, the compressed data are decoded by inverse processes corresponding to the entropy coding, quantization, transform, and prediction.

SUMMARY

[4] In various implementations, methods and devices are disclosed that adjust the derivation a luma mapping with chroma scaling function LMCS to effective MIN/MAX values of the video.

[5] According to a first aspect, a method of video encoding is disclosed that comprises determining a luma forward mapping function applied to a prediction of luma samples that maps codewords used to represent luma samples within an input luma range into a coding luma range by reallocating a total number of codewords of coding luma range into a fewer number of pieces than an initial number of pieces. For instance, on condition that a minimum value or a maximum value of the input luma range is not a multiple of an initial average number of codewords per piece, the number of codeword for a first piece comprising the minimum value or for a last piece comprising the maximum value is adjusted such that all or a maximized number of codewords of coding luma range are used to represent luma samples within the input luma range. The method further comprising encoding video by at least applying the luma forward mapping function to a prediction of the luma samples of the video. i [6] According to a second aspect, a method of video decoding is disclosed that comprises obtaining a minimum value and a maximum value of an input luma range; and determining a luma forward mapping function applied to a prediction of luma samples that maps codewords used to represent luma samples within an input luma range into a coding luma range by reallocating a total number of codewords of coding luma range into a fewer number of pieces than an initial number of pieces. For instance, on condition that a minimum value or a maximum value of the input luma range is not a multiple of an initial average number of codewords per piece, the number of codeword for a first piece comprising the minimum value or for a last piece comprising the maximum value is adjusted such that all or a maximized number of codewords of coding luma range are used to represent luma samples within the input luma range. The method further comprising decoding video by at least applying an inverse function of the luma forward mapping function to a reconstruction of the luma samples of the video.

[7] One or more embodiments also provide an apparatus for encoding or decoding video comprising one or more processors and at least one memory coupled to said one or more processors, wherein said one or more processors are configured to perform the encoding or decoding method according to any of the embodiments described herein.

[8] One or more embodiments also provide a computer program comprising instructions which when executed by one or more processors cause the one or more processors to perform the encoding or decoding method according to any of the embodiments described herein. One or more of the present embodiments also provide a computer readable storage medium having stored thereon instructions for encoding or decoding a video according to the methods described herein.

[9] One or more embodiments also provide a computer readable storage medium having stored thereon video data generated according to the methods described above. One or more embodiments also provide a method and apparatus for transmitting or receiving the video data generated according to the methods described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[10] FIG. 1 illustrates a block diagram of a system within which aspects of the present embodiments may be implemented.

[11] FIG. 2 illustrates a block diagram of an embodiment of a video encoder.

[12] FIG. 3 illustrates a block diagram of an embodiment of a video decoder. [13] FIG. 4 illustrates an example of forward mapping LUT according to prior art.

[14] FIG. 5 illustrates an example of forward mapping LUT according to a first variant of a second embodiment.

[15] FIG. 6 illustrates an example of forward mapping LUT according to a second variant of a second embodiment.

[16] FIG. 7 illustrates an example of forward mapping LUT according to a first and second variant of a third embodiment.

[17] FIG. 8 illustrates a method for determining a luma forward mapping function according to an embodiment.

DETAILED DESCRIPTION

[18] FIG. 1 illustrates a block diagram of an example of a system in which various aspects and embodiments can be implemented. System 100 may be embodied as a device including the various components described below and is configured to perform one or more of the aspects described in this application. Examples of such devices, include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 100, singly or in combination, may be embodied in a single integrated circuit, multiple ICs, and/or discrete components. For example, in at least one embodiment, the processing and encoder/decoder elements of system 100 are distributed across multiple ICs and/or discrete components. In various embodiments, the system 100 is communicatively coupled to other systems, or to other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports. In various embodiments, the system 100 is configured to implement one or more of the aspects described in this application.

[19] The system 100 includes at least one processor 110 configured to execute instructions loaded therein for implementing, for example, the various aspects described in this application. Processor 110 may include embedded memory, input output interface, and various other circuitries as known in the art. The system 100 includes at least one memory 120 (e.g., a volatile memory device, and/or a non-volatile memory device). System 100 includes a storage device 140, which may include non-volatile memory and/or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drive, and/or optical disk drive. The storage device 140 may include an internal storage device, an attached storage device, and/or a network accessible storage device, as non-limiting examples.

[20] System 100 includes an encoder/decoder module 130 configured, for example, to process data to provide an encoded video or decoded video, and the encoder/decoder module 130 may include its own processor and memory. The encoder/decoder module 130 represents module(s) that may be included in a device to perform the encoding and/or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Additionally, encoder/decoder module 130 may be implemented as a separate element of system 100 or may be incorporated within processor 110 as a combination of hardware and software as known to those skilled in the art.

[21] Program code to be loaded onto processor 110 or encoder/decoder 130 to perform the various aspects described in this application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. In accordance with various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder/decoder module 130 may store one or more of various items during the performance of the processes described in this application. Such stored items may include, but are not limited to, the input video, the decoded video or portions of the decoded video, the bitstream, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[22] In several embodiments, memory inside of the processor 110 and/or the encoder/decoder module 130 is used to store instructions and to provide working memory for processing that is needed during encoding or decoding. In other embodiments, however, a memory external to the processing device (for example, the processing device may be either the processor 110 or the encoder/decoder module 130) is used for one or more of these functions. The external memory may be the memory 120 and/or the storage device 140, for example, a dynamic volatile memory and/or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[23] The input to the elements of system 100 may be provided through various input devices as indicated in block 105. Such input devices include, but are not limited to, (i) an RF portion that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a Composite input terminal, (iii) a USB input terminal, and/or (iv) an HDMI input terminal.

[24] In various embodiments, the input devices of block 105 have associated respective input processing elements as known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) down converting the selected signal, (iii) band- limiting again to a narrower band of frequencies to select (for example) a signal frequency band which may be referred to as a channel in certain embodiments, (iv) demodulating the down converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF portion of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include a tuner that performs various of these functions, including, for example, down converting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. In one set-top box embodiment, the RF portion and its associated input processing element receives an RF signal transmitted over a wired (for example, cable) medium, and performs frequency selection by filtering, down converting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and/or add other elements performing similar or different functions. Adding elements may include inserting elements in between existing elements, for example, inserting amplifiers and an analog-to-digital converter. In various embodiments, the RF portion includes an antenna.

[25] Additionally, the USB and/or HDMI terminals may include respective interface processors for connecting system 100 to other electronic devices across USB and/or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed- Solomon error correction, may be implemented, for example, within a separate input processing IC or within processor 110 as necessary. Similarly, aspects of USB or HDMI interface processing may be implemented within separate interface ICs or within processor 110 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110, and encoder/decoder 130 operating in combination with the memory and storage elements to process the datastream as necessary for presentation on an output device.

[26] Various elements of system 100 may be provided within an integrated housing, Within the integrated housing, the various elements may be interconnected and transmit data therebetween using suitable connection arrangement 115, for example, an internal bus as known in the art, including the I2C bus, wiring, and printed circuit boards.

[27] The system 100 includes communication interface 150 that enables communication with other devices via communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and to receive data over communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network card and the communication channel 190 may be implemented, for example, within a wired and/or a wireless medium.

[28] Data is streamed to the system 100, in various embodiments, using a Wi-Fi network such as IEEE 802. 11. The Wi-Fi signal of these embodiments is received over the communications channel 190 and the communications interface 150 which are adapted for WiFi communications. The communications channel 190 of these embodiments is typically connected to an access point or router that provides access to outside networks including the Internet for allowing streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 100 using a set- top box that delivers the data over the HDMI connection of the input block 105. Still other embodiments provide streamed data to the system 100 using the RF connection of the input block 105.

[29] The system 100 may provide an output signal to various output devices, including a display 165, speakers 175, and other peripheral devices 185. The other peripheral devices 185 include, in various examples of embodiments, one or more of a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide a function based on the output of the system 100. In various embodiments, control signals are communicated between the system 100 and the display 165, speakers 175, or other peripheral devices 185 using signaling such as AV. Link, CEC, or other communications protocols that enable device-to-device control with or without user intervention. The output devices may be communicatively coupled to system 100 via dedicated connections through respective interfaces 160, 170, and 180. Alternatively, the output devices may be connected to system 100 using the communications channel 190 via the communications interface 150. The display 165 and speakers 175 may be integrated in a single unit with the other components of system 100 in an electronic device, for example, a television. In various embodiments, the display interface 160 includes a display driver, for example, a timing controller (T Con) chip. [30] The display 165 and speaker 175 may alternatively be separate from one or more of the other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments in which the display 165 and speakers 175 are external components, the output signal may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[31] FIG. 2 illustrates an example video encoder 200, such as a VVC (Versatile Video Coding) encoder. FIG. 2 may also illustrate an encoder in which improvements are made to the VVC standard or an encoder employing technologies similar to VVC.

[32] In the present application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “encoded” or “coded” may be used interchangeably, and the terms “image,” “picture” and “frame” may be used interchangeably. Usually, but not necessarily, the term “reconstructed” is used at the encoder side while “decoded” is used at the decoder side.

[33] Before being encoded, the video sequence may go through pre-encoding processing (201), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to get a signal distribution more resilient to compression. Metadata can be associated with the pre-processing, and attached to the bitstream.

[34] In the encoder 200, a picture is encoded by the encoder elements as described below. The picture to be encoded is partitioned (202) and processed in units of, for example, CUs. Each unit is encoded using, for example, either an intra or inter mode. When a unit is encoded in an intra mode, it performs intra prediction (260). In an inter mode, motion estimation (275) and compensation (270) are performed. The encoder decides (205) which one of the intra mode or inter mode to use for encoding the unit, and indicates the intra/inter decision by, for example, a prediction mode flag. After prediction, prediction enhancement (285) is applied to the prediction block. Prediction residuals are calculated, for example, by subtracting (210) the predicted block from the original image block.

[35] The prediction residuals are then transformed (225) and quantized (230). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the non-transformed residual signal. The encoder can bypass both transform and quantization, i.e., the residual is coded directly without the application of the transform or quantization processes. [36] The encoder decodes an encoded block to provide a reference for further predictions. The quantized transform coefficients are de-quantized (240) and inverse transformed (250) to decode prediction residuals. Combining (255) the decoded prediction residuals and the predicted block, an image block is reconstructed. In-loop filters (265) are applied to the reconstructed picture to perform, for example, deblocking/SAO (Sample Adaptive Offset) filtering to reduce encoding artifacts. The filtered image is stored at a reference picture buffer (280).

[37] FIG. 3 illustrates a block diagram of an example video decoder 300. In the decoder 300, a bitstream is decoded by the decoder elements as described below. Video decoder 300 generally performs a decoding pass reciprocal to the encoding pass as described in FIG. 2. The encoder 200 also generally performs video decoding as part of encoding video data.

[38] In particular, the input of the decoder includes a video bitstream, which can be generated by video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other coded information. The picture partition information indicates how the picture is partitioned. The decoder may therefore divide (335) the picture according to the decoded picture partitioning information. The transform coefficients are de-quantized (340) and inverse transformed (350) to decode the prediction residuals. Combining (355) the decoded prediction residuals and the predicted block, an image block is reconstructed. The predicted block can be obtained (370) from intra prediction (360) or motion-compensated prediction (i.e., inter prediction) (375). After prediction, prediction enhancement (390) is applied to the prediction block. In-loop filters (365) are applied to the reconstructed image. The filtered image is stored at a reference picture buffer (380).

[39] The decoded picture can further go through post-decoding processing (385), for example, an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping performing the inverse of the remapping process performed in the preencoding processing (201). The post-decoding processing can use metadata derived in the preencoding processing and signaled in the bitstream.

[40] Recent additions to video compression technology include various industry standards, versions of the reference software and/or documentations such as Enhanced Compression Model (ECM) being developed by the JVET (Joint Video Exploration Team) group. The aim is to make further improvements to the existing VVC (Versatile Video Coding) standard. In most of the sequences especially for captured content one’s, data range of the luma is not corresponding to 1024 levels as expected for a 10-bit signal but it is corresponding to the limited range of 64-940 (also named standard range). In VVC and ECM, the LMCS function allows to modify the luma of the content to better exploit the 10-bit capabilities. A mapping function is applied to luma based on a pre-calculated LUT, the pre-calculated LUT being computed at the encoder side. Values of the LUT are defined by the encoder based on specific characteristic of the content. For instance, the content is a 10-bit SDR content. For this purpose, the 10-bit representation (1024 levels) is split into 16 pieces (intervals) of 64 levels. According to a current implementation of LMCS mapping function derivation in ECM encoder, the following steps are applied in the Random- Access mode.

[41] In a first step, video parameters are computed. For instance, for a given video or video chunk, some parameters are extracted, typically on I- frames such as the histogram and the local variance of the frame per pieces. The result of the histogram is given with 16 values (corresponding to the 16 pieces), each corresponding to 1/16 of the range of values between 0 and 1023 for a 10 bits signal. Hist[0] is corresponding to the occurrence of samples having a luma between 0 and 63 on the whole picture, Hist[ 1] the occurrence of sample having a luma between 64 and 127 and so on. Histfi] for i=0 to 15 are normalized so that HistfO] + ... + Hist[15] = 1.0. The variance of the sample P having the value of luma P(x,y) is calculated as the sum of square value of pixel on a 5*5 window divided by the size of the window minus the square of the average value on this window. This variance is summed with variances of the same piece, the piece is determined by the luma value of the current sample.

[42] In a second step, a decision is made to apply LMCS or not. For instance, if Hist[0] and Hist[15] are both below a threshold (e.g. 0.001), the LCMS function is activated meaning that the Code Words from 0-63 and 960-1023 are not currently used for coding this sequence.

[43] In a third step, a calculation of the global number of code words is processed. For instance, 896 codes were used to code values between 64 and 959. The new Number of Code Words (NbCW) can be between 896 and 1022 to encode these 896 values between 64 and 959. At this stage NbCW is always a multiple of 14. NbCW is determined based on different thresholds calculated from histogram and variance values.

[44] In a fourth step, the code words CW per piece are refined. At the previous step, the number of CW per piece is the same for all the pieces, ie 73 (NbCW is a multiple of 14). The refinement process will use the histogram and the variance value of each piece to slightly modified the number of CW per piece around the initial value calculated at the third step. This step adjusts the allocation of code words such that more code words are allocated to pieces with smooth regions (represented by a lower normalized average variance normVar[i] for the ith piece) and fewer code words are allocated to pieces with other regions (represented by a higher normalized average variance normVar[i] for the ith piece). The amplitude of the adjustment is higher for pieces with higher occurrence (represented by a higher Hist[i] for ith piece). The sign of the adjustment is depending on the variance. For smooth regions the adjustment is positive and negative for high variance regions. This corresponds to the following pseudo-code: for (int i = 0; i < 16; i++) { if (Hist[i] > 0.001) { hist = Hist[i] > 0.4 ? 0.4 : Hist[i] ; deltal = (uintl6 t) (10.0 * hist + 0.5) ; delta2 = (u hist + 0.5) ; if (normVar[ m_binCW[i] = m_binCW[i] + delta2;

} else if (normVar[i] < 0.9) { m_binCW[i] = m_binCW[i] + deltal;

} if (normVar[i] > 1.2) { m_binCW[i] = m_binCW[i] - delta2;

} else if (normVar[i] > 1.1) { m_binCW[i] = m_binCW[i] - deltal;

} }

[45] A correction step is applied to ensure the sum of number of CW per piece is not above 1023.

[46] In a fifth step, a specific rate adaptation processing is applied when encoding with low values of QP that preserve most spatial details in a picture. For QP<=22 the NbCW is modified and is set to 924 (66 CW per piece).

[47] In a sixth step, the LUT corresponding to the number of CW per piece is generated. It is called the Forward_Mapping LUT and it will be used each time a conversion from the nonmapped to the mapped domain is required. In parallel the Inverse_Mapping LUT is also generated based on the same number of CW per pieces. It will be used each time a conversion from the mapped to the non-mapped domain is required. The Forward_Mapping and Inverse_Mapping LUTs are applied to the video for any other frames than I-frames.

[48] Besides, the LMCS parameters are signaled by the encoder in the APS with the syntax of Table 1.

Table 1. LMCS syntax table.

[49] At decoder side, the decoder decodes this information, rebuilds the forward and inverse mapping functions, and uses these functions during the decoding process (forward mapping is applied to inter prediction samples, and inverse mapping is applied after the reconstruction process and before the in-loop filtering).

[50] The skilled in the art will note that the syntax in VVC allows computing and signaling LMCS parameters for any frame. In practice, in the latest VTM and ECM implementations, LMCS parameters are computed by the encoder and signaled only for 1-frames.

[51] The LMCS function as it is derived in the ECM encoder considers only testing for a 10-bit luma signal, values 64 and 960 as MIN and MAX values for the luma. If a number of samples has a luma level below 64 or above 960 (for a 10-bit video signal) the LMCS function is not applied. But for a captured video content in limited range, MIN/MAX values may not be 64/960 respectively but rather 64/940.

[52] EIG. 4 illustrates an example of forward mapping LUT according to prior art. In the example of EIG. 4, MIN/MAX values for LMCS are respectively set to 64 and 960. Then if LMCS is applied, the 64 and 960 are respectively the starting point and the ending point of the forward mapping LUT as shown on EIG.4. Lor each piece, 73 CW are defined from the initial 64 values which gives at the end 1022 CW. However, for some sequences, depending on the content, the MIN value is equal to or higher than 64 and the MAX value is equal to or lower than 960. The skilled in the art will recognize that the forward LUT is not optimized since there is no interest to encode video values lower than the MIN value or/and higher than the MAX value. There is still room for improvement of the performance of the LMCS mapping.

[53] The present document proposes considering the 64/940 video levels and MIN/MAX values of the video to derive the LMCS function.

[54] In a first embodiment, the activation of the LMCS function is determined by comparing the histogram of the video with the 64/940 thresholds.

[55] In a second embodiment, the number of CW per piece is based on the 64/940 luma values.

[56] In a third embodiment, the number of CW per piece is based on the MIN/MAX luma values.

[57] In a fourth embodiment, the number of pieces and the number of CW per piece are based on the MIN/MAX luma values.

[58] In a fifth embodiment, the LMCS function is also applied to chrominance using the corresponding MIN/MAX values of each chrominance component to calculate corresponding number of CW.

[59] A generic embodiment for determining a LMCS function optimizing on MIN/MAX values.

[60] FIG. 8 illustrates a method for determining a luma forward mapping function according to an embodiment. The method 800 of FIG. 8 may be implemented in an encoder of FIG. 2 or a decoder of FIG. 3. For instance, a luma forward mapping function is applied in an encoder to a prediction of the luma samples and the luma forward mapping function maps codewords used to represent luma samples within the input luma range ([MIN, MAX]) into a coding luma range ([0, 2^AB]) by reallocating the total number (2^AB) of codewords of the coding luma range into a fewer number of pieces than an initial number of pieces, a piece being a division of the coding luma range by an initial number of pieces being an integer. For instance, an inverse luma mapping function is applied in a decoder to a reconstruction of the luma samples before the inloop filtering. Besides, the forward mapping function is also applied in a decoder to the inter prediction of the luma samples obtained from the DPB. According to a particular feature, the activation of the forward luma mapping function in the encoder and decoder is determined by comparing the histogram of the video with the effective input luma range. According to variant embodiments, the forward mapping function is either determined in the encoder and signaled to a decoder, or the forward mapping function is determined a same way in the encoder and the decoder based on the effective input luma range. In the following, the term codewords, levels or code values may be used indifferently to represent the values for coding the signal on an input bit-depth B. According to a particular embodiment, on condition that a minimum value or a maximum value of the input luma range is not a multiple of an initial average number ([2^AB/N]) of codewords per piece, the number of codeword for at least a first piece comprising the minimum value or for a last piece comprising the maximum value is adjusted such that all or a maximized of codewords of coding luma range are used to represent luma samples within the input luma range. In a first step 810, an average number of codewords per piece A_NBCW is determined. For instance, an initial average number of codewords per piece is obtained by dividing a total number of codewords 2^AB of the coding luma range by the initial number of pieces N. This initial average number of codewords per piece also corresponds to a length of a piece L = 2^AB/N. A mapping ratio is also computed by dividing the coding luma range 2^AB by the input luma range MAX-MIN. As the MAX value is lower than 2^AB and the MIN value is higher than or equal to 0, MAX-MIN is lower than 2^AB, and the mapping ratio is larger than 1. Then, an average number of codewords per piece is obtained as being an integer (either from rounding or truncation) value of the multiplication of the initial average number of codewords per piece by the mapping ratio. In a second step 820, the average number of codewords per piece A_NBCW is adjusted for at least a first or a last piece in the sub-range of pieces to optimize the number of codewords of the coding domain used by the signal. When the maximum value MAX of the input luma range is not a multiple of an initial average number of codewords per piece, some of the codewords of coding luma range may not be used to code the input signal. In that case, the number of codeword for the last piece (ie K-lth piece having a non-null histogram value) is adjusted to maximize the number of codewords of coding luma range used to represent the input luma range. Similarly, when the minimum value MIN of the input luma range is not a multiple of an initial average number of codewords per piece, the number of codeword for the first piece (ie first piece having a non-null histogram value) is adjusted. Various embodiments are described in the following that may be used in combination or independently.

[61] A first embodiment with activation of the LMCS function determined by comparing the histogram of the video with the 64/940 levels.

[62] As described above, the histogram of the luma component is calculated per piece. There are 16 pieces, each of them corresponding to 1/16 of the video range which means 64 levels for a 10-bit content. Besides, the LMCS function is activated if both Hist[0] and Hist[15] are below a given threshold. Since a captured content in limited range has its luma in between 64 and 940, Hist[0] and Hist[15] are supposed to be at 0. If the content has been processed before the histogram calculation, some samples may have values below 64 or above 940 but most of the time it corresponds to a very limited number of samples so most of the time for captured contents the LMCS activation is ON.

[63] But Hist[15] corresponds to sample values in between 960 and 1023.

[64] In the first embodiment we modify the way we calculate the histogram by modifying the range of video level which define Hist[15] and Hist[14] as follows:

• Instead of 896/960, Hist[14] counts the number of samples having a level between 896 and 940.

• Instead of 960/1023, Hist[15] counts the number of samples having a level between 940 and 1023.

[65] The test to activate the LMCS function remains the same by testing if Hist[0] and Hist[15] are below a threshold.

[66] A second embodiment with Number of CW per piece based on 64/940 values.

[67] In case of LMCS activation (Hist[0] and Hist[15] are below a threshold) the Number of CW (NBCW) per piece is calculated as described above with the third step. At this stage the number of CW is the same for each piece including the piece number 14. In the second embodiment it is proposed to calculate the CW repartition per piece taking into account the fact that there is no need to encode values between 940 and 960. In this case, the average value of CW per piece is calculated like this:

Average NBCW per piece = 1024 / (940 - 64) *64 = 74.81 Eq (l)

[68] In a first variant of this second embodiment, the 13 first pieces have a number of CW equal to the rounded value of the average NBCW (i.e. 75) and the 14^th piece has the remaining ones (i.e. 49 = 1024-13*75). FIG. 5 illustrates forward mapping LUT according to the first variant of the second embodiment, i.e with 75 CW per piece.

[69] In a second variant of this second embodiment, the 13 first pieces have a number of CW equal to the truncated value of the average NBCW (i.e. 74) and the 14^th piece has the same density of CW (i.e. 51) than the 13 first pieces. Equal density means the same ratio between the number of values to be coded by the number of CW associated. For the 14^th piece, the calculation is 74* (940-896)/64. FIG. 6 illustrates forward mapping LUT according to the second variant of the second embodiment, i.e with 74 CW per piece and equal density.

[70] In yet a third variant of this second embodiment, the 1023 CW are spread onto the 14 pieces with different number of CW per piece but still considering not having CW to encode values between 940 and 960.

[71] A third embodiment with Number of CW per piece based on MIN/MAX values.

[72] The third embodiment proposes to replace the fixed values of 64 and 940 of the second embodiment by the specific MIN and MAX values of the video. MIN is supposed, for a captured content, to be above than or equal to 64 and MAX below than or equal to 940. The equation (1) becomes now:

Average NBCW per piece = 1024 / (MAX - MIN) *64 = A_NBCW Eq(2 )

[73] If MIN = 64 and MAX = 940, this corresponds to the second embodiment.

[74] If MIN > 64 or/and MAX<940 then A_NBCW >74.81.

[75] FIG. 7 illustrates an example of forward mapping LUT according to a first and second variant of a third embodiment.

[76] In a first variant of this third embodiment, the 13 first pieces have a number of CW equal to the rounded value of the average NBCW and the 14^th piece has the remaining ones.

[77] In a second variant of this third embodiment, the 13 first pieces have a number of CW equal to the truncated value of the average NBCW and the 14^th piece has the same density of CW than the 13 first pieces.

[78] In yet a third variant of this third embodiment, the 1023 CW are spread onto the 14 pieces with different number of CW per piece but still considering not having CW to encode values below MIN and above MAX.

[79] According to yet other generalization of the second/third embodiments, Eq(2) can be generalized for a signal with bit-depth B, and for an LMCS function defined using N pieces, as follows:

Average NBCW per piece = (2^AB ) / (MAX - MIN) * (2^AB / N) = A_NBCW

[80] If N pieces are used, the range is split into pieces of length of L = 2^AB / N.

[81] If the signal of bit-depth B is of limited range, the signal goes (by definition of “limited range” of a SDR signal on 8 bits) from ( 16 * 2^A(B - 8) ) to ( 235 * 2^A(B - 8) ). The total used range is therefore not 2^AB, but ( 219 * 2^A(B - 8) ). Therefore there are only K = ( 219 * 2^A(B - 8) ) / L = ( 219 * 2^A(B - 8) ) / (2^AB / N ) = ( N * 219 / 2^A8 ) = ( N * 219 / 256 ) pieces to be considered for mapping calculation.

[82] In a first variant of this embodiment, the ( K - 1 ) first pieces have a number of CW equal to the rounded value of the average NBCW and the K^th piece has the remaining ones.

[83] In a second variant of this embodiment, the ( K - 1 ) first pieces have a number of CW equal to the truncated value of the average NBCW and the K'¹' piece has the same density of CW than the ( K - 1 ) first pieces.

[84] In yet a third variant of this embodiment, the (2^AB - 1) CW are spread onto the K pieces with different number of CW per piece but still considering not having CW to encode values below MIN and above MAX.

[85] A fourth embodiment with a number of pieces and a number of CW per piece based on MIN/MAX values.

[86] In this fourth embodiment compared to the third embodiment, the number of pieces and the number of CW per pieces are not fixed. They can be adapted to the MIN and MAX values in order for instance to have the same number of CW per piece and such that the product NB_piece * NB level per piece gives the lower than and nearest value to 1024.

[87] As an example, if MIN=80 and MAX=919 there are 840 levels as input of the forward mapping LUT. A first solution is to have 14 pieces of 60 video levels each. In the LMCS process, these 60 levels of each piece are mapped to 73 levels (14*73 =1022). Another solution could be to consider only 10 pieces of 84 levels mapped to 102 levels (10*102 = 1020). According to a particular variant, the number of pieces and the number of CW is determined such that the product NB_piece * NB level per piece gives the lower than and nearest value to 1024.

[88] In another variant embodiment, the 1024 CW are spread onto a variable number of pieces with different number of CW per piece but still considering not having CW to encode values below MIN and above MAX levels.

[89] A fifth embodiment with a number of pieces and number of CW per piece based on MIN/MAX values for LMCS function applied to chrominance components.

[90] In a fifth embodiment, the LMCS function is also applied to chrominance using the corresponding MIN/MAX values of each chrominance component to calculate corresponding number of CW. The same principle can be used as in the fourth embodiment. In one embodiment, for each chrominance component the number of pieces and the number of CW per pieces are not fixed. They can be adapted to the MIN and MAX values in order for instance to have the same number of CW per piece. [91] In another variant embodiment, the number of CW and the number of pieces are calculated separately for the two chrominance components.

[92] A sixth embodiment performing the LMCS function determination at both encoder and decoder side.

[93] In a sixth embodiment, the MIN and MAX values of the signal are signaled by the encoder to the decoder, and the LMCS function is deduced by the encoder and by the decoder from those values, based on the preceding embodiments.

[94] At decoder, after entropy decoding, when LMCS is enabled (the flag related to enabling/disabling LMCS has been parsed and indicates that LMCS is enabled), the MIN and MAX values of the samples are parsed, as well as the number of pieces of the LMCS function. Other parameters may also be parsed, such as the length of each piece when the range is not equally divided in piece of uniform length. Then the decoder performs a new step for computing the LMCS function as described above.

[95] Various methods are described herein, and each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and/or use of specific steps and/or actions may be modified or combined. Additionally, terms such as “first”, “second”, etc. may be used in various embodiments to modify an element, component, step, operation, etc., for example, a “first decoding” and a “second decoding”. Use of such terms does not imply an ordering to the modified operations unless specifically required. So, in this example, the first decoding need not be performed before the second decoding, and may occur, for example, before, during, or in an overlapping time period with the second decoding.

[96] Various methods and other aspects described in this application can be used to modify modules, for example, the motion compensation module (270) of a video encoder 200 as shown in FIG. 2. Moreover, the present aspects are not limited to ECM and VVC, and can be applied, for example, to other standards and recommendations, and extensions of any such standards and recommendations. Unless indicated otherwise, or technically precluded, the aspects described in this application can be used individually or in combination.

[97] Various numeric values are used in the present application. The specific values are for example purposes and the aspects described are not limited to these specific values. [98] Various implementations involve decoding. “Decoding,” as used in this application, may encompass all or part of the processes performed, for example, on a received encoded sequence in order to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or generally to the broader decoding process will be clear based on the context of the specific descriptions and is believed to be well understood by those skilled in the art.

[99] Various implementations involve encoding or decoding. In an analogous way to the above discussion about “decoding”, “encoding” as used in this application may encompass all or part of the processes performed, for example, on an input video sequence in order to produce an encoded bitstream.

[100] Note that the syntax elements as used herein are descriptive terms. As such, they do not preclude the use of other syntax element names.

[101] The implementations and aspects described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed may also be implemented in other forms (for example, an apparatus or program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, for example, computers, cell phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.

[102] Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment. [103] Additionally, this application may refer to “determining” various pieces of information. Determining the information may include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

[104] Further, this application may refer to “accessing” various pieces of information. Accessing the information may include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[105] Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information may include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

[106] It is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of’, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.

[107] Also, as used herein, the word “signal” refers to, among other things, indicating something to a corresponding decoder. For example, in certain embodiments the encoder signals a quantization matrix for de-quantization. In this way, in an embodiment the same parameter is used at both the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter. Conversely, if the decoder already has the particular parameter as well as others, then signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter. By avoiding transmission of any actual functions, a bit savings is realized in various embodiments. It is to be appreciated that signaling can be accomplished in a variety of ways. For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun.

[108] As will be evident to one of ordinary skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry the bitstream of a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[109] We describe a number of embodiments. Features of these embodiments can be provided alone or in any combination, across various claim categories and types. Further, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types.

Claims

1. A method comprising: determining a luma forward mapping function applied to a prediction of luma samples that maps codewords used to represent luma samples within an input luma range into a coding luma range by reallocating a total number of codewords of coding luma range into a fewer number of pieces than an initial number of pieces, a piece being a division of the coding luma range by an initial number of pieces being an integer; wherein on condition that a minimum value or a maximum value of the input luma range is not a multiple of an initial average number of codewords per piece, adjusting a number of codeword for a first piece comprising the minimum value or for a last piece comprising the maximum value such that an optimized number of codewords of coding luma range are used to represent luma samples within the input luma range; and encoding a video by applying the luma forward mapping function to a prediction of the luma samples of the video.

2. A method comprising: obtaining a minimum value and a maximum value of an input luma range; determining a luma forward mapping function applied to a prediction of luma samples that maps codewords used to represent luma samples within the input luma range ([MIN, MAX]) into a coding luma range ([0, 2^AB]) by reallocating a total number (2^AB) of codewords of the coding luma range into a fewer number of pieces than an initial number of pieces, a piece being a division of the coding luma range by an initial number of pieces being an integer; wherein on condition that a minimum value or a maximum value of the input luma range is not a multiple of an initial average number ([2^AB/N]) of codewords per piece, adjusting a number of codeword for a first piece comprising the minimum value or for a last piece comprising the maximum value such that an optimized number of codewords of coding luma range are used to represent luma samples within the input luma range; decoding a video by at least applying an inverse function of the luma forward mapping function to a reconstruction of the luma samples of the video.

3. The method of any of claims 1 or 2, further comprising: determining the initial average number of codewords per piece by dividing a total number of codewords of the coding luma range by the initial number of pieces, determining a mapping ratio by dividing the coding luma range by the input luma range; determining an average number of codewords per piece by obtaining an integer value of the multiplication of the initial average number of codewords per piece by the mapping ratio; and on condition that the maximum value of the input luma range is not a multiple of an initial average number of codewords per piece, adjusting a number of codeword for a last piece comprising the maximum value.

4. The method of claim 3, wherein the average number of codewords per piece is a rounded integer value of the multiplication of the initial average number of codewords per piece by the mapping ratio, wherein the fewer number of pieces have a number of codewords equal to the average number of codewords per piece and wherein the last piece has an adjusted number of codewords equal to a remaining number of codewords.

5. The method of claim 3, wherein the average number of codewords per piece is a truncated integer value of the multiplication of the initial average number of codewords per piece by the mapping ratio, wherein the fewer number of pieces have a number of codewords equal to the average number of codewords per piece and wherein the last piece has an adjusted number of codewords with a same density of codeword than previous pieces.

6. The method of any of claims 1 or 2, wherein the total number of codewords of the coding luma range is reallocated into a fewer number of pieces, each piece having an adjusted number of codewords with no codeword below than the minimum value and no codeword above than the maximum value.

7. The method of any of claims 1 or 2 further comprising: determining the initial average number of codewords per piece by dividing a total number of codewords of the coding luma range into the initial number of pieces, determining a mapping ratio by dividing the coding luma range by input luma range; determining an average number of codewords per piece by obtaining an integer value of the multiplication of the initial average number L of codewords per piece by the mapping ratio; on condition that a minimum value of the input luma range is not a multiple of an initial average number of codewords per piece, adjusting a number of codeword for a first piece comprising the minimum value.

8. The method of claim 7, wherein the average number of codewords per piece is a rounded integer value of the multiplication of the initial average number of codewords per piece by the mapping ratio, wherein the fewer number of pieces have a number of codewords equal to the average number of codewords per piece and wherein the first piece has an adjusted number of codewords equal to a remaining number of codewords.

9. The method of claim 7, wherein the average number of codewords per piece is a truncated integer value of the multiplication of the initial average number of codewords per piece by the mapping ratio, wherein the fewer number of pieces have a number of codewords equal to the average number of codewords per piece and wherein the first piece has an adjusted number of codewords with a same density of codeword than previous pieces.

10. The method of any of claims 1 to 9, wherein the coding luma range is determined by a bit-depth used for coding luma samples.

11. The method of any of claims 1 to 10, wherein the minimum value of the input luma range is an integer value equal to or higher than 0 and wherein the maximum value of the input luma range is an integer value lower than two raised at a power of a bit-depth of the coding luma range.

12. The method of any of claims 1 to 11, wherein the minimum value of the input luma range is 64, wherein the maximum value of the input luma range is 940 and wherein a bit-depth of the coding luma range is 10.

13. The method of any of claims 1 to 12, further comprising adjusting the initial number of pieces such that a product of the initial number of pieces by the number of codewords per pieces results in a lower value than and nearest value to the coding luma range.

14. The method of any of claims 1 to 13, further comprising determining a chroma forward mapping function applied to a prediction of chroma samples that maps codewords used to represent chroma samples within an input chroma range into a coding chroma range by reallocating a total number of codewords of the coding chroma range into a fewer number of pieces than an initial number of pieces as performed for the luma samples.

15. An apparatus, comprising one or more processors, wherein the one or more processors are configured to perform the method of any of claims 1-14.

16. A signal comprising video data, formed by performing the method of any one of claim 1 and claims 3-14.

17. A computer readable storage medium having stored thereon instructions for video encoding according to the method of any one of claim 1 and claims 3-14.