EP4666581A1 - Local reshaping using tensor-product b-spline with coordinates wide view video - Google Patents
Local reshaping using tensor-product b-spline with coordinates wide view videoInfo
- Publication number
- EP4666581A1 EP4666581A1 EP24711955.5A EP24711955A EP4666581A1 EP 4666581 A1 EP4666581 A1 EP 4666581A1 EP 24711955 A EP24711955 A EP 24711955A EP 4666581 A1 EP4666581 A1 EP 4666581A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- wide view
- view images
- images
- reshaping
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Definitions
- DR dynamic range
- HVS human visual system
- DR may relate to a capability of the human visual system (HVS) to perceive a range of intensity (e.g., luminance, luma) in an image, e.g., from darkest blacks (darks) to brightest whites (highlights).
- DR relates to a “scene-referred” intensity.
- DR may also relate to the ability of a display device to adequately or approximately render an intensity range of a particular breadth. In this sense, DR relates to a “display-referred” intensity.
- HDR high dynamic range
- HVS human visual system
- EDR enhanced dynamic range
- VDR visual dynamic range
- HVS human visual system
- n ⁇ 8 e.g., color 24-bit JPEG images
- images where n > 8 may be considered images of enhanced dynamic range.
- a reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between color values (e.g., luminance) of an input video signal to output screen color values (e.g., screen luminance) produced by the display.
- information about its EOTF may be embedded in the bitstream as (image) metadata.
- metadata herein relates to any auxiliary information transmitted as part of the coded bitstream and assists a decoder to render a decoded image.
- Such metadata may include, but are not limited to, color space or gamut information, reference display parameters, and auxiliary signal parameters, as those described herein.
- PQ perceptual luminance amplitude quantization.
- the human visual system responds to increasing light levels in a very nonlinear way. A human’s ability to see a stimulus is affected by the luminance of that stimulus, the size of the stimulus, the spatial frequencies making up the stimulus, and the luminance level that the eyes have adapted to at the particular moment one is viewing the stimulus.
- a perceptual quantizer function maps linear input gray levels to output gray levels that better match the contrast sensitivity thresholds in the human visual system.
- SMPTE High Dynamic Range EOTF of Mastering Reference Displays
- LDR lower dynamic range
- SDR standard dynamic range
- EDR content may be displayed on EDR displays that support higher dynamic ranges (e.g., from 1,000 nits to 5,000 nits or more). Such displays may be defined using alternative EOTFs that support high luminance capability (e.g., 0 to 10,000 or more nits).
- An example of such an EOTF is defined in SMPTE 2084 and Rec. ITU-R BT.2100, “Image parameter values for high dynamic range television for use in production and international programme exchange,” (06/2017).
- improved techniques for generating high quality video content data with high dynamic range, high local contrast and vivid color are desired.
- FIG.1A and FIG.1B illustrate example encoder side and decoder side architectures
- FIG.2A illustrates an example process or method for reference base layer image generation
- FIG.2B illustrates example overlapped and non-overlapped patches
- FIG.2C illustrates example patch based forward reshaping mapping generation
- FIG.2D illustrates an example method or process flow for patch fusion
- FIG.2E through FIG.2I illustrates example process flows for optimizing reshaping mappings
- FIG.3A illustrates example distributions of block based standard deviations
- FIG.3B illustrates an example weighting map
- FIG.3C illustrates example operations in connection with patch-based weighting maps
- FIG.4A and FIG.4B illustrate example process
- User-generated and professional wide view (e.g., greater than a viewer’s entire vision field, greater than a user’s field of view, greater than 90-degrees, 360-degree, etc.) content is already being distributed using popular content sharing platforms.
- relatively wide view video such as up to 360-degree HDR video.
- the wide view video often has much larger image dimensions, contains much richer spatial information from up to all front and/or rear viewing angles not necessarily limited to only within an individual viewer’s vision field, and includes a much higher pixel count in each image.
- the wide view video can exhibit much higher dynamic ranges and much wider color gamuts as well as much larger spatial variations of dynamic ranges and color gamuts in different spatial locations of a wide view scene/image, and therefore present a much greater challenge in terms of supporting a widely diverse range of local luminances and color distributions in a relatively wide field of view (FOV) as compared with the 2D SDR or HDR video.
- Many existing video codecs that support or adopt a global reshaping method in processing the 2D SDR or HDR video may be ill equipped to handle much higher dynamic range (HDR) and wider color gamut (WCG) in wide view video content in a compression efficient manner.
- Fused patch-based local reshaping techniques as described herein can be implemented to support wide view video efficiently and effectively, and to resolve issues and handle challenges associated with the wide view video.
- a reference base layer (BL) signal can be generated from original (input or source) wide view video using a fused patch-based algorithm/method to address or preserve local dynamic range and color distribution.
- Some or all of these techniques can operate with wide view video represented in an Equi-Rectangular Projection (ERP) format or other formats different from the ERP forma.
- ERP Equi-Rectangular Projection
- intermediate reference BL video content may be first generated patch-wisely to provide sufficient codewords for preventing banding visual artifacts and preserving colors or color precisions in various local regions.
- This (final fused) reference BL signal can be used to increase video compression efficiency globally and preserve HDR/WCG properties of the original or source wide view video locally in a reconstructed HDR signal, in order to help prevent highlight/dark area clipping, alleviate banding visual artifacts and achieve color fidelity in the reconstructed HDR signal.
- a relatively compression- efficient and revertible forward reshaped BL signal can be generated by available video codecs on the encoder side using a forward reshaping function to approximate the reference BL signal.
- the reconstructed HDR signal can be generated or constructed by available video codecs on the decoder side from the forward reshaped BL signal using a backward reshaping function corresponding to the forward reshaping function.
- highly varying characteristics of local dynamic range and local color distribution in wide view HDR/WCG video data can be efficiently and effectively addressed or preserved Tensor-Product B-Spline with Coordinates (TPB with Coordinates or TPBC).
- TPB is a tool to model cross-channel complex mapping or reshaping functions.
- B-splines or basis splines can be used as functions to fit a given one dimensional curve using polynomial functions with continuity constraints at knot points.
- multiple B-spline functions can be fused together by (tensor) multiplication to approximate, estimate or fit higher dimensional curves while maintaining smooth connectivity at knot points.
- Example TPB reshaping functions are described in U.S. Provisional Application Ser. No.62/908,770, titled “TENSOR- PRODUCT B-SPLINE PREDICTOR,” filed on October 1, 2019, which are incorporated by reference in its entirety as if fully set forth herein.
- Example BESA algorithm/method can be found in U.S. Provisional Patent Application Ser. No. 63/013,063, “Reshaping functions for HDR imaging with continuity and reversibility constraints,” filed on April 21, 2020; U.S. Provisional Patent Application Ser. No. 63/013,807, “Iterative optimization of reshaping functions in single-layer HDR image codec,” filed on April 22, 2020; PCT Application Ser. No.
- TPB or TPBC as described herein can be implemented with a Backward Error Subtraction Algorithm (BESA) to reach or achieve a relatively high degree of revertability between a pair of corresponding forward and backward reshaping functions.
- BESA Backward Error Subtraction Algorithm
- algorithms or methods as described herein can be developed or implemented in a manner that reduces memory footage and computational load.
- a three-stage optimization may be implemented in an adaptive algorithm that applies or uses incremental datasets to minimize prediction errors.
- a relatively small bit depth domain (e.g., SDR, etc.) image may be transformed into a relatively large bit depth domain (e.g., HDR, WCG, etc.) image with a relatively small model, for example, built with a relatively small set of pixels selected from among all pixels represented in the images.
- Optimized operational parameters derived with the adaptive algorithm can be used to generate reconstructed wide view HDR/WCG video content that is visually lossless to the original or source wide view HDR/WCG video content.
- One or more reference wide view images of a first domain are generated from one or more source wide view images of a second domain.
- a forward reshaping mapping is generated to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain.
- a backward reshaping mapping is generated to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain.
- Each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data.
- the one or more forward reshaped wide view images and corresponding image metadata are encoded into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images.
- the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings.
- the corresponding image metadata includes operational parameters specifying the backward reshaping mappings, but does not include operational parameters specifying the forward reshaping mappings which are applied at the encoder side only.
- Example embodiments described herein relate to decoding coded packets relating to reconstructed images.
- One or more forward reshaped wide view images of a first domain and corresponding image metadata are decoded from a bitstream.
- the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping.
- the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping.
- One or more reconstructed wide view images of the second domain are generated from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping.
- One or more display images are generated from the one or more reconstructed wide view images.
- the one or more display images are rendered on an image display.
- FIG.1A illustrates an example encoder side architecture that may be implemented with one or more computing devices.
- the encoder side architecture may include a plurality of processing blocks or components to receive input or source wide view HDR/WCG images (or image frames); to use the received wide view HDR/WCG images/frames to generate reference BL images/frames; to generate forward reshaping functions to forward reshape the received wide view HDR/WCG images/frames to generate forward reshaped BL images approximating the reference BL images/frames; to encode the forward reshaped BL images along with image metadata specifying backward reshaping functions corresponding to the forward reshaping functions in a coded bitstream; to deliver or transmit the coded bitstream from an upstream device (e.g., some or all of the one or more computing devices implementing the encoder side, etc.) to a downstream device over data communication links or paths between the upstream device and the downstream device; etc.
- an upstream device e.g., some or all of the one or more computing devices implementing the encoder side, etc.
- some or all of these foregoing operations may be performed by the one or more computing devices at the encoder side on line in real time, for example while the upstream device communicates the bitstream or a portion thereof to the downstream device.
- at least some of the foregoing operations may be performed by the one or more computing devices at the encoder side off line in non-real time, for example before the upstream device communicates any portion of the bitstream to the downstream device.
- the encoder side architecture may include a processing block/component for reference BL generation, which takes the input or source wide view HDR/WCG images (or image frames) as inputs and use the input wide view HDR/WCG images/frames to generate or derive corresponding BL images to serve as reference for forward reshaped BL images to approximate.
- a processing block/component for reference BL generation takes the input or source wide view HDR/WCG images (or image frames) as inputs and use the input wide view HDR/WCG images/frames to generate or derive corresponding BL images to serve as reference for forward reshaped BL images to approximate.
- the forward reshaped BL images can be encoded in the coded bitstream with relatively high coding efficiency and subsequently decoded and used by a recipient device to generate reconstructed wide view HDR/WCG images approximate the input or source wide view HDR/WCG in a manner that avoids banding visual artifacts and preserves colors in each local (spatial) region of some or all local (spatial) regions in the reconstructed wide view HDR/WCG images corresponding to some or all local (spatial) regions represented in the input or source wide view HDR/WCG.
- the encoder side architecture may include a processing block/component for forward/backward TPBC optimization, which generates forward reshaping functions and backward reshaping functions corresponding to the forward reshaping functions using the input wide view HDR/WCG images and the reference BL images as input.
- the forward/backward TPBC optimization (1) minimizes differences between the reference BL images and forward reshaped BL images (such that the banding prevention and color preservation can be maintained as well as underlying video compression efficiency can be achieved or improved), and (2) minimizes distortions between the reconstructed wide view HDR/WCG images and the original wide view HDR/WCG images.
- the encoder side architecture may include a processing block/component for forward reshaping, which uses the forward reshaping functions to forward reshape the input wide view HDR/WCG images into the forward reshaped BL images that approximate the reference BL images.
- the encoder side architecture may include a processing block/component for metadata generation, which generates image metadata or portions thereof to define or specify the backward reshaping functions.
- the backward reshaping functions can be used, for example by a recipient decoding device of the image metadata, to generate the reconstructed wide view HDR/WCG images that approximate the input wide view HDR/WCG images.
- the encoder side architecture may include a processing block/component for video compression, which compresses or encodes the forward reshaped BL images into the (video) coded bitstream along with the image metadata generated by the metadata generation.
- a processing block/component for video compression which compresses or encodes the forward reshaped BL images into the (video) coded bitstream along with the image metadata generated by the metadata generation.
- available video codecs that support compressing or encoding relatively narrow view 2D video content can be used or enhanced to implement the processing block/component for video compression of FIG.1A.
- FIG.1B illustrates an example decoder side architecture that may be implemented with one or more computing devices.
- the decoder side architecture may include a plurality of processing blocks or components to receive a coded bitstream encoded with forward reshaped wide view BL images along with image metadata specifying backward reshaping functions corresponding to forward reshaping functions used to generate the forward reshaped wide view BL images; to apply the backward reshaping functions to the forward reshaped wide view BL images to generate corresponding reconstructed or backward reshaped wide view HDR/WCG images; to render display images derived from the reconstructed or reshaped wide view HDR/WCG images on image display(s); etc.
- the decoder side architecture may include a processing block/component for metadata extraction, which extracts the image metadata from (image metadata container or data fields carried or included in) the coded bitstream.
- the decoder side architecture may include a processing block/component for video decompression, which decodes or decompresses the forward reshaped wide view BL images from (encoded video data carried or included in) the coded bitstream.
- the decoder side architecture may include a processing block/component for backward reshaping, which backward reshapes – or applies the backward reshaping functions to – the forward reshaped wide view BL images (denoted as “Reshaped BL”) into the reconstructed or reshaped wide view HDR/WCG images (or video).
- Reshaped BL forward reshaped wide view BL images
- the first of the above mentioned two processing blocks/components, or the reference BL generation includes extracting overlapped patches from image data of each input or source wide view HDR/WCG image in each color channel of an input or source color space in which the input or source wide view HDR/WCG image is represented.
- the input or source color space may include a luma channel and two chroma channels.
- BLKSTD block- based standard deviations based method/algorithm
- Example BLKSTD methods/algorithms are described in U.S. Patent No. 10,032,262, issued on 24 July 2018; U.S. Patent No.10,223,774, issued on 5 March 2019, the entire contents of all of which are hereby incorporated by reference as if fully set forth herein.
- a luma-chroma channel energy ratio may be determined and applied to generate to generate chroma image data of the reference BL image patches from corresponding image patches of the original (input or source) wide view HDR/WCG images/frames. After determining or generating the luma and chroma image data of the reference BL image patches, these patches can be fused together, for example with a predefined weight function, into a reference wide view BL image in which visual patch boundary artifacts are reduced, removed or otherwise alleviated.
- the second of the above mentioned two processing blocks/components, or the forward/backward TPBC optimization includes determining an optimized spatial positional (information or data) encoding for the input or source wide view HDR/WCG image. More specifically, optimized operational parameters for forward reshaping TPBC operations are generated and used on the encoder side to forward reshaping the input or source wide view HDR/WCG image into a forward reshaped wide view BL image approximating the reference wide view BL image. Optimized operational parameters for backward reshaping TPBC operations are generated and used on the decoder side to backward reshaping the forward reshaped wide view BL image into a reconstructed or reshaped wide view HDR/WCG image.
- a three-stage optimization process may be used to generate these optimized operational parameters for the forward and backward TPBC operations.
- TPBC operations can be performed on an increased-size dataset using the previously mentioned BESA method or algorithm.
- Reference Base Layer Generation [0043]
- global reshaping may be applied to reshape an input image with a global reshaping function applied to image data in all spatial regions of the input image into a reshaped image at a relatively high compression efficiency.
- Such global reshaping techniques or functions may work well with images of a relatively narrow field of view such as 2D images.
- a wide view HDR/WCG image/picture covers a much wider field of view as compared with an 2D image of a relatively narrow field of view.
- wide view HDR/WCG video or images/pictures exhibit relatively great diversity in different local regions within the same image/picture.
- dynamic ranges and color distributions in different spatial regions of the same wide view HDR/WCG image may exhibit different characteristics. Codewords needed to avoid banding visual artifacts in one particular luminance range in one particular local area or spatial region of the wide view HDR/WCG image can be quite different from codewords needed to avoid banding visual artifacts in another particular luminance range in another particular local area or spatial region of the same wide view HDR/WCG image.
- Global reshaping would subject all these spatial regions with diversely different luminance ranges to the same global reshaping mapping or function and cannot adequately or efficiently handle varying needs in these different spatial regions for different sets of codewords in reshaping operations.
- local reshaping operations/methods may be implemented, performed or used to apply different local reshaping functions or mappings to different spatial regions of the same wide view image.
- These local reshaping functions or mappings such as TPBC based reshaping functions or mappings may be determined or generated to meet respective needs of the different spatial regions of the same wide view image.
- FIG.2A illustrates an example two-stage process or method for reference BL image generation, which may be implemented, for example, by a video encoder or processing blocks/components therein. This two-stage process or method can be used to help tackle or perform local reshaping operations by way of creating a reference wide view base layer images (or video signal) from an input or source wide view HDR/WCG images (or video signal).
- the first stage is implemented with a processing block/component for LUT generation, which generates a respective lookup table (LUT) in each local patch – among a plurality of local (HDR/WCG) patches identified from an input or source wide view HDR/WCG image in the input or source wide view HDR/WCG images (or video signal) – using the BLKSTD method/algorithm.
- Respective LUTs generated for the plurality of local (HDR/WCG) patches can be different for different local or spatial patches in the same input or source wide view HDR/WCG image.
- LUTs can be applied to the plurality of local (HDR/WCG) patches in the input or source wide view HDR/WCG image into corresponding locally forward reshaped (BL) patches.
- the second stage is implemented with a processing block/component for fusion, which fuses the locally forward reshaped (BL) patches to smooth out boundary or discontinuity artifacts along patch boundaries to generate a corresponding reference wide view BL image in the reference wide view base layer images (or video signal).
- a respective patch based LUT (e.g., one-dimensional LUT or 1D- LUT for the luma channel, etc.) representing a local forward reshaping function for a local patch can be generated for each local patch in a plurality of local patches in the input or source wide view HDR/WCG image.
- These local patches may represent relatively small areas or spatial regions in the input or source wide view HDR/WCG image and may be used to generate patch based LUTs.
- a group of T (input or source wide view HDR/WCG) image/frames within the same scene may be processed or partitioned using the same set of local patches.
- superscript C may be Cb or Cr to denote a Cb or Cr chroma channel in an input YCbCr color space or domain.
- superscript C may be P or T to denote a P or T channel in an input IPT (e.g., IPTPQc2, etc.) color space or domain.
- IPT e.g., IPTPQc2, etc.
- An input or source wide view HDR/WCG image, or the t-th image/frame may be partitioned into a plurality of non-overlapped local patches each of which has a patch size ⁇ ⁇ ⁇ ⁇ ⁇ , where ⁇ ⁇ represents an integer greater than two (2).
- the luma component may include the corresponding luma pixel values number of pixels ⁇ ⁇ ⁇ ⁇ ⁇ if the k-th non-overlapped patch is an interior patch in the t-th image/frame. If the k-th non-overlapped patch is next to or on a border/boundary of the image/frame, the patch may have fewer luma pixel values than the full size of ⁇ ⁇ ⁇ ⁇ ⁇ .
- a non-luma component e.g., Cb or Cr chroma in an YCbCr color space, P or T in an IPT color space – or corresponding non-luma pixel values of the k- th non-overlapped patch in the t-th image/frame as as ⁇ ⁇ ⁇ , , ⁇ ⁇ .
- the input or source wide view is represented in a 4:2:0 color space sampling format.
- [0057] Denote the luma and non-luma pixel values at a (e.g., relative to the local patch, etc.) pixel location (m, n) in the k-th non-overlapped patch of the t-th image/frame, respectively, as ⁇ ⁇ ⁇ , ⁇ ( ⁇ , ⁇ ) and ⁇ ⁇ ⁇ , ⁇ ( ⁇ , ⁇ ), where m represents a (e.g., relative to the top left pixel or pixel location of the local patch, etc.) row index of the pixel or pixel location; and n represents a (e.g., relative to the top left pixel or pixel location of the local patch, etc.) column index of the pixel or pixel location.
- a straightforward solution may be to construct respective patch-based 1D- LUTs for the non-overlapped patches in the input or source wide view HDR/WCG image and to generate a reference wide view base layer image by applying the BLKSTD method/algorithm to each non-overlapped patch.
- relatively significant boundary artifacts can be observed occurring along the patch boundaries.
- Those high-frequency visual artifacts can degrade reshaping accuracy, reduce video coding efficiency, and propagate those visual artifacts to a reconstructed or backward reshaped wide view HDR/WCG image.
- an overlapped patch based method or algorithm as described herein may be implemented or performed to use overlapped patches to collect or gather additional neighboring patch data in the LUT generation and to further use overlapped patches to enable or provide a relatively smooth transition along patch boundaries in the reference wide view BL image generation.
- overlapped patches for the LUT generation may be different from overlapped patches for the BL image generation.
- the overlapped patches for the LUT generation may be of different sizes from sizes of overlapped patches for the BL image generation.
- the t-th image may be partitioned into a plurality of non-overlapped local patches, for example arranged in a two-dimensional spatial array.
- Two corresponding overlapped local patches may be defined or determined for each non-overlapped local patch in the plurality of non-overlapped local patches.
- Pixel value statistics can be collected in the first of the two corresponding overlapped local patches to construct a local patch-based LUT, which can be applied to the second of the two corresponding overlapped local patches to help generate a patch- based portion of the reference wide view base layer image.
- the first of the two corresponding local patches – assuming they are located in the interior of the t-th image – is represented by a first square or a first luma patch size ⁇ ⁇ ⁇ ⁇ ⁇ . This first overlapped local patch may be used for the LUT generation.
- the second of the two corresponding local patches is represented by a second square or a second luma patch size ⁇ ⁇ ⁇ ⁇ ⁇ . This second overlapped local patch may be used for the BL image generation.
- the patch sizes of the non-overlapped and overlapped local patches may satisfy an inequality as follows: ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ (1)
- the total number of the non-overlapped local patches may be the same as the total number of the overlapped local patches for each of the LUT generation and the BL image generation and is given or specified by ⁇ ⁇ .
- the four corners of the overlapped luma local patch for the LUT generation, ⁇ ⁇ ⁇ , , ⁇ ⁇ has a pixel distance ( ⁇ ⁇ ) horizontally and vertically to the corresponding four corners of the non-overlapped luma local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ .
- the four corners of the overlapped luma local patch for the BL image generation, ⁇ ⁇ ⁇ , , ⁇ ⁇ have a pixel distance ( ⁇ ⁇ ⁇ ) horizontally and vertically to the corresponding four corners of ⁇ ⁇ ⁇ , , ⁇ ⁇ .
- the four corners of ⁇ ⁇ ⁇ , , ⁇ ⁇ has pixel distance ( ⁇ ⁇ ) horizontally and vertically to the corresponding four corners of the non-overlapped luma local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ .
- FIG.2C illustrates an example method or process for patch based forward reshaping function/mapping generation, which may be implemented, for example, by a video encoder or processing blocks/components therein.
- the process flow can be implemented or performed to generate patch-based or patch-specific forward reshaping functions/mappings such as LUTs for forward reshaping patches extracted from input or source wide view HDR/WCG image(s).
- the process flow comprises a processing block/component for patch extraction, which extracts overlapped HDR/WCG local patches for LUT generation at a plurality of different local patch locations from input or source wide view HDR/WCG image(s).
- the process flow further comprises a processing block/component for LUT generation, which generates respective LUTs for forward reshaping HDR/WCG local patches at the plurality of different local patch locations in the input or source wide view HDR/WCG image(s) into corresponding BL local patches.
- the corresponding BL local patches may be combined or fused into reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s).
- a respective LUT such as a specific 1D mapping LUT can be constructed to prevent or reduce banding artifacts and preserve colors or color precision in each non-overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ of an input or source wide view HDR/WCG image.
- a corresponding overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ can be used instead of the non-overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ to generate the LUT.
- bit depth e.g., the number of bits used to encode a luma or non-luma pixel value or codeword, etc.
- bit depth e.g., the number of bits used to encode a luma or non-luma pixel value or codeword, etc.
- needed codewords to avoid or prevent banding artifacts may be estimated or computed with a function derived with a block-based standard deviation (BLKSTD) method, which measures block-based standard deviations (BLKSTDs) in multiple mutually exclusive luminance sub-ranges that make up the entire luminance range in the overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ of the input or source wide view HDR/WCG image. More specifically, in each luminance sub-range, pixels – in the overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ – whose pixel values or codewords fall into this considered luminance sub-range may be collected.
- BLKSTD block-based standard deviation
- the input or source wide view HDR/WCG image containing the overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ belongs to a sequence of (e.g., consecutive, sequential, etc.) input or source wide view HDR/WCG in a video.
- pixels or pixel values (or codewords) represented in corresponding overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ of a group of one or more input or source wide view HDR/WCG images within a scene may be collected across all images in the group and used to derive average BLKSTDs for different luminance sub-ranges that make up the entire luminance range represented in the corresponding overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ of the group of one or more input or source wide view HDR/WCG images.
- Respective BLKSTDs in all the luminance sub-range can be used to construct a BLKSTD function that estimates or computes respective needed codewords for all the luminance sub-range represented in the overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ .
- the LUT such as a 1D-LUT can be built using the BLKSTD function constructed from all the average BLKSTDs in all the luminance sub-ranges.
- the processing block or component for the LUT generation may include optimizing the LUT derived from the BLKSTDs by identifying and re-using previously unused codewords in a codeword space and smoothing filtering to make the LUT or 1D-LUT relatively smooth or less discontinuous for the purpose of avoiding or reducing video compression artifacts.
- One or both of the input HDR/WCG pixel values (or codewords) and the output base layer pixel values (or codewords) represented in the LUT may be normalized into a normalized value range [01].
- the LUT – e.g., optimized and smoothened LUT – may be used as a forward reshaping mapping or function to convert the pixel values or codewords encoded in corresponding overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ of the input or source wide view HDR image – or in the group of the one or more input or source wide view HDR images within the same scene – to corresponding base layer pixel values or codewords in corresponding overlapped BL local patches.
- the overlapped BL local patches may be used to construct corresponding reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s).
- the maximum mapped or BL value may be set or normalized to the maximum value of an applicable normalized value range such as 1.
- the minimum mapped or BL value may be set or normalized to the minimum value of the applicable normalized value range such as 0.
- a codeword space that includes all available codewords comprises 2 %/ codewords.
- the LUT may contain 2 %/ entries each of which maps or forward reshapes a respective input or source pixel value or codeword into a mapped (e.g., forward reshaped, etc.) BL pixel value or codeword as output.
- the output value range may also be in [01], as the output BL signal is encoded with codewords or pixel values of ! # ( ⁇ !
- FIG.3A illustrates two example distributions of BLKSTDs in two local patches, respectively, of the same input or source wide view HDR/WCG images. As illustrated, the distributions of BLKSTDs in different local patches of the same image can be widely different. Global reshaping under other approaches would significantly degrade reshaping operations and would likely fail to prevent banding artifacts and negatively impact video compression performance. In contrast, local reshaping as described herein applies different reshaping functions/mappings in different local patches even if they are from the same image.
- the local reshaping operations can be used to effectively prevent or reduce banding artifacts as well as improve video compression performance.
- the same LUT or the 1D-LUT may be applied to – for example to forward reshape luma component or pixel values/codewords in – each corresponding local patch of some or all of (e.g., a group of, consecutive, sequential, with different frame indexes, etc.) images with the same scene.
- the LUT or 1D-LUT construction may be performed with respect to all like local patches at the same patch location in some or all images within the scene.
- 5 0,1, ... . , .
- the input values and/or the output values can be a range such as [01].
- respective LUTs or 1D-LUTs constructed to represent non-linear forward reshaping functions/mappings using the BLKSTD method or algorithm can be applied to forward reshape input luma pixel values or codewords in local patches at some or all local patches of input or source wide view HDR/WCG image(s).
- linear forward reshaping functions/mappings – such as first order polynomials, which can also be represented by LUTs such as 1D- LUTs – can be applied to forward reshape input non-luma pixel values or codewords in local patches at some or all local patches of input or source wide view HDR/WCG image(s).
- LUTs such as 1D- LUTs –
- 5 0,1, ... . , .
- a LUT or 1D-LUT for mapping or forward reshaping chroma values may be a relatively simple first order polynomial or a first order function with a scaling factor and an offset.
- the offset may be used to shift the center of a (e.g., valid, etc.) codeword range or sub-range to the middle of all codewords (for coding efficiency) in the codeword range or sub-range.
- the slope or scaling factor may be chosen using an energy ratio, which is defined as a ratio specified with the previously determined maximum and minimum values in the luma and non-luma channels.
- FIG.2D illustrates an example method or process flow for patch fusion, which may be implemented, for example, by a video encoder or processing blocks/components therein. The process flow can be implemented or performed to fuse overlapped patches together to generate reference wide view base layer image(s).
- the process flow comprises a processing block/component for patch extraction, which extracts overlapped HDR/WCG local patches for reference generation at a plurality of different local patch locations from input or source wide view HDR/WCG image(s).
- the process flow further comprises a processing block/component for applying LUT for reference (signal or image) generation, which applies respective LUTs to forward reshape HDR/WCG local patches for reference generation at the plurality of different local patch locations in the input or source wide view HDR/WCG image(s) into corresponding BL local patches for reference generation.
- the process flow also comprises a processing block/component for weight map (for patches) creation, which creates a weight map for each of the BL local patches.
- the process flow comprises a processing block/component for fusion, which fuses the BL local patches using respective weight maps for the BL local patches into overall reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s).
- the LUTs can be first applied to corresponding overlapped local patches to generate forward reshaped BL local patches. Fusion operations may be performed on the forward reshaped local patches in the luma and chroma channels to generate the overall wide view BL image(s). It should be noted that the overlapped local patches for reference generation may be of different sizes from the corresponding overlapped local patches used to generate the LUTs.
- Weighting Matrix To ensure relatively smooth transitions across patch boundaries, a weighting map with individual weighting factors for different pixels may be applied to the overlapped local patches used in image future. For a pixel which is covered by multiple overlapped local patches, its fused pixel value may be a weighed combination of pixel values from all the multiple overlapped patches multiplied with a normalization factor summed from weight factors used to generate the weighted combination.
- a (luma channel) weighting map as described herein for an overlapped local patch – with a total number ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ of pixels in the luma channel – used for image fusion (in the luma channel) can be defined or specified with a ( ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ in the luma channel) weighting matrix comprising respective weighting factors for (luma pixel values or codewords of) all pixels represented in the overlapped local patch.
- non-luma channel weighting maps or matrix for image fusion in the non-luma channels can be similarly defined or specified.
- a weighting map or matrix may be specifically selected or configured such that a contour formed by equal values of a subset of the weighting factors in the weighting map or matrix for the overlapped local patch may be of the same or similar shape as compared with a spatial shape of the overlapped local patch such as square or rectangle, rather than an isotropic circular shape.
- Each of these matrices in expressions (8) and (9) above specifies equal weights (or weighting factor values) along a rectangle contour, which is the same as or similar to the spatial shape of a corresponding overlapped local patch, with the highest weights at the center and non-center weights decreasing
- These two matrices in expressions (8) and (9) above or their corresponding weighting maps may be fixed for each of some or all of the overlapped local patches used for image fusion, except for possibly boundary overlapped local patches which do not have full patch size.
- FIG.3C illustrates example operations that apply patch-based weighting matrixes or maps, as defined in expressions (8) and (9) above, to mapped or forward reshaped overlapped local patches to generate an overall or fused reference wide view BL image corresponding to an input or source wide view HDR/WCG image.
- the mapped or forward reshaped overlapped local patches may be generated from patch- based forward reshaping or LUT mapping operations corresponding input overlapped local patches derived from the input or source wide view HDR/WCG image.
- the overall or fused reference wide view BL image depicts the same visual semantic content as – and may be in a lower bit depth or dynamic range than – the input or source wide view HDR/WCG image.
- path-based or patch-specific mapped or forward reshaped pixel values or codewords in the luma channel as ⁇ o ⁇ ⁇ ,n .
- path-based or patch-specific mapped or forward reshaped pixel values or codewords in the C non-luma channel as ⁇ o ⁇ ⁇ ,n .
- path-based or patch-specific mapped or forward reshaped pixel values or codewords in either the luma channel or the C non-luma channel as ⁇ o ⁇ n .
- Denote accumulated weights or weighting factors in the luma channel as ⁇ ⁇ ⁇ , ⁇ .
- two memory spaces may be prepared – e.g., initiated to pixel values of all zeros (0), as represented by the two black rectangles on the left side of FIG.3C – with the same dimension (E ⁇ D) as that of the input or source wide view HDR/WCG image.
- the first (to store accumulated pixel values or codewords ⁇ ⁇ n ; which may be simply referred to as the memory space ⁇ ⁇ n ) of the two memory spaces may be used to accumulate respective weighted patch-based reshaped pixel values or codewords for all pixels (to be) represented in the (overall or fused) reference wide view BL image.
- the second (to store accumulated weights or weighting factors ⁇ ⁇ ⁇ ; which may be simply referred to as the memory space ⁇ ⁇ ⁇ ) of the two memory spaces may be used to accumulates respective weights or weighting factors for all the pixels.
- the mapped or forward reshaped pixel values or codewords can be multiplied by weights or weighting factors in a corresponding patch-based weighting matrix. All weighted mapped pixel values or codewords at each pixel (location) covered by overlapped local patches can be accumulated in the corresponding pixel location in the memory space ⁇ ⁇ n .
- weight maps for the luma and non-luma channels are fixed for some or all (input or reshaped) images.
- the accumulated weights or weighting factors such as ⁇ ⁇ ⁇ , ⁇ and ⁇ ⁇ ⁇ , ⁇ are fixed for these images, and can be computed once, for example at the system boot up and then stored in its allocated memory space.
- TPBC Tensor-Product B-Spline with Coordinate
- a corresponding backward reshaping mapping/function corresponding to the forward reshaping mapping/function may be constructed to be used to backward reshape forward reshaped wide view BL image to generate a reconstructed or backward reshaped wide view HDR/WCG image that approximates the input or source wide view HDR/WCG image.
- the forward and backward reshaping mappings or functions may be built or constructed as TPBC reshaping functions.
- TPB provides a relatively high flexibility and accuracy for reshaping operations that may include dynamic range conversion and color mapping.
- TPB forward and backward reshaping functions can be derived using the previously mentioned BESA method or algorithm, which builds a reversible paired of TPB transforms representing respectively a TPB forward reshaping function used to forward reshape the input or source wide view HDR/WCG image into the forward reshaped image and a corresponding TPB backward reshaping function to backward reshape the forward reshaped image to the reconstructed or backward reshaped image that relatively closely approximate (e.g., with minimized errors or differences, etc.) the input or source wide view HDR/WCG image.
- a TPB reshaping function as described herein may use a combination of luma pixel values or codewords of pixels in an input (or to-be reshaped) image and their positional information such as pixel coordinates – or a positional functional or encoded form of the pixel coordinates – as inputs to predict (e.g., luma, etc.) pixel values or codewords of pixels in an output (or predicted) image as output.
- the TPB reshaping function may use non-luma pixel values or codewords of the pixels in the input image as a part of the inputs in addition to or in place of some or all in the combination of the luma pixel values or codewords and the positional information.
- pixel coordinates may be directly used as the positional information in the inputs to a TPB reshaping function.
- a positional functional or encoded form of the pixel coordinates may be used as the positional information in the inputs to a TPB reshaping function.
- the selection of specific type(s) of positional encoding functions can be based at least in part on a comparison of respective performances among some or all candidate positional encoding functions and a selection of specific positional encoding functions corresponding to the best performance.
- the selected positional encoding functions can be signaled by image metadata provided by an upstream device to a downstream recipient device.
- the encoded positional information ( ⁇ y , ⁇ z) may be used in reshaping (or prediction) functions as described herein in place of the pixel coordinates ( ⁇ , ⁇ ).
- forward path TPB Optimization A process flow portion with forward reshaping operations may be referred to as a forward path herein.
- forward reshaping mappings/functions takes input pixel values or codewords in input or source wide view HDR/WCG image(s) as input to predict or estimate output pixel values or codewords that collectively constitute forward reshaped wide view BL image(s) corresponding to – or depicting the same visual semantic content as – the input or source wide view HDR/WCG image(s).
- a TPB forward reshaping mapping/function can be used to take (1) an input luma pixel value or codeword (denoted as ⁇ ⁇ ⁇ ( ⁇ , ⁇ ), where m and n represent pixel (location) coordinates of a pixel) of an (or t-th) input or source wide view HDR/WCG image, (2) input x-axis positional information represented by ⁇ y encoded from m, and (3) input y-axis positional information represented by ⁇ y encoded from n to predict an output luma pixel value or codeword (denoted as ⁇ s ⁇ ⁇ ( ⁇ , ⁇ )) in a (or t-th) forward reshaped wide view BL image for the luma channel in the forward path.
- TPB basis functions and input pixel values or codewords of P pixels selected or collected from T input or source wide view HDR/WCG images within the same scene as follows: s ( ⁇ ⁇ %, ⁇ ù ⁇ q û frame index for the k-th pixel; (mk, nk) represent the pixel coordinates or encoded positional information for the k-th pixel.
- a ground truth vector (denoted as s ⁇ ⁇ i ( ⁇ , ⁇ ) or s ⁇ ⁇ [ ( ⁇ , ⁇ )) for the chroma channel may also be constructed from channel pixel values or codewords of the corresponding P pixels in the corresponding (e.g., fused, etc.) reference wide view BL images within the same scene.
- a matrix (denoted as ⁇ ⁇ ⁇ , where C is C0 or C1) or a vector (denoted as ⁇ ⁇ ⁇ ) for the chroma channel can be constructed similarly to the matrix or the vector for the luma channel in expressions (25) above.
- optimized values ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ⁇ of TPB (prediction) coefficients can be obtained or represented as ⁇ with backward reshaping operations may be referred to as a backward path herein.
- TPB optimization can be performed in the backward path similar to the foregoing TPB optimization in the forward path.
- a TPB backward reshaping mapping/function can be used to take (1) an input luma pixel value or codeword (denoted as ⁇ ⁇ ⁇ ( ⁇ , ⁇ ), where m and n represent pixel (location) coordinates of a pixel) of an (or t-th) forward reshaped wide view BL image, (2) input x-axis positional information represented by ⁇ y encoded from m, and (3) input y-axis positional information represented by ⁇ y encoded from n to predict an output luma pixel value or codeword (denoted as ⁇ ⁇ ⁇ ( ⁇ , ⁇ )) in a (or t-th) reconstructed or backward reshaped wide view image for the luma channel in the backward path.
- a ground truth vector (denoted as ⁇ ⁇ ) for measuring prediction errors of the predicted output luma pixel values or codewords ( ⁇ ⁇ ) may be constructed from the input or source luma pixel values or codewords of
- a ground truth vector for the chroma channel – similar to the ground truth vector in expression (32) above – may also be constructed from corresponding chroma channel pixel values or codewords of the corresponding P pixels in the corresponding input or source wide view HDR/WCG images within the same scene.
- a matrix or a vector for the chroma channel can be constructed similarly to the matrix or the vector for the luma channel in expressions (34) above.
- images with local reshaping based at least in part on positional encoding can support a relatively high degree of color precision, thereby preventing, avoiding or reducing color shift in the reshaped or predicted images.
- differences in PSNR measurements/values between reconstructed or backward reshaped images generated from the local reshaping with positional encoding and corresponding reconstructed or backward reshaped images generated from the global reshaping without positional encoding may be relatively significant (e.g., ⁇ 60dB, etc.) in favor of the local reshaping.
- different types of positional encoding such as linear positional encoding, non-linear positional encoding with sine/cosine functions may yield different image quality measurements/values such as different PSNR measurements/values in reconstructed or backward reshaped images generated from the local reshaping performed with the different types of positional encoding.
- a specific type among the different types of positional encoding corresponding to the best image quality measurements/values can be selected, implemented and/or signaled with image metadata to downstream recipient device(s) for the purpose of allowing the downstream recipient device(s) to generate relatively high quality reconstructed or backward reshaped images.
- forward reshaping may be performed on an input or source wide view HDR/WCG signal or images therein to make forward reshaped output wide view BL signal or images therein as close to (e.g., fused, etc.) reference wide view BL images as possible (e.g., with minimized prediction errors, etc.).
- backward reshaping may be performed on the forward reshaped output wide view BL signal or images therein to make reconstructed or backward reshaped output wide view HDR/WCG signal or images therein as close to the input or source wide view HDR/WCG signal or images therein as possible (e.g., with minimized prediction errors, etc.).
- an iterative algorithm such as the BESA algorithm may be used to generate optimized values for prediction/reshaping operations such as TPB prediction/reshaping operations.
- the algorithm can modify the reference BL signal iteratively, such that prediction errors between the (final; post-iteration) reconstructed or backward reshaped output wide view HDR/WCG signal or images therein and the input or source wide view HDR/WCG signal or images therein are minimized.
- the BESA algorithm can be deployed or performed in each of some or all (e.g., three, etc.) color channels of a color space.
- these color channels may be luma and chroma channels and denoted as ch, which can be Y, Cb, and Cr.
- a superscript, ⁇ may be added as the iteration index in (1) the TPB forward reshaping function ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ⁇ , (2) the TPB backward reshaping function ⁇ ( ⁇ %, ⁇ ,( ⁇ ) % , and (3) the F forward reshaped wide view base layer images ⁇ ⁇ ,( ⁇ ) ⁇ within ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ⁇ p ⁇ ⁇ ⁇ ⁇ ( ⁇ , ⁇ ), ⁇ y, ⁇ zq [0148]
- the same superscript, ⁇ may be added as the iteration index in the corresponding TPBC coefficient matrices ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ( ⁇ %, ⁇ ,( ⁇ ) ⁇ and ⁇ % .
- FIG.2E illustrates example process flow implementing the BESA algorithm to optimize reshaping mappings/functions, which may be implemented, for example, by a video encoder or processing blocks/components therein.
- the process flow comprises a processing block/component for initialization (not shown in FIG.
- the process flow for the BESA algorithm comprises a processing block/component for forward reshaping, which at each iteration, generates or optimizes the TPB (forward reshaping) coefficients or parameters ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ( ⁇ %, ⁇ , ⁇ in the forward reshaping function ⁇ ( ⁇ ) ⁇ to minimize prediction errors or differences between forward reshaped wide view BL pixel values or codewords ⁇ s ⁇ ⁇ ⁇ ,( ⁇ ) ( ⁇ , ⁇ ) at the current iteration and the modified reference wide view BL pixel
- This error amount ⁇ ⁇ ⁇ ⁇ ,( ⁇ ) ( ⁇ , ⁇ ) may be used to modify or update the (current) modified reference wide view BL pixel values or codewords for the entire set ⁇ into the modified reference wide view BL pixel values or codewords s o ⁇ ,( ⁇ a[) ⁇ ( ⁇ , ⁇ ) for the next iteration, and may be a reduced amount as compared with
- pixel values or codewords of all available pixels of each of all images in a scene may be used to generate optimized values for TPBC coefficients in TPB reshaping functions/mappings.
- a process flow under this approach may use relatively large amounts of memory and computing resources.
- pixel values or codewords of a relatively small subset of all available pixels of each of all images in a scene may be used to generate optimized values for TPBC coefficients in TPB reshaping functions/mappings.
- a process flow under this approach may use relatively limited amounts of memory and computing resources and may contain multiple stages such as three stages illustrated in FIG.2F.
- TPBC reshaping functions can be generated or optimized by fitting or modeling on an incremental data set, which includes an addition of the most distorted pixels (e.g., as measured or compared with an applicable distortion threshold, etc.) from a previous stage.
- TPBC reshaping functions are generated or optimized by fitting or modeling on a first data set comprising pixel values or codewords in down-sampled images derived from input or source wide view HDR/WCG images ⁇ ⁇ ⁇ ⁇ in a scene for a color channel ch such as a luma or non-luma channel, non-luma as well as derived from corresponding (fused) wide view BL images s ⁇ ⁇ ⁇ in the scene for a corresponding color channel ch such as the same luma or non-luma channel with a different bit depth or dynamic ranges.
- FIG.2G illustrates an example detailed process flow corresponding to stage 1 of the process flow of FIG.2F.
- the process flow of FIG.2G in stage 1 comprises a processing block or component for downsampling, which (e.g., uniformly, etc.) downsamples ⁇ ′ ⁇ ⁇ ⁇ by a constant factor denoted as ⁇ ⁇ (e.g., a prime number, 101, etc.).
- ⁇ ⁇ e.g., a prime number, 101, etc.
- FIG.2G in stage 1 comprises a processing block or component or performs a first round of BESA optimization using pixel set ⁇ [ to determine TPB coefficients for constructing TPB forward and backward reshaping functions/mappings denoted respectively as ⁇ ⁇ ( , o ⁇ % ⁇ and ⁇ % ( , ⁇ o % ⁇ , as follows: [) a processing block or component denoted as “Apply TPBC 1”, which applies the TPB forward reshaping function ⁇ ⁇ ( , o ⁇ % ⁇ to the input or source wide view HDR/WCG images ⁇ ⁇ ⁇ ⁇ to obtain or generate
- (1 ⁇ 4 ⁇ r ⁇ ⁇ ⁇ (£) ⁇ ⁇ ⁇ r ⁇ , , ⁇ o ⁇ ⁇ (£)1 ⁇ 4 ⁇ ⁇ 3 ⁇ 4 ⁇ , o ⁇ ⁇ ) ⁇ O( 1 ⁇ 4s r ⁇ ⁇ ⁇ (£) ⁇ ⁇ s r ⁇ , , ⁇ o ⁇ ⁇ (£)1 ⁇ 4 ⁇ ⁇ ' , ⁇ o ⁇ ) ⁇ ⁇ ⁇ r ⁇ , , ⁇ o ⁇ ⁇ 3h ⁇ £ ⁇ _3_ ⁇ h ⁇ ( ⁇ ⁇ ⁇ ⁇ , ⁇ o ⁇ ) ) 3 ⁇ 4 ⁇
- FIG.2H illustrates an example detailed process flow corresponding to stage 2 of the process flow of FIG.2F.
- the process flow of FIG.2H in stage 2 comprises processing blocks or components similar to those in the process flow of FIG.2G in stage 1.
- a second data set ⁇ ⁇ may be in BESA operations.
- (1 ⁇ 4 ⁇ r ⁇ ⁇ ⁇ (£) ⁇ ⁇ ⁇ r ⁇ , , ⁇ o ⁇ l (£)1 ⁇ 4 ⁇ ⁇ 3 ⁇ 4 ⁇ , ⁇ o l ) ⁇ O( 1 ⁇ 4s r ⁇ ⁇ ⁇ (£) ⁇ ⁇ s r ⁇ , , ⁇ o ⁇ l (£)1 ⁇ 4 ⁇ ⁇ ' ⁇ , ⁇ o l ) ⁇ where ⁇ ⁇ r ⁇ , , ⁇ o ⁇ l 3h ⁇ £ ⁇ _3_ ⁇ h ⁇ ( ⁇ ⁇ ⁇ ⁇ , ⁇ o l )
- FIG.2I illustrates an example detailed process flow corresponding to stage 3 of the process flow of FIG.2F.
- the process flow of FIG.2I in stage 3 comprises processing blocks or components similar to those in the process flow of FIG.2G or FIG.2H in stage 1 or 2.
- a third data set ⁇ ⁇ may be in BESA operations.
- Other processing blocks or components in stage 3 may be performed with this second data set ⁇ ⁇ similar to show the corresponding processing blocks or components in stage 2 is performed with the second data set ⁇ ⁇ .
- more or fewer stages other than three stages may be implemented.
- the 3-stage optimization as described herein can generate images with image qualities, for example as measured with PSNR values illustrated in TABLE 3 below, better than or comparable to all- or full-pixel optimization that utilizes all (non-downsampled) pixels or pixel values.
- the 3- stage optimization can be implemented or perform to reduce memory usage as well as provide better reconstruction quality.
- FIG.4A illustrates an example process flow according to an embodiment.
- one or more computing devices or components e.g., an encoding device/module, a transcoding device/module, an upstream device, a sender, etc. may perform this process flow.
- an image processing system generates one or more reference wide view images of a first domain (e.g., a first color space, a first dynamic range, a first bit depth image/signal, etc.) from one or more source wide view images of a second domain (e.g., a second color space, a second dynamic range, a second bit depth image/signal, etc.).
- a first domain e.g., a first color space, a first dynamic range, a first bit depth image/signal, etc.
- a second domain e.g., a second color space, a second dynamic range, a second bit depth image/signal, etc.
- the image processing system generates a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain.
- Each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data.
- the image processing system encodes the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images.
- the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings.
- the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth;
- the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth.
- the positional data is derived from the pixel locations using one of a linear function or a non-linear function.
- the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch-specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images; the patch-specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non- overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain.
- a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel.
- one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches.
- at least one of the forward and backward reshaping mappings is optimized using a Backward Error Subtraction Algorithm (BESA) algorithm.
- BESA Backward Error Subtraction Algorithm
- At least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images.
- the pixels in the downsampled pixel set are downsampled from a sequence of pixel rows, ordered in a lexicographical order, in the one or more source wide view images and the one or more reference wide view images.
- At least one of the forward and backward reshaping mappings is optimized using multiple stages; in each of the multiple stages, a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a proper set of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images.
- BESA Backward Error Subtraction Algorithm
- at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images.
- FIG.4B illustrates an example process flow according to an embodiment.
- one or more computing devices or components may perform this process flow.
- an image processing system decodes one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream.
- the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping.
- the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping.
- the image processing system generates one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping. [0203] In block 456, the image processing system generates one or more display images from the one or more reconstructed wide view images. [0204] In block 458, the image processing system renders the one or more display images on an image display. [0205] In an embodiment, the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display.
- a computing device such as a display device, a mobile device, a set-top box, a multimedia device, etc.
- an apparatus comprises a processor and is configured to perform any of the foregoing methods.
- a non- transitory computer readable storage medium storing software instructions, which when executed by one or more processors cause performance of any of the foregoing methods.
- a computing device comprising one or more processors and one or more storage media storing a set of instructions which, when executed by the one or more processors, cause performance of any of the foregoing methods.
- Embodiments of the present invention may be implemented with a computer system, systems configured in electronic circuitry and components, an integrated circuit (IC) device such as a microcontroller, a field programmable gate array (FPGA), or another configurable or programmable logic device (PLD), a discrete time or digital signal processor (DSP), an application specific IC (ASIC), and/or apparatus that includes one or more of such systems, devices or components.
- IC integrated circuit
- FPGA field programmable gate array
- PLD configurable or programmable logic device
- DSP discrete time or digital signal processor
- ASIC application specific IC
- the computer and/or IC may perform, control, or execute instructions relating to the adaptive perceptual quantization of images with enhanced dynamic range, such as those described herein.
- the computer and/or IC may compute any of a variety of parameters or values that relate to the adaptive perceptual quantization processes described herein.
- the image and video embodiments may be implemented in hardware, software, firmware and various combinations thereof.
- Certain implementations of the inventio comprise computer processors which execute software instructions which cause the processors to perform a method of the disclosure. For example, one or more processors in a display, an encoder, a set top box, a transcoder or the like may implement methods related to adaptive perceptual quantization of HDR images as described above by executing software instructions in a program memory accessible to the processors.
- Embodiments of the invention may also be provided in the form of a program product.
- the program product may comprise any non-transitory medium which carries a set of computer- readable signals comprising instructions which, when executed by a data processor, cause the data processor to execute a method of an embodiment of the invention.
- Program products according to embodiments of the invention may be in any of a wide variety of forms.
- the program product may comprise, for example, physical media such as magnetic data storage media including floppy diskettes, hard disk drives, optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, flash RAM, or the like.
- the computer-readable signals on the program product may optionally be compressed or encrypted.
- the special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques.
- the special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques.
- FIG.5 is a block diagram that illustrates a computer system 500 upon which an embodiment of the invention may be implemented.
- Computer system 500 includes a bus 502 or other communication mechanism for communicating information, and a hardware processor 504 coupled with bus 502 for processing information.
- Hardware processor 504 may be, for example, a general purpose microprocessor.
- Computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 502 for storing information and instructions to be executed by processor 504.
- Main memory 506 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 504.
- Computer system 500 further includes a read only memory (ROM) 508 or other static storage device coupled to bus 502 for storing static information and instructions for processor 504.
- ROM read only memory
- a storage device 510 such as a magnetic disk or optical disk, is provided and coupled to bus 502 for storing information and instructions.
- Computer system 500 may be coupled via bus 502 to a display 512, such as a liquid crystal display, for displaying information to a computer user.
- An input device 514 is coupled to bus 502 for communicating information and command selections to processor 504.
- cursor control 516 such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 504 and for controlling cursor movement on display 512.
- This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
- Computer system 500 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 500 to be a special-purpose machine.
- the techniques as described herein are performed by computer system 500 in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage device 510. Execution of the sequences of instructions contained in main memory 506 causes processor 504 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.
- storage media refers to any non-transitory media that store data and/or instructions that cause a machine to operation in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media.
- Non-volatile media includes, for example, optical or magnetic disks, such as storage device 510.
- Volatile media includes dynamic memory, such as main memory 506.
- Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.
- Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media.
- transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 502.
- Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
- Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 504 for execution.
- the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer.
- the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
- a modem local to computer system 500 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal.
- An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 502.
- Bus 502 carries the data to main memory 506, from which processor 504 retrieves and executes the instructions.
- the instructions received by main memory 506 may optionally be stored on storage device 510 either before or after execution by processor 504.
- Computer system 500 also includes a communication interface 518 coupled to bus 502.
- Communication interface 518 provides a two-way data communication coupling to a network link 520 that is connected to a local network 522.
- communication interface 518 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line.
- ISDN integrated services digital network
- communication interface 518 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
- LAN local area network
- Network link 520 typically provides data communication through one or more networks to other data devices.
- network link 520 may provide a connection through local network 522 to a host computer 524 or to data equipment operated by an Internet Service Provider (ISP) 526.
- ISP 526 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 528.
- Internet 528 uses electrical, electromagnetic or optical signals that carry digital data streams.
- Computer system 500 can send messages and receive data, including program code, through the network(s), network link 520 and communication interface 518.
- a server 530 might transmit a requested code for an application program through Internet 528, ISP 526, local network 522 and communication interface 518.
- the received code may be executed by processor 504 as it is received, and/or stored in storage device 510, or other non-volatile storage for later execution.
- a method comprising: generating one or more reference wide view images of a first domain from one or more source wide view images of a second domain; generating a forward reshaping mapping to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain; generating a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain; wherein each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data; encoding the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mapping
- EEE2 The method of EEE1, wherein the forward reshaping mapping receives source pixel values of the one or more source wide view images and source positional data derived from source pixel coordinates as input and generates forward reshaped pixel values of the one or more forward reshaped wide view images.
- EEE3 The method of EEE1 or EEE2, wherein the backward reshaping mapping receives forward reshaped pixel values of the one or more forward reshaped wide view images and source positional data derived from source pixel coordinates as input and generates backward reshaped pixel values of the one or more reconstructed wide view images.
- EEE4 The method of EEE1, wherein the forward reshaping mapping receives source pixel values of the one or more source wide view images and source positional data derived from source pixel coordinates as input and generates backward reshaped pixel values of the one or more reconstructed wide view images.
- the forward reshaping mapping includes a chroma forward reshaping mapping that maps source chrominance pixel values of the one or more source wide view images into forward reshaped chrominance pixel values of the one or more forward reshaped wide view images based at least in part on a scaling factor; wherein the scaling factor represents a ratio of a first difference between maximum and minimum source luminance pixel values and a second difference between maximum and minimum source chrominance pixel values.
- EEE6 The method of any of EEE1-EEE4, wherein the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth; the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth.
- EEE6 The method of any of EEE1-EEE5, wherein the positional data is derived from the pixel locations using one of a linear function or a non-linear function.
- EEE7 The method of any of EEE1-EEE6, wherein the non-linear function is from a sinusoidal function family.
- EEE8 The method of any of EEE8.
- any of EEE1-EEE7 wherein the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch-specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images; wherein the patch- specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; wherein the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain.
- EEE9 The method of any of EEE1-EEE8, wherein a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel.
- EEE10 The method of EEE9, wherein the weighted combination is derived from weighting factors that are given by Gaussian distribution functions with rectangular contours of equal values.
- EEE11 The method of EEE9, wherein one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches.
- EEE13 The method of any of EEE1-EEE12, wherein at least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images.
- BESA Backward Error Subtraction Algorithm
- EEE15 The method of any of EEE1-EEE14, wherein at least one of the forward and backward reshaping mappings is optimized using multiple stages; wherein the multiple stages include a beginning stage in which a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a first proper subset of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images.
- BESA Backward Error Subtraction Algorithm
- EEE15 wherein a first set of pixels with relatively large prediction errors is identified after the BESA algorithm in the beginning stage has been performed; wherein the multiple stages include a second stage, following the beginning stage, in which the BESA algorithm is performed based on a second proper subset of pixels derived as a set union of the first proper subset of pixels and the first set of pixels with the relatively large prediction errors.
- EEE16 wherein a second set of pixels with relatively large prediction errors is identified after the BESA algorithm in the second stage has been performed; wherein the multiple stages include a third stage, following the second stage, in which the BESA algorithm is performed based on a third proper subset of pixels derived as a second set union of the second proper subset of pixels and the second set of pixels with the relatively large prediction errors.
- EEE18 The method of any of EEE1-EEE17, wherein at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images.
- a method comprising: decoding one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream, wherein the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping; generating one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping; generating one or more display images from the one or more reconstructed wide view images; rendering the one or more display images on an image display.
- EEE20 The method of EEE19, wherein the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display.
- EEE21 An apparatus comprising a processor and configured to perform any one of the methods recited in EEE1-EEE20.
- EEE22 A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with any of the methods recited in EEE1-EEE20.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202363485427P | 2023-02-16 | 2023-02-16 | |
| EP23156989 | 2023-02-16 | ||
| PCT/US2024/015944 WO2024173649A1 (en) | 2023-02-16 | 2024-02-15 | Local reshaping using tensor-product b-spline with coordinates wide view video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4666581A1 true EP4666581A1 (en) | 2025-12-24 |
Family
ID=90365217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP24711955.5A Pending EP4666581A1 (en) | 2023-02-16 | 2024-02-15 | Local reshaping using tensor-product b-spline with coordinates wide view video |
Country Status (2)
| Country | Link |
|---|---|
| EP (1) | EP4666581A1 (en) |
| WO (1) | WO2024173649A1 (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10593028B2 (en) * | 2015-12-03 | 2020-03-17 | Samsung Electronics Co., Ltd. | Method and apparatus for view-dependent tone mapping of virtual reality images |
| US10223774B2 (en) | 2016-02-02 | 2019-03-05 | Dolby Laboratories Licensing Corporation | Single-pass and multi-pass-based polynomial approximations for reshaping functions |
| US10032262B2 (en) | 2016-02-02 | 2018-07-24 | Dolby Laboratories Licensing Corporation | Block-based content-adaptive reshaping for high dynamic range images |
| TWI812874B (en) * | 2019-10-01 | 2023-08-21 | 美商杜拜研究特許公司 | Tensor-product b-spline predictor |
| WO2022103902A1 (en) * | 2020-11-11 | 2022-05-19 | Dolby Laboratories Licensing Corporation | Wrapped reshaping for codeword augmentation with neighborhood consistency |
-
2024
- 2024-02-15 WO PCT/US2024/015944 patent/WO2024173649A1/en not_active Ceased
- 2024-02-15 EP EP24711955.5A patent/EP4666581A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024173649A1 (en) | 2024-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11277627B2 (en) | High-fidelity full reference and high-efficiency reduced reference encoding in end-to-end single-layer backward compatible encoding pipeline | |
| US10701375B2 (en) | Encoding and decoding reversible production-quality single-layer video signals | |
| US10264287B2 (en) | Inverse luma/chroma mappings with histogram transfer and approximation | |
| US10560707B2 (en) | Chroma subsampling and gamut reshaping | |
| EP3168809B1 (en) | Image processing for hdr images | |
| US10972759B2 (en) | Color appearance preservation in video codecs | |
| US11388408B2 (en) | Interpolation of reshaping functions | |
| ES3062794T3 (en) | Adaptive local reshaping for sdr-to-hdr up-conversion | |
| CN105052143A (en) | Encoding perceptually-quantized video content in multi-layer VDR coding | |
| US11818400B2 (en) | Adjustable trade-off between quality and computation complexity in video codecs | |
| US20240007682A1 (en) | Wrapped reshaping for codeword augmentation with neighborhood consistency | |
| JP7797682B2 (en) | Image optimization in mobile capture and editing applications | |
| JP7627389B2 (en) | Noise removal for SDR-HDR local reconstruction | |
| WO2024173649A1 (en) | Local reshaping using tensor-product b-spline with coordinates wide view video | |
| JP7827882B2 (en) | Image optimization in mobile capture and editing applications | |
| KR20180054623A (en) | Determination of co-localized luminance samples of color component samples for HDR coding / decoding | |
| EP4690094A1 (en) | Image enhancement acceleration by patch-wise regression | |
| HK1248904A1 (en) | Inverse luma/chroma mappings with histogram transfer and approximation | |
| HK1257716B (en) | Encoding and decoding reversible production-quality single-layer video signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20250903 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| P01 | Opt-out of the competence of the unified patent court (upc) registered |
Free format text: CASE NUMBER: UPC_APP_0004280_4666581/2026 Effective date: 20260206 |