EP4666581A1 - Local reshaping using tensor-product b-spline with coordinates wide view video - Google Patents

Local reshaping using tensor-product b-spline with coordinates wide view video

Info

Publication number
EP4666581A1
EP4666581A1 EP24711955.5A EP24711955A EP4666581A1 EP 4666581 A1 EP4666581 A1 EP 4666581A1 EP 24711955 A EP24711955 A EP 24711955A EP 4666581 A1 EP4666581 A1 EP 4666581A1
Authority
EP
European Patent Office
Prior art keywords
wide view
view images
images
reshaping
source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP24711955.5A
Other languages
German (de)
French (fr)
Inventor
Janos Horvath
Guan-Ming Su
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dolby Laboratories Licensing Corp
Original Assignee
Dolby Laboratories Licensing Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dolby Laboratories Licensing Corp filed Critical Dolby Laboratories Licensing Corp
Publication of EP4666581A1 publication Critical patent/EP4666581A1/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Definitions

  • DR dynamic range
  • HVS human visual system
  • DR may relate to a capability of the human visual system (HVS) to perceive a range of intensity (e.g., luminance, luma) in an image, e.g., from darkest blacks (darks) to brightest whites (highlights).
  • DR relates to a “scene-referred” intensity.
  • DR may also relate to the ability of a display device to adequately or approximately render an intensity range of a particular breadth. In this sense, DR relates to a “display-referred” intensity.
  • HDR high dynamic range
  • HVS human visual system
  • EDR enhanced dynamic range
  • VDR visual dynamic range
  • HVS human visual system
  • n ⁇ 8 e.g., color 24-bit JPEG images
  • images where n > 8 may be considered images of enhanced dynamic range.
  • a reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between color values (e.g., luminance) of an input video signal to output screen color values (e.g., screen luminance) produced by the display.
  • information about its EOTF may be embedded in the bitstream as (image) metadata.
  • metadata herein relates to any auxiliary information transmitted as part of the coded bitstream and assists a decoder to render a decoded image.
  • Such metadata may include, but are not limited to, color space or gamut information, reference display parameters, and auxiliary signal parameters, as those described herein.
  • PQ perceptual luminance amplitude quantization.
  • the human visual system responds to increasing light levels in a very nonlinear way. A human’s ability to see a stimulus is affected by the luminance of that stimulus, the size of the stimulus, the spatial frequencies making up the stimulus, and the luminance level that the eyes have adapted to at the particular moment one is viewing the stimulus.
  • a perceptual quantizer function maps linear input gray levels to output gray levels that better match the contrast sensitivity thresholds in the human visual system.
  • SMPTE High Dynamic Range EOTF of Mastering Reference Displays
  • LDR lower dynamic range
  • SDR standard dynamic range
  • EDR content may be displayed on EDR displays that support higher dynamic ranges (e.g., from 1,000 nits to 5,000 nits or more). Such displays may be defined using alternative EOTFs that support high luminance capability (e.g., 0 to 10,000 or more nits).
  • An example of such an EOTF is defined in SMPTE 2084 and Rec. ITU-R BT.2100, “Image parameter values for high dynamic range television for use in production and international programme exchange,” (06/2017).
  • improved techniques for generating high quality video content data with high dynamic range, high local contrast and vivid color are desired.
  • FIG.1A and FIG.1B illustrate example encoder side and decoder side architectures
  • FIG.2A illustrates an example process or method for reference base layer image generation
  • FIG.2B illustrates example overlapped and non-overlapped patches
  • FIG.2C illustrates example patch based forward reshaping mapping generation
  • FIG.2D illustrates an example method or process flow for patch fusion
  • FIG.2E through FIG.2I illustrates example process flows for optimizing reshaping mappings
  • FIG.3A illustrates example distributions of block based standard deviations
  • FIG.3B illustrates an example weighting map
  • FIG.3C illustrates example operations in connection with patch-based weighting maps
  • FIG.4A and FIG.4B illustrate example process
  • User-generated and professional wide view (e.g., greater than a viewer’s entire vision field, greater than a user’s field of view, greater than 90-degrees, 360-degree, etc.) content is already being distributed using popular content sharing platforms.
  • relatively wide view video such as up to 360-degree HDR video.
  • the wide view video often has much larger image dimensions, contains much richer spatial information from up to all front and/or rear viewing angles not necessarily limited to only within an individual viewer’s vision field, and includes a much higher pixel count in each image.
  • the wide view video can exhibit much higher dynamic ranges and much wider color gamuts as well as much larger spatial variations of dynamic ranges and color gamuts in different spatial locations of a wide view scene/image, and therefore present a much greater challenge in terms of supporting a widely diverse range of local luminances and color distributions in a relatively wide field of view (FOV) as compared with the 2D SDR or HDR video.
  • Many existing video codecs that support or adopt a global reshaping method in processing the 2D SDR or HDR video may be ill equipped to handle much higher dynamic range (HDR) and wider color gamut (WCG) in wide view video content in a compression efficient manner.
  • Fused patch-based local reshaping techniques as described herein can be implemented to support wide view video efficiently and effectively, and to resolve issues and handle challenges associated with the wide view video.
  • a reference base layer (BL) signal can be generated from original (input or source) wide view video using a fused patch-based algorithm/method to address or preserve local dynamic range and color distribution.
  • Some or all of these techniques can operate with wide view video represented in an Equi-Rectangular Projection (ERP) format or other formats different from the ERP forma.
  • ERP Equi-Rectangular Projection
  • intermediate reference BL video content may be first generated patch-wisely to provide sufficient codewords for preventing banding visual artifacts and preserving colors or color precisions in various local regions.
  • This (final fused) reference BL signal can be used to increase video compression efficiency globally and preserve HDR/WCG properties of the original or source wide view video locally in a reconstructed HDR signal, in order to help prevent highlight/dark area clipping, alleviate banding visual artifacts and achieve color fidelity in the reconstructed HDR signal.
  • a relatively compression- efficient and revertible forward reshaped BL signal can be generated by available video codecs on the encoder side using a forward reshaping function to approximate the reference BL signal.
  • the reconstructed HDR signal can be generated or constructed by available video codecs on the decoder side from the forward reshaped BL signal using a backward reshaping function corresponding to the forward reshaping function.
  • highly varying characteristics of local dynamic range and local color distribution in wide view HDR/WCG video data can be efficiently and effectively addressed or preserved Tensor-Product B-Spline with Coordinates (TPB with Coordinates or TPBC).
  • TPB is a tool to model cross-channel complex mapping or reshaping functions.
  • B-splines or basis splines can be used as functions to fit a given one dimensional curve using polynomial functions with continuity constraints at knot points.
  • multiple B-spline functions can be fused together by (tensor) multiplication to approximate, estimate or fit higher dimensional curves while maintaining smooth connectivity at knot points.
  • Example TPB reshaping functions are described in U.S. Provisional Application Ser. No.62/908,770, titled “TENSOR- PRODUCT B-SPLINE PREDICTOR,” filed on October 1, 2019, which are incorporated by reference in its entirety as if fully set forth herein.
  • Example BESA algorithm/method can be found in U.S. Provisional Patent Application Ser. No. 63/013,063, “Reshaping functions for HDR imaging with continuity and reversibility constraints,” filed on April 21, 2020; U.S. Provisional Patent Application Ser. No. 63/013,807, “Iterative optimization of reshaping functions in single-layer HDR image codec,” filed on April 22, 2020; PCT Application Ser. No.
  • TPB or TPBC as described herein can be implemented with a Backward Error Subtraction Algorithm (BESA) to reach or achieve a relatively high degree of revertability between a pair of corresponding forward and backward reshaping functions.
  • BESA Backward Error Subtraction Algorithm
  • algorithms or methods as described herein can be developed or implemented in a manner that reduces memory footage and computational load.
  • a three-stage optimization may be implemented in an adaptive algorithm that applies or uses incremental datasets to minimize prediction errors.
  • a relatively small bit depth domain (e.g., SDR, etc.) image may be transformed into a relatively large bit depth domain (e.g., HDR, WCG, etc.) image with a relatively small model, for example, built with a relatively small set of pixels selected from among all pixels represented in the images.
  • Optimized operational parameters derived with the adaptive algorithm can be used to generate reconstructed wide view HDR/WCG video content that is visually lossless to the original or source wide view HDR/WCG video content.
  • One or more reference wide view images of a first domain are generated from one or more source wide view images of a second domain.
  • a forward reshaping mapping is generated to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain.
  • a backward reshaping mapping is generated to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain.
  • Each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data.
  • the one or more forward reshaped wide view images and corresponding image metadata are encoded into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images.
  • the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings.
  • the corresponding image metadata includes operational parameters specifying the backward reshaping mappings, but does not include operational parameters specifying the forward reshaping mappings which are applied at the encoder side only.
  • Example embodiments described herein relate to decoding coded packets relating to reconstructed images.
  • One or more forward reshaped wide view images of a first domain and corresponding image metadata are decoded from a bitstream.
  • the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping.
  • the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping.
  • One or more reconstructed wide view images of the second domain are generated from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping.
  • One or more display images are generated from the one or more reconstructed wide view images.
  • the one or more display images are rendered on an image display.
  • FIG.1A illustrates an example encoder side architecture that may be implemented with one or more computing devices.
  • the encoder side architecture may include a plurality of processing blocks or components to receive input or source wide view HDR/WCG images (or image frames); to use the received wide view HDR/WCG images/frames to generate reference BL images/frames; to generate forward reshaping functions to forward reshape the received wide view HDR/WCG images/frames to generate forward reshaped BL images approximating the reference BL images/frames; to encode the forward reshaped BL images along with image metadata specifying backward reshaping functions corresponding to the forward reshaping functions in a coded bitstream; to deliver or transmit the coded bitstream from an upstream device (e.g., some or all of the one or more computing devices implementing the encoder side, etc.) to a downstream device over data communication links or paths between the upstream device and the downstream device; etc.
  • an upstream device e.g., some or all of the one or more computing devices implementing the encoder side, etc.
  • some or all of these foregoing operations may be performed by the one or more computing devices at the encoder side on line in real time, for example while the upstream device communicates the bitstream or a portion thereof to the downstream device.
  • at least some of the foregoing operations may be performed by the one or more computing devices at the encoder side off line in non-real time, for example before the upstream device communicates any portion of the bitstream to the downstream device.
  • the encoder side architecture may include a processing block/component for reference BL generation, which takes the input or source wide view HDR/WCG images (or image frames) as inputs and use the input wide view HDR/WCG images/frames to generate or derive corresponding BL images to serve as reference for forward reshaped BL images to approximate.
  • a processing block/component for reference BL generation takes the input or source wide view HDR/WCG images (or image frames) as inputs and use the input wide view HDR/WCG images/frames to generate or derive corresponding BL images to serve as reference for forward reshaped BL images to approximate.
  • the forward reshaped BL images can be encoded in the coded bitstream with relatively high coding efficiency and subsequently decoded and used by a recipient device to generate reconstructed wide view HDR/WCG images approximate the input or source wide view HDR/WCG in a manner that avoids banding visual artifacts and preserves colors in each local (spatial) region of some or all local (spatial) regions in the reconstructed wide view HDR/WCG images corresponding to some or all local (spatial) regions represented in the input or source wide view HDR/WCG.
  • the encoder side architecture may include a processing block/component for forward/backward TPBC optimization, which generates forward reshaping functions and backward reshaping functions corresponding to the forward reshaping functions using the input wide view HDR/WCG images and the reference BL images as input.
  • the forward/backward TPBC optimization (1) minimizes differences between the reference BL images and forward reshaped BL images (such that the banding prevention and color preservation can be maintained as well as underlying video compression efficiency can be achieved or improved), and (2) minimizes distortions between the reconstructed wide view HDR/WCG images and the original wide view HDR/WCG images.
  • the encoder side architecture may include a processing block/component for forward reshaping, which uses the forward reshaping functions to forward reshape the input wide view HDR/WCG images into the forward reshaped BL images that approximate the reference BL images.
  • the encoder side architecture may include a processing block/component for metadata generation, which generates image metadata or portions thereof to define or specify the backward reshaping functions.
  • the backward reshaping functions can be used, for example by a recipient decoding device of the image metadata, to generate the reconstructed wide view HDR/WCG images that approximate the input wide view HDR/WCG images.
  • the encoder side architecture may include a processing block/component for video compression, which compresses or encodes the forward reshaped BL images into the (video) coded bitstream along with the image metadata generated by the metadata generation.
  • a processing block/component for video compression which compresses or encodes the forward reshaped BL images into the (video) coded bitstream along with the image metadata generated by the metadata generation.
  • available video codecs that support compressing or encoding relatively narrow view 2D video content can be used or enhanced to implement the processing block/component for video compression of FIG.1A.
  • FIG.1B illustrates an example decoder side architecture that may be implemented with one or more computing devices.
  • the decoder side architecture may include a plurality of processing blocks or components to receive a coded bitstream encoded with forward reshaped wide view BL images along with image metadata specifying backward reshaping functions corresponding to forward reshaping functions used to generate the forward reshaped wide view BL images; to apply the backward reshaping functions to the forward reshaped wide view BL images to generate corresponding reconstructed or backward reshaped wide view HDR/WCG images; to render display images derived from the reconstructed or reshaped wide view HDR/WCG images on image display(s); etc.
  • the decoder side architecture may include a processing block/component for metadata extraction, which extracts the image metadata from (image metadata container or data fields carried or included in) the coded bitstream.
  • the decoder side architecture may include a processing block/component for video decompression, which decodes or decompresses the forward reshaped wide view BL images from (encoded video data carried or included in) the coded bitstream.
  • the decoder side architecture may include a processing block/component for backward reshaping, which backward reshapes – or applies the backward reshaping functions to – the forward reshaped wide view BL images (denoted as “Reshaped BL”) into the reconstructed or reshaped wide view HDR/WCG images (or video).
  • Reshaped BL forward reshaped wide view BL images
  • the first of the above mentioned two processing blocks/components, or the reference BL generation includes extracting overlapped patches from image data of each input or source wide view HDR/WCG image in each color channel of an input or source color space in which the input or source wide view HDR/WCG image is represented.
  • the input or source color space may include a luma channel and two chroma channels.
  • BLKSTD block- based standard deviations based method/algorithm
  • Example BLKSTD methods/algorithms are described in U.S. Patent No. 10,032,262, issued on 24 July 2018; U.S. Patent No.10,223,774, issued on 5 March 2019, the entire contents of all of which are hereby incorporated by reference as if fully set forth herein.
  • a luma-chroma channel energy ratio may be determined and applied to generate to generate chroma image data of the reference BL image patches from corresponding image patches of the original (input or source) wide view HDR/WCG images/frames. After determining or generating the luma and chroma image data of the reference BL image patches, these patches can be fused together, for example with a predefined weight function, into a reference wide view BL image in which visual patch boundary artifacts are reduced, removed or otherwise alleviated.
  • the second of the above mentioned two processing blocks/components, or the forward/backward TPBC optimization includes determining an optimized spatial positional (information or data) encoding for the input or source wide view HDR/WCG image. More specifically, optimized operational parameters for forward reshaping TPBC operations are generated and used on the encoder side to forward reshaping the input or source wide view HDR/WCG image into a forward reshaped wide view BL image approximating the reference wide view BL image. Optimized operational parameters for backward reshaping TPBC operations are generated and used on the decoder side to backward reshaping the forward reshaped wide view BL image into a reconstructed or reshaped wide view HDR/WCG image.
  • a three-stage optimization process may be used to generate these optimized operational parameters for the forward and backward TPBC operations.
  • TPBC operations can be performed on an increased-size dataset using the previously mentioned BESA method or algorithm.
  • Reference Base Layer Generation [0043]
  • global reshaping may be applied to reshape an input image with a global reshaping function applied to image data in all spatial regions of the input image into a reshaped image at a relatively high compression efficiency.
  • Such global reshaping techniques or functions may work well with images of a relatively narrow field of view such as 2D images.
  • a wide view HDR/WCG image/picture covers a much wider field of view as compared with an 2D image of a relatively narrow field of view.
  • wide view HDR/WCG video or images/pictures exhibit relatively great diversity in different local regions within the same image/picture.
  • dynamic ranges and color distributions in different spatial regions of the same wide view HDR/WCG image may exhibit different characteristics. Codewords needed to avoid banding visual artifacts in one particular luminance range in one particular local area or spatial region of the wide view HDR/WCG image can be quite different from codewords needed to avoid banding visual artifacts in another particular luminance range in another particular local area or spatial region of the same wide view HDR/WCG image.
  • Global reshaping would subject all these spatial regions with diversely different luminance ranges to the same global reshaping mapping or function and cannot adequately or efficiently handle varying needs in these different spatial regions for different sets of codewords in reshaping operations.
  • local reshaping operations/methods may be implemented, performed or used to apply different local reshaping functions or mappings to different spatial regions of the same wide view image.
  • These local reshaping functions or mappings such as TPBC based reshaping functions or mappings may be determined or generated to meet respective needs of the different spatial regions of the same wide view image.
  • FIG.2A illustrates an example two-stage process or method for reference BL image generation, which may be implemented, for example, by a video encoder or processing blocks/components therein. This two-stage process or method can be used to help tackle or perform local reshaping operations by way of creating a reference wide view base layer images (or video signal) from an input or source wide view HDR/WCG images (or video signal).
  • the first stage is implemented with a processing block/component for LUT generation, which generates a respective lookup table (LUT) in each local patch – among a plurality of local (HDR/WCG) patches identified from an input or source wide view HDR/WCG image in the input or source wide view HDR/WCG images (or video signal) – using the BLKSTD method/algorithm.
  • Respective LUTs generated for the plurality of local (HDR/WCG) patches can be different for different local or spatial patches in the same input or source wide view HDR/WCG image.
  • LUTs can be applied to the plurality of local (HDR/WCG) patches in the input or source wide view HDR/WCG image into corresponding locally forward reshaped (BL) patches.
  • the second stage is implemented with a processing block/component for fusion, which fuses the locally forward reshaped (BL) patches to smooth out boundary or discontinuity artifacts along patch boundaries to generate a corresponding reference wide view BL image in the reference wide view base layer images (or video signal).
  • a respective patch based LUT (e.g., one-dimensional LUT or 1D- LUT for the luma channel, etc.) representing a local forward reshaping function for a local patch can be generated for each local patch in a plurality of local patches in the input or source wide view HDR/WCG image.
  • These local patches may represent relatively small areas or spatial regions in the input or source wide view HDR/WCG image and may be used to generate patch based LUTs.
  • a group of T (input or source wide view HDR/WCG) image/frames within the same scene may be processed or partitioned using the same set of local patches.
  • superscript C may be Cb or Cr to denote a Cb or Cr chroma channel in an input YCbCr color space or domain.
  • superscript C may be P or T to denote a P or T channel in an input IPT (e.g., IPTPQc2, etc.) color space or domain.
  • IPT e.g., IPTPQc2, etc.
  • An input or source wide view HDR/WCG image, or the t-th image/frame may be partitioned into a plurality of non-overlapped local patches each of which has a patch size ⁇ ⁇ ⁇ ⁇ ⁇ , where ⁇ ⁇ represents an integer greater than two (2).
  • the luma component may include the corresponding luma pixel values number of pixels ⁇ ⁇ ⁇ ⁇ ⁇ if the k-th non-overlapped patch is an interior patch in the t-th image/frame. If the k-th non-overlapped patch is next to or on a border/boundary of the image/frame, the patch may have fewer luma pixel values than the full size of ⁇ ⁇ ⁇ ⁇ ⁇ .
  • a non-luma component e.g., Cb or Cr chroma in an YCbCr color space, P or T in an IPT color space – or corresponding non-luma pixel values of the k- th non-overlapped patch in the t-th image/frame as as ⁇ ⁇ ⁇ , , ⁇ ⁇ .
  • the input or source wide view is represented in a 4:2:0 color space sampling format.
  • [0057] Denote the luma and non-luma pixel values at a (e.g., relative to the local patch, etc.) pixel location (m, n) in the k-th non-overlapped patch of the t-th image/frame, respectively, as ⁇ ⁇ ⁇ , ⁇ ( ⁇ , ⁇ ) and ⁇ ⁇ ⁇ , ⁇ ( ⁇ , ⁇ ), where m represents a (e.g., relative to the top left pixel or pixel location of the local patch, etc.) row index of the pixel or pixel location; and n represents a (e.g., relative to the top left pixel or pixel location of the local patch, etc.) column index of the pixel or pixel location.
  • a straightforward solution may be to construct respective patch-based 1D- LUTs for the non-overlapped patches in the input or source wide view HDR/WCG image and to generate a reference wide view base layer image by applying the BLKSTD method/algorithm to each non-overlapped patch.
  • relatively significant boundary artifacts can be observed occurring along the patch boundaries.
  • Those high-frequency visual artifacts can degrade reshaping accuracy, reduce video coding efficiency, and propagate those visual artifacts to a reconstructed or backward reshaped wide view HDR/WCG image.
  • an overlapped patch based method or algorithm as described herein may be implemented or performed to use overlapped patches to collect or gather additional neighboring patch data in the LUT generation and to further use overlapped patches to enable or provide a relatively smooth transition along patch boundaries in the reference wide view BL image generation.
  • overlapped patches for the LUT generation may be different from overlapped patches for the BL image generation.
  • the overlapped patches for the LUT generation may be of different sizes from sizes of overlapped patches for the BL image generation.
  • the t-th image may be partitioned into a plurality of non-overlapped local patches, for example arranged in a two-dimensional spatial array.
  • Two corresponding overlapped local patches may be defined or determined for each non-overlapped local patch in the plurality of non-overlapped local patches.
  • Pixel value statistics can be collected in the first of the two corresponding overlapped local patches to construct a local patch-based LUT, which can be applied to the second of the two corresponding overlapped local patches to help generate a patch- based portion of the reference wide view base layer image.
  • the first of the two corresponding local patches – assuming they are located in the interior of the t-th image – is represented by a first square or a first luma patch size ⁇ ⁇ ⁇ ⁇ ⁇ . This first overlapped local patch may be used for the LUT generation.
  • the second of the two corresponding local patches is represented by a second square or a second luma patch size ⁇ ⁇ ⁇ ⁇ ⁇ . This second overlapped local patch may be used for the BL image generation.
  • the patch sizes of the non-overlapped and overlapped local patches may satisfy an inequality as follows: ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ (1)
  • the total number of the non-overlapped local patches may be the same as the total number of the overlapped local patches for each of the LUT generation and the BL image generation and is given or specified by ⁇ ⁇ .
  • the four corners of the overlapped luma local patch for the LUT generation, ⁇ ⁇ ⁇ , , ⁇ ⁇ has a pixel distance ( ⁇ ⁇ ) horizontally and vertically to the corresponding four corners of the non-overlapped luma local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ .
  • the four corners of the overlapped luma local patch for the BL image generation, ⁇ ⁇ ⁇ , , ⁇ ⁇ have a pixel distance ( ⁇ ⁇ ⁇ ) horizontally and vertically to the corresponding four corners of ⁇ ⁇ ⁇ , , ⁇ ⁇ .
  • the four corners of ⁇ ⁇ ⁇ , , ⁇ ⁇ has pixel distance ( ⁇ ⁇ ) horizontally and vertically to the corresponding four corners of the non-overlapped luma local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ .
  • FIG.2C illustrates an example method or process for patch based forward reshaping function/mapping generation, which may be implemented, for example, by a video encoder or processing blocks/components therein.
  • the process flow can be implemented or performed to generate patch-based or patch-specific forward reshaping functions/mappings such as LUTs for forward reshaping patches extracted from input or source wide view HDR/WCG image(s).
  • the process flow comprises a processing block/component for patch extraction, which extracts overlapped HDR/WCG local patches for LUT generation at a plurality of different local patch locations from input or source wide view HDR/WCG image(s).
  • the process flow further comprises a processing block/component for LUT generation, which generates respective LUTs for forward reshaping HDR/WCG local patches at the plurality of different local patch locations in the input or source wide view HDR/WCG image(s) into corresponding BL local patches.
  • the corresponding BL local patches may be combined or fused into reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s).
  • a respective LUT such as a specific 1D mapping LUT can be constructed to prevent or reduce banding artifacts and preserve colors or color precision in each non-overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ of an input or source wide view HDR/WCG image.
  • a corresponding overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ can be used instead of the non-overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ to generate the LUT.
  • bit depth e.g., the number of bits used to encode a luma or non-luma pixel value or codeword, etc.
  • bit depth e.g., the number of bits used to encode a luma or non-luma pixel value or codeword, etc.
  • needed codewords to avoid or prevent banding artifacts may be estimated or computed with a function derived with a block-based standard deviation (BLKSTD) method, which measures block-based standard deviations (BLKSTDs) in multiple mutually exclusive luminance sub-ranges that make up the entire luminance range in the overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ of the input or source wide view HDR/WCG image. More specifically, in each luminance sub-range, pixels – in the overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ – whose pixel values or codewords fall into this considered luminance sub-range may be collected.
  • BLKSTD block-based standard deviation
  • the input or source wide view HDR/WCG image containing the overlapped local patch ⁇ ⁇ ⁇ , , ⁇ ⁇ belongs to a sequence of (e.g., consecutive, sequential, etc.) input or source wide view HDR/WCG in a video.
  • pixels or pixel values (or codewords) represented in corresponding overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ of a group of one or more input or source wide view HDR/WCG images within a scene may be collected across all images in the group and used to derive average BLKSTDs for different luminance sub-ranges that make up the entire luminance range represented in the corresponding overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ of the group of one or more input or source wide view HDR/WCG images.
  • Respective BLKSTDs in all the luminance sub-range can be used to construct a BLKSTD function that estimates or computes respective needed codewords for all the luminance sub-range represented in the overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ .
  • the LUT such as a 1D-LUT can be built using the BLKSTD function constructed from all the average BLKSTDs in all the luminance sub-ranges.
  • the processing block or component for the LUT generation may include optimizing the LUT derived from the BLKSTDs by identifying and re-using previously unused codewords in a codeword space and smoothing filtering to make the LUT or 1D-LUT relatively smooth or less discontinuous for the purpose of avoiding or reducing video compression artifacts.
  • One or both of the input HDR/WCG pixel values (or codewords) and the output base layer pixel values (or codewords) represented in the LUT may be normalized into a normalized value range [01].
  • the LUT – e.g., optimized and smoothened LUT – may be used as a forward reshaping mapping or function to convert the pixel values or codewords encoded in corresponding overlapped local patch(es) ⁇ ⁇ ⁇ , , ⁇ ⁇ of the input or source wide view HDR image – or in the group of the one or more input or source wide view HDR images within the same scene – to corresponding base layer pixel values or codewords in corresponding overlapped BL local patches.
  • the overlapped BL local patches may be used to construct corresponding reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s).
  • the maximum mapped or BL value may be set or normalized to the maximum value of an applicable normalized value range such as 1.
  • the minimum mapped or BL value may be set or normalized to the minimum value of the applicable normalized value range such as 0.
  • a codeword space that includes all available codewords comprises 2 %/ codewords.
  • the LUT may contain 2 %/ entries each of which maps or forward reshapes a respective input or source pixel value or codeword into a mapped (e.g., forward reshaped, etc.) BL pixel value or codeword as output.
  • the output value range may also be in [01], as the output BL signal is encoded with codewords or pixel values of ! # ( ⁇ !
  • FIG.3A illustrates two example distributions of BLKSTDs in two local patches, respectively, of the same input or source wide view HDR/WCG images. As illustrated, the distributions of BLKSTDs in different local patches of the same image can be widely different. Global reshaping under other approaches would significantly degrade reshaping operations and would likely fail to prevent banding artifacts and negatively impact video compression performance. In contrast, local reshaping as described herein applies different reshaping functions/mappings in different local patches even if they are from the same image.
  • the local reshaping operations can be used to effectively prevent or reduce banding artifacts as well as improve video compression performance.
  • the same LUT or the 1D-LUT may be applied to – for example to forward reshape luma component or pixel values/codewords in – each corresponding local patch of some or all of (e.g., a group of, consecutive, sequential, with different frame indexes, etc.) images with the same scene.
  • the LUT or 1D-LUT construction may be performed with respect to all like local patches at the same patch location in some or all images within the scene.
  • 5 0,1, ... . , .
  • the input values and/or the output values can be a range such as [01].
  • respective LUTs or 1D-LUTs constructed to represent non-linear forward reshaping functions/mappings using the BLKSTD method or algorithm can be applied to forward reshape input luma pixel values or codewords in local patches at some or all local patches of input or source wide view HDR/WCG image(s).
  • linear forward reshaping functions/mappings – such as first order polynomials, which can also be represented by LUTs such as 1D- LUTs – can be applied to forward reshape input non-luma pixel values or codewords in local patches at some or all local patches of input or source wide view HDR/WCG image(s).
  • LUTs such as 1D- LUTs –
  • 5 0,1, ... . , .
  • a LUT or 1D-LUT for mapping or forward reshaping chroma values may be a relatively simple first order polynomial or a first order function with a scaling factor and an offset.
  • the offset may be used to shift the center of a (e.g., valid, etc.) codeword range or sub-range to the middle of all codewords (for coding efficiency) in the codeword range or sub-range.
  • the slope or scaling factor may be chosen using an energy ratio, which is defined as a ratio specified with the previously determined maximum and minimum values in the luma and non-luma channels.
  • FIG.2D illustrates an example method or process flow for patch fusion, which may be implemented, for example, by a video encoder or processing blocks/components therein. The process flow can be implemented or performed to fuse overlapped patches together to generate reference wide view base layer image(s).
  • the process flow comprises a processing block/component for patch extraction, which extracts overlapped HDR/WCG local patches for reference generation at a plurality of different local patch locations from input or source wide view HDR/WCG image(s).
  • the process flow further comprises a processing block/component for applying LUT for reference (signal or image) generation, which applies respective LUTs to forward reshape HDR/WCG local patches for reference generation at the plurality of different local patch locations in the input or source wide view HDR/WCG image(s) into corresponding BL local patches for reference generation.
  • the process flow also comprises a processing block/component for weight map (for patches) creation, which creates a weight map for each of the BL local patches.
  • the process flow comprises a processing block/component for fusion, which fuses the BL local patches using respective weight maps for the BL local patches into overall reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s).
  • the LUTs can be first applied to corresponding overlapped local patches to generate forward reshaped BL local patches. Fusion operations may be performed on the forward reshaped local patches in the luma and chroma channels to generate the overall wide view BL image(s). It should be noted that the overlapped local patches for reference generation may be of different sizes from the corresponding overlapped local patches used to generate the LUTs.
  • Weighting Matrix To ensure relatively smooth transitions across patch boundaries, a weighting map with individual weighting factors for different pixels may be applied to the overlapped local patches used in image future. For a pixel which is covered by multiple overlapped local patches, its fused pixel value may be a weighed combination of pixel values from all the multiple overlapped patches multiplied with a normalization factor summed from weight factors used to generate the weighted combination.
  • a (luma channel) weighting map as described herein for an overlapped local patch – with a total number ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ of pixels in the luma channel – used for image fusion (in the luma channel) can be defined or specified with a ( ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ in the luma channel) weighting matrix comprising respective weighting factors for (luma pixel values or codewords of) all pixels represented in the overlapped local patch.
  • non-luma channel weighting maps or matrix for image fusion in the non-luma channels can be similarly defined or specified.
  • a weighting map or matrix may be specifically selected or configured such that a contour formed by equal values of a subset of the weighting factors in the weighting map or matrix for the overlapped local patch may be of the same or similar shape as compared with a spatial shape of the overlapped local patch such as square or rectangle, rather than an isotropic circular shape.
  • Each of these matrices in expressions (8) and (9) above specifies equal weights (or weighting factor values) along a rectangle contour, which is the same as or similar to the spatial shape of a corresponding overlapped local patch, with the highest weights at the center and non-center weights decreasing
  • These two matrices in expressions (8) and (9) above or their corresponding weighting maps may be fixed for each of some or all of the overlapped local patches used for image fusion, except for possibly boundary overlapped local patches which do not have full patch size.
  • FIG.3C illustrates example operations that apply patch-based weighting matrixes or maps, as defined in expressions (8) and (9) above, to mapped or forward reshaped overlapped local patches to generate an overall or fused reference wide view BL image corresponding to an input or source wide view HDR/WCG image.
  • the mapped or forward reshaped overlapped local patches may be generated from patch- based forward reshaping or LUT mapping operations corresponding input overlapped local patches derived from the input or source wide view HDR/WCG image.
  • the overall or fused reference wide view BL image depicts the same visual semantic content as – and may be in a lower bit depth or dynamic range than – the input or source wide view HDR/WCG image.
  • path-based or patch-specific mapped or forward reshaped pixel values or codewords in the luma channel as ⁇ o ⁇ ⁇ ,n .
  • path-based or patch-specific mapped or forward reshaped pixel values or codewords in the C non-luma channel as ⁇ o ⁇ ⁇ ,n .
  • path-based or patch-specific mapped or forward reshaped pixel values or codewords in either the luma channel or the C non-luma channel as ⁇ o ⁇ n .
  • Denote accumulated weights or weighting factors in the luma channel as ⁇ ⁇ ⁇ , ⁇ .
  • two memory spaces may be prepared – e.g., initiated to pixel values of all zeros (0), as represented by the two black rectangles on the left side of FIG.3C – with the same dimension (E ⁇ D) as that of the input or source wide view HDR/WCG image.
  • the first (to store accumulated pixel values or codewords ⁇ ⁇ n ; which may be simply referred to as the memory space ⁇ ⁇ n ) of the two memory spaces may be used to accumulate respective weighted patch-based reshaped pixel values or codewords for all pixels (to be) represented in the (overall or fused) reference wide view BL image.
  • the second (to store accumulated weights or weighting factors ⁇ ⁇ ⁇ ; which may be simply referred to as the memory space ⁇ ⁇ ⁇ ) of the two memory spaces may be used to accumulates respective weights or weighting factors for all the pixels.
  • the mapped or forward reshaped pixel values or codewords can be multiplied by weights or weighting factors in a corresponding patch-based weighting matrix. All weighted mapped pixel values or codewords at each pixel (location) covered by overlapped local patches can be accumulated in the corresponding pixel location in the memory space ⁇ ⁇ n .
  • weight maps for the luma and non-luma channels are fixed for some or all (input or reshaped) images.
  • the accumulated weights or weighting factors such as ⁇ ⁇ ⁇ , ⁇ and ⁇ ⁇ ⁇ , ⁇ are fixed for these images, and can be computed once, for example at the system boot up and then stored in its allocated memory space.
  • TPBC Tensor-Product B-Spline with Coordinate
  • a corresponding backward reshaping mapping/function corresponding to the forward reshaping mapping/function may be constructed to be used to backward reshape forward reshaped wide view BL image to generate a reconstructed or backward reshaped wide view HDR/WCG image that approximates the input or source wide view HDR/WCG image.
  • the forward and backward reshaping mappings or functions may be built or constructed as TPBC reshaping functions.
  • TPB provides a relatively high flexibility and accuracy for reshaping operations that may include dynamic range conversion and color mapping.
  • TPB forward and backward reshaping functions can be derived using the previously mentioned BESA method or algorithm, which builds a reversible paired of TPB transforms representing respectively a TPB forward reshaping function used to forward reshape the input or source wide view HDR/WCG image into the forward reshaped image and a corresponding TPB backward reshaping function to backward reshape the forward reshaped image to the reconstructed or backward reshaped image that relatively closely approximate (e.g., with minimized errors or differences, etc.) the input or source wide view HDR/WCG image.
  • a TPB reshaping function as described herein may use a combination of luma pixel values or codewords of pixels in an input (or to-be reshaped) image and their positional information such as pixel coordinates – or a positional functional or encoded form of the pixel coordinates – as inputs to predict (e.g., luma, etc.) pixel values or codewords of pixels in an output (or predicted) image as output.
  • the TPB reshaping function may use non-luma pixel values or codewords of the pixels in the input image as a part of the inputs in addition to or in place of some or all in the combination of the luma pixel values or codewords and the positional information.
  • pixel coordinates may be directly used as the positional information in the inputs to a TPB reshaping function.
  • a positional functional or encoded form of the pixel coordinates may be used as the positional information in the inputs to a TPB reshaping function.
  • the selection of specific type(s) of positional encoding functions can be based at least in part on a comparison of respective performances among some or all candidate positional encoding functions and a selection of specific positional encoding functions corresponding to the best performance.
  • the selected positional encoding functions can be signaled by image metadata provided by an upstream device to a downstream recipient device.
  • the encoded positional information ( ⁇ y , ⁇ z) may be used in reshaping (or prediction) functions as described herein in place of the pixel coordinates ( ⁇ , ⁇ ).
  • forward path TPB Optimization A process flow portion with forward reshaping operations may be referred to as a forward path herein.
  • forward reshaping mappings/functions takes input pixel values or codewords in input or source wide view HDR/WCG image(s) as input to predict or estimate output pixel values or codewords that collectively constitute forward reshaped wide view BL image(s) corresponding to – or depicting the same visual semantic content as – the input or source wide view HDR/WCG image(s).
  • a TPB forward reshaping mapping/function can be used to take (1) an input luma pixel value or codeword (denoted as ⁇ ⁇ ⁇ ( ⁇ , ⁇ ), where m and n represent pixel (location) coordinates of a pixel) of an (or t-th) input or source wide view HDR/WCG image, (2) input x-axis positional information represented by ⁇ y encoded from m, and (3) input y-axis positional information represented by ⁇ y encoded from n to predict an output luma pixel value or codeword (denoted as ⁇ s ⁇ ⁇ ( ⁇ , ⁇ )) in a (or t-th) forward reshaped wide view BL image for the luma channel in the forward path.
  • TPB basis functions and input pixel values or codewords of P pixels selected or collected from T input or source wide view HDR/WCG images within the same scene as follows: s ( ⁇ ⁇ %, ⁇ ù ⁇ q û frame index for the k-th pixel; (mk, nk) represent the pixel coordinates or encoded positional information for the k-th pixel.
  • a ground truth vector (denoted as s ⁇ ⁇ i ( ⁇ , ⁇ ) or s ⁇ ⁇ [ ( ⁇ , ⁇ )) for the chroma channel may also be constructed from channel pixel values or codewords of the corresponding P pixels in the corresponding (e.g., fused, etc.) reference wide view BL images within the same scene.
  • a matrix (denoted as ⁇ ⁇ ⁇ , where C is C0 or C1) or a vector (denoted as ⁇ ⁇ ⁇ ) for the chroma channel can be constructed similarly to the matrix or the vector for the luma channel in expressions (25) above.
  • optimized values ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ⁇ of TPB (prediction) coefficients can be obtained or represented as ⁇ with backward reshaping operations may be referred to as a backward path herein.
  • TPB optimization can be performed in the backward path similar to the foregoing TPB optimization in the forward path.
  • a TPB backward reshaping mapping/function can be used to take (1) an input luma pixel value or codeword (denoted as ⁇ ⁇ ⁇ ( ⁇ , ⁇ ), where m and n represent pixel (location) coordinates of a pixel) of an (or t-th) forward reshaped wide view BL image, (2) input x-axis positional information represented by ⁇ y encoded from m, and (3) input y-axis positional information represented by ⁇ y encoded from n to predict an output luma pixel value or codeword (denoted as ⁇ ⁇ ⁇ ( ⁇ , ⁇ )) in a (or t-th) reconstructed or backward reshaped wide view image for the luma channel in the backward path.
  • a ground truth vector (denoted as ⁇ ⁇ ) for measuring prediction errors of the predicted output luma pixel values or codewords ( ⁇ ⁇ ) may be constructed from the input or source luma pixel values or codewords of
  • a ground truth vector for the chroma channel – similar to the ground truth vector in expression (32) above – may also be constructed from corresponding chroma channel pixel values or codewords of the corresponding P pixels in the corresponding input or source wide view HDR/WCG images within the same scene.
  • a matrix or a vector for the chroma channel can be constructed similarly to the matrix or the vector for the luma channel in expressions (34) above.
  • images with local reshaping based at least in part on positional encoding can support a relatively high degree of color precision, thereby preventing, avoiding or reducing color shift in the reshaped or predicted images.
  • differences in PSNR measurements/values between reconstructed or backward reshaped images generated from the local reshaping with positional encoding and corresponding reconstructed or backward reshaped images generated from the global reshaping without positional encoding may be relatively significant (e.g., ⁇ 60dB, etc.) in favor of the local reshaping.
  • different types of positional encoding such as linear positional encoding, non-linear positional encoding with sine/cosine functions may yield different image quality measurements/values such as different PSNR measurements/values in reconstructed or backward reshaped images generated from the local reshaping performed with the different types of positional encoding.
  • a specific type among the different types of positional encoding corresponding to the best image quality measurements/values can be selected, implemented and/or signaled with image metadata to downstream recipient device(s) for the purpose of allowing the downstream recipient device(s) to generate relatively high quality reconstructed or backward reshaped images.
  • forward reshaping may be performed on an input or source wide view HDR/WCG signal or images therein to make forward reshaped output wide view BL signal or images therein as close to (e.g., fused, etc.) reference wide view BL images as possible (e.g., with minimized prediction errors, etc.).
  • backward reshaping may be performed on the forward reshaped output wide view BL signal or images therein to make reconstructed or backward reshaped output wide view HDR/WCG signal or images therein as close to the input or source wide view HDR/WCG signal or images therein as possible (e.g., with minimized prediction errors, etc.).
  • an iterative algorithm such as the BESA algorithm may be used to generate optimized values for prediction/reshaping operations such as TPB prediction/reshaping operations.
  • the algorithm can modify the reference BL signal iteratively, such that prediction errors between the (final; post-iteration) reconstructed or backward reshaped output wide view HDR/WCG signal or images therein and the input or source wide view HDR/WCG signal or images therein are minimized.
  • the BESA algorithm can be deployed or performed in each of some or all (e.g., three, etc.) color channels of a color space.
  • these color channels may be luma and chroma channels and denoted as ch, which can be Y, Cb, and Cr.
  • a superscript, ⁇ may be added as the iteration index in (1) the TPB forward reshaping function ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ⁇ , (2) the TPB backward reshaping function ⁇ ( ⁇ %, ⁇ ,( ⁇ ) % , and (3) the F forward reshaped wide view base layer images ⁇ ⁇ ,( ⁇ ) ⁇ within ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ⁇ p ⁇ ⁇ ⁇ ⁇ ( ⁇ , ⁇ ), ⁇ y, ⁇ zq [0148]
  • the same superscript, ⁇ may be added as the iteration index in the corresponding TPBC coefficient matrices ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ( ⁇ %, ⁇ ,( ⁇ ) ⁇ and ⁇ % .
  • FIG.2E illustrates example process flow implementing the BESA algorithm to optimize reshaping mappings/functions, which may be implemented, for example, by a video encoder or processing blocks/components therein.
  • the process flow comprises a processing block/component for initialization (not shown in FIG.
  • the process flow for the BESA algorithm comprises a processing block/component for forward reshaping, which at each iteration, generates or optimizes the TPB (forward reshaping) coefficients or parameters ⁇ ( ⁇ %, ⁇ ,( ⁇ ) ( ⁇ %, ⁇ , ⁇ in the forward reshaping function ⁇ ( ⁇ ) ⁇ to minimize prediction errors or differences between forward reshaped wide view BL pixel values or codewords ⁇ s ⁇ ⁇ ⁇ ,( ⁇ ) ( ⁇ , ⁇ ) at the current iteration and the modified reference wide view BL pixel
  • This error amount ⁇ ⁇ ⁇ ⁇ ,( ⁇ ) ( ⁇ , ⁇ ) may be used to modify or update the (current) modified reference wide view BL pixel values or codewords for the entire set ⁇ into the modified reference wide view BL pixel values or codewords s o ⁇ ,( ⁇ a[) ⁇ ( ⁇ , ⁇ ) for the next iteration, and may be a reduced amount as compared with
  • pixel values or codewords of all available pixels of each of all images in a scene may be used to generate optimized values for TPBC coefficients in TPB reshaping functions/mappings.
  • a process flow under this approach may use relatively large amounts of memory and computing resources.
  • pixel values or codewords of a relatively small subset of all available pixels of each of all images in a scene may be used to generate optimized values for TPBC coefficients in TPB reshaping functions/mappings.
  • a process flow under this approach may use relatively limited amounts of memory and computing resources and may contain multiple stages such as three stages illustrated in FIG.2F.
  • TPBC reshaping functions can be generated or optimized by fitting or modeling on an incremental data set, which includes an addition of the most distorted pixels (e.g., as measured or compared with an applicable distortion threshold, etc.) from a previous stage.
  • TPBC reshaping functions are generated or optimized by fitting or modeling on a first data set comprising pixel values or codewords in down-sampled images derived from input or source wide view HDR/WCG images ⁇ ⁇ ⁇ ⁇ in a scene for a color channel ch such as a luma or non-luma channel, non-luma as well as derived from corresponding (fused) wide view BL images s ⁇ ⁇ ⁇ in the scene for a corresponding color channel ch such as the same luma or non-luma channel with a different bit depth or dynamic ranges.
  • FIG.2G illustrates an example detailed process flow corresponding to stage 1 of the process flow of FIG.2F.
  • the process flow of FIG.2G in stage 1 comprises a processing block or component for downsampling, which (e.g., uniformly, etc.) downsamples ⁇ ′ ⁇ ⁇ ⁇ by a constant factor denoted as ⁇ ⁇ (e.g., a prime number, 101, etc.).
  • ⁇ ⁇ e.g., a prime number, 101, etc.
  • FIG.2G in stage 1 comprises a processing block or component or performs a first round of BESA optimization using pixel set ⁇ [ to determine TPB coefficients for constructing TPB forward and backward reshaping functions/mappings denoted respectively as ⁇ ⁇ ( , o ⁇ % ⁇ and ⁇ % ( , ⁇ o % ⁇ , as follows: [) a processing block or component denoted as “Apply TPBC 1”, which applies the TPB forward reshaping function ⁇ ⁇ ( , o ⁇ % ⁇ to the input or source wide view HDR/WCG images ⁇ ⁇ ⁇ ⁇ to obtain or generate
  • (1 ⁇ 4 ⁇ r ⁇ ⁇ ⁇ (£) ⁇ ⁇ ⁇ r ⁇ , , ⁇ o ⁇ ⁇ (£)1 ⁇ 4 ⁇ ⁇ 3 ⁇ 4 ⁇ , o ⁇ ⁇ ) ⁇ O( 1 ⁇ 4s r ⁇ ⁇ ⁇ (£) ⁇ ⁇ s r ⁇ , , ⁇ o ⁇ ⁇ (£)1 ⁇ 4 ⁇ ⁇ ' , ⁇ o ⁇ ) ⁇ ⁇ ⁇ r ⁇ , , ⁇ o ⁇ ⁇ 3h ⁇ £ ⁇ _3_ ⁇ h ⁇ ( ⁇ ⁇ ⁇ ⁇ , ⁇ o ⁇ ) ) 3 ⁇ 4 ⁇
  • FIG.2H illustrates an example detailed process flow corresponding to stage 2 of the process flow of FIG.2F.
  • the process flow of FIG.2H in stage 2 comprises processing blocks or components similar to those in the process flow of FIG.2G in stage 1.
  • a second data set ⁇ ⁇ may be in BESA operations.
  • (1 ⁇ 4 ⁇ r ⁇ ⁇ ⁇ (£) ⁇ ⁇ ⁇ r ⁇ , , ⁇ o ⁇ l (£)1 ⁇ 4 ⁇ ⁇ 3 ⁇ 4 ⁇ , ⁇ o l ) ⁇ O( 1 ⁇ 4s r ⁇ ⁇ ⁇ (£) ⁇ ⁇ s r ⁇ , , ⁇ o ⁇ l (£)1 ⁇ 4 ⁇ ⁇ ' ⁇ , ⁇ o l ) ⁇ where ⁇ ⁇ r ⁇ , , ⁇ o ⁇ l 3h ⁇ £ ⁇ _3_ ⁇ h ⁇ ( ⁇ ⁇ ⁇ ⁇ , ⁇ o l )
  • FIG.2I illustrates an example detailed process flow corresponding to stage 3 of the process flow of FIG.2F.
  • the process flow of FIG.2I in stage 3 comprises processing blocks or components similar to those in the process flow of FIG.2G or FIG.2H in stage 1 or 2.
  • a third data set ⁇ ⁇ may be in BESA operations.
  • Other processing blocks or components in stage 3 may be performed with this second data set ⁇ ⁇ similar to show the corresponding processing blocks or components in stage 2 is performed with the second data set ⁇ ⁇ .
  • more or fewer stages other than three stages may be implemented.
  • the 3-stage optimization as described herein can generate images with image qualities, for example as measured with PSNR values illustrated in TABLE 3 below, better than or comparable to all- or full-pixel optimization that utilizes all (non-downsampled) pixels or pixel values.
  • the 3- stage optimization can be implemented or perform to reduce memory usage as well as provide better reconstruction quality.
  • FIG.4A illustrates an example process flow according to an embodiment.
  • one or more computing devices or components e.g., an encoding device/module, a transcoding device/module, an upstream device, a sender, etc. may perform this process flow.
  • an image processing system generates one or more reference wide view images of a first domain (e.g., a first color space, a first dynamic range, a first bit depth image/signal, etc.) from one or more source wide view images of a second domain (e.g., a second color space, a second dynamic range, a second bit depth image/signal, etc.).
  • a first domain e.g., a first color space, a first dynamic range, a first bit depth image/signal, etc.
  • a second domain e.g., a second color space, a second dynamic range, a second bit depth image/signal, etc.
  • the image processing system generates a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain.
  • Each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data.
  • the image processing system encodes the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images.
  • the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings.
  • the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth;
  • the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth.
  • the positional data is derived from the pixel locations using one of a linear function or a non-linear function.
  • the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch-specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images; the patch-specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non- overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain.
  • a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel.
  • one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches.
  • at least one of the forward and backward reshaping mappings is optimized using a Backward Error Subtraction Algorithm (BESA) algorithm.
  • BESA Backward Error Subtraction Algorithm
  • At least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images.
  • the pixels in the downsampled pixel set are downsampled from a sequence of pixel rows, ordered in a lexicographical order, in the one or more source wide view images and the one or more reference wide view images.
  • At least one of the forward and backward reshaping mappings is optimized using multiple stages; in each of the multiple stages, a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a proper set of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images.
  • BESA Backward Error Subtraction Algorithm
  • at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images.
  • FIG.4B illustrates an example process flow according to an embodiment.
  • one or more computing devices or components may perform this process flow.
  • an image processing system decodes one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream.
  • the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping.
  • the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping.
  • the image processing system generates one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping. [0203] In block 456, the image processing system generates one or more display images from the one or more reconstructed wide view images. [0204] In block 458, the image processing system renders the one or more display images on an image display. [0205] In an embodiment, the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display.
  • a computing device such as a display device, a mobile device, a set-top box, a multimedia device, etc.
  • an apparatus comprises a processor and is configured to perform any of the foregoing methods.
  • a non- transitory computer readable storage medium storing software instructions, which when executed by one or more processors cause performance of any of the foregoing methods.
  • a computing device comprising one or more processors and one or more storage media storing a set of instructions which, when executed by the one or more processors, cause performance of any of the foregoing methods.
  • Embodiments of the present invention may be implemented with a computer system, systems configured in electronic circuitry and components, an integrated circuit (IC) device such as a microcontroller, a field programmable gate array (FPGA), or another configurable or programmable logic device (PLD), a discrete time or digital signal processor (DSP), an application specific IC (ASIC), and/or apparatus that includes one or more of such systems, devices or components.
  • IC integrated circuit
  • FPGA field programmable gate array
  • PLD configurable or programmable logic device
  • DSP discrete time or digital signal processor
  • ASIC application specific IC
  • the computer and/or IC may perform, control, or execute instructions relating to the adaptive perceptual quantization of images with enhanced dynamic range, such as those described herein.
  • the computer and/or IC may compute any of a variety of parameters or values that relate to the adaptive perceptual quantization processes described herein.
  • the image and video embodiments may be implemented in hardware, software, firmware and various combinations thereof.
  • Certain implementations of the inventio comprise computer processors which execute software instructions which cause the processors to perform a method of the disclosure. For example, one or more processors in a display, an encoder, a set top box, a transcoder or the like may implement methods related to adaptive perceptual quantization of HDR images as described above by executing software instructions in a program memory accessible to the processors.
  • Embodiments of the invention may also be provided in the form of a program product.
  • the program product may comprise any non-transitory medium which carries a set of computer- readable signals comprising instructions which, when executed by a data processor, cause the data processor to execute a method of an embodiment of the invention.
  • Program products according to embodiments of the invention may be in any of a wide variety of forms.
  • the program product may comprise, for example, physical media such as magnetic data storage media including floppy diskettes, hard disk drives, optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, flash RAM, or the like.
  • the computer-readable signals on the program product may optionally be compressed or encrypted.
  • the special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques.
  • the special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques.
  • FIG.5 is a block diagram that illustrates a computer system 500 upon which an embodiment of the invention may be implemented.
  • Computer system 500 includes a bus 502 or other communication mechanism for communicating information, and a hardware processor 504 coupled with bus 502 for processing information.
  • Hardware processor 504 may be, for example, a general purpose microprocessor.
  • Computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 502 for storing information and instructions to be executed by processor 504.
  • Main memory 506 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 504.
  • Computer system 500 further includes a read only memory (ROM) 508 or other static storage device coupled to bus 502 for storing static information and instructions for processor 504.
  • ROM read only memory
  • a storage device 510 such as a magnetic disk or optical disk, is provided and coupled to bus 502 for storing information and instructions.
  • Computer system 500 may be coupled via bus 502 to a display 512, such as a liquid crystal display, for displaying information to a computer user.
  • An input device 514 is coupled to bus 502 for communicating information and command selections to processor 504.
  • cursor control 516 such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 504 and for controlling cursor movement on display 512.
  • This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
  • Computer system 500 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 500 to be a special-purpose machine.
  • the techniques as described herein are performed by computer system 500 in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage device 510. Execution of the sequences of instructions contained in main memory 506 causes processor 504 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.
  • storage media refers to any non-transitory media that store data and/or instructions that cause a machine to operation in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media.
  • Non-volatile media includes, for example, optical or magnetic disks, such as storage device 510.
  • Volatile media includes dynamic memory, such as main memory 506.
  • Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.
  • Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media.
  • transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 502.
  • Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
  • Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 504 for execution.
  • the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer.
  • the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
  • a modem local to computer system 500 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal.
  • An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 502.
  • Bus 502 carries the data to main memory 506, from which processor 504 retrieves and executes the instructions.
  • the instructions received by main memory 506 may optionally be stored on storage device 510 either before or after execution by processor 504.
  • Computer system 500 also includes a communication interface 518 coupled to bus 502.
  • Communication interface 518 provides a two-way data communication coupling to a network link 520 that is connected to a local network 522.
  • communication interface 518 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line.
  • ISDN integrated services digital network
  • communication interface 518 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
  • LAN local area network
  • Network link 520 typically provides data communication through one or more networks to other data devices.
  • network link 520 may provide a connection through local network 522 to a host computer 524 or to data equipment operated by an Internet Service Provider (ISP) 526.
  • ISP 526 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 528.
  • Internet 528 uses electrical, electromagnetic or optical signals that carry digital data streams.
  • Computer system 500 can send messages and receive data, including program code, through the network(s), network link 520 and communication interface 518.
  • a server 530 might transmit a requested code for an application program through Internet 528, ISP 526, local network 522 and communication interface 518.
  • the received code may be executed by processor 504 as it is received, and/or stored in storage device 510, or other non-volatile storage for later execution.
  • a method comprising: generating one or more reference wide view images of a first domain from one or more source wide view images of a second domain; generating a forward reshaping mapping to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain; generating a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain; wherein each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data; encoding the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mapping
  • EEE2 The method of EEE1, wherein the forward reshaping mapping receives source pixel values of the one or more source wide view images and source positional data derived from source pixel coordinates as input and generates forward reshaped pixel values of the one or more forward reshaped wide view images.
  • EEE3 The method of EEE1 or EEE2, wherein the backward reshaping mapping receives forward reshaped pixel values of the one or more forward reshaped wide view images and source positional data derived from source pixel coordinates as input and generates backward reshaped pixel values of the one or more reconstructed wide view images.
  • EEE4 The method of EEE1, wherein the forward reshaping mapping receives source pixel values of the one or more source wide view images and source positional data derived from source pixel coordinates as input and generates backward reshaped pixel values of the one or more reconstructed wide view images.
  • the forward reshaping mapping includes a chroma forward reshaping mapping that maps source chrominance pixel values of the one or more source wide view images into forward reshaped chrominance pixel values of the one or more forward reshaped wide view images based at least in part on a scaling factor; wherein the scaling factor represents a ratio of a first difference between maximum and minimum source luminance pixel values and a second difference between maximum and minimum source chrominance pixel values.
  • EEE6 The method of any of EEE1-EEE4, wherein the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth; the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth.
  • EEE6 The method of any of EEE1-EEE5, wherein the positional data is derived from the pixel locations using one of a linear function or a non-linear function.
  • EEE7 The method of any of EEE1-EEE6, wherein the non-linear function is from a sinusoidal function family.
  • EEE8 The method of any of EEE8.
  • any of EEE1-EEE7 wherein the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch-specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images; wherein the patch- specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; wherein the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain.
  • EEE9 The method of any of EEE1-EEE8, wherein a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel.
  • EEE10 The method of EEE9, wherein the weighted combination is derived from weighting factors that are given by Gaussian distribution functions with rectangular contours of equal values.
  • EEE11 The method of EEE9, wherein one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches.
  • EEE13 The method of any of EEE1-EEE12, wherein at least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images.
  • BESA Backward Error Subtraction Algorithm
  • EEE15 The method of any of EEE1-EEE14, wherein at least one of the forward and backward reshaping mappings is optimized using multiple stages; wherein the multiple stages include a beginning stage in which a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a first proper subset of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images.
  • BESA Backward Error Subtraction Algorithm
  • EEE15 wherein a first set of pixels with relatively large prediction errors is identified after the BESA algorithm in the beginning stage has been performed; wherein the multiple stages include a second stage, following the beginning stage, in which the BESA algorithm is performed based on a second proper subset of pixels derived as a set union of the first proper subset of pixels and the first set of pixels with the relatively large prediction errors.
  • EEE16 wherein a second set of pixels with relatively large prediction errors is identified after the BESA algorithm in the second stage has been performed; wherein the multiple stages include a third stage, following the second stage, in which the BESA algorithm is performed based on a third proper subset of pixels derived as a second set union of the second proper subset of pixels and the second set of pixels with the relatively large prediction errors.
  • EEE18 The method of any of EEE1-EEE17, wherein at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images.
  • a method comprising: decoding one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream, wherein the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping; generating one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping; generating one or more display images from the one or more reconstructed wide view images; rendering the one or more display images on an image display.
  • EEE20 The method of EEE19, wherein the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display.
  • EEE21 An apparatus comprising a processor and configured to perform any one of the methods recited in EEE1-EEE20.
  • EEE22 A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with any of the methods recited in EEE1-EEE20.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Reference wide view images of a first domain are generated from source wide view images of a second domain. A forward reshaping mapping is generated to forward reshape the source wide view images into forward reshaped wide view images of the first domain. A backward reshaping mapping is generated to backward reshape the forward reshaped wide view images into reconstructed wide view images of the second domain. Each of the reshaping mappings is generated based on pixel level image data and positional data derived from pixel locations. The forward reshaped wide view images and corresponding image metadata are encoded into a bitstream to enable a recipient device of the bitstream to generate display images from the reconstructed wide view images. The corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings.

Description

LOCAL RESHAPING USING TENSOR-PRODUCT B-SPLINE WITH COORDINATES WIDE VIEW VIDEO CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefit of priority from U.S. Provisional Application No. 63/485,427, filed on 16 February 2023, and European Patent Application No.23156989.8, filed on 16 February 2023, each of which is incorporated by reference herein in its entirety. TECHNOLOGY [0002] The present disclosure relates generally to image processing operations. More particularly, an embodiment of the present disclosure relates to video codecs. BACKGROUND [0003] As used herein, the term “dynamic range” (DR) may relate to a capability of the human visual system (HVS) to perceive a range of intensity (e.g., luminance, luma) in an image, e.g., from darkest blacks (darks) to brightest whites (highlights). In this sense, DR relates to a “scene-referred” intensity. DR may also relate to the ability of a display device to adequately or approximately render an intensity range of a particular breadth. In this sense, DR relates to a “display-referred” intensity. Unless a particular sense is explicitly specified to have particular significance at any point in the description herein, it should be inferred that the term may be used in either sense, e.g. interchangeably. [0004] As used herein, the term high dynamic range (HDR) relates to a DR breadth that spans the some 14-15 or more orders of magnitude of the human visual system (HVS). In practice, the DR over which a human may simultaneously perceive an extensive breadth in intensity range may be somewhat truncated, in relation to HDR. As used herein, the terms enhanced dynamic range (EDR) or visual dynamic range (VDR) may individually or interchangeably relate to the DR that is perceivable within a scene or image by a human visual system (HVS) that includes eye movements, allowing for some light adaptation changes across the scene or image. As used herein, EDR may relate to a DR that spans 5 to 6 orders of magnitude. While perhaps somewhat narrower in relation to true scene referred HDR, EDR nonetheless represents a wide DR breadth and may also be referred to as HDR. [0005] In practice, images comprise one or more color components (e.g., luma Y and chroma Cb and Cr) of a color space, where each color component is represented by a precision of n-bits per pixel (e.g., n=8). Using non-linear luminance coding (e.g., gamma encoding), images where n ≤ 8 (e.g., color 24-bit JPEG images) are considered images of standard dynamic range, while images where n > 8 may be considered images of enhanced dynamic range. [0006] A reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between color values (e.g., luminance) of an input video signal to output screen color values (e.g., screen luminance) produced by the display. For example, ITU Rec. ITU-R BT.1886, “Reference electro-optical transfer function for flat panel displays used in HDTV studio production,” (March 2011), which is incorporated herein by reference in its entirety, defines the reference EOTF for flat panel displays. Given a video stream, information about its EOTF may be embedded in the bitstream as (image) metadata. The term “metadata” herein relates to any auxiliary information transmitted as part of the coded bitstream and assists a decoder to render a decoded image. Such metadata may include, but are not limited to, color space or gamut information, reference display parameters, and auxiliary signal parameters, as those described herein. [0007] The term “PQ” as used herein refers to perceptual luminance amplitude quantization. The human visual system responds to increasing light levels in a very nonlinear way. A human’s ability to see a stimulus is affected by the luminance of that stimulus, the size of the stimulus, the spatial frequencies making up the stimulus, and the luminance level that the eyes have adapted to at the particular moment one is viewing the stimulus. In some embodiments, a perceptual quantizer function maps linear input gray levels to output gray levels that better match the contrast sensitivity thresholds in the human visual system. An example PQ mapping function is described in SMPTE ST 2084:2014 “High Dynamic Range EOTF of Mastering Reference Displays” (hereinafter “SMPTE”), which is incorporated herein by reference in its entirety, where given a fixed stimulus size, for every luminance level (e.g., the stimulus level, etc.), a minimum visible contrast step at that luminance level is selected according to the most sensitive adaptation level and the most sensitive spatial frequency (according to HVS models). [0008] Displays that support luminance of 200 to 1,000 cd/m2 or nits typify a lower dynamic range (LDR), also referred to as a standard dynamic range (SDR), in relation to EDR (or HDR). EDR content may be displayed on EDR displays that support higher dynamic ranges (e.g., from 1,000 nits to 5,000 nits or more). Such displays may be defined using alternative EOTFs that support high luminance capability (e.g., 0 to 10,000 or more nits). An example of such an EOTF is defined in SMPTE 2084 and Rec. ITU-R BT.2100, “Image parameter values for high dynamic range television for use in production and international programme exchange,” (06/2017). As appreciated by the inventors here, improved techniques for generating high quality video content data with high dynamic range, high local contrast and vivid color are desired. [0009] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, issues identified with respect to one or more approaches should not assume to have been recognized in any prior art on the basis of this section, unless otherwise indicated.
BRIEF DESCRIPTION OF THE DRAWINGS [0010] An embodiment of the present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which: [0011] FIG.1A and FIG.1B illustrate example encoder side and decoder side architectures; [0012] FIG.2A illustrates an example process or method for reference base layer image generation; FIG.2B illustrates example overlapped and non-overlapped patches; FIG.2C illustrates example patch based forward reshaping mapping generation; FIG.2D illustrates an example method or process flow for patch fusion; FIG.2E through FIG.2I illustrates example process flows for optimizing reshaping mappings; [0013] FIG.3A illustrates example distributions of block based standard deviations; FIG.3B illustrates an example weighting map; FIG.3C illustrates example operations in connection with patch-based weighting maps; [0014] FIG.4A and FIG.4B illustrate example process flows; and [0015] FIG.5 illustrates a simplified block diagram of an example hardware platform on which a computer or a computing device as described herein may be implemented.
DESCRIPTION OF EXAMPLE EMBODIMENTS [0016] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, that the present disclosure may be practiced without these specific details. In other instances, well-known structures and devices are not described in exhaustive detail, in order to avoid unnecessarily occluding, obscuring, or obfuscating the present disclosure. Summary [0017] From virtual reality (VR) to robotics, innovative applications exploiting omnidirectional images and videos are expected to become more and more widespread in the future. Fully omni-directional cameras, able to capture up to 360- degree real-world scenes, have recently started to appear in products and professional tools. User-generated and professional wide view (e.g., greater than a viewer’s entire vision field, greater than a user’s field of view, greater than 90-degrees, 360-degree, etc.) content is already being distributed using popular content sharing platforms. Hence, there is an increasing need to support relatively wide view video such as up to 360-degree HDR video. As compared with relatively narrow view video such as 2D SDR or HDR video (e.g., less than 45 degrees, less than a viewer’s field of vision, etc.) in a traditional format with a relatively narrow field of view (FOV), the wide view video often has much larger image dimensions, contains much richer spatial information from up to all front and/or rear viewing angles not necessarily limited to only within an individual viewer’s vision field, and includes a much higher pixel count in each image. As a result, the wide view video can exhibit much higher dynamic ranges and much wider color gamuts as well as much larger spatial variations of dynamic ranges and color gamuts in different spatial locations of a wide view scene/image, and therefore present a much greater challenge in terms of supporting a widely diverse range of local luminances and color distributions in a relatively wide field of view (FOV) as compared with the 2D SDR or HDR video. Many existing video codecs that support or adopt a global reshaping method in processing the 2D SDR or HDR video may be ill equipped to handle much higher dynamic range (HDR) and wider color gamut (WCG) in wide view video content in a compression efficient manner. [0018] Fused patch-based local reshaping techniques as described herein can be implemented to support wide view video efficiently and effectively, and to resolve issues and handle challenges associated with the wide view video. Under these techniques, a reference base layer (BL) signal can be generated from original (input or source) wide view video using a fused patch-based algorithm/method to address or preserve local dynamic range and color distribution. Some or all of these techniques can operate with wide view video represented in an Equi-Rectangular Projection (ERP) format or other formats different from the ERP forma. [0019] More specifically, intermediate reference BL video content may be first generated patch-wisely to provide sufficient codewords for preventing banding visual artifacts and preserving colors or color precisions in various local regions. Multiple neighboring patches represented by the intermediate reference BL video content can be fused together to ensure smooth transitions crossing boundaries between or among the neighboring patches to generate the (final fused) reference BL signal. [0020] This (final fused) reference BL signal can be used to increase video compression efficiency globally and preserve HDR/WCG properties of the original or source wide view video locally in a reconstructed HDR signal, in order to help prevent highlight/dark area clipping, alleviate banding visual artifacts and achieve color fidelity in the reconstructed HDR signal. For example, a relatively compression- efficient and revertible forward reshaped BL signal can be generated by available video codecs on the encoder side using a forward reshaping function to approximate the reference BL signal. The reconstructed HDR signal can be generated or constructed by available video codecs on the decoder side from the forward reshaped BL signal using a backward reshaping function corresponding to the forward reshaping function. [0021] Additionally, optionally or alternatively, highly varying characteristics of local dynamic range and local color distribution in wide view HDR/WCG video data can be efficiently and effectively addressed or preserved Tensor-Product B-Spline with Coordinates (TPB with Coordinates or TPBC). With TPB or TPBC, banding artifact alleviation or prevention in HDR video content and color preservation in WCG video content represented in the wide view HDR/WCG video data can be realized under techniques as described herein in a more local manner as compared with global reshaping based approaches. [0022] TPB is a tool to model cross-channel complex mapping or reshaping functions. B-splines or basis splines can be used as functions to fit a given one dimensional curve using polynomial functions with continuity constraints at knot points. In a TPB framework, multiple B-spline functions can be fused together by (tensor) multiplication to approximate, estimate or fit higher dimensional curves while maintaining smooth connectivity at knot points. Example TPB reshaping functions are described in U.S. Provisional Application Ser. No.62/908,770, titled “TENSOR- PRODUCT B-SPLINE PREDICTOR,” filed on October 1, 2019, which are incorporated by reference in its entirety as if fully set forth herein. Example BESA algorithm/method can be found in U.S. Provisional Patent Application Ser. No. 63/013,063, “Reshaping functions for HDR imaging with continuity and reversibility constraints,” filed on April 21, 2020; U.S. Provisional Patent Application Ser. No. 63/013,807, “Iterative optimization of reshaping functions in single-layer HDR image codec,” filed on April 22, 2020; PCT Application Ser. No. PCT/US2021/028475, filed on April 21, 2021, the contents of all of which are entirely incorporated herein by reference as if fully set forth herein. Spatial information such as spatial coordinates can be incorporated into the TPB framework or tool to address or model local or spatial diversity in wide view video. [0023] In some operational scenarios, TPB or TPBC as described herein can be implemented with a Backward Error Subtraction Algorithm (BESA) to reach or achieve a relatively high degree of revertability between a pair of corresponding forward and backward reshaping functions. [0024] In some operational scenarios, algorithms or methods as described herein can be developed or implemented in a manner that reduces memory footage and computational load. By way of example but not limitation, a three-stage optimization may be implemented in an adaptive algorithm that applies or uses incremental datasets to minimize prediction errors. A relatively small bit depth domain (e.g., SDR, etc.) image may be transformed into a relatively large bit depth domain (e.g., HDR, WCG, etc.) image with a relatively small model, for example, built with a relatively small set of pixels selected from among all pixels represented in the images. Optimized operational parameters derived with the adaptive algorithm can be used to generate reconstructed wide view HDR/WCG video content that is visually lossless to the original or source wide view HDR/WCG video content. [0025] Example embodiments described herein relate to encoding coded packets relating to source images. One or more reference wide view images of a first domain are generated from one or more source wide view images of a second domain. A forward reshaping mapping is generated to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain. A backward reshaping mapping is generated to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain. Each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data. The one or more forward reshaped wide view images and corresponding image metadata are encoded into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images. The corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings. In some operational scenarios, the corresponding image metadata includes operational parameters specifying the backward reshaping mappings, but does not include operational parameters specifying the forward reshaping mappings which are applied at the encoder side only. [0026] Example embodiments described herein relate to decoding coded packets relating to reconstructed images. One or more forward reshaped wide view images of a first domain and corresponding image metadata are decoded from a bitstream. The one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping. The corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping. One or more reconstructed wide view images of the second domain are generated from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping. One or more display images are generated from the one or more reconstructed wide view images. The one or more display images are rendered on an image display. Fused Patch-Based Local Reshaping Framework [0027] A framework for fused patch-based local reshaping under techniques as described herein includes both encoder side and receiver side. [0028] FIG.1A illustrates an example encoder side architecture that may be implemented with one or more computing devices. The encoder side architecture may include a plurality of processing blocks or components to receive input or source wide view HDR/WCG images (or image frames); to use the received wide view HDR/WCG images/frames to generate reference BL images/frames; to generate forward reshaping functions to forward reshape the received wide view HDR/WCG images/frames to generate forward reshaped BL images approximating the reference BL images/frames; to encode the forward reshaped BL images along with image metadata specifying backward reshaping functions corresponding to the forward reshaping functions in a coded bitstream; to deliver or transmit the coded bitstream from an upstream device (e.g., some or all of the one or more computing devices implementing the encoder side, etc.) to a downstream device over data communication links or paths between the upstream device and the downstream device; etc. [0029] In some operational scenarios, some or all of these foregoing operations may be performed by the one or more computing devices at the encoder side on line in real time, for example while the upstream device communicates the bitstream or a portion thereof to the downstream device. In some operational scenarios, at least some of the foregoing operations may be performed by the one or more computing devices at the encoder side off line in non-real time, for example before the upstream device communicates any portion of the bitstream to the downstream device. [0030] As shown in FIG.1A, the encoder side architecture may include a processing block/component for reference BL generation, which takes the input or source wide view HDR/WCG images (or image frames) as inputs and use the input wide view HDR/WCG images/frames to generate or derive corresponding BL images to serve as reference for forward reshaped BL images to approximate. The forward reshaped BL images can be encoded in the coded bitstream with relatively high coding efficiency and subsequently decoded and used by a recipient device to generate reconstructed wide view HDR/WCG images approximate the input or source wide view HDR/WCG in a manner that avoids banding visual artifacts and preserves colors in each local (spatial) region of some or all local (spatial) regions in the reconstructed wide view HDR/WCG images corresponding to some or all local (spatial) regions represented in the input or source wide view HDR/WCG. [0031] The encoder side architecture may include a processing block/component for forward/backward TPBC optimization, which generates forward reshaping functions and backward reshaping functions corresponding to the forward reshaping functions using the input wide view HDR/WCG images and the reference BL images as input. The forward/backward TPBC optimization (1) minimizes differences between the reference BL images and forward reshaped BL images (such that the banding prevention and color preservation can be maintained as well as underlying video compression efficiency can be achieved or improved), and (2) minimizes distortions between the reconstructed wide view HDR/WCG images and the original wide view HDR/WCG images. [0032] The encoder side architecture may include a processing block/component for forward reshaping, which uses the forward reshaping functions to forward reshape the input wide view HDR/WCG images into the forward reshaped BL images that approximate the reference BL images. [0033] The encoder side architecture may include a processing block/component for metadata generation, which generates image metadata or portions thereof to define or specify the backward reshaping functions. The backward reshaping functions can be used, for example by a recipient decoding device of the image metadata, to generate the reconstructed wide view HDR/WCG images that approximate the input wide view HDR/WCG images. [0034] The encoder side architecture may include a processing block/component for video compression, which compresses or encodes the forward reshaped BL images into the (video) coded bitstream along with the image metadata generated by the metadata generation. In some operational scenarios, available video codecs that support compressing or encoding relatively narrow view 2D video content can be used or enhanced to implement the processing block/component for video compression of FIG.1A. [0035] FIG.1B illustrates an example decoder side architecture that may be implemented with one or more computing devices. The decoder side architecture may include a plurality of processing blocks or components to receive a coded bitstream encoded with forward reshaped wide view BL images along with image metadata specifying backward reshaping functions corresponding to forward reshaping functions used to generate the forward reshaped wide view BL images; to apply the backward reshaping functions to the forward reshaped wide view BL images to generate corresponding reconstructed or backward reshaped wide view HDR/WCG images; to render display images derived from the reconstructed or reshaped wide view HDR/WCG images on image display(s); etc. [0036] In some operational scenarios, some or all of these foregoing operations may be performed a downstream device implemented by the one or more computing devices at the decoder side on line in real time, for example while the upstream device communicates the bitstream or a portion thereof to the downstream device. In some operational scenarios, at least some of the foregoing operations may be performed by the one or more computing devices at the decoder side off line in non-real time. [0037] As shown in FIG.1B, the decoder side architecture may include a processing block/component for metadata extraction, which extracts the image metadata from (image metadata container or data fields carried or included in) the coded bitstream. [0038] The decoder side architecture may include a processing block/component for video decompression, which decodes or decompresses the forward reshaped wide view BL images from (encoded video data carried or included in) the coded bitstream. [0039] The decoder side architecture may include a processing block/component for backward reshaping, which backward reshapes – or applies the backward reshaping functions to – the forward reshaped wide view BL images (denoted as “Reshaped BL”) into the reconstructed or reshaped wide view HDR/WCG images (or video). [0040] In the entire pipeline implemented by the encoder side architecture of FIG. 1A and the decoder side architecture of FIG.1B, there are two processing blocks/components at the encoder side, namely (1) reference BL generation and (2) forward/backward TPBC optimization. [0041] The first of the above mentioned two processing blocks/components, or the reference BL generation, includes extracting overlapped patches from image data of each input or source wide view HDR/WCG image in each color channel of an input or source color space in which the input or source wide view HDR/WCG image is represented. By way of example but not limitation, the input or source color space may include a luma channel and two chroma channels. For the luma channel, a block- based standard deviations based method/algorithm (denoted as BLKSTD) may be implemented or performed to generate luma image data of reference BL image patches. Example BLKSTD methods/algorithms are described in U.S. Patent No. 10,032,262, issued on 24 July 2018; U.S. Patent No.10,223,774, issued on 5 March 2019, the entire contents of all of which are hereby incorporated by reference as if fully set forth herein. For the chroma channels, a luma-chroma channel energy ratio may be determined and applied to generate to generate chroma image data of the reference BL image patches from corresponding image patches of the original (input or source) wide view HDR/WCG images/frames. After determining or generating the luma and chroma image data of the reference BL image patches, these patches can be fused together, for example with a predefined weight function, into a reference wide view BL image in which visual patch boundary artifacts are reduced, removed or otherwise alleviated. [0042] The second of the above mentioned two processing blocks/components, or the forward/backward TPBC optimization, includes determining an optimized spatial positional (information or data) encoding for the input or source wide view HDR/WCG image. More specifically, optimized operational parameters for forward reshaping TPBC operations are generated and used on the encoder side to forward reshaping the input or source wide view HDR/WCG image into a forward reshaped wide view BL image approximating the reference wide view BL image. Optimized operational parameters for backward reshaping TPBC operations are generated and used on the decoder side to backward reshaping the forward reshaped wide view BL image into a reconstructed or reshaped wide view HDR/WCG image. In some operational scenarios, a three-stage optimization process may be used to generate these optimized operational parameters for the forward and backward TPBC operations. At each of these three stages, TPBC operations can be performed on an increased-size dataset using the previously mentioned BESA method or algorithm. Reference Base Layer Generation [0043] Under some approaches, global reshaping may be applied to reshape an input image with a global reshaping function applied to image data in all spatial regions of the input image into a reshaped image at a relatively high compression efficiency. Such global reshaping techniques or functions may work well with images of a relatively narrow field of view such as 2D images. [0044] In contrast, a wide view HDR/WCG image/picture covers a much wider field of view as compared with an 2D image of a relatively narrow field of view. As noted, wide view HDR/WCG video or images/pictures exhibit relatively great diversity in different local regions within the same image/picture. For example, dynamic ranges and color distributions in different spatial regions of the same wide view HDR/WCG image may exhibit different characteristics. Codewords needed to avoid banding visual artifacts in one particular luminance range in one particular local area or spatial region of the wide view HDR/WCG image can be quite different from codewords needed to avoid banding visual artifacts in another particular luminance range in another particular local area or spatial region of the same wide view HDR/WCG image. Global reshaping would subject all these spatial regions with diversely different luminance ranges to the same global reshaping mapping or function and cannot adequately or efficiently handle varying needs in these different spatial regions for different sets of codewords in reshaping operations. [0045] Under techniques as described herein, local reshaping operations/methods may be implemented, performed or used to apply different local reshaping functions or mappings to different spatial regions of the same wide view image. These local reshaping functions or mappings such as TPBC based reshaping functions or mappings may be determined or generated to meet respective needs of the different spatial regions of the same wide view image. These local reshaping functions/mappings or operational parameters used in the local reshaping operations/methods can be optimized respectively for each local region or for each luminance range in each such local region to achieve relatively high efficiency and effectiveness. [0046] FIG.2A illustrates an example two-stage process or method for reference BL image generation, which may be implemented, for example, by a video encoder or processing blocks/components therein. This two-stage process or method can be used to help tackle or perform local reshaping operations by way of creating a reference wide view base layer images (or video signal) from an input or source wide view HDR/WCG images (or video signal). [0047] The first stage is implemented with a processing block/component for LUT generation, which generates a respective lookup table (LUT) in each local patch – among a plurality of local (HDR/WCG) patches identified from an input or source wide view HDR/WCG image in the input or source wide view HDR/WCG images (or video signal) – using the BLKSTD method/algorithm. Respective LUTs generated for the plurality of local (HDR/WCG) patches can be different for different local or spatial patches in the same input or source wide view HDR/WCG image. These respective LUTs can be applied to the plurality of local (HDR/WCG) patches in the input or source wide view HDR/WCG image into corresponding locally forward reshaped (BL) patches. [0048] The second stage is implemented with a processing block/component for fusion, which fuses the locally forward reshaped (BL) patches to smooth out boundary or discontinuity artifacts along patch boundaries to generate a corresponding reference wide view BL image in the reference wide view base layer images (or video signal). Patch Generation [0049] To address different needs for codewords in different luminance ranges in different local regions in forward reshaping an input or source (e.g., 12+ bits, etc.) wide view HDR/WCG image into a corresponding reference (e.g., 10 bits, etc.) wide view BL image, a respective patch based LUT (e.g., one-dimensional LUT or 1D- LUT for the luma channel, etc.) representing a local forward reshaping function for a local patch can be generated for each local patch in a plurality of local patches in the input or source wide view HDR/WCG image. These local patches may represent relatively small areas or spatial regions in the input or source wide view HDR/WCG image and may be used to generate patch based LUTs. [0050] Denote the width and height of an input or source wide view HDR/WCG image, respectively, as W and H. To maintain temporal stability and compression efficiency, a group of T (input or source wide view HDR/WCG) image/frames within the same scene may be processed or partitioned using the same set of local patches. [0051] Denote the t-th image/frame in the group of T image/frames within the same scene as ^^ . Denote the luma channel and non-luma channels of the t-th image/frame, respectively, as ^^ ^ and ^^ ^. In some operational scenarios, superscript C may be Cb or Cr to denote a Cb or Cr chroma channel in an input YCbCr color space or domain. In some operational scenarios, superscript C may be P or T to denote a P or T channel in an input IPT (e.g., IPTPQc2, etc.) color space or domain. [0052] Denote the luma and non-luma pixel values at a pixel location (m, n) in the t-th image/frame, respectively, as ^^ ^(^, ^) and ^^ ^(^, ^), where m represents a row index of the pixel or pixel location; and n represents a column index of the pixel or pixel location. [0053] An input or source wide view HDR/WCG image, or the t-th image/frame, may be partitioned into a plurality of non-overlapped local patches each of which has a patch size ^^ × ^^, where ^^ represents an integer greater than two (2). [0054] These non-overlapped local patches collectively cover the entire spatial ^ ^ area represented in the t-th image/frame. Hence, there are ^^ = ^^^^ × ^ ^^^ patches in the t-th image/frame. [0055] Denote the luma component or corresponding values of the k-th non-overlapped patch in the t-th image/frame as ^^ ^, , ^^. The luma component may include the corresponding luma pixel values number of pixels ^^ × ^^ if the k-th non-overlapped patch is an interior patch in the t-th image/frame. If the k-th non-overlapped patch is next to or on a border/boundary of the image/frame, the patch may have fewer luma pixel values than the full size of ^^ × ^^. [0056] Denote a non-luma component – e.g., Cb or Cr chroma in an YCbCr color space, P or T in an IPT color space – or corresponding non-luma pixel values of the k- th non-overlapped patch in the t-th image/frame as as ^^ ^, , ^^. By way of example but not limitation, the input or source wide view is represented in a 4:2:0 color space sampling format. The non-luma component may include the corresponding non-luma pixel values for a total number of pixels ^^ ^ × ^^ ^ , where ^^ ^ = ^^/2, if the k-th non-overlapped patch is an interior patch in the t-th image/frame. If the k-th non-overlapped patch is next to or on a border/boundary of the image/frame, the patch may have fewer non-luma pixel values than the full size of ^^ ^ × ^^ ^ . [0057] Denote the luma and non-luma pixel values at a (e.g., relative to the local patch, etc.) pixel location (m, n) in the k-th non-overlapped patch of the t-th image/frame, respectively, as ^^ ^,^ (^, ^) and ^^ ^,^ (^, ^), where m represents a (e.g., relative to the top left pixel or pixel location of the local patch, etc.) row index of the pixel or pixel location; and n represents a (e.g., relative to the top left pixel or pixel location of the local patch, etc.) column index of the pixel or pixel location. [0058] A straightforward solution may be to construct respective patch-based 1D- LUTs for the non-overlapped patches in the input or source wide view HDR/WCG image and to generate a reference wide view base layer image by applying the BLKSTD method/algorithm to each non-overlapped patch. However, relatively significant boundary artifacts can be observed occurring along the patch boundaries. Those high-frequency visual artifacts can degrade reshaping accuracy, reduce video coding efficiency, and propagate those visual artifacts to a reconstructed or backward reshaped wide view HDR/WCG image. [0059] To alleviate these problems, an overlapped patch based method or algorithm as described herein may be implemented or performed to use overlapped patches to collect or gather additional neighboring patch data in the LUT generation and to further use overlapped patches to enable or provide a relatively smooth transition along patch boundaries in the reference wide view BL image generation. In some operational scenarios, as illustrated in FIG.2B, overlapped patches for the LUT generation may be different from overlapped patches for the BL image generation. For example, the overlapped patches for the LUT generation may be of different sizes from sizes of overlapped patches for the BL image generation. [0060] As shown in FIG.2B, the t-th image may be partitioned into a plurality of non-overlapped local patches, for example arranged in a two-dimensional spatial array. Two corresponding overlapped local patches may be defined or determined for each non-overlapped local patch in the plurality of non-overlapped local patches. Pixel value statistics can be collected in the first of the two corresponding overlapped local patches to construct a local patch-based LUT, which can be applied to the second of the two corresponding overlapped local patches to help generate a patch- based portion of the reference wide view base layer image. [0061] As illustrated in FIG.2B, the first of the two corresponding local patches – assuming they are located in the interior of the t-th image – is represented by a first square or a first luma patch size ^^ × ^^. This first overlapped local patch may be used for the LUT generation. The corresponding first chroma patch size for the LUT generation is ^^ ^ × ^^ ^ , where ^^ ^ = ^^/2 for the 4:2:0 sampling format. [0062] The second of the two corresponding local patches is represented by a second square or a second luma patch size ^^ × ^^. This second overlapped local patch may be used for the BL image generation. The corresponding second chroma patch size for the BL image generation is ^^ ^ × ^^ ^ , where ^^ ^ = ^^/2 for the 4:2:0 sampling format. [0063] In some operational scenarios, the patch sizes of the non-overlapped and overlapped local patches may satisfy an inequality as follows: ^^ < ^^ ≤ ^^ (1) [0064] The total number of the non-overlapped local patches may be the same as the total number of the overlapped local patches for each of the LUT generation and the BL image generation and is given or specified by ^^. [0065] Denote the k-th overlapped luma patches in the t-th frame used in the LUT generation and BL image generation as ^^ ^, , ^^ and ^^ ^, , ^^, respectively. Denote the k-th overlapped non-luma patches in the t-th frame used in the LUT generation and BL image generation as ^^ ^, , ^^ and ^^ ^, , ^^, respectively. [0066] As shown in FIG.2B, a non-overlapped local patch and its two corresponding overlapped local patches share the same patch center. Hence, all three k-th luma local patches, ^^ ^, , ^^, ^^ ^, , ^^, and ^^ ^, , ^^, have the same patch center. Likewise, all three k-th chroma local patches for the same non-luma channel C, ^^ ^, , ^^, ^^ ^, , ^^, and ^^ ^, , ^^, have the same patch center. [0067] By way of illustration but not limitation, in the case of luma local patches, the four corners of the overlapped luma local patch for the LUT generation, ^^ ^, , ^^, has a pixel distance ( ^^^^^ ^ ) horizontally and vertically to the corresponding four corners of the non-overlapped luma local patch ^^ ^, , ^^. The four corners of the overlapped luma local patch for the BL image generation, ^^ ^, , ^^, have a pixel distance ( ^ ^^^ ^ ) horizontally and vertically to the corresponding four corners of ^^ ^, , ^^. The four corners of ^^ ^, , ^^ has pixel distance ( ^^^^ ^ ) horizontally and vertically to the corresponding four corners of the non-overlapped luma local patch ^^ ^, , ^^. Patched-Based LUT Construction Within a Scene [0068] FIG.2C illustrates an example method or process for patch based forward reshaping function/mapping generation, which may be implemented, for example, by a video encoder or processing blocks/components therein. The process flow can be implemented or performed to generate patch-based or patch-specific forward reshaping functions/mappings such as LUTs for forward reshaping patches extracted from input or source wide view HDR/WCG image(s). As shown, the process flow comprises a processing block/component for patch extraction, which extracts overlapped HDR/WCG local patches for LUT generation at a plurality of different local patch locations from input or source wide view HDR/WCG image(s). The process flow further comprises a processing block/component for LUT generation, which generates respective LUTs for forward reshaping HDR/WCG local patches at the plurality of different local patch locations in the input or source wide view HDR/WCG image(s) into corresponding BL local patches. The corresponding BL local patches may be combined or fused into reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s). [0069] Under techniques as described herein, a respective LUT such as a specific 1D mapping LUT can be constructed to prevent or reduce banding artifacts and preserve colors or color precision in each non-overlapped local patch ^^ ^, , ^^ of an input or source wide view HDR/WCG image. In some operational scenarios, a corresponding overlapped local patch ^^ ^, , ^^ can be used instead of the non-overlapped local patch ^^ ^, , ^^ to generate the LUT. [0070] Denote the bit depth (e.g., the number of bits used to encode a luma or non-luma pixel value or codeword, etc.) of the input or source wide view HDR/WCG image as !". Denote the bit depth in a corresponding wide view base layer image derived from the input or source wide view HDR/WCG image as !#. [0071] As noted, needed codewords to avoid or prevent banding artifacts may be estimated or computed with a function derived with a block-based standard deviation (BLKSTD) method, which measures block-based standard deviations (BLKSTDs) in multiple mutually exclusive luminance sub-ranges that make up the entire luminance range in the overlapped local patch ^^ ^, , ^^ of the input or source wide view HDR/WCG image. More specifically, in each luminance sub-range, pixels – in the overlapped local patch ^^ ^, , ^^ – whose pixel values or codewords fall into this considered luminance sub-range may be collected. These pixel values or codewords can be used to compute an individual BLKSTD for each of these pixels. Individual BLKSTDs for all the pixels can be used to compute an average BLKSTD for that luminance sub-range represented in the overlapped local patch ^^ ^, , ^^. [0072] In some operational scenarios, the input or source wide view HDR/WCG image containing the overlapped local patch ^^ ^, , ^^ belongs to a sequence of (e.g., consecutive, sequential, etc.) input or source wide view HDR/WCG in a video. In these scenarios, pixels or pixel values (or codewords) represented in corresponding overlapped local patch(es) ^^ ^, , ^^ of a group of one or more input or source wide view HDR/WCG images within a scene may be collected across all images in the group and used to derive average BLKSTDs for different luminance sub-ranges that make up the entire luminance range represented in the corresponding overlapped local patch(es) ^^ ^, , ^^ of the group of one or more input or source wide view HDR/WCG images. [0073] Respective BLKSTDs in all the luminance sub-range can be used to construct a BLKSTD function that estimates or computes respective needed codewords for all the luminance sub-range represented in the overlapped local patch(es) ^^ ^, , ^^. The LUT such as a 1D-LUT can be built using the BLKSTD function constructed from all the average BLKSTDs in all the luminance sub-ranges. [0074] The processing block or component for the LUT generation may include optimizing the LUT derived from the BLKSTDs by identifying and re-using previously unused codewords in a codeword space and smoothing filtering to make the LUT or 1D-LUT relatively smooth or less discontinuous for the purpose of avoiding or reducing video compression artifacts. One or both of the input HDR/WCG pixel values (or codewords) and the output base layer pixel values (or codewords) represented in the LUT may be normalized into a normalized value range [01]. [0075] The LUT – e.g., optimized and smoothened LUT – may be used as a forward reshaping mapping or function to convert the pixel values or codewords encoded in corresponding overlapped local patch(es) ^^ ^, , ^^ of the input or source wide view HDR image – or in the group of the one or more input or source wide view HDR images within the same scene – to corresponding base layer pixel values or codewords in corresponding overlapped BL local patches. The overlapped BL local patches may be used to construct corresponding reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s). [0076] In some operational scenarios, to ensure maximum utilization of all available codewords in a codeword space, the maximum mapped or BL value may be set or normalized to the maximum value of an applicable normalized value range such as 1. Likewise, the minimum mapped or BL value may be set or normalized to the minimum value of the applicable normalized value range such as 0. For simplicity, the foregoing operations used to construct the LUT or 1D-LUT denoted as $%^&'() from a collection or group of images {^} may be represented as follows: ,-. = $%^&'()({^}) (2) [0077] Since there are !" bits to each or source value or or non- or etc.) in the input or source wide view HDR/WCG image(s), a codeword space that includes all available codewords comprises 2%/ codewords. Hence, the LUT may contain 2%/ entries each of which maps or forward reshapes a respective input or source pixel value or codeword into a mapped (e.g., forward reshaped, etc.) BL pixel value or codeword as output. Even though the output value range may also be in [01], as the output BL signal is encoded with codewords or pixel values of !# (<!") bits, quantization occurs in mapping the input 2%/ HDR/WCG pixel values or codewords to some or all 2%3 BL pixel values or codewords. [0078] FIG.3A illustrates two example distributions of BLKSTDs in two local patches, respectively, of the same input or source wide view HDR/WCG images. As illustrated, the distributions of BLKSTDs in different local patches of the same image can be widely different. Global reshaping under other approaches would significantly degrade reshaping operations and would likely fail to prevent banding artifacts and negatively impact video compression performance. In contrast, local reshaping as described herein applies different reshaping functions/mappings in different local patches even if they are from the same image. As a result, the local reshaping operations can be used to effectively prevent or reduce banding artifacts as well as improve video compression performance. [0079] To ensure temporal stability, the same LUT or the 1D-LUT may be applied to – for example to forward reshape luma component or pixel values/codewords in – each corresponding local patch of some or all of (e.g., a group of, consecutive, sequential, with different frame indexes, etc.) images with the same scene. To achieve this, the LUT or 1D-LUT construction may be performed with respect to all like local patches at the same patch location in some or all images within the scene. [0080] For example, for the k-th overlapped patch location in all T input or source wide view HDR/WCG images in the same scene, input pixel values or codewords in all T overlapped local patches {^^ ^, , ^^|5 = 0,1, … . , . − 1} at that patch location may be collected, computed or used the k-th LUT by performing BLKSTD related operations the function $%^&'() as follows: ,-.^ ^() = $%^&'()({^^ ^, , ^^|5 = 0,1, … . , . − 1} ) (3) [0081] the input values and/or the output values can be a range such as [01]. [0082] In some operational scenarios, respective LUTs or 1D-LUTs constructed to represent non-linear forward reshaping functions/mappings using the BLKSTD method or algorithm can be applied to forward reshape input luma pixel values or codewords in local patches at some or all local patches of input or source wide view HDR/WCG image(s). In comparison, linear forward reshaping functions/mappings – such as first order polynomials, which can also be represented by LUTs such as 1D- LUTs – can be applied to forward reshape input non-luma pixel values or codewords in local patches at some or all local patches of input or source wide view HDR/WCG image(s). [0083] More specifically, the maximum and minimum values for input luma and non-luma pixel values or codewords in all images with the same scene may be determined or identified in each luma or non-luma color channel, as follows: <=>? = maxC{^^ ^(^, ^)|5 = 0,1, … . , . − 1, ^ = 0,1, … . , D − 1, ^ = 0,1, … . , E − 1}F (4- (4- (4- K=GH − ^ − ^ = 0,1, … . , E − 1}M (4- 4) [0084] In addition, the maximum and minimum values for input luma and non- luma pixel values or codewords in local patches at the k-th local patch location in all images with the same scene may be determined or identified in each luma or non- luma color channel, as follows: <^ =>? 0,1, … . , . − 1, ^ = 0,1, … . , ^^ − 1, ^ = 0,1, … . , ^^ − 1FM (5- 1) 0,1, … . , . − 1, ^ = 0,1, … . , ^^ − 1, ^ = 0,1, … . , ^^ − 1FM (5- 2) K^ =>? = max LC^^ ^,^ (^, ^)N5 = 0,1, … . , . − 1, ^ = 0,1, … . , ^^ ^ − 1, ^ = 0,1, … . , ^^ ^ − 1FM (5- 3) K^ =GH (5- 4) [0085] As noted, a LUT or 1D-LUT for mapping or forward reshaping chroma values may be a relatively simple first order polynomial or a first order function with a scaling factor and an offset. The offset may be used to shift the center of a (e.g., valid, etc.) codeword range or sub-range to the middle of all codewords (for coding efficiency) in the codeword range or sub-range. The slope or scaling factor may be chosen using an energy ratio, which is defined as a ratio specified with the previously determined maximum and minimum values in the luma and non-luma channels. More specifically, a constrain factor denoted as O^ ^ may be computed as follows: QRS QTU O^ ^P ^^P ^ = ^P QRS ^^P QTU (6) [0086] , Y ,-. ^^ QTU [^\ Y ^^(V^ ^ " ,^ ) = O^ ^X,P P ^QRS^^QTU + ( ^ ) (7) [0087] FIG.2D illustrates an example method or process flow for patch fusion, which may be implemented, for example, by a video encoder or processing blocks/components therein. The process flow can be implemented or performed to fuse overlapped patches together to generate reference wide view base layer image(s). As shown, the process flow comprises a processing block/component for patch extraction, which extracts overlapped HDR/WCG local patches for reference generation at a plurality of different local patch locations from input or source wide view HDR/WCG image(s). The process flow further comprises a processing block/component for applying LUT for reference (signal or image) generation, which applies respective LUTs to forward reshape HDR/WCG local patches for reference generation at the plurality of different local patch locations in the input or source wide view HDR/WCG image(s) into corresponding BL local patches for reference generation. The process flow also comprises a processing block/component for weight map (for patches) creation, which creates a weight map for each of the BL local patches. The process flow comprises a processing block/component for fusion, which fuses the BL local patches using respective weight maps for the BL local patches into overall reference wide view BL image(s) corresponding to the input or source wide view HDR/WCG image(s). [0088] As noted, LUTs or 1D-LUTs for forward reshaping HDR/WCG local patches may be constructed or trained using patch-based or patch-specific codeword distributions computed or generated from overlapped local patch, for example located at a plurality of different patch location in some or all input or source wide view HDR/WCG images (for frame index t=0, 1, …, T-1) within the same scene. Although such construction or training of the LUTs can be done with the overlapped local patches, directly applying these LUTs to corresponding non-overlapped local patch may still show boundary artifacts along patch boundaries. In some operational scenarios, to make inter-patch transitions relatively smooth, the LUTs can be first applied to corresponding overlapped local patches to generate forward reshaped BL local patches. Fusion operations may be performed on the forward reshaped local patches in the luma and chroma channels to generate the overall wide view BL image(s). It should be noted that the overlapped local patches for reference generation may be of different sizes from the corresponding overlapped local patches used to generate the LUTs. Weighting Matrix [0089] To ensure relatively smooth transitions across patch boundaries, a weighting map with individual weighting factors for different pixels may be applied to the overlapped local patches used in image future. For a pixel which is covered by multiple overlapped local patches, its fused pixel value may be a weighed combination of pixel values from all the multiple overlapped patches multiplied with a normalization factor summed from weight factors used to generate the weighted combination. [0090] A (luma channel) weighting map as described herein for an overlapped local patch – with a total number ^^ ^ × ^^ ^ of pixels in the luma channel – used for image fusion (in the luma channel) can be defined or specified with a (^^ ^ × ^^ ^ in the luma channel) weighting matrix comprising respective weighting factors for (luma pixel values or codewords of) all pixels represented in the overlapped local patch. It should be noted that non-luma channel weighting maps or matrix for image fusion in the non-luma channels can be similarly defined or specified. [0091] In some operational scenarios, as illustrated in FIG.3B, a weighting map or matrix may be specifically selected or configured such that a contour formed by equal values of a subset of the weighting factors in the weighting map or matrix for the overlapped local patch may be of the same or similar shape as compared with a spatial shape of the overlapped local patch such as square or rectangle, rather than an isotropic circular shape. [0092] By way of example but not limitation, the (^^ ^ × ^^ ^ in the luma channel) weighting map or matrix may include equal weighting factors (or values) in rectangle contours given (e.g., using Gaussian distribution function(s), etc.) as follows: ]^ ^ = 0.2 ∙ ^^ ^ (8- 1) ^ (^, ^ ab a = H ^ ) = ^_` 2 ∗ ^ d − 0.5fb , b2 ∗ a ^ d − 0.5fbf (8- 2) ^i. g^(^, ^) =∙ h (m d )l (8- 3) × a non- map or may include equal weighting (or values) in rectangle contours given (e.g., using Gaussian distribution (s), etc.) as follows: ]^ ^ = 0.2 ∙ ^^ ^ (9- 1) = H ^^(^, ^) = ^_` ab2 ∗ a^ Y − 0.5fb , b2 ∗ a ^ Y − 0.5fbf (9- 2) kY (Q, l ^i.j U) (^, ^) = h (m Y ) l (9- 3) [0094] Each of these matrices in expressions (8) and (9) above specifies equal weights (or weighting factor values) along a rectangle contour, which is the same as or similar to the spatial shape of a corresponding overlapped local patch, with the highest weights at the center and non-center weights decreasing outwards. [0095] These two matrices in expressions (8) and (9) above or their corresponding weighting maps may be fixed for each of some or all of the overlapped local patches used for image fusion, except for possibly boundary overlapped local patches which do not have full patch size. In some operational scenarios, the same matrixes as other (interior) overlapped local patches can be used by aligning the top-left corners of all of these overlapped local patches, for example to a common (relative rather than absolute; relative to each overlapped local patch) pixel location value of (m=0, n=0). we still apply the same equation by aligning the top-left corner. It may be noted that in this boundary patch case, the center of the boundary overlapped local patch might not have the highest weighting factor value as the peak weighting factor value of the center of an interior overlapped local patch. [0096] FIG.3C illustrates example operations that apply patch-based weighting matrixes or maps, as defined in expressions (8) and (9) above, to mapped or forward reshaped overlapped local patches to generate an overall or fused reference wide view BL image corresponding to an input or source wide view HDR/WCG image. The mapped or forward reshaped overlapped local patches may be generated from patch- based forward reshaping or LUT mapping operations corresponding input overlapped local patches derived from the input or source wide view HDR/WCG image. The overall or fused reference wide view BL image depicts the same visual semantic content as – and may be in a lower bit depth or dynamic range than – the input or source wide view HDR/WCG image. [0097] Denote accumulated mapped or forward reshaped pixel values or codewords in the luma channel as ^^ ^ ,n. Denote accumulated mapped or forward reshaped pixel values or codewords in a C non-luma channel as ^^ ^ ,n. For simplicity, denote accumulated mapped or forward reshaped pixel values or codewords in either the luma channel or the C non-luma channel as ^^ n. [0098] Denote path-based or patch-specific mapped or forward reshaped pixel values or codewords in the luma channel as ^o^ ^ ,n. Denote path-based or patch-specific mapped or forward reshaped pixel values or codewords in the C non-luma channel as ^o^ ^ ,n. For simplicity, denote path-based or patch-specific mapped or forward reshaped pixel values or codewords in either the luma channel or the C non-luma channel as ^o ^ n. [0099] Denote accumulated weights or weighting factors in the luma channel as ^^ ^ ,^. Denote accumulated weights or weighting factors in the C non-luma channel as ^^ ^ ,^. For simplicity, denote accumulated weights or weighting factors in either the luma channel or the C non-luma channel as ^^ ^ . [0100] As illustrated in FIG.3C, two memory spaces may be prepared – e.g., initiated to pixel values of all zeros (0), as represented by the two black rectangles on the left side of FIG.3C – with the same dimension (E × D) as that of the input or source wide view HDR/WCG image. [0101] The first (to store accumulated pixel values or codewords ^^ n; which may be simply referred to as the memory space ^^ n) of the two memory spaces may be used to accumulate respective weighted patch-based reshaped pixel values or codewords for all pixels (to be) represented in the (overall or fused) reference wide view BL image. The second (to store accumulated weights or weighting factors ^^ ^ ; which may be simply referred to as the memory space ^^ ^ ) of the two memory spaces may be used to accumulates respective weights or weighting factors for all the pixels. [0102] For each overlapped patch such as luma overlapped local patch ^^ ^, , ^^ and non-luma overlapped local patch ^^ ^, , ^^ at a respective patch location (e.g., the k-th patch location, etc.) in the plurality of patch locations in the input or source wide view HDR/WCG image, a respective patch-based or patch-specific forward reshaping function or LUT can be applied to generate a corresponding mapped or forward reshaped overlapped local patch such as mapped or forward reshaped luma overlapped local patch ^o^ ^, , ^^ and mapped or forward reshaped non-luma overlapped local patch ^o^ ^, , ^^, as follows: ^o^ ^, , ^^ = ,-.^ ^p^^ ^, , ^^q (10-1) [0103] The mapped or forward reshaped pixel values or codewords can be multiplied by weights or weighting factors in a corresponding patch-based weighting matrix. All weighted mapped pixel values or codewords at each pixel (location) covered by overlapped local patches can be accumulated in the corresponding pixel location in the memory space ^^ n. The weights or weighting factors – to be used for normalization – can be accumulated similarly in the memory space ^^ ^ , as follows: ^^ ^ ,n(^r, ^r) = ^^ ^ ,n(^r, ^r) + ^o^ ^, , ^^(^, ^) ∙ g^(^, ^) (11-1) ^^ ^ ,n(^r, ^r) = ^^ ^ ,n(^r, ^r) + ^o^ ^, , ^^(^, ^) ∙ g^(^, ^) (11-2) ^^ ^ ,^(^r, ^r) = ^^ ^ ,^(^r, ^r) + g^(^, ^) (11-3) ^^ ^ ,^(^r, ^r) = ^^ ^ ,^(^r, ^r) + g^(^, ^) where (^r, ^r) denotes a pixel location in the original image coordinate of the input or source wide view HDR/WCG image – which may be the same as the image coordinate of the (overall or fused) reference wide view BL image derived from the input or source wide view HDR/WCG image – corresponding to the pixel location (^, ^) in the relative image coordinate of the k-th patch coordinate. [0104] After scanning or processing all ^^ mapped or forward reshaped overlapped local patches generated from reshaping corresponding overlapped local patches in the input or source wide view HDR/WCG image, accumulated weighted pixel values or codewords in the memory space ^^ ^ ,n can be respectively divided (elementwise) by accumulated weights or weighting factors in the memory space ^^ ^ ,^, as follows: d,t u u s^ ^r r ^X p= ,H q ^ ( , ^ ) = ^ d X ,v (= u ,H u ) q [0105] Similar operations to those for the luma channel may be performed for a non-luma channel. [0106] In some operational scenarios, weight maps for the luma and non-luma channels are fixed for some or all (input or reshaped) images. Hence, the accumulated weights or weighting factors such as ^^ ^ ,^ and ^^ ^ ,^are fixed for these images, and can be computed once, for example at the system boot up and then stored in its allocated memory space. Tensor-Product B-Spline with Coordinate (TPBC) [0107] After a (fused) reference wide view base layer image is built or derived from an input or source wide view HDR/WCG image, a local forward reshaping mapping/function may be generated or constructed to forward reshape the input or source wide view HDR/WCG image to generate a (relatively compression efficient) forward reshaped wide view BL image that approximates the reference wide view BL image. A corresponding backward reshaping mapping/function corresponding to the forward reshaping mapping/function may be constructed to be used to backward reshape forward reshaped wide view BL image to generate a reconstructed or backward reshaped wide view HDR/WCG image that approximates the input or source wide view HDR/WCG image. [0108] In some operational scenarios, the forward and backward reshaping mappings or functions may be built or constructed as TPBC reshaping functions. [0109] TPB provides a relatively high flexibility and accuracy for reshaping operations that may include dynamic range conversion and color mapping. In some operational scenarios, TPB forward and backward reshaping functions can be derived using the previously mentioned BESA method or algorithm, which builds a reversible paired of TPB transforms representing respectively a TPB forward reshaping function used to forward reshape the input or source wide view HDR/WCG image into the forward reshaped image and a corresponding TPB backward reshaping function to backward reshape the forward reshaped image to the reconstructed or backward reshaped image that relatively closely approximate (e.g., with minimized errors or differences, etc.) the input or source wide view HDR/WCG image. [0110] In some operational scenarios, a TPB reshaping function as described herein may use a combination of luma pixel values or codewords of pixels in an input (or to-be reshaped) image and their positional information such as pixel coordinates – or a positional functional or encoded form of the pixel coordinates – as inputs to predict (e.g., luma, etc.) pixel values or codewords of pixels in an output (or predicted) image as output. Additionally, optionally or alternatively, the TPB reshaping function may use non-luma pixel values or codewords of the pixels in the input image as a part of the inputs in addition to or in place of some or all in the combination of the luma pixel values or codewords and the positional information. [0111] In some operational scenarios, pixel coordinates may be directly used as the positional information in the inputs to a TPB reshaping function. [0112] In some operational scenarios, to improve reshaping function generation performance and/or reshaping operation performance, a positional functional or encoded form of the pixel coordinates may be used as the positional information in the inputs to a TPB reshaping function. [0113] Denote the positional functional or encoding form of the pixel coordinates as a w?() function in the x-axis or row direction and a wx() function in the y-axis or column direction, as follows: ^y = w?(^) (13-1) ^z = wx (^) (13-2) where ^y and ^z represents the positional information along both the row and column directions to be used as a part of the inputs in TPB mapping operations. [0114] In an example, the functions in expressions (13) above can be simple linear functions as follow: w ( ) = ? ^ = ^ (14-1) wx(^) = H ^ [0115] example, the functions in expressions (13) above can be simple trigonometry functions as follow: ?( s (~ = w ^) = co ^^) (15-1) x in (~ H w (^) = s ^^) (15-2) where W and H in expressions (14) and (15) above denote the width and height of the input image for reshaping operation such as the input or source wide view HDR/WCG image. The selection of specific type(s) of positional encoding functions can be based at least in part on a comparison of respective performances among some or all candidate positional encoding functions and a selection of specific positional encoding functions corresponding to the best performance. In operational scenarios in which multiple candidate positional encoding functions, the selected positional encoding functions can be signaled by image metadata provided by an upstream device to a downstream recipient device. [0116] Hence, in some operational scenarios, the encoded positional information (^y , ^z) may be used in reshaping (or prediction) functions as described herein in place of the pixel coordinates (^, ^). Forward Path TPB Optimization [0117] A process flow portion with forward reshaping operations may be referred to as a forward path herein. In the forward path, forward reshaping mappings/functions takes input pixel values or codewords in input or source wide view HDR/WCG image(s) as input to predict or estimate output pixel values or codewords that collectively constitute forward reshaped wide view BL image(s) corresponding to – or depicting the same visual semantic content as – the input or source wide view HDR/WCG image(s). [0118] By way of illustration but not limitation, a TPB forward reshaping mapping/function can be used to take (1) an input luma pixel value or codeword (denoted as ^^ ^(^, ^), where m and n represent pixel (location) coordinates of a pixel) of an (or t-th) input or source wide view HDR/WCG image, (2) input x-axis positional information represented by ^y encoded from m, and (3) input y-axis positional information represented by ^y encoded from n to predict an output luma pixel value or codeword (denoted as^ s^ ^(^, ^)) in a (or t-th) forward reshaped wide view BL image for the luma channel in the forward path. [0119] Given three sets of knot points in the three dimensions, respectively, of the inputs as enumerated above to the TDB forward reshaping mapping, three sets of independent B-spline basis functions – denoted as !^,i ^d (^^ ^(^, ^)), !^,[ ^S (^y ), and !^ ^^,^(^z) – can be defined or identified. Indexes of the three sets of know points can be respectively denoted as 5^, 5?, and 5x in the luma channel denoted as Y, x and y dimensions. Denote three total numbers of basis functions in the three sets of independent B-spline basis functions (!^,i ^d (^^ ^(^, ^)), !^,[ ^,^ ^S (^y), and !^^ (^z)) as ^i ^ (in the luma channel), ^[ ^ (in the x dimension), and ^^ ^ (in the y dimension), respectively. [0120] From these three sets of independent B-spline basis functions (!^,i(^^(^, ^)), !^ S ,[(^y), an ^ ^ ,^ z (^%,^ ^d ^ ^ d !^ (^)), TPB basis functions (denoted as !^d,^S,^^ ; for prediction in luma be constructed by multiplying three B-Spline basis functions from these sets of independent B-spline basis functions (!^,i ^d (^^ ^(^, ^)), !^,[ ^S (^y), and !^,^ ^^ (^z)), as follows: !( ^d ^% ,^S ,^ ,^^ (^^ ^(^, ^), ^y, ^z) = !^ ^d ,i(^^ ^(^, ^)) ∙ !^ ^S,[(^y) ∙ !^ ^^,^(^z) the t-th wide view BL image can be performed using the TPB basis functions !(^%,^ ^d,^S,^^ in expression (16) as follows: s ^^ ^ ( ^, ^ ) = ^^ (^%,^( ^ ^ ^ ( ^, ^ ) , ^ y , ^ z) )^ d^[ )d [ ld ^[ ^z) where prediction errors and serve as operational parameters for TPB prediction/reshaping related operations. [0122] For simplicity, the three-dimensional (3D) indexes 5^, 5?, and 5x of the three sets of knot points may be vectorized as a one-dimensional (1D) index denoted as 5) to simplify the expression. Hence, the TPB basis function functions denoted with the 3D indexes may be alternatively denoted using the 1D index 5) as follows: !(^%,^ ^^ (^^ ^(^, ^), ^y, ^z) = !(^%,^ ^d,^S,^^ (^^ ^(^, ^), ^y, ^z) expression (17) above can alternatively expressed as follows: )^ ^[ ^ s^ ^(^, ^) = ^^ (^%,^(^^ ^(^, ^), ^y, ^z) = ^ ^ ^(^%,^ (^%,^ ^,^^ ∙ !^^ (^^ ^(^, ^), ^y, ^z) TPB basis functions and input pixel values or codewords of P pixels selected or collected from T input or source wide view HDR/WCG images within the same scene as follows: s( ^^%,^ = ù ú qû frame index for the k-th pixel; (mk, nk) represent the pixel coordinates or encoded positional information for the k-th pixel. [0125] The TPB (prediction) coefficients in the forward path may be collectively represented as a vector as follows: é ^ (^%,^ ^,i ( ù ^( ê ^^ ^ ú ú ú û in the forward path – forward reshaping or or codewords (with coordinate information) of the P pixels selected or collected from the T input or source wide view HDR/WCG images within the same scene to corresponding output luma pixel values or codewords (denoted as ^^^ ) of corresponding P pixels in T output wide view BL images within the same scene – can be represented as follows: ^^^ = s( ^^%,^^( ^^%,^ [0127] vector (denoted as ^^ ) for measuring prediction errors of the predicted output luma pixel values or codewords (^^^ ) may be constructed from reference luma pixel values or codewords of corresponding P pixels in corresponding (e.g., fused, etc.) reference wide view BL images within the same scene, as follows: é s ^ ^^ (^i, ^i) ^ ù ú ú ú û [0128] TPB (prediction) coefficient (^%,x s ^^ can be obtained via the least squared solution to the problem of minimizing prediction errors as measured between the ground truth vector (^^ ) and the predicted output luma pixel values or codewords (^^^ ) using the design matrix in expression (20) above and the ground truth vector in expression (23) above, as follows: ^(^%,^,(^^^) ^ = ((s( ^^%,^)(s( ^^%,^)^[((s( ^^%,^)(^^ ) vector can be constructed using the design matrix in expression (20) above and the ground truth vector in expression (23) above as follows: ^^ ^ = ((s( ^^%,^)(s( ^^%,^ (25-1) ^^ ^ = (s( ^^%,^)(^^ [0130] the optimized values ^(^%,^,(^^^) ^ of the TPB (prediction) coefficients ^(^%,x ^ can be obtained as follows: (^%,^,(^ ^ ^^) ^ =(^^ ^)^[^^ ^ [0131] such as a C0 (or Cb) or C1 (or Cr) chroma pixel values or codewords (denoted as ^^ ^ i(^, ^) or ^^ ^[(^, ^)) of the P pixels in the T input or source wide view HDR/WCG images may be used along with positional information of the pixels to construct a design matrix (denoted as s( ^^%,^i or s( ^^%,^[) for the chroma channel similar to the design matrix in expression above for the luma channel. A ground truth vector (denoted as s^ ^i(^, ^) or s^ ^ [(^, ^)) for the chroma channel may also be constructed from channel pixel values or codewords of the corresponding P pixels in the corresponding (e.g., fused, etc.) reference wide view BL images within the same scene. In addition, a matrix (denoted as ^^ ^ , where C is C0 or C1) or a vector (denoted as ^^ ^) for the chroma channel can be constructed similarly to the matrix or the vector for the luma channel in expressions (25) above. Hence, optimized values ^(^%,^,(^^^) ^ of TPB (prediction) coefficients can be obtained or represented as ^ with backward reshaping operations may be referred to as a backward path herein. TPB optimization can be performed in the backward path similar to the foregoing TPB optimization in the forward path. [0133] For example, a TPB backward reshaping mapping/function can be used to take (1) an input luma pixel value or codeword (denoted as ^^^ ^(^, ^), where m and n represent pixel (location) coordinates of a pixel) of an (or t-th) forward reshaped wide view BL image, (2) input x-axis positional information represented by ^y encoded from m, and (3) input y-axis positional information represented by ^y encoded from n to predict an output luma pixel value or codeword (denoted as ^^^ ^(^, ^)) in a (or t-th) reconstructed or backward reshaped wide view image for the luma channel in the backward path. [0134] The prediction in the backward path for the luma channel of the t-th backward reshaped wide view image can be performed using similar TPB basis functions in expressions (16) and (18) above as follows: ^^^ ^(^, ^) = ^% (^%,^(^^^ ^ ^[ ^(^, ^), ^y, ^z) = ∑) ^^ ^i ^(^%,^ (^%,^ %,^^ ∙ !^^ (^^^ ^(^, ^), ^y, ^z) values can be found or determined through an optimization process in the path similar to that of the optimization process in the forward path. [0135] A design matrix s( %^%,^ may be constructed for the backward path using the TPB basis functions and input pixel values or codewords of P pixels selected or collected from T forward reshaped wide view BL images within the same scene as follows: s( %^%,^ = ^ ^ ù ú [ frame index for the k-th pixel; (mk, nk) represent the pixel coordinates or encoded positional information for the k-th pixel. [0136] The TPB (prediction) coefficients in the backward path may be collectively represented as a vector as follows: ^ (^%,^ é %,i ù ú ú ú û [0137] in the backward path – backward reshaping or mapping the forward reshaped pixel values or codewords (with coordinate information) of the P pixels selected or collected from the T forward reshaped wide view BL images within the same scene to corresponding output luma pixel values or codewords (denoted as ^^^ ) of corresponding P pixels in T output backward reshaped wide view images within the same scene – can be represented as follows: ^^^ = s( %^%,^^( %^%,^ (31) [0138] A ground truth vector (denoted as ^^ ) for measuring prediction errors of the predicted output luma pixel values or codewords (^^^ ) may be constructed from the input or source luma pixel values or codewords of corresponding P pixels in the corresponding input or source wide view HDR/WCG images within the same scene, as follows: é ^ ^ ^^ (^i, ^i) ^ ù ^^^ ^ ú ú ú û [0139] TP (^%,x B (prediction) coefficients ^% can be obtained via the least squared solution to the problem of minimizing prediction errors as measured between the ground truth vector (^^ ) and the predicted output luma pixel values or codewords (^^^ ) using the design matrix in expression (29) above and the ground truth vector in expression (32) above, as follows: ^(^%,^,(^^^) % = ((s( %^%,^)(s( %^%,^)^[((s( %^%,^)(^^ ) [0140] vector can be constructed using the design matrix in expression (29) above and the ground truth vector in expression (32) above as follows: ^^ % = ((s( %^%,^)(s( %^%,^ ^^ % = (s( %^%,^)(^^ (34-2) [0141] Correspondingly, the optimized values ^(^%,^,(^^^) % of the TPB (prediction) coefficients ^( %^%,^ can be as follows: ^(^%,^,(^^^) % =(^^ % )^[^^ % [0142] such as a C0 (or Cb) or C1 (or Cr) chroma channel, forward reshaped chroma BL pixel values or codewords of the P pixels in the T forward reshaped wide view BL images may be used along with positional information of the pixels to construct a design matrix for the chroma channel similar to the design matrix in expression (29) above for the luma channel. A ground truth vector for the chroma channel – similar to the ground truth vector in expression (32) above – may also be constructed from corresponding chroma channel pixel values or codewords of the corresponding P pixels in the corresponding input or source wide view HDR/WCG images within the same scene. In addition, a matrix or a vector for the chroma channel can be constructed similarly to the matrix or the vector for the luma channel in expressions (34) above. Hence, optimized values ^(^%,^,(^^^) % of TPB (prediction) coefficients can be obtained or represented as follows: ^(^%,^,(^^^) % =(^^ % )^[^^ % global reshaping can suffer a relatively large amount of color shift in the reshaped or predicted images. In comparison, images with local reshaping based at least in part on positional encoding can support a relatively high degree of color precision, thereby preventing, avoiding or reducing color shift in the reshaped or predicted images. Also, it may be observed that, in many operational scenarios, differences in PSNR measurements/values between reconstructed or backward reshaped images generated from the local reshaping with positional encoding and corresponding reconstructed or backward reshaped images generated from the global reshaping without positional encoding may be relatively significant (e.g., ~60dB, etc.) in favor of the local reshaping. In some operational scenarios, different types of positional encoding such as linear positional encoding, non-linear positional encoding with sine/cosine functions may yield different image quality measurements/values such as different PSNR measurements/values in reconstructed or backward reshaped images generated from the local reshaping performed with the different types of positional encoding. In these operational scenarios, a specific type among the different types of positional encoding corresponding to the best image quality measurements/values can be selected, implemented and/or signaled with image metadata to downstream recipient device(s) for the purpose of allowing the downstream recipient device(s) to generate relatively high quality reconstructed or backward reshaped images. Joint Forward and Backward TPBC Optimization using BESA [0144] In the forward path, forward reshaping may be performed on an input or source wide view HDR/WCG signal or images therein to make forward reshaped output wide view BL signal or images therein as close to (e.g., fused, etc.) reference wide view BL images as possible (e.g., with minimized prediction errors, etc.). At the same time, in the backward path, backward reshaping may be performed on the forward reshaped output wide view BL signal or images therein to make reconstructed or backward reshaped output wide view HDR/WCG signal or images therein as close to the input or source wide view HDR/WCG signal or images therein as possible (e.g., with minimized prediction errors, etc.). [0145] In some operational scenario, an iterative algorithm such as the BESA algorithm may be used to generate optimized values for prediction/reshaping operations such as TPB prediction/reshaping operations. The algorithm can modify the reference BL signal iteratively, such that prediction errors between the (final; post-iteration) reconstructed or backward reshaped output wide view HDR/WCG signal or images therein and the input or source wide view HDR/WCG signal or images therein are minimized. [0146] The BESA algorithm can be deployed or performed in each of some or all (e.g., three, etc.) color channels of a color space. By way of example but not limitation, these color channels may be luma and chroma channels and denoted as ch, which can be Y, Cb, and Cr. [0147] A superscript, λ, may be added as the iteration index in (1) the TPB forward reshaping function ^(^%,^^,(^) ^ , (2) the TPB backward reshaping function ^(^%,^^,(^) % , and (3) the F forward reshaped wide view base layer images ^^^^,(^) ^ within ^(^%,^^,(^) ^ p^^ ^^(^, ^), ^y, ^zq [0148] The same superscript, λ, may be added as the iteration index in the corresponding TPBC coefficient matrices ^(^%,^^,(^) (^%,^^,(^) ^ and ^% . [0149] Denote all the P (selected) pixel location (m, n) at time or frame index t within the scene as the set Φ. It will be explained in detail later how this set may be constructed. [0150] FIG.2E illustrates example process flow implementing the BESA algorithm to optimize reshaping mappings/functions, which may be implemented, for example, by a video encoder or processing blocks/components therein. The process flow comprises a processing block/component for initialization (not shown in FIG. 2E), which, at the first iteration (λ =0), sets or initializes modified reference wide view BL pixel values or codewords as those from the (e.g., original, post-fusion, fused, etc.) reference wide view BL images as follows: so^^,(^) ^ (^, ^) = s^ ^^(^, ^) (38) [0151] As illustrated in FIG.2E, the process flow for the BESA algorithm comprises a processing block/component for forward reshaping, which at each iteration, generates or optimizes the TPB (forward reshaping) coefficients or parameters ^(^%,^,(^) (^%,^^, ^ in the forward reshaping function ^ (^) ^ to minimize prediction errors or differences between forward reshaped wide view BL pixel values or codewords^ s^ ^^,(^)(^, ^) at the current iteration and the modified reference wide view BL pixel values or codewords so^ ^^,(^)(^, ^) at the current iteration, as follows: ( ^^%,^^,(^) ^ ^^,(^) ^ − s^ ( ) ^ ^ = _¡¢^£^ ^ ¤ so (^, ) ^ ^ ^, ^ (^, ^)¤ ^ where ^ s^^,(^) ^ (^, ^) = ^(^%,^^,(^) ^ p^^ ^^(^, ^), ^y, ^zq also comprises a processing block/component for backward reshaping, which, at each iteration, generates or optimizes the TPB (backward reshaping) coefficients or parameters ^(^%,^^,(^) % in the backward reshaping function ^(^%,^^,(^) % to minimize prediction errors or differences between backward reshaped wide view HDR/WCG pixel values or codewords ^^^ ^^,(^)(^, ^) at the current iteration and the input or source wide view HDR/WCG pixel values or codewords ^^ ^^(^, ^) at the current iteration, as follows: ( ^ ^( %^%,^^, ^) = _¡¢^£^ ^ ¤ ^^ ^^(^, ^) − ^^^ ^^,(^)(^, ^)¤ ^ where ^^^ ^^,(^)(^, ^) = ^(^%,^^,(^) ^^,(^) % ^^^ ^ (^, ^), ^y, ^z  further comprises a processing block/component for computing errors, which, at the end of the current (or each) iteration, based on the backward reshaped wide view HDR/WCG pixel values or codewords and the (original) input or source wide view HDR/WCG pixel values or codewords, determines an amount denoted as §¨̃^ ^^,(^)(^, ^) in the computed prediction errors/values. This error amount §¨̃^ ^^,(^)(^, ^) may be used to modify or update the (current) modified reference wide view BL pixel values or codewords for the entire set Φ into the modified reference wide view BL pixel values or codewords so^^,(^ª[) ^ (^, ^) for the next iteration, and may be a reduced amount as compared with In some operational scenarios, the error amount §¨̃^ ^^,(^)(^, ^) may be computed or determined as a function of ^^^ ( ) ^^, ^ (^, ^) and ^^ ^^(^, ^), as follows: §¨̃ ( ) ( ) ^^^, ^ (^, ^) = ¢(^^ ^^(^, ^), ^^^ ^^, ^ (^, ^)) can be as simple as a subtraction function as follows: §¨̃^ ^^,(^)(^, ^) = ^^ ^^(^, ^) − ^^^ ^^,(^)(^, ^) amount §¨̃^ ^^,(^)(^, ^) or the ¢() function may at least in part depend on a gradient of the backward reshaping function. [0156] Hence, the modified reference BL value is updated for the next iteration for the entire set Φ as follows: so^^,(^ª[) ^ (^, ^) = so^^,(^) ^ (^, ^) + §¨̃^ ^^,(^)(^, ^) [0157] flow for the BESA termination are met. For example, prediction errors or differences/distortions of the backward reshaped wide view HDR/WCG pixel values or codewords and of the forward reshaped wide view BL pixel values or codewords can be respectively computed as follows: ^^,( ^ ^" ^) = (^,=,H)∈¦ ¤ ^^ ^ ^(^, ^) − ^^^ ^^,(^)(^, ^)¤ ^ [0158] If the termination conditions are not satisfied, the iteration index is incremented as follows: « = « + 1 (47) [0159] And the process flow goes back to the processing block/component for forward reshaping. [0160] On the other hand, the process flow can go to the next iteration until a pre- set or preconfigured maximum iteration count is reached or until a weighted combination of the prediction errors or differences or distortions in expressions (46) falls below a weighted prediction error threshold ¬, as follows: ^^^,(^) = g" ⋅ ^" ^^,(^) + g# ⋅ ^# ^^,(^) [0161] If one cares (e.g., only, etc.) about minimizing HDR/WCG distortions, then g" can be set to one (1) and g# can be set to zero (0). [0162] The foregoing BESA optimization process flow can be denoted in a functional form as follows: {^(^%,^^,(^^^), ^( ( ) %^%,^^, ^^^ } = !®¯°( ^^ ^^ , s^ ^^ , ±) Example Memory and Computational Efficient Optimization [0163] In some operational scenarios, pixel values or codewords of all available pixels of each of all images in a scene may be used to generate optimized values for TPBC coefficients in TPB reshaping functions/mappings. A process flow under this approach may use relatively large amounts of memory and computing resources. [0164] In some operational scenarios, pixel values or codewords of a relatively small subset of all available pixels of each of all images in a scene may be used to generate optimized values for TPBC coefficients in TPB reshaping functions/mappings. A process flow under this approach may use relatively limited amounts of memory and computing resources and may contain multiple stages such as three stages illustrated in FIG.2F. [0165] In each (e.g., later, etc.) stage of the process flow of FIG.2F, TPBC reshaping functions can be generated or optimized by fitting or modeling on an incremental data set, which includes an addition of the most distorted pixels (e.g., as measured or compared with an applicable distortion threshold, etc.) from a previous stage. [0166] In stage 1 of the process flow of FIG.2F, TPBC reshaping functions are generated or optimized by fitting or modeling on a first data set comprising pixel values or codewords in down-sampled images derived from input or source wide view HDR/WCG images ^^ ^ ^ in a scene for a color channel ch such as a luma or non-luma channel, non-luma as well as derived from corresponding (fused) wide view BL images s^ ^ ^ in the scene for a corresponding color channel ch such as the same luma or non-luma channel with a different bit depth or dynamic ranges. [0167] FIG.2G illustrates an example detailed process flow corresponding to stage 1 of the process flow of FIG.2F. [0168] The process flow of FIG.2G in stage 1 comprises processing blocks or components for lexical ordering, which apply a lexicographical or dictionary order to (pixel) rows in the input or source wide view HDR/WCG images and concatenate the ordered rows into a first image construct (e.g., a first row vector, an artificial HDR/WCG image not for rendering, etc.) ^′^ ^^ = ³h`£´_³_¶¡^h¡(^^ ^^), and apply the same order used in ordering the (pixel) rows in the input or source HDR/WCG images to order and concatenate (pixel) rows in the corresponding (fused) reference wide view BL images into a second image construct (e.g., a second row vector, an artificial BL image not for rendering, etc.) s′^ ^ ^ = ³h`£´_³_¶¡^h¡(s^ ^ ^). [0169] The process flow of FIG.2G in stage 1 comprises a processing block or component for downsampling, which (e.g., uniformly, etc.) downsamples ^′^ ^^ by a constant factor denoted as ^· (e.g., a prime number, 101, etc.). For example, all rows in ^′^ ^ ^ with and all corresponding row in s′^ ^ ^ with indices i such that £ % ^· = 1 are collected into the first data set ±[, as follows: ±[ = { ∀£|£ % ^· = 1} [0170] FIG.2G in stage 1 comprises a processing block or component or performs a first round of BESA optimization using pixel set ±[ to determine TPB coefficients for constructing TPB forward and backward reshaping functions/mappings denoted respectively as ^^ ( ,º ^% ^ and ^% ( ,^ º% ^, as follows: [) a processing block or component denoted as “Apply TPBC 1”, which applies the TPB forward reshaping function ^^ ( ,º ^% ^ to the input or source wide view HDR/WCG images ^^ ^^ to obtain or generate corresponding forward reshaped wide view BL images denoted as^ s^ ^, ^ º^ , and applies the TPB backward reshaping function ^% ( ,^ º% ^ to the forward reshaped wide view BL images^ s^ ^, ^ º^ to obtain or generate corresponding reconstructed or backward reshaped wide view HDR/WCG images denoted as ^ ^^ ^, ^ º^ , as follows: ^ s^ ^, ^ º^ = ^^ ( ,º ^% ^(^^ ^^) ) [0172] The process flow of FIG.2G in stage 1 comprises a processing block or component for determining a pixel set ±[,» with relatively large distortions after determining^ s^ ^, ^ º^ and ^^^ ^, ^ º^ . The pixel set (±[,») can be determined by comparing the reshaped pixel with corresponding pixel values or codewords in the input wide view HDR/WCG images and the (fused) reference wide view BL images, as follows: ±[,» = { ∀£|(¼^r^ ^^(£) − ^^r ^, ,^ º^ ^ (£)¼ ≥ ¬¾ ^ ,º ^ ^ )¿O( ¼sr^ ^^(£) −^ sr ^, ,^ º^ ^ (£)¼ ≥ ¬' ^ ,^ º^ ) } ^^r ^, ,^ º^ ^ = ³h`£´_³_¶¡^h¡(^^^ ^, ^ º^ ) ) ¾^ , ^ º^ º^ represent two distortion thresholds for the channel ch. values of these distortion thresholds for ch= Y, Cb, and Cr are illustrated in 1 below (in 10 bits precision). TABLE 1 ch Y Cb Cr ¬¾ ^ ,º ^ ^ 20 4 4 ¬' ^ ,^ º^ 2 2 2 [0173] FIG.2H illustrates an example detailed process flow corresponding to stage 2 of the process flow of FIG.2F. [0174] The process flow of FIG.2H in stage 2 comprises processing blocks or components similar to those in the process flow of FIG.2G in stage 1. As shown, in this stage, a second data set ±^ may be in BESA operations. The second data set ±^ combines both (1) the down sampled image pixels or pixel values with indices ±[ and (2) pixels or pixel values with the relatively large distortions with indices ± from stage 1, as follows: ±^ = ±[ ∪ ±[,» [0175] Other processing blocks or components in stage 2 may be performed with this second data set ±^ similar to show the corresponding processing blocks or components in stage 1 is performed with the first data set ±[. [0176] For example, the second round of BESA optimization may be performed using pixel set ±^ to determine TPB coefficients for constructing TPB forward and backward reshaping functions/mappings denoted respectively as ^^ ( ,º ^% l and ^% ( ,^ º% l, as follows: {^( ^^ ,^ %,^^,(^^^), ^( %^ ,^ %,^^,(^^^)} = ({^^ ^ ^}, {s^ ^ ^}, ±^) be applied to the input or source or corresponding forward reshaped wide view BL images denoted as^ s^ ^, ^ ºl , and the TPB backward reshaping function ^% ( ,^ º% l may be applied to the forward reshaped wide view BL images ^ s^ ^, ^ ºl to obtain or generate corresponding reconstructed or backward reshaped wide view HDR/WCG images denoted as ^^^ ^, ^ ºl , as follows: ^ s^ ^, ^ ºl = ^^ ( ,º ^% l(^^ ^^) (57-1) ^^^ ) [0178] A pixel set ±^,» with relatively large distortions may be determined after determining ^ s^ ^, ^ ºl and ^^^ ^, ^ ºl . The pixel set (±^,») can be determined by comparing the reshaped pixel with corresponding pixel values or codewords in the input wide view HDR/WCG images and the (fused) reference wide view BL images, as follows: ±^,» = { ∀£|(¼^r^ ^^(£) −^ ^r ^, ,^ º^ l (£)¼ ≥ ¬¾ ^ ,^ ºl )¿O( ¼sr^ ^^(£) −^ sr ^, ,^ º^ l (£)¼ ≥ ¬' ^ ,^ ºl ) } where ^^r ^, ,^ º^ l = ³h`£´_³_¶¡^h¡(^^^ ^, ^ ºl ) ) ¬¾ ^ ,^ ºl and ¬' ^ ,^ ºl represent two predefined distortion thresholds for the channel ch. Example values of these distortion thresholds for ch= Y, Cb, and Cr are illustrated in TABLE 2 below (in 10 bits precision). TABLE 2 ch Y Cb Cr ¬¾ ^ ,^ ºl 20 4 4 ¬' ^ ,^ ºl 2 2 2 [0179] FIG.2I illustrates an example detailed process flow corresponding to stage 3 of the process flow of FIG.2F. [0180] The process flow of FIG.2I in stage 3 comprises processing blocks or components similar to those in the process flow of FIG.2G or FIG.2H in stage 1 or 2. As shown, in this stage, a third data set ±Á may be in BESA operations. The second data set ±Á combines both (1) the down sampled image pixels or pixel values with indices ±[, (2) pixels or pixel values with the relatively large distortions with indices ±[,» from stage 1, and (3) pixels or pixel values with the relatively large distortions with indices ±^,» from stage 2, as follows: ±Á = ±^ ∪ ±^,» (60) [0181] Other processing blocks or components in stage 3 may be performed with this second data set ±Á similar to show the corresponding processing blocks or components in stage 2 is performed with the second data set ±^. [0182] For example, the third round of BESA optimization may be performed using pixel set ±Á to determine TPB coefficients for constructing TPB forward and backward reshaping functions/mappings denoted respectively as ^^ ( ,º ^%  and ^% ( ,^ º% Â, as follows: {^( ^^%,^^,(^^^), ^( %^%,^^,(^^^)} = ({^^ ^ ^}, {s^ ^ ^}, ±Á) the final stage. In some other operational scenarios, more or fewer stages other than three stages may be implemented. [0184] In many operational scenarios, the 3-stage optimization as described herein can generate images with image qualities, for example as measured with PSNR values illustrated in TABLE 3 below, better than or comparable to all- or full-pixel optimization that utilizes all (non-downsampled) pixels or pixel values. Hence, the 3- stage optimization can be implemented or perform to reduce memory usage as well as provide better reconstruction quality. TABLE 3 Y Cb Cr Stage 1 38.02 59.05 55.83 Stage 1,2 56.84 53.10 52.48 Stage 1,2,3 57.12 59.15 59.92 No down sampling 55.52 59.41 58.79 [0185] Hence, local reshaping that incorporates positional information may be implemented or performed to efficiently and effectively compress input or source wide view HDR/WCG videos or images therein using a wide variety of available video codecs including but not limited to regular gamma-based10-bit video codecs. This local reshaping approach can prevent banding artifacts locally and preserve colors. In some operational scenarios, a patch-based mapping method with a relatively advanced fusion method can be used to generate reference wide view base layer videos or images therein corresponding to the input or source wide view HDR/WCG videos or images therein. In some operational scenarios, local reshaping can be implemented with TPB forward and backward reshaping functions with coordinate or positional information. Example Process Flows [0186] FIG.4A illustrates an example process flow according to an embodiment. In some embodiments, one or more computing devices or components (e.g., an encoding device/module, a transcoding device/module, an upstream device, a sender, etc.) may perform this process flow. In block 402, an image processing system generates one or more reference wide view images of a first domain (e.g., a first color space, a first dynamic range, a first bit depth image/signal, etc.) from one or more source wide view images of a second domain (e.g., a second color space, a second dynamic range, a second bit depth image/signal, etc.). [0187] In block 404, the image processing system generates a forward reshaping mapping to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain. [0188] In block 406, the image processing system generates a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain. [0189] Each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data. [0190] In block 408, the image processing system encodes the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images. The corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings. [0191] In an embodiment, the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth; the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth. [0192] In an embodiment, the positional data is derived from the pixel locations using one of a linear function or a non-linear function. [0193] In an embodiment, the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch- specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images; the patch-specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non- overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain. [0194] In an embodiment, a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel. [0195] In an embodiment, one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches. [0196] In an embodiment, at least one of the forward and backward reshaping mappings is optimized using a Backward Error Subtraction Algorithm (BESA) algorithm. [0197] In an embodiment, at least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images. [0198] In an embodiment, the pixels in the downsampled pixel set are downsampled from a sequence of pixel rows, ordered in a lexicographical order, in the one or more source wide view images and the one or more reference wide view images. [0199] In an embodiment, at least one of the forward and backward reshaping mappings is optimized using multiple stages; in each of the multiple stages, a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a proper set of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images. [0200] In an embodiment, at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images. [0201] FIG.4B illustrates an example process flow according to an embodiment. In some embodiments, one or more computing devices or components (e.g., a decoding device/module, a transcoding device/module, a downstream device, a recipient, etc.) may perform this process flow. In block 452, an image processing system decodes one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream. The one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping. The corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping. [0202] In block 454, the image processing system generates one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping. [0203] In block 456, the image processing system generates one or more display images from the one or more reconstructed wide view images. [0204] In block 458, the image processing system renders the one or more display images on an image display. [0205] In an embodiment, the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display. [0206] In an embodiment, a computing device such as a display device, a mobile device, a set-top box, a multimedia device, etc., is configured to perform any of the foregoing methods. In an embodiment, an apparatus comprises a processor and is configured to perform any of the foregoing methods. In an embodiment, a non- transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of any of the foregoing methods. [0207] In an embodiment, a computing device comprising one or more processors and one or more storage media storing a set of instructions which, when executed by the one or more processors, cause performance of any of the foregoing methods. [0208] Note that, although separate embodiments are discussed herein, any combination of embodiments and/or partial embodiments discussed herein may be combined to form further embodiments. Example Computer System Implementation [0209] Embodiments of the present invention may be implemented with a computer system, systems configured in electronic circuitry and components, an integrated circuit (IC) device such as a microcontroller, a field programmable gate array (FPGA), or another configurable or programmable logic device (PLD), a discrete time or digital signal processor (DSP), an application specific IC (ASIC), and/or apparatus that includes one or more of such systems, devices or components. The computer and/or IC may perform, control, or execute instructions relating to the adaptive perceptual quantization of images with enhanced dynamic range, such as those described herein. The computer and/or IC may compute any of a variety of parameters or values that relate to the adaptive perceptual quantization processes described herein. The image and video embodiments may be implemented in hardware, software, firmware and various combinations thereof. [0210] Certain implementations of the inventio comprise computer processors which execute software instructions which cause the processors to perform a method of the disclosure. For example, one or more processors in a display, an encoder, a set top box, a transcoder or the like may implement methods related to adaptive perceptual quantization of HDR images as described above by executing software instructions in a program memory accessible to the processors. Embodiments of the invention may also be provided in the form of a program product. The program product may comprise any non-transitory medium which carries a set of computer- readable signals comprising instructions which, when executed by a data processor, cause the data processor to execute a method of an embodiment of the invention. Program products according to embodiments of the invention may be in any of a wide variety of forms. The program product may comprise, for example, physical media such as magnetic data storage media including floppy diskettes, hard disk drives, optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, flash RAM, or the like. The computer-readable signals on the program product may optionally be compressed or encrypted. [0211] Where a component (e.g. a software module, processor, assembly, device, circuit, etc.) is referred to above, unless otherwise indicated, reference to that component (including a reference to a "means") should be interpreted as including as equivalents of that component any component which performs the function of the described component (e.g., that is functionally equivalent), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated example embodiments of the invention. [0212] According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques. [0213] For example, FIG.5 is a block diagram that illustrates a computer system 500 upon which an embodiment of the invention may be implemented. Computer system 500 includes a bus 502 or other communication mechanism for communicating information, and a hardware processor 504 coupled with bus 502 for processing information. Hardware processor 504 may be, for example, a general purpose microprocessor. [0214] Computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 502 for storing information and instructions to be executed by processor 504. Main memory 506 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 504. Such instructions, when stored in non-transitory storage media accessible to processor 504, render computer system 500 into a special-purpose machine that is customized to perform the operations specified in the instructions. [0215] Computer system 500 further includes a read only memory (ROM) 508 or other static storage device coupled to bus 502 for storing static information and instructions for processor 504. A storage device 510, such as a magnetic disk or optical disk, is provided and coupled to bus 502 for storing information and instructions. [0216] Computer system 500 may be coupled via bus 502 to a display 512, such as a liquid crystal display, for displaying information to a computer user. An input device 514, including alphanumeric and other keys, is coupled to bus 502 for communicating information and command selections to processor 504. Another type of user input device is cursor control 516, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 504 and for controlling cursor movement on display 512. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane. [0217] Computer system 500 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 500 to be a special-purpose machine. According to one embodiment, the techniques as described herein are performed by computer system 500 in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage device 510. Execution of the sequences of instructions contained in main memory 506 causes processor 504 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. [0218] The term “storage media” as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operation in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 510. Volatile media includes dynamic memory, such as main memory 506. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge. [0219] Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 502. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications. [0220] Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 504 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 500 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 502. Bus 502 carries the data to main memory 506, from which processor 504 retrieves and executes the instructions. The instructions received by main memory 506 may optionally be stored on storage device 510 either before or after execution by processor 504. [0221] Computer system 500 also includes a communication interface 518 coupled to bus 502. Communication interface 518 provides a two-way data communication coupling to a network link 520 that is connected to a local network 522. For example, communication interface 518 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 518 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 518 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information. [0222] Network link 520 typically provides data communication through one or more networks to other data devices. For example, network link 520 may provide a connection through local network 522 to a host computer 524 or to data equipment operated by an Internet Service Provider (ISP) 526. ISP 526 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 528. Local network 522 and Internet 528 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 520 and through communication interface 518, which carry the digital data to and from computer system 500, are example forms of transmission media. [0223] Computer system 500 can send messages and receive data, including program code, through the network(s), network link 520 and communication interface 518. In the Internet example, a server 530 might transmit a requested code for an application program through Internet 528, ISP 526, local network 522 and communication interface 518. [0224] The received code may be executed by processor 504 as it is received, and/or stored in storage device 510, or other non-volatile storage for later execution. Equivalents, Extensions, Alternatives and Miscellaneous [0225] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is claimed embodiments of the invention, and is intended by the applicants to be claimed embodiments of the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. Enumerated Exemplary Embodiments [0226] The invention may be embodied in any of the forms described herein, including, but not limited to the following Enumerated Example Embodiments (EEEs) which describe structure, features, and functionality of some portions of embodiments of the present invention. [0227] EEE1. A method comprising: generating one or more reference wide view images of a first domain from one or more source wide view images of a second domain; generating a forward reshaping mapping to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain; generating a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain; wherein each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data; encoding the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings. [0228] EEE2. The method of EEE1, wherein the forward reshaping mapping receives source pixel values of the one or more source wide view images and source positional data derived from source pixel coordinates as input and generates forward reshaped pixel values of the one or more forward reshaped wide view images. [0229] EEE3. The method of EEE1 or EEE2, wherein the backward reshaping mapping receives forward reshaped pixel values of the one or more forward reshaped wide view images and source positional data derived from source pixel coordinates as input and generates backward reshaped pixel values of the one or more reconstructed wide view images. [0230] EEE4. The method of any of EEE1-EEE3, wherein the forward reshaping mapping includes a chroma forward reshaping mapping that maps source chrominance pixel values of the one or more source wide view images into forward reshaped chrominance pixel values of the one or more forward reshaped wide view images based at least in part on a scaling factor; wherein the scaling factor represents a ratio of a first difference between maximum and minimum source luminance pixel values and a second difference between maximum and minimum source chrominance pixel values. [0231] EEE5. The method of any of EEE1-EEE4, wherein the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth; the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth. [0232] EEE6. The method of any of EEE1-EEE5, wherein the positional data is derived from the pixel locations using one of a linear function or a non-linear function. [0233] EEE7. The method of any of EEE1-EEE6, wherein the non-linear function is from a sinusoidal function family. [0234] EEE8. The method of any of EEE1-EEE7, wherein the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch-specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images; wherein the patch- specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non-overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; wherein the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain. [0235] EEE9. The method of any of EEE1-EEE8, wherein a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel. [0236] EEE10. The method of EEE9, wherein the weighted combination is derived from weighting factors that are given by Gaussian distribution functions with rectangular contours of equal values. [0237] EEE11. The method of EEE9, wherein one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches. [0238] EEE12. The method of any of EEE1-EEE11, wherein at least one of the forward and backward reshaping mappings is optimized using a Backward Error Subtraction Algorithm (BESA) algorithm. [0239] EEE13. The method of any of EEE1-EEE12, wherein at least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images. [0240] EEE14. The method of EEE13, wherein the pixels in the downsampled pixel set are downsampled from a sequence of pixel rows, ordered in a lexicographical order, in the one or more source wide view images and the one or more reference wide view images. [0241] EEE15. The method of any of EEE1-EEE14, wherein at least one of the forward and backward reshaping mappings is optimized using multiple stages; wherein the multiple stages include a beginning stage in which a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a first proper subset of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images. [0242] EEE16. The method of EEE15, wherein a first set of pixels with relatively large prediction errors is identified after the BESA algorithm in the beginning stage has been performed; wherein the multiple stages include a second stage, following the beginning stage, in which the BESA algorithm is performed based on a second proper subset of pixels derived as a set union of the first proper subset of pixels and the first set of pixels with the relatively large prediction errors. [0243] EEE17. The method of EEE16, wherein a second set of pixels with relatively large prediction errors is identified after the BESA algorithm in the second stage has been performed; wherein the multiple stages include a third stage, following the second stage, in which the BESA algorithm is performed based on a third proper subset of pixels derived as a second set union of the second proper subset of pixels and the second set of pixels with the relatively large prediction errors. [0244] EEE18. The method of any of EEE1-EEE17, wherein at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images. [0245] EEE19. A method comprising: decoding one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream, wherein the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping; generating one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping; generating one or more display images from the one or more reconstructed wide view images; rendering the one or more display images on an image display. [0246] EEE20. The method of EEE19, wherein the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display. [0247] EEE21. An apparatus comprising a processor and configured to perform any one of the methods recited in EEE1-EEE20. [0248] EEE22. A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with any of the methods recited in EEE1-EEE20.

Claims

CLAIMS What is claimed is: 1. A method comprising: generating one or more reference wide view images of a first domain from one or more source wide view images of a second domain, including extracting overlapped patches from each of the one or more source wide view images of the second domain; generating a forward reshaping mapping to forward reshape the one or more source wide view images into one or more forward reshaped wide view images of the first domain which approximate the one or more reference wide view images; generating a backward reshaping mapping to backward reshape the one or more forward reshaped wide view images into one or more reconstructed wide view images of the second domain which approximate the one or more source wide view images; wherein each of the forward and backward reshaping mappings is generated based at least in part on inputs that include pixel level image data and positional data derived from pixel locations represented in the pixel level image data; encoding the one or more forward reshaped wide view images and corresponding image metadata into a bitstream to enable a recipient device of the bitstream to generate one or more display images from the one or more reconstructed wide view images, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward and backward reshaping mappings.
2. The method of Claim 1, wherein the forward reshaping mapping receives source pixel values of the one or more source wide view images and source positional data derived from source pixel coordinates as input and generates forward reshaped pixel values of the one or more forward reshaped wide view images.
3. The method of Claim 1 or 2, wherein the backward reshaping mapping receives forward reshaped pixel values of the one or more forward reshaped wide view images and source positional data derived from source pixel coordinates as input and generates backward reshaped pixel values of the one or more reconstructed wide view images.
4. The method of any of Claims 1-3, wherein the forward reshaping mapping includes a chroma forward reshaping mapping that maps source chrominance pixel values of the one or more source wide view images into forward reshaped chrominance pixel values of the one or more forward reshaped wide view images based at least in part on a scaling factor; wherein the scaling factor represents a ratio of a first difference between maximum and minimum source luminance pixel values and a second difference between maximum and minimum source chrominance pixel values.
5. The method of any of Claims 1-4, wherein the first domain is represented by one of a first dynamic range, a first color gamut, or a first bit depth; the second domain is represented by one of a second dynamic range higher than the first dynamic range, a second color gamut wider than the first color gamut, or a second bit depth higher than the first bit depth.
6. The method of any of Claims 1-5, wherein the positional data is derived from the pixel locations using one of a linear function or a non-linear function.
7. The method of any of Claims 1-6, wherein the non-linear function is from a sinusoidal function family.
8. The method of any of Claims 1-7, wherein the one or more source wide view images are partitioned into non-overlapped image patches of the second domain; wherein patch-specific forward reshaping functions are generated from patch-specific image data statistics computed from image data of first overlapped image patches of the second domain, corresponding to the non- overlapped image patches, in the one or more source wide view images; wherein the patch-specific forward reshaping functions are applied to second overlapped image patches of the second domain, corresponding to the non- overlapped image patches, in the one or more source wide view images to generate forward reshaped overlapped image patches of the first domain; wherein the forward reshaped overlapped image patches of the first domain are fused into the one or more reference wide view images of the first domain.
9. The method of Claims 8, wherein a pixel value at a pixel of the one or more reference wide view images of the first domain is derived as a weighted combination of one or more pixel values of the pixel from one or more forward reshaped overlapped image patches covering the pixel.
10. The method of Claim 9, wherein the weighted combination is derived from weighting factors that are given by Gaussian distribution functions with rectangular contours of equal values.
11. The method of Claim 9, wherein one or more individual weights of the one or more pixel values are specified in one or more patch-specific weight maps of the one or more forward reshaped overlapped image patches.
12. The method of any of Claims 1-11, wherein at least one of the forward and backward reshaping mappings is optimized using a Backward Error Subtraction Algorithm (BESA) algorithm.
13. The method of any of Claims 1-12, wherein at least one of the forward and backward reshaping mappings is optimized using a downsampled pixel set that includes pixels downsampled from the one or more source wide view images and the one or more reference wide view images.
14. The method of Claim 13, wherein the pixels in the downsampled pixel set are downsampled from a sequence of pixel rows, ordered in a lexicographical order, in the one or more source wide view images and the one or more reference wide view images.
15. The method of any of Claims 1-14, wherein at least one of the forward and backward reshaping mappings is optimized using multiple stages; wherein the multiple stages include a beginning stage in which a Backward Error Subtraction Algorithm (BESA) algorithm is performed based on a first proper subset of pixels in all pixels represented in the one or more source wide view images and the one or more reference wide view images.
16. The method of Claim 15, wherein a first set of pixels with relatively large prediction errors is identified after the BESA algorithm in the beginning stage has been performed; wherein the multiple stages include a second stage, following the beginning stage, in which the BESA algorithm is performed based on a second proper subset of pixels derived as a set union of the first proper subset of pixels and the first set of pixels with the relatively large prediction errors.
17. The method of Claim 16, wherein a second set of pixels with relatively large prediction errors is identified after the BESA algorithm in the second stage has been performed; wherein the multiple stages include a third stage, following the second stage, in which the BESA algorithm is performed based on a third proper subset of pixels derived as a second set union of the second proper subset of pixels and the second set of pixels with the relatively large prediction errors.
18. The method of any of Claims 1-17, wherein at least one of the forward and backward reshaping mappings is optimized to minimize prediction errors between predicted pixel values in the one or more reconstructed wide view images and source pixel values in the one or more source wide view images.
19. A method comprising: decoding one or more forward reshaped wide view images of a first domain and corresponding image metadata from a bitstream, wherein the one or more forward reshaped wide view images have been generated by an upstream device from forward reshaping one or more source wide view images of a second domain based at least in part on a forward reshaping mapping, wherein the corresponding image metadata includes operational parameters specifying at least one of the forward reshaping mapping or a backward reshaping mapping corresponding to the forward reshaping mapping; generating one or more reconstructed wide view images of the second domain from backward reshaping the one or more forward reshaped wide view images based at least in part on the backward reshaping mapping; generating one or more display images from the one or more reconstructed wide view images; rendering the one or more display images on an image display.
20. The method of Claim 19, wherein the one or more display images are adapted from the one or more reconstructed wide view images based at least in part on device capabilities of the image display.
21. An apparatus comprising a processor and configured to perform any one of the methods recited in Claims 1-20.
22. A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with any of the methods recited in Claims 1-20.
EP24711955.5A 2023-02-16 2024-02-15 Local reshaping using tensor-product b-spline with coordinates wide view video Pending EP4666581A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202363485427P 2023-02-16 2023-02-16
EP23156989 2023-02-16
PCT/US2024/015944 WO2024173649A1 (en) 2023-02-16 2024-02-15 Local reshaping using tensor-product b-spline with coordinates wide view video

Publications (1)

Publication Number Publication Date
EP4666581A1 true EP4666581A1 (en) 2025-12-24

Family

ID=90365217

Family Applications (1)

Application Number Title Priority Date Filing Date
EP24711955.5A Pending EP4666581A1 (en) 2023-02-16 2024-02-15 Local reshaping using tensor-product b-spline with coordinates wide view video

Country Status (2)

Country Link
EP (1) EP4666581A1 (en)
WO (1) WO2024173649A1 (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10593028B2 (en) * 2015-12-03 2020-03-17 Samsung Electronics Co., Ltd. Method and apparatus for view-dependent tone mapping of virtual reality images
US10223774B2 (en) 2016-02-02 2019-03-05 Dolby Laboratories Licensing Corporation Single-pass and multi-pass-based polynomial approximations for reshaping functions
US10032262B2 (en) 2016-02-02 2018-07-24 Dolby Laboratories Licensing Corporation Block-based content-adaptive reshaping for high dynamic range images
TWI812874B (en) * 2019-10-01 2023-08-21 美商杜拜研究特許公司 Tensor-product b-spline predictor
WO2022103902A1 (en) * 2020-11-11 2022-05-19 Dolby Laboratories Licensing Corporation Wrapped reshaping for codeword augmentation with neighborhood consistency

Also Published As

Publication number Publication date
WO2024173649A1 (en) 2024-08-22

Similar Documents

Publication Publication Date Title
US11277627B2 (en) High-fidelity full reference and high-efficiency reduced reference encoding in end-to-end single-layer backward compatible encoding pipeline
US10701375B2 (en) Encoding and decoding reversible production-quality single-layer video signals
US10264287B2 (en) Inverse luma/chroma mappings with histogram transfer and approximation
US10560707B2 (en) Chroma subsampling and gamut reshaping
EP3168809B1 (en) Image processing for hdr images
US10972759B2 (en) Color appearance preservation in video codecs
US11388408B2 (en) Interpolation of reshaping functions
ES3062794T3 (en) Adaptive local reshaping for sdr-to-hdr up-conversion
CN105052143A (en) Encoding perceptually-quantized video content in multi-layer VDR coding
US11818400B2 (en) Adjustable trade-off between quality and computation complexity in video codecs
US20240007682A1 (en) Wrapped reshaping for codeword augmentation with neighborhood consistency
JP7797682B2 (en) Image optimization in mobile capture and editing applications
JP7627389B2 (en) Noise removal for SDR-HDR local reconstruction
WO2024173649A1 (en) Local reshaping using tensor-product b-spline with coordinates wide view video
JP7827882B2 (en) Image optimization in mobile capture and editing applications
KR20180054623A (en) Determination of co-localized luminance samples of color component samples for HDR coding / decoding
EP4690094A1 (en) Image enhancement acceleration by patch-wise regression
HK1248904A1 (en) Inverse luma/chroma mappings with histogram transfer and approximation
HK1257716B (en) Encoding and decoding reversible production-quality single-layer video signals

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250903

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

P01 Opt-out of the competence of the unified patent court (upc) registered

Free format text: CASE NUMBER: UPC_APP_0004280_4666581/2026

Effective date: 20260206