EP4674129A1 - Guided conversions between two different dynamic ranges with new metadata - Google Patents
Guided conversions between two different dynamic ranges with new metadataInfo
- Publication number
- EP4674129A1 EP4674129A1 EP24705492.7A EP24705492A EP4674129A1 EP 4674129 A1 EP4674129 A1 EP 4674129A1 EP 24705492 A EP24705492 A EP 24705492A EP 4674129 A1 EP4674129 A1 EP 4674129A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- hdr
- sdr
- metadata
- interoperability
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/98—Adaptive-dynamic-range coding [ADRC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20208—High dynamic range [HDR] image processing
Definitions
- At least one of the present embodiments generally relates to the field of production of High Dynamic Range (HDR) video and more particularly to a method, a device and an equipment implementing a signaling mechanism allowing identifying an appropriate HDR to SDR or SDR to HDR conversion in a single master production and distribution system. This signaling mechanism allows interoperability of different equipment and technologies in a real workflow. 3.
- HDR High Dynamic Range
- High-dynamic-range video describes video having a dynamic range greater than that of standard-dynamic-range video (SDR video).
- HDR video involves capture, production, content/encoding, and display.
- HDR capture and display devices are capable of brighter whites and deeper blacks.
- HDR encoding standards allow for a higher maximum luminance and use at least a 10-bit dynamic range (compared to 8-bit for non-professional and 10-bit for professional SDR video) in order to maintain precision across this extended range.
- HDR production is a new domain and there will be a transition phase during which both HDR contents and SDR contents will coexist.
- a same live content will be produced simultaneously in a HDR and a SDR version.
- a user can then display the HDR or the SDR version of the content depending on his preferences or capabilities.
- the current trend of the content production industry is first, to produce HDR contents and then to derive automatically SDR contents from the HDR contents using 1 2023PF00065 automatic tools (this is known as single-master production and distribution workflow); and, second, to apply a controlled and safe approach for the HDR production to avoid delivering bad HDR or SDR contents to users.
- one or more of the present embodiments provide a method comprising obtaining input video data in an input dynamic range; obtaining interoperability metadata; transmitting the input video data and the interoperability 2 2023PF00065 metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value.
- the input dynamic range is HDR or SDR.
- the interoperability metadata are transmitted using ancillary channels of a SDI interface.
- the interoperability metadata are embedded in SL-HDR1 metadata. In an embodiment, wherein the interoperability metadata are transmitted using a SEI message. In a second aspect, one or more of the present embodiments provide a method comprising: obtaining input video data in an input dynamic range and interoperability metadata; converting the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value.
- the input dynamic range is HDR or SDR.
- the interoperability metadata are used to define a tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is HDR and the output dynamic range is SDR; and, the interoperability metadata are used to define an inverse tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is SDR and the output dynamic range is HDR.
- the interoperability metadata are received using ancillary channels of a SDI interface. 3 2023PF00065
- the interoperability metadata are embedded in SL-HDR1 metadata.
- the interoperability metadata are transmitted using a SEI message.
- one or more of the present embodiments provide a device comprising electronic circuitry configured for: obtaining input video data in an input dynamic range; obtaining interoperability metadata; transmitting the input video data and the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value.
- the input dynamic range is HDR or SDR.
- the interoperability metadata are transmitted using ancillary channels of a SDI interface.
- the interoperability metadata are embedded in SL-HDR1 metadata. In an embodiment, the interoperability metadata are transmitted using a SEI message. In a fourth aspect, one or more of the present embodiments provide a device comprising electronic circuitry configured for: obtaining input video data in an input dynamic range and interoperability metadata; converting the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value.
- the input dynamic range is HDR or SDR. 4 2023PF00065
- the interoperability metadata are used to define a tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is HDR and the output dynamic range is SDR and the interoperability metadata are used to define an inverse tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is SDR and the output dynamic range is HDR.
- the interoperability metadata are received using ancillary channels of a SDI interface.
- the interoperability metadata are embedded in SL-HDR1 metadata.
- the interoperability metadata are transmitted using a SEI message.
- one or more of the present embodiments provide a non-transitory information storage medium storing program code instructions for implementing the method according to the first or the second aspect.
- one or more of the present embodiments provide a signal comprising interoperability metadata comprising at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value.
- one or more of the present embodiments provide a computer program comprising program code instructions for implementing the method according to the first or the second aspect. 5.
- Fig.1A illustrates a scale of luminance values in which appears the diffuse white
- Fig.1B illustrates the separation of a scale of luminance values when the diffuse white is fixed to “203” nits
- Fig.1C illustrates schematically a context of various embodiments
- Fig.2 illustrates a Single-master HDR/SDR workflow
- Fig. 3A illustrates an example of luminance ITM curve compliant with characteristics provided by interoperability metadata
- Fig. 3B illustrates an example of luminance TM curve compliant with characteristics provided by interoperability metadata
- Fig.4 illustrates a Single-master HDR/SDR workflow according to an embodiment
- FIG. 5 illustrates an example of process executed by a video source according to an embodiment
- Fig.6 illustrates an example process executed by an equipment in charge of converting video data from a first dynamic range to a second dynamic range according to an embodiment
- Fig. 7 provide an example of process for introducing interoperability metadata in the SL-HDR1 metadata
- Fig. 8A illustrates schematically an example of hardware architecture of a processing module able to implement various aspects and embodiments
- Fig. 8B illustrates a block diagram of an example of a first system in which various aspects and embodiments are implemented
- Fig.8C illustrates a block diagram of an example of a second system in which various aspects and embodiments are implemented; 6.
- Diffuse White As mentioned earlier, the BT.2408-3 report proposed some recommendations and in particular a constraint of diffuse white.
- the diffuse white is defined in BT.2408- 3 report as “the white provided by a card that approximates to a perfect reflecting diffuser by being spectrally grey, not just calorimetrically grey, by minimizing specular highlights and minimizing spectral power absorptance”.
- a “perfect reflecting diffuser” 6 2023PF00065 is defined as an “ideal isotropic, nonfluorescent diffuser with a spectral radiance factor equal to unity at each wavelength of interest”.
- the diffuse white is a luminance level of a video signal that separates: ⁇ the scene with all the details, corresponding to the luminance levels that are below the diffuse white; ⁇ the speculars: very bright pixels, generally close to white and with very few details, corresponding to the luminance levels that are above the diffuse white level.
- Fig. 1A illustrates a scale of luminance values in which appears the diffuse white.
- the diffuse white separates the sets of all possible luminance values in two parts.
- the diffuse white notion is valid for HDR signals and for SDR signals.
- the BT.2408-3 report specifies that HDR Diffuse White is equal to “203” nits. However, the “203” nits constraint is only a recommendation and many content producers disagree with that recommendation.
- a HDR diffuse white equal to “203” nits brings a major issue: the HDR content is constrained, i.e., for a typical “1000” nits HDR content, only a small amount of the HDR luminance range [0-203 nits] is dedicated to the details of a scene, while the largest part of the HDR luminance range [203 – 1000 nits] is reserved for speculars that bring no detail.
- Fig.1B illustrates the separation of a scale of luminance values when the diffuse white is fixed to “203” nits.
- One of the reasons for this restriction is a need for controlled and “very safe” live HDR content production.
- this restriction has the following advantages: ⁇ the implementation of the conversion from HDR to SDR (i.e. the tone mapping (TM)) is simpler as the HDR diffuse white defined at “203” nits needs to be mapped to SDR diffuse white that is generally defined between 90% and 100% SDR (i.e. between 90 nits and 100 nits). The tone mapping can be therefore implemented using a very basic static 3D- LUTs. ⁇ the implementation of the conversion from SDR to HDR, (i.e. the inverse tone mapping (ITM)) is also simpler for the same reason and the 7 2023PF00065 inverse tone mapping can also be implemented with very basic static 3D-LUTs.
- Narrow Range and Narrow-Full Range As described in recommendations ITU-R BT.709-6 (Recommendation ITU-R BT.709-6, Parameter values for the HDTV standards for production and international program exchange, 06/2015) and ITU-R BT. 2100-2 (Recommendation ITU-R BT.2100-2, Image parameter values for high dynamic range television for use in production and international program exchange, 07/2018), typical video signals are transmitted using a YCbCr representation, also named YUV representation.
- quantization levels are defined from a lowest value to a highest value. For instance, when using a “10” bits quantization, luminance values are defined between the lowest value “64” (black) and the highest value “940” (white) and chrominance values are defined between the lowest value “64” and the highest value “960”.
- These allowed ranges are commonly named Legal Range or Narrow Range or Normal Range. In the latter, we will use the term Narrow Range (NR).
- NR Narrow Range
- recommendations ITU-R BT.709-6 and ITU-R BT.2100-2 also state that video data can take values from “4” to “1019”.
- the EBU R 103 (Recommendation EBU R103, Video signal tolerance in digital video systems version 3.0, 05/2020) recommendation gives some explanations on why some tolerance is needed when generating a YCbCr signal and recommends to not exceed the range [20-984]. However, some implementations do not follow the range [20-984] recommended by the recommendation EBU R103 and make use of the range [4-1019] or any range starting with a value in the range [4-63] and ending with a value in the range [941-1019].
- NFR Narrow Full Range
- the range [4-63] is called Infra blacks in the latter.
- the range [941-1019] is called Super Whites in the latter.
- the 3D-LUTs implementations are using these Infra Blacks and Super Whites ranges.
- the Super Whites range is typically used for carrying extra dynamic range that can be useful when doing SDR to HDR or HDR to SDR conversions.
- FIG. 1C illustrates an example context in which various embodiments are implemented.
- a live production system 20 is communicating with a master central control system 21.
- the live production system 20 provides simultaneously a HDR version and a SDR version of a same live content.
- the master central control system 21 then encodes the same or an enriched version of these SDR and HDR versions and provides these encoded versions to devices 22A and 22B.
- the master central control system 21 encodes the HDR and SDR versions using an AVC ((ISO/CEI 14496-10 / ITU-T H.264) encoder, an HEVC (ISO/IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265)) encoder, a VVC (ISO/IEC 23090-3 – MPEG-I, Versatile Video Coding/ ITU- T H.266) encoder or any other encoder.
- Devices 22A and 22B are display devices such as a PC, a TV, a smartphone, a tablet or a head mounted display ora device connected to a display device such as a set top box.
- the device 22A that have HDR capabilities receives an encoded HDR version.
- the device 22B that have only SDR capabilities receives an encoded SDR version.
- Fig. 2 illustrates a Single-master HDR/SDR workflow. Fig. 2 provides details on the live production system 20 and the master central control system 21.
- the live production system 20 comprises two sources: a HDR source 200 and a SDR source 201. Each source comprises at least one of a camera, a playback system or a system that generates graphics.
- the SDR source 201 is connected to a plurality of ITM tools: ⁇ one ITM tool 202A (ITM1) for SDR camera output up-conversion to HDR; 9 2023PF00065 ⁇ one ITM tool 202B (ITM2) for SDR playback content up-conversion to HDR; and, ⁇ one ITM tool 202C (ITM3) for SDR graphics (score insertion for instance) content up-conversion to HDR.
- a HDR content routing and switching system 203 ingests the multiple HDR inputs that come either from the HDR source 200 or from the SDR source 201 and then generates multiple HDR outputs.
- TM tools are used: ⁇ A TM tool 204B (TM1) for generating a predictive SDR output that is used by shaders operators that assess a quality of the generated SDR contents that are sent to the master central control system 21.
- TM2 ⁇ A TM tool 204A (TM2) for generating a SDR provided to the master central control system 21.
- the master central control system 21 comprises a HDR master control system 212 and a SDR master control system 213.
- the HDR and SDR master control systems are in charge of distributing SDR/HDR contents.
- the master central control system 21 comprises a source 210 generating adverts in SDR and an ITM tool 211 that convert the adverts from SDR to HDR.
- the HDR master control system 212 receives these adverts converted in HDR and mixes these adverts with the HDR content it receives.
- the HDR (respectively the SDR) master control system 212 (respectively 213) encode the HDR (respectively the SDR data) it receives or resulting from a mixture of the HDR (respectively SDR) data it receives with other data.
- the SDR and HDR data are encoded by an AVC encoder, an HEVC encoder, a VVC encoder or any other encoder.
- a HDR content may originate from the native HDR source 200.
- TM1 and/or TM tool 204A TM2
- characteristics of the native HDR source such as the HDR diffuse white value, the NFR value and at which SDR diffuse white mapping the HDR diffuse white.
- TM2 TM1
- TM2 TM2
- characteristics of the native HDR source such as the HDR diffuse white value, the NFR value and at which SDR diffuse white mapping the HDR diffuse white.
- TM tool 204A or 204B
- uses a specific static 3D-LUT the only solution for now is to make sure that the HDR source 200 is configured correctly to produce a video signal compatible with this specific 3D-LUT.
- All of these static LUTs are different, i.e.: ⁇ the HDR and SDR diffuse white values are different; ⁇ the maximum specular HDR values are different, due to different usage of the SDR maximum NFR and HDR maximum NFR values; ⁇ the NFR usage of these LUTs are different; ⁇ the HDR value mapping to the maximum SDR NR value is different; ⁇ the Tone Mapping and Inverse Tone Mapping curves are different; ⁇ the source and target transfer function may be different (HDR transfer function can be HLG or PQ).
- Table TAB1 represents values of SDR and HDR diffuse white, of SDR maximum NR, of HDR value from SDR maximum NR, of SDR maximum NFR and HDR value from SDR maximum NFR: 12 2023PF00065 Luminance (nits) HDR HDR SDR HDR SDR value from SDR value Diffuse Diffuse max SDR max max from SDR L UT name White White NR NR NFR max NFR NBCU LUT 1 100 203 100 203 124 252 NBCU LUT 2 100 203 100 203 124 252 ITM NBCU LUT 4 100 203 100 203 124 248 BBC LUT 3 100 203 100 203 124 260 BBC LUT 5 100 313 100 313 124 676 Luminance (nits) HDR value mapped SDR max HDR SDR to SDR HDR NFR from Diffuse Diffuse max SDR max max HDR max LUT name White White NR NR value value NBCU LUT 3 86 294
- a tone mapping tool use a dynamic solution, it can adapt to the content thanks to its dynamic nature.
- the conversion may not be optimal, as the tone mapping tool is lacking information such as the HDR and SDR Diffuse White values and NFR usage. Knowing this information would allow the dynamic tone mapping tools to produce a more optimal HDR to SDR conversion.
- the output of the HDR master control system 212 is a mixture of a main HDR video content with native SDR content. Such a situation commonly occurs in the case of diffusion of Live HDR contents. For example, during a diffusion of a content representing a live video event, logos, scores and adverts are mixed to a main HDR video representing the live sport event.
- the added contents can be SDR and need therefore to be converted in HDR before being mixed with the main HDR video content. Since the resulting mixed HDR content 13 2023PF00065 is likely to be converted in SDR, a new constraint appears: the SDR content resulting from the so-called SDR-HDR-SDR round trip conversion (i.e. ITM conversion followed by a TM conversion (for SDR delivery)) of these added contents must be identical to the original SDR content.
- SDR-HDR-SDR round trip constraint exist when a content producer generates a HDR content from an original SDR content but wishes that a SDR content generated, for any reason, from this HDR content to be identical to the original SDR content.
- the interoperability metadata comprise six information: ⁇ a HDR Diffuse White value HDR_DW: HDR_DW gives a HDR Diffuse White value for the HDR content.
- HDR pixels with luminance above HDR_DW are considered as speculars in the HDR content.
- a potential subsequenty SDR to HDR conversion it indicates on which HDR Diffuse White value a SDR Diffuse White (defined below) value should be mapped. It can be expressed in nits or any coding representation that relates to values in nits.
- SDR_DW gives a SDR Diffuse White value for a desired subsequent HDR to SDR (TM) conversion.
- SDR pixels with luminance above SDR_DW are considered as speculars in the SDR content resulting from the TM conversion.
- HDR_DW It indicates on which SDR Diffuse White value the HDR Diffuse White value HDR_DW should be mapped. It can be expressed in nits or any coding representation that relates to values in nits. If the HDR content is a native HDR content from a HDR source and if SDR_DW is not present in the 14 2023PF00065 metadata, then the TM tool is free to select the SDR Diffuse White value. If the HDR content originates from a SDR source that has been converted to HDR, SDR_DW can indicate either: o the SDR Diffuse White value of the SDR content that has been mapped on the HDR Diffuse White value HDR_DW during the ITM conversion.
- This SDR Diffuse White value SDR_DW may be different from the source SDR Diffuse White value, as it is, for instance, in the NBCU LUTs (source SDR Diffuse White is “100” nits and output SDR Diffuse White is “86” nits).
- HDR_NR gives the HDR luminance value that needs to be mapped on the Narrow Range maximum luminance value (i.e., “940” in “10” bits codeword) of the SDR content during a subsequent TM conversion.
- SDR_NFR_MAX gives the highest luminance allowed value used in the upper part of the SDR Narrow Full Range, corresponding to the HDR_NFR value of the corresponding HDR content.
- the value can be expressed by either a “8” bits, “10” bits or “12” bits value, or by a percentage of the narrow range. For instance, this value equals “1019” in 10 bits (or 109% in percentage of the narrow range) for SDI / Total Video signal range, and “984” in “10” bits codeword (or 105% in percentage of the narrow range) per EBU R103 recommendation.
- NFR_MIN gives the lowest allowed value used in the lower part of the Narrow Full Range.
- the value can be expressed by either a “8” bits, “10” bits or “12” bits value, or by a percentage of the narrow range. For instance, this value equals “4” in “10” bits codeword (or -6.84% in percentage of the narrow range) for SDI / Total Video signal range, and “20” in “10” bits codeword (or -5% in percentage of the narrow range) per EBU R103 recommendation.
- the Interoperability metadata (and in particular the “6” information) allows defining characteristics that must be respected by TM and ITM curves (implemented in any static conversions in the form of 3D-LUTs for instance or implemented in any dynamic solutions) used by TM and ITM tools in a single-master HDR/SDR workflow.
- Fig. 3A illustrates an example of luminance ITM curve compliant with the characteristics provided by the interoperability metadata.
- ⁇ The NFR_MIN value (for instance equal to “4” for a “10” bits codeword) of the input SDR content is mapped to the same NFR_MIN value (i.e., “4” for a “10” bits codeword in this example) of the output HDR content.
- the Narrow Range minimum value (“64” for a “10” bits codeword) of the input SDR content is mapped to the same Narrow Range minimum value (“64” for a “10” bits codeword) of the output HDR content.
- the SDR_DW value of the input SDR content is mapped to the HDR_DW value of output HDR content.
- the Narrow Range maximum value (“940” for a “10” bits codeword) of the input SDR content is mapped on the HDR_NR value of output HDR content.
- the SDR_NFR_MAX value (for instance equal to “1019” for a “10” bits codeword) of the input SDR content is mapped to the HDR_NFR value of output HDR content.
- 16 2023PF00065 Fig. 3B illustrates an example of luminance TM curve compliant with the characteristics provided by the interoperability metadata.
- the NFR_MIN value (for instance equal to “4” for a “10” bits codeword) of the input HDR content is mapped to the same NFR_MIN value (i.e., “4” for a “10” bits codeword in this example) of the output SDR content.
- the Narrow Range minimum value (“64” for a “10” bits codeword) of the input HDR content is mapped to the same Narrow Range minimum value (“64” for a “10” bits codeword) of the output SDR content.
- the HDR_DW value of the input HDR content is mapped to the SDR_DW value of output SDR content.
- the HDR_NR value of the input HDR content is mapped to the Narrow Range maximum value (“940” for a “10” bits codeword) of the output SDR content.
- the HDR_NFR value of input HDR is mapped to the SDR_NFR_MAX value (for instance equal to “1019” for a “10” bits codeword) of the output SDR content.
- Any static 3D-LUT can be characterized using the interoperability metadata.
- table TAB2 we mapped the “6” information provided by the Interoperability metadata on the values of SDR and HDR diffuse white, of SDR maximum NR, of HDR value from SDR maximum NR, of SDR maximum NFR and HDR value from SDR maximum NFR characterizing the existing 3D-LUTs represented in table TAB1.
- any static converter can be characterized using the interoperability metadata.
- FIG.4 illustrates the Single-master HDR/SDR workflow of Fig. 2 in which the interoperability metadata (IMD) are used.
- IMD interoperability metadata
- each equipment generating HDR data transmits interoperability metadata along the HDR data.
- the equipment generating HDR data comprises the HDR source 200, the ITM tools 202A, 202B and 202C and 211.
- the interoperability metadata are provided to the TM tools 204A and 204B to guide the TM process applied to the HDR data to generate the SDR data.
- the interoperability metadata are distributed along the HDR data by the HDR master control system 212.
- the interoperability metadata are used to guide a TM process applied on the distributed HDR data.
- the interoperability metadata are generated by an equipment generating SDR data.
- Equipment generating SDR data is for example the SDR source 18 2023PF00065 201, the TM tools 204A and 204B.
- the interoperability metadata can be distributed along the SDR data by the SDR master control system 213.
- the interoperability metadata are used to guide an ITM process applied on the distributed SDR data.
- the interoperability metadata could allow characterizing either a TM process or a ITM process.
- interoperability metadata characterizing a TM process allows for instance a TM tool to identify directly which TM process (i.e., which TM curve or 3D LUT defining a TM curve) applying.
- interoperability metadata characterizing a ITM process allows for instance a TM tool to identify characteristics of the HDR data or which ITM process (i.e., which ITM curve or 3D LUT defining a ITM curve) was applied to obtain the HDR data.
- the TM tool can determine a TM process compatible with the characteristics of the HDR data or with the ITM process applied to obtain the HDR data. In some embodiments, the determined TM process could ensure the SDR-HDR-SDR round trip constraint.
- interoperability metadata characterizing a ITM process allows for instance a ITM tool to identify directly which ITM process (i.e., which ITM curve or 3D LUT defining a ITM curve) applying.
- interoperability metadata characterizing a TM process allows for instance a ITM tool to identify characteristics of the SDR data or which TM process (i.e., which TM curve or 3D LUT defining a TM curve) was applied to obtain the SDR data.
- the ITM tool can determine an ITM process compatible with the characteristics of the SDR data or with the TM process applied to obtain the SDR data. Again, the determined TM process could ensure the SDR-HDR-SDR round trip constraint.
- Fig. 5 illustrates an example of process executed by a video source according to an embodiment. The process of Fig. 5 is executed by a processing module of the video source.
- the video source is for example, the HDR source 200, the SDR source 201, the ITM tool 202A, 202B, 202C or 211 or the TM tool 204A or 204B.
- the processing module obtains input video data in an input dynamic range.
- the input dynamic range could be HDR or SDR. 19 2023PF00065
- the processing module obtain interoperability metadata.
- the interoperability metadata could characterize either the input video data in the input dynamic range or output video data to be obtained from the input video data after conversion from the input dynamic range to an output dynamic range.
- the output dynamic range is SDR when the input dynamic range is HDR. Otherwise, the output dynamic range is HDR when the input dynamic range is SDR.
- the processing module transmits the input video data and the interoperability metadata.
- the input video data and the interoperability metadata are for example transmitted from the HDR source 200 to the TM tool 204A and 204B.
- the input video data and the interoperability metadata are transmitted from the HDR source 200 to a client equipment via HDR master control system 212.
- Fig. 6 illustrates an example process executed by an equipment in charge of converting video data from a first dynamic range to a second dynamic range according to an embodiment.
- the process of Fig. 6 is executed by a processing module of the equipment in charge of converting video data from a first dynamic range to a second dynamic range.
- the equipment is for example, the TM tool 204A or 204B.
- the processing module obtains input video data in an input dynamic range and interoperability metadata.
- the input dynamic range could be HDR or SDR.
- the input video data and the interoperability metadata are received by the TM tool 204A or 204B from the HDR source 200 or from the ITM tool 202A or 202B or 202C.
- the input video data are received by an equipment from the HDR source 200 or from the ITM tool 202A or 202B or 202C via the HDR control system 212 or via the SDR master control system 213.
- the processing module converts the input video data in output video data in a second dynamic range using the interoperability metadata.
- the output dynamic range is SDR when the input dynamic range is HDR and, the output dynamic range is HDR when the input dynamic range is SDR.
- Adapted solutions are needed to transport HDR or SDR data along with interoperability metadata either between modules of a production system (such as for example between the HDR source 200 or from the ITM tool 202A or 202B or 202C and the TM tools 204A and 204B of the live production system 20) or for distributing 20 2023PF00065 the HDR or SDR data and the interoperability metadata to external equipment after their production (for example via the Master central control system 21).
- SDI Serial Digital Interface
- Ancillary channels of a SDI interface allow exchanging additional information such as metadata.
- metadata can be exchanged using a vertical ancillary channels (SDI VANC in the following) as described in the standard SMPTE ST 2108-1 (SMPTE ST 2108-1:2018, HDR/WCG Metadata Packing and Signaling in the Vertical Ancillary Data Space, 10/2018).
- SDI VANC vertical ancillary channels
- the interoperability metadata could also be exchanged using the same SDI based mechanism.
- a first solution for transporting HDR or SDR data along with interoperability metadata is based on SL-HDR1.
- SL-HDR1 ETSI TS 103 433-1 v1.4.1, High- Performance Single Layer High Dynamic Range (HDR) System for use in Consumer Electronics devices; Part 1: Directly Standard Dynamic Range (SDR) Compatible HDR System (SL-HDR1), 08/2021) is a standard allowing representing an HDR content by an SDR signal with dynamic metadata.
- the SDR signal results from a tone mapping (TM) of the HDR content applying a tone mapping tool based on a tone mapping curve.
- TM tone mapping
- the dynamic metadata represent an inverse tone mapping curve allowing back converting the SDR signal to a HDR signal.
- SL-HDR1 As described in sections A.2.2.2 and A.2.2.4 of SL-HDR1, if a flag sl_hdr_extension_present_flag is set to “1”, then any information can be embedded in SL-HDR1 metadata in the form of SL-HDR extension data bytes sl_hdr_extension_data_byte. In an embodiment, this option of SL-HDR1 is used to transport the interoperability metadata.
- Fig. 7 provide an example of process for introducing the Interoperability metadata in the SL-HDR1 metadata. This process is implemented by a processing module of a HDR source (such as the HDR source 200) or by an ITM tool (such as the ITM tools 202A, 202B, 202C or 211).
- a step 701 the processing module set the flag sl_hdr_extension_present_flag to one and a syntax element sl_hdr_extension_length to “12”.
- the syntax element sl_hdr_extension_length represents the length in bytes of the SL-HDR extension data bytes.
- the processing module signals the value in nits of the SDR diffuse white SDR_DW on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[0] and sl_hdr_extension_data_byte[1], sl_hdr_extension_data_byte[0] being the least significant byte and sl_hdr_extension_data_byte[1] being the most significant byte of a 16 bits integer value.
- the processing module signals the value in nits of the HDR diffuse white HDR_DW on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[2] and sl_hdr_extension_data_byte[3], sl_hdr_extension_data_byte[2] being the least significant byte and sl_hdr_extension_data_byte[3] being the most significant byte of a “16” bits integer value .
- the processing module signals the HDR Narrow Range value (in nits) HDR_NR on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[4] and sl_hdr_extension_data_byte[5], sl_hdr_extension_data_byte[4] being the least significant byte and sl_hdr_extension_data_byte[5] being the most significant byte of a “16” bits integer value.
- the processing module signals the HDR Narrow Full range value (in nits) HDR_NFR on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[6] and sl_hdr_extension_data_byte[7], sl_hdr_extension_data_byte[6] being the least significant byte and sl_hdr_extension_data_byte[7] being the most significant byte of a “16” bits integer value.
- the processing module signals the Narrow Full Range minimum value NFR_MIN on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[8] and sl_hdr_extension_data_byte[9], sl_hdr_extension_data_byte[8] being the least significant byte and sl_hdr_extension_data_byte[9] being the most significant byte of a “16” bits integer value.
- the signaled value of NFR_MIN can be defined by convention to be representative of a “10” bits value for instance.
- the processing module signals the SDR Narrow Full Range maximum value SDR_NFR_MAX on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[10] and sl_hdr_extension_data_byte[11], sl_hdr_extension_data_byte[10] being the least significant byte and sl_hdr_extension_data_byte[11] being the most significant byte of a “16” bits integer value.
- the signaled value of SDR_NFR_MAX can be defined by convention to be representative of a 10 bits value for instance. For systems that are managing signals with different bit depth such as 8bits or 12bits content, this 10bits codeword and can therefore be converted for 8bits codeword, 12bits codeword, ... If some metadata are already present in the SL-HDR extension data bytes, those 6 new metadata can also be added before or after these already existing metadata. Any other representation of the “6” information in the SL-HDR1 metadata is also possible for instance, with less or more bytes for each information, not byte aligned, etc. Another solution for transporting the video data along with the interoperability metadata consist in transporting the metadata in SEI messages.
- Video compression standards such as VVC, HEVC and AVC defines messages, called supplemental enhancement information (SEI) messages, comprising metadata providing information relative to a video stream.
- SEI message is a data container or a syntax structure associated with the video stream.
- SEI messages offer a solution for transporting the interoperability metadata.
- a new SEI message dedicated to the transport of the Interoperability metadata could be defined.
- SEI messages could be transported in the form of SDI VANC messages as defined in SMPTE ST 2108-1.
- SEI messages could alternatively be inserted in a transport stream along with an encoded video stream distributed by the master central control system.
- the method used by the conversion module is not restricted to the methods proposed methods. 25 2023PF00065 3rd scenario: the interoperability metadata are used to mimic existing single- master HDR/SDR workflow.
- some existing single-master HDR/SDR workflow for example, the one proposed by NBCU (see NBCUniversal Single-Master Broadcast Production and Distribution Recommendations #NBCU-Rec-UHD-HDR-01.13, September 23nd, 2022, https://github.com/digitaltvguy/NBCUniversal-UHD-HDR-SDR-Single-Master-Master-Workflow-Recommendation-LUTs)
- the ITM tools 202A, 202B and 202C use the ITM 3D-LUT NBCU LUT 1 (see table TAB2) and the TM tools 204A and 204B uses the counterpart TM 3D-LUTs NBCU LUT 3.
- the ITM tool fills the interoperability metadata with SDR_DW, HDR_DW , NFR_MIN , SDR_NFR_MAX , HDR_NR and HDR_NFR values corresponding to the counterpart TM 3D-LUT NBCU LUT 3.
- the TM tools 204A and 204B select the 3D-LUT specified by the interoperability metadata, i.e., select the TM 3D- LUT NBCU LUT 3.
- the use of the interoperability metadata makes the selection of the appropriate TM 3D-LUT automatic and reliable.
- 4th scenario the interoperability metadata are used to improve a SDR-HDR- SDR round trip in a single-master HDR/SDR workflow. While in the previous scenario, the SDR-HDR-SDR round trip is not ensured, one objective of the 4 th scenario is to show how the SDR-HDR-SDR round trip could be improved using the interoperability metadata.
- the ITM tool when generating a HDR content with the ITM 3D-LUT NBCU LUT 1, in the 4 th scenario, the ITM tool fills the interoperability metadata with SDR_DW, HDR_DW , NFR_MIN , SDR_NFR_MAX , HDR_NR and HDR_NFR values corresponding to the ITM 3D-LUT NBCU LUT 1.
- the equipment When the equipment receives a native HDR content along with the interoperability metadata, the equipment configures the dynamic solution with the “6” information, allowing the dynamic solution to deliver the proper TM conversion.
- the equipment receives a HDR content resulting from ITM conversion along with the interoperability metadata that describes the characteristics of the ITM conversion, the equipment configures the dynamic system with the “6” information, allowing the dynamic system to deliver the proper TM conversion, i.e. it is able to respect the SDR-HDR-SDR round-trip constraint.
- FIG. 8A illustrates schematically an example of hardware architecture of a processing module 80 comprised in the live production system 20, in a system or module comprised in the live production system 20 such as the ITM tools 202A, 202B and 202C or the TM tools 204B and 204A, in the master central control system 21 or in a system or module of the master central control system 21 such as the ITM tool 211, or in the devices 22A and 22B.
- the processing module 80 comprises, connected by a communication bus 805: a processor or CPU (central processing unit) 800 encompassing one or more microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting 27 2023PF00065 examples; a random access memory (RAM) 801; a read only memory (ROM) 802; a storage unit 803, which can include non-volatile memory and/or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and/or optical disk drive, or a storage medium reader, such as a SD (secure digital) card reader and/or a hard disc drive (HDD) and/or a network accessible storage device; at least one communication interface 804 for exchanging data with other modules, devices, systems or equipment.
- the communication interface 804 can include, but is not limited to, a transceiver configured to transmit and to receive data over a communication network 81.
- the communication interface 804 can include, but is not limited to, a modem or a network card.
- the communication interface 804 enables for instance the processing module 80 to HDR or SDR data along with interoperability metadata.
- the processor 80 is capable of executing instructions loaded into the RAM 801 from the ROM 802, from an external memory (not shown), from a storage medium, or from a communication network. When the processing module 80 is powered up, the processor 800 is capable of reading instructions from the RAM 801 and executing them.
- These instructions form a computer program causing, for example, the implementation by the processor 800 of the HDR source 200, of the ITM tools 202A, 202B and 202C and 211 of the method of Fig.5 and the implementation by the processor 800 of the TM tools 204A and 204B of the method of Fig.6.
- All or some of the algorithms and steps of said processes may be implemented in software form by the execution of a set of instructions by a programmable machine such as a DSP (digital signal processor) or a microcontroller, or be implemented in hardware form by a machine or a dedicated component such as a FPGA (field- programmable gate array) or an ASIC (application-specific integrated circuit).
- Fig.8C illustrates a block diagram of an example of system A that corresponds to device 22A or 22B in which various aspects and embodiments are implemented.
- System A can be embodied as a device including various components or modules and is configured to generate a SDR or HDR content adapted to be displayed on adapted display devices. Examples of such system include, but are not limited to, 28 2023PF00065 various electronic systems such as personal computers, laptop computers, smartphones, tablet, TV, or set top boxes.
- Components of system A, singly or in combination can be embodied in a single integrated circuit (IC), multiple ICs, and/or discrete components.
- the system A comprises one processing module 80 that implements a decoding of a SDR or HDR content.
- the system A is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports.
- the input to the processing module 80 can be provided through various input modules as indicated in block 60.
- Such input modules include, but are not limited to, (i) a radio frequency (RF) module that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input module (or a set of COMP input modules), (iii) a Universal Serial Bus (USB) input module, and/or (iv) a High Definition Multimedia Interface (HDMI) input module.
- RF radio frequency
- COMP component
- USB Universal Serial Bus
- HDMI High Definition Multimedia Interface
- the input modules of block 60 have associated respective input processing elements as known in the art.
- the RF module can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the down-converted and band- limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets.
- the RF module of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers.
- the RF portion can include a tuner that performs various of these functions, including, for example, down-converting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband.
- Various embodiments rearrange the order of the above- described (and other) elements, remove some of these elements, and/or add other elements performing similar or different functions.
- Adding elements can include inserting elements in between existing elements, such as, for example, inserting 29 2023PF00065 amplifiers and an analog-to-digital converter.
- the RF module includes an antenna.
- the USB and/or HDMI modules can include respective interface processors for connecting system A to other electronic devices across USB and/or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed-Solomon error correction, can be implemented, for example, within a separate input processing IC or within the processing module 80 as necessary. Similarly, aspects of USB or HDMI interface processing can be implemented within separate interface ICs or within the processing module 80 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to the processing module 80.
- Various elements of system A can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards.
- I2C Inter-IC
- the processing module 80 is interconnected to other elements of said system A by the bus 805.
- the communication interface 804 of the processing module 80 allows the system A to communicate on the communication network 81.
- the communication network 81 can be implemented, for example, within a wired and/or a wireless medium.
- Wi-Fi Wireless Fidelity
- IEEE 802.11 IEEE refers to the Institute of Electrical and Electronics Engineers
- the Wi- Fi signal of these embodiments is received over the communications network 81 and the communications interface 804 which are adapted for Wi-Fi communications.
- the communications network 81 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications.
- Still other embodiments provide streamed data to the system A using the RF connection of the input block 60.
- various embodiments provide data in a non- streaming manner, for example, when the system A is a smartphone or a tablet.
- various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network.
- the system A can provide an output signal to various output devices using the communication network 81 or the bus 805.
- the system A can provide a decoded SDR or HDR signal.
- the system A can provide an output signal to various output devices, including a display 64 (if for example the system A is a set top box provided a decoded SDR or HDR signal to a display device), speakers 65, and other peripheral devices 66.
- the display 64 of various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and/or a foldable display.
- OLED organic light-emitting diode
- the display 64 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or other devices.
- the display 64 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop).
- the display device 64 is SDR or HDR content compatible.
- the other peripheral devices 66 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and/or a lighting system.
- DVR digital versatile disc
- Various embodiments use one or more peripheral devices 66 that provide a function based on the output of the system A. For example, a disk player performs the function of playing the output of the system A.
- control signals are communicated between the system A and the display 64, speakers 65, or other peripheral devices 66 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communications protocols that enable device-to-device control with or without user intervention.
- the output devices can be communicatively coupled to system A via dedicated connections through respective interfaces 61, 62, and 63. Alternatively, the output devices can be connected to system A using the communication network 81 via the communication interface 804.
- the display 64 and speakers 65 can be integrated in a single unit with the other components of system A in an electronic device such as, for example, a television.
- the display interface 61 includes a display driver, such as, for example, a timing controller (T Con) chip.
- the display 64 and speakers 65 can alternatively be separate from one or more of the other components, for example, if the RF module of input 60 is part of a separate set-top box.
- the output signal can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.
- Fig. 8B illustrates a block diagram of an example of the system B adapted to implement the live production system 20 or a module or device of the live production system 20 or the master central control system 21, or a module or a device of the master central control system 21 in which various aspects and embodiments are implemented.
- System B can be embodied as a device including the various components and modules described above and is configured to perform one or more of the aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, a camera, a smartphone and a server. Elements or modules of system B, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and/or discrete components.
- the system B comprises one processing module 80 that implement either an ITM tool (202A, 202B, 202C, 211) or a TM tool (204A, 204B).
- the system B is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports.
- the input to the processing module 80 can be provided through various input modules as indicated in block 60 already described in relation to Fig.8C.
- Various elements of system B can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards.
- I2C Inter-IC
- the processing module 80 is interconnected to other elements of said system B by the bus 805.
- the communication interface 804 of the processing module 80 allows the system B to communicate on the communication network 81.
- the communication network 81 can be implemented, for example, within a wired and/or a wireless medium. Data is streamed, or otherwise provided, to the system B, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers).
- IEEE 802.11 IEEE refers to the Institute of Electrical and Electronics Engineers.
- the Wi- 32 2023PF00065 Fi signal of these embodiments is received over the communications network 81 and the communications interface 804 which are adapted for Wi-Fi communications.
- the communications network 81 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications. Still other embodiments provide streamed data to the system B using the RF connection of the input block 60.
- various embodiments provide data in a non- streaming manner.
- a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of a corresponding apparatus.
- a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of a corresponding method/process.
- the implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program).
- An apparatus can be implemented in, for example, appropriate hardware, software, and firmware.
- processors refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device.
- Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs"), smartphones, tablets, and other devices that facilitate communication of information between end-users.
- PDAs portable/personal digital assistants
- this application may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, 33 2023PF00065 estimating the information, calculating the information, predicting the information, retrieving the information from memory or obtaining the information for example from another device, module or from user. Further, this application may refer to “accessing” various pieces of information.
- Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information. Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory).
- “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information. It is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of”, “one or more of” for example, in the cases of “A and/or B” and “at least one of A and B”, “one or more of A and B” is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B).
- implementations or embodiments can produce a variety of signals formatted to carry information that can 34 2023PF00065 be, for example, stored or transmitted.
- the information can include, for example, instructions for performing a method, or data produced by one of the described implementations or embodiments.
- a signal can be formatted to carry a HDR or SDR image or video sequence and interoperability metadata of a described embodiment.
- Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal.
- the formatting can include, for example, encoding a HDR or SDR image or video sequence with interoperability metadata in an encoded stream and modulating a carrier with the encoded stream.
- the information that the signal carries can be, for example, analog or digital information.
- the signal can be transmitted over a variety of different wired or wireless links, as is known.
- the signal can be stored on a processor-readable medium.
- a TV, set-top box, cell phone, tablet, or other electronic device that receives (e.g. using an antenna) a signal over the air that includes SDR or HDR data 35 2023PF00065 and interoperability metadata, and performs at least one of the embodiments described.
- a server, camera, cell phone, tablet, personal computer or other electronic device that tunes (e.g. using a tuner) a channel to transmit a signal including SDR or HDR data and interoperability metadata, and performs at least one of the embodiments described.
- a server camera, cell phone, tablet, personal computer or other electronic device that transmits (e.g. using an antenna) a signal over the air that includes SDR or HDR data and interoperability metadata, and performs at least one of the embodiments described.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
A method comprising: obtaining (601) input video data in an input dynamic range and interoperability metadata; converting (602) the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value.
Description
2023PF00065 GUIDED CONVERSIONS BETWEEN TWO DIFFERENT DYNAMIC RANGES WITH NEW METADATA 1. CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to European Application No.23305273.7, filed March 2, 2023, which is incorporated herein by reference in its entirety. 2. TECHNICAL FIELD At least one of the present embodiments generally relates to the field of production of High Dynamic Range (HDR) video and more particularly to a method, a device and an equipment implementing a signaling mechanism allowing identifying an appropriate HDR to SDR or SDR to HDR conversion in a single master production and distribution system. This signaling mechanism allows interoperability of different equipment and technologies in a real workflow. 3. BACKGROUND Recent advancements in display technologies allow for an extended dynamic range of color, luminance and contrast in images to be displayed. The term image refers here to an image content that can be for example a video or a still picture or image. High-dynamic-range video (HDR video) describes video having a dynamic range greater than that of standard-dynamic-range video (SDR video). HDR video involves capture, production, content/encoding, and display. HDR capture and display devices are capable of brighter whites and deeper blacks. To accommodate this, HDR encoding standards allow for a higher maximum luminance and use at least a 10-bit dynamic range (compared to 8-bit for non-professional and 10-bit for professional SDR video) in order to maintain precision across this extended range. HDR production is a new domain and there will be a transition phase during which both HDR contents and SDR contents will coexist. During this coexistence phase, a same live content will be produced simultaneously in a HDR and a SDR version. A user can then display the HDR or the SDR version of the content depending on his preferences or capabilities. The current trend of the content production industry is first, to produce HDR contents and then to derive automatically SDR contents from the HDR contents using 1
2023PF00065 automatic tools (this is known as single-master production and distribution workflow); and, second, to apply a controlled and safe approach for the HDR production to avoid delivering bad HDR or SDR contents to users. In that respect, some recommendations have been introduced by the ITU-R document “Report ITU-R BT.2408-3, Guidance for operational practices in HDR television production, 07/2019”, just called BT.2408-3 report in the following. One important recommendation introduced in the BT.2408-3 report is a constraint of HDR Diffuse White set to a fixed value equal to “203” nits. This constraint allows using fixed 3D-LUTs (Look-Up Tables) to implement SDR to HDR conversions (i.e. Inverse Tone Mapping (ITM)) and HDR to SDR conversions (Tone Mapping (TM)). Different static LUTs solutions are available on the market. These static LUTs have different characteristics (i.e., they use different diffuse white values, different maximum specular HDR values, different TM and ITM curves, different source and target transfer functions and they have different extended range usage). Using these static LUTs makes the system being static, i.e., the conversions (either ITM or TM) are fixed and not adapted to dynamic characteristics of a content. Dynamic conversions, based on a use a configurable and dynamic HDR Diffuse White level for both HDR to SDR (Tone Mapping) and SDR to HDR (Inverse Tone Mapping) conversions, offer solutions to deal with this issue. A real workflow is made with many different equipment from different providers. Nothing prevent the different providers to implement different solutions in their equipment, such as different static 3D-LUTs or dynamic solutions. However, equipment implementing different solutions are generally not interoperable. It is desirable to overcome the above drawbacks. It is particularly desirable to propose mechanisms that allow ensuring the selection of the appropriate conversions and that allow ensuring interoperability of different equipment and technologies in a real workflow. 4. BRIEF SUMMARY In a first aspect, one or more of the present embodiments provide a method comprising obtaining input video data in an input dynamic range; obtaining interoperability metadata; transmitting the input video data and the interoperability 2
2023PF00065 metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. In an embodiment, the input dynamic range is HDR or SDR. In an embodiment, the interoperability metadata are transmitted using ancillary channels of a SDI interface. In an embodiment, the interoperability metadata are embedded in SL-HDR1 metadata. In an embodiment, wherein the interoperability metadata are transmitted using a SEI message. In a second aspect, one or more of the present embodiments provide a method comprising: obtaining input video data in an input dynamic range and interoperability metadata; converting the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. In an embodiment, the input dynamic range is HDR or SDR. In an embodiment, the interoperability metadata are used to define a tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is HDR and the output dynamic range is SDR; and, the interoperability metadata are used to define an inverse tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is SDR and the output dynamic range is HDR. In an embodiment, the interoperability metadata are received using ancillary channels of a SDI interface. 3
2023PF00065 In an embodiment, the interoperability metadata are embedded in SL-HDR1 metadata. In an embodiment, the interoperability metadata are transmitted using a SEI message. In a third aspect, one or more of the present embodiments provide a device comprising electronic circuitry configured for: obtaining input video data in an input dynamic range; obtaining interoperability metadata; transmitting the input video data and the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. In an embodiment, the input dynamic range is HDR or SDR. In an embodiment, the interoperability metadata are transmitted using ancillary channels of a SDI interface. In an embodiment, the interoperability metadata are embedded in SL-HDR1 metadata. In an embodiment, the interoperability metadata are transmitted using a SEI message. In a fourth aspect, one or more of the present embodiments provide a device comprising electronic circuitry configured for: obtaining input video data in an input dynamic range and interoperability metadata; converting the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. In an embodiment, the input dynamic range is HDR or SDR. 4
2023PF00065 In an embodiment, the interoperability metadata are used to define a tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is HDR and the output dynamic range is SDR and the interoperability metadata are used to define an inverse tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is SDR and the output dynamic range is HDR. In an embodiment, the interoperability metadata are received using ancillary channels of a SDI interface. In an embodiment, the interoperability metadata are embedded in SL-HDR1 metadata. In an embodiment, the interoperability metadata are transmitted using a SEI message. In a fifth aspect, one or more of the present embodiments provide a non-transitory information storage medium storing program code instructions for implementing the method according to the first or the second aspect. In a sixth aspect, one or more of the present embodiments provide a signal comprising interoperability metadata comprising at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. In a seventh aspect, one or more of the present embodiments provide a computer program comprising program code instructions for implementing the method according to the first or the second aspect. 5. BRIEF SUMMARY OF THE DRAWINGS Fig.1A illustrates a scale of luminance values in which appears the diffuse white; Fig.1B illustrates the separation of a scale of luminance values when the diffuse white is fixed to “203” nits; 5
2023PF00065 Fig.1C illustrates schematically a context of various embodiments; Fig.2 illustrates a Single-master HDR/SDR workflow; Fig. 3A illustrates an example of luminance ITM curve compliant with characteristics provided by interoperability metadata; Fig. 3B illustrates an example of luminance TM curve compliant with characteristics provided by interoperability metadata; Fig.4 illustrates a Single-master HDR/SDR workflow according to an embodiment; Fig. 5 illustrates an example of process executed by a video source according to an embodiment; Fig.6 illustrates an example process executed by an equipment in charge of converting video data from a first dynamic range to a second dynamic range according to an embodiment; Fig. 7 provide an example of process for introducing interoperability metadata in the SL-HDR1 metadata; Fig. 8A illustrates schematically an example of hardware architecture of a processing module able to implement various aspects and embodiments; Fig. 8B illustrates a block diagram of an example of a first system in which various aspects and embodiments are implemented; and, Fig.8C illustrates a block diagram of an example of a second system in which various aspects and embodiments are implemented; 6. DETAILED DESCRIPTION In the following, we introduce several notions used in various embodiments. Diffuse White: As mentioned earlier, the BT.2408-3 report proposed some recommendations and in particular a constraint of diffuse white. The diffuse white is defined in BT.2408- 3 report as “the white provided by a card that approximates to a perfect reflecting diffuser by being spectrally grey, not just calorimetrically grey, by minimizing specular highlights and minimizing spectral power absorptance”. A “perfect reflecting diffuser” 6
2023PF00065 is defined as an “ideal isotropic, nonfluorescent diffuser with a spectral radiance factor equal to unity at each wavelength of interest”. In other words, the diffuse white is a luminance level of a video signal that separates: ^ the scene with all the details, corresponding to the luminance levels that are below the diffuse white; ^ the speculars: very bright pixels, generally close to white and with very few details, corresponding to the luminance levels that are above the diffuse white level. Fig. 1A illustrates a scale of luminance values in which appears the diffuse white. As can be seen, the diffuse white separates the sets of all possible luminance values in two parts. The diffuse white notion is valid for HDR signals and for SDR signals. The BT.2408-3 report specifies that HDR Diffuse White is equal to “203” nits. However, the “203” nits constraint is only a recommendation and many content producers disagree with that recommendation. Indeed, a HDR diffuse white equal to “203” nits brings a major issue: the HDR content is constrained, i.e., for a typical “1000” nits HDR content, only a small amount of the HDR luminance range [0-203 nits] is dedicated to the details of a scene, while the largest part of the HDR luminance range [203 – 1000 nits] is reserved for speculars that bring no detail. Fig.1B illustrates the separation of a scale of luminance values when the diffuse white is fixed to “203” nits. One of the reasons for this restriction is a need for controlled and “very safe” live HDR content production. In addition, this restriction has the following advantages: ^ the implementation of the conversion from HDR to SDR (i.e. the tone mapping (TM)) is simpler as the HDR diffuse white defined at “203” nits needs to be mapped to SDR diffuse white that is generally defined between 90% and 100% SDR (i.e. between 90 nits and 100 nits). The tone mapping can be therefore implemented using a very basic static 3D- LUTs. ^ the implementation of the conversion from SDR to HDR, (i.e. the inverse tone mapping (ITM)) is also simpler for the same reason and the 7
2023PF00065 inverse tone mapping can also be implemented with very basic static 3D-LUTs. However, such ratio between the luminance values allocated to the details of the scene and the luminance values allocated to the speculars induced by the diffuse white at “203” nits render the resulting HDR images very dull and not appealing. Narrow Range and Narrow-Full Range: As described in recommendations ITU-R BT.709-6 (Recommendation ITU-R BT.709-6, Parameter values for the HDTV standards for production and international program exchange, 06/2015) and ITU-R BT. 2100-2 (Recommendation ITU-R BT.2100-2, Image parameter values for high dynamic range television for use in production and international program exchange, 07/2018), typical video signals are transmitted using a YCbCr representation, also named YUV representation. In that representation, quantization levels are defined from a lowest value to a highest value. For instance, when using a “10” bits quantization, luminance values are defined between the lowest value “64” (black) and the highest value “940” (white) and chrominance values are defined between the lowest value “64” and the highest value “960”. These allowed ranges are commonly named Legal Range or Narrow Range or Normal Range. In the latter, we will use the term Narrow Range (NR). However, recommendations ITU-R BT.709-6 and ITU-R BT.2100-2 also state that video data can take values from “4” to “1019”. The EBU R 103 (Recommendation EBU R103, Video signal tolerance in digital video systems version 3.0, 05/2020) recommendation gives some explanations on why some tolerance is needed when generating a YCbCr signal and recommends to not exceed the range [20-984]. However, some implementations do not follow the range [20-984] recommended by the recommendation EBU R103 and make use of the range [4-1019] or any range starting with a value in the range [4-63] and ending with a value in the range [941-1019]. The combinations of the ranges [4-63] and [941-1019] (respectively [961- 1019]) with the NR range [64-940] (respectively [64-960]) is named Narrow Full Range (NFR) in the latter. 8
2023PF00065 The range [4-63] is called Infra blacks in the latter. The range [941-1019] (respectively [941-1019]) is called Super Whites in the latter. Typically, the 3D-LUTs implementations are using these Infra Blacks and Super Whites ranges. For example, the Super Whites range is typically used for carrying extra dynamic range that can be useful when doing SDR to HDR or HDR to SDR conversions. Fig. 1C illustrates an example context in which various embodiments are implemented. In Fig.1C, a live production system 20 is communicating with a master central control system 21. The live production system 20 provides simultaneously a HDR version and a SDR version of a same live content. The master central control system 21 then encodes the same or an enriched version of these SDR and HDR versions and provides these encoded versions to devices 22A and 22B. In an embodiment, the master central control system 21 encodes the HDR and SDR versions using an AVC ((ISO/CEI 14496-10 / ITU-T H.264) encoder, an HEVC (ISO/IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265)) encoder, a VVC (ISO/IEC 23090-3 – MPEG-I, Versatile Video Coding/ ITU- T H.266) encoder or any other encoder. Devices 22A and 22B are display devices such as a PC, a TV, a smartphone, a tablet or a head mounted display ora device connected to a display device such as a set top box. The device 22A that have HDR capabilities receives an encoded HDR version. The device 22B that have only SDR capabilities receives an encoded SDR version. Fig. 2 illustrates a Single-master HDR/SDR workflow. Fig. 2 provides details on the live production system 20 and the master central control system 21. The live production system 20 comprises two sources: a HDR source 200 and a SDR source 201. Each source comprises at least one of a camera, a playback system or a system that generates graphics. The SDR source 201 is connected to a plurality of ITM tools: ^ one ITM tool 202A (ITM1) for SDR camera output up-conversion to HDR; 9
2023PF00065 ^ one ITM tool 202B (ITM2) for SDR playback content up-conversion to HDR; and, ^ one ITM tool 202C (ITM3) for SDR graphics (score insertion for instance) content up-conversion to HDR. A HDR content routing and switching system 203 ingests the multiple HDR inputs that come either from the HDR source 200 or from the SDR source 201 and then generates multiple HDR outputs. From these HDR outputs, multiple TM tools are used: ^ A TM tool 204B (TM1) for generating a predictive SDR output that is used by shaders operators that assess a quality of the generated SDR contents that are sent to the master central control system 21. ^ A TM tool 204A (TM2) for generating a SDR provided to the master central control system 21. The master central control system 21 comprises a HDR master control system 212 and a SDR master control system 213. The HDR and SDR master control systems are in charge of distributing SDR/HDR contents. In the example of Fig. 2, the master central control system 21 comprises a source 210 generating adverts in SDR and an ITM tool 211 that convert the adverts from SDR to HDR. The HDR master control system 212 receives these adverts converted in HDR and mixes these adverts with the HDR content it receives. The HDR (respectively the SDR) master control system 212 (respectively 213) encode the HDR (respectively the SDR data) it receives or resulting from a mixture of the HDR (respectively SDR) data it receives with other data. For example, the SDR and HDR data are encoded by an AVC encoder, an HEVC encoder, a VVC encoder or any other encoder. As can be seen in Fig. 2, a HDR content may originate from the native HDR source 200. As the HDR content needs to be converted to SDR, currently there is no way for the TM tool 204B (TM1) and/or the TM tool 204A (TM2) to know characteristics of the native HDR source such as the HDR diffuse white value, the NFR value and at which SDR diffuse white mapping the HDR diffuse white. If a TM tool (204A or 204B) uses a specific static 3D-LUT, the only solution for now is to make sure that the HDR source 200 is configured correctly to produce a video signal compatible with this specific 3D-LUT. However, nothing prevents an HDR source provider to configure the HDR source 200 in a different way that is not 10
2023PF00065 compatible anymore with this specific 3D-LUT. Similarly, nothing prevents a TM tool configurator to choose a different static 3D-LUT that is not compatible with the HDR source 200. Indeed, different static 3D-LUTs solutions are available on the market. One can mention the following static 3D-LUTs solutions: ^ BBC LUTs (Release Notes for HLG Format Conversion LUTs v1.5, BBC Research, 05/2021, http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_HLG_ LUT_Release_Notes_v1-5.pdf): Current version 1.5 of the BBC LUTs solution describes “20” different 3D-LUTs for different usages/configurations; ^ NBCU LUTs (NBCUniversal Single-Master Broadcast Production and Distribution Recommendations #NBCU-Rec-UHD-HDR-01.13, September 23nd, 2022, https://github.com/digitaltvguy/NBCUniversal- UHD-HDR-SDR-Single-Master-Production-Workflow- Recommendation-LUTs): Current version 1.13 of the NBCU LUTs solution describes “5” different 3D-LUTs for different usages/configurations; ^ Recently during the FIFA World Cup in Qatar, HBS has designed its own HDR to SDR 3D-LUT, (see https://www.sportsvideo.org/2022/12/07/live-from-the-fifa-world-cup- hbs-ceo-dan-miodownik-cto-christian-gobbel-reflect-on-efforts-to- date/). All of these static LUTs are different, i.e.: ^ the HDR and SDR diffuse white values are different; ^ the maximum specular HDR values are different, due to different usage of the SDR maximum NFR and HDR maximum NFR values; ^ the NFR usage of these LUTs are different; ^ the HDR value mapping to the maximum SDR NR value is different; ^ the Tone Mapping and Inverse Tone Mapping curves are different; ^ the source and target transfer function may be different (HDR transfer function can be HLG or PQ). 11
2023PF00065 Table TAB1 represents values of SDR and HDR diffuse white, of SDR maximum NR, of HDR value from SDR maximum NR, of SDR maximum NFR and HDR value from SDR maximum NFR: 12
2023PF00065 Luminance (nits) HDR HDR SDR HDR SDR value from SDR value Diffuse Diffuse max SDR max max from SDR LUT name White White NR NR NFR max NFR NBCU LUT 1 100 203 100 203 124 252 NBCU LUT 2 100 203 100 203 124 252 ITM NBCU LUT 4 100 203 100 203 124 248 BBC LUT 3 100 203 100 203 124 260 BBC LUT 5 100 313 100 313 124 676 Luminance (nits) HDR value mapped SDR max HDR SDR to SDR HDR NFR from Diffuse Diffuse max SDR max max HDR max LUT name White White NR NR value value NBCU LUT 3 203 86 294 100 1810 1006 NBCU LUT 5 203 86 289 100 24030 1019 TM BBC LUT 8 203 78 1000 100 1810 959 Table TAB1 This variety of static 3D LUTs creates a risk of misconfiguration. If a tone mapping tool use a dynamic solution, it can adapt to the content thanks to its dynamic nature. However, the conversion may not be optimal, as the tone mapping tool is lacking information such as the HDR and SDR Diffuse White values and NFR usage. Knowing this information would allow the dynamic tone mapping tools to produce a more optimal HDR to SDR conversion. Another aspect of Fig. 2 is that the output of the HDR master control system 212 is a mixture of a main HDR video content with native SDR content. Such a situation commonly occurs in the case of diffusion of Live HDR contents. For example, during a diffusion of a content representing a live video event, logos, scores and adverts are mixed to a main HDR video representing the live sport event. As in the example of Fig. 2, the added contents can be SDR and need therefore to be converted in HDR before being mixed with the main HDR video content. Since the resulting mixed HDR content 13
2023PF00065 is likely to be converted in SDR, a new constraint appears: the SDR content resulting from the so-called SDR-HDR-SDR round trip conversion (i.e. ITM conversion followed by a TM conversion (for SDR delivery)) of these added contents must be identical to the original SDR content. The same SDR-HDR-SDR round trip constraint exist when a content producer generates a HDR content from an original SDR content but wishes that a SDR content generated, for any reason, from this HDR content to be identical to the original SDR content. In a context wherein the use static LUTs and/or dynamic solutions is not controlled, the risk of misconfiguration and therefore the risk of being unable to respect the SDR-HDR-SDR round trip constraint is high. There is therefore a need to find solutions allowing interoperability between equipment of a single-master HDR/SDR workflow. These solutions would therefore allow to respect the SDR-HDR-SDR round trip constraint. In the following, various embodiments based on a use of additional metadata, called interoperability metadata, are proposed. In an embodiment, the interoperability metadata comprise six information: ^ a HDR Diffuse White value HDR_DW: HDR_DW gives a HDR Diffuse White value for the HDR content. HDR pixels with luminance above HDR_DW are considered as speculars in the HDR content. In the case of a potential subsequenty SDR to HDR conversion, it indicates on which HDR Diffuse White value a SDR Diffuse White (defined below) value should be mapped. It can be expressed in nits or any coding representation that relates to values in nits. ^ a SDR Diffuse White value SDR_DW: SDR_DW gives a SDR Diffuse White value for a desired subsequent HDR to SDR (TM) conversion. SDR pixels with luminance above SDR_DW are considered as speculars in the SDR content resulting from the TM conversion. It indicates on which SDR Diffuse White value the HDR Diffuse White value HDR_DW should be mapped. It can be expressed in nits or any coding representation that relates to values in nits. If the HDR content is a native HDR content from a HDR source and if SDR_DW is not present in the 14
2023PF00065 metadata, then the TM tool is free to select the SDR Diffuse White value. If the HDR content originates from a SDR source that has been converted to HDR, SDR_DW can indicate either: o the SDR Diffuse White value of the SDR content that has been mapped on the HDR Diffuse White value HDR_DW during the ITM conversion. It allows a subsequent TM conversion to map the HDR Diffuse White value to the SDR Diffuse White value SDR_DW. This information allows therefore to meet the SDR- HDR-SDR round trip constraint, o the SDR Diffuse White value to which the HDR Diffuse White value HDR_DW should be mapped by a subsequent TM conversion. This SDR Diffuse White value SDR_DW may be different from the source SDR Diffuse White value, as it is, for instance, in the NBCU LUTs (source SDR Diffuse White is “100” nits and output SDR Diffuse White is “86” nits). ^ a HDR Narrow Range (NR) value HDR_NR: HDR_NR gives the HDR luminance value that needs to be mapped on the Narrow Range maximum luminance value (i.e., “940” in “10” bits codeword) of the SDR content during a subsequent TM conversion. ^ a HDR Narrow Full Range value HDR_NFR: HDR_NFR gives the HDR maximum luminance value that needs to be mapped to the narrow full range maximum luminance value SDR_NFR_MAX of the SDR content during a subsequent TM conversion. ^ a SDR Narrow Full Range maximum value SDR_NFR_MAX: SDR_NFR_MAX gives the highest luminance allowed value used in the upper part of the SDR Narrow Full Range, corresponding to the HDR_NFR value of the corresponding HDR content. The value can be expressed by either a “8” bits, “10” bits or “12” bits value, or by a percentage of the narrow range. For instance, this value equals “1019” in 10 bits (or 109% in percentage of the narrow range) for SDI / Total Video signal range, and “984” in “10” bits codeword (or 105% in percentage of the narrow range) per EBU R103 recommendation. 15
2023PF00065 SDR_NFR_MAX cannot be lower than the maximum Narrow Range value, i.e., “940” in 10 bits (or 100%). ^ a Narrow Full Range minimum value NFR_MIN: NFR_MIN gives the lowest allowed value used in the lower part of the Narrow Full Range. The value can be expressed by either a “8” bits, “10” bits or “12” bits value, or by a percentage of the narrow range. For instance, this value equals “4” in “10” bits codeword (or -6.84% in percentage of the narrow range) for SDI / Total Video signal range, and “20” in “10” bits codeword (or -5% in percentage of the narrow range) per EBU R103 recommendation. This value cannot be higher than the minimum Narrow Range value, i.e., “64” in “10” bits (or 0%). The Interoperability metadata (and in particular the “6” information) allows defining characteristics that must be respected by TM and ITM curves (implemented in any static conversions in the form of 3D-LUTs for instance or implemented in any dynamic solutions) used by TM and ITM tools in a single-master HDR/SDR workflow. Fig. 3A illustrates an example of luminance ITM curve compliant with the characteristics provided by the interoperability metadata. In the example of Fig.3A: ^ The NFR_MIN value (for instance equal to “4” for a “10” bits codeword) of the input SDR content is mapped to the same NFR_MIN value (i.e., “4” for a “10” bits codeword in this example) of the output HDR content. ^ The Narrow Range minimum value (“64” for a “10” bits codeword) of the input SDR content is mapped to the same Narrow Range minimum value (“64” for a “10” bits codeword) of the output HDR content. ^ The SDR_DW value of the input SDR content is mapped to the HDR_DW value of output HDR content. ^ The Narrow Range maximum value (“940” for a “10” bits codeword) of the input SDR content is mapped on the HDR_NR value of output HDR content. ^ The SDR_NFR_MAX value (for instance equal to “1019” for a “10” bits codeword) of the input SDR content is mapped to the HDR_NFR value of output HDR content. 16
2023PF00065 Fig. 3B illustrates an example of luminance TM curve compliant with the characteristics provided by the interoperability metadata. In the example of Fig.3B: ^ The NFR_MIN value (for instance equal to “4” for a “10” bits codeword) of the input HDR content is mapped to the same NFR_MIN value (i.e., “4” for a “10” bits codeword in this example) of the output SDR content. ^ The Narrow Range minimum value (“64” for a “10” bits codeword) of the input HDR content is mapped to the same Narrow Range minimum value (“64” for a “10” bits codeword) of the output SDR content. ^ The HDR_DW value of the input HDR content is mapped to the SDR_DW value of output SDR content. ^ The HDR_NR value of the input HDR content is mapped to the Narrow Range maximum value (“940” for a “10” bits codeword) of the output SDR content. ^ The HDR_NFR value of input HDR is mapped to the SDR_NFR_MAX value (for instance equal to “1019” for a “10” bits codeword) of the output SDR content. Any static 3D-LUT can be characterized using the interoperability metadata. As an example, in table TAB2, we mapped the “6” information provided by the Interoperability metadata on the values of SDR and HDR diffuse white, of SDR maximum NR, of HDR value from SDR maximum NR, of SDR maximum NFR and HDR value from SDR maximum NFR characterizing the existing 3D-LUTs represented in table TAB1. 17
2023PF00065 Luminance (nits) Code Word (10 bits) LUT name SDR_DW HDR_DW HDR_NR HDR_NFR NFR_MIN SDR_NFR_MAX ITM NBCU LUT 1 100 203 203 252 4 1019 NBCU LUT 2 100 203 203 252 4 1019 NBCU LUT 4 100 203 203 248 4 1019 BBC LUT 3 100 203 203 260 4 1019 BBC LUT 5 100 313 313 676 4 1019 Luminance (nits) Code Word LUT name SDR_DW HDR_DW HDR_NR HDR_NFR NFR_MIN SDR_NFR_MAX TM NBCU LUT 3 86 203 294 1810 4 1006 NBCU LUT 5 86 203 289 24030 4 1019 BBC LUT 8 78 203 1000 1810 4 959 Table TAB2 More generally, any static converter can be characterized using the interoperability metadata. Similarly any dynamic converter that is able to manage these interoperability metadata can produce dynamic conversion curves that are compatible with the interoperability metadata. Fig.4 illustrates the Single-master HDR/SDR workflow of Fig. 2 in which the interoperability metadata (IMD) are used. As can be seen in Fig. 4, each equipment generating HDR data, transmits interoperability metadata along the HDR data. The equipment generating HDR data comprises the HDR source 200, the ITM tools 202A, 202B and 202C and 211. In an embodiment, the interoperability metadata are provided to the TM tools 204A and 204B to guide the TM process applied to the HDR data to generate the SDR data. In an embodiment, the interoperability metadata are distributed along the HDR data by the HDR master control system 212. In this embodiment, the interoperability metadata are used to guide a TM process applied on the distributed HDR data. In an embodiment, it is also possible to associate interoperability metadata to SDR data. In that case the interoperability metadata are generated by an equipment generating SDR data. Equipment generating SDR data is for example the SDR source 18
2023PF00065 201, the TM tools 204A and 204B. In the example of Fig.4, when associated to SDR data, the interoperability metadata can be distributed along the SDR data by the SDR master control system 213. In this case, the interoperability metadata are used to guide an ITM process applied on the distributed SDR data. As can be seen in the example of table TAB2, the interoperability metadata could allow characterizing either a TM process or a ITM process. When transmitted along HDR data, interoperability metadata characterizing a TM process allows for instance a TM tool to identify directly which TM process (i.e., which TM curve or 3D LUT defining a TM curve) applying. When transmitted along HDR data, interoperability metadata characterizing a ITM process allows for instance a TM tool to identify characteristics of the HDR data or which ITM process (i.e., which ITM curve or 3D LUT defining a ITM curve) was applied to obtain the HDR data. In that case, from the interoperability metadata, the TM tool can determine a TM process compatible with the characteristics of the HDR data or with the ITM process applied to obtain the HDR data. In some embodiments, the determined TM process could ensure the SDR-HDR-SDR round trip constraint. When transmitted along SDR data, interoperability metadata characterizing a ITM process allows for instance a ITM tool to identify directly which ITM process (i.e., which ITM curve or 3D LUT defining a ITM curve) applying. When transmitted along SDR data, interoperability metadata characterizing a TM process allows for instance a ITM tool to identify characteristics of the SDR data or which TM process (i.e., which TM curve or 3D LUT defining a TM curve) was applied to obtain the SDR data. In that case, from the interoperability metadata, the ITM tool can determine an ITM process compatible with the characteristics of the SDR data or with the TM process applied to obtain the SDR data. Again, the determined TM process could ensure the SDR-HDR-SDR round trip constraint. Fig. 5 illustrates an example of process executed by a video source according to an embodiment. The process of Fig. 5 is executed by a processing module of the video source. The video source is for example, the HDR source 200, the SDR source 201, the ITM tool 202A, 202B, 202C or 211 or the TM tool 204A or 204B. In a step 501, the processing module obtains input video data in an input dynamic range. The input dynamic range could be HDR or SDR. 19
2023PF00065 In a step 502, the processing module obtain interoperability metadata. The interoperability metadata could characterize either the input video data in the input dynamic range or output video data to be obtained from the input video data after conversion from the input dynamic range to an output dynamic range. The output dynamic range is SDR when the input dynamic range is HDR. Otherwise, the output dynamic range is HDR when the input dynamic range is SDR. In a step 503, the processing module transmits the input video data and the interoperability metadata. The input video data and the interoperability metadata are for example transmitted from the HDR source 200 to the TM tool 204A and 204B. In another example, the input video data and the interoperability metadata are transmitted from the HDR source 200 to a client equipment via HDR master control system 212. Fig. 6 illustrates an example process executed by an equipment in charge of converting video data from a first dynamic range to a second dynamic range according to an embodiment. The process of Fig. 6 is executed by a processing module of the equipment in charge of converting video data from a first dynamic range to a second dynamic range. The equipment is for example, the TM tool 204A or 204B. In a step 601, the processing module obtains input video data in an input dynamic range and interoperability metadata. Again, the input dynamic range could be HDR or SDR. In an example, the input video data and the interoperability metadata are received by the TM tool 204A or 204B from the HDR source 200 or from the ITM tool 202A or 202B or 202C. In another example, the input video data are received by an equipment from the HDR source 200 or from the ITM tool 202A or 202B or 202C via the HDR control system 212 or via the SDR master control system 213. In a step 602, the processing module converts the input video data in output video data in a second dynamic range using the interoperability metadata. Again, the output dynamic range is SDR when the input dynamic range is HDR and, the output dynamic range is HDR when the input dynamic range is SDR. Adapted solutions are needed to transport HDR or SDR data along with interoperability metadata either between modules of a production system (such as for example between the HDR source 200 or from the ITM tool 202A or 202B or 202C and the TM tools 204A and 204B of the live production system 20) or for distributing 20
2023PF00065 the HDR or SDR data and the interoperability metadata to external equipment after their production (for example via the Master central control system 21). Typical production systems (such as the live production system 20) use SDI (Serial Digital Interface) interconnections following the suite of SMPTE standards - for exchanging data inside the system. Ancillary channels of a SDI interface allow exchanging additional information such as metadata. For instance, metadata can be exchanged using a vertical ancillary channels (SDI VANC in the following) as described in the standard SMPTE ST 2108-1 (SMPTE ST 2108-1:2018, HDR/WCG Metadata Packing and Signaling in the Vertical Ancillary Data Space, 10/2018). The interoperability metadata could also be exchanged using the same SDI based mechanism. A first solution for transporting HDR or SDR data along with interoperability metadata is based on SL-HDR1. SL-HDR1 (ETSI TS 103 433-1 v1.4.1, High- Performance Single Layer High Dynamic Range (HDR) System for use in Consumer Electronics devices; Part 1: Directly Standard Dynamic Range (SDR) Compatible HDR System (SL-HDR1), 08/2021) is a standard allowing representing an HDR content by an SDR signal with dynamic metadata. The SDR signal results from a tone mapping (TM) of the HDR content applying a tone mapping tool based on a tone mapping curve. The dynamic metadata represent an inverse tone mapping curve allowing back converting the SDR signal to a HDR signal. As described in sections A.2.2.2 and A.2.2.4 of SL-HDR1, if a flag sl_hdr_extension_present_flag is set to “1”, then any information can be embedded in SL-HDR1 metadata in the form of SL-HDR extension data bytes sl_hdr_extension_data_byte. In an embodiment, this option of SL-HDR1 is used to transport the interoperability metadata. Fig. 7 provide an example of process for introducing the Interoperability metadata in the SL-HDR1 metadata. This process is implemented by a processing module of a HDR source (such as the HDR source 200) or by an ITM tool (such as the ITM tools 202A, 202B, 202C or 211). This process is for example executed in step 503. In a step 701, the processing module set the flag sl_hdr_extension_present_flag to one and a syntax element sl_hdr_extension_length to “12”. The syntax element sl_hdr_extension_length represents the length in bytes of the SL-HDR extension data bytes. 21
2023PF00065 In a step 702, the processing module signals the value in nits of the SDR diffuse white SDR_DW on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[0] and sl_hdr_extension_data_byte[1], sl_hdr_extension_data_byte[0] being the least significant byte and sl_hdr_extension_data_byte[1] being the most significant byte of a 16 bits integer value. In a step 703, the processing module signals the value in nits of the HDR diffuse white HDR_DW on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[2] and sl_hdr_extension_data_byte[3], sl_hdr_extension_data_byte[2] being the least significant byte and sl_hdr_extension_data_byte[3] being the most significant byte of a “16” bits integer value . In a step 704, the processing module signals the HDR Narrow Range value (in nits) HDR_NR on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[4] and sl_hdr_extension_data_byte[5], sl_hdr_extension_data_byte[4] being the least significant byte and sl_hdr_extension_data_byte[5] being the most significant byte of a “16” bits integer value. In a step 705, the processing module signals the HDR Narrow Full range value (in nits) HDR_NFR on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[6] and sl_hdr_extension_data_byte[7], sl_hdr_extension_data_byte[6] being the least significant byte and sl_hdr_extension_data_byte[7] being the most significant byte of a “16” bits integer value. In a step 706, the processing module signals the Narrow Full Range minimum value NFR_MIN on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[8] and sl_hdr_extension_data_byte[9], sl_hdr_extension_data_byte[8] being the least significant byte and sl_hdr_extension_data_byte[9] being the most significant byte of a “16” bits integer value. The signaled value of NFR_MIN can be defined by convention to be representative of a “10” bits value for instance. For systems that are managing signals with different bit depth such as 8bits or 12bits content, this 10bits codeword can therefore easily be converted for 8bits codeword, 12bits codeword, … 22
2023PF00065 In a step 707, the processing module signals the SDR Narrow Full Range maximum value SDR_NFR_MAX on two bytes of the SL-HDR extension data bytes sl_hdr_extension_data_byte[10] and sl_hdr_extension_data_byte[11], sl_hdr_extension_data_byte[10] being the least significant byte and sl_hdr_extension_data_byte[11] being the most significant byte of a “16” bits integer value. The signaled value of SDR_NFR_MAX can be defined by convention to be representative of a 10 bits value for instance. For systems that are managing signals with different bit depth such as 8bits or 12bits content, this 10bits codeword and can therefore be converted for 8bits codeword, 12bits codeword, … If some metadata are already present in the SL-HDR extension data bytes, those 6 new metadata can also be added before or after these already existing metadata. Any other representation of the “6” information in the SL-HDR1 metadata is also possible for instance, with less or more bytes for each information, not byte aligned, etc. Another solution for transporting the video data along with the interoperability metadata consist in transporting the metadata in SEI messages. Video compression standards such as VVC, HEVC and AVC defines messages, called supplemental enhancement information (SEI) messages, comprising metadata providing information relative to a video stream. A SEI message is a data container or a syntax structure associated with the video stream. SEI messages offer a solution for transporting the interoperability metadata. A new SEI message dedicated to the transport of the Interoperability metadata could be defined. These SEI messages could be transported in the form of SDI VANC messages as defined in SMPTE ST 2108-1. One can note that these SEI messages could alternatively be inserted in a transport stream along with an encoded video stream distributed by the master central control system. 23
2023PF00065 As already mentioned earlier, there are many existing video equipment that are embedding HDR to SDR (TM) or SDR to HDR (ITM) conversions. These conversions could be realized either using static 3D-LUTs or using dynamic solutions. In this context, multiple scenarios of use of the interoperability metadata are possible. Some different scenarios are provided in the following. 1st scenario: the interoperability metadata matches a static 3D-LUT known by the module in charge of the conversion. When the “6” information contained in the interoperability metadata matches with a 3D-LUT known by the module in charge of the conversion, the selection of the 3D-LUT to apply is straight-forward for the conversion module. For example, if the single-master HDR/SDR workflow is using a HLG content and if the “6” information are as follows: ^ SDR_DW = 86 ^ HDR_DW = 203 ^ NFR_MIN = 4 ^ SDR_NFR_MAX = 1006 ^ HDR_NR = 294 ^ HDR_NFR = 1810 Then, the NBCU LUT 3 as described in table TAB2 is the appropriate choice. 2nd scenario: the interoperability metadata does not match any static 3D-LUT known by the module in charge of the conversion. When the “6” information of the interoperability metadata are not matching with one static 3D-LUT known by the conversion module, the equipment determines which 3D-LUT would be the most appropriate. For that, many different algorithms can be defined. For instance: ^ Make the decision based on HDR_DW value only and select the 3D- LUT corresponding to the HDR diffuse white value the closer the HDR_DW value; 24
2023PF00065 ^ Make the decision based on HDR_DW value only and select the 3D- LUT corresponding to the HDR diffuse white value that is closer to but lower than the HDR_DW value; ^ Make the decision based on HDR_DW value only and select the 3D- LUT corresponding to the HDR diffuse white value that is closer to but higher than the HDR_DW value; ^ Make the decision based on SDR_DW value only and select the 3D-LUT corresponding to the SDR diffuse white value the closer the SDR_DW value; ^ Make the decision based on SDR_DW value only and select the 3D-LUT corresponding to the SDR diffuse white value that is closer to but lower than the SDR_DW value; ^ Make the decision based on SDR_DW value only and select the 3D-LUT corresponding to the SDR diffuse white value that is closer to but higher than the SDR_DW value; ^ Make the decision based on both HDR_DW and SDR_DW values and select the 3D-LUT corresponding to the HDR diffuse white and SDR diffuse white values the closest to the HDR_DW value and the SDR_DW value. ^ Make the decision based on HDR_DW and SDR_DW values and select the 3D-LUT corresponding to the HDR diffuse white and the SDR diffuse white values that are closer to but lower than the HDR_DW and SDR_DW values; ^ Make the decision based on HDR_DW and SDR_DW values only and select the 3D-LUT corresponding to the HDR and SDR diffuse white values that are closer to but higher than the HDR_DW and SDR_DW values; ^ Make the decision based on any combination of “1” to “6” of the information and select the 3D-LUT for which the combination of the “1” to “6” information is the closest to the “1” to “6” characteristics characterizing the 3D-LUT. The method used by the conversion module is not restricted to the methods proposed methods. 25
2023PF00065 3rd scenario: the interoperability metadata are used to mimic existing single- master HDR/SDR workflow. In some existing single-master HDR/SDR workflow (for example, the one proposed by NBCU (see NBCUniversal Single-Master Broadcast Production and Distribution Recommendations #NBCU-Rec-UHD-HDR-01.13, September 23nd, 2022, https://github.com/digitaltvguy/NBCUniversal-UHD-HDR-SDR-Single-Master- Production-Workflow-Recommendation-LUTs)), it is recommended to use specific static 3D-LUTs for SDR to HDR conversion and for HDR to SDR conversion. For instance, if the NCBU workflow is used in Fig.2, the ITM tools 202A, 202B and 202C use the ITM 3D-LUT NBCU LUT 1 (see table TAB2) and the TM tools 204A and 204B uses the counterpart TM 3D-LUTs NBCU LUT 3. One possibility to ensure the compatibility with the NCBU workflow, when generating a HDR content with the ITM 3D-LUT NBCU LUT 1, the ITM tool fills the interoperability metadata with SDR_DW, HDR_DW , NFR_MIN , SDR_NFR_MAX , HDR_NR and HDR_NFR values corresponding to the counterpart TM 3D-LUT NBCU LUT 3. When receiving the interoperability metadata, the TM tools 204A and 204B select the 3D-LUT specified by the interoperability metadata, i.e., select the TM 3D- LUT NBCU LUT 3. The use of the interoperability metadata makes the selection of the appropriate TM 3D-LUT automatic and reliable. 4th scenario: the interoperability metadata are used to improve a SDR-HDR- SDR round trip in a single-master HDR/SDR workflow. While in the previous scenario, the SDR-HDR-SDR round trip is not ensured, one objective of the 4th scenario is to show how the SDR-HDR-SDR round trip could be improved using the interoperability metadata. Taking again the example of the single-master HDR/SDR workflow proposed by NBCU, when generating a HDR content with the ITM 3D-LUT NBCU LUT 1, in the 4th scenario, the ITM tool fills the interoperability metadata with SDR_DW, HDR_DW , NFR_MIN , SDR_NFR_MAX , HDR_NR and HDR_NFR values corresponding to the ITM 3D-LUT NBCU LUT 1. 26
2023PF00065 In that case, when an equipment (for instance the TM tools 204A and 204B) receives a HDR content along with interoperability metadata corresponding to the ITM 3D-LUT NBCU LUT 1, it can select a new-defined TM 3D-LUT that matches perfectly the characteristics of the ITM 3D-LUT NBCU LUT 1. Therefore, this new TM 3D- LUT would allow a perfect SDR-HDR-SDR round trip. 5th scenario: Dynamic HDR to SDR converter configuration Let’s consider that an equipment (for example the TM tools 204A and 204B) embeds a dynamic tone mapping solution that is compatible with the interoperability metadata, i.e. it is able to produce a tone mapping curve as shown in Fig.3B. When the equipment receives a native HDR content along with the interoperability metadata, the equipment configures the dynamic solution with the “6” information, allowing the dynamic solution to deliver the proper TM conversion. When the equipment receives a HDR content resulting from ITM conversion along with the interoperability metadata that describes the characteristics of the ITM conversion, the equipment configures the dynamic system with the “6” information, allowing the dynamic system to deliver the proper TM conversion, i.e. it is able to respect the SDR-HDR-SDR round-trip constraint. Whenever the values of the interoperability metadata is changing, either from a native HDR content or from a HDR content resulting from an ITM conversion, the dynamic solution is always able to adapt dynamically to new HDR content characteristics. Fig. 8A illustrates schematically an example of hardware architecture of a processing module 80 comprised in the live production system 20, in a system or module comprised in the live production system 20 such as the ITM tools 202A, 202B and 202C or the TM tools 204B and 204A, in the master central control system 21 or in a system or module of the master central control system 21 such as the ITM tool 211, or in the devices 22A and 22B. The processing module 80 comprises, connected by a communication bus 805: a processor or CPU (central processing unit) 800 encompassing one or more microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting 27
2023PF00065 examples; a random access memory (RAM) 801; a read only memory (ROM) 802; a storage unit 803, which can include non-volatile memory and/or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and/or optical disk drive, or a storage medium reader, such as a SD (secure digital) card reader and/or a hard disc drive (HDD) and/or a network accessible storage device; at least one communication interface 804 for exchanging data with other modules, devices, systems or equipment. The communication interface 804 can include, but is not limited to, a transceiver configured to transmit and to receive data over a communication network 81. The communication interface 804 can include, but is not limited to, a modem or a network card. For example, the communication interface 804 enables for instance the processing module 80 to HDR or SDR data along with interoperability metadata. The processor 80 is capable of executing instructions loaded into the RAM 801 from the ROM 802, from an external memory (not shown), from a storage medium, or from a communication network. When the processing module 80 is powered up, the processor 800 is capable of reading instructions from the RAM 801 and executing them. These instructions form a computer program causing, for example, the implementation by the processor 800 of the HDR source 200, of the ITM tools 202A, 202B and 202C and 211 of the method of Fig.5 and the implementation by the processor 800 of the TM tools 204A and 204B of the method of Fig.6. All or some of the algorithms and steps of said processes may be implemented in software form by the execution of a set of instructions by a programmable machine such as a DSP (digital signal processor) or a microcontroller, or be implemented in hardware form by a machine or a dedicated component such as a FPGA (field- programmable gate array) or an ASIC (application-specific integrated circuit). Fig.8C illustrates a block diagram of an example of system A that corresponds to device 22A or 22B in which various aspects and embodiments are implemented. System A can be embodied as a device including various components or modules and is configured to generate a SDR or HDR content adapted to be displayed on adapted display devices. Examples of such system include, but are not limited to, 28
2023PF00065 various electronic systems such as personal computers, laptop computers, smartphones, tablet, TV, or set top boxes. Components of system A, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and/or discrete components. For example, in at least one embodiment, the system A comprises one processing module 80 that implements a decoding of a SDR or HDR content. In various embodiments, the system A is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports. The input to the processing module 80 can be provided through various input modules as indicated in block 60. Such input modules include, but are not limited to, (i) a radio frequency (RF) module that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input module (or a set of COMP input modules), (iii) a Universal Serial Bus (USB) input module, and/or (iv) a High Definition Multimedia Interface (HDMI) input module. Other examples, not shown in FIG.8C, include composite video. In various embodiments, the input modules of block 60 have associated respective input processing elements as known in the art. For example, the RF module can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the down-converted and band- limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF module of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion can include a tuner that performs various of these functions, including, for example, down-converting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. Various embodiments rearrange the order of the above- described (and other) elements, remove some of these elements, and/or add other elements performing similar or different functions. Adding elements can include inserting elements in between existing elements, such as, for example, inserting 29
2023PF00065 amplifiers and an analog-to-digital converter. In various embodiments, the RF module includes an antenna. Additionally, the USB and/or HDMI modules can include respective interface processors for connecting system A to other electronic devices across USB and/or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed-Solomon error correction, can be implemented, for example, within a separate input processing IC or within the processing module 80 as necessary. Similarly, aspects of USB or HDMI interface processing can be implemented within separate interface ICs or within the processing module 80 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to the processing module 80. Various elements of system A can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in the system A, the processing module 80 is interconnected to other elements of said system A by the bus 805. The communication interface 804 of the processing module 80 allows the system A to communicate on the communication network 81. The communication network 81 can be implemented, for example, within a wired and/or a wireless medium. Data is streamed, or otherwise provided, to the system A, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi- Fi signal of these embodiments is received over the communications network 81 and the communications interface 804 which are adapted for Wi-Fi communications. The communications network 81 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications. Still other embodiments provide streamed data to the system A using the RF connection of the input block 60. As indicated above, various embodiments provide data in a non- streaming manner, for example, when the system A is a smartphone or a tablet. Additionally, various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network. 30
2023PF00065 The system A can provide an output signal to various output devices using the communication network 81 or the bus 805. For example, the system A can provide a decoded SDR or HDR signal. The system A can provide an output signal to various output devices, including a display 64 (if for example the system A is a set top box provided a decoded SDR or HDR signal to a display device), speakers 65, and other peripheral devices 66. The display 64 of various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and/or a foldable display. The display 64 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or other devices. The display 64 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop). The display device 64 is SDR or HDR content compatible. The other peripheral devices 66 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and/or a lighting system. Various embodiments use one or more peripheral devices 66 that provide a function based on the output of the system A. For example, a disk player performs the function of playing the output of the system A. In various embodiments, control signals are communicated between the system A and the display 64, speakers 65, or other peripheral devices 66 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communications protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to system A via dedicated connections through respective interfaces 61, 62, and 63. Alternatively, the output devices can be connected to system A using the communication network 81 via the communication interface 804. The display 64 and speakers 65 can be integrated in a single unit with the other components of system A in an electronic device such as, for example, a television. In various embodiments, the display interface 61 includes a display driver, such as, for example, a timing controller (T Con) chip. The display 64 and speakers 65 can alternatively be separate from one or more of the other components, for example, if the RF module of input 60 is part of a separate set-top box. In various embodiments in which the display 64 and speakers 65 are 31
2023PF00065 external components, the output signal can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs. Fig. 8B illustrates a block diagram of an example of the system B adapted to implement the live production system 20 or a module or device of the live production system 20 or the master central control system 21, or a module or a device of the master central control system 21 in which various aspects and embodiments are implemented. System B can be embodied as a device including the various components and modules described above and is configured to perform one or more of the aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, a camera, a smartphone and a server. Elements or modules of system B, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and/or discrete components. For example, in at least one embodiment, the system B comprises one processing module 80 that implement either an ITM tool (202A, 202B, 202C, 211) or a TM tool (204A, 204B). In various embodiments, the system B is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and/or output ports. The input to the processing module 80 can be provided through various input modules as indicated in block 60 already described in relation to Fig.8C. Various elements of system B can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in the system B, the processing module 80 is interconnected to other elements of said system B by the bus 805. The communication interface 804 of the processing module 80 allows the system B to communicate on the communication network 81. The communication network 81 can be implemented, for example, within a wired and/or a wireless medium. Data is streamed, or otherwise provided, to the system B, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi- 32
2023PF00065 Fi signal of these embodiments is received over the communications network 81 and the communications interface 804 which are adapted for Wi-Fi communications. The communications network 81 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications. Still other embodiments provide streamed data to the system B using the RF connection of the input block 60. As indicated above, various embodiments provide data in a non- streaming manner. When a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of a corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of a corresponding method/process. The implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented, for example, in a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs"), smartphones, tablets, and other devices that facilitate communication of information between end-users. Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment. Additionally, this application may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, 33
2023PF00065 estimating the information, calculating the information, predicting the information, retrieving the information from memory or obtaining the information for example from another device, module or from user. Further, this application may refer to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information. Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information. It is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of”, “one or more of” for example, in the cases of
“A and/or B” and “at least one of A and B”, “one or more of A and B” is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, “one or more of A, B and C” such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed. As will be evident to one of ordinary skill in the art, implementations or embodiments can produce a variety of signals formatted to carry information that can 34
2023PF00065 be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data produced by one of the described implementations or embodiments. For example, a signal can be formatted to carry a HDR or SDR image or video sequence and interoperability metadata of a described embodiment. Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting can include, for example, encoding a HDR or SDR image or video sequence with interoperability metadata in an encoded stream and modulating a carrier with the encoded stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium. We described above a number of embodiments. Features of these embodiments can be provided alone or in any combination. Further, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types: ^ A bitstream or signal that includes one or more of the described SDR or HDR data and the interoperability metadata, or variations thereof. ^ Creating and/or transmitting and/or receiving and/or decoding a bitstream or signal that includes one or more of the described SDR or HDR data and/or interoperability metadata, or variations thereof. ^ A server, camera, TV, set-top box, cell phone, tablet, personal computer or other electronic device that performs at least one of the embodiments described. ^ A TV, set-top box, cell phone, tablet, personal computer or other electronic device that performs at least one of the embodiments described, and that displays (e.g. using a monitor, screen, or other type of display) a resulting image. ^ A TV, set-top box, cell phone, tablet, personal computer or other electronic device that tunes (e.g. using a tuner) a channel to receive a signal including encoded SDR or HDR data and interoperability metadata, and performs at least one of the embodiments described. ^ A TV, set-top box, cell phone, tablet, or other electronic device that receives (e.g. using an antenna) a signal over the air that includes SDR or HDR data 35
2023PF00065 and interoperability metadata, and performs at least one of the embodiments described. ^ A server, camera, cell phone, tablet, personal computer or other electronic device that tunes (e.g. using a tuner) a channel to transmit a signal including SDR or HDR data and interoperability metadata, and performs at least one of the embodiments described. ^ A server, camera, cell phone, tablet, personal computer or other electronic device that transmits (e.g. using an antenna) a signal over the air that includes SDR or HDR data and interoperability metadata, and performs at least one of the embodiments described. 36
Claims
2023PF00065 Claims 1. A method comprising: obtaining (501) input video data in an input dynamic range; obtaining (502) interoperability metadata representative of the input video data in the input dynamic range and allowing a conversion of the input video data in output video data in an output dynamic range; transmitting (503) the input video data and the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. 2. The method of claim 1 wherein the input dynamic range is HDR or SDR. 3. The method of claim 1 or 2 wherein the interoperability metadata are transmitted using ancillary channels of a SDI interface. 4. The method of claim 3 wherein the interoperability metadata are embedded in SL-HDR1 metadata. 5. The method of any previous claim from claim 1 to 3 wherein the interoperability metadata are transmitted using a SEI message. 6. A method comprising: obtaining (601) input video data in an input dynamic range and interoperability metadata representative of the input video data in the input dynamic range and allowing a conversion of the input video data in output video data in an output dynamic range; converting (602) the input video data in the output video data in the output dynamic range using the interoperability metadata; 37
2023PF00065 wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. 7. The method of claim 6 wherein the input dynamic range is HDR or SDR. 8. The method of claim 6 or 7 wherein the interoperability metadata are used to define a tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is HDR and the output dynamic range is SDR and the interoperability metadata are used to define an inverse tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is SDR and the output dynamic range is HDR. 9. The method of claim 6, 7 or 8 wherein the interoperability metadata are received using ancillary channels of a SDI interface. 10. The method of claim 9 wherein the interoperability metadata are embedded in SL-HDR1 metadata. 11. The method of any previous claim from claim 6 to 9 wherein the interoperability metadata are transmitted using a SEI message. 12. A device comprising electronic circuitry configured for: obtaining (501) input video data in an input dynamic range; obtaining (502) interoperability metadata representative of the input video data in the input dynamic range and allowing a conversion of the input video data in the input dynamic range in output video data in an output dynamic range; transmitting (503) the input video data and the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an 38
2023PF00065 SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. 13. The device of claim 12 wherein the input dynamic range is HDR or SDR. 14. The device of claim 12 or 13 wherein the interoperability metadata are transmitted using ancillary channels of a SDI interface. 15. The device of claim 14 wherein the interoperability metadata are embedded in SL-HDR1 metadata. 16. The device of any previous claim from claim 12 to 14 wherein the interoperability metadata are transmitted using a SEI message. 17. A device comprising electronic circuitry configured for: obtaining (601) input video data in an input dynamic range and interoperability metadata representative of the input video data in the input dynamic range and allowing a conversion of the input video data in the input dynamic range in output video data in an output dynamic range; converting (602) the input video data in the output video data in the output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. 18. The device of claim 17 wherein the input dynamic range is HDR or SDR. 39
2023PF00065 19. The device of claim 17 or 18 wherein the interoperability metadata are used to define a tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is HDR and the output dynamic range is SDR and the interoperability metadata are used to define an inverse tone mapping process allowing converting the input video data from the input to the output dynamic range responsive to the input dynamic range is SDR and the output dynamic range is HDR. 20. The device of claim 17, 18 or 19 wherein the interoperability metadata are received using ancillary channels of a SDI interface. 21. The device of claim 20 wherein the interoperability metadata are embedded in SL-HDR1 metadata. 22. The device of any previous claim from claim 17 to 20 wherein the interoperability metadata are transmitted using a SEI message. 23. Non-transitory information storage medium storing program code instructions for implementing the method according to any previous claims from claim 1 to 11. 24. A signal comprising interoperability metadata representative of input video data in an input dynamic range and allowing a conversion of the input video data in the input dynamic range in output video data in an output dynamic range comprising at least one information among an information representing an HDR diffuse white value, an information representing an SDR diffuse white value, an information representing an HDR narrow range value, an information representing an HDR narrow full range value, an information representing an SDR narrow full range maximum value and an information representing narrow full range minimum value. 25. A computer program comprising program code instructions for implementing the method according to any previous claim from claim 1 to 11. 40
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP23305273 | 2023-03-02 | ||
| PCT/EP2024/053976 WO2024179856A1 (en) | 2023-03-02 | 2024-02-16 | Guided conversions between two different dynamic ranges with new metadata |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4674129A1 true EP4674129A1 (en) | 2026-01-07 |
Family
ID=85726186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP24705492.7A Pending EP4674129A1 (en) | 2023-03-02 | 2024-02-16 | Guided conversions between two different dynamic ranges with new metadata |
Country Status (5)
| Country | Link |
|---|---|
| EP (1) | EP4674129A1 (en) |
| JP (1) | JP2026508871A (en) |
| KR (1) | KR20250155016A (en) |
| CN (1) | CN120814239A (en) |
| WO (1) | WO2024179856A1 (en) |
-
2024
- 2024-02-16 WO PCT/EP2024/053976 patent/WO2024179856A1/en not_active Ceased
- 2024-02-16 CN CN202480015220.3A patent/CN120814239A/en active Pending
- 2024-02-16 KR KR1020257028830A patent/KR20250155016A/en active Pending
- 2024-02-16 EP EP24705492.7A patent/EP4674129A1/en active Pending
- 2024-02-16 JP JP2025551151A patent/JP2026508871A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| JP2026508871A (en) | 2026-03-13 |
| CN120814239A (en) | 2025-10-17 |
| WO2024179856A1 (en) | 2024-09-06 |
| KR20250155016A (en) | 2025-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2014335620A1 (en) | Decoding device and decoding method, and coding device and coding method | |
| US20260010992A1 (en) | Method for preventing clipping in sl-hdrx systems | |
| US20250131545A1 (en) | Coupled inverse tone mapping and tone mapping | |
| CN116508054B (en) | Methods, apparatus, and devices for avoiding chroma clipping in tone mappers while maintaining saturation and preserving hue. | |
| US20250005725A1 (en) | Tone mapping with configurable hdr and sdr diffuse white levels | |
| EP4723650A1 (en) | Guided conversions between two different dynamic ranges with new metadata related to reference level | |
| US12519960B2 (en) | Method for correcting SDR pictures in a SL-HDR1 system | |
| EP4674129A1 (en) | Guided conversions between two different dynamic ranges with new metadata | |
| US12477090B2 (en) | Method for limiting effects of quantization in a color gamut modification process applied to a video content | |
| EP4625405A1 (en) | Display energy reduction mdcv message | |
| EP4738813A1 (en) | Limitation of a quantization effect when exchanging video content using a channel with different color gamuts by yuv adjustment | |
| US20260101024A1 (en) | Method for reducing a quantization effect in a color gamut modification process applied to a video content | |
| EP4718372A1 (en) | Constant-time video processing for sdr to sl-hdr converter | |
| WO2025176438A1 (en) | Display energy reduction sl-hdr sei message | |
| WO2026092986A1 (en) | Limitation of a quantization effect when exchanging video content using a channel with different color gamuts by yuv adjustment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20250728 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |