RU2790178C2 - Multirange encoding of video with expanded dynamic range - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: invention relates to methods and devices for encoding of images with an expanded dynamic range. Encoder (900) of video with an expanded dynamic range is made with the possibility of reception through image input (920) of an input image (MsterHDR) with an expanded dynamic range, which has the first maximum pixel luminosity (PB_C_H50). At the same time, the encoder is made with the possibility of reception through metadata input (921) of the main brightness display function (FL_50t1), which determines the relationship between normalized brightness of the input image with an expanded dynamic range and normalized brightness of the corresponding image (Im_LDR) with a standard dynamic range, having the maximum pixel luminosity of preferably 100 nits. In this case, the encoder contains metadata input (923) for reception of the second maximum pixel luminosity (PB_CH). The encoder additionally contains HDR function generation block (901) made with the possibility of use of a standardized algorithm for conversion of the main brightness display function (FL_50t1) into an adapted brightness display function (F_H2hCI), which links normalized brightness of the input image with an expanded dynamic range with normalized brightness of an image with an intermediate dynamic range (hereinafter – IDR), which differs in the maximum possible luminosity equal to the second maximum pixel luminosity (PB_CH); IDR image calculation block (902) made with the possibility of use of the adapted brightness display function (F_H2hCI) to brightness of pixels of the input image (MsterHDR) with an expanded dynamic range to obtain brightness of pixels of IDR image; and IDR display function generator (903) made with the possibility of reception, based on the main brightness display function (FL_50t1) and the adapted brightness display function (F_H2hCI), of a channel brightness display function (F_I2sCI), which determines, as output data, corresponding normalized brightness of the image (Im_LDR) with a standard dynamic range, when corresponding normalized brightness of IDR image is set as input data. In this case, the encoder additionally outputs IDR image, the second maximum pixel luminosity (PB_CH) as the first metadata, the channel brightness display function (F_I2sCI) as the second metadata, and the first maximum pixel luminosity (PB_C_H50) as the third metadata.

EFFECT: provision of encoding of images with an expanded dynamic range with adapted brightness data for displays with different capacities of display of a dynamic range of image brightness.

13 cl, 28 dwg

Description

FIELD OF TECHNOLOGY

The invention relates to methods and apparatus for encoding images with a high dynamic range and, in particular, video, which is a time sequence of images that can be compressed in accordance with compression methods such as the High Efficiency Video Coding (HEVC) standard developed by the Motion Picture Expert Group (MPEG) images (e.g. television broadcasts), in particular using the image(s) with a second dynamic range (for transmission to a decoder) in order to present the (main) image(s) with a first dynamic range, the dynamic range change of which provides for changing the luminances of image pixels (eg, from a first value normalized to 1.0 to a second value normalized to 1.0) with functions to be co-transmitted with the second dynamic range image(s), typically as metadata.

BACKGROUND OF THE INVENTION

About 5 years ago, new High Dynamic Range (HDR) video coding methods were introduced (leading, for example, to special Blu-ray HDR discs that must be played on "UltraHD Premium" TVs at 1000 nits).

This new way of technically processing images is in many ways technically in contrast to the legacy video coding that has been used to encode all videos for the past 50 years up to this point, and which is now called standard dynamic range (SDR) video coding (also known as video coding). low dynamic range (LDR) video). Image representation requires digitally encoded representations of pixel colors, and the SDR luminance code determination function (also known as the opto-electrical transfer function (OETF)) from Rec. The 709 could encode (using 8 or 10-bit luma words) a luminosity dynamic range of only about 1000:1 due to its roughly square root shape (luminance: Y=sqrt(Luminance L)). However, this was ideally suited for encoding images to be displayed on displays of the time, which have standard luminance reproduction capabilities (for all displays at the time) of approximately 0.1 to 100 nits, the latter being the so-called peak luminance (PB) , also known as maximum luminosity.

Due to the fact that the function of determining the brightness code from the recommendation of Rec. 709 cannot mathematically describe the huge range of HDR scene image luminances (e.g. 0.001 nits to 10000 nits of required peak image coding PB_C brightness), HDR researchers initially addressed this problem by developing a new HDR code distribution that was much more logarithmic in shape , so that many more luminosities can be encoded (because the visual system requires less precision, i.e., fewer code values for brighter areas compared to darker areas, it follows that by distributing, for example, 50 codes out of 2 ^8 = 256 (where "^" denotes a power function) for each decade of luminosities, it is already possible to encode a dynamic range of 100000:1). This is a simple "natural" way to encode the colors of an HDR image using the so-called perceptual quantizer (PQ) function standardized in SMPTE 2084.

It might be naive to think that this is all it takes to encode and decode HDR images, but it's not that simple; therefore, additional approaches to encoding have appeared, in particular, the previously developed approach to encoding and processing HDR video proposed by the present applicant.

FIG. 1 summarizes some of the important aspects to get a good idea of what is involved and needed in HDR video coding.

Assume that on the left side there is a luminosity range of all possible (PQ-decoded) HDR luminosities up to PB_C = 5000 nits. Suppose for a moment that in order to make this image look perfect as required, all the object pixels of what we will call the main HDR image are created on a computer (how to proceed, for example, in the case of a broadcast television camera is explained below with FIG 2). The problem with the native HDR codec (which simply offers the technology to encode luminances up to 10,000 nits, i.e. up to 5,000 nits as required in this example) is that if the consumer also has an expensive display with peak brightness (PB_D ) display of 5000 nits (and if he is viewing the image in standard viewing environment conditions), he can watch the video exactly as the creator intended (for example, a filmmaker), but if he has a different display (for example, with PB_D = 750 nits or PB_D = 100 nits), then there is the following unsolved and also non-trivial problem: how to display an image with PB_C 5000 nits on a display with PB_D 750 nits? There doesn't seem to be a simple elegant solution to this problem. Applying accurate luminance mapping will perfectly render all objects with luminance up to 750 nits, but will cut off all brighter object pixels to the same PB_D = 750 nits, resulting in many objects in the image in the area of white spots, which, of course doesn't look pretty. One might think that the solution is to linearly scale the "content" or content (dividing all HDR luminances by 5000/750, which is the so-called "white content on white display" approach), but then darker objects such as a human in the dark area of the cave in the ImSCN3 image example, scenes with HDR luminosities (0.05 nits), which can also be too low for lower dynamic range displays, acquire an imperceptible low luminosity on a 750 nits display (0.05*750/5000 = 0.0075 nits).

FIG. 1 also shows that different HDR images of different typical HDR scenes can have quite different requirements as to how to "compress" different (if possible, at "arbitrary" luminance positions in the DR_2 range of HDR luminosities) HDR luminosities to much smaller ones, for example , dynamic range DR_1 of LDR luminosities.

Luminances in the real world can vary, for example, when indoor and outdoor objects are simultaneously present in the view in relation to: lighting contrast (illumination_contrast) * object reflection contrast (object_reflection_contrast) = (1:100)*(1:100), and while the luminances in an image representing the scene will generally not necessarily be identical to the original scene luminosities, for a good representative HDR image one would expect pixel luminances to be at least 1000 nits, starting from a minimum of at least 0 .1 nit or less (hence DR_im >= 10,000:1). In addition, good HDR images may have more to do with cleverly distributing different object luminosities over a luminosity range than with the physical dynamic range itself (not to mention the misunderstanding that everything is determined by the number of bits - this is not true for non-linear luminance code definition functions, since a 10-bit luma image could just as well be some HDR image as well as an SDR image).

FIG. 1 shows a couple of typical illustrative examples of the many possible HDR scenes that an HDR system of the future (e.g. connected to a display with a PB_D of 1000 nits) may need to be able to handle correctly, in particular by displaying appropriate luminosities for all objects/pixels on the image.

For example, ImSCN1 is an image of a sunlit outdoor scene from a western (which mostly contains bright areas that are brighter than average, which would correspond to a dim day image, and these areas should ideally be reproduced brighter than a 100 nit display to provide a sunnier view than on a rainy day, for example, with an average luminosity of, say, 400 nits). ImSCN2, on the other hand, is a completely different kind of image, namely a night scene image that is dominated by dark areas (and, for example, their good visibility), and what makes this image an HDR image and not just a dark SDR -image, is also the presence of bright pixels in spots under street lamps and possibly in illuminated windows of houses, and even the presence of very bright pixels (for example, 3000 nits) on the surfaces of lamps of street lamps.

What makes such an ImSCN1 image "sunny" compared to a dark ImSCN2 image? Optionally, relative luminosities, at least not in the SDR paradigm (pixel luminosities will be in the entire range from 0.1 to 100 nits, possibly for both images, although the spatial distribution of such luminosities and, in particular, the histogram may differ). What makes HDR image reproduction different from what it always was in the SDR era, which ended just a couple of years ago, is that SDR represented such a limited dynamic range (about PB = 100 nits, and a minimum MB black level of about 0.1 to 1 nit) and that basically only the intrinsic reflectances of objects (which would be in the range of 90% for good white to 1% for good black) could be shown in SDR. This would be good for recognizing objects (having a specific degree of brightness due to their reflection and, of course, their chromaticity) under uniform technically controlled lighting, but in natural scenes there are not many beautiful variations in the lighting itself that can affect viewers ( a sunbeam coming out of a window, or plasma coming from a witch). If the display allows it, and hence the technology of coding and image processing, then it would be possible, walking through the forest, to really see how the sun shines through the trees, i.e. instead of just a slightly more yellowish picture as on an SDR display, one would like to see bright and multi-colored sunlit clothes when a person steps out of the shadows into the sun. And, for the same reason, fire and explosions should have optimal visual impact, at least as far as PB_D allows.

In SDR, it is possible to make an image of a night scene somewhat darker than an image of a normal lit scene, as seen in the brightness histogram, but not too much, otherwise it will simply be displayed as too dark and unpleasant (perhaps even largely invisible) image (namely therefore, an agreement was reached to keep images of night scenes still relatively bright, but blue). Also, on a 100 nit TV or 100 nit encoding, there's simply no room for anything too bright. Thus, it was necessary to show objects regardless of their lighting, and at the same time it was impossible to accurately show all the sometimes very contrasting scene lighting that could occur. In practice, this meant that a very bright sunny scene had to be reproduced with approximately the same display luminosities (0-100 nits) as a dim rainy day or even a night scene.

In real life, human vision can also adapt to the amount of light available, but not as much (most people in real life still understand when it starts to get dark, or that they are in a darker or fairly bright environment). And it should not be forgotten that television showing images is not a simulation of an adapted eye, but rather a simulation of real environmental conditions, as far as possible, given viewing conditions and other technical limitations. Thus, one would like to display images with all visible local and also temporal lighting effects that can be artistically embodied in images to produce much more realistic reproducible images, at least if the target viewer has an HDR display. What kind of luminosity would be suitable for, say, a lightsaber in a dark room, we will leave to the discretion of a color grading specialist who creates the main (main) graduation (s) (for simplicity of presenting the idea of the invention in this application, we will assume that images with different dynamic range, at least two with very different dynamic range are created by a human grading specialist, but similarly, images can be created by automatic software), and this application will focus on the necessary technical components for creating and processing such images for various market participants with potentially different needs.

On the left axis in FIG. 1 shows the luminosities of objects as one would like to see them (for example) in a basic HDR grading with PB_C 5000 nits when directly displayed on a (reference) display with PB_D 5000 nits a home HDR HDR TV will have a PB_D of 5000 nits and it will be able to actually be present in the view of that home viewing room and calibrate on that grading display). If it is desired to convey not just the approximate illusion of the original HDR scene that was captured, but the actual feeling of the cowboy being in a bright, sunlit environment, these pixel luminosities must be set and made sufficiently bright, e.g., averaging approximately 500 nits.

For a night scene, mostly low luminosities are desirable, but the main character on a motorcycle must be well recognizable, i.e. not too dark (e.g. approximately 5 nits) while there may be pixels with fairly high luminosity, e.g. street light pixels, e.g. having approximately 3000 nits on a 5000 nits display or approximately peak brightness on any HDR display (e.g. with PB_D 1000 nits). The third sample image of ImSCN3 shows what is now also possible on HDR displays: it is possible to simultaneously display many (semantically more important than just a lamp, i.e. with more detail within the area, such as sunlit trees) very bright and many important very dark pixels. The ImSCN3 image shows that a typical and relatively difficult to process image of an HDR scene can be considered a dark cave with a hole through which one can see the sunny environment. This scene may need to make sunlit objects such as a tree somewhat less bright than in a scene that aims to give the impression of only a bright sunny landscape, for example, approximately 400 nits, which should be more consistent with the essentially dark character inside the cave . If necessary, the color grading specialist may need to optimally coordinate the luminosities of all objects (already in the main HDR image with PB_HDR = 5000 nits) so that nothing looks out of place dark or bright and there is good contrast, for example, a person standing in the dark in this cave , can be encoded on the main graduated HDR image using approximately 0.05 nits.

Once this basic HDR image is created, the following artistic question arises (even before it is formulated when the technology is involved): how this image should be re-graded into images with different dynamic range, for example, at least a legacy SDR image with a PB_C of 100 nits.

This helps to understand when relationships between luminosities are given, so we will do so in this application when it is convenient. In fact, from a technical point of view, luminosities will be encoded as luminances via a luminance code distribution function, also known as an opto-electrical transfer function (OETF), and hence one can also formulate all relationships between luminosities, for example, a function to calculate the output luminosity L_out from input luminosity L_in, also as relationships between equivalent luminances.

This may be a bit confusing, but it is also possible to express the luminosities in a normalized (i.e., using the maximum normalized luminosity of 1.0) way and define all actions on those normalized luminosities. This has the advantage that (assuming both image pixel colors are defined in the same set of RGB primaries) the normalized HDR gamut exactly overlaps the LDR gamut, and thus luminosity changes can be displayed in that single normalized colors. Obviously, the relative position of the normalized LDR luminosity, which should be displayed at exactly the same absolute luminosity as the HDR luminosity, which is given in the HDR luminosity range with PB_C = 5000 nits, will have a different relative height (i.e. can then be shown in such a gamut representation, a mapping of the luminosity for a particular HDR pixel luminosity needed to generate the corresponding LDR image pixel luminosity as a relative/normalized height change in that normalized gamut). The relationship between absolute and relative luminosities is simple: L_norm = L_abs/PB_C where PB_C is any maximum encoding luminosity, eg 5000 nits for HDR encoding and 100 nits for SDR by standard convention.

The last thing that is important to learn from FIG. 1 (because the whole technology should work accordingly) is that depending on what type of object (i.e., the luminosities of its pixels) in what kind of HDR scene you have to deal with, you can use different high-level approaches to how to recalibrate, i.e. convert the specified luminosity(s) of the pixels.

For example, an object in the dark, such as a motorcyclist, can be reproduced by equalizing the absolute luminosity (which involves a corresponding rescaling for normalized luminosity) for the entire regraded image, in particular the original HDR main image on the left, the corresponding SDR image on the right, and any an image with a medium dynamic range (MDR) in between, such as one shown with PB_C = PB_MDR = 800 nits, which is optimized (using the correct object luminosities) for direct display on a display with a PB_D of 800 nits (for example, for a consumer who has purchased such a display and receives HDR images with PB_C 5000 nits, for example, from your cable provider or via a satellite set-top box, or from the Internet, etc.). This makes sense because the content creator wants to convey a dark atmosphere in which the bike is just visible, and it would be bad to make it brighter on a brighter display for the simple reason that such a display can do this, since it has a larger range of luminosities, ending higher PB_D, to display all the luminosities of objects in the scene.

An object like the sun would probably follow a completely different philosophy, namely a way of displaying white on white that always gives it the highest possible value in any representation of the image, i.e. PB_C. Obviously, other kinds of objects may obey other kinds of rules, and one could go on and on (e.g. a cowboy would fit the scalable moderate gray philosophy), but it's enough for the reader to understand that there must be a technology that provides an almost "arbitrary" distribution of all pixel luminosities, and not, for example, a fixed one, similar to what simple technologies dictate.

Although FIG. 1 simply fills in the gaps in creating a generic HDR image (covering various technically limited applications such as movies, real sports broadcasting, etc.), it is still an open question for developers of HDR technology how to encode HDR images, and how to convert HDR images to be able to display them optimally on any display with a lower PB_D than the encoded PB_C (i.e. the brightest pixel that can appear at least once in a video). Capturing images of an HDR scene and, not least, decorating and lighting an HDR scene are also technical skills, but it is not necessary to focus on this aspect in this application.

The simplest option is to just encode the HDR pixel luminosities (ignoring the complexity of Display Adaptation (DA), i.e. how to map an image from PB_C1 to an image for a less capable display). The problem was that the Rec. 709 for OETF, it is possible to encode only the dynamic range of luminosities of 1000:1, i.e. it was necessary to invent a new OETF for HDR (or, in fact, its inverse EOTF). The first HDR codec called HDR10 was brought to market, which is used for example to create new Blu-ray HDR discs with black tapes, and it uses as OETF a more logarithmic function called the Perceptual Quantizer (PQ) function, which is standardized in SMPTE 2084 and which allows you to set the brightness for luminances from 1/10000 nits to 10000 nits, which is enough for the practical production of HDR video. Also, it has the good property that the luminance codes it generates match how human vision works (the kind of perceptual gray values that the brain uses to determine the different luminances in a scene, which is a nice property as for efficient regrading of specific gray objects, and for efficient representation of luminosities, as the brain does). After calculating the luminances, there was only a 10-bit pixel plane (or rather, only two 3-bit chrominance planes Cb and Cr), which can classically be processed further sequentially, "as if" they were an SDR image from a mathematical point of view, for example , compressed by MPEG (this is an important limitation because it avoids the redesign and redeployment of several pre-existing technologies in a common video processing pipeline).

A significant technical difficulty with HDR10 images still lies in how to properly display them on less capable displays (eg, those with less than PB_C 2000 nits functionality for which HDR content was created). If, for example, one simply maps rectilinearly white to white (the maximum white color of the encoded image, also known as the peak luminance PB_C of the encoding, for example, to the peak luminance PB_D of an SDR display), the most interesting (darker) parts of the image with PB_C = 1000 nits, would typically look 10 times darker, which would mean that the night scene in the ImSCN2 image would become unviewable. Due to the logarithmic nature of OETF PQ, HDR10 images are viewable (when they are simply rendered as luma, i.e. decoded with the wrong EOTF), but have an unsightly degraded contrast ratio, causing them to look, among other things, washed out and have the wrong brightness.

A simple system for creating HDR video content, such as in a broadcast scenario, is explained with FIG. 2. As before, the details of the distribution of non-linear luminance codes or colors of R'G'B' pixels are not yet considered in order to simplify interpretation (the so-called opto-optical approach: OOTF, with normal (absolute) luminosities in the entire sequence). With the exposure (EXP) of the camera (201), it is possible to choose which object luminosities are accurately recorded and at what relative value (because the camera functions as a relative luminosity meter for all spatial positions, or rather as a relative colorimeter providing RGB triplets). Since both the camera sensor and the N-bit mathematical representation of the color component have a practically finite range, from the minimum value to the maximum value, it makes sense not to expose the details of the sun, which corresponds to a billion nits, but to cut off at least these luminosity or RGB values to their maximum. Over an essentially unrestricted range, exposure selection can be "corrected" by subsequent remapping of the luminosity, but in any case this fact shows the reader that there is no "natural" obvious mapping of scene luminosities into the luminosities to be displayed (this mapping of luminosities is known as colorimetry associated with the display, and is really what matters in the end). The LIN_HDR image with linear luminosity is usually first subjected to OOTF mapping (202). It has existed to some extent already in the SDR era and aims to correct the fact that human vision in the usually darker viewing environment in the evening living room where TV is watched requires higher contrast for a similar visual experience, therefore OOTF, as is usually a soft gamma function. However, especially when displaying a high dynamic range scene on a typical lower dynamic range (205) display (even if it is a high quality 4000 nit reference monitor), some artistic optimization of the varying luminosity of object pixels may be required by applying a potentially arbitrary curve, referred to herein as graduation, using the block 203 graduation. Especially for high-quality offline productions, significant grading efforts may be required to create a so-called creative vision or look at the main HDR image MAST_HDR (which, according to the present invention, is still subject to further technical processing, for example, preferably encoding). The resulting image then looks optimal and can be sent via any image connection 204 to the display 205 where the human calibrator can check if the image already matches the desired one or continue to adjust at least one luminance display function via a user interface control unit 206 (for example, a calibration control panel). This arbitrary grading forms the main view - not to be confused with the arbitrary re-grading (secondary view) to obtain, for example, the best possible corresponding SDR picture, which can be called the main SDR picture (for example, when forming part of the video coding philosophy, as described below). While just another simple topology has been explained to the reader, the reader will be able to appreciate that there may be different actual implementations in practice depending on, for example, whether the actual broadcast is using only one HDR camera or a set of combined SDR and HDR cameras, or using a pre-defined HDR image and a corresponding re-graded SDR main image, which now need to be jointly encoded according to coding principles (eg ETSI1 or ETSI2 principles - see below), etc.

As already explained with FIG. 1, Applicant has realized that since there is a mathematical re-grading relationship between the various possible re-graded MDR images starting from the main HDR, provided one can technically capture these features pragmatically, it is actually possible to encode a whole range of different dynamic range features by sending only one of them and at least one luminosity display function for creating another image from the one actually sent. For the first time this possibility and the technical concept of coding resulting from it were described in WO2011107905.

It has been found that it makes sense to define a luminosity mapping function F_L to convert the luminances of the main HDR image (for example, with PB_C 5000 nits) to the luminosities of the SDR image, i.e. cause the calibrator to determine the necessary way to implement re-grading between the most extreme image representations, and then recalculate the display-adapted luminance mapping functions F_L_DA to calculate an intermediate pixel luminance of the MDR image corresponding to any possible luminance of the M_HDR image with a PB_C of 5000 nits.

As subsequently standardized by the Applicant, there are two logical options for actually transmitting an image (as a single image for the entire spectrum of re-graded images with different dynamic range, in particular with a PB_C endpoint, since it can often be assumed that the lower MB endpoint will be approximately fixed, e.g. 0.01 nits) to either the main HDR image or the corresponding SDR image (you should stop for a second to understand that in this situation, instead of HDR images, simple SDR images are actually transmitted, and due to the F_L function is also actually being passed HDR images, because L_HDR_reconstructed = F_L_inverse[L_SDR]).

The second encoding option, which is quite useful when the technical limitation is that many legacy displays must be used without distortion (in fact, the old SDR display simply receives an SDR image, and without requiring knowledge of what is also encoded in it HDR image, it can directly display the SDR image and immediately get a very good SDR reproduction of the HDR scene, actually as good as the display can display such an HDR scene), was standardized first in ETSI TS 103 433 -1 (note -1; abbreviation ETSI1 will be used in the following). It should be noted that there are technical limitations, such as the need for the colors of an SDR image to be reversible in order to be able to reconstruct the original HDR main image at any receiving end with sufficient fidelity, which was part of the technical consideration leading to this given standard approach to (de) coding.

ETSI TS 103 433-2 (ETSI2) is an encoding alternative in which the main HDR image is actually transmitted to the receivers and in which the F_L function(s) (in fact, as will be shown below, although for purposes of explanation one can think of the system as , as if one global function F_L was used for all luminances of pixels in the transferred image, for technical reasons a set of subsequently applied display functions is used) serves to calculate images for optimal display on displays with PB_D < PB_C_master (i.e. for so-called display adaptation). Different consumers can choose which system they want to use, for example, a cable TV operator transmitting HDR ETSI2 will provide its users with a set-top box (STB) that will decode and optimize for any display that the user has at home.

FIG. 3 first provides an overview of the components of a typical system (encoder + decoder) for transmitting HDR video with a single image and functions, not limited to a typical SDR communication type system, in order to explain the basic concepts.

The color converter 302 receives as input the MAST_HDR images (for example, as they were captured by the camera and graduated by the system shown in FIG. 2, and then transmitted through some professional video transmission system to the encoder 321 on the side of the broadcaster, which , for example, will transmit television programs over a wireless connection or through a television cable network) from the source 301 of the image. A set of color conversion functions F_ct is then applied (in this example, specifically defined by automatic grading software such as Applicant's technology for automatic HDR to SDR conversion, which defines the F_ct functions based on image characteristics such as histogram, etc. ; specific details can be ignored in the materials of this application, since it simply requires the presence of such optimized functions for any image or set of images sequential in time) containing at least a luminance mapping function F_L in order to obtain the corresponding SDR luminosities for the luminosities of the pixels of the main HDR images (MAST_HDR). For ease of understanding, the reader may, for the sake of simplicity, assume that F_L is a luminance mapping function to the 4th power (L_out_SDR = power(L_in_HDR; ¼)) to obtain normalized to 1.0 output SDR luminances of pixels in the output SDR image Im_LDR with PB_C 100 nits (i.e., the range of luminosities on the right in FIG. 1).

Since there is now a "normal" SDR image, it can be compressed using a standard video compression technique, such as an MPEG standard such as HEVC or MPEG2, or a similar standard such as AV1, with the compression performed by the video compressor 303.

Since the receivers must be able to reconstruct the main HDR image from the received corresponding compressed SDR image Im_COD, in addition to the actual pixel images to be transmitted, also the color mapping functions F_ct must be fed to the video compressor. Without limitation, it can be assumed that these functions are stored in the metadata, for example, using the SEI (Supplemental Optimization Information) mechanism or a similar technique. Finally, the formatter 304 does everything necessary to format (package into data blocks, etc.) the video stream for any technical communication environment 305, such as formatting for storage on a Blu-ray disc or for DVB communication via satellite, etc. . (the details of this can be found by a person skilled in the relevant fields of technology and are not relevant to understanding the ideas of the present invention).

After the MPEG decompression in the video receiver 320, performed by the video decompressor 307 (after passing through the reverse formatter 306), the SDR image can be interpreted by the receiver using the standard EOTF from Rec. 709 (in order to obtain an image for an SDR display), but the receiver may also decode the received Im_COD image differently to obtain a reconstructed HDR image Im_RHDR.

This is performed by color converter 308, which is configured to convert an Im_RLDR decompressed SDR image to an image of any non-SDR dynamic range (i.e. with a PB_C above 100 nits and typically at least 6 times higher) . For example, the original main image Im_RHDR with 5000 nits can be recovered by applying the inverse color transforms IF_ct from the color transforms F_ct used on the encoding side to create Im_LDR from MAST_HDR (which were received in the metadata and passed to the color converter 308). Or, a display adaptor 309 may be included that converts the SDR image Im_RLDR to a different dynamic range, such as an Im3000nit image that is optimally graduated if the display 310 is a 3000 nit PB display, or a 1500 nit or 1000 nit PB image. for corresponding displays with lower PB_D etc. It has been assumed, without limitation, that the video decoder and the color converter reside in the same video receiver 320. The experienced reader can appreciate that many different topologies can be designed in the same way, for example, with the decoding functionality implemented in the set-top box to be connected to a display that simply functions as a primitive display for pre-optimized images received, or that performs further color transformations of the image, and so on.

FIG. 4 summarizes the principles of Applicant's technology for displaying luminosity and color standardized in ETSI2 (in fact, this technology details the color converter 302, which was generally presented in FIG. 3 in accordance with the ETSI2 decoding philosophy (or similar to the ETSI1 coding philosophy) ) as it is necessary to know to understand some of the more specific techniques in the embodiments of the present application.

The input data is assumed to be YCbCr pixel colors given by PQ (ie, per-pixel color components of Y luma and Cb and Cr chrominance). First, the luminance is linearized to normal linear luminances L_in by the EOTF applicator 401, which must use the EOTF PQ from SMPTE 2084. Then, the entire recalibration process to obtain the SDR pixel output color from the HDR pixel input color can be determined again using the normal (universally defined through the physical SI and CIE) luminosities. Thereafter, luminosity processing can be performed by the luminance processor 401, which implements the full F_L mapping as desired, but with wisely chosen sub-blocks (these blocks 402, 403, etc. have been technically designed to meet the needs of various HDR applications such as automatic calibration, ease of human calibration, complexity of IC design, etc.).

First, the luminosity uniformizer applies a fixed-curve transformation whose shape depends only on the peak brightness PB_C_H of the input HDR image (for example, PB_C_H = 5000 nits), using one PB-dependent curve from a family of curves defined as:

Y'HP = log(1+(RHO-1)*power(L_in/PB_C_H; 1/(2,4)))/log(RHO) [Lv. 1]

Where

RHO = 1+32*power(PB_C_H/10000; 1/2.4) [Lv. 2]

This curve maps all luminosities to perceptually uniform gray Y'HP luminances. If PB_C_HDR = 10000 nits, then this curve exactly matches the PQ curve from SMPTE 2084, which is known to be perceptually uniform. For input images with lower PB_C_HDR, the curve scales well (actually it is a sub-curve ending, for example, at 3000 nits on a 10000 nits curve in an absolute sense), resulting in a less steep log-gamma curve for the darkest colors in normalized representation of the input/output luminosity axis [0-1.0]/[0-1.0]. In other words, the rest of the processing is already in progress, being well pre-normalized.

Subsequently, the black-white level shifter 403 may optionally apply some additional white level shift WLO and some black level shift BLO.

The usefulness of white level offset can be understood from the following. Suppose a content creator calibrates their images on a system set to PB_C = 4000 nits (for example, their reference monitor for grading has a PB_D of 4000 nits), however throughout the video, they actually never produce an image with a higher maximum pixel brightness than, eg 1500 nits (maximum for video, which is different from the encoded maximum of PB_D). Then, since the dynamic range of SDR luminances is quite small on its own, it makes sense to rescale the input HDR by removing the unused values of 1500-4000 nits (because it uses dynamically adjusted luminance mappings that can be optimized for each image/video at any given time). 1500/4000 corresponds to a normalized (input) HDR luminance of 0.375, so this value can be mapped to the maximum Y'HPS HDR scaled luminance by dividing by 2.6.

To be more precise, according to the ETSI2 standard, the following calculation is performed:

Y'HPS = (Y'HP-BLO)/(1-WLO-BLO) [Lv. 3]

In this case, WLO and BLO are transmitted in the metadata jointly transmitted or associated with the received video images.

Black level offset is useful for obtaining a more contrasty look for the corresponding re-graded SDR images, but remember that the received ESTI1 images must remain mapped back to the HDR image, i.e. too much black pixel detail should not be lost (which is why there is a parallel gain limiter not shown in FIG. 4).

In principle, one can simplistically interpret the black level offset as setting some HDR "black" color to 0.0 in SDR, or more specifically, through block 403 preparing to map HDR luminosity to SDR luminosity (i.e. with normalized luminances all still in HDR, namely with the relative distribution used to get a good look on an HDR display and a bad, not yet optimized look on an SDR display).

Coarse dynamic range converter 404 then applies a primary luminosity transform to obtain SDR luminosities (ie, with a good first redeployment of object luminances to get a reasonable look on SDR displays). To do this, ETSI2 uses a curve that consists of a linear segment with adjustable slope for the lowest normalized HDR luminosities (the slope of this segment is called shadow gain), another linear compressible part for the highest normalized Y'HPS HDR input luminances (with a slope control parameter, called highlight boost) and a controlled parabolic part that smooths them together, offering a nice looking SDR look for the midtones (with a control parameter called midtone width, as well as mathematical relationships that can be seen in the standard and which in this application are only re-explained (as simple and understandable as possible) to the extent necessary to understand new embodiments of the invention in accordance with the present ideas). Thus, the Y′CL output luminances of this coarse dynamic range converter 404 are first given in the SDR range or SDR relative luminance distribution statistics.

The technical (and artistic) suggestion for the content creator of this block 404 is that the grading specialist can perfectly optimize how bright he should make the darkest pixels, at the expense (due to the limited luminosity range of SDR) of the intra-object contrast of other objects, containing brighter pixels, but it can co-adjust, for example, glare enhancement. The intensification of the shadow can be perceived, for example, as follows: a person with a luminosity of 0.05 nits, standing in the dark shaded areas of the cave. If it were to be displayed on an SDR display with a white-on-white criterion, i.e. with a normalized mapping function that is the identity function with a 45 degree slope being the diagonal of the plot of the normalized luminosity function, it would be found that its normalized luminosity in HDR is 0.05/5000 and remains equal to the normalized luminosity due to the identity mapping for roughly mapped SDR luminosities, i.e. after they become absolute, these pixels should be displayed on the SDR display with (1/100000)*100, i.e. with a minimum of black (control signal "0") on this display and be invisible. Therefore, it is necessary to greatly enhance such luminosities already in more logarithmic uniformized relative gray values ​​for HDR and SDR or luminosity representation in order to obtain SDR luminosities that are sufficiently noticeable and lead to distinguishability of object textures within a human object (for example, the luminosity of human pixels in range of 0.3-1.2 nits shown on the SDR display). Therefore, depending on how deep one gets into the HDR luminosity range (which, as mentioned above, will depend on how the combination of factors such as HDR scene building, scene lighting, camera exposure, and artistic basic HDR grading , has been chosen by the content creator), the encoder (e.g., a human scaler creating a suitable F_L part that is this first choice of coarse luminosity mapping to recalibrate the main HDR input into optimal or appropriate corresponding SDR pixel luminosities) will select the appropriate gain shadows to handle the specified darkest pixels of that particular image (i.e. content-optimized image). It should be noted that ETSI actually defines the shadow gain SG as a correction for automatic scaling based on the ratio of the peak brightnesses of the input and output images - at least their brightness representations. In line with the philosophy of equal brightness, it makes sense to boost luminosities represented in a normalized range of brightnesses that correspond to, for example, only PB_C 200 nits (or rather the value according to Equations 1 and 2 above:

Y'HP = Y'200 = v(PB_C_H/200; RHO(200)), where v is the pseudo-log equation 1 above, starting with normalized HDR luminosities as: L_200 = Y'200*L_HDR. However, this usually results in an image that is too bright and low-contrast, so the calibrator can use exposure gain correction: SG = expgain*Y'200, which will be a dimming factor that returns SG to a diagonal value of 1.0 and adds some darkness back into the SDR image (generally it doesn't choose expgain = 1/Y'200 because then the normalized SDR luminances will be equal to the normalized HDR luminances and again too low; for example, SG will be in the range between 1.0 and 1, 8).

This kind of curve implements a non-linear luminosity compression "spring" to compress multiple HDR luminosities from a potentially much larger dynamic range (DR) of luminosities into a much smaller DR SDR. Since no fixed curve is used, which "on average should never be too implausible", but the encoder can apply an already optimized curve, the resulting SDR image will not be bad for many HDR scenes (not all HDR scenes are equally complex, for example, sometimes there is just a faint shaded area next to a uniformly sunlit area, and then while the simplest systems will create problems such as clipping to white, not too complex HDR-to-SDR smart mapping, such as the block 404 three-part curve, will typically , will already do a good job of producing a suitable re-graded SDR image for the main HDR image (for example, the one coming out of a content creator's HDR camera capturing a real event).

However, some other scenes may be more complex, and some content creators may also have a higher level of professional challenge when fine-tuning their feature content (for example, a Hollywood film director or director of photography (DOP)).

Therefore, the next block, i.e. a custom curve application block 405, allows the content creator (again, be it a human or an intelligent automatic vehicle with various rules encoded in its algorithm) to apply a customizable and optionally arbitrarily shaped luminance mapping function F_L_CU for precise grading to pre-graded luminances Y'CL, resulting in graduated Y'GL LDR luminances (the only requirement for the function is that it is non-decreasing and is typically monotonically increasing, and is typically at least chosen according to ETSI2 for mapping input value 1.0 to output value 1.0). In practice, the form of this F_L_CU function can be passed to decoders either as a set of parameters defining its form, such as polynomial coefficients, or as a LUT, etc.

Such precise grading may be required because the visual system has a complex way of determining grayscale impressions in the perceived subject of an image, and/or because compressing a large range of HDR luminances into a limited DR SDR may sometimes require some ingenuity, and/or because the content creator openly desires to add some additional artistic flair to this custom F_L_CU curve (the shape of which will then be determined, typically by computer hardware of a different color user interface and connected software on the encoding side, which are not shown in the figures). In fact, on the one hand, it can be said that all MDR images should be some compressed representation of all information (exclusively) in the main HDR image, but, on the other hand (because this can give a rather weak impression of images, for example, with too low contrast, as if they are being looked at through fog) another important requirement for content creation might be that all images up to and including the SDR image look - given their more limited DR capabilities - as realistic as possible, like an HDR scene, or at least as pretty as it gets. Human vision is very non-linear and highly organized, and it can quickly catch when a function that is too simple has been used. Thus, a content creator can intelligently use the custom F_L_CU function in addition to the coarse luminosity compression F_C function to succeed in creating an almost impossible SDR image that still looks as good as possible compared to the HDR scene and preferably the same. as well as an HDR scene (for example, reducing the brightness of a certain sub-range of pixel luminances in order to only slightly improve the contrast between objects, for example, for the brightness of a stained-glass window in comparison with the interior of a church, or the visual contrast between inside and outside rooms in an SDR image, or with the goal of optimizing chroma versus luminosity for some objects in the scene, choosing a special local shape for the F_L_CU curve, etc.).

For the purpose of enlightening the reader and providing him with the minimum necessary understanding of the customizable luminosity display function, FIG. 6 shows one simple example of a "Shadow Man" image.

FIG. 6A shows geometrically what can be seen in the image, and FIG. 6B is a functional relationship between the L_HDR and L_SDR luminosities. The image shows a dark space station (DRKSPST) that robot 602 is moving through. At a certain time in the image presentation, it encounters shadow man 601, which is color-coded as a set of very bright HDR pixels with little difference in luminosity between different pixels. , which make up the body of a human-shadow. This is because he is outside a window in a brightly lit environment with a hazy atmosphere. The fog adds a component to the luminosities emanating from the shadow man's body (for example, his clothes), forming the final luminosities in the direction of the viewer in the HDR image, for example, L_pants = 20 nits + L_fog = 4500 nits = 4520 nits, L_shirt = 50 nits + L_fog = 4800 nits = 4850 nits etc. The problem with using the coarse luminosity mapping function with too little slope for the brightest pixels is that the shadow person can become low contrast and hard to see in images with a lower dynamic range, such as an SDR image. The solution is to set the F_L_CU function so that it locally has a larger slope in the HDR luminosity input region of 4500-5000 nits, which would result in a larger RS subrange of SDR luminosities for the shadow man, making him and his details, for example, a tie, which it wears, more visible in the fog even in the SDR image. It can be understood that there are many more situations in which it may be advantageous to also have some additional recalibration control capability, and not just the coarse mapping function F_C.

Returning to FIG. 4, after determining the appropriate (uniform visual representation) SDR luminances, the linearizer 406 converts them into (normalized) SDR luminances Ls. For this reason, it applies an inversion to Equation 1 above, but since the SDR luminosity is to be obtained this time, the RHO corresponding to PB_C_S = 100 nits (which is entered in block 406) is used rather than the 5000 nits that was used for the perceptual uniformization at the beginning of the luminosity processing sequence.

Colors are of course non-one-dimensional (unless we are talking about achromatic grayscale images), which makes the dynamic range conversion and encoding much more difficult, but in any case, this requires a parallel processing channel for the Cb and Cr chrominances of the pixels to get more suitable corresponding SDR chromaticities, or in fact, as shown in FIG. 4, ultimately suitable RGB SDR colors as output color components Rs, Gs and Bs.

The ETSI2 chromatic processing channel 450 performs the following processing (which is briefly explained again to the extent necessary). The input pixel chromaticities Cb and Cr are similarly multiplied by the F_C[Y] value by the multiplier 452, resulting in the output chromaticities Cb*, Cr*. The challenge is to always get the right output chromaticities, taking into account the existing many difficulties: irregularly shaped color gamut for realizable colors (see explanation in FIG. 5), non-linearity of the mathematical model and, in other cases, the human visual system, etc. . In addition, as will be shown below in the embodiments of the present invention, there are more and more desires in the market, leading to even more complex HDR processing systems.

ETSI2 uses a saturation processing determiner 451 that can load, for example, a LUT that determines the output values to be sent to the multiplier, depending on what luminance Y value the input pixel had. Again, it is up to the content creator to define/optimize the shape of this saturation multiplier vs. luminance function - at least to the extent necessary, because, as will be seen later, an ingenious mathematical color model is sometimes required to define this LUT F_C [Y].

Matrix application block 453 simply converts the Cb, Cr color characteristics to the corresponding normalized RGB representation (the mathematical model for this operation is not of interest within the scope of this application, and the interested reader can find it in ETSI2 in combination with ETSI1).

It is possible to determine the actual RGB triplet by multiplying the normalized R/Lh values and other "no HDR luminosity" values by the normalized Ls value calculated in the luminosity processing channel 401 . It should be noted that the resulting values of RN, GN and BN are in fact still normalized luminosities, and not absolute SDR luminosities (Rs, etc.), while they are "SDR-correct" normalized luminosities, because they now take into account that , what luminosity the SDR colors (Ls) received.

In order for the reader to quickly understand this perhaps initially slightly complex concept for someone who is not a specialist in the field of colorimetry, FIG. 5 explains what happens in normalized (universal, i.e. SDR and HDR gamuts overlap well when normalized as described above, but of course HDR colors must be shifted so that they become suitable SDR colors, even if the conversion is not was very reasonable and optimized for the needs of the current image of the HDR scene, but consisted simply of equating the SDR absolute luminance to the HDR input absolute luminance) YCbCr color gamut.

The net luminosity conversion will be in the vertical direction, so usually the HDR luminosity or its corresponding Y luminosity (i.e. ColHDR) is moved up to the optimal new position (ColSDR) because to map HDR luminosity to SDR luminosity the F_L curve on the normalized axes plot will always be above the diagonal (i.e. the input normalized luminosities or HDR luminances at a particular x coordinate also have as the y coordinate the height of the diagonal at the x coordinate position, and the normalized output luminosity of the SDR function, which always lies above that diagonal, therefore, always will produce a higher normalized output value). Which actual (absolute) SDR luminosity corresponds to this normalized luminance Y value is determined by first applying the EOTF to the normalized luminosity (which was done by block 406 because the processed luminances from Y'HP through Y'GL were determined by applying the appropriate EOTF in the equation 1), and these normalized luminosities are simply multiplied by 100 by a factor of 455 (for example, 0.7*100 = 70 nits). Those. the reader can now see that, using these principles, one can determine everything from the input color of an HDR image, specifically its PQ-specified luminosity Y (e.g. stored on a Blu-ray HDR disc), all the way to the absolute SDR luminosity of the corresponding pixel , which must be displayed on the SDR display to show the optimal corresponding SDR image for the input HDR image (and the resulting decoding of the SDR image from the received HDR image).

Based on the foregoing, the reader now understands the basic starting point of HDR encoding, at least in accordance with the Applicant's coding philosophy as standardized in ETSI. For most consumers, choosing ETSI1 or ETSI2 (and then whatever happens technically) will be sufficient for their purposes, ie. supplying their market with beautiful HDR images (of course, they will still need to create these beautiful HDR images based on the good shape definition of at least the F_C function and preferably also the F_L_CU function, or at least when these functions are not manually optimized, according to their own artistic wishes, acquire and use the applicant's automatic tool, which automatically generates a fairly beautiful look for each type of HDR image, as well as subsequent forms of codec functions). For example, consumers who will go for a complete upgrade to a promising high quality universal HDR system can deploy an ETSI2 system, and market participants who value either their SDR images more or SDR consumers can deploy their HDR system as an ETSI1 system ( this may also be subject to different discussions depending on who is involved in the HDR video processing sequence, e.g. content creator or cable TV system operator, and transcoding may be involved, etc.).

However, there is another market need or market offering for customers who don't want to deploy ETSI1 or ETSI2 exactly to the standard. It would make sense that in the event that someone decides to transmit HDR images as the only images representing the full range of images needed for all the various displays with PB_D, that someone transmits the main HDR images themselves (for example, with PB_C 5000 nits) not only because these images are already available, but also due to the best quality representation of HDR scene images (in fact, they are the "treasure" of the content creator, i.e. images that he specifically created and approved, and is also often the starting point of a film with a creative vision; or the only one he actively created, in the case where the rest of the recalibration works automatically according to the chosen technology). However, especially in the coming years, there will be situations in the market where a different complementary approach may come in handy. Unfortunately, not all televisions available on the market (or video decoding or processing devices in general) that are not primitive legacy SDR displays (i.e. unable to perform all the mathematical calculations associated with HDR decoding, display adaptation, etc.) .d.) will always automatically be TVs with ETSI2 (or ETSI1) functionality. There are a number of TVs on the market that take a completely different approach to HDR encoding and display, such as following the recently standardized hybrid log gamut approach. Or, for example, some TVs can only decode HDR images encoded using PQ luminances, but nothing else. Perhaps some TVs can only use this approach, so probably the best they can do is not process incoming ETSI2 HDR video at all. Similarly, there may be some TVs on the market that do not follow any standard philosophy, at least in terms of display adaptation, i.e. re-grading, for example, a received image with 2000 nits, for example, into an image with 900 nits for a display with PB_D 900 nits. Such a TV would need decoding capability to understand what pixel colors and, in particular, luminosities are contained in the received image, but it could use its own heuristic algorithm (for tone mapping) to determine how to create a 900 nit image. The disadvantage, at least from the point of view of a content creator who would like every consumer to be able to watch his film in the same good quality as he originally created it, is that such variability will create a high degree of uncertainty as to what a specific brand TV will do with any accepted HDR image. For example, a simple rethinking of HDR image display in the recent past has resulted in an absolute reproduction of HDR image luminosities. It means that all HDR image luminances up to 900 nits are displayed at exactly the same luminosity as encoded in the image, but all higher luminosities are limited to the display's maximum possible white color (i.e. PB_D). For an example image such as the space station of FIG. 7, this could mean that some parts of the Earth are cut off to an unsightly white spot (where the Sun eclipses the Earth from the right). While this TV will still be a great HDR TV to some extent, because it will show in good quality the vibrant blues of most of the Earth as seen through the space station's top viewing portal, contrasting beautifully with the dark interior, but at least one part of the image will look ugly (and some other scenes may have much more severe defects, at least on some TVs, such as clipping of every detail of the image outside the cave of FIG. 1 or the bazaar, etc.). Using another simplified rethinking of tone mapping, such as linear compression of luminosities like a white-on-white strategy, can lead to several other problems. In private, although such a system could function and produce some kind of HDR image for the end viewer (for example, in our ETSI2 system, such a TV could use only the PQ function of block 401, but ignore all other metadata of the luminance display function and subsequent serial displays 402, 403, luminosity, etc., which in ESTI2 implement the display adaptation functionality), the results will be neither better visual quality nor - probably worse - predictable.

This has led to a new coding topology based on a second type of HDR image in addition to the main HDR image, namely the so-called intermediate dynamic range (IDR) image, which was first described in WO2016020189. The advantage is that one can define a secondary HDR picture (an IDR encoded picture that will be sent to receivers instead of the main HDR picture in the classic ETSI2 codec philosophy) with a PB_C that is in the range of many current TVs (e.g. 1000 nits or 750 nits; though you could also use 500 nits with the same technique, or maybe even PB_IDR with 400 nits). But it's still possible to make any desired primary HDR with PB_MHDR, as long as it suits artistic preferences or really technical constraints (e.g. an available monitor for grading). The idea is that whatever display rethinking technique (including tone mapping) is used in the TV should be smooth in the sense that the processing should not deviate too much from the received image if PB_D is close to PB_IDR, i.e. the peak brightness of the received IDR image. For example, even a TV set that is so primitive that it simply cuts off all pixel luminosities above PB_D should not cut off too much (for example, not all of the Earth or Sun outside the cave in the image). And the content creator takes back some control, because even if, on the one hand, he wants to create beautiful super-bright areas of the image, for example, with an average of approximately 4000 nits, on the main image with a PB_C_H of 5000 nits, he can control the way he re graduates these areas in the IDR image so that they are, for example, sufficiently below 1000 nits, so that even a primitive 800 nit TV should only cut off the brightest and visually least destructive pixels, for example, only the sun's rays in the space station example of FIG. . 7. Thus, some new technology is needed to provide this new approach.

FIG. 7 shows how the codec philosophy of WO 2016020189 provides an approach to channel adaptation (since the image transmitted over the channel is an IDR image, it can be said that a particular communication channel is configured to send, for example, images with a PB_CH of 1000 nits). The example is again chosen to be interesting for explaining some of the basic concepts. One thing to be aware of though, it can be helpful if all images with different PB_Cs in the range are exactly what, or at least very close to, what a content creator would do if they graduated each one individually and without restriction in any technical system, this is not necessarily always the case, especially for an IDR image. There may be some relaxation here (on the other hand, there may also be disputes as to when and why a particular image grading of category X of an HDR scene compared to category Y is optimal, and what deviation seems to be sufficient deviation; for example, one can imagine, that the pixel luminosity of a street light is less critical than the pixel luminance of a face, especially if it is assumed to be half-hidden in darkness, and also because in real life any street light may well be a little brighter or less bright anyway).

WO 2016020189 discloses a means for defining functions (different functions) using an IDR image as a midpoint, i. up towards the main HDR image to be reconstructed from the IDR image received by the receivers, and down to perform display adaptation for any MDR display with PB_D<PB_IDR. With such a technology, the main HDR range could be chosen to always be fixed as the PB_C range of 10000 nits, which is the range associated with the PQ function.

It can be seen that again different considerations can be used on how to transform the different possible luminosities, which may differ significantly on the left and right sides from the selected IDR image. This is because something else might actually be conceptually done. On the left, a secondary ("smaller") HDR image is formed from the main HDR image. So one consideration might be that this IDR image should be "as good" (despite a lower PB_IDR) as the main HDR image (and how then to elegantly resolve this seeming contradiction?). On the right, there is compression towards even lower PB_MDR (which can be significant for some high degree of complexity, implying, among other things, the presence of many critical objects distributed over the entire luminosity range and images with high PB_C_H), i.e., apparently , there is another task of forming an image adapted for the display. Thus, one can imagine that this could lead to a (completely) different technical processing, in particular, luminance display functions with different shapes/configurations, given our idea of image + luminance display.

In this example, the luminosities of a dark space station can be displayed on any acceptable TV (at least in theory) as long as they are darker than 60 nits. But first, the brighter pixels must be compressed quite neatly into an IDR image, and then the less compression was done in the first part, the more compression needs to be done with respect to the SDR image. Again, there could be different criteria for the approximate two bright objects: a bright blue Earth versus a much brighter but almost colorless Sun and its rays. As shown by the luminosity subranges, respectively, in the main range (BE) of the HDR image luminosities and the range (Be2) of the IDR luminosities for a bright earth object, ideally this content creator may wish that the maximum brightness of the Earth never exceeds, for example, 750 nits, regardless of the PB_C characteristic of any image or display (because otherwise the Earth may start to glow too much and look unrealistic). However, how the luminosities of the Sun should look depends on several factors, not only artistic preference, but also the amount of luminosity left to encode a solar object above 750 nits in the selected IDR image (with PB_IDR 800 nits) (of course, in some situations, the sender content may choose another higher PB_IDR value, but in this case it was assumed that any device connected on the receiving side to the communication channel always expects a PB_IDR of 800 nits for any video content, be it a Hollywood movie or a news program). The final selected luminosity mapping function F_H2h to create the IDR image luminosities from the main HDR image luminosities for all of these brightest pixels is shown as a subset by two arrows: it was decided to define an overall squeezing action for the two objects together, which also slightly reduces the luminosities of the least bright earth objects. This is an example of a situation where the content creator's ideal desired regrading is not 100% perfect (because it may be due to some other technical difficulty), but the IDR image is adequate for most people. It doesn't really matter much if the Earth's pixels are only slightly darker in the IDR image, and this might be expected even for a lower quality HDR image. But the important point is that this IDR image can still meet all the requirements of the original ETSI2 philosophy (and at the same time, with this additional codec step, the requirement is also fulfilled that primitive displays with a PB_D of approximately 800 nits cannot degrade too much received IDR image before displaying it): all MDR images up to the main SDR image can still be generated by the receivers using the luminance transform functions on the right side, at the request of the content creator, and (even with the bright pixels of the earth object darkened) the main HDR -an image with a PB_C of 2000 nits or a PB_C of 10000 nits can be reconstructed by inverting the F_H2h function (which itself can also be optimized for each image or set of consecutive images encoding a particular movie frame, according to its technical and/or artistic needs).

Two documents worth discussing due to their irrelevance (because different technical aspects should not be confused) rather than their importance (but due to potential confusion they deserve discussion) are the following: US 20160307602 and EP2689392 (also published as WO 2012127401) , which both refer to so-called "display optimization" and not to the structure of the video coding paradigm. This main difference for those skilled in the art is illustrated with the help of FIG. 23, which shows a typical example of the entire video processing sequence. It is assumed that the content creation side 2301 is capturing an HDR scene with the camera 2302 in real time (or a previously recorded HDR scene). The human grading specialist (or colorist) determines, for example, among other things, the basic grading of the captured HDR scene (i.e. the relative position of the luminosities of the pixels of the various image objects in the dynamic range of luminosities for the main HDR image, which ends with the maximum representable value, for example, PB_C_H50 = 5000 nits, and starts with some small black value, for example, MB_C_H50 = 0.001 nits, which in this discussion can be considered equal to zero: for example, for a space station, it changes by image processing the data captured by the source camera so that the pixels The suns had 4500 nits, the bright blue color of the Earth corresponded to, for example, 200 nits, etc.). Secondly, the calibrator in our approach usually wants to specify at least one luminance mapping function (in practice, such a luminance mapping function can have a different shape for sequential HDR video images, and our ETSI standards have even explained how this can be technically convenient enough to specify multiple functions even for a single video snapshot, but these additional complexities are not needed to explain the current inventive contribution to the art), which will typically be a function specifying how PB_C_H50 normalized 5000 nit luminosities of the main HDR must be re-calibrated to normalized to 100 nits LDR luminosity, i.e. FL_50t1.

The third important aspect then is the coding technique for coding the main HDR image to be transmitted to one or more receivers (using at least one coding technique). At the beginning of HDR video research, and accordingly in the simpler versions standardized by the applicant, this would be a relatively simple encoding, such as for example a 100 nit LDR image, which then has excellent backward compatibility so that it can be shown directly in good general terms on older LDR TVs that do not have the capability or are not configured for HDR processing. The coding approach disclosed in WO2016020189 and the principles of the present invention are an example of a more advanced second generation approach that is more complex but may satisfy additional desires in some specific technical ecosystems for HDR video transmission and processing. The calibration, for example, performed by a human calibration specialist 2304 (in case it is not automated, as is often the case in real broadcast programs) is carried out on the calibration device 2303 (which usually contains several tools for changing pixel luminosities, however, for the purposes of this explanation, it can be assumed that it consists of elements of providing a user interface for specifying the form FL_50t1 and passing this form of the function (for example, in the form of metadata containing several parameters that determine the form of the function)).

Although the video encoder 2305 (which will perform, without any restrictions, assuming that its input HDR main image is a set of luminances for all pixels, all techniques for implementing the actual encoding of the HDR main image, i.e., for example, to obtain 8 -bit, 10-bit, or 12-bit pixel matrix of typical YCbCr pixel color triplets, together with metadata describing all additional information, such as luminance mapping functions given depending on which coding technique has been chosen), could in principle be included part of the 2303 calibration device, it was shown as a plug-in accessory. This is a simplification for the reader, sufficient to explain the present invention, which summarizes various practical options, for example, capture (and possibly calibration) occurs at an external mobile television station, and encoding possibly occurs at some intermediate relay station communications, for example, after local commercials have been embedded in the signal, etc. (harmonization of different image content may also be involved in this connection, but the details of this need not be explained in detail). It is important to understand what happens on the production side (see, for example, the difference between delivery and distribution), which can be defined as the formal completion when the finally encoded video signal is transmitted to some consumer, for example, using a satellite dish 2306 and a communication satellite 2340 (or any equivalent video transmission channel, such as over the Internet, etc.).

The receiving side typically has consumer devices in the end user's home, such as, for example, a satellite set-top box or any equivalent decoding and finishing device 2352 connected on the input side to the local satellite dish 2351 and on the output side to the HDR display 2353 , which can have different display capabilities, for example with a PB_D of 1000 nits, or 700 nits, or 2500 nits. While it may be that the set top box only needs to do the decoding again to obtain the luminance values to be displayed by the decoder 2381, which then performs the inverse of the encoder operations, this will generally only be useful in a limited number of situations. Typically, the display optimizer 2382 performs a display optimization process which consists of next changing the absolute appropriately normalized luminance distribution (either the received LDR picture, for example, or the decoded HDR main picture, for example, with 5000 nits), since the main picture may have been encoded , for example, for PB_C_H50 5000 nits, i.e. potentially contain pixels with a luminosity of 2000 nits, although a particular consumer's HDR display may, for example, only display up to 700 nits (its whitest display color).

Thus, on the one hand, there are significant technical differences between the equipment (and its technological design philosophy, etc.) on both sides, for example, that the creating / encoding / transmitting side will only have a video encoder 2370 for encoding main HDR video (MsterHDR) as some channel-encoded IDR image with intermediate dynamic range, while the receiving side can also display and optimize the reconstructed HDR image (RecHDR) with 5000 nits on an optimized display, such as an ImDA image with PB_C 700 nits, optimal for connected display with PB_D 700 nits. The technical difference between the two can be seen in that display optimization can be done as (optional) post-processing, while encoding/decoding is purely an image restoration technique, usually without any display optimization instructions. Devices (and control processes, etc.) on both sides are also usually handled by completely different skilled people. Content creation devices may be designed by professional video equipment manufacturers and controlled by broadcast engineers, etc. Set-top boxes and televisions are usually made by consumer electronics manufacturers, such as those of Asian origin.

US 20160307602 is Applicant's first patent document on display optimization. Briefly, the idea is that a content creator can propose ground rules and algorithms for a way to implement regrading for different (at least two) modes that may exist in an image (with a mode being a concept that is both a set of pixels in an image). and the necessary way to implement re-grading for these pixels in the presence of different displays with different dynamic range). Although this initially made it possible to establish a link between the wishes of the content creators and the actual display at the end-use location, in reality it is at this end-use location that a controlled way of implementing display adaptation should take place. And ideally, a set-top box or TV manufacturer, if at least display adaptation occurs on that TV, will largely follow what the content creator has determined is a good way to implement for various mode objects on video images (for example, someone coming out of the dark area doesn't get too visible or too invisible on displays of any capability, even LDR displays with PB_D 100 nits), because that's exactly what this content requires, rather than blindly doing anything on one's own. But this is clearly the FINAL way of implementation to be done on the consumer side, and is completely independent of how the video transmission technology provider wants to design or, accordingly, any developer wants to deploy any particular video codec philosophy. It should also not be confused with any ad hoc tone mapping technology, if only because such a mapping would typically not be reversible, a property that encoding with a lower dynamic range IDR image should have.

WO 2012127401 also describes early HDR era technology for defining a way to implement display optimization, which can be done with various implementations of the DATGRAD structure that defines how different image content should be re-graded for different luminosity dynamic range capabilities. This DATGRAD structure will be used to generate any required Medium Dynamic Range (MDR) image for a PB_D MDR display between the core HDR PB_C encoded peak luminance (i.e. PB_C_H50 in the current notation) and a PB_C LDR of 100 nits as per the minimum required replay specification. graduations (see page 16 in WO2012127401). The acquisition of an MDR image is optimally performed not only using the image re-grading requirements encoded in the DATGRAD data structure, but also using specific aspects of viewing from the display side, such as, for example, the brightness of the viewing environment or the final settings of the viewer's preferences (see page .5 in WO2012127401).

It should be clear that, initially based on such ideas, but without using any additional rather specific values, a person skilled in the art will not succeed in redesigning codecs.

In addition to the differences in the creation of specific functions compared to what already exists in the state of the art, but more importantly, the most innovative codec structure/paradigm, it is also necessary to mention that the actual transmission of the second PB_C value (the highest value of the main content along with the lower from the actually transmitted IDR picture) is also different from the (optional) type characterization that can be used according to WO2016020189. Apart from the fact that these two concepts are not literally the same, the numbered object can have a different role, especially when looking at the details of this paradigm in comparison with the ideas of the present invention. Such a characterization in WO2016020189 can be useful in the case, for example, when there are two luminosity mapping functions for re-grading upwards. Then it can be useful to choose which one should get something like an exact reconstruction of the main HDR image on the content creation side. But such information is neither strictly necessary nor mandatory for use in the prior art. You can use the upscaling function extracted from the main HDR image to get a 4000 or 6000 nit reconstructed image instead of a 5000 nit reconstructed image. Since the intermediate image has two sides, the downscaling function is usually one that contains significant image content (especially content that should be displayed brightly enough and faithfully on all displays with PB_D), but the upscaling function will be particularly different in this way to determine the nature of the re-grading for very bright objects (eg car headlights, sunbeams, etc.). These are typical HDR objects/effects, which, however, are not reproduced correctly anyway, since it is in the upper region of the luminosity range that displays with different functionality differ the most. Therefore, creating a reconstructed image with PB_C 4000 nits from, for example, an IDR image with 600 nits may result in some car headlights being slightly dark compared to their ideal luminance value (even though such a value could be represented in the luminosity range 4000 nits), but it will still be a pretty nice HDR image if you just apply e.g. a polylinear normalized re-grading function on the [0-1]/[0-1] axis system, where the horizontal axis is PB_C normalized luminosities IDR images, and the vertical axis corresponds to what is chosen as the calculated PB_C of the reconstructed HDR image, which is not too far from the (perhaps unknown, but only estimated) PB_C of the main HDR. In our present technology, essentially the luminosity value PB_C_H50 is actively transmitted because it is also used in the decoder algorithm.

The inventors of this patent application wished to limit the general IDR approach in a number of ways, in particular around the ETSI2 coding philosophy and systems already developed today (ICs, TVs, set-top boxes).

DISCLOSURE OF THE INVENTION

The inventors paid attention to a number of technical features. On the one hand, they would like their system to be compatible with the ETSI2 decoders already deployed. Therefore, if, for example, an IDR image with 1500 nits was transmitted (and the ETSI2 decoder knows nothing about the principles of constructing IDR, as a result of which it assumes that this was just the original main HDR image of an HDR scene), then along with it the luminance mapping function F_L_IDR (and all other color mapping information according to ETSI2), which is the F_I2s function of FIG. 7 and which performs the correct display adaptation. Therefore, whether additional IDR technology was used or not, an ETSI2 decoder, also known as SLHDR2, should be able to normally produce all MDR pictures up to and including the SDR picture, and they should (ideally) look like that's what the content creator wanted. Ideally, any new decoder according to the present principles, which we will call the SLHDR2PLUS decoder, should also exactly or at least approximately provide the same appearance for all images from IDR to SDR (i.e. at least one of IDR- and SDR -images should preferably not deviate too much from the MDR image, which might be the result that the grading specialist or, more generally, the content creator would like or at least agree to see). On the other hand, a very important criterion is that the underlying HDR can be almost perfectly reconstructed (albeit with perhaps a few minor rounding errors that are introduced, for example, when applying the Discrete Cosine Transform (DCT) on an IDR image in the compression phase MPEG for the purpose of transmission, which will result in very small acceptable errors in the restoration of the main HDR at the receiving end). Of course, some systems can be used with some relaxation in terms of the quality of the restoration of the main HDR (some content providers may consider the IDR image to be more important, at least if some temporal aspect is applied, for example, in broadcast or even in a single broadcast to a limited audience, and not, for example, for the purpose of storage, for example, on Blu-ray discs), but in general, at least one main party involved in the video processing sequence will find it important to be able to perfectly restore the main HDR image ( which is different from blindly trying to create a slightly higher dynamic range look starting with IDR).

Finally, although it can be seen that in order to perform service without the need to redesign and redeploy ETSI2 decoders, it is necessary to share the F_I2s function (i.e., it is preferable to reuse the (de)coding scheme of the SLHDR2 system to the maximum, but at least the video signals , including their luminosity and color mapping functions, must still conform to the standardized definition so that, among other things, legacy SLHDR2 systems know what they are receiving, except for some metadata that they do not require and can be ignored), the desire of a content grading specialist can usually be to define its own luminosity (and color) mapping functions between the main HDR he created and some corresponding version of the SDR (i.e. PB_C 100 nits) (which he, for example, could create using the system shown in FIG. 4 ). Thus, the F_Mt1 function (see FIG. 10) is neither the F_H2h function nor the F_I2s function of FIG. 7, but rather a function that covers all re-grading efforts between main HDR and main SDR (i.e., this F_Mt1 determines the need for re-grading an HDR scene image between the most different representations of the dynamic range of said HDR scene). Thus, a technique is needed to elegantly correlate these two situations, in particular within or around the philosophy of the ETSI2 paradigm (for example, the SLHDR2PLUS decoding method results in the same types of MDR images as an ETSI2 receiver display adapting to the received IDR- images and functions D_I2s; for each point in time, one or more functions partially recalibrate between the input dynamic range image for that point in time and the desired output dynamic range image for that point in time).

As will be shown below, this can be implemented in several ways according to the different views of various inventors, depending on which type of system is desired, as well as which desired constraint condition should be relaxed more and which should be relaxed less (also considering specific practical technical factors , for example, how many cycles or transistors would be required for different choices, which may make some choices more desirable than others, but we don't need to go into those details within the scope of this patent application).

However, there are some basic fundamental principles that are used in all approaches. At least two decision methods can be summarized with FIG. 8. The receivers receive an IDR image (which must somehow be restored to an HDR Mster image, in this example with PB_C = 4000 nits) and they also have the F_I2s feature. But they must somehow find the function F_ for each IDR luminosity?? to calculate the required corresponding normalized and therefore absolute HDR core luminosity (which will exactly restore that HDR core as it was originally, but whose image(s) were never transmitted(s)). One can either create a new SLHDR2PLUS decoder color conversion system that can define the desired function (but which, at least according to its processing integrated circuits (ICs) or software core, remains the same as shown in FIG. 4, with the presence of a processing channel luminosity with its features (at least some of the sub-blocks are used) and chromatic processing channel), or you can also try to apply all the ingenuity in relation to the encoder so that you can use the standard approach to color conversion in ETSI2 encoding in the native form (if not to think that its novelty will lie in its new software to restore the original HDR from 4000 nits by receiving the metadata of this second, desired peak brightness PB_C_H50, which usually involves loading the corresponding LUTs of full or partial processing of luminance and chrominance), regardless on whether it is configured for extrapolation, and not to adapt the display of images with a smaller PB_D value than the PB_IDR value. Both approaches and their implementations will require some common new technical components and fall under the overall new SLHDR2PLUS coding philosophy.

Based on shown in FIG. 9 of the basic design of the SLHDR2PLUS encoder 900 (which will be explained in more detail below), which is a variation of the general IDR encoder, you can see the difference with conventional HDR coding, in particular ETSI2 HDR video coding: two peak brightnesses are now jointly encoded in the metadata, and namely, firstly, the “normal” peak brightness (which should be called the peak brightness of the PB_CH channel, i.e. the peak brightness of the received IDR image, whatever technology is used for it, i.e. whatever the peak brightness level , which looks optimal for the creator, owner or transcoder of the content, and whatever mathematical technical method is used to calculate the IDR pixel luminances), indicating what is the maximum encoded luminosity of the transmitted and later received video, i.e. IDR images (IDR images) [this is what a regular ETSI2 decoder would see, ignoring all other new approaches]. But secondly, there is now also the peak brightness of the original main HDR image(s), namely the peak brightness of PB_C_H50 content (eg 5000 nits). The second PB_C_H50 could be determined in some embodiments many months before the creation of the IDR image, when the main HDR images were created (for example, based on camera action, in a computer, etc.), and PB_CH could be set to as an external input to the encoder 900 through many different possible means during channel encoding (for example, a cable TV operator may have a fixed value set in memory that may be updated annually to reflect the current average condition of their customer's HDR displays, or an optimized PB_CH may also be calculated considering some luminosity or other image characteristics for at least one video image or associated metadata, even in theory metadata specifically provided to control subsequent IDR transcoding, etc.). (Single) peak luminance co-transmission is useful for HDR coding, at least with respect to the ETSI2 system (which had only what was once thought to be the only thing needed, namely the peak luminance of received images), but with full transparency use for legacy ETSI2 decoders which are said to have a PB_CH (otherwise they would not be able to do their normal display adaptation calculations). PB_C_H50, on the other hand, is necessary in order to be able to calculate the function F_ ?? according to FIG. 8 and finally, using this function, the desired reconstructed HDR main image from the received IDR image.

So this immediately shows the difference in the data stream of legacy ETSI2 video encoding, and legacy ETSI2 decoders will not know this extra metadata and will simply ignore it, because ETSI2 decoders are not required to define any pictures with a PB_C higher than PB_C_H, which they accept in metadata indicating the maximum possible luminosity in the image they receive (because according to the true ETSI2 philosophy, the received image is always the best quality image; in fact, it would be the main HDR image created by the content creator). But, as shown in FIG. 11, the SLHDR2PLUS generic decoder not only receives and reads the PB_C_H50 value, but also uses it to reconstruct the REC_M_HDR image, which is an almost perfect reconstruction of the main HDR image created by the content creator (in reality, such a decoder will use the PB_C_H50 value to calculate the necessary function ( functions) F_?? from the received function(s) F_I2sCI). This decoder may also successfully output pictures with a lower PB_C, such as an MDR_300 picture with a PB_C of 400 nits, but one may choose to use the standard ETSI2 compute core for such pictures with a lower PB_C than PB_CH, or one may perform the computation in an embodiment of a new compute SLHDR2PLUS core (but to accurately restore images with a higher PB rather than PB_CH, new developments are definitely needed, since this cannot be done easily with ETSI2 technology).

Thus, the tasks to be solved by the new technology are realized by a high dynamic range video encoder (900) capable of being received via an input (920) for a high dynamic range input image (MsterHDR) image that has a first maximum luminosity (PB_C_H50) of pixels, for which the encoder has a first input (922) for metadata, and capable of being received via a second input (921) for metadata of the main luminance mapping function (FL_50t1), wherein the luminance mapping function determines the relationship between the normalized luminances of the input image with extended dynamic range and normalized brightnesses of the corresponding image (Im_LDR) with a narrow dynamic range, having a maximum luminance of LDR pixels, preferably equal to 100 nits, characterized in that it additionally contains a third input (923) for metadata for receiving the second maximum luminance (PB_CH) of pixels , while the encoder is additionally different in that it contains:

- an HDR function generating unit (901) configured to apply a standardized algorithm for converting the main luminance mapping function (FL_50t1) into an adapted luminance mapping function (F_H2hCI) that relates the normalized luminances of an extended dynamic range input image to the normalized image luminosities (IDRs) ) with an intermediate dynamic range, which is characterized by a maximum possible luminance equal to the second maximum luminance (PB_CH) of the pixels;

- an IDR image calculation block (902), configured to apply an adapted function (F_H2hCI) of mapping brightness to pixel brightnesses of the input image (MsterHDR) with an extended dynamic range to obtain the pixel brightnesses of the image (IDR) with an intermediate dynamic range, which is output by this block ; And

- an IDR mapping function generator (903) configured to obtain, based on the main luminance mapping function (FL_50t1) and the adapted luminance mapping function (F_H2hCI), of the channel luminance mapping function (F_I2sCI) that outputs the corresponding normalized image luminances (Im_LDR) ) with a narrow dynamic range, when the corresponding normalized image luminances (IDR) with an intermediate dynamic range are given as input, which, in turn, correspond to the corresponding luminances of the input image (MsterHDR) with an extended dynamic range; while the encoder is additionally different in that it contains:

- an output (930) for an image output image (IDR) with an intermediate dynamic range;

- the first output (932) for metadata to output the second maximum luminosity (PB_CH) pixels;

- the second output (931) for metadata to output the function (F_I2sCI) display the brightness of the channel and

- the third output (933) for metadata to output the first maximum luminosity (PB_C_H50) pixels.

First, it should be noted that while it is purely theoretically shown that there is a separate input for each required data element of the present encoder, one skilled in the art will appreciate that in practice one or more of these inputs (and similarly for the outputs) may be the same, depending on which video input technology can be considered (for example, some older HDMI image inputs cannot handle dynamically changing - i.e. potentially different for each video image in time sequence - basic functions (FL_50t1) of displaying brightness, in which case this data can be transmitted synchronously, for example, via a Wi-Fi connection, etc.). How various inputs are entered may also depend on where they are generated, i.e. in which or which other system the encoder is connected (which may depend on whether real-time encoding is required at the same time as the camera captures the event, or later encoding for some video transmission system, such as a cable television system, which at any time time takes all the data from the original content creator in order to optimally distribute it, taking into account the limitations or objectives of this particular cable system, etc.).

It can be assumed, without conceivable limitations, that the MsterHDR image was recently graded by a human color grading specialist using color grading software on a computer, and that he specified the FL_50t1 functions that define the corresponding image with a narrower dynamic range, typically SDR- image with 100 nits (although currently the lower end of the spectrum of re-graded images, according to the model convention, corresponds to the image with PB_C 100 nits, so it is unlikely to change; such the lowest of the three images, i.e. the LDR image, may have a maximum LDR luminosity in future embodiments that does not correspond exactly to 100 nits, but possibly a number k times 100 nits, for example, it is preferable that k be up to 3x, i.e. the maximum LDR luminance in this embodiment implementation of the present system is 300 nits), which corresponds to the image MsterHDR (whereby the SDR image preferably looks as similar as possible to the MsterHDR image, given the much lower dynamic range of luminances), which usually at least adequately conveys the desired views for visual and optimal reproduction of, for example, the necessary movie story (also various applications videos may have different objectives, such as different color criteria, possibly with different technical limitations regarding FL_50t1 functions).

The value of PB_CH is somewhat different from other metadata in that it is actually a setting for intermediate dynamic range coding. So it may or may not come from a calibrator. It may, for example, be a fixed value for a particular video coding system (eg, a satellite broadcast system), which may, for example, be retrieved from a permanent memory connected to or included in the encoder. In Internet delivery, it is also possible that this PB_CH value may be conveyed as something desired by the end user for whom the IDR images are generated. For example, a consumer with a poor quality mobile display might just request an image with a PB_IDR of 500 nits to be computed by a server on the other side of the Internet, such as a video-on-demand company's server, while another consumer might request a version with a PB_IDR of 1000 nits, and in such a case, the requested value PB_CH = PB_IDR will be injected into the encoder.

So while there was the highest quality MsterHDR image on the encoding side (in fact with the highest PB_C), they will not correspond to the image that the receivers (complementing decoders) receive, but rather the IDR image (and they will need to accurately reconstruct MsterHDR image by calculating the REC_M_HDR image). The techniques are best implemented by formulating everything as normalized to brightnesses of 0.0-1.0. In fact, when it comes to the luminance display function, this is in fact also equivalent to the luminance display function (due to the relationship between luminances and their respective luminosities, e.g. usually the luminosities to be displayed), but strictly speaking, from a technical point of view, our calculations are preferable relate to luminance display functions, preferably considering a psycho-visually uniform definition of luminance given by the Philips v-function (see Equations 1 in conjunction with Equation 2).

As explained above, our approach to HDR video processing, in particular encoding not just one or two differently re-graded images with different dynamic range and specific peak brightness, but the entire spectrum of corresponding different DR re-gradeds, is to establish a relationship between different possible normalized brightnesses that the pixels of such at least two linked images can have, for example, 0.2 in image_1, which corresponds to 0.4 in image_2, and so on. This is what defines the brightness mapping functions between the same situation, i.e. re-grading of one variety, and any other chosen situation.

What is meant by a standardized algorithm is that there must be some fixed way of relating the first set of possible functions (which may have many different forms and definitions) to the second set of corresponding functions. In other words, it simply means that in some communication technologies (or even in all of them), the encoder and decoder designer has specified a way that unambiguously specifies how to transform the shape (usually along axes that are normalized to 1.0) of any input function into the form of the output function. There may be many such algorithms, so in principle the codec designer may decide that he may need to pass the sequence number of any such algorithm - for example, agreed algorithm number 3 - to the decoder, etc., but usually this complexity is not necessary, since how our method will work ideally and as simply as possible due to only preliminary agreement on one fixed standardized function transformation algorithm, for example, what is described below in the auxiliary mathematical model.

For a quick understanding by the reader, a simple example of such an algorithm is given below. Assume that the input functions are power functions: power(x_in; P), then the algorithm can obtain the corresponding functions power(x_in; P-1). By inversion, the original features can also be re-inferred when the corresponding features are received (using the +1 algorithm).

One should not be mistaken that the standardized algorithm itself is usually not passed to the receivers, but only the resulting corresponding output functions are passed. That is why it is standardized or pre-agreed, ie. must be set so that the decoder can know what happened on the encoding side. How it was agreed is not so important for understanding the technology being patented. For example, there may be 5 different predefined algorithms, and a cable TV operator may choose to encode with algorithm 3 and provide their customers with set-top boxes appropriately configured to decode with predetermined algorithm 3 (even though the STB may be reconfigured in some cases for a different video connection). eg with algorithm 4; but algorithm changes will not normally be required, although PB_CH changes may be of interest, eg for different cable channels).

Careful attention should also be paid to the fact that it is not usually the corresponding adapted luminance mapping function F_H2hCI that is sent to the receivers, but another, then additionally extracted channel luminance mapping function (F_I2sCI), whereby the decoder also needs to somehow reconsider this double extraction. In fact, the complete regrading mapping is divided into two parts, so if the first part is standardized, the second part is also defined, whereby the inversion of this IDR coding by decoders can be (albeit difficultly) considered as plausibly possible (which makes the structure and correct functioning possible). new codec SLHDR2PLUS).

Let us explain this concept of a standardized algorithm for changing the function depending on the encoded peak brightness in a little more detail with the help of FIG. 24. So, on the left, it is shown that there can be various occurrence events of the FL_50t1 function developed by the calibration specialist or the automatic recalibration function determination tool on the content creation side. For example, FL_50t1_1 could be defined for an HDR scene that has fairly deep blacks with important action going on that needs to be brightened enough to be sufficiently visible on displays with a narrower dynamic range, but the brightest parts are not as bright. significant - such as, for example, street lamps - and can be represented with one almost maximum white luminosity on any display (or on an image calculated for this, i.e. containing exactly those absolute luminosities that should be reproduced on this display, or, more precisely, usually the luminosity codes in which these luminosities are encoded). In contrast, FL_50t1_2 was created for an image or snapshot of sequential images that contains a different type of HDR scene, in which there is both an important object (or more precisely, a mode) with a lower brightness, and an object with a higher brightness, which resulted in specially tuned re-grading curve shapes on either side of the "middle". FL_50t1_3 is another possible input function in the standardized algorithm applied by the HDR function generation block 901, which may, for example, appear in the case of a daytime scene with very bright content (and not too high main PB_C) and some local darker areas such as , such as the entrance to a temple on a street scene in India.

For any of these three situations, and countless other situations, block 901 will define an output function. The property of this algorithm is that this function will have a similar shape, but will be closer to diagonal (because if the original function is a regrading, for example, between an image with PB_C X nits and the corresponding one - similar enough, as far as possible, - an image with PB_C2 Y nits, say a 100 nit image, then re-grading from X to an image with PB_C Z nits, where Z, for example, lies midway between X and Y, would require a similar re-grading, but to a lesser extent; if it was necessary to map X to X, then the identity transformation corresponding to the diagonal would be required).

There are a number of ways in which such a standardized algorithm can be defined to unambiguously obtain the output luminance mapping functions F_H2hCI_1, F_H2hCI_2 and F_H2hCI_3 corresponding to the respective input functions, and the details of this are not really an essential element of the present invention, except for the fact that it is necessary have some standardized algorithm that behaves accordingly. For example, it is usually possible to define some metric (quantifying the dependence of PB_C_CH on the selected encoded maximum PB_C_CH IDR image luminance) that can be used to shift the points y(x) of the input function for any normalized input luminance towards the diagonal in some way (e.g. , mostly evenly or unevenly, etc.). While it is also possible to move vertically, a preferred embodiment that works quite well, as detailed below, involves shifting such function points along paths orthogonal to the diagonal from [0,0] to [1,1].

According to a preferred embodiment, the high dynamic range video encoder (900) is characterized in that the standardized algorithm of the HDR function generation unit (901) applies compression to the diagonal of the main luminance mapping function (FL_50t1) to obtain an adapted luminance mapping function (F_H2hCI), wherein compression includes scaling all output luminance values of the function using a scaling factor that depends on the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels.

There may be variously defined F_L50t1 functions (the para function definition below is one example) and they may be scaled in various ways by a standardized algorithm, but scaling will typically be involved depending on the initial PB_C_H50 and the target PB_CH = PB_IDR. This can be done with different metrics, but over time, the applicant has found it convenient to set the scaling factor based on psycho-visually unified peak brightness values and ratios, sending them through a v-function, i.e. specifying a scaling factor based on the output luminance values of the v-function corresponding to the two peak luminances (and possibly the third PB_C SDR image).

According to a preferred embodiment, the extended dynamic range video encoder (900) comprises a limiter (1804) configured to re-determine the slope of the channel luminance mapping function (F_I2sCI) for a subband of normalized luminances containing the brightest normalized luminance equal to 1.0. This is not required for many implementations, but in particular it is a useful way to handle the particular choice of encoding HG_COD flare gains in para functions that has been standardized in ETSI2, and therefore this particular implementation is useful for ensuring full compatibility with all of this. .

The corresponding mirror technology for the encoder, effectively canceling all encoding processing by being able to re-extract all the necessary information (even if such information was not actually transmitted), is a high dynamic range video decoder (1100) having an image input (1110) for receiving an intermediate dynamic range (IDR) image that has a second maximum luminance (PB_CH) of pixels that is lower by a multiplicative factor, preferably equal to 0.8 or less than the first maximum luminance (PB_C_H50) of the pixels of the main image (MsterHDR) with extended dynamic range, wherein the second maximum luminosity (PB_CH) of the pixels is received via the second input (1112) for metadata, the decoder having the first input (1111) for metadata for receiving a luminance mapping function (F_I2sCI) that specifies a transformation of all possible normalized image luminances (IDR ) from the interval accurate dynamic range to the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range and maximum luminance of LDR pixels, wherein the decoder is characterized in that it has a third input (1113) for metadata for receiving the first maximum luminance (PB_C_H50) of pixels, and the decoder comprises:

- a luminance function determiner (1104) configured to apply a standardized algorithm for converting a luminance mapping function (F_I2sCI) into a luminance mapping decoding function (F_ENCINV_H2I) that determines, as output for any possible input, normalized image pixel luminance (IDR) with an intermediate dynamic range corresponding to the normalized HDR brightness of the main image (MsterHDR) with extended dynamic range, and the standardized algorithm uses the values of the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels; And

- a color converter (1102) configured to sequentially apply to the input normalized image luminances (IDR) with an intermediate dynamic range of the luminance mapping decoding function (F_ENCINV_H2I) to obtain normalized reconstructed luminances (L_RHDR) of pixels of the reconstructed main HDR image (REC_M_HDR); wherein the decoder further has a picture output (1120) for outputting the reconstructed HDR main picture (REC_M_HDR). The maximum LDR luminance is again preferably the standardized SDR luminance of 100 nits, although it is possible to conceive of similarly functioning further options in which the image with the narrowest (i.e., the narrowest) dynamic range (i.e., the maximum luminosity) in the transmitted spectrum of the re-graded image is, for example, an image with 200 nits.

Thus, the MsterHDR image is not actually received as an image, and yet it is uniquely determined by the received data (so although formally this MsterHDR image is the corresponding main image that existed at the corresponding location of the matched encoder, and the decoder only reconstructs an almost identical image REC_M_HDR from the received IDR picture, the various functions actually determine the properties of the MsterHDR picture even at any point in the decoding execution). Different consumers may choose different values for both PB_C_H50 and PB_IDR. The former may be chosen by the content creator for various reasons, such as because they have purchased a 4000 nit graduation monitor, or because they like to give their main content a particular better quality (e.g. create/set everything with a PB_C of at least 10000 nits), or because certain types of images require a certain quality, such as PB_C_H50, at least in the opinion of the creator (for example, a spectacular fireworks show, light show, or pop concert may deserve a higher PB_C_H50 than, for example, a sufficiently evenly lit tennis match or mode news reading).

The value of PB_IDR may be chosen based on various technical considerations, such as evaluating a typical customer of a video communication company, and as already mentioned, the communication provider may often not be the same as the content creator company.

As a general rule, it doesn't make much sense to create re-graded IDR content that differs by less than 20% in PB_C (i.e. by a factor of 0.8, although in principle PB_C values can be closer to each other, for example, with factor of 0.9), but often in most cases there will be a factor of 2 or more between the PB_C values (e.g. a base material with 2000 nits sent at some PB_CH below 1000 nits, for example 800, 700 or 600 nits, but usually above 500 nits ). PB_C_H50 at the decoding execution site is usually similar to other metadata and in particular to the PB_CH value, so it is usually received as metadata associated with video data, such as non-limiting SEI messages or special packets in a video communication protocol, etc. (whether it be a single logical data structure or multiple structures, whichever is best for each standardized or non-standardized video communication protocol; this is a minor detail of the new technology presented). Since the decoder used a standardized algorithm to ultimately obtain the IDR image and its ETSI2 compliant metadata, an appropriate standardized algorithm can be developed for and within the decoder that ultimately determines the necessary luma mapping function F_ENCINV_H2I to restore the pixel luminances of the REC_M_HDR image (regardless of what is then done with this image; its display is a typical application, and, for example, saving to a hard disk recorder is another application).

According to an interesting embodiment, the extended dynamic range video decoder (1100) is characterized in that the standardized algorithm of the luminance function determiner (1104) calculates a scaling factor that depends on the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels. As already mentioned, this can suitably be done in a variety of ways, but psycho-visually unified v-function based scaling factors are very practical for well-controlled HDR processing and satisfy various even critical artistic needs, while also while maintaining control over technological complexity.

According to a useful embodiment, the extended dynamic range video decoder (1100) has a luminance mapping function (F_I2sCI) defined by a luminance mapping that consists of a first line segment having a first slope (SG_gr) for the range of normalized low luminances, a second linear segment having a second slope (HG_gr) for the range of normalized high brightnesses, and a parabolic bar for brightnesses between the specified two ranges. The corresponding mathematical model includes, inter alia, solving a second-order equation to obtain the necessary channel-adapted glare amplification to perform the reconstruction. This is a useful approach to first-order HDR re-grading, and is suitable for markets with less demanding pixel color management, such as real-time television broadcasting (as opposed to, for example, full color management, which is sometimes used, such as blockbuster films). As mentioned below, in some further decoupled embodiments, this may either be a single component specifying the entire F_L50t1 function and all retrieved functions (e.g., functions passed along with the IDR picture: F_I2S), or it may also be a partial definition of this recalibration function. , for example, defining a common recalibration along with a custom function, as shown in FIG. 4.

According to a useful embodiment, the extended dynamic range video decoder (1100) has a color converter (1102) configured to calculate pixel luminances of an average dynamic range image (MDR_300) having a maximum pixel luminosity (PB_MDR) that is not equal to 100 nits. , a first maximum luminance (PB_C_H50) of pixels, and a second maximum luminance (PB_CH) of pixels, wherein the decoder has an image output (1122) for outputting an image (MDR_300) with a medium dynamic range. Although REC_M_HDR image restoration may be all that is required for some devices in some sub-markets (and possibly all sorts of other transformations are applied to the restored image), it would be preferable if, in some embodiments, our SLHDR2PLUS decoder could, in addition to simply restoring the main HDR image also calculate the corresponding images from another PB_C, for example, an MDR image directly displayed on some display from any PB_D. This will also use the mathematical principles of the present invention, for example, as explained using FIG. 16, or in any equivalent way.

According to another useful embodiment, the high dynamic range video decoder (1100) has an output (1121) for metadata for outputting a luminance mapping function (F_L_subsq) that determines, for all normalized luminances, a reconstructed HDR main picture (REC_M_HDR) or, alternatively, an image (MDR_300) with a medium dynamic range corresponding to the luminances of an image having a different maximum pixel luminance, this other maximum pixel luminance being preferably 100 nits or equal to a value higher or lower than the maximum luminance value of the reconstructed HDR main image (REC_M_HDR), respectively, or , alternatively, images (MDR_300) with medium dynamic range. It may happen that the received IDR image is reconstructed as a REC_M_HDR image not for direct display on a primitive control display, but for sending to some system that performs further colorimetric calculations. It is then useful that the decoder of this embodiment can also output a suitable luminance mapping function, namely typically a luminance mapping function that is associated with an output image, such as a REC_M_HDR image (generally associated with the fact that the input normalized luminances of this given function are the normalized brightnesses of the image that is being output together, and the output values of the function are the normalized brightnesses of some main image, which is usually an SDR image since it is standardized to have PB_C = 100 nits, which is usually the lowest quality one could wish for in HDR era; this does not preclude that one may want to apply the ideas of the present invention with PB_C for the output ordinate defining the jointly transmitted function, for example, 80 or 50 nits).

Whatever is stated above for devices (or parts or sets of devices) can be equivalently expressed as signals, memory products containing images such as Blu-ray discs, methods, etc., for example:

as a method for encoding a high dynamic range video for a received input image (MsterHDR) with a high dynamic range that has a first maximum luminosity (PB_C_H50) of pixels, the method including receiving a main luminance mapping function (FL_50t1), wherein the luminance mapping function determines the relationship between the normalized luminances of an input high dynamic range image and the normalized luminances of a corresponding low dynamic range image (Im_LDR) having a maximum LDR pixel luminance preferably a value of 100 nits, the method being characterized in that the encoding further comprises receiving a second maximum luminosity (PB_CH) pixels, and the encoding includes:

- application of a standardized algorithm for converting the main luminance mapping function (FL_50t1) into an adapted luminance mapping function (F_H2hCI) that relates the normalized luminances of an extended dynamic range input image to the normalized image luminances (IDR) with an intermediate dynamic range that differs in the maximum possible luminosity, equal to the second maximum luminosity (PB_CH) of the pixels;

- applying an adapted function (F_H2hCI) of mapping brightness to input image pixel brightnesses (MsterHDR) with extended dynamic range to obtain image pixel brightnesses (IDR) with intermediate dynamic range;

- based on the main luminance mapping function (FL_50t1) and the adapted luminance mapping function (F_H2hCI) of the channel luminance mapping function (F_I2sCI), which determines as output the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range, when as input the corresponding normalized image luminances (IDR) with intermediate dynamic range are used, which, in turn, correspond to the corresponding luminances of the input image (MsterHDR) with extended dynamic range;

- image output (IDR) with an intermediate dynamic range and

- output of the second maximum luminosity (PB_CH) of the pixels, the function (F_I2sCI) of the display of the channel brightness and the first maximum luminosity (PB_C_H50) of the pixels.

- or in the form of a method for decoding video with an extended dynamic range for a received image (IDR) with an intermediate dynamic range, while the image has a second maximum luminance (PB_CH) of pixels, which is lower by a multiplicative factor, preferably 0.8 or less than the first maximum luminosity (PB_C_H50) of the main image (MsterHDR) pixels with extended dynamic range, wherein the second maximum luminance (PB_CH) of the pixels is received as intermediate dynamic range image metadata, the decoding method also includes receiving in the metadata of a luminance display function (F_I2sCI) which defines the transformation of all possible normalized image luminances (IDR) with an intermediate dynamic range into the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range and maximum luminance of the LDR pixels, and the decoding method differs in that it takes the first maximum luminosity (PB_C_H50) of pixels, as well as those that include:

- application of a standardized algorithm for converting the luminance mapping function (F_I2sCI) into a luminance mapping decoding function (F_ENCINV_H2I), which determines as output for any possible input normalized image pixel luminance (IDR) with an intermediate dynamic range the corresponding normalized luminance HDR of the main image (MsterHDR ) with an extended dynamic range, the standardized algorithm using the values of the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels;

applying to the normalized image luminances (IDR) with intermediate dynamic range the luminance mapping decoding function (F_ENCINV_H2I) to obtain normalized reconstructed luminances (L_RHDR) of the pixels of the reconstructed main HDR image (REC_M_HDR), and

- output of the restored main HDR image (REC_M_HDR).

FIG. 25 well shows the real encoded signal (2501) SLHDR2PLUS HDR image, i.e., for example, a color matrix of 2502 pixels of size 4kx2k (YCbCr or any color representation that can be converted to the desired representation of YCbCr using known colorimetric equations by the preprocessor [not shown]), and the necessary metadata: a luminosity mapping function F_I2sCI and two PB_C values. In the event that this HDR image signal is transmitted and received by a standard SLHDR2 decoder 2510, since F_I2sCI is a normal function, this decoder would expect to recalibrate from its received image peak luminance (PB_CH, in this example, equal to 800 nits) to any more low peak brightness, and it can optimize display for example for a connected display with 350 nits MDR by calculating Im350 by optimizing the display (which, as said, is not a key aspect of this application, and is only possible as one of the initial design criteria when moving to a new codec paradigm, and to optimize the display, you can use, for example, the method disclosed in US 20160307602, or a similar method). But now it is also possible that anyone with a 2520 SLHDR2PLUS decoder, such as one recently deployed by a cable operator who decides to implement this service, etc., can create other pictures with a PB_C higher than the PB_CH value, for example, perform restoration of an approximate a main HDR image with 5000 nits (i.e. Im5000 output image) or any display-adapted image with brightness between PB_C_H50 and PB_CH, or potentially even higher than PB_C_H50, like Im1250 display-adapted image, etc.

Interesting decoder embodiments relating to chrominance processing, among others, are the following:

- an extended dynamic range video decoder (1100), in which the luminance function determiner (1104) is configured to additionally determine the original saturation gain characteristic curve (2801) depending on the channel saturation gain characteristic curve (2804) and the mapping function (F_I2sCI) channel brightness.

- a wide dynamic range video decoder (1100) in which the original saturation gain characteristic curve (2804) further depends on a saturation position correction function (FSNL) that corresponds to an equation including a power function for the normalized brightness value. The normalized luminances best used for this are the normalized luminances given, for example, by dividing by the peak brightness PB_C_MHDR of the main HDR image, eg 5000 nits.

The same corresponds on the encoder side to, among other things, a high dynamic range video encoder (900), in which the HDR function generation unit (901) is configured to determine a saturation gain characteristic curve depending on the original saturation gain characteristic curve (2801). and an adapted luminance display function (F_H2hCI). In the same way, color processing methods are performed in any technical device, for example, a decoding integrated circuit in a set-top box, etc., or on a mobile phone, or an encoder integrated circuit in a production device in an external broadcast channel, or a final encoder or transcoder in at the premises of a cable television operator, or are performed on the server of an Internet video content provider or a cinema content provider, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the method and apparatus according to the present invention will be apparent from the implementations and embodiments described below and explained with reference to them, as well as with reference to the accompanying drawings, which serve simply as non-limiting specific illustrations explaining more general concepts, and in which dotted lines are used to indicate that a component is optional, where components shown without dashed lines are not necessarily essential. Dashed lines can also be used to indicate elements that are considered essential but hidden within an object, or to indicate intangible things such as object/area selections (and indicate how they might be displayed).

On the drawings:

FIG. 1 schematically shows a number of typical color transformations that occur when a high dynamic range image is optimally mapped to a corresponding optimally color-graded and similar looking (as similar as desired and possible, given the differences in first and second dynamic ranges DR_1 and DR_2, respectively) an image with a narrower dynamic range, for example, an image with a standard dynamic range and a maximum brightness of 100 nits, which, in the case of reversibility (mode 2), would also correspond to the display of an SDR image received by receivers (decoders), and the SDR image is actually encodes an HDR scene;

FIG. 2 shows what an HDR imaging system might look like;

FIG. 3 explains a possible way of transmitting HDR images as some images with a specific (different) peak luminance and luminance mapping functions, commonly defined as luminance mapping functions, conveyed in metadata together, for example, as legacy SDR images Im_LDR, for example, according to the Applicant's preferred approach, at a high conceptual level, to provide introductory explanations for readers new to the technology;

FIG. 4 shows various additional features of HDR image coding in accordance with Applicant's specific approach standardized in ETSI2, aspects of which are necessary to understand the various features of the various ideas of the new SLHDR2PLUS codec approach outlined below;

FIG. 5 explains how color transforms, and in particular regraded luminance transforms, work in the YCbCr color gamut;

FIG. 6 explains our concept of a tuned curve in more detail, with an explanation of its useful application;

FIG. 7 explains a basic understanding of intermediate dynamic range (IDR) coding and transmission of the original HDR main image, as well as a concept that should not be confusing and is used for medium dynamic range (MDR) images, which are usually calculated from any the received image to optimize it for display on any available specific PB_D peak brightness display, such as a specific HDR TV that has been purchased by any end user intending to view received HDR video;

FIG. 8 further explains how the IDR problem should be approached, in particular, its solution in a specific way automatically performed in decoders, and in particular if at least some of the decoders potentially receiving content are ETSI2 compliant decoders already on the market. and possibly not easily updated with the new SLHDR2PLUS technology (eg because the owner of the TV or STB does not update it);

FIG. 9 shows the general design of the technical parts that are typically needed for the new SLHDR2PLUS encoder disclosed in this application;

FIG. 10 explains some basic technical aspects related to sequentially obtaining by the encoder of various corresponding luminance mapping functions, which are, in particular, illustrated by the example of the luminance mapping function para;

FIG. 11 shows a possible exemplary high-level design of a new SLHDR2PLUS decoder that follows some of the embodiment ideas for the various SLHDR2PLUS video transmission implementations described below;

FIG. 12 further explains some aspects of the black and white offset when it is selected by the content creator in its main luminance mapping functions, determining re-grading needs for its video content according to its presentation;

FIG. 13 describes the technical principles of a preferred approach to a fixed algorithm for obtaining a channel-adapted version of the para function in accordance with the principle of diagonal scaling;

FIG. 14 explains some aspects of the inverse curve for the para function, the so-called abcara function;

FIG. 15 details some aspects of the SLHDR2PLUS decoder;

FIG. 16 illustrates a useful way to implement display adaptation to compute an MDR image for any particular PB_D display in conjunction with the new SLHDR2PLUS coding techniques disclosed in this application;

FIG. 17 explains some additional aspects of channel adaptation of black and white offsets (BLO and WLO) to complement and simplify the mathematical model below, as well as provide a general physical and technical justification;

FIG. 18 shows another embodiment of SLHDR2PLUS encoding (or indeed several ideas for various combinations of embodiments explained in one figure), which is particularly useful since encoded pictures can be directly decoded by standard ETSI2 decoders already deployed;

FIG. 19 explains how to determine the appropriate versions of the original master tuning curve for different dependent peak brightness image representations (whether input or output image), for example, IDR images with a joint tuning curve specification to fine-tune coarsely mapped IDR luminances in order to obtain accurate final SDR luminances, for example, as a filmmaker's grading specialist would like;

FIG. 20 shows the underlying technical principles of the FIG. 18 for the spectrum of re-graded images;

FIG. 21 explains the extrapolation of adapting a luminance display function (eg, the para function) beyond the highest value of the initial image (corresponding to a unit transform or a diagonal in a graph), as well as the relationship between a particular chosen form of the para function and the corresponding abcara function;

FIG. 22 schematically illustrates how a particular embodiment of the flare gain limitation for the para function works with luminance recalibration between input and output normalized luminances;

FIG. Figure 23 is a schematic of the entire HDR transmission and processing flow to clearly point out the difference between video encoding and display optimization (with encoding referring to the very definition of images as such, while display optimization simply refers to optimizing display on any particular display with specific dynamic range functionality). , which is still freely configurable, preferably taking into account the re-grading needs of different, possibly quite different and rather complex HDR scene images);

FIG. 24 further explains how a standardized function change algorithm would work, which would take any form of input function given, for example, by a scaler for a content creator, to indicate how the normalized dynamic range luminosity with the first peak luminance should be mapped to the normalized presentation luminances. images with a second peak brightness, and how a function can be uniquely calculated from this input function to display between another representation, in particular, with a lower third peak brightness, and a second representation for any given input function;

FIG. 25 schematically shows how the SLHDR2PLUS video coding engine and signal described in this application is well compatible with legacy SLHDR2 decoders, while offering additional capabilities for deployed SLHDR2PLUS decoders;

FIG. 26 schematically summarizes in block form all of the following decoding mathematical model for restoring all necessary luminance mapping functions to perform all necessary luminance decoding processing to obtain a desired image from a received IDR image;

FIG. 27 schematically illustrates an excellent pragmatic approach to numerically obtaining a specially tuned version of the tuned curve, if any; And

FIG. 28 explains a practical approach to handling pixel chromaticities in the SLHDR2PLUS approach.

IMPLEMENTATION OF THE INVENTION

FIG. 9 is an overview of the new SLHDR2PLUS encoder 900. As input, it receives a basic HDR image (e.g., an MsterHDR image with a PB_C of 5000 nits), which the reader, without loss of generality, can consider as being created at or about the time of encoding by a human color grading specialist using color grading software , for example, starting from a RAW HDR image captured by the camera (whereby the MsterHDR image is optimized, for example, for a typical evening dark TV viewing environment, i.e. in relation to its average ambient luminosity, etc.; the present technologies can also work with different or variable environments, but this is more a matter of display adaptation rather than new ways of creating or encoding HDR images). The scaler also created at least one good luminosity downgrading function FL_50t1 to convert a 5000 nit HDR base image to a corresponding good looking SDR image (i.e. with a regular 100 nit PB_C), and he did this by substituting some partial aspects of the re-grading in blocks 403, 404 and 405, as well as some good color correction F_C[Y] in accordance with the chromatic processing block 451, which he checked on his reference SDR monitor (other methods, for example, when broadcasting real events, may involve calculating applicable forms of the function on the fly, and then there may be some director who looks at the result as a whole or even does not, but the principle is that a good FL_50t1 function arises either from just one from partial blocks, or due to the general functioning of all blocks, etc.).

This function FL_50t1 must also be entered as seed information for the new encoder 900. Static peak luminance metadata PB_C_H50 (for the entire movie or the entire broadcast) is also entered as it will be used, but is also output by the encoder as the resulting IDR coding signal -video (IDR+F_I2sCI+PB_CH+PB_C_H50, wherein the image is typically compressed or uncompressed according to some suitable video communication standard, e.g. HEVC, and other metadata can be transmitted according to any available or configurable metadata transfer mechanism, ranging from MPEG SEI to dedicated Internet packages, etc.).

The HDR function generating unit 901 will calculate the HDR to IDR luminance mapping function F_H2hCI that is needed to calculate the IDR image from the MsterHDR image, and will need to select the PB_CH IDR that it is supposed to get from some other input (for example, it may have been selected by the cable operator and held somewhere in memory for download by configuration software); let's assume that PB_CH is 1000 nits (for explanation purposes only; usually this value will be a couple of times higher than PB_C SDR, for example, 4 times higher, with technical aspects slightly different in particular implementations depending on which value selected).

FIG. 10 shows how this HDR function generating block 901 can function.

Assume that the calibrator has defined some function (here, in an illustrative example, a linear-parabolic-linear function - or the para function for short - that the applicant uses in accordance with the standardized ETSI codec philosophy, in order to perform the first and already pretty good repeated balancing the brightness of the dominant areas of the image (for example, this gives sufficient visibility to dark areas in the SDR image due to jointly controlled compression of the areas with the highest luminosities).

Such a function correlates the input luminances (in the psycho-visual balanced representation by converting the pixel luminosities according to Equations 1 and 2 above) of the lowest luminance sub-range (L<Ld) to the desired output luminances through a linear relationship with a controlled slope SG_gr optimally chosen for this HDR image by a grading specialist:

Ln_XDR = SG_gr*Ln_Mster_HDR if (Ln_Mster_HDR<Ld) [Lv. 4]

(where Ln_Mstr_HDR and Ln_XDR, respectively, are the luminances - i.e. psycho-visually uniformized representations of the respective pixel luminosities - of an input HDR master image pre-graded as an optimal initial image by a grader, and Ln_XDR is a generalization of multiple output luminances images with different dynamic range and, in particular, peak brightness PB_C, which are all shown on the same normalized vertical axis for the purpose of explaining the concepts underlying the present invention and its embodiments). In particular, when the grader starts to re-calibrate the corresponding optimal SDR image for his already optimally graded Mster_HDR image, XDR will be a variant of SDR, and the corresponding form of the luminance mapping function is shown as F_Mt1 [here, the abbreviation xty is used to indicate whether the function displays brightness from some initial value x PB_C to some final value y PB_C, where x and y can either generically denote the PB_C of the image, as "M" for "Master", or numerically indicate an approximate value, which then discards two zeros, i.e. ., for example, "50" means "5000" and "1" means "100" nits].

Similarly, for input luminances Ln_Mster_HDR higher than Lb, there is again a controlled linear relationship:

Ln_SDR = HG_gr*Ln_Mster_HDR+(1-HG_gr) if (Ln_Mster_HDR>Lb) [Lv. 5]

The parabolic part of the para function between Ld = mx-WP and Lb = mx+WP has the function definition L_XDR = a*x^2+b*x+c, from which the coefficients a, b and c can be computed by calculating the point where the linear tangents to the curve coming from its extreme values, and then its abscissa mx (as defined in the ETSI1 standard; mx = (1-HG)/(SG-HG)).

The general idea behind the invention is as follows (and can be explained in multiplicative terms). Any HDR base luminosity can be converted to itself by applying the identity transformation (diagonal). If at the end of the spectrum of re-graded images, i.e. to create the corresponding SDR luminosity (XDR = SDR), it is necessary to obtain the output luminosity L_SDR = F_Mt1(Ln_M), where Ln_M is some specific value of the Ln_Mstr_HDR luminosities, then this can be considered as a multiplicative gain of the input luminosity L_SDR = b_SDR(Ln_M)*Ln_M. If now it is possible to determine some intermediate function F_Mt1_ca, then the final processing is the sequential application of two functions F_IDRt1(F_Mt1_ca(Ln_Mster_HDR)), while F_IDRt1 performs the final mapping of the luminosity to the SDR luminosity of any pixel (or object), starting from the already calculated luminosity of the IDR pixels (derived from the main HDR luminosity). Using the multiplicative terms, we can say that L_SDR = b_IDR*b_ca*Ln_M, where the two gains correspond to the intermediate function (or channel adaptation function), and the remaining re-grading function (which turns out to be the function that is transmitted along with the IDR image to perform HDR encoding -video compatible with the ETSI2 standard). It should be noted that these gains are themselves functions of Ln_Mster_HDR (or in fact any intermediate brightness associated with them).

In such a case, it would be convenient if there was no need to transfer any additional functions (which, for example, may be lost if metadata management is imperfect, etc.).

Therefore, it may be useful if the SLHDR2PLUS philosophy uses a pre-agreed fixed way of converting the scaler's F_Mt1 function (i.e., a mechanism for whatever form of function he wishes to use) into a channel-adapted function corresponding to PB_IDR (i.e., the value , which is also typically sent to receivers as a PB_CH in accordance with the ETSI2 coding approach). It can be shown that there is then no need to co-transmit in the metadata associated with IDR pictures the upgrading function F_H2h because it is fixed and known to decoders, so the inverse F_?? can probably be computed from the received function F_I2s, as will be actually shown below (if PB_C_H50 is also transmitted to receivers). The novelty of the decoder lies in a new approach to obtaining images from PB_C>PB_IDR. In principle, any fixed approach to deriving the F_Mt1_ca function from the underlying F_Mt1 may be suitable, provided that it is mathematically reversible, or at least decodable as required, but it is desirable to choose a view class approach in which regrading from HDR to IDR is performed (i.e. the shape of F_Mt1_ca is obtained) so that its further deformation to obtain MDR images is compatible with what is obtained based on the ETSI2 standard (in principle, ETSI2 images are only standardized between PB_C and 100 nits, so you can start with the requirement approximate equality of image views, i.e. all luminances and pixel colors, for all images of dynamic range between PB_IDR and 100 nits, but one can also try to impose on the solution to be obtained technical limitations in that the images obtained upgradation from the received IDR image to the main HDR image, i.e. using F_ ??, to to be computed by the SLHDR2PLUS decoder should have the same form as would be obtained by adapting an ETSI2 display that was to receive, for example, Mster_HDR images with a PB_C of 5000 nits, and a common luminosity remapping function F_Mt1.

First, it will be explained how such a preferred channel adaptation (i.e., calculation of F_Mt1_ca or F_H2hCI shown in FIG. 9; and corresponding IDR picture(s)) can be designed which is useful for several SLHDR2PLUS approaches/embodiments.

FIG. 12a shows the white level offset WLO_gr optimally selected by the calibrator (or automatic tool) and, if present, the black level offset (BLO_gr), which correspond to block 403 of FIG. 4.

So far, we can assume that this is the only dynamic range setting, i.e. luminosity mapping operation to obtain an SDR image from the initial Mster_HDR image (this white-on-white and black-on-black mapping is a rather primitive dynamic range transformation that produces low-quality LDR images that no longer have the correct average brightness or average visual contrast ratio, not to mention the higher quality descriptors of the resulting image, which are desirable; but this is a good step if considered as the first step in the re-grading sequence according to Applicant's approach, and this step and the channel adaptation itself need to be explained first). The idea is that if (despite the ability to encode luminances up to PB_HDR = 5000 nits) there are in fact no pixel luminances higher than the MXH value in the image currently to be displayed (or a frame of images in the video of the same scene, in case someone decides to map the same function for all of these time-sequential images), then it makes sense to map this highest MXH value to the maximum brightness code in SDR (i.e. 1024, corresponding to luminance of 100 nits, for example). Any other mapping approach (for example mapping "white in HDR to white in SDR") will make all the actual luminosities even lower, and this is not optimal, given that the range of SDR luminosities is already quite small, while it should optimally contain the corresponding emulation of a wide range of HDR luminosities.

The question then arises as to whether this WLO value for the IDR image should be adjusted (as seen in FIG. 12b, the highest brightness in the intermediate images may already be approaching PB_IDR while still having a final transition to display at 1.0 for the re-rendered grading the SDR image; this display can also be shown equivalently in the normalized luminance range of the HDR image with 5000 nits, which is indicated by "ON"). With the first approach, this is not necessary (because there is some freedom in how to develop an algorithm for calculating the function F_Mt1_ca), but in case of scaling it, the following approach can be used.

In order to be able to scale the luminosity mapping function, it is necessary to define a scaling factor for such horizontal scaling, which in this case are its WLO_ca parameters and, similarly, the scaled BLO_gr (denoted as "BLO_ca"). If you want this parameter to scale linearly with PB_IDR, then the constraints should be that the action is performed in full, i.e. the offset has its maximum value BLO_gr when PB_IDR = PB_SDR. On the other hand, for an HDR image, BLO or WLO should be zero, since nothing needs to be adjusted since we have the same transformation to map Mster_HDR with 5000 nits to Mster_HDR with 5000 nits.

Therefore, we can formulate the following definition of parameters:

WLO_ca = scaleHor*WLO_gr (0 <= ScaleHor <= 1)

BLO_ca = scaleHor*BLO_gr [Lv. 6]

The question then becomes how to define ScaleHor.

FIG. 12b shows a spectrum of images with different dynamic range, more precisely, with different PB_C, which are arranged along the horizontal axis. They are located along the perceived positions of the peak brightness PB_C of each image. Thus, we put them in position with abscissa v(PB_C), where v is a function from Equation 1, where the PB_C value is used for the L_in parameter, and the RHO value from Equation 2 is calculated for the maximum brightness of the Mster_HDR image that has been graduated (t i.e., for example, RHO is 25 for an Mster_HDR image with a PB_C of 5000 nits). If the luminances L are also plotted on the y-axis, parameterized according to the function v (on the vertical axis) with the same RHO = 25, then the PB_C values exactly follow a straight line, and definitions and calculations can be performed in this paradigm. For example, it is possible to project the luminances for the peak luminances PB_C of any intermediate image onto the main (5000 nits) luminance axis. The notation "P_I1oI2" is used, meaning a luminance value corresponding to applying the v-function with the peak luminance (which is the normal luminance) of the image I1 when it is represented in the luminance range of the image I2. So, for example, P_IoH is the brightness for the peak brightness of the selected IDR image in the Mster_HDR brightness range, and P_SoH is the brightness of 100 nits (note that 1.0 in this range corresponds to the PB_C of the Mster_HDR image, so a position of, for example, 100 nits, for example, 0.5 will vary depending on the chosen image representation Mster_HDR, and therefore equations 1 and 2 are a family of curves parameterized by RHO).

Then the appropriate function for ScaleHor would be one that starts with 1-P_IoH. This function will indeed increase as PB_IDR decreases, i.e. the further to the right we select our IDR image representation for the MsterHDR image. And it will give 0 if P_IoH = 1, which happens when an IDR image with 5000 nits is selected (purely for the theoretical explanation of the scaleHor equation, because it doesn't make sense from a technical point of view). However, this equation does not equal 1.0 when IDR = SDR, so it needs to be scaled by a factor k.

It can be checked that the normalization is correct if k = 1-P_SoH (which is in conflict with the value of the variable P_IoH corresponding to the various fixed value IDR positions), so:

ScaleHor = (1-P_IoH)/(1-P_SoH) [Lv. 7]

Determining the correct para function (see FIG. 4, block 404) for channel mapping is more complex and is explained using FIG. 13.

In this case, the inventors decided to transform the function in a diagonal direction orthogonal to the diagonal ([0,0]-[1,1]) of the identity matrix. It must be transformed with an equivalent choice of parameters in the normal representation of the Mster_HDR/XDR coordinate system for all functional recalibrations.

The base scaling is set in the axis system rotated by 45 degrees, which changes the diagonal to the horizontal axis (see FIG. 13a). You can see the Fx function, which is, for example, the rotated para function. It makes sense to scale any dY value for a point on the rotated diagonal, i.e. on the new x-axis (on which dX corresponds to some abscissa, i.e. the brightness L_Mster_HDR in the original axis system), by the coefficient La/K, where K is the full action of the function, i.e. the full value of dY, and the scaling value dY_ca will correspond to (La/K)*dY in this rotated system.

We set sc_r = La/K, where La = 1/P_IoH and K = 1/P_SoH (it should be noted that the brightness value I2 on the I1 axis can be reformulated as the brightness value I1 on the I2 axis, in particular, for example, 1/P_IoH = P_HoI; for example, if P_IoH = 0.7, that means PB_Mstr_HDR will stick to 1/0.7 above PB_IDR).

Now we need to calculate the equivalent vertical scaling sc* with respect to the diagonal sc_r.

This can be done by applying a mathematical model of reverse rotation (in fact, by first setting K and La to 1.0 instead of 1.4), transferring the representation of FIG. 13a on the diagonal of FIG. 13b. This results in the use of a rotation matrix (any x_r, y_r in the diagonal system, e.g. 1, dY rotated to the main representation) to the following:

[x1, y1] = [cos(pi/4) -sin(pi/4); sin(pi/4) cos(pi/4)]*[1, P_HoI = 1/La]

[x2, y2] = [cos(pi/4) -sin(pi/4); sin(pi/4) cos(pi/4)]*[1, P_HoS = 1/K] [Lv. 8]

Note that due to diagonal scaling, the x and y coordinates will change, but SG and HG, and any other change in the scaled point, are given as slopes, not angles, anyway.

A rotation starting from a line from (0,0) in FIG. 13b to a square representing the diagonally scaled point of the luminance mapping function, to a line from (0,0) to a circle being the origin of the luminance mapping function, or vice versa, can be found by dividing the slopes by any fixed value of the abscissa, for example (a change in angle corresponds to vertical change of the normalized scaling factor sc*):

sc* = (y2/x2)/(y1/x1) = [(1+1/K)/(1-1/K)]/[(1+1/La)/(1-1/La)] = [(K+1)/(K-1)]/[(La+1)/(La-1)] = [(La-1)*(K+1)]/[(La+1)* (K-1)] [Lv. 8]

The actual y-distance n corresponding to full vertical scaling (sc* = 1) must then be calculated, and this can be done by recognizing that, due to the 45 degree angle involved in the diagonal scaling, mip is the midpoint having a distance Fd under it to the diagonal and above it to the point (mx, my) of the intersection of two linear segments of the para function. Hence, n = Fd is equal to half of the differential slope of SG-1 at mx, i.e. mx*(SG-1)/2.

After that, you need to calculate the offset point (mxca, myca) of the intersection, which looks like this:

mxca = mx+d = mx+[mx*(SG-1)/2]*(1-sc*)

myca = my-d = SG*mx-(mxca-mx) = -mxca+mx*(SG+1) [Lv. 9]

With the location of the new point determined, one can finally compute the channel-adapted shadow gain (SG_ca, see FIG. 10) and the channel-adapted highlight gain HG_ca:

SG_ca = myca/mxca

HG_ca = (myca-1)/(mxca-1) [Lv. 10]

Finally, there are several approaches/options for the parabolic middle segment.

One approach that gives good visual results in practice is to take WP_ca = WP_gr, where WP_gr is the original width of the parabolic segment, optimized by a scaler or content creator's automatic tool that determines the relationship between the main HDR image and the main SDR image, and WP_ca is the width for the channel-adapted para function. Another approach is to define WP_ca = v(abs(sc*), 100)*WP_gr, with the v-function being again given by Equations 1 and 2 above.

If available, this available technology can be used to formulate a suitable IDR definition for SLHDR2PLUS.

Returning to FIG. 10, the above equations define how the F_Mt1_ca function can be uniquely set, for example, for a selected PB_IDR of 1000 nits, starting from, for example, a main HDR image with 5000 nits. If this function is determined by block 901 generate HDR function, it can be output as F_H2hCI and sent as input to block 902 calculate IDR image. This block will apply this function to all the pixel luminances of the MsterHDR image it receives as input image [L_IDR = F_H2hCI(L_MsterHDR) = F_Mt1_ca(L_MsterHDR)] to obtain the corresponding pixel luminosities of the IDR image, and it will output the IDR image.

Now the question still remains what luminance mapping function should be added to the metadata of an IDR image so that it looks like it is a normal ETSI2 image (i.e. so that any legacy ETSI2 decoder can decode it normally, outputting SDR image or any MDR image as it should look).

This secondary IDR luminance mapping function F_I2sCI, which will also be a para function, can be defined as follows (and it will be calculated by the IDR mapping function generator 903). The shadow enhancement for the IDR image SG_IDR can be thought of as the residual multiplication (or slope) after the transition from Mster_HDR to the IDR image (i.e. the remaining relative increase in brightness to obtain an SDR image starting from the IDR image):

Y_out(x_in) = SG_gr*x_in; = F_I2sCI(L_IDR = SG_ca*x_in)

We also know that the same line segment mapping of the para function for the darkest pixels is applied to the new input IDR brightness values:

Y_out = SG_IDR*L_IDR

From here:

SG_gr = SG_IDR*SG_ca [Lv. eleven]

(for example, if you take the input value x_in = L_Mster_HDR = 0.2, then it is mapped diagonally to L_IDR = 0.3 = (0.3/0.2)*x_in, which is finally mapped to Y_out = 0.4 = k*0.3 where k = 0.4/0.3 Y_out = SG_gr*0.2 = (0.4)*0.2 = (0.4/0.3)*(0.3 /0.2)*0.2).

Thus, Equation 11 implies the following way to calculate the required SG_IDR (assuming a fixed approach to determining SG_ca has been used as described above):

SG_IDR = SG_gr/SG_ca [Lv. 12]

Similarly:

HG_IDR = HG_gr/HG_ca [Lv. 13]

Here, HG_gr is again the optimal specular gain defined by the content creator defining the relationship between the view of the main SDR image and the view of the main HDR image (i.e. its brightness distribution), and HG_ca is the channel-adapted specular gain corresponding to the original gain HG_gr glare.

It should be noted that the basic shadow gain control can be determined relative to the expected simple shadow gain based on the difference in peak brightness between the SDR image and the IDR image as follows: ShadBst = SG_IDR/P_IoS. As said, P_IoS is the maximum encoded luminosity of an IDR image when it is represented on the normalized luma axis of an SDR image, i.e. 7.0 for example.

It should be noted that there are some practical implementations in which the glare gain cannot be greater than a predetermined number (in the way that the ETSI standard encodes the glare gains), in which case further recalculation of the glare gain is required, see below, but this not necessary for all embodiments. This can be implemented, for example, as follows:

If HG_IDR>KLIM, then HG_IDR_adj = KLIM, [Lv. 14]

wherein KLIM is preferably 0.5.

Indeed, if we assume that the calibrator made HG_gr close to the maximum value of 0.5, and the corresponding HG_ca (which, as a softer mapping, should have HG_ca closer to the diagonal, i.e. be greater than HG_gr), for example equal to 0.75, then it can be found that the division result is 0.67, which is higher than the maximum value that can be transmitted according to the ETSI2 standardized pure HDR video signal. The solution, for example, is to redefine a smaller HG_gr so that HG_IDR is not higher than 0.5, i.e. standardized maximum. This again requires significant computation, considering all aspects of recalibration, as will be shown below. Another option, for example, is to make the metadata + IDR signal compatible by limiting HG_IDR to 0.5 while passing the exact unrestricted HG_IDR as additional metadata. HG_gr will usually depend not only on the PB_C of the Mster_HDR image, but also on what type of image objects are in the image (e.g. bright colored objects that are important enough not to compress their luminosities too much, with the image being an extreme example a bright planet near a powerful sun that is calibrated with many very high L_Mster_HDR brightness values and a few low brightness values). HG_ca will typically depend on, among other things, how close the selected PB_IDR is to PB_Mster_HDR.

Also, suppose WP_IDR = WP_gr [Lv. 15]

As has been said, other embodiments are possible, but for the purpose of simplifying the explanation of the principles, the above assumption will be used.

Using Equations 6, the corresponding channel-adapted black offset and white offset values were calculated (assuming such offsets were defined by the content creator). It now remains to calculate (by the IDR video encoder) the corresponding BLO_IDR and WLO_IDR values.

First, the glim value is calculated in the preferred encoding:

glim = {log[1+(rhoSDR-1)*power((0.1/100); 1/2.4)]/log(rhoSDR)}/{log[1+(rhoHDR-1)*power( 1/PB_Mster_HDR;1/2,4)]/log(rhoHDR)} [Lv. 16]

where rhoSDR = 1+32*power(100/10000; 1/2.4), and

rhoHDR = 1+32*power(PB_Mster_HDR/10000; 1/2.4).

This will simplify the adaptation of BLO, since in fact there is a parallel with the luminosity processing sequence in the standard ETSI1 and ETSI2 approach to HDR coding (i.e. between blocks 402-406 of FIG. 4 and blocks 1502-1506 of FIG. 15) , with the figures explaining for ease of understanding only partial successive recalibration steps in Applicant's approach, as well as a linear gain limiter applying a linear curve with angle glim to the perceptual Y'HP and performing a comparison with the Y'GL value calculated using the explained blocks, and choosing the maximum of two values computed in parallel (this is particularly important for the reversibility of ETSI1 so that the lowest HDR luminances can be recovered).

It can now be shown that, due to the action of this limiter, the BLO values can be easily adapted to the channel using the following equation:

BLO_IDR = BLO_gr*glim [Lv. 17]

glim, as shown above, depends on the particular choice of PB_Mster_HDR and may, for example, be equal to 0.6.

This is illustrated in FIG. 17. In FIG. 17b shows the increase in the lowest luminances in the full range luminance display shown in FIG. 17a. The different functions are again shown in the normalized plot, which correspond to different input PB_Cs and output PB_Cs.

FL_gr is a function created by the content creator to display, for example, a 4000 nit Mster_HDR to a 4000 nit SDR. The dotted curve FL_ca represents the channel adaptation to obtain, for example, a 500 nit IDR from Mster_HDR. The dotted curve FL_IDR is a curve for mapping IDR luminances to SDR luminances. On the enlarged graph in FIG. 17b shows that the FL_gr curve has a sharp kink at an input value of about 0.03 where the parallel gain limiter kicks in (i.e. its linear output value y = glim*Y'HP is chosen as the output value of the function for low input luminance values instead of the Y'GL value resulting from the operation of all the blocks in the sequence as shown in FIG 4 (for a complete description of the circuit, see FIG 4 related to the ETSI1 standard)).

The BLO value of any curve represents the intersection with the horizontal axis that would occur if there were no gain limitation, ie, for example, BLO_gr, shown by extending the local slope above the 0.3 value of the FL_gr curve, as shown by the dotted line.

For the purposes of this application, it is sufficient to know that one can also extend the FL_IDR curve to obtain a BLO_IDR value (note that there is a glim_IDR value to be used in the ETSI2 standard that is different from glim_gr), and that this lower BLO_IDR value can be found as glim *BLO_gr (Note that the only glim value that needs to be computed for SLHDR2PLUS is the one shown in FIG. 17b as glim_gr).

The following calculations are then performed to obtain the WLO_IDR.

FIG. 17a also shows that there are three different WLOs, namely WLO_gr, originally created by the scaler as his main mapping strategy from HDR to SDR (also shown as "ON" in FIG. 12b), WLO_ca adapted for the channel, in which the FL_ca curve intersects the upper horizontal line and which is the WLO_gr luminance mapping onto the IDR luminance axis (which can be represented with representations like those shown in FIG. 12 where MXH is projected onto MXI), and finally there is also WLO_IDR which is WLO, remaining for the IDR to SDR luminance conversion (which is not the same as the scaled WLO_ca because the definition of the normalized abscissa for luminance changes starting from the associated PB_C = 5000 for WLO_gr and WLO_ca because the input image for regrading with these functions is Mster_HDR with 5000 nits, ending with PB_C = 1000 nits for IDR-related definitions for force recalibration needs that in this ETSI2 compliant representation, the start image for deriving other images from what is received is, for example, an IDR image with a PB_C of 1000 nits).

FIG. 17c zoomed in on this top corner (about [1,1]) of the function graph. The value of WLO_IDR results from using the value of WLO_gr as input to the FL_ca curve, as shown by a circular projection from the (normalized) position on the y-axis to the position on the abscissa. FIG. 12b, it can be seen that the MXI position is indeed a normalized position on the IDR luminance axis, which maps to an SDR luminance of 1.0, whereby it conforms to the definition of WLO_IDR.

At first glance, one might think that if the mapping curve, through which the WLO value subsequently enters the encoding side, is a para function (see in Fig. 4 mapping by block 404 after block 403), then it is usually the upper linear segment of the para function that will be involved. .

However, depending on how the para function is set, any part of it can be involved (there are even settings where just the special SG value of the para function sets a very high crossover point, theoretically moving above 1.0, causing the function to behave in this case up to the highest luminances is only determined by the shadow gain slope, resulting in a linear curve that is useful for regrading in an SDR HDR image containing mostly very high luminances, such as a desert planet illuminated by 5 suns in a sci-fi movie). Therefore, a fairly complex calculation is required in the case where it is necessary to check which of the three segments of the para function is applicable, while the preferred mathematical implementation is as follows:

WLO_co = 255*WLO_ca/510

BLO_co = 255*BLO_ca/2040

Xh = (1-HG_ca)/(SG_ca-HG_ca)+WP_ca

WW = (1-WLO_gr*255/510-BLO_co)/(1-WLO_co-BLO-co)

IF WW>= Xh, TO WLO_IDR = HG_ca*(1-WW)*510/255 [upper line segment]

ELSE {

Xs = (1-HG_ca)/(SG_ca-HG_ca)-WP_ca

IF WW>Xs

{[input value, i.e. WLO_gr, should be rendered via a parabolic subpart of the channel-adapted para function]

A = -0.5*(SG_ca-HG_ca/(2*WP_ca))

B = (1-HG_ca)/(2*WP_ca)+(SG_ca+HG_ca)/2

C = -[(SG_ca-HG_ca)*(2*WP_ca)-2*(1-HG_ca)]^2 / (8*(SG_ca-HG_ca)*2*WP_ca)

WLO_IDR = (1-(A*WW*WW+B*WW+C))*510/255

}

ELSE [in special cases where a subpart of the para function is used to enhance the shadow]

WLO_IDR = (1-SG_ca*WW)*510/255

}

These parameters SG_IDR, HG_IDR, WP_IDR, BLO_IDR, WLO_IDR (and, if necessary, similar additional parameters for the custom curve) are parameters that characterize and are therefore output as a function of F_I2sCI (whether or not these parameters characterizing the shape of this desired curve with the purpose of performing display adaptation, or whether the LUT characterizing the function is output depends only on the choice of implementation; the main thing is that the correct form of the luminance display function F_I2sCI in a normalized to 1.0 axis system is transmitted along with the IDR image (IDR images) in the form of metadata ).

The encoder is now characterized according to the new SLHDR2PLUS approach. The question then becomes how the decoder should be designed. It should be understood that this decoder will now only receive the F_I2sCI function, so it must somehow calculate the F_?? function needed to reconstruct the original Mster_HDR image from the received IDR image. In this SLHDR2PLUS approach to encoding, this would be the inverse function of the F_H2hCI function used in the encoder to generate the IDR luminances, but such a function must still be fed to the calculation.

In the shown in FIG. 11 in the general view of the SLHDR2PLUS video decoder 1100, the luminance function determiner 1104 must calculate F_ ?? function based only on the information it receives, i.e. F_I2sCI, as well as two peak brightnesses PB_CH and PB_C_H50. Once this function is defined, it can be used to reconstruct the original Mster_HDR luminosities by applying it (in the color mapper 1102) to the received IDR luminances: L_REC_M_HDR = F_??(L_IDR), from which the corresponding HDR luminosities can be calculated by applying the equations that are inverse to equations 1 and 2, to these L_REC_M_HDR luminances. Finally, the reconstructed HDR main image (REC_M_HDR) can be output by the color converter 1102 in any desired format, such as PQ-based YCbCr color representation, etc. Decoder 1100 may in preferred embodiments also be configured to compute any display adapted image, e.g. MDR_300, in case the connected display with a PB_D of 300 nits is to be provided with a better equivalent of the received HDR image, and this may be done either by mathematical SLHDR2PLUS model, or simply using conventional ETSI2 decoding, since the corresponding image (IDR) and luminance mapping function (F_I2sCI) are already available as input to the color converter 1102).

FIG. 14 shows what exactly is used in the para function to reconstruct the REC_M_HDR image from the received IDR image (similar calculations will be performed for WLO and BLO, and for custom curve shape points where applicable (note that, as discussed below, some embodiments will not apply a tailor-made curve philosophy between Mster_HDR and IDR, but only SDR downscaling technology, i.e. between IDR and SDR).

Now it is necessary to calculate the new shadow gain (SG_REC) and the new highlight gain (HG_REC) to restore the main HDR, and also calculate the inverse parabolic equation for the parabolic segment to find the necessary form F_L_RHDR of the para function to display the luminosity when restored (it should be noted that it is simple in for purposes of illustration, the inverse luminosity mapping function from SDR to Mster_HDR is also shown in this normalized plot as a dotted line, note that due to the property of the inverse mapping function of SDR to HDR, the shadow gain of this SG_RM curve is 1/SG_gr etc.).

FIG. 15 first explains some aspects of a typical basic computation topology of the decoder 1502. As can be seen, this is roughly the same structure as the encoder, despite the fact that it recalibrates in the opposite direction (recovery of REC_M_HDR from IDR), which is convenient because such a computing topology can be easily reconfigured if necessary. If the luminosity mapper 1501 receives the full LUT (of all partial recalibration steps), it will indeed function in the same manner as the encoder.

Of course, some differences need to be generated in order for the decoder to perform the correct recalibration when HDR is restored. First, L_in will now be the normalized IDR luminosity and the output luminosity Lh will be the normalized luminosity that scales correctly, for example, to display on a display with a PB_D of 5000 nits. It is also clear that the last multiplier, which gives the colors (Rs, Gs, Bs) of the pixels of the REC_M_HDR image, is now multiplied by the PB_C_H50 value adopted in the metadata. In fact, the outer perceptualization calculation loop, performed by the perceptualizer 1502 and the linearizer 1506, applies the values of PB_CH and PB_C_H50, respectively, to Equations 1, 2, and their inverses. It should also be noted that the order of the various partial recalibrations, to the extent that they are present, is now reversed: first, the perceived luminance Y'IP IDR is finely calibrated by the inverse tuning curve in fine grading block 1503, resulting in recalibrated Y'IPG luminances. IDR. Thereafter, the first mapping to the HDR luma axis (i.e., to the respective redistributed luminances for the respective correct HDR view, in fact, the Mster_HDR view with PB_C_H50 5000 nits) is performed by coarse luminance mapping block 1504, which applies the inverse para function of FIG. 14, which still needs to be computed correctly, and will result in the original Y'HC HDR luminances. Finally, the black and white inverse bias block 1505 will produce the correct normalized luminances (Y'HR) REC_M_HDR to be used in further chromaticity calculations to obtain the full 3D color for each pixel. As explained, block 1504 will typically receive the computed SG_REC, and so on. (or a LUT version of the brightness mapping function to be applied according to those three values). It should be noted that if the various PW values remained identical, then WP_REC would again be WP_gr. Block 1505 will similarly obtain the black and white offset to restore Mster_HDR (WLO_REC, BLO_REC). The lower part of the main chromatic processing unit (color processor 1550) will be similar to the encoder topology of FIG. 4, except for loading the correct C_LUT F_C [Y] into the chromatic processing determination block 1551 (see below for explanations of this calculation).

Now the question is whether and how to calculate the parameters of the functions that should be used in a decoder programmed to recover Mster_HDR from IDR (this situation did not occur before when decoding HDR video).

For example, you can understand the approach to enhancing the shadow.

Before calculating SG_REC, one may wonder if the total shadow enhancement SG_RM from SDR to Mster_HDR can be determined and based on this, SG_REC can then be determined by dividing in Equation 12.

So SG_IDR = SG_gr/SG_ca.

It can also be shown that SG_ca = (mx/mxca)*(SG_gr+1)-1.

This can be seen based on the fact that myca = SG_ca*mxca (by definition of the lower linear segment of the para function adapted for the channel), and also myca = my-d = mx*SG_gr+(mx-mxca).

The second ratio mxca/mx is obtained by dividing the top of equations 9 by mx.

Since, by substituting the first relation into the second (removing the mx/mxca part), one can write SG_ca through SG_gr, the final relationship is now formed between SG_IDR and SG_gr:

SG_ca = (SG_gr+1)/[(SG_gr-1)*(1-sc*)/2+1]-1.

Where:

SG_IDR = SG_gr/{(SG_gr+1)/[(SG_gr-1)*(1-sc*)/2+1]-1} [Lv. 18]

This equation can now be solved for the unknown SG_gr given the known (received) SG_IDR (and sc* was calculated only from the peak luminances, which are also known since both PB_CH, i.e. PB_IDR and PB_C_H50 are received, and PB_SDR is usually 100 nits, but otherwise it could also be placed in the signal's metadata).

Using SG_IDR = y and SG_gr = x for simplified notation, we then get that:

y = [(x-1)*(1-sc*)*x/2+x]/[x-(x-1)*(1-sc*)/2].

Therefore: x^2+x*(y-1)*[(sc*+1)/(sc*-1)]-y = 0 [Eq. 19]

[these coefficients (called below A', B', C'), which are functions of y and sc*, will be used below to solve the quadratic equation in the general system of equations for reconstructing the luminosities of the Mster_HDR image].

To determine all the parameters shaping the luminance mapping reconstructing function, the following equations can typically be used in one of the embodiments (this reconstructs the reverse function of that used to generate the IDR image on the encoder side). First, the correct para function is determined, from which the black and white offsets are subsequently calculated.

rhoSDR is again calculated as above and rhoCH is calculated as:

rhoCH = 1+32*power(PB_CH/10000; 1/2.4)

mu = log[1+(rhoSDR-1)*power(PB_CH/PB_SDR; 1/2,4)]/log(rhoSDR)

K, La and sc* are calculated as above with K = P_HoS and La = P_HoI.

A'= 1

B '= (SG_IDR-1)*(sc*+1)/(sc*-1)

C'=-SG_IDR

Once it has been possible at the decoder side to determine the necessary parameters of all required functions (i.e., from other received available SG_IDR parameters, etc.), the rest of the decoding is performed on the basis that the possibility of reversibility, which is simply the application of the reverse (reverse) coding curve(s), for example, a para function like that shown in FIG. 14 (which has been appropriately shaped by calculating the respective defining parameters 1/SG_REC, etc.) will override the para IDR encoding function as shown in FIG. 10, i.e. determine IDR re-decoding with respect to Mster_HDR luminances, etc.).

From this it follows that:

SG_gr = [-B'+SQRT(B'^2-4*A'*C')]/2*A',

where ^2 means square power.

SG_REC = SG_gr/SG_IDR [Lv. 20]

Thus, the inverse gain (1/SG_REC) of the shadow for channel adaptation is already known.

Similarly, the required glare amplification can be calculated.

A'' = (SG_REC*HG_IDR-SG_gr)*(SG_gr+1)/(SG_REC+1)

B'' = SG_gr-HG_IDR-(SG_REC*HG_IDR-1)*(SG_gr+1)/(SG_REC+1)

C'' = HG_IDR-1

MxRec = [-B''+SQRT(B''^2-4*A''*C'')]/2*A''

IF MxRec = 1, THEN HG_REC = 0

ELSE = HG_REC = max[0, (MxRec*SG_gr-1)/(MxRec-1)]

Since the para function is defined from its parameters, after they have been evaluated, the required para function is determined.

The following equations are used to obtain BLO_REC and WLO_REC:

mx = (1-HG_gr)/(SG_gr-HG_gr)

mxca = mx*(SG_gr-1)*(1-sc*)/2+mx

myca = mx*(SG_gr+1)-mxca

SG_ca = myca/mxca

IF mxca = 1, THEN HG_ca = 0, ELSE HG_ca = max[0, (myca-1)/(mxca-1)]

ScaleHor = (1-1/La)/(1-1/K)

RHO = 1+32*power(PB_C_H50/10000; 1/2.4)

glim = {log[1+(rhoSDR-1)*(0.1/100)^(1/2.4)]/log(rhoSDR)}/{log[1+(RHO-1)*(1/ PB_C_H50)^(1/2,4)]/log(RHO)}; [as before, the same glim that is used by the encoder, because in the ETSI approach it is a fixed parallel workaround in the Im_PB_C_1 <> Im_PB_C_2 mechanism, with these two images defined as re-graded starting from the same PB_C_1, and in this specific SLHDR2PLUS approach are, respectively, the Mster_HDR and IDR images]

BLO_gr = BLO_IDR/glim [reciprocal equation of equation 17, so it is relatively easy to determine without resorting to higher order equations, and subsequently it is only necessary to apply a fixed channel adaptation mechanism to obtain the necessary WLO_REC, which is equal to the WLO_ca used in encoding, but now will be inverted, with addition becoming subtraction]

BLO_REC = BLO_ca = BLO_REC*ScaleHor

After that, WLO_REC is calculated by projecting it through the para function, as was done in the encoding principle, for subsequent inversion.

IF HG_ca = 0 WLO_REC = 0

ELSE

{

BLO_co = 255*BLO_ca/2040

Xh = (1-HG_REC)/(SG_REC-HG_REC)+WP_REC

Xh_REC = HG_REC*Xh+1-HG_REC

WW_REC = 1-WLO_IDR*255/510

IF WW_REC>=Xh_REC THEN WCA = 1-(1-WW_REC)/HG_REC

ELSE

Xs = (1-HG_REC)/(SG_REC-HG_REC)-WP_REC

Xsca = SG_REC*Xs

IF WW_REC>Xsca

{

A'''= -0.5*(SG_REC-HG_REC)/(2*WP_REC)

B'''= (1-HG_REC)/(2*WP_REC)+(SG_REC+HG_REC)/2

C'''= -[(SG_REC-HG_REC)*(2*WP_REC)-2*(1-HG_REC)]^2/[8*(SG_REC-HG_REC)*(2*WP_RE)]

WCA = (-B'''+SQRT(B'''^2-4*A'''*{C'''-WW_REC})/(2*A''')

WCA = min(WCA, 1)

}

ELSE WCA=WW_REC/SG_REC

WLO_REC = (1-WCA)*(1-BLO_co)/[(1-WCA*ScaleHor)*(510/255)]

It should be noted that while BLO is indeed a purely additive contribution in terms of display, WLO is converted to multiplicative scaling to the maximum (for example, in FIG. 4):

Y'HPS = (Y'HP-BLO)/(1-BLO-WLO) [Lv. 21]

All of this information can usually be contained in a single luminosity processing LUT that refers to, for example, the perceptual region Y'IP to Y'HR (or better yet, a common LUT that specifies Lh for each value of L_in). This will restore the REC_M_HDR image.

As mentioned above, it is also useful if the decoder can directly output a display-adapted image, such as MDR_300.

For this, the following technology, illustrated in FIG. 16 (where two partial LUTs are used, it is most useful in practice to simply load one LUT, called P_LUT, since the upper luminance calculation channel passes through the preferred main calculation units, e.g., a per-pixel color processor in a dedicated decoding IC, usually implemented simply as a LUT. Y_IDR luminance values are input (eg, typically encoded with YCbCr based on PQ) and these are converted into normalized L_in luminosities by a linearizer 1601. The perceptualizer 1602 functions as explained above (see Equations 1 and 2) and uses the RHO value for peak luminance PB_IDR IDR, for example, 1000 nits This results in the perceived luminances Y'IP IDR The luminance mapping unit 1603 reconstructs the main HDR image as explained above, i.e. it obtains all calculated parameters defining the reconstructive mapping function F_L_REC luminosity from IDR to MsterHDR, or usually the LUT form of this function. t as a result of the restored brightness Y'HPR Mster_HDR. This image is a good basis for computing images with a narrower dynamic range/low PB_C peak brightness. It basically works like an ETSI2 mechanism, provided that the correct functions are applied. These functions may be scaled from the F_L_IDR shared as metadata, or computed from the reconstructed F_50t1 function, which is a reconstruction of what the content creator has determined at its end to be the optimal function for calculating the main SDR image from the Mster_HDR image. This F_50t1 function can then be mapped to the corresponding display adaptation function F_L_DA, for example for PB_D 300 nits, according to the principles defined in the ETSI2 standard (the reader is referred to this standard for details). It is loaded into HDR to MDR luminosity mapper 1604, if any. In practice, a single P_LUT will contain the full action F_L_REC and then F_L_DA.

Finally, the obtained MDR relative luminosities are sent to the first multiplier 454 of FIG. 4 to perform the same processing (also using the correct companion function F_C[Y]).

Finally, it is necessary to calculate the corresponding C_LUT (F_C[Y] in FIG. 4 or FIG. 15, respectively) that illuminates the re-calibrated output colors with their respective chroma (to have as close as possible to the Mster_HDR image, i.e., the chroma pixels of the output image and the Mster_HDR image should be as nearly identical as possible, given the excellent lower dynamic range).

The C_LUT for Mster_HDR recovery is as follows (other C-LUT calculations in recalibration are performed according to similar principles, for example, taking into account the principles of ETSI2).

First, a CP-LUT is computed which is the inverse of the above P_LUT that was applied in the encoder to map the Mster_HDR image to an IDR image (so the decoder will use this inverse chroma correction to deconvert from the received Cb and Cr chroma of the IDR image to the reconstructed chroma mster_HDR).

Then the C_LUT for Mster-HDR recovery can be calculated as:

XH = v(PB_M_HDR; 10000)

XS = v(PB_SDR = 100; 10000)

XD = v(PB_D; 10000)

XC = v(PB_CH; 10000)

Here, v is again the function of v(x, RHO) given by Equations 1 and 2 above.

CfactCH = 1-(XC-XS)/(XH-XS)

CfactCA = 1-(XD-XS)/(XH-XS)

C_LUT[Y} = [1+CfactCA*power(CP_LUT [Y]; 2,4)]/[Y*{1+CfactCH*power(CP_LUT[Y]; 2,4)}] [Lv. 22]

The target display PB_D may be set equal to the PB_Mster_HDR for recovery, in which case only the delimiter remains as the C_LUT qualifier. In a practical embodiment, a power of 2.4 may also be included in the LUT, such as CPP_LUT = power(CP_LUT[Y]; 2.4), which may simplify some calculations in some embodiments.

It has been said above that in some practical embodiments (for current ETSI2 metadata definition compatibility) the SLHDR2PLUS encoder recalculates HG_gr for compatible HG_IDR values. This can be done in the following way.

For example, the metadata could reserve an 8-bit codeword for the HG para function, i.e. in this case, since the IDR image + its metadata must be an ETSI2 compliant signal, the question is whether the required HG_IDR will fit in the dedicated code. The standard usually uses a code allocation function to convert a physically necessary HG_IDR to some HG_COD: HG_COD in [0,255] = F_COD[HG_IDR]. For example, the FCOD may be 128*(4*HG_IDR), which means that a maximum of 255 corresponds to a maximum HG_IDR of 0.5.

You need to make sure that the IDR image is formed in such a way that the HG_IDR simply "fits" into the code range, i.e. in a pragmatic embodiment, this can be implemented by somewhat adapting the HG_gr of the scaler (so as to avoid overflow using fixed channel adaptation and the IDR metadata definition based on it).

The calculations for this (optional) embodiment could, for example, be:

Set HG_IDR = (254*2)/(255*4);

Exposure = shadow/4+0.5 [with shadow being the ETSI2 encoding for shadow enhancement SG_gr]

SG_gr = K*exposure

A = SG_gr*(HG_IDR-1)-0.5*(SG_gr-1)*(1-sc*)*(HG_IDR+SG_gr)

B = SG_gr-HG_IDR+1+0.5*(SG_gr-1)*(1-sc*)*(HG_IDR+1)

C=HG_IDR-1

MxLM = [-B+sqrt(B*B-4*A*C)]/(2*A)

IF MxLM = 1, TO HG_gr_LM = 0

ELSE HG_gr_LM = max[0, (MxLM*SG_gr-1)/(MxLM-1)]

Where HG_gr_LM is the adjusted value of HG_gr. The rest of the algorithm will work as described above, as if the calibrator had chosen the optimal HG_gr_LM value from the start.

One method for solving the problem of developing a new SLHDR2PLUS codec is described in detail here. There are alternative ways depending on the technical choice made, in particular which aspects are considered key, which other aspects can be relaxed.

The above mathematical model defines a completely new way to implement an HDR decoder, whether or not it supports at least a basic computational approach compatible with ETSI1 and ETSI2 approaches: in particular, although the P-LUT and C_LUT functions of different shapes will computed as shown above (although FIG. 4 and FIG. 15 detail the physico-technical philosophy behind how our HDR coding approach works, and why, in practice, all luma processing that is equivalent to luminance processing [in the one-dimensional color aspect, both of them are related through a non-linear image-dependent functional transformation] in the luminance processing channel 401, respectively 1501, is performed by loading the correct general form of the luminance mapping function P_LUT, and similarly for the C-LUT called F_C [Y] , in blocks 451 and 1551, respectively), the computational topology is reusable, which is a very useful property for consumers (they have to buy an IC, such as STB, once, and it can be reconfigured for various new coding philosophies, reprogramming the metadata processing, but retaining the per-pixel color conversion engine).

It is also possible to develop an IDR coding technique that reuses the same ETSI2 decoding math model in full detail (i.e., the sequence of partial recalibrations 1503-1505) by simply instructing the ETSI2 decoder to appropriate extrapolation instead of its usual task of degrading the received image, and the display adapts it to display with PB_D<PB_IDR. It should be emphasized that this is not a "blind" extrapolation that gives "just any" kind of image with a higher dynamic range, corresponding to the kind (i.e., in particular, the statistical distribution of the relative brightnesses or absolute luminances of IDR pixels) of an IDR image, but actually creates "automatically" with this encoding method an output HDR image that looks as close as possible to the original Mster_HDR image on the content creation side (which also in such embodiments is still not actually received, nor is it metadata, such as SG_gr). This, of course, is not so simple by default and requires a proper approach on the content encoding side. For the decoder in embodiments of this philosophy, the received secondary peak luminance PB_C_H50 is used in programming the main per pixel decoder just as if it were the desired display luminance PB_D (which is then, for example, 5 times higher than PB_IDR).

FIG. 18 explains this approach (a flowchart of how the encoder math model works conceptually). Also, for simplicity, assume (although these choices are not necessarily related to this example) that the freedom to choose a fixed channel adaptation algorithm has been defined to perform only the para function transformation linking Mster_HDR and IDR, while leaving any BLOs and WLOs ( if it is still appropriate for the current image or image frame) and a custom curve for the secondary transformation, i.e. recalibration from IDR to SDR, and passing the metadata belonging to the ETSI2 compatible IDR signal to receivers (whether legacy ETSI2 receivers or SLHDR2PLUS decoding receivers).

First, we need some introductory definitions:

The inverse of the para curve shown in FIG. 10, i.e. a curve whose shape is defined according to the ETSI standard as formulated in Equations 4 and 5 above, and whose parabolic middle part is given as a*x^2+b*x+c, is a curve, which in this document will be called abcara for brevity. According to section 7 of ETSI1 (HDR signal recovery), it is defined as:

L_out = 1/SG*L_in (if 0<=L_in<=xS)

L_out = -b/2a+sqrt(b^2-4*a*(c-L_in))/2a (if xS<L_in<xH)

L_out = 1/HG*(L_in-1)+1 (if xH<=L_in) [Lv. 23]

In this case, xS and xH are the points where the linear segments go to the parabolic middle segment, in accordance with how the para function was defined for encoding (or any other use).

The basic principle of what the embodiment of the video encoder of FIG. 18 is shown in FIG. 20 (in this example, an exemplary PB_C of 500 nits IDR has been chosen for clarification, but without the intention of indicating that this method is somehow limited or better suited to a lower PB_IDR).

If we have a fixed mechanism (either in an ETSI2 compliant decoder or a legacy ETSI2 decoder) to extrapolate from IDR to higher PB_Cs than PB_IDR (using the PB_C setting as if it were peak display brightness), then also one could develop an encoder that inverts this process, i.e. creates an IDR image using the F_ENCINV_H2I inversion of a suitably adapted luminance mapping extrapolation function F_E_I2S (adapted from the F_I2S function that conforms to the ETSI2 specification adopted by IDR signal receivers, i.e. IDR image + metadata including the F_I2S function), and then adds the correct metadata, which is said to be F_I2S, and which must be derived from a generic luminosity mapping function F_H2S (e.g. F_50t1) created by a content creator, e.g. a scaler, or an automated tool in any intermediate encoding process in real time, etc.

Relations can also be formulated in multiplicative representation:

L_SDR = m_F_I2S*m_F_ENCINV_H2I*L_HDR = m_F_I2S*L_IDR

L_HDR = m_F_E_I2S*L_IDR

Here, m_F_I2S or rather m_F_I2S(L_HDR) is the appropriate multiplier needed to implement luminosity recalibration for any selected L_HDR value corresponding to the shape of the F_I2S luminance mapping function, and similarly for other multipliers.

Thus, it is necessary to explain that the function inverse to the para function acting from HDR to IDR, i.e. the abcara function acting from IDR to HDR has the same effect as some para functions extrapolated to PB_HDR (starting from any L_IDR).

To understand this somewhat better, use FIG. 21. In normal interpolation mode from a higher image input value PB_C (i.e. when working with any normalized input luminances L_in_X that match the actual luminosities via PB_Ch that is higher than the PB_D of the normalized output image luminances) to a lower value PB_D of the original the para F_H2S function of the calibrator (received in the metadata via the standard ESTI2 encoded video link sequence) would be scaled diagonally, following the arrow towards the [0,0]-[1,1] diagonal, resulting in F_ENCIV_H2I (which now matches visually unified ratio of the pseudo-log distance PB_IDR/PB_HDR to PB_SDR/PB_HDR, i.e., for example, v(100/5000)/v(500; 5000) = 0.54/0.72 [where v(x; y) is a function from equation 1 with abscissa x and RHO corresponding to y through equation 2]). It can be imagined that continuing to recalibrate like this, from any higher PB_D situation to any lower PB_D situation, using PB_HDR to PB_HDR identity processing mapping, would result in the curves falling off steeply, in fact for luminosity mapping curves. of the para function type, they would mathematically become abcara functions. Indeed, the required function F_E_I2S to extrapolate any received IDR image (based on the initial luminance mapping function F_H2S received in the metadata using the display adaptation mechanism from ETSI2 chapter 7.3) would be the mirror function obtained by diagonally mirroring F_ENCINV_H2I (and vice versa ).

Therefore, given that one wants to reuse the standard ETSI2 calculation mechanisms to implement SLHDR2PLUS functionality, it remains only to determine the appropriate encoder, as explained in FIG. 18.

For example, the SG for F_ENCINV_H2I corresponds to the abcara function definition: 1/SG*L_in_X.

The outcome from SG_COD (i.e. the ETSI-specified codification of the above physical and mathematical gain SG of the shadow), we get (SG_COD = SGC*255/2, taking into account ETSI1 level C23 exposure = SGC/4+0.5, taking into account C24 expgain = v (PB_HDR = 5000/PB_target = 500; PB_target) based on C27 SG = expgain*exposure):

1 / [(SGC/4+0.5)*v(5000/500; 500)] = (X/4+0.5)*v(500/5000; 500) [Lv. 24]

The above equation is to be solved to find the unknown para parameter shadow gain control parameter X (ie, X is SG for F_ENCINV_H2I).

Those. the decoder determines, for any choice of F_H2S by the scaler, what the form F_E_I2S will look like (using the ETSI2 7.3 algorithm), but we need to interpret this as an ETSI1 abcara function so that we can associate this abcara function with the appropriate inverse function para F_ENCINV_H2I so that to eventually use the appropriate para function in the new SLHDR2PLUS encoder to calculate the luminosities of an IDR image (in the first preferred embodiment of this approach to a specific variation of the general SLHDR2PLUS approach, i.e. derived calculation of luminance mapping functions using the second peak luminance ; white and black offsets will be ignored in this flavor, at least in the HDR<>IDR sub-band, because they will be applicable to the HDR<>SDR sub-band of another image spectrum with PB_C, as shown in FIG. 7).

In practice, the encoder functions in a different order (but with the same relationships to keep the system ETSI2 compliant). The channel adaptation block 1801 calculates (based on the received form of function F50t1) the para function needed to convert L_HDR luminances to, for example, PB_C L_IDR luminances of 500 nits (you can use the channel adaptation mathematical model of the above previous embodiment, but ignore WLO and BLO adaptation , i.e. the para function simply operates between two brightness representations 0-1.0 without any offsets, but simply by applying only the para functions). The inversion block 1802 calculates the corresponding function abcara using the inversion of Equation 24 (ie, when 1/X on the left is computed given the known SGC on the right side of the equation). This is the mapping that will reconstruct the L_HDR pixel luminances from the received L_IDR luminances. Assuming, for example, that WP remains constant in the codec determination sequence, inversion block 1802 will therefore calculate shadow gain SG_abc and highlight gain HG_abc for function abcara. The lower channel performing metadata management will eventually need to calculate F_L_IDR (= F_I2S), so adaptation block 1803 determines the necessary mapping function F_I2S (in particular its SG_IDR and HG_IDR) by applying the ETSI2 7.3 algorithm backwards (implementing the remaining F_I2S transformation from the full F_H2S conversion if partial luminosity regrading has already been done for IDR image luminances with F_ENCINV_H2I).

As mentioned above, in some scenarios it may happen that the value of HG_IDR is higher than what can be encoded according to ETSI2 as HG_COD. What can be done in such a scenario is to limit the indicated value of HG_IDR to its maximum and revert back in sequence to what it means, in particular, what other original calibrator's F_H2S function would this correspond to. Then all calculations can be restarted from this situation, and this is exactly what the optional blocks shown in dashed lines do in one line of sequential processing.

FIG. 22, it is explained that the limiter 1804 is used to change the shape of the luminosity display curve. The dashed line shows the initial F_H2S and how the F_ENCINV_H2I function can be derived from it using the fixed channel adaptation algorithm, and also how the (original) remaining F_I2S_or partial recalibration function (original F_IDR in case there were no additional specific restrictions in ETSI2, which are currently formulated to require a more detailed approach to a specific implementation option, as explained now). The HG_IDR_or of this function, given that this is a completely new approach to HDR video coding, may not match the HG_COD definition, i.e. require a value above its 8-bit maximum of 255 that can be carried in ETSI2 compliant HDR encoded video. Therefore, HG_IDR_or must be lowered to a maximum of the still encoded value HG_IDR_LIM (which is 2.0 in current ETSI2 implementations, but this is not a fundamental limitation of the approach). This creates a para function that has a linear cut for the highlights a little closer to the upper horizontal border (L_out_X = 1.0), which corresponds to slightly brighter IDR images, but this is not a fundamental problem (as mentioned above, there is some room for easing in the system for designing various options). This will mean that the areas with the highest luminosity in the HDR scene image will get a lower contrast IDR representation (although the original primary HDR is fully recoverable, and the SDR view and all MDR regradings will also look good), but this is not a real problem, since the graduation is done from the higher main HDR PB_Cs and this corresponds to being in the range of 3000-5000 nits, for example, which lamps and the like usually have, which may have a slight degradation (because in any case it is always required some degrading mapping, and this is partly expected for such superbright regions). The second channel adaptation block 1805 will then again apply all of the above mathematical model, but now considering the limited HG_IDR situation (so that an equivalent F_H2S can be calculated first, which, as said in this category of embodiments, can be done by extrapolating the limited F_I2S_LIM to PB_D = PB_Mster_HDR, after which channel adaptation can be applied again).

This resulting F_H2I_LIM (i.e. mapping L_HDR luminances to L_IDR luminances) can now be applied by image pixel luminance mapper 1806 to determine, pixel by pixel, all IDR luminances (or in fact all YCbCr IDR colors using also ETSI2 chromatic processing, i.e. e. C_LUT set therein, corresponding to the shape of the luminance display function F_H2I_LIM). Finally, the IDR metadata determiner 1807 computes a complete set of metadata to implement regrading based on ETSI2 compliant metadata for lower PB_C images (for any display PB_D), i.e. lower than PB_IDR (or by extrapolation higher than PB_IDR). So, again SG_IDR, HG_IDR and WP_IDR are defined according to any of the possible combinations forming the embodiment as explained above. BLO_IDR and WLO_IDR are now also defined (as explained above, a particular brightness on the Mster_HDR brightness axis can be mapped to 1.0 on the SDR brightness axis, and this can be reformulated as a display of a suitably scaled IDR brightness, i.e., a WLO_IDR definition, and similar for BLO_IDR).

Finally, the custom curve can be optimized for the new IDR metadata situation using the custom curve optimizer 1808 (in the case of custom curves, because in some implementations of codec technology available in sub-markets, such as real-time broadcasting, it is possible , it was decided never to use custom curves, and then the previous math model "para + offset function" is applied).

FIG. 19 explains how the custom curve adaptation works. It always consists of two conceptual components (applied directly in only one direction or in the opposite direction). The first component can be understood by focusing on an object: suppose for a moment that one of the control points of an adjustable curve with many line segments corresponds to a pair of trousers (so a particular normalized brightness xo1I L_in_S is, for example, the average brightness of all the pixels of the trousers). A transform is used, for example, to lighten those pixels of the pants (around and in a particular one of the control points) to output normalized luminances that the grading technician (or automatic software) thinks are better than the luminances for those trousers. Also in FIG. 4, it can be seen that in the ETSI approach this occurs as the last (optional) step of fine-scale coding in the encoder (block 405) and, accordingly, the first step in the decoder. So, in fact, this luma transform is actually defined in the SDR luminance domain (after a rough luminance transform from HDR to SDR with respect to the pair+offset function, if any).

Thus, we can assume that for any brightness a transformation is required (for this object!), which can be written in multiplicative representation as L_out = m(L_in_SDR)*L_in_SDR.

The required multiplicative change in brightness (percentage) may be different for any other image, such as an IDR image, but what you can rely on is that the fine grading correction corresponds to the specific "object" for which re-evaluation is needed (even if a custom curve is used for another one of its benefits in addition to specific fine object grading, such as improving the shape of the luminosity mapping curve at coarse grading, it can still physically be interpreted as such an object-oriented improvement, be it by defining a set of virtual objects corresponding to some brightness subrange). Thus, if you track objects in a different DR brightness range, the normalized abscissa value may change, but not the main essence of the object (for example, for a person on a motorcycle, the normalized brightness in HDR, namely 5/5000, differs from that in SDR, but exactly 5/100). So we have to recalculate the function for this new normalized luminance position distribution (this can be done for any number of intermediate partial regrading luminance mapping functions, even up and down the different partial channels, no matter how difficult it is to design a video coding implementation HDR). Thus, in FIG. 19a shows this in a general way: the original brightness xo1I of the SDR object (for example, the end point of the linear segment of the adjustable curve) is moved to xo1N (this could happen, for example, by applying the function abcara, which is the inverse of F_I2S of FIG. 20). The same happens with other points, such as the segment point of a pentagon (usually it can be assumed that there are sufficiently well-scattered segment points, for example 16, of which, for example, 10 can be automatically set by the calibration software if the calibration specialist , for example, applies coarse linear tunable recalibration over a relatively large sub-range of lower brightnesses). Thus, having shifted all these points, it is now possible, based on the original CC_gr curve of the main content metadata grading block (F_H2S with CC in the SDR luminance range), to determine the intermediate CC_XRM curve by applying the original CC_gr offsets, i.e. L_out_SDR = CC_gr[L_in_S], where the L_in_S values were the original xo1I values, and so on. (but now the L_out values are applied to the remapped IDR xo1N luminance positions (resulting in a dotted curve). Of course, these will not be appropriate display multipliers from HDR to IDR (or more precisely from IDR to IDR), so the correction is done in step 2 as shown in FIG 19b.

As can again be seen in FIG. 19b, multiplicative fine-tuning can be interpreted as a scalable process that switches between no correction (Mster_HDR pixel brightnesses are already correct by default, because this image was optimally graded by the content creator at the very beginning) and full correction for the most strongly different (from Mster_HDR) images with PB_C in the spectrum of re-graded images, which in Applicant's approach is typically a 100 nit SDR image (in which the total correction for a particular pixel is, for example, mso1, which can be written as an absolute offset as well as a multiplicative correction yio1 = mso1*xso1 (any form yio1 = F_L(xso1) of the luminance mapping curve can be formulated as a curve of luminance dependent multiplication values).

Since the multiplicative representation of the correction can be formulated as offsets from the diagonal, where yio1 = xso1, we can introduce a vertical scaling factor:

ScaleVer = max[(1-La)/(1-K); 0] [Lv. 25]

with La and K as defined above.

The required adapted values of the tuning curve are then found as:

yiDA = Min[(yio1-xso1)*ScaleVer+xio1; 1] [Lv. 26]

and the calculation is performed for all xso1 values.

FIG. 27 shows another way to determine the endpoints of a segment of a fine-scaled tuning curve in a technically elegant way for decoders. It has already been described how the parameters of the para curve can be recalculated for coarse grading (and if black and/or white offset is present, but for now let's simplify this explanation by focusing on the para function). Assume that the para function performs a coarse grading from any initial dynamic range to a final dynamic range, such as the LDR dynamic range. Black and white shifting can take into account normalized range differences if needed, so the custom curve simply changes the positioning of the relative luminosities of certain modes along the normalized axes. Therefore, the curve will start at the point (0,0) and, accordingly, end at the point (1,1), and between them there will be points of connection of the segments, in this example 2 points for defining the shape of the curve (for example, (x1, y1)) . It also makes sense that in any representation and its regrading, the number of line segments and points should be the same, since the nature of the modes does not change (the mode of the darkest colors, for example, indoors, may end up with a different (usually perceptually uniform) normalized brightness, for example , in the image with PB_C 200 nits compared to the image with PB_C 1500 nits, but the fact that there are two modes - indoor and outdoor - does not change when recalibrating).

Therefore, to redefine the form of the multilinear regrading function, one only needs to find the appropriate endpoints (xnew, ynew).

It is possible to use another property that should be satisfied (ideally), namely whether the main HDR image will be directly regraded using the general range function FL_50t1 (which in this case will consist of two functions applied in succession: the general function para 2710 and full polyline function 2711), or re-grading is done in two steps, first from the main image with 5000 nits to the IDR image with 700 nits (again using two functions: garnishing function para 2701 IDR and generating multilinear function 2702 IDR), and then downgrading to a 100 nit LDR image from there (using the para 2703 channel function and the 2704 channel polylinear function) and the result should be the same: the same LDR image, because it is the LDR image that always follows generate for the main HDR image, i.e. the one that the content creator encoded and transmitted (using the forms of the downgraded luminosity mapping function). In other words, whichever of all possible x1_MH HDR normalized input luminances is chosen, the final LDR output luminance must be the same. Therefore, this will also be true for input luminances that are mapped (via previous mappings) to the x-coordinate of the multilinear channel function: x1_CH_L. This can be used to recalculate segments, since if the y-coordinates of the ordinate are equal, you only need to calculate x_new for a specific segment of the corresponding polylinear custom curve in a different dynamic range.

Thus, on the coding side, for any input value x1_MH, it is possible to calculate a channel-adapted Y_CHA value using a scaled standardized algorithm. This Y_CHA value will form the corresponding input x coordinate for the next block, which goes into the para function defined for the channel with PB_C, the equations for which were given above. The value of yi_CH is already known, since it is equal to the value of y1_L for a total recalibration from 5000 nits to 100 nits, which of course is known directly on the encoding side, as opposed to the decoding side (done by the calibrator, for example). By doing this for all points of the multilinear function, you can get all its characterizing parameters that will be recorded in the video signal (as part of F_I2sCI).

On the decoder side, the same rationale can again be used to arrive at a slightly different algorithm, since some unknown parameters now need to be computed. So, now it is necessary to calculate the x1_ML values corresponding to the accepted, hence known, x1_CH_L values, because the first step was to restore the general recalibration function(s). As a rule, functions have digital precision, for example, 256 quantized x values (i.e. not specific, for example, two or three points between the segments, but all points, as well as points on the lines between them), so you can simply numerically build a table LUT for all points of the custom curve as it has been tuned, i.e. y1_L of this curve is known, and the required x1_ML corresponds to x1_CH_L.

By mapping LDR luminance to IDR luminance, we get x1_CH for any yi_CH, and this value can be mapped back via the para 2703 function. It is also possible to determine which of all possible x1_MH values corresponds to this Y_CHA value if the para 2701 function and the 2702 polylinear function are known. From the above, it can be understood how to calculate the para 2701 function from the function metadata received at the decoder side, as explained above. We don't know about multiline function 2702 (yet), but we don't need it at the moment. Because the tuned curve 2702 is also known to obey the vertical scaling equation of the standardized algorithm. Any tested X1_MH can be converted to the corresponding X_CHA and the corresponding (and required) Y_CHA value follows from: Y_CHA = (y1_L-x1_ML)*scaleVer+X_CHA and x1_ML can be computed from x1_MH by applying the para 2710 generic function.

Therefore, exactly one corresponding x1_MH value can be found, respectively, for the x1_ML value, which will reconstruct the complete multilinear function 2711. between the main image with 5000 nits and the IDR image with 700 nits can also be determined; from here, everything is decoded, i.e. functions have been determined, and processing of all pixel colors of the IDR image can begin, as explained in FIG. 26.

FIG. 26 again conceptually explains in a generalized way (to provide a better degree of understanding for the reader) everything that the experienced reader could already find in the above detailed explanations. The upper blocks of the channel are related to the recalculation of metadata, i.e. to various steps in the definitions of various luminance display functions (lower block 2650, etc. are those blocks that perform the actual pixel color processing). The two-step calculation is now clearly visible, corresponding to the application of the standardized algorithm in the HDR function block 901, namely the determination of the IDR mapping function generator function 903 in the encoder, but now from the SLHDR2PLUS video decoder side. As explained, the decoders receive the F_I2sCI function, which defines how to implement luminance recalibration between the received selected IDR picture with its PB_CH channel peak luminance and the 100 nit graduation. But we need to define a larger range function between 100 nits and the peak brightness of the main HDR image, for example, PB_C_H50 = 6000 nits, i.e. function FL_50t1 (or, more specifically, a function that has an inverse form compared to that used on the encoding side), which will be used by the calculator 2601 of the original function. But this has not yet been achieved, we need to decode the normalized IDR luminances (or more precisely, in our typical decoding topology, perceptually unified normalized pixel luminances) into the reconstructed luminosities of the main HDR image. Thus, neither the originally received F_I2sCI function nor the FL_50t1 function defines a re-grading between the main picture with PB_C_H50 expressed in nits and the picture with 100 nits, nor the IDR picture with PB_CH expressed in nits, neither of the other two, therefore, we need to define a function F_IDRt50 that will be applied to the received IDR pixel intensities to obtain (perceptually unified) reconstructed pixel intensities YpMstr of the main HDR image, which is what the reconstructor function definition block 2602 will do. It has been shown that the display can be adapted in the form of a display optimization function calculator 2603, shown in dotted line, because, as said, although it will generally also be used in our full-featured SLHDR2PLUS decoding ICs, it is in principle optional for SLHDR2PLUS decoding. The peak luminance of the PB_CH channel will be used to convert normally encoded (e.g., 10-bit YCbCr) IDR pixel luminances into perceptually uniform IDR pixel luminances YpIDR, from which luminance reconstruction will typically be performed in our preferred SLHDR2PLUS ICs (although one skilled in the art understands how the principles of the present invention can be implemented in alternative circuitry or software that does not use perceptual uniformization or other method, etc.). To this end, perceptual uniformization block 2650 applies Equations 1 and 2 with PB_C_H = PB_CH. The luminance up-mapping unit 2651 restores the luminances of the main HDR image simply by applying a specific function, i. e. YpMstr = F_IDRt50(YpIDR). In the event that display adaptation is necessary to create, for example, an image with a PB_C of 350 nits, the display optimizer 2652 simply applies a certain display optimization function to do so, resulting in display-optimized brightness pixels: Yglim = F_DO(YpMstr). These can be converted to the actual normalized L pixel luminosities with a linearizer 2653 that applies inverse equations 1 and 2, but now with, for example, PB_C_DO 350 nits in display optimization instead of PB_CH. Finally, there can typically be an optional luma code generator 2654 that uses the perceptual EOTF quantizer from SMPTE 2084 to produce YPQ output luminances in the popular HDR10 format, if desired.

FIG. 28 provides an exemplary chrominance processing specification (i.e. affecting pixel color saturation rather than luminosity). Assume that a processing device topology is used (eg, typically an integrated circuit within some, eg, consumer device) shown in FIG. 15, in which the chrominance processing (multiplier) should obtain the corresponding multiplicative value for each pixel, or more specifically, each possible YCbCr color of a pixel. This value is determined by the chromatic processing determination block 1551 with a function that allows a different value to be represented for different pixel brightness values, i.e. allows changing the differential saturation in the color range, i.e., in particular, along the Y height of the color range.

FIG. 28 shows an example of such a function F_C(Xi), where Xi is now a normalized representation of Y on a scale ending, for example, in 1.0, which typically represents a fixed value of 10,000 nits. It is a multilinear curve (given solely as a non-limiting example) that defines a saturation factor (Ys=F_C(Xs), eg 0.33) for several possible pixel brightnesses. In this example, the value 1/2 means chromaticity identity, i.e. neither boost nor desaturate. In this particular example, values below 0.5 indicate a chroma boost, and values above a decrease in saturation; therefore, it is clear that this particular saturation enhancement characteristic curve F_C(Xi), which, for example, a content creator or automatic content creator can freely choose with any shape depending on the needs of a particular HDR scene and its images (for example, a night street with colorful characters TL or a fire explosion requiring a different optimal chroma setting to convert an image with a first luminosity dynamic range to an optimally matched output image with a different second luminance dynamic range) reduces the saturation for brighter objects to match the upper gamut, for example.

In this example, it is assumed that our original main HDR image had an encoded peak luminance or maximum emerging pixel luminance of 4000 nits, which corresponds to some normalized uniform PQ value: PQ_4000 = 0.9.

Since there is nothing above 4000 nits in this main HDR image, saturation gain values above 4000 nits generally do not need to be specified, but you can, for example, specify them to remain constant and equal to the Ys3 value of 4000 nits. It can be seen that the original saturation enhancement characteristic curve, denoted by small letters "s", is defined by the points (Xs1, Ys1), (Xs2, Ys2), etc., with the X coordinate being the normalized brightness (on a perceptually unified axis, using in example SMPTE 2084 EOTF) and the y-coordinate is the applicable multiplicative gain for that X value, i.e. 0.88 multiplied by Cb and Cr in the same way.

This initial situation and the initial saturation gain function 2801 determine how the pixel chromaticities should be handled when moving from the first reference dynamic range, in this case HDR PB_C 4000 nits, to the second reference dynamic range, which in this example corresponds to a typical PB_C_SDR image with 100 nits with its brightnesses and chromaticities of Cb, Cr pixels.

The task of SLHDR2PLUS again is to represent this as effectively a separate processing associated with the channel PB_C, for example, if PB_CH = 600 nits, one should find a channel saturation gain characteristic curve 2804 that corresponds to the original saturation gain characteristic curve 2801 in the sense that it applies the same chroma change as if we had to start with an IDR link image corresponding to the original main HDR image.

In other words, if one maps any pixel in the main HDR image (Y_HDR, Cb_HDR, Cr_HDR) to any secondary dynamic range color, such as an SDR pixel (Y_SDR, Cb_SDR, Cr_SDR), or the corresponding pixel color for an MDR image with 250 nits for optimal display management with PB_D 250 nits (Y_MDR2, Cb_ MDR2, Cr_ MDR2), i.e. since this involves defined and generally metadata-transmitted luminance mapping function(s) and the original F_C function, it is necessary to obtain exactly the same, or at least well-matched, pixel color(s), e.g. (Y_MDR2, Cb_ MDR2, Cr_ MDR2) if you start with the colors (Y_CDR6, Cb_ CDR6, Cr_ CDR6) of the channel image, and then apply the corresponding channel saturation gain characteristic curve 2804 (i.e., this function can then be loaded in block 1551 chromatic processing definitions, and the device itself can start bulk processing of incoming pixels); as well as correctly displaying the channel luminance, as explained in detail in the various possible embodiments above.

And more interestingly, decoders equipped with these luminance and chrominance mapping functions associated with the channel PB_C should be able to recover the original situation (in particular, determine the original saturation gain characteristic curve from the received channel saturation gain characteristic curve), e.g., recover HDR -image, etc. or even perform any mapping up or down to any secondary dynamic range (MDR) but starting from the received IDR image and its pixel colors (and preferably using the same pixel processing topology, preferably as shown in FIG. 15).

On the encoding side, an IDR image for channel transmission may be generated, for example, by performing mapping of a 4000 nit PB_C_master_HDR image to a 600 nit PB_CH image for normalized luminances, as explained by any of the above embodiments in the luminance mapping part (see , for example, F_H2hCI of FIG. 9). This is shown by horizontally moving a distance PL(Xs2) of the main function bar origin point 2802 to the channel-adapted bar point 2803 (the effect of this shift alone is shown by a thin dashed curve, which is simply shown but not technically used). Since the mapping from a 600 nit IDR image to a 100 nit image implements only part of the full chroma processing from 4000 to 100 nits, the encoder still needs to calculate new output Y values for the points corresponding to the vertical shift (e.g., point 2805 of the channel-adapted value amplification of the chromaticity that corresponds to the circle above it, i.e. has the same Xc value).

The saturation enhancement function must be changed in the first place in order to map the main image to the channel view (for example, PB_C_MHDR = 4000 nits in PB_CH = 600 nits), and this according to this example is preferably done as follows:

First, the modification factor is calculated according to

MF = 1-(invPQ[PB_CH]-invPQ[PB_SDR]/(invPQ [PB_C_MHDR]-invPQ[PB_SDR) [Lv. 27]),

with PB_SDR = 100 nits in this example, and invPQ is an abbreviation for the inverse PQ EOTF function standardized in SMPTE 2084.

The channel-adapted function g(Xn) is then computed and is defined as:

g(Xn) = F_C[Xn]*MF+(1-MF)/Rs [Lv. 28]

where Rs is a constant that is typically chosen to be 2.0, and where Xn is any possible normalized representation of the brightness on the PQ scale for the brightness of a pixel, as shown in FIG. 28.

The new processing LUT to be loaded into the chromatic processing determination block 1551 may, for example, be defined in the preferred embodiment as

F_C_chadap[Xn] = Min{LC0; (1+MF*POWER[Xn; 2,4])/(Xn*Max[Rs/255; Rs*g(Xn)])*S} [Lv. 29]

Wherein, F_C_chadap is an adapted saturation gain function corresponding to the F_C of the original content creator for the main HDR content, but for a particular PB_CH and the corresponding intermediate dynamic range image, LC0 is a constant, which is preferably 1/8 for example, POWER denotes a power function, Rs is another constant, which is preferably equal to 2, and S is a constant depending on the number of bits that the word length of the luma codes uses, S = 1/(POWER [2; word length]), for example, 1/1023 for 10 -bit input images.

What is interesting is the power function in the numerator in Equation 29. This is an example of a well-functioning function for correcting for non-linearity in saturation behavior resulting from the codification of luminances and chroma in the region defined by PQ (as different from the normal behavior of the square root function as and at one time the original definition of YCbCr in SDR/MPEG).

The modification step of the first method is only the starting point for further calculation of the saturation gain characteristic curve 2804 (actually, this is the "half" of the conversion from 4000 nits to 600 nits - starting from the original specification from 4000 to 100 nits - but what we are really interested in is the characteristic curve 2804 saturation gain from 600 to 100 nits, in connection with which it is necessary to divide the total action into two parts and, having already calculated the first part at the first stage of modification, we now calculate the remaining part of the entire set).

To obtain a 2804 curve, the SLHDR2PLUS encoder must do two things.

First, he needs to calculate new definition points for the linear segments of the F_C_chacomm[Xc] curve 2804 (or similarly with the continuous representation of the F_C curve), tracking such points by displaying the brightness.

In other words, each seed point (from the original 4000-100 nit F_C curve supplied with the metadata), eg Xs2, needs to be mapped to a new normalized Xc2 point, and so on.

This is done by applying the overall luminance mapping PL(Xs2) as determined in any situation of the embodiment described above, i.e. the PL curve is the F_H2hCI curve of FIG. 9.

For example, if the luminance mapping was only determined using the para function, a suitably deformed para function (according to the above equations) will be used to display the positions of the PB_C_MHDR 4000 nit normalized luminances [denoted by the subscript "s" in FIG. 28] to the respective positions of the normalized luminances PB_CH 600 nits [indicated by the subscript "c" in FIG. 28]. If the luminance mapping also included a white offset and/or a custom curve shape, etc., all of these would need to be similarly adjusted for the 600 nit situation and the corresponding positions (Xc…) of the 600 nit normalized luminances would be calculated, from where curve definition 2804 starts .

But adjusting the chroma for all of these (redefined) Xc positions requires only the remainder of the correction from 600 nits (or any other PB_CH IDR) to the lowest dynamic range reference level, which is typically a PB_SDR of 100 nits.

Yc values are calculated according to

Yc = Ys/{Rs*(Ys*MF+(1-MF)/Rs)} [Lv. thirty]

It should be noted that Ys = F_C[Xs].

It should be noted that this only explains the completion of the more important part of the curve, namely the situation:

Xs<=invPQ[PB_C_MHDR]; for normalized PQ luminances above, for example a value of 0.9 corresponding to an approximate PB_C_MHDR = 4000 nits, it is preferable to use the following equation to maintain the correct scaling ratio:

Xc = Xs*invPQ[PB_CH]/invPQ[PB_C_MHDR] [Lv. 31]

The Yc values for these upper Xc values will be calculated in the same way or kept constant at the last significant value if this part of the curve is not used in practice.

Finally, some rounding to some level of quantization can usually be used, for example:

(Xcr, Ycr) = round[255*(Xc, Yc)]/255 [Lv. 32]

The SLHDR2PLUS decoder needs to recover the original pairs (Xs, Ys) from the transmitted and received pairs (Xc, Yc), namely the pairs (Xcr, Ycr) of defining the F_C_chacomm[Xc] curve (i.e., channel saturation gain characteristic curve 2804). Therefore, two reverse encoder steps are applied.

First, the decoder needs to remap the definition points of the linear segment of the saturation gain curve from their Xc positions to the Xs position. It has been shown above how to calculate the luminance mapping from IDR, for example, PB_CH luminance positions of 600 nits, to the original luminance positions of the main HDR image, starting from the channel-adapted luminance mapping function which is adopted in the metadata co-transmitted with the IDR images.

We will call this operation in the PQ region (as shown in FIG. 28, the values of Xs and Xc are determined in the PQ region using SMPTE 2084):

Xs = PQICA(Xc) [Eq. 33]

with the form of this inverse channel adaptation function again depending, among other things, on which definition of the luminance mapping function was used, for example, the para function alone, and not, for example, the para function together with the adjustable curve, and, on the other hand, specific parameter values such as ShadowGain defining the particular para function that applies to the HDR cave exploration scene, etc. (all of which have been explored in detail for the various embodiments above).

In this situation, the adjustment again looks like this:

If Xc>invPQ(PB_CH), then Xs = Xc*invPQ[PB_C_MHDR]/invPQ[PB_CH] [Lv.34]

The required Ys values are calculated according to:

Ys = (MF-1)*Yc/(Rs*MF*Yc-1) [Eq. 35]

Finally, rounding can again be involved, for example in increments of 1/255.

Finally, although it is useful for recovering only the original colors (Y, Cb, Cr) of the main HDR image from the received IDR image, it is preferable if the SLHDR2PLUS decoder can immediately convert to (i.e. calculate the corresponding colors, for example, ( Y_MDR2, Cb_ MDR2, Cr_ MDR2)) any required medium dynamic range image for optimal display control with PB_D_MDR, for example 250 or 400 nits, and do this of course in one go, i.e. starting with channel-adapted colors (Y_CDR6, Cb_CDR6, Cr_CDR6) IDR; i.e., for example, by loading the respective luminance processing functions (or LUTs, etc.) and the chrominance multiplier determination function F_C_MDR into the pixel color processing topology as shown in FIG. 15.

To this end, the particular one of Equations 29 which defines the F_C function can be advantageously applied.

Both the numerator and the denominator must be adapted to the new situation, i.e. new chrominance mapping from PB_CH to PB_MDR, the latter being 250, for example, when a 250 nit display needs to deliver an optimal display-optimized image (of what was once the original HDR main image, and, speaking of a decoder, corresponding an incoming IDR image at 600 nits, neither of which is yet good for display on a 250 nit display).

For this, two modification coefficients are first calculated:

MF_ch = 1-(invPQ[PB_CH]-invPQ[PB_SDR]/(invPQ[PB_C_MHDR]-invPQ [PB_SDR) [Lv. 36]

MF_do = 1-(invPQ[PB_MDR]-invPQ[PB_SDR]/(invPQ[PB_C_MHDR]-invPQ[PB_SDR) [Lvl 37]

If CPL[Xc] is the luminance display function, which is defined as follows:

For each normalized input luminance Xc defined with PB_CH:

First apply EOTF PQ, then transform to perceptual domain using Philips Proposed Perceptualizer Function (PPF) with RHO parameter for PB_CH value, then apply luma mapping function form from PB_CH back to PB_C_MHDR as described in accordance with any of the possible implementations and shapes of the content creator function, then convert to linear domain using the inverse of the Philips Proposed Perceptualizer Function (PPF) but now with the RHO parameter set to PB_C_MHDR, and finally use the inverse EOTF PQ according to SMPTE 2084, which will result in the corresponding CPL[Xc] value.

Then the CLY[Xc] function is calculated as:

CLY[Xc] = (1+MF_do*POWER[CPL[Xc]; 2.4])/(1+MF_ch*POWER[CPL[Xc]; 2.4]) [Lv. 38]

This includes adjusting the old (no longer applicable) situation of this part of the function (usually C_LUT) to determine the chroma multipliers, which we will call the saturation position adjustment (FSNL) function for the new situation of this equation, in order to optimize the display.

The two g-functions are then computed as follows:

Gch[Xn] = F_C_[CPL[Xc]]*MF_ch+(1-MF_ch)/Rs;

Gdo[Xn] = F_C_[CPL [Xc]]*MF_do+(1-MF_do)/Rs [Lv. 39]

(wherein F_C_[CPL[Xc]] is the original content creator's chroma multiplier resulting in the original saturation gain function 2801 calculated from F_C_chacomm[Xc] being the chroma adjustment curve received by the SLHDR2PLUS decoder in the metadata, i.e. corresponding to curve 2804, i.e. as performed, for example, in accordance with the above calculation of points (Xs, Ys)).

Finally, the C-LUT resulting in suitable chroma multipliers for optimizing IDR-to-MDR mapping is computed as:

F_C_DO[Xn] = CLY[Xc]*max{Rs/255; Rs*Gch[Xn]}/max{Rs/255; Rs*Gdo[Xn]} [Lv. 40]

This function F_C_DO[Xn] can be directly loaded into block 1551 before a new incoming image arrives to start a pixel color processor to obtain a properly displayed optimized MDR image in time for display or the like, such as storage (one skilled in the art the technician understands that other implementations are possible, but then all calculations need to be modified accordingly).

Although some embodiments/ideas have been presented to clarify certain aspects that may be modified, individually or collectively, it can be appreciated that several additional embodiments may be generated based on the same basic principles: re-deriving luminance mapping equations from different image metadata with intermediate dynamic range received in accordance with ETSI2 HDR video link or similar, in order to restore the main HDR image that has been optimally graded at the point of content creation. The algorithmic components disclosed herein may be (in whole or in part) implemented as hardware (eg, parts of a dedicated IC) or as software running on a dedicated digital signal processor or general purpose processor, etc.

One skilled in the art should understand from the description above which components may be optional enhancements and may be implemented in combination with other components, and how the (optional) method steps correspond to the corresponding device features, and vice versa. The term "device" is used in this application in its broadest sense, namely as a group of means enabling a specific purpose to be realized, and therefore can be, for example, a (small circuit part) of an IC or a dedicated device (e.g. a device with a display) or part of a network systems, etc. The term "arrangement" is also intended to be used in its broadest sense, so it may include, among other things, a single device, a part of a device, a set of cooperating device(s), and so on.

The term "computer program product" is to be understood as encompassing any physical implementation of a set of instructions enabling a general purpose or special purpose processor, after a sequence of loading steps (which may include intermediate transformation steps such as translation into an intermediate language and the final language of the processor), to enter instructions into the processor and perform any of the characteristic functions of the invention. In particular, a computer program product may be embodied as data on a medium such as a disk or tape, data present in memory, data transmitted over a network connection, wired or wireless, or program code on paper. In addition to the program code, the characteristic data required for the program may also be embodied as a computer program product.

Some of the steps required for the operation of the method may already be present in the functionality of the processor instead of those described in the computer program product, such as data input and output steps.

It should be noted that the above embodiments illustrate and do not limit the invention. Where one skilled in the art can easily map the presented examples to other areas of the claims, all of these options have not been mentioned in detail for the sake of brevity. In addition to combinations of elements of the invention, as combined in the claims, other combinations of elements are possible. Any combination of elements can be implemented in one specialized element.

No reference position in brackets in the claims is intended to limit a claim. The word "comprises" does not exclude the presence of elements or aspects not listed in a claim. The singular form for an element does not preclude the presence of multiple such elements.

Jargon and abbreviations used:

PB_C: maximum encoded image luminosity, usually specified for any situation, with "C" meaning encoding (not to be confused with bit depth), e.g. an HDR image can have PB_C_HDR = 4000 nits (which also defines all relative luminances below because L_norm = L/PB_C where L_norm is the normalized luminosity lying between 0.0 and 1.0

PB_D: The maximum displayable luminosity (also known as peak brightness) of any display, for example current HDR displays typically have a PB_D of 1000 nits (but nowadays you can also get values up to 600 nits or up to 2000 and even 4000 nits, and in the future PB_D may be higher).

IDR (Intermediate Dynamic Range): A mechanism for representing an image that was originally (i.e., the main image) defined with PB_C1, e.g., 10000 nits, actually as a secondary HDR image with PB_C2<PB_C1 (i.e., typically a factor of 2 or even lower, and PB_C2 is usually >= 500 nits).

MDR (Medium Dynamic Range; certainly not to be confused with IDR): An image with PB_C_MDR, typically lying between the PB_C of the received HDR image (PB_C_H) and PB_C_SDR = 100 nits (as agreed upon in the video processing field), with PB_C_MDR set to PB_D any display (thus an incoming HDR image that has an incorrect dynamic range, and therefore, more importantly, an incorrect relative statistical distribution of normalized brightnesses relative to each other, can be optimally recalibrated for a particular available display with lower dynamic range capabilities, i.e. PB_D<PB_C_H).

Para: a specific very pragmatically useful function for mapping luminances defined in the first range of normalized luminances corresponding to PB_C1 to normalized by PB_C2, the function being defined in this document above with Equations 4 and 5 and a parabolic intermediate or formally in ETSI TS 103 433 -1 V1.2.1 (2017-08) [abbreviated as ETSI1], see page 70 S-20.

Abcara: An inverse function for any para function (i.e. with parameters that uniquely define its form), whose inverse form can also be intuitively found by rearranging the axis (but sometimes it needs to be calculated mathematically).

WLO (white level shift): normalized brightness in the normalized brightness range of the first image (im1), which is mapped to 1.0 in the normalized brightness range of the second image, with PB_C_im1>PB_C_im2. In this application, there are several different WLOs for different images with different PB_Cs when defining the encoding process, so to make them easy to distinguish, they contain suffixes, for example, WLO_gr.

BLO (Black Level Offset): The normalized brightness in the normalized brightness range of the first image, which is mapped to 0.0 in the normalized brightness range of the second image, with PB_C_im1>PB_C_im2. In this application, there are several different BLOs for different images with different PB_C when defining the encoding process, so to make them easy to distinguish, they contain suffixes, for example, BLO_IDR.

P_LUT: the full mapping (consisting of the partial recalibrations in our codec approach as explained with FIG. 4) needed to convert any possible normalized luminance of the first image to the corresponding normalized luminance of the second image, with PB_C_im1 != PB_C_im2 (generally , they differ by at least a factor of 1.02). Since P_LUT[L], which is usually dependent on the content of the image, and, for example, is optimized by an intelligent automatic image analyzer or by a human, changes the relative distribution of the normalized luminances, i.e. histogram is a key aspect of dynamic range transformation, such as that involved in defining an IDR image, which is key to the current philosophy of the new HDR codec.

C_LUT: A pixel-luminance dependent mapping of the chromaticity (also known as chrominance) of pixel colors, which, together with P_LUT, completes the color transformation (YCbCr_out = T[Y_cbCr_in]).

Philips Proposed Perceptualizer Function (PPF): A function (defined in Equation 1) to parametrically convert luminosities defined in the range 0 to PB_C into perceptually uniform luminosities, where the PB_C value via the RHO parameter is the shape control parameter of the PPF and, hence, a distribution of visually uniform encoding luminances for different input luminosities.

Claims (30)

1. Encoder (900) video with extended dynamic range, configured to receive through the input (920) for the image of the input image (MsterHDR) with extended dynamic range, which has a first maximum luminance (PB_C_H50) pixels, for which the encoder has a first input ( 922) for metadata, and capable of being received via a second input (921) for metadata of a luminance mapping main function (FL_50tl) that defines a relationship between the normalized luminances of the input high dynamic range image and the normalized luminances of the corresponding low dynamic range image (Im_LDR). , having a maximum LDR pixel luminosity, preferably equal to 100 nits, characterized in that it additionally contains a third input (923) for metadata for receiving a second maximum luminance (PB_CH) of pixels, which is lower than the first maximum luminance (PB_C_H50) of pixels, while the encoder further differs in that what contains an HDR function generating unit (901) configured to apply a standardized algorithm for converting the main luminance mapping function (FL_50tl) into an adapted luminance mapping function (F_H2hCI) that relates the normalized luminances of an extended dynamic range input image to the normalized image luminances (IDR) with an intermediate dynamic range, which is characterized by a maximum possible luminance equal to the second maximum luminance (PB_CH) of the pixels; an IDR image calculation unit (902), configured to apply an adapted function (F_H2hCI) of luminance to input image pixel luminances (MsterFIDR) with extended dynamic range to obtain pixel luminances of an intermediate dynamic range image (IDR) that the block outputs; And an IDR mapping function generator (903) configured to obtain, based on the main luminance mapping function (FL_50tl) and the adapted luminance mapping function (F_H2hCI) of the channel luminance mapping function (F_I2sCI), which outputs the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range, when the corresponding normalized image luminances (IDR) with an intermediate dynamic range are given as input, which, in turn, correspond to the corresponding luminances of the input image (MsterHDR) with an extended dynamic range; while the encoder is additionally different in that it contains: an image output (930) for image output (IDR) with an intermediate dynamic range; a first metadata output (932) for outputting a second maximum luminosity (PB_CH) of pixels; the second output (931) for metadata to output the function (F_I2sCI) display the brightness of the channel and a third output (933) for metadata for outputting the first maximum luminosity (PB_C_H50) of pixels. 2. The encoder (900) of the video with an extended dynamic range according to claim 1, characterized in that the standardized algorithm of the block (901) for generating the HDR function applies compression in the diagonal direction of the main function (FL_50tl) of the luminance display to obtain an adapted function (F_H2hCI) of the display luminance, wherein the compression includes scaling all output luminance values of the function with a scaling factor that depends on the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels. 3. Encoder (900) video with extended dynamic range according to one of the above items for the encoder, which contains a limiter (1804), configured to re-determine the steepness of the function (F_I2sCI) map luminance of the channel for the subrange of normalized luminances containing the brightest normalized luminance , equal to 1.0. 4. Encoder (900) video with extended dynamic range according to one of the above paragraphs for the encoder, in which the block (901) generating HDR functions is configured to determine the characteristic curve of saturation gain depending on the original characteristic curve (2801) of saturation gain and adapted function (F_H2hCI) display brightness. 5. An extended dynamic range video decoder (1100) having an image input (1110) for receiving an intermediate dynamic range (IDR) image that has a second maximum luminance (PB_CH) of pixels that is lower by a multiplicative factor, preferably equal to 0, 8 or less than the first maximum luminance (PB_C_H50) of the pixels of the main image (MsterHDR) with extended dynamic range, and the second maximum luminance (PB_CH) of the pixels is received through the second input (1112) for metadata, while the decoder has the first input (1111) for metadata for receiving a luminance mapping function (F_I2sCI) that defines the transformation of all possible normalized image luminances (IDR) with an intermediate dynamic range into the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range and with a maximum luminance of the LDR pixels, the decoder being characterized in that contains the third input (1113) for met data for receiving the first maximum luminosity (РВ_С_Н50) pixels, while the decoder contains a luminance function determiner (1104) configured to apply a standardized algorithm for converting a luminance mapping function (F_I2sCI) into a luminance mapping decoding function (F_ENCINV_H2I) that determines as output for any possible input normalized image pixel luminance (IDR) with an intermediate dynamic range corresponding to the normalized HDR brightness of the main image (MsterHDR) with extended dynamic range, and the standardized algorithm uses the values of the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels; And a color converter (1102) configured to sequentially apply to the input normalized image luminances (IDR) with an intermediate dynamic range of the luminance mapping decoding function (F_ENCINV_H2I) to obtain normalized reconstructed luminances (L_RHDR) of pixels of the reconstructed main HDR image (RECMHDR); wherein the decoder further comprises a picture output (1120) for outputting the reconstructed HDR main picture (REC_M_HDR). 6. The extended dynamic range video decoder (1100) according to claim 5, characterized in that the standardized algorithm of the luminance function determiner (1104) calculates a scaling factor that depends on the first maximum luminance (PB WITH H50) pixels and the second maximum luminance (PB_CH) pixels. 7. The extended dynamic range video decoder (1100) according to one of the above for a decoder in which the luminance mapping function (F_I2sCI) is defined by a luminance mapping that consists of a first linear segment having a first steepness (SG_gr) for a range of low normalized luminances , a second linear segment having a second steepness (HG_gr) for the range of high normalized brightnesses, and a parabolic segment for brightnesses between these two ranges. 8. The extended dynamic range video decoder (1100) according to one of the above paragraphs for the decoder, in which the color converter (1102) is configured to calculate the brightness of the pixels of an image (MDR_300) with an average dynamic range having a maximum luminosity (PB_MDR) of pixels, which is not equal to the values of the maximum luminance of the LDR, the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels, and the decoder contains an output (1122) for an image to output an image (MDR_300) with an average dynamic range. 9. High dynamic range video decoder (1100) according to one of the above items for a decoder that has a metadata output (1121) for outputting a luminance mapping function (F_L_subsq) that determines for all normalized luminances of the reconstructed HDR main image (REC_M_HDR ) or alternatively, an image (MDR_300) with a medium dynamic range corresponding to the brightness of an image having a different maximum pixel luminance, said different maximum pixel luminance preferably being 100 nits or a value above or below the maximum luminance value, respectively, of the reconstructed HDR main image ( REC_M_HDR) or alternatively images (MDR_300) with medium dynamic range. 10. Decoder (1100) video with extended dynamic range according to one of the above paragraphs for the decoder, in which the block (1104) determine the luminance function is made with the additional ability to determine the original characteristic curve (2801) gain saturation depending on the characteristic curve (2804) channel saturation gain; and a function (F_I2sCI) to display the channel luminance. 11. Decoder (1100) video with extended dynamic range according to one of paragraphs. 8, 9, or 10 for a decoder in which the original saturation gain characteristic curve further depends on a saturation position correction function (FSNL) that corresponds to an equation including a power function for the normalized luminance value. 12. A method for encoding a high dynamic range video for a received high dynamic range input image (MsterHDR) that has a first maximum luminance (PB_C_H50) of pixels, comprising receiving a main luminance mapping function (FL_50t1) that determines a relationship between the normalized luminances of the input image with extended dynamic range and normalized luminances of a corresponding low dynamic range (Im_LDR) image having a maximum LDR pixel luminance preferably having a value of 100 nits, characterized in that the encoding further comprises receiving a second maximum luminance (PB_CH) of the pixels, wherein the encoding includes application of a standardized algorithm to convert the main luminance mapping function (FL_50t1) into an adapted luminance mapping function (F_H2hCI) that relates the normalized luminances of an extended dynamic range input image to normalized image luminances (IDR) with an intermediate dynamic range that differs by the maximum possible luminance equal to second maximum luminosity (PB_CH) pixels; applying an adapted function (F_H2hCI) of mapping luminance to input image pixel luminances (MsterHDR) with extended dynamic range to obtain image pixel luminances (IDR) with intermediate dynamic range; based on the main function (FL_50tl) of the luminance mapping and the adapted luminance mapping function (F_H2hCI) of the channel luminance mapping function (F_I2sCI) that determines the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range as output when given as input corresponding normalized image luminances (IDR) with intermediate dynamic range, which, in turn, correspond to the corresponding luminances of the input image (MsterHDR) with extended dynamic range; image output (IDR) with intermediate dynamic range and outputting the second maximum luminance (PB_CH) of the pixels, the channel luminance display function (F_I2sCI) and the first maximum luminance (PB_C_H50) of the pixels. 13. An extended dynamic range video decoding method for an intermediate dynamic range received image (IDR) that has a second maximum luminance (PB_CH) of pixels that is lower by a multiplicative factor, preferably equal to 0.8 or less than the first maximum luminance (PB_C_H50). ) pixels of the main image (MsterHDR) with extended dynamic range, and the second maximum luminance (PB_CH) of the pixels is received as metadata of the image with an intermediate dynamic range, while the decoding method also includes receiving in the metadata of the luminance display function (F_I2sCI) that determines the transformation of all possible normalized image luminances (IDR) with an intermediate dynamic range into the corresponding normalized image luminances (Im_LDR) with a narrow dynamic range and with a maximum luminosity of the LDR pixels, while the decoding method differs in that it includes receiving the first maximum luminosity (РВ_С_Н50) pixels, as well as those that include application of a standardized algorithm to convert a luminance mapping function (F_I2sCI) into a luminance mapping decoder function (F_ENCINV_H2I) that determines as output for any possible input normalized image pixel luminance (IDR) with intermediate dynamic range the corresponding normalized main image luminance HDR (MsterHDR) with extended dynamic range, wherein the standardized algorithm uses the values of the first maximum luminance (PB_C_H50) of the pixels and the second maximum luminance (PB_CH) of the pixels; applying a luminance mapping decoding function (F_ENCINV_H2I) to normalized intermediate dynamic range (IDR) image luminances to obtain normalized reconstructed luminances (L_RHDR) pixels of the reconstructed HDR main image (REC_M_HDR), and output of the restored main HDR image (REC_M_HDR).

Images