TW202034692A - Multi-range hdr video coding - Google Patents

Abstract

Although recently high dynamic range video codec technologies have been invented, at least some applications in the total HDR video market desire some more sophisticated methods, which can work defined around more than two different dynamic range re-graded images of a same original HDR scene. Thereto the inventors developed a number of technical ways to realize a high dynamic range video encoder (900), arranged to receive via an image input (920) an input high dynamic range image (MsterHDR) which has a first maximum pixel luminance (PB_C_H50) for which the encoder has a first metadata input (922), and arranged to receive via a second metadata input (921) a master luma mapping function (FL_50t1), which luma mapping function defines the relationship between normalized lumas of the input high dynamic range image and normalized lumas of a corresponding standard dynamic range image (Im_LDR) having a maximum pixel luminance of preferably 100 nit, characterized in that the encoder further comprises a third metadata input (923) to receive a second maximum pixel luminance (PB_CH), and the encoder further being characterized in that it comprises: - a HDR function generation unit (901) arranged to apply a standardized algorithm to transform the master luma mapping function (FL_50t1) into a adapted luma mapping function (F_H2hCI), which relates normalized lumas of the input high dynamic range image to normalized luminances of an intermediate dynamic range image (IDR) which is characterized by having a maximum possible luminance being equal to the second maximum pixel luminance (PB_CH); - an IDR image calculation unit (902) arranged to apply the adapted luma mapping function (F_H2hCI) to lumas of pixels of the input high dynamic range image (MsterHDR) to obtain lumas of pixels of the intermediate dynamic range image (IDR) which is output of this unit; and- an IDR mapping function generator (903) arranged to derive on the basis of the master luma mapping function (FL_50t1) and the adapted luma mapping function (F_H2hCI) a channel luma mapping function (F_I2sCI), which defines as output the respective normalized lumas of the standard dynamic range image (Im_LDR) when given as input the respective normalized lumas of the intermediate dynamic range image (IDR), which in turn correspond to respective lumas of the input high dynamic range image (MsterHDR); the encoder being further characterized to have: - an image output (930) to output the intermediate dynamic range image (IDR); - a first metadata output (932) to output the second maximum pixel luminance (PB_CH); - a second metadata output (931) to output the channel luma mapping function (F_I2sCI); and - a third metadata output (933) to output the first maximum pixel luminance (PB_C_H50), and a corresponding HDR video decoder.

Description

Multi-range HDR video encoding

The present invention relates to a video that is used to encode high dynamic range images, and is particularly used for time series of images that can be compressed according to compression techniques like MPEG-HEVC (for example, television broadcasting), especially by using a second dynamic range (Multiple) images (used to be communicated to the decoder) to represent the first dynamic range (multiple) (main) image method and equipment, the dynamic range change involves the application of general data and (multiple) (A) The second dynamic range image collectively conveys the function to change the image pixel brightness (for example, from the first normalized to a value of 1.0 to the second normalized to a value of 1.0).

About 5 years ago, novel technologies for high dynamic range video coding were introduced into the world (for example, special HDR Blu-ray discs that were watched on UltraHD Premium TVs that were 1000 nits).

This novel way of technically processing images is technically in contrast to the many ways in which all videos were coded according to the traditional (legacy) video coding in the past 50 years until then. This traditional video coding is now called Standard Dynamic Range (Standard). Dynamic Range, SDR) video coding (also known as low dynamic range (low dynamic range) video coding; LDR). In order to represent the image, the digitally coded representation of the pixel color is necessary, and the definition of the SDR luma code of Rec. 709 (also known as the Opto-electrical transfer function (OETF)) is because of its approximate square root function shape (square root function shape) (brightness: Y=sqrt (brightness L)) and can encode (using 8 or 10-bit brightness blocks) only about 1000:1 brightness dynamic range. However, this is perfectly suitable for encoding images to be displayed on displays with general brightness rendering capabilities (of all displays at the time) between 0.1 and 100 nits in these times, and the latter value is the so-called peak brightness (peak brightness, PB), also known as maximum brightness.

See Rec. 709 brightness code definition function can not mathematically represent the huge range of HDR scene image brightness (for example, between 0.001 nits and 10,000 nits expected image coding peak brightness PB_C), HDR researchers initially designed new The HDR code allocation solves this problem. The new HDR code allocation is more logarithmic in shape, making it possible to encode more brightness (because the visual system requires lower accuracy, that is, the code value of the brighter area is less than the darker The area is small, so it can be understood that each decimal brightness is allocated with 50 codes in 2^8=256 (where ^ represents the power function), and the dynamic range of 100,000:1 can be encoded). This is a simple "natural" way of encoding HDR image colors by using the SMPTE 2084-standardized (so-called perceptual quantizer (PQ)) function.

It is naive to think that encoding and decoding HDR images is all, but things are not so simple. Therefore, additional encoding methods have emerged, especially the previous development methods of the applicant's HDR video encoding and processing.

To get a proper understanding of what is involved or needed in HDR video coding, Figure 1 summarizes some important aspects.

Assuming that there are all possible (PQ decoded) HDR luminance ranges on the left, at most PB_C=5000 nits. For the time being, suppose that in order to make this image look perfect as expected, all the object pixels called the main HDR image are created on the computer (see Figure 2 below to illustrate how to start from, for example, a broadcast camera). The problem with using the inherent HDR codec (which only provides technology that encodes brightness up to 10,000 nits, that is, as expected up to 5000 nits in this example) is that if the consumer also has an expensive 5000 nit display Peak brightness (PB_D) display (and if he views the image under standardized viewing environment conditions), he can perfectly watch the image intended by the creator (for example, a film director), but if he has a different display (for example, PB_D) =750 nits, or PB_D=100 nits), there is an unsolved and not simple problem: how to display 5000 nits of PB_C images on a 750 nits PB_D display? There seems to be no elegant simple solution. Applying accurate brightness display will perfectly display all objects with a brightness of up to 750 nits, but trim all brighter object pixels to the same PB_D=750 nits, so that many image objects disappear in the white spot area, which of course is not good looking. It can be considered that the linear scaling of content is a solution (divide all HDR brightness by 5000/750, which is the so-called map content-white-on-display-white approach), but it is more Dark objects (such as people with HDR brightness (0.05 nits) in the dark area of the cave in the example scene image ImSCN3, which may have been too low for a smaller dynamic range display), so you get imperceptible darkness on the 750 nit display Brightness (0.05*750/5000= 0.0075 nits).

Figure 1 also teaches that different HDR images (for different typical HDR scenes) can have quite different requirements. These requirements are about how to squeeze various (possibly at an "arbitrary" brightness position along the HDR brightness range DR_2) HDR brightness into The smaller one, for example, LDR luminance dynamic range DR_1.

The real-world brightness range can be, for example, when both indoor and outdoor objects are viewed through illumination_contrast*object_reflection_contrast =(1:100)*(1:100), and although the brightness in the image representing the scene is generally not necessary Or it will not be equal to the original scene brightness. For a good HDR representation image, the range of expected pixel brightness may be at least 1000 nits, and start with a minimum value of at least 0.1 nits or less (so DR_im >= 10,000:1 ). In addition, a good HDR image is more about the intelligent distribution of the brightness of various objects along the brightness range than it is about the dynamic range of the entity itself (not to mention misunderstanding the amount of bits it is guiding, and its definition of non-linear brightness code is not True, and 10-bit brightness images can only be some HDR images and SDR images).

Figure 1 shows several typical illustrative examples of many possible HDR scenarios. Future HDR systems (for example, connected to a 1000-nit PB_D display) may need to be able to process correctly, that is, by displaying all objects/pixels in the image The appropriate brightness.

For example, ImSCN1 is a sunny outdoor image from Western movies (it has most of the bright areas, which are brighter than the average of cloudy images. Ideally, these areas should appear brighter than on a 100 nit monitor to provide a better A more sunny appearance on a rainy day, for example, with an average brightness of, say, 400 nits). On the other hand, ImSCN2 is a very different type of image, that is, a night image dominated by dark areas (and, for example, good visibility). However, the reason why this is an HDR image rather than a simple dark SDR image is also There are bright pixels in the spot under the street light, and possibly in the light-emitting windows of the house, and there are even very bright pixels (for example, 3000 nits) on the surface of the street light.

Relative to the darkness of ImSCN2, what makes this ImSCN1 clear? It is not necessarily the relative brightness, at least not in the SDR paradigm (there will be pixel brightness spreading over the range between 0.1 and 100 nits for both images. , Although the spatial distribution of this kind of brightness, and especially the histogram can be different). What makes HDR image presentation different from how it always did in the SDR era, which ended only a few years ago, is that SDR has this limited dynamic range (approximately PB=100 nits, and the minimum black level is approximately MB). 0.1 to 1 nit), only most of the intrinsic reflectivity of the object can be displayed in SDR (it will fall between 90% for good white and 1% for good black). Under technically controlled uniform lighting, it will be good for recognizing objects (having a certain amount of brightness from their equivalent reflections and of course from their equivalent chromaticity), but for the beauty of the lighting itself that can be possessed in natural scenes In terms of change, and what kind of impact it can have on the viewer (sunlight from a window, or plasma radiated by a wizard) is not so good. If the display allows it (and therefore the image coding and processing technology should be), walking in the forest really sees the sunlight passing through the trees, that is, instead of the slightly more yellowish impression on the SDR display, when people walk from the shadows When you step into the sun, you want to see bright and vivid clothes shining in the sun. And at least when PB_D allows, fire and explosion should also have the best visual impact.

In SDR, the night image can be made darker than the normal lighting image. For example, it can be perceived in the brightness histogram, but it will not be too dark, otherwise it will only be too dark and ugly (maybe most of them are not See) (this is the reason for introducing the conventional method of making night images still relatively bright (but blue)). Moreover, in 100-nit TV or in 100-nit encoding, there is just no room for anyone who is too bright. Therefore, the object must be displayed independently of its lighting, and all the high-contrast lighting that can sometimes occur in the scene cannot be faithfully displayed at the same time. In practice, it means that a highly bright sunny scene must be displayed with almost the same display brightness (0 to 100 nits) as a dark rainy scene and even a night scene.

In real life, human vision will also adapt to the amount of light available, but not so much (most people do recognize gradual dimming in real life, or they are in a darker or fairly bright environment). And it should not be forgotten that the television that displays images is not a simulation of adaptive eyes, but a simulation of real life environments, such as those that are good in view of the viewing environment and other technical limitations. Therefore, it is desired to display the image with a lighting effect that can be artistically designed to all the spectacular parts of the image and time, so as to obtain a more realistic rendering of the image (at least if the end viewer has an available HDR display). Exactly what is the appropriate brightness, such as a lightsaber in a dark room, will be left to the color grader who establishes (multiple) master grading (for the simplicity of the teaching in this patent, assuming various dynamics) Range images, at least two of the most extreme different dynamic ranges are created by human graders, but images can be similarly created by automatic software), and this application will focus on the creation and processing of various market players with potentially different needs The required technical components of such images.

On the left axis of Figure 1 is the brightness of the object that can be directly displayed on the 5000-nit PB_D (reference) monitor when you want to see it with (for example) 5000 nits PB_C master HDR classification (that is, the image grader It is assumed that the general high-quality HDR TV in the home will have an image of 5000 nits PB_D, and he can actually sit in this family viewing room and the hierarchical representation on this hierarchical display). If you want to deliver not only the approximate phantom of the captured original HDR scene, but also the actual scene of a cowboy in a bright sunlight environment, the brightness of these pixels must be specified and the performance must be bright enough, for example, an average of 500 nits .

For night scenes, you want to focus on the dark brightness, but the main characters on the motorcycle should be well-recognizable, that is, not too dark (for example, about 5 nits), and at the same time, such as street lights have a fairly high brightness Pixels, such as around 3000 nits on a 5000-nit display, or around the peak brightness on any other HDR display (for example, 1000-nit PB_D). The third example, ImSCN3, shows which is also possible on the HDR display: it can show many at the same time (more semantically related to not only lights, that is, related to details in many areas, such as sunlit trees) very bright and many Both important and very dark pixels. ImSCN3 shows what can be regarded as a typical and relatively difficult HDR scene image, with a dark cave with an opening on the outside of which can be seen through it. For this scene, you may want to produce a sunlit object (such as a tree) that is less bright than a scene where you only want to show the impression of a bright and sunny landscape. For example, about 400 nits, it should be the same as the inside of the cave. The characters on the dark are more coordinated. Color graders may want to optimally coordinate the brightness of all objects (already in the main HDR image of PB_HDR=5000 nits) so that nothing looks inappropriately dark or bright with good contrast, such as standing in this cave People in the dark can be encoded in the main HDR graded image at about 0.05 nits.

With this master HDR image already established, the art issue (even before it is formulated in a technical way) is how to reclassify this image to a different dynamic range, such as at least 100 nits PB_C traditional SDR image.

It contributes to clarity when the relationship between brightness is given, so it will be done in this patent when it is convenient. In fact, in terms of technology, the brightness will be coded as brightness by the brightness code distribution function (also known as the photoelectric transfer function (OETF)), and therefore all the relationships between the brightness can be formulated, for example, the output brightness L_out can be calculated from the input L_in The relationship between the function and the equivalent brightness is also true.

It may be a little confusing. You can also formulate the brightness in a normalized (for example, using a maximum normalized brightness equal to 1.0) and define all actions on such normalized brightness. This has the advantage that the normalized HDR color gamut accurately overlaps the LDR color gamut (the condition is that the image pixel colors of the two images are defined in the same RGB primary color group), and therefore, the single normalized color gamut can display brightness changes. Obviously, the relative position of the normalized LDR brightness that should be displayed with the absolute brightness that is exactly the same as the HDR brightness defined in the HDR brightness brightness range with PB_C=5000 will have a different relative height (that is, the same color can then be displayed). The domain representation displays the brightness mapping of the specific HDR pixel brightness required when establishing the corresponding LDR image pixel brightness according to the relative/normalized change of the height in the normalized color gamut. The relationship between absolute and relative brightness is simple: L_norm=L_abs/PB_C, where PB_C is any maximum brightness coded, such as 5000 nits for HDR coding, and 100 nits for SDR according to the standard protocol.

The last important thing to learn from Figure 1 (because all technologies must behave accordingly) depends on what type of object (ie, its pixel brightness) is processed in what type of HDR scene, and how to reclassify the ( There can be different high-level methods on the brightness of a plurality of pixels (that is, brightness transformation).

For example, objects in the dark (such as a motorcycle rider) can be rendered by equalizing the absolute brightness of all reclassified images (which involves the corresponding proportional adjustment changes in normalized brightness), especially at the beginning on the left The main HDR image, the corresponding SDR image on the right, and any medium dynamic range (MDR) images in between, for example, for a PB_D display at 800 nits (for example, for a monitor that has been purchased, For example, consumers who get 5000 nits of PB_C HDR images from his cable TV provider, or via satellite set-top boxes, or from the Internet) directly display optimization (with correct object brightness) PB_C= PB_MDR=800 nits display. This is reasonable, because the creator of the content wants the conveyor to be only a visible dark atmosphere, and only for the reason that this display can do so (because it has a higher PB_D end to display all the objects in the scene. The larger brightness range), it is not good to make it appear brighter on a brighter display.

Objects (like the sun) may follow a completely different principle, namely, the map white-on-white method, where the highest possible value in any image representation is always given, that is PB_C. Obviously, other types of objects can follow other types of rules and can continue (for example, cowboys will follow the principle of proportionally adjusted intermediate gray), but readers understand that they must have technology that allows almost "arbitrary" distribution of all pixel brightness One of them is sufficient, not for example a fixed one (as simple techniques will specify).

Figure 1 briefly summarizes the requirements for diversified HDR image creation (across such applications that are restricted by different technologies, such as movies, real-time sports broadcasting, etc.), and then the problem for HDR technology developers is how to encode HDR images, and also How to transform the HDR image so that it can be best displayed on any display with a smaller PB_D than the encoded PB_C (that is, the brightest pixel that may occur at least once in the video). Capturing HDR scene images, and equally important art direction and lighting HDR scenes are also technical skills, but this application need not focus on this aspect.

The simplest thing envisaged is to encode only the HDR pixel brightness (ignoring the complexity of display adaptation (DA), that is, how to map the PB_C1 image to an image for a display with less capability). The problem is that Rec. 709 OETF can only encode a brightness dynamic range of 1000:1, that is, a new HDR OETF (or actually its trans, EOTF) must be invented. The first HDR codec called HDR10 has been introduced into the market, which is used for example to create a new black belt HDR Blu-ray disc, and it will be standardized in SMPTE 2084 called Perceptual Quantizer (PQ) function update The logarithmically shaped function is used as OETF, and it allows to define the brightness for the brightness between 1/10,000 nits and 10,000 nits (enough for practical HDR video production). In addition, it has a good property that the brightness code it generates adapts to how human vision works (the brain is used to characterize the perceptual gray value types of different brightness in the scene, which is used to efficiently reclassify certain grays Value objects, and good properties for efficiently representing both brightness (as the brain does). After the brightness calculation, there is only a 10-bit pixel plane (or more precisely, two chrominance planes, Cb and Cr 3-bit planes), which can be further processed classically below the line , "As if" they are mathematically SDR images, for example, compressed by MPEG (this is an important limitation because it avoids the redesign and redeployment of several pre-existing technologies in the overall video pipeline).

The significant technical difficulty of using HDR10 images is still how to properly display them on displays with less capabilities (for example, the capabilities are less than 2000 PB_C for HDR content). For example, if only white is mapped linearly on white (the encoded image is the largest white, also known as encoding the peak brightness PB_C to, for example, the peak brightness PB_D of an SDR display), the image with PB_C=1000 nits is most interesting (darker The part will generally look too dark at 10x, which will mean that the night scene ImSCN2 becomes invisible. Due to the logarithmic nature of PQ OETF, HDR10 images are visible (when only the brightness is rendered, that is, decoded with the wrong EOTF), but with ugly deteriorating contrast, making it look faded and faded among other things. The brightness is incorrect.

Use Figure 2 to explain a simple system for creating HDR video content (for example, in a broadcast scene). Furthermore, in order to keep the description simple, the details of non-linear brightness or R'G'B' pixel color code allocation are still not considered (the so-called Opto-optical approach: OOTF, which is normally used throughout the chain). (Absolute) brightness). Using the camera (201) exposure (EXP), you can choose which object's brightness to be recorded faithfully and its relative value (because the camera functions as a relative brightness meter for all spatial positions, or generates RGB triplet (triplet) Relative colorimeter). Because the camera sensor and the N-bit mathematical representation of the color component actually have a final range (starting at the minimum and ending at the maximum), it is reasonable not to expose the details of the billion nits of the sun, but At least trim the brightness or RGB value to its maximum value. In a substantially infinite range, the exposure selection can be “corrected” by the brightness remapping later, but in any case, this fact shows the reader that there is no “natural” obvious mapping of the scene brightness to the brightness to be displayed (the latter brightness is The reference frame is known as the display-related colorimetry, and it is in fact the most important one). The linear luminance image LIN_HDR is generally first subjected to OOTF mapping (202). This has already existed to some extent in the SDR era, and corrected the fact that human vision in the generally darker viewing environment in the living room at night watching TV requires a higher contrast for similar visual experiences, so OOTF is generally soft. Soft gamma function. However, especially when a scene with a considerable dynamic range is mapped on a general monitor (205) with a smaller dynamic range (even when it is a high-quality 4000 nit reference monitor), a certain art of pixel brightness of various objects The optimization can be prepared by applying a possibly arbitrary curve (which will be referred to as a classification herein) by the classification unit 203. Especially for offline high-quality production, the grading effect may be considerable, so as to put the so-called creative vision or appearance in the main HDR image MAST_HDR (which still must be further technically processed according to the present invention, for example, advantageously encoded) . The resulting image then looks best and can be sent to the display 205 via an image communication connection 204, where the human grader can check whether the image is as expected, or via the user interface control unit 206 (for example, Hierarchical console) continue to fine-tune at least one brightness mapping function. This arbitrary grading forms the main pattern (not to be confused with any re-grading (secondary pattern)) to obtain, for example, the best possible corresponding SDR image, which can be called the primary SDR image (for example, when the video is formed as described below Part of the coding principle). Although only a simpler topology is explained for the reader, the reader can understand that the practicality depends on, for example, whether there is a real-life broadcast using only a single HDR camera or many mixed SDR and HDR cameras, or the previously determined HDR image and The corresponding reclassified SDR main image now needs to be co-coded according to the coding principle (for example, the principle of ETSI1 or ETSI2 below) and may have different practical embodiments.

The applicant understands that because there is a mathematical reclassification relationship between various possible reclassification MDR images starting from the main HDR, if these functions can be extracted technically and pragmatically, the overall spectrum of different dynamic range functions can be actually encoded (by By sending only one of them) and at least one luminance mapping function to create another image from the one that has actually been sent (illustrated in Figure 1). The first introduction of this possibility and the subsequent technical coding concept was completed in WO2011107905.

It has been found that it is reasonable to define the brightness mapping function F_L used to transform (for example, 5000 nits PB_C) the brightness of the main HDR image into the brightness of the SDR image, that is, to allow the classifier to define what is needed between the most extreme image representations Re-classification behavior, and then recalculate the display-adapted brightness mapping function F_L_DA for calculating the inter-MDR image pixel brightness corresponding to any possible 5000 nits PB_CM_HDR image brightness.

In the subsequent standardization by the applicant, there are two logical choices for the image to be actually transmitted (as a single image for the overall spectrum of the rescaleable image with different dynamic ranges, especially the PB_C endpoint, because it can often be assumed The lower endpoint MB is roughly fixed, such as 0.01 nits) to any receiver: the main HDR image or the corresponding SDR image (should stop for one second to understand that in this case, because L_HDR_reconstructed=F_L_inverse[L_SDR], and in fact because HDR images are still transmitted to the F_L function, so the actual ordinary SDR images are transmitted instead of HDR images).

The second encoding option was first standardized under ETSI TS 103 433-1 (note the -1 ; abbreviate it as ETSI1 ). This second encoding option is equivalent when many traditional displays need to be served without interference due to technical limitations Useful (in fact, old SDR monitors only get SDR images, and you don’t need to know that it also encodes HDR images. It can directly display SDR images and immediately get a very good SDR performance of the HDR scene. In fact, the monitor can display as much as possible This HDR scene). Note that there are technical limitations (such as the need for the reversibility of SDR image colors) to be able to reconstruct the original master HDR image with sufficient precision on any receiving side, which leads to one of the standard encoding (de)coding methods defined by it. Part of the technical vision.

ETSI TS 103 433-2 ( ETSI2 ) is a coding alternative that actually conveys the main HDR image to the receiver, and the function(s) F_L (actually as shown below, although for illustration, you can The system is conceived as if there is a single global F_L function for the brightness of all pixels in the communicated image. For technical reasons, a set of subsequently applied mapping functions are used to calculate the optimal value for the display with PB_D<PB_C_master The displayed image (that is, used for so-called display adaptation). Various consumers can choose the system they want to adopt. For example, a cable TV operator that communicates ETSI2 HDR will deploy STB to its users, which will decode and optimize any display that the user happens to have at home.

Figure 3 first shows the components of a general single-image-plus-function HDR video communication system (encoder+decoder) in a bird’s-eye view layer level, and does not limit the type of SDR communication used for the purpose of explaining the basic concept General system.

The color converter 302 obtains the MAST_HDR image from the image source 301 as input (for example, when it is captured by a camera and graded by the system illustrated in Figure 2, and then transmitted to the broadcast station side encoder through some professional video communication system 321, for example, it will transmit TV programs through the air or via a TV cable network). Then apply a set of color transformation function F_ct (in this example, for example, by grading automatic software determination, such as applicant's automatic HDR to SDR conversion technology, which defines the F_ct function based on image characteristics (such as histogram, etc.); Specific details can be put on hold for the description of this application, because it only requires the existence of such optimization functions for any image or time continuous image group), including at least the brightness mapping function F_L to obtain the main HDR image (MAST_HDR) pixels The brightness corresponds to the SDR brightness. For ease of understanding, readers can assume that F_L is a quarter-to-four brightness mapping function for simplicity (L_out_SDR=power(L_in_HDR; ¼)), which is used to derive the normalized PB_C SDR output image Im_LDR (also That is, the 1.0 SDR output luminance of the pixel in the luminance range on the right of Figure 1).

Because there are now "normal" SDR images, they can be compressed using standard video compression technologies (for example, MPEG standards (such as HEVC or MPEG2), or similar standards (such as AV1)), and the compression is performed by the video compressor 303.

Because the receiver must be able to reconstruct the main HDR image from the received corresponding compressed SDR image Im_COD, in addition to the actual pixelated image to be transmitted, the color mapping function F_ct must also enter the video compressor. Without limitation, it can be assumed that the function is stored in meta data, for example, by means of SEI (supplemental enhancement information) mechanism or similar technology. Finally, the formatter 304 performs formatting (putting data blocks, etc.) video streams for all those in need with the communication media 305 for any technology, for example, for storage on Blu-ray discs, or formatting for DVB communication via satellites ( This detail can be found by those with ordinary knowledge in the respective technical fields and has nothing to do with understanding the concept of the present invention).

After the MPEG decompression in the video receiver 320 is executed by the video decompressor 307 (after passing through the unformatter 306), the SDR image can be decoded by the receiver by applying the standard Rec. 709 EOTF (with Obtain the image for SDR display), but the receiver can also decode the received Im_COD image differently to obtain the reconstructed HDR image Im_RHDR.

This is performed by the color converter 308, which is configured to transform SDR images such as compressed Im_RLDR into any non-SDR dynamic range (ie, PB_C higher than 100 nits, and generally at least higher than 6x) Of the image. For example, the original main image Im_RHDR of 5000 nits can be reconstructed by applying the inverse color transformation IF_ct of the color transformation F_ct used on the encoding side to generate Im_LDR from MAST_HDR (and it is received in the meta data and passed through the color converter 308). Alternatively, a display adjustment unit 309 may be included, which transforms the SDR image Im_RLDR into different dynamic ranges, for example, if the display 310 is a 3000 nit PB display, the best graded Im3000 nit, or for a lower PB_D display PB image of 1500 nits or 1000 nits, etc. It has been assumed without limitation that the video decoder and color converter are in a single video receiver 320. Readers with general knowledge can understand that it is possible to similarly design many different topologies with, for example, the decoding function separated in the set-top box to be connected to the display, which is only used for pre-optimization as received The basic type of image display (dumb display), or further image color conversion, etc.

Figure 4 briefly summarizes the principle of the applicant's luminance and color mapping technology as standardized in ETSI2 (in fact it details the color converter 302, which is based on the ETSI2 decoding principle (or similar ETSI1 coding principle) roughly in Figure 3 Introduction), because it must be understood to understand some of the more specific embodiment techniques of this application.

The input should be the YCbCr pixel color defined by PQ (that is, the brightness Y and chromaticity Cb and Cr color components per pixel). First, the brightness is linearized to the normal linear brightness L_in by the EOTF applying unit 401 that must use the SMPTE 2084 PQ EOTF. The normal (generally defined by entity SI and CIE) luminance definition can then be used again to obtain a complete re-grading procedure for the SDR output pixel color from the input HDR pixel color. After that, the luminance processing can be performed by the luminance processor 401, which is as desired but with wisely selected sub-units (such units 402, 403, etc.) are technically designed to facilitate the needs of various HDR applications, such as, Automatic classification, easy human classification, IC design complexity, etc.) to achieve total F_L mapping.

First, the luminance homogenizer applies a fixed curve transformation whose shape depends only on the peak brightness of the input HDR image PB_C_H (PB_C_H=for example, 5000 nits) by applying one of the PB-dependent curve families defined as follows: Y’HP=log(1+(RHO-1)*power(L_in/PB_C_H; 1/(2.4)))/log(RHO) [Equation 1] among them RHO= 1+32*power(PB_C_H/10000;1/2.4) ［Equation 2］

This maps all the brightness to a perceptually uniform gray brightness Y'HP. If PB_C_HDR = 10000 nits, this curve closely corresponds to the SMPTE 2084 PQ curve, which is known to be uniform in perception. For input images with lower PB_C_HDR, the curve scales well (in fact, it is expressed in absolute terms on the 10000 nits curve, for example, the sub-curve ending at 3000 nits), resulting in a normalized [0- 1.0]/[0-1.0] The logarithmic gamma curve of the darkest color in the input/output luminance axis representation is less steep. That is, the rest of the process has started to pre-normalize well.

Subsequently, the black-white level shifter 403 may apply a certain additional white level shift WLO and a certain black level shift BLO where desired.

The effect of white level shift can be understood as follows. Suppose a content creator grades his image on a system set at PB_C=4000 nits (that is, for example, his reference graded monitor has a PB_D of 4000 nits). However, in the entire video, he never uses high For, for example, the maximum brightness of the pixel of 1500 nits (the maximum value of video, which is different from the maximum value of PB_D that can be encoded) actually produces an image. Then, because the SDR brightness dynamic range is small enough as it is, it is reasonable to re-scale the input HDR that reduces the unused value by 1500 to 4000 nits (because the dynamically adjustable brightness mapping is being used, it can be /Video instantly optimizes it). 1500/4000 corresponds to the normalized (input) HDR brightness of 0.375, so this value can be mapped to the maximum value of the proportionally adjusted HDR brightness Y’HPS by dividing by 2.6.

For accuracy, according to the ETSI2 standard, the following calculations are performed: Y’HPS=(Y’HP-BLO)/(1-WLO-BLO) [Equation 3]

Among them, WLO and BLO are communicated in the meta-data, which is communicated with the received video image or can be associated with it.

The black level shift is useful for obtaining a higher contrast appearance of the SDR corresponding reclassified image, but it should be noted that the received image of ESTI1 should be reversibly mapped to the HDR image, that is, too much black pixel details should not be lost (It is also due to the parallel gain limiter, not shown in Figure 4).

Basically, the black level offset can be simply understood as placing a certain HDR "black" color in the SDR 0.0, or more precisely, through the unit 403 (that is, the unit prepared for HDR to SDR brightness mapping) , Using the normalized brightness that is still in HDR means that it has a relative distribution that can be used to obtain a good appearance on an HDR display, and a relatively unoptimized appearance on an SDR display).

Subsequently, the coarse dynamic range converter 404 applies the main luminance transformation to obtain the SDR luminance (ie, has a good first redistribution of the object luminance to obtain a reasonable appearance on the SDR display). In this regard, ETSI2 uses a slope controllable linear section for the darkest HDR normalized brightness (the slope of this section is called shadow gain), and another for the brightest normalized HDR input brightness Y'HPS A linear compression part (high brightness gain with a slope control parameter), and a controllable parabolic part that smoothes it together by providing a good SDR appearance for the midtones (with a control parameter midtone width, and the mathematics is available in the standard Read it, and in this application, only reinterpret (as needed in a simple and digestible manner) to the extent required to understand the new invention embodiment based on this insight) composed of the curve. Therefore, the output brightness Y'CL of the rough dynamic range converter 404 is first defined in the SDR range, or SDR relative brightness distribution statistics.

The content creator’s technical (and artistic) proposal for this unit 404 is that the grader can optimize the brightness of other objects with brighter pixels at the expense of intra-object contrast (because of the limited SDR brightness range). To produce the darkest pixels, but he can be tuned together, for example, high brightness gain. The shadow gain can be understood to be used for, for example, people standing in a dark shadow area of a cave with a brightness of 0.05 nits. If the criterion of white on white (ie, the normalized mapping function, which is an identity function with a 45-degree slope of the diagonal of the normalized luminance function graph) is displayed on the SDR monitor, he will be found The normalized luminance in HDR is 0.05/5000. It stays at the same normalized luminance due to the identity mapping used to roughly map the SDR luminance, that is, after making it equal, the pixels should It is displayed on the SDR display with (1/100000)*100, that is, the minimum black ("0" drive signal) on the display and is invisible. Therefore, such brightness must be increased considerably, even in a more logarithmic uniform HDR and SDR relative gray value or brightness representation, to obtain full visibility and cause personal objects (for example, display on SDR displays that span 0.3 to 1.2 Ni Unique individual pixel brightness) within the object texture discernibility SDR brightness. Therefore, it depends on how deep the person happens to fall in the HDR brightness range (which, as taught above, will depend on such things as HDR scene architecture, scene lighting, camera exposure, and the master HDR grading selected by the content creator. What is the combination of class factors), the encoder (for example, the appropriate F_L part is the first rough luminance mapping choice to reclassify the main HDR input to the best or suitable human classifier corresponding to the SDL pixel brightness) will choose Appropriate shadow gain of the darkest pixels used to process this particular image (ie, image content optimization). Note that in fact, in ETSI, the shadow gain SG is defined as an automatic scaling correction based on the ratio of the peak brightness of the input and output images (at least the equivalent brightness). Under the principle of equilumina, by starting from the normalized HDR brightness as: L_200=Y'200*L_HDR, it is reasonable to increase the brightness in the normalized brightness range. This normalized The brightness range corresponds to, for example, only 200 nits PB_C (or more precisely, according to the values of Equation 1 and Equation 2 above: Y'HP=Y'200=v(PB_C_H/200; RHO(200)), v is the pseudo-logarithmic equation of Equation 1 above). However, this generally provides too bright and low-contrast images, so the grader can use exposure gain correction: SG=expgain*Y'200, which will move the SG back towards the diagonal value of 1.0 and bring some darkness back to SDR In the image (he generally would not choose expgain=1/Y'200, because the normalized brightness of SDR will then be equal to the normalized brightness of HDR and will be too dark again; SG will fall between 1.0 and 1.8, for example). Dimming factor.

This curve is similar to the non-linear luminance compression "spring" in the SDR DR that squeezes many HDR luminances in the much larger luminance dynamic range to much smaller SDR DR. Because the fixed curve of "never should never be too unreasonable on average" is used, the encoder can apply an optimized curve, and the resulting SDR image will not be bad for many HDR scenes (not all HDR The scenes are all equally complex, for example, sometimes there is a weak shadow area next to a uniform sunlight area, and then although the simplest system will produce a problem like trimming to white, but not too complicated smart HDR to SDR mapping (Such as the three-part curve of unit 404) will often be well done in creating an appropriate SDR re-graded image of the HDR master image (for example, the one from the HDR camera of the creator of the real life event capture content).

However, several other scenarios can be more complex, and some content creators may also have a higher degree of professional demand when fine-tuning their artistic content (for example, Hollywood film directors or DOP).

Therefore, the next unit (customizable curve applicator 405) allows content creators (again, whether humans or intelligent automation with various rules coded in their algorithms) to customize and possibly arbitrarily shaped fine The graded luminance mapping function F_L_CU is applied to the pre-graded luminance of Y'CL to produce graded LDR luminance Y'GL (the only requirement for this function is that it does not decrease, and generally even increases monotonically, and generally at least as Optionally map 1.0 input to 1.0 output in ETSI2). In practice, the shape of this function F_L_CU can be communicated to the decoder as a set of shape definition parameters (for example, coefficients of a polynomial) or as one of the LUTs.

Because the visual system has a complicated way to determine the impression of the gray value of the perceived image object, and/or because the HDR brightness of a large span is squeezed into a limited SDR DR, considerable understanding ability may sometimes be required, and/or because of the content The creator obviously expects to put some extra artistic flavor on this customized curve F_L_CU (then the shape will generally be determined by the computer hardware of another color user interface and the software (not shown) connected to the encoding side), This fine classification may be necessary. In fact, on the one hand, it can be said that all MDR images should be (only) some kind of compressed representation of all the information in the main HDR image, but on the other hand (because it can give a rather weak image, for example, With too little contrast, as if seen through the fog), other important requirements for content creation can be to make up all images into SDR imagery-given their more limited DR capabilities-as real as possible or at least as real as HDR scenes May be beautiful. Human vision is highly nonlinear and sophisticated, and if it has been used for too simple functions, it can be perceived quickly. Therefore, in addition to the rough brightness squeeze function F_C, content creators may use the ability to understand the customizable function F_L_CU in order to produce it looks as good as possible for HDR scenes and better like SDR images of HDR scenes are better (for example, reduce the brightness of a certain luminance sub-range of pixels to produce just more contrast between objects, for example, the brightness used for stained glass windows is for church interiors, or SDR images The visual contrast between indoor and outdoor in, or by selecting a special local shape through the F_L_CU curve to optimize the color and brightness of some objects in the scene, etc.).

The single simple example of the "shadow man" image shown in Figure 6 can be used to inspire the reader and provide him with the minimum necessary understanding of the customizable brightness mapping function.

Figure 6A shows geometrically what can be seen in the image, and Figure 6B shows the functional relationship between L_HDR luminance and L_SDR luminance. The image shows the robot 602 moving through its dark space station (DRKSPST). At a certain image presentation time, it encounters the shadow person 601 (its chromaticity is defined as a set of very bright HDR pixels), in which there is a slight difference in brightness between the various pixels that make up the shadow person's body. This happened because he was standing behind a window in a strong lighting environment filled with fog. The fog adds a component to the brightness derived from the shadowed person's body (for example, his clothes), for example, L_pants=20 nits + L_mist = 4500 nits = 4520 nits, L_shirt = 50 nits + L_mist = 4800 nits = The final brightness in the HDR image of 4850 nits is given to the viewer. The problem when using a progressive luminance mapping function with a slope that is too small for the brightest pixel is that shadow people can become insufficiently contrasted and difficult to see in smaller dynamic range images (such as SDR images). One solution is to define the F_L_CU function so that it locally has a larger slope in the input HDR luminance region of 4500 to 5000 nits, resulting in a larger SDR luminance sub-range RS of the shadow person, so that he and his details (for example, The tie it is wearing is more visible in the fog (even in the SDR image). It can be understood that there may be many situations where it may be advantageous to have more additional reclassification control than only the rough mapping function F_C.

Returning to FIG. 4, after the appropriate (uniform visual representation) SDR brightness has been defined, the linearizer 406 converts it into a (normalized) SDR brightness Ls. However, since the SDR luminance is now generated by RHO corresponding to PB_C_S=100 nits (which is input into unit 406) instead of 5000 nits used to perceive unevenness at the beginning of the luminance processing chain, the above equation 1 is applied The inverse equation.

The color is of course not one-dimensional (unless only the achromatic gray value image is used for operation), which makes dynamic range conversion and coding more complicated, but it needs to be used for the parallel processing track of the pixel chromaticity Cb and Cr In any case, to obtain more suitable corresponding SDR chromaticities as the output color components Rs, Gs, and Bs, or in fact suitable SDR RGB colors as finally shown in FIG. 4.

The chrominance processing track 450 of ETSI2 performs the following (only briefly explained again to the degree required). The input pixel chromaticity Cb and Cr are similarly multiplied by the value F_C[Y] by the multiplier 452 to produce output chromaticities Cb* and Cr*. The difficulty lies in obtaining the appropriate output chromaticity at all times. Knowing that there are many difficulties: the achievable color gamut of irregular shapes (see the explanation in Figure 5), mathematical nonlinearity, and the human visual system of the viewer. In addition, as will be shown in the following embodiments of the present application, there is even more demand in the market, resulting in an even more complex HDR processing system.

ETSI2 uses a saturation process determiner 451, which can load a LUT that defines the output value to be sent to the multiplier depending on which brightness value Y the input pixel happens to have. Furthermore, content creators can freely define/optimize the shape of the saturation multiplier definition function dependent on brightness. At least because, as will be seen below, sometimes the invention of color mathematics requires defining this F_C[Y] LUT to the desired degree.

The matrix application unit 453 simply converts the Cb and Cr color specifications into corresponding normalized RGB representations (this mathematics is not interesting for this application, and interested readers can find them in ETSI2 and ETSI1).

The RGB triplet can be defined by multiplying the normalized R/Lh equivalent value of the "non-HDR luminance" by the normalized Ls value calculated in the luminance processing track 401. Note that the obtained RN, GN, and BN values are in fact still normalized brightness, not absolute SDR brightness (Rs, etc.), but they are "SDR correct" normalized brightness, because they now coincide with SDR The obtained brightness (Ls) is taken into consideration.

There is a possibility that people who do not have general knowledge in colorimetric technology may initially have some difficult concepts. In order to speed up the reader, the normalization in Figure 5 (commonly, that is, when as explained above During normalization, the SDR and HDR color gamuts overlap well, but of course the HDR color must be shifted to become a suitable SDR color. Even if the current HDR scene image requirements for this transformation are not highly sophisticated and optimized, it is simple The absolute SDR luminance is equated with the input HDR absolute luminance) What happens in the YCbCr color gamut.

The pure brightness transformation will occur in the vertical direction, so it is general to move the HDR brightness or its corresponding brightness Y (that is, the brightness of ColHDR) up to the best new position (ColSDR), because for HDR to SDR brightness mapping, The F_L curve will always fall above the diagonal on the normalized axis graph (that is, the input HDR normalized brightness or brightness with a certain x coordinate also has the height of the diagonal at the position of the x coordinate The normalized SDR output brightness of the function that is always above the diagonal will always produce a higher normalized output value). Which actual (absolute) SDR brightness corresponds to this normalized brightness value Y by first EOTF to normalized brightness (executed by unit 406, because the processing brightness from Y'HP up to Y'GL is performed by applying The corresponding EOTF of Equation 1 is defined), and the normalized luminance is simply multiplied by 100 (for example, 0.7*100=70 nits) by the multiplier 455. In other words, readers now see that using this framework, whatever is needed can be defined from the input HDR image color, especially the brightness Y defined by PQ (for example, as stored on the HDR Blu-ray Disc). Up to the absolute SDR brightness of the corresponding pixel to be displayed on the SDR display to display the best corresponding SDR image as an HDR input image (and the resulting decoding of the SDR image from the received HDR image).

So far, the reader now understands the basic starting point of HDR encoding at least according to the applicant's ETSI standardized encoding principles. For most consumers, the choice of either ETSI1 or ETSI2 (and everything that happens technically afterwards) will be sufficient for their purpose, that is, to supply their market with beautiful HDR images (of course They will still need to generate these beautiful HDR images, including determining good shapes for at least the F_C function and preferably also for the F_L_CU function, or at least when manually optimizing the functions according to their own specific artistic needs , Purchase and use the applicant's automation that automatically generates a pretty good look for each HDR image type, and the subsequent codec function shape). For example, consumers who will be completely patched to obtain future-proof high-quality and diverse HDR can deploy the ETSI2 system, and market players that value one of its SDR images or SDR consumers can be They also deployed their HDR system as an ETSI1 system (this may also involve various discussions depending on where in the HDR video processing chain, for example, content creators are related to cable television communication system operators, and may involve transcoding, etc.) .

However, the market has another demand or a market that proposes to consumers who do not like to deploy accurate standardized ETSI1 or ETSI2. If you choose to communicate the HDR image as a single image representing the overall spectrum of the images required by all various PB_D displays, it is very reasonable to communicate (for example, 5000 nits PB_C) the main HDR image itself, not only because these images are already available, but also Because the best quality of the image HDR scene represents (in fact, it is the "wealth" of content creators, specially created and recognized by him, and often the starting point of creative visual movies, if not the only thing he actively created , If the rest of the reclassification works automatically by the selected technology). However, especially in the next few years, there are market conditions that can benefit from another additional method. Unfortunately, all TVs (or general video decoding or processing devices) that are not non-basic traditional SDR displays in the market (that is, cannot perform all the mathematics involved in HDR decoding, display adjustment, etc.) will always have ETSI2 immediately. (Or ETSI1) capable TV. There are many TVs on the market that apply very different methods (like, for example, according to the recently standardized hybrid logarithmic gamma method) to HDR encoding and display. Or maybe some tv can only decode HDR images encoded with PQ brightness, nothing more. Maybe some TVs can only use this method, so it is likely that the best thing they can do is not to process the incoming ETSI2 HDR video at all. Similarly, there may be some TVs in the market that do not follow any standard principles, at least not about display adaptation, that is, when receiving, for example, 2000 nits images are reclassified into, for example, 900 nits images for 900 nit PB_D displays. . This TV will need decoding capabilities to understand what pixel colors and especially the brightness the received image contains, but it can use its own (tone mapping) heuristics on how to generate 900 nits of images. At least from the point of view of content creators who want every consumer to see movies that are as good as they were originally created. The disadvantage is that this variability will create a high degree of uncertainty. Deterministically produce any received HDR images. For example, the simple display re-interpretation of HDR images performed recently is the absolute rendering of HDR image brightness. This means that all HDR image brightness of up to 900 nits is accurately displayed as the brightness coded in the image, but all higher brightness is clipped to the whitest possible white of the display (ie, PB_D). Using an example image, such as the space station in Figure 7, means that some parts of the earth will be trimmed into ugly white spots (where the sun on the right is over-illuminating the earth). And this TV will still be a beautiful HDR TV to a certain extent, because it will display most of the very bright blue earth that is seen through the top viewing port of the space station in good contrast with the dark interior. This image At least part of will look ugly (and some other scenes can show more serious errors on at least some TVs, such as, for example, trimming out every image detail on the outside of the cave or market in Figure 1). Performing another simplified tone mapping reinterpretation (like, for example, linear compression of luminance, like a white on white strategy) can cause several other problems. Therefore, although this system can operate for terminal viewers and generate some kind of HDR images (for example, in our ETSI2 system, this TV can only use the PQ function of 401, but after ignoring all other luminance mapping functions Suppose the data and the subsequent sequential brightness mapping 402, 403, etc., perform the display adjustment function in ESTI2), but the result will not be the best visual quality or predictable (it may be worse) of).

This leads to a new coding topology based on the second type of HDR image, the so-called intermediate dynamic range (IDR) image, in addition to the main HDR image, which was first introduced in WO2016020189. Then the advantage is that the PB_C (for example, 1000 nits, or 750 nits; PB_IDR of 500 nits using the same technology, or perhaps even 400 nits PB_IDR) can be defined in the range of many TVs in the field. Class secondary HDR image (IDR-encoded image, which will be transmitted to the receiver instead of the main HDR image in the typical ETSI2 codec principle). But in terms of required artistic or practical technical limitations (for example, available graded monitors), it can still be expected to generate PB_MHDR master HDR anyway. The idea is that regardless of the display reinterpretation (including tone mapping) technology used by any TV, it should be smooth. In this sense, if PB_D is close to PB_IDR (the peak brightness of the received IDR image), the processing should not be The received image deviates too much. For example, even if it is such a basic TV that only trims all pixels higher than PB_D, it should not be trimmed too much at that time (for example, not the entire earth or the sunny outside of the cave image). And the content creator takes back some control, because even if he expects to produce beautiful super bright image areas on the one hand, for example, the main image of 5000 nits PB_C_H has an average of 4000 nits, he can control him to reclassify IDR The way these areas in the image drop to, for example, sufficiently below 1000 nits, so that even a basic TV of 800 nits should only trim the brightest and least visually destructive pixels, for example, only Trim the rays of the sun in the space station example in Figure 7. Therefore, some new technology is needed to cater to the new method.

Figure 7 shows the codec principle of WO2016020189 that caters to the channel adaptation method (the channel conveying the image is an IDR image, so that it can be said that a specific communication channel is configured for sending, for example, a 1000 nit PB_CH image). Again, select interesting examples that illustrate some of the main concepts. One thing that should be understood is that if all the different PB_C images along the range are exactly or at least very close to what would be produced if the content creators graded them separately and not restricted by any technical system, Although it can be useful, this need is not always the case, especially for IDR images. It may involve some kind of relaxation (on the other hand, there may also be some kind of debate about when or why a certain image classification of the HDR scene category X is the best, and there seems to be enough deviation for this deviation; for example It is conceivable that the brightness of the pixel of a street lamp is less important than the pixel of a face, especially if it should be regarded as half hidden in the dark, because in real life, any street lamp is likely to be brighter or brighter anyway. Not brighter).

WO2016020189 provides definitions from an IDR image as an intermediate point (that is, upward toward the main HDR to be reconstructed from the IDR image when received by the receiver, and downward for display adjustment for the MDR display of PB_D <PB_IDR) Function (different function) method. Using this technique, the main HDR range can be well selected to always be fixed at 10000 nits PB_C range, which is linked to the range of the PQ function.

It is seen that there may again be different considerations concerning how to transform the various possible brightnesses, and these may advantageously be quite different on the left side of the selected IDR image than on the right side. Because in fact, different things may be being done conceptually. On the left, secondary ("smaller") HDR images are generated from the main HDR image. So one consideration can be that this IDR image must be "as good" as the main HDR image (despite the lower PB_IDR) (and then how to elegantly resolve the apparent contradiction?). On the right, it is moving towards even smaller PB_MDR compression (which is considerable for some high-complexity, meaning many key objects in the luminance range among other things, and high PB_C_H images), that is, it seems There are different display-adapted image generation tasks. Therefore, it is conceivable that this can lead to (quite) different technical treatments, especially in our image + brightness mapping visually shaped/designed brightness mapping function.

In this example, the dark space station brightness can (at least in principle) be displayed on every reasonable TV because it is darker than 60 nits. But the brighter pixels must first be compressed to the IDR image fairly gently, and then less compression is done in the first part, and more must be done towards the SDR image. And again there may be different criteria for the two illustrative bright objects, the bright blue earth to the far bright but almost colorless sun and its rays. When the luminance sub-ranges respectively in the main HDR image luminance range (BE) and the IDR luminance range (Be2) for bright earth objects indicate that the maximum brightness that content creators can expect for the earth has never been higher, for example , 750 nits, regardless of the PB_C capability of any image or display (because otherwise the earth may start to glow too much to look unreal). However, what the sun's brightness must do then varies with several factors, not only for artistic needs, but also for encoding the brightness of solar objects higher than 750 nits in the selected (800 nits PB_IDR) IDR image. (Of course, in some cases, the content transmitter can choose another higher PB_IDR value, but it has been assumed here that any device connected to the receiving end of the communication channel is always expected to be used for any video content (whether it is a Hollywood movie or news program) 800 nits PB_IDR). Use two arrows to display the final selected F_H2h brightness mapping function for all these brightest pixels as a subset of the IDR image brightness from the main HDR image brightness: select the solution to define the one used for the two objects together The total compression action also reduces the brightness of some of the least bright earth objects. This is an example of a situation where the ideal reclassification requirements of content creators are not 100% fully satisfied (because it may correspond to some other technical difficulties), and IDR images are close enough for most people. It really doesn't matter whether the earth pixels are only darker in IDR images, and some may even be expected to be used for lower quality HDR images. But the key point is that this IDR image can still meet all the requirements of the original ETSI2 principle (at the same time, this additional codec step also meets the requirement of not degrading the received IDR image too much before displaying the received IDR image on a basic display close to 800 nits PB_D) : By adopting the right brightness transformation function, all MDR images up to the SDR main image can still be generated by the receiver according to the content creator’s expectations, and (even with darkened bright earth object pixels) the main HDR 2000 nits PB_C or The 10,000 nits PB_C image can still be reconstructed by the inverse F_H2h function (it can also be optimized by itself for each image and a set of continuous images of a specific lens of the encoded movie according to its technical and/or artistic needs).

It is worth discussing two documents about their irrelevance (because different technical aspects should not be confused) rather than their importance (but due to potential confusion, they are worth discussing): US20160307602 and EP2689392 (also known as WO2012127401) , Both of them consider the so-called "display optimization" rather than video image coding framework design. This main difference is explained to those with general knowledge using Figure 23, which shows a general example of the total video processing chain. On the content creation side 2301, it is assumed that there is a live (or previously recorded) capture of an HDR scene by means of a camera 2302. The human grader (or shader) judges, for example, the main grading captured among other things (that is, the relative position of the luminance of various image objects in the luminance dynamic range of the main HDR image-which is in, for example, PB_C_H50= It ends at the maximum value of 5000 nits; and starts at a small black value, for example, MB_C_H50=0.001 nits, which can be assumed to be equal to zero for the current discussion: for example, for the space station, he uses image processing Changing the original camera capture makes the sun pixel become 4500 nits and the bright blue of the earth becomes, for example, 200 nits, etc.). Secondly, the graders in our method generally want to use at least one brightness mapping function (in practice, this brightness mapping function can be shaped differently for continuous images of HDR video, and even in our ETSI The standard explains how it is technically convenient to even define several functions for a single instant video image, but their further complexity is not necessary to explain the contribution of this innovation in the field) indicating which is generally designated How to reclassify the normalized main HDR luminance of 5000 nits PB_C_H50 to 100 nits LDR normalized luminance: FL_50t1.

Then the third important aspect is the encoding technique used for encoding the main HDR image to be communicated (via at least one encoding technique) to one or more receivers. At the beginning of the HDR video research, and correspondingly in the simpler version standardized by the applicant, this will be a relatively simple encoding, such as, for example, an LDR 100 nit image, and then it is well retrospectively compatible so that it can work well. The visual appearance is directly displayed on the old LDR TV without HDR understanding or processing capabilities. WO2016020189 encoding method and this teaching are examples of more advanced second-generation methods, which are more complex but can cater to additional needs in some specific HDR video communication or processing technology ecosystems. The classification performed by, for example, the human grader 2304 (if this is not automated, such as usually in a real-life broadcast program) is performed in the grading device 2303 (which generally will contain several tools to change the pixel brightness. This description can still It is assumed to be completed by providing a user interface to specify the shape of FL_50t1 and conveying the shape of this function (for example, as a component that includes some parameters defining the shape of the function).

Although the video encoder 2305 (without limitation assumes that its input main HDR image is a set of luminance for all pixels, it will perform all the actual encoding techniques that produce the main HDR image, that is, for example, and general YCbCr pixel color three The 8-bit, 10-bit, or 12-bit pixelized matrix of tuples together with the meta-data describing all further information (such as the luminance mapping function) in view of which coding technology is selected can be included in the classification in principle In the device 2303, it is generally displayed as a device that can be further connected. This representation suffices to illustrate the simplification of the present invention for the reader, where it summarizes what happens in the outer broadcast track, for example, the capture (and possibly the classification), and perhaps the encoding that occurs in some intermediate communication relay stations (for example, in the Local advertisements are inserted in the signal, which can also involve the individual reconciliation of various video content, but it is a variety of actual changes that need not be explained. The point is to understand what happens on the creative side (see, for example, the difference between contribution and distribution), which can be defined as when, for example, by means of satellite antenna 2306 and communication satellite 2340 (or any equivalent video communication channel, for example, It officially ends when the final encoded video signal is delivered to some consumers via the Internet, etc.

On the receiving side, generally facing consumer equipment at the end consumer’s home, such as a satellite tv connected to a local satellite dish 2351 on the input side and a satellite TV connected to an HDR display 2353 on the output side Box, or any equivalent decoding and final processing device 2352, the display can have various display capabilities, for example, a PB_D of 1000 nits, or 700 nits, or 2500 nits. It may be sufficient for the set-top box to use the decoder 2381 to decode the luminance value that needs to be displayed. The decoder often performs the reverse operation of the encoder. This is generally only useful in a limited amount of cases. . Usually there will be a display optimization process by the display optimizer 2382, which again changes the absolutely individually normalized luminance distribution (received, for example, LDR image or decoded master HDR, For example, any of the 5000 nits image), because the main image may have been encoded for, for example, 5000 nits PB_C_H50, that is, may contain 2000 nits of brightness pixels, the HDR display of a specific consumer can still only be, for example, Display up to 700 nits (which can display the whitest white).

Therefore, on the one hand, there are major technical differences between the appliances on one side (and their technical design principles, etc.). For example, the authoring/encoding/transmission side will only have a video encoder 2370 to convert the main HDR The video (MsterHDR) is encoded as a channel-encoded intermediate dynamic range image IDR, and the receiving side can also display the optimized 5000 nit HDR image (RecHDR) in, for example, 700 nit PB_C image ImDA optimized Among the displays, it is best suited for the connected 700-nit PB_D display. The technical difference between the two can be seen in that one can perform display optimization as an (optional) post-processing, while encoding/decoding is only an image reconstruction technology, and generally does not require any teaching related to display optimization. The equipment (and operating procedures, etc.) on the two sides are generally handled by quite different technicians. Content creation equipment can be designed by professional video equipment manufacturers and operated by broadcast engineers. Set-top boxes and TVs are generally manufactured by Asian consumer electronic appliance manufacturers.

US20160307602 is the applicant's first display optimization patent. In summary, the idea here is that the content creator can give various (at least two) areas that can exist in the image (area is a concept, the concept is a group of pixels in the image, and when there are various The dynamic range of the various available displays when the pixels require re-grading behavior (two) guide the re-grading behavior rules and algorithms. Although this first realizes the connection between the needs of the content creator and the actual display of the final consumption location, in fact, the controlled behavior of display adaptation must occur at this terminal location. And ideally, the maker of a set-top box or a TV (provided that at least the display adaptation takes place in the TV) will mostly follow the content creator’s designation of various regional objects in the video image as good actors (for example, someone from dark The area fades in and is neither too visible nor too invisible in any display capability, even a 100-nit PB_D LDR display), because it is needed for this content, rather than blindly doing anything. But this is clearly the final behavior that takes place on the consumer side, and is completely orthogonal to the principles of how video communication technology providers want to develop, and how any real author wants to deploy any specific video codec. It should not be confused with any ad-hoc tone mapping technology. It has been in view of the fact that this mapping will generally be irreversible and should have the property of being encoded by IDR images with lower dynamic range.

WO2012127401 is also an early HDR chronological technology for specifying display optimization behaviors, which can be accomplished by various embodiments of the DATGRAD structure, which specifies how various image content should be reclassified for different luminance dynamic range capabilities. This DATGRAD structure will be used to generate anything between the main HDR-encoded peak brightness PB_C (that is, PB_C_H50 in this notation) for the MDR display PB_D and the 100 nit LDR PB_C of the minimum necessary reclassification specification (p. 16) The required medium dynamic range image (MDR). The derivation of MDR images is not only based on the use of reclassification requirements such as the images encoded in the DATGRAD data structure, but also the use of specific display-side viewing aspects (such as, for example, viewing environment brightness or final viewer preference settings (see p. 5)) Best done.

It should be clear that from the beginning, when there is no further and quite specific insight, such teachings have not brought anything to the general knowledge about codec redesign.

Compared with those found in the prior art, in addition to the difference in generating specific functions, the more important is the innovative codec structure/framework itself. The second PB_C value should also be mentioned (in addition to the actual IDR In addition to the lower one of the image, the actual communication of the highest one of the main content is also different from the (optional) category characterizer that can be used in WO2016020189. Except for the fact that the two are not the same literally, enumerators can play different roles, especially if you consider the details of the framework compared to the teachings. This characterizer of '189 can be useful in situations where there are, for example, two upwardly reclassified luminance mapping functions. Then it might be useful to choose which one to get anything (like a tight reconstruction of the main HDR image on the creative side). However, such information is neither strictly required nor necessary to be applied to prior art. The upgrade function from the main HDR image can be used to replace the 5000 nit reconstruction image to obtain a 4000 or 6000 nit reconstruction image. There are two sides of the intermediate image, the downgrade function usually has one of the critical image content (especially the content that must be bright enough and faithfully displayed on all PB_D monitors), but the upgrade function is for the brightest objects (like It is the headlights, daylight reflections, etc.) that specify the reclassification behavior in this way will be specifically different. It is a general HDR affecting purpose/effect, but it cannot be replicated correctly in any way, because it is in the upper region of the brightness range where displays of various capabilities change the most. Therefore, for example, a 4000-nit PB_C reconstructed image generated from a 600-nit IDR image may have an ideal luminance value (even if this value can be expressed in the 4000-nit luminance range) for some car heads that are slightly too dark. Light, but if only applied, for example, the multi-linear normalized re-grading function is on the [0-1]/[0-1] axis system, it will still be a pretty good-looking HDR image, where the horizontal axis represents the IDR image The luminance normalized by PB_C, and the vertical axis corresponds to anything selected to calculate the reconstructed HDR image PB_C, which is not too unreasonably far away from the main HDR PB_C (it may not be known but just an assumption). In our technology, the PB_C_H50 brightness value itself is actively conveyed in the meta-data because it is also used in the algorithm of the decoder.

The inventor of this patent application wants to restrict the general IDR method in many ways, especially around the ETSI2 coding principles and systems (IC, TV, set-top box) deployed today.

Many technical considerations are generated by the inventor. On the one hand, the systems they want to wait for are compatible with the deployed ETSI2 decoders. Therefore, if an IDR image of, for example, 1500 nits is communicated (the ETSI2 decoder does not know anything about the principle of the IDR architecture, so it is assumed that this is only an original HDR main image of the HDR scene), it should be communicated together as shown in Figure 7. A F_L_IDR luminance mapping function (and all other color mapping information according to ETSI2) of the F_I2s function that performs the correct display adjustment. Therefore, regardless of whether an IDR additional technology has been used, the ETSI2 (also known as SLHDR2) decoder should be able to create all MDR images up to SDR images normally, and they should (ideally) look like the content creator expects of. Ideally, any new decoder (which will be referred to as an SLHDR2PLUS decoder ) according to this principle should also target all images between IDR and SDR (that is, at least one of the IDR and SDR images is preferably not It should accurately or at least approximately produce the same appearance instead of how much deviation will result from the MDR image that the color grader or the content creator would want or at least accept to see it. On the other hand, a very important criterion is that the main HDR can be reconstructed almost perfectly (but may be unknowingly caused by minor trimming errors, for example, when the IDR image is DCTized in the MPEG compression phase for transmission, It will cause a very minor unobjectionable reconstruction error of the main HDR on the receiving side). Of course, there may be some systems with some relaxation in the quality of the main HDR reconstruction (some content providers may regard IDR images as more important, at least in cases involving some temporary aspects, such as broadcasting or even single betting. To a limited audience, not, for example, for storage, such as on Blu-ray discs), but usually at least one major party involved in the video processing chain will find that the perfect reconstruction of the master HDR image is important (it is similar to the establishment of a certain A blind attempt to distinguish between a higher dynamic range appearance).

Finally, although it is seen that services without the need to redesign and redeploy the ETSI2 decoder, the F_I2s function must be communicated together (that is, it is better to reuse the encoding (decoding) circuit system of the SLHDR2 system as much as possible, But at least the video signals including their luminance and color mapping functions should still conform to the standardized definition, so that, in particular, except for some unnecessary and negligible meta-data, the traditional SLHDR2 system knows what they obtain), The content grader may generally want to specify between the master HDR he has established and some corresponding SDR (ie, 100 nit PB_C) version (he may have established it, for example, using one of the systems shown in Figure 4) His brightness (and color) mapping function. So that the F_Mt1 function (see Figure 10) is neither the F_H2h nor the F_I2s function of Figure 7, but is a function that spans the overall reclassification effect between the main HDR and the main SDR (that is, this F_Mt1 is an HDR The reclassification requirement of the scene image is defined between the most different dynamic range representations of the HDR scene). Therefore, a technique is needed to elegantly connect these two situations, especially in or around the ETSI2 framework principle (for example, the SLHDR2PLUS decoding method produces the same MDR as the ETSI2 receiver that adapts the received IDR image and the D_I2s function. Image appearance; for each instant in time, one or more functions partially perform the reclassification between the input dynamic range image at that instant and the expected output dynamic range image at that instant).

As will be seen below, it can be done in several ways based on the different insights of the various inventors, depending on which type of system is exactly desired, and which one wants to relax more restrictive conditions and which relaxes more. (Also take such specific actual technical factors into consideration, such as, for example, how many cycles or transistors will be required for various options, which makes some options more desirable than others, but does not need to apply for a patent Delve into these details).

There are some basic principles that all methods use. Figure 8 can be used to summarize at least two solutions. The receiver receives IDR images (which should be reconstructed into Mster HDR images in some way, in this example with PB_C = 4000 nits), and they also have the function F_I2s. But they must find the function F_?? for each IDR brightness in some way to calculate the required corresponding normalized and therefore absolute master HDR brightness (which accurately reconstructs the master HDR into its original state, but never conveys it. (Multiple) images). A novel SLHDR2PLUS decoder color conversion system that can determine the required functions can be constructed (it is still at least based on its processing IC or software core as shown in Figure 4, with a luminance processing track and its details (using at least some of the subunits) , And chroma processing track), or try to put all the intelligence into the encoder, so that the standard ETSI2 encoding color conversion method can be used as it is (except for its new plan to receive this second expected peak In addition to the novelty of reconstructing the original HDR of 4000 nits from the meta-information of the luminance PB_C_H50, it generally involves the loading of all or part of the luminance and chrominance processing LUT), which is set as extrapolation instead of being used for comparison with PB_IDR The smaller value of PB_D display adjustment image. The embodiments of the two methods and others will require some general new technical components, although they fall under the general new coding principles of SLHDR2PLUS.

From the basic structure of the SLHDR2PLUS encoder 900 type of the universal IDR encoder shown in Figure 9 (which will be explained in more detail below), we can see the difference from the normal HDR encoding, especially the ETSI2 HDR video encoding: now there are two The peak brightness is coded together in the meta-data, that is, the “normal” peak brightness (which should be called the channel peak brightness PB_CH, that is, the peak brightness of the received IDR image. Any technique used for It means any peak brightness level that looks best to the content creator, owner, or transcoder, and any mathematical technique used to calculate the IDR pixel brightness), the normal peak brightness indicator conveys and The maximum encodable brightness of the video received later (ie, IDR image(s)) [this is what a normal ETSI2 decoder will see, ignoring all other novel methods]. Second, but now there is also the peak brightness of the original master HDR image(s), that is, the content peak brightness PB_C_H50 (for example, 5000 nits). In some embodiments, many months before the IDR image is created, the PB_C_H50 of the second person may have been specified when the main HDR image was created (for example, based on the camera capture action, on the computer, etc.), and can be used in During channel encoding, the PB_CH is set as an external input to the decoder 900 in many different possible ways (for example, a cable TV operator may have a fixed value set in memory, which may be set on an annual basis Upgrade to reflect the current average status of his consumer’s HDR display, or once optimized, PB_CH may be able to provide certain brightness or other image details of at least one image of the video, or its associated meta-data, or even possibly In particular, include the meta-data used to guide the later IDR re-encoding into consideration and calculation). Make a (single) peak brightness collectively convey the encoding of HDR, at least for the ETSI2 system (it was considered the only thing they needed at the time, that is, the "the" peak brightness of the received image) It is useful, but in view of the fully transparent availability of traditional ETSI2 displays, they must be PB_CH as described (otherwise they cannot do their normal display adjustment calculations). On the other hand, PB_C_H50 must be able to calculate the F_?? function of Figure 8, and using this function, the final expected main HDR reconstruction image comes from the received IDR image.

Therefore, the difference between a traditional ETSI2 video encoding data stream is immediately displayed, and the traditional ETSI2 decoder will not know this additional meta-data and simply ignore it, because the ETSI2 decoder does not need to use it in the meta-data. If the received PB_C_H is higher than the PB_C, any image is judged. The meta data indicates the brightest possible brightness of the image received (because according to a pure ETSI2 principle, the received image is always the best quality image , In fact, it is the highest quality master HDR image created by the content creator). However, as shown in Figure 11, a general SLHDR2PLUS decoder will not only receive and read the PB_C_H50 value, but also use it to reconstruct the REC_M_HDR image, which is an almost perfect reconstruction of the main HDR image created by the content creator ( In fact, this decoder will use the PB_C_H50 value to calculate the required F_?? function(s) from the received F_I2sCI function(s). This decoder can also advantageously output lower PB_C images, such as, for example, 400 nits of PB_C MDR_300 images, but it can choose to use a standard ETSI2 computing core for such images of PB_C lower than PB_CH, or you can This calculation is performed in an embodiment of the new SLHDR2PLUS computing core (but for the accurate reconstruction of images with a higher PB than PB_CH, new insights are definitely needed because it cannot be done trivially using ETSI2 technology).

Therefore, the tasks set to be solved by the new technology are achieved by a high dynamic range video encoder (900) that is configured to receive an input high through an image input (920). Dynamic range image (MsterHDR), the input high dynamic range image has a first maximum pixel brightness (PB_C_H50), the encoder has a first post data input (922) for the first maximum pixel brightness, and the high The dynamic range video encoder is configured to receive a primary brightness mapping function (FL_50t1) via a second meta data input (921), which defines the normalized brightness and corresponding brightness of the input high dynamic range image The relationship between the normalized brightness of a low dynamic range image (Im_LDR), the low dynamic range image preferably has an LDR maximum pixel brightness equal to 100 nits, characterized in that the encoder further includes a The third meta data input (923) is used to receive a second maximum pixel brightness (PB_CH), and the encoder is further characterized in that it includes: -An HDR function generating unit (901), which is configured to apply a standardized algorithm to transform the main brightness mapping function (FL_50t1) into an adjusted brightness mapping function (F_H2hCI), and the adjusted brightness mapping function makes The normalized brightness of the input high dynamic range image is related to the normalized brightness of an intermediate dynamic range image (IDR). The intermediate dynamic range image is characterized by having a maximum value equal to the second maximum pixel brightness (PB_CH). Possible brightness -An IDR image calculation unit (902), which is configured to apply the adjusted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the output of this unit The brightness of the pixels of the intermediate dynamic range image (IDR); and -An IDR mapping function generator (903), which is configured to derive a channel brightness mapping function (F_I2sCI) on the basis of the main brightness mapping function (FL_50t1) and the adjusted brightness mapping function (F_H2hCI), The channel brightness mapping function, when the respective normalized brightness of the intermediate dynamic range image (IDR) is given as input, the respective normalized brightness of the low dynamic range image (Im_LDR) Defined as output, the brightness corresponds to the respective brightness of the input high dynamic range image (MsterHDR); further features of the encoder are: -An image output (930) to output the intermediate dynamic range image (IDR); -A first meta data output (932) to output the second maximum pixel brightness (PB_CH); -A second meta data output (931) to output the channel brightness mapping function (F_I2sCI); and -A third meta data output (933) to output the first maximum pixel brightness (PB_C_H50).

First notice that although a separate input for each required data item of this encoder is conceptually shown, in practice, readers with general knowledge understand that one or more of these inputs (and similarly for outputs) can be The same depends on what a video input technology can handle (for example, some earlier HDMI video inputs cannot handle this dynamic change-that is, the main brightness mapping function (FL_50t1) may be different for consecutive video images in time ), in this case, the data can be communicated in a synchronized manner through, for example, a Wi-Fi connection). How the various input data is entered can also depend on where they are generated, that is, the encoder is connected to it or to which other system it is connected to (it can depend on whether real-time encoding is expected at the same time the camera captures the event, or It is later coded for a certain video communication system, such as, for example, a cable TV distribution system, which receives all the data from the original content creator at any later time in order to optimally distribute it in view of the limitations or needs of this particular cable TV system Its etc.).

Without deliberate restrictions, it can be assumed that the MsterHDR image has been graded by a human color grader using a color grading software on a computer, and he has defined the FL_50t1 function, which defines a corresponding lower dynamic The range image is generally a 100-nit SDR image (although the current lowest end of the reclassified image spectrum is a 100-nit PB_C image according to the standard protocol, it seems impossible to change the lowest image of the three, namely The LDR image may have an LDR maximum brightness that is not exactly 100 nits in future embodiments, but may be a k multiple of 100 nits, for example, k is preferably at most 3x, that is, in the embodiment of the system The maximum brightness of the LDR in implementation is 300 nits), which corresponds to the MsterHDR image (considering the considerable lower brightness dynamic range, the SDR image preferably looks as similar to the MsterHDR image as possible), which Generally, at least reasonably convey the desired appearance for the best visual telling (for example, movie stories) according to needs (different video applications may also have different requirements, such as different color criteria, which may involve different FL_50t1 functions Technical limitations).

The PB_CH value is somewhat different from other meta-data in that it is actually used for the setting of the intermediate dynamic range encoding. Therefore, it may or may not come from the grader. It can be, for example, a fixed value used in a specific video coding system (for example, a satellite broadcasting system, for example), which can be extracted from, for example, an encoder or a fixed memory attached to the encoder. In Internet-based delivery, the PB_CH value can be communicated according to the needs of an end consumer generated by the IDR image system. For example, a consumer with a poor-quality mobile display may only request a 500-nit PB_IDR image to be calculated by a server on the other side of the Internet, such as the server of an on-demand video company, and some others Consumers can request a 1000-nit PB_IDR version, and in this case, the requested PB_CH=PB_IDR will be input into the encoder.

Therefore, there is a MsterHDR image with the highest quality (in fact, the highest PB_C) on the encoding side. This is not the image that the receiver (complementary decoder) will receive, but the IDR image (and they will need to be calculated by calculating REC_M_HDR And closely reconstruct the MsterHDR image). These techniques are best achieved by formulating everything to be normalized between 0.0 and 1.0 brightness. In fact, when talking about a brightness mapping function, this is actually equivalent to a brightness mapping function (because of the relationship between brightness and their corresponding brightness (for example, the brightness to be displayed in general)), but technically strict In other words, our calculations work better with the brightness mapping function, and are preferably defined by the visually psychologically homogenized brightness, as calculated by the Philips v function (see Equation 1 and Equation 2).

As explained above, our method of handling HDR video (especially not only encodes a single or two different reclassified images in different dynamic ranges of specific peak brightness, but also encodes the whole of the corresponding different DR reclassifications The spectrum) is related to various possible normalized brightness. Such at least two pixels of the relevant image may have, for example, 0.4 in image_2 and 0.2 in image_1. This is defined by the brightness mapping function, between one situation (that is, a reclassification) and any other selected different situations.

Using standardized algorithms means that there must be some fixed way to relate a first set of possible functions (which can have many different shapes and definitions) to a second set of corresponding functions. That is, this only means that in some communication technologies (or even all technologies), the designer of the encoder and the decoder has defined and uniquely specified how the shape of any input function (usually normalized to 1.0 A method of transforming to the shape of the output function. There can be various such algorithms, so in principle the codec designer can determine the order number of any such algorithms that he may want to convey to the decoder, etc.-for example, the number of algorithms is 3, but it is not normally required It is so complicated because our method will work perfectly and most simply by pre-agreeing a fixed standardized function transformation algorithm, such as one of the supporting mathematics below.

To make readers understand quickly, the following will be a simple example of this algorithm. Assuming that the input function is a power function: power(x_in; P), the algorithm can derive the corresponding function power(x_in; P-1). By reversing, when the corresponding functions (by the +1 algorithm) are received, the original functions can be derived again.

It should not be misunderstood that the standardized algorithm itself is generally not communicated to the receiver, only the resulting output corresponding function. This is why it is standardized or scheduled, that is, it must be fixed so that the decoder can know what has happened on the encoding side. The way this is agreed is not so related to understanding the patented technology. For example, there may be 5 different fixed algorithms, and a cable TV operator may decide to encode with Algorithm 3, and supply the corresponding settings to his consumer set-top box to decode fixed Algorithm 3 (even if the STB can In some cases, reset to some other video communication, such as algorithm 4; but the algorithm change will usually not be necessary, although the change on the PB_CH used for different cable TV channels, for example, can be Of interest).

Careful attention should also be paid to the fact that the corresponding adjusted brightness mapping function F_H2hCI is generally not communicated to the receivers, but then another channel brightness mapping function (F_I2sCI) that can be further derived is communicated, and the decoder also needs to use Somehow reverse this double derivation. In fact, the total reclassification map is divided into two parts, so if the first part is standardized, the second part can also be defined, so the inversion of the IDR encoding by the decoder can be regarded as probably possible (Although difficult) (to make the architecture and correct function of the new SLHDR2PLUS codec possible).

The concept of the standardized coded peak brightness dependent function change algorithm has been further illustrated in FIG. 24 . Therefore, on the left, you can see the occurrence of various FL_50t1 functions automatically designed by the grader or content authoring side re-grading function. For example, FL_50t1_1 may have been determined for an HDR scene in which an important action is taking place with a fairly deep black. It must be considerably brightened to still be visible enough on a low dynamic range display, but the brightest parts are not It is so critical-like, for example, a street lamp-and can be represented by a single almost maximum white luminance on any display (or the image calculated accordingly, which contains exactly the absolute luminance as it is waiting to appear on that display, or more To be precise, it is usually the same brightness code that encodes the same brightness). In contrast, FL_50t1_2 is created for an image or a continuous image containing another type of HDR scene, in which there is an important lower-brightness object (or more precisely, an area) and a higher-brightness object. On either side of the "middle" results a specially tuned regrading curve shape. FL_50t1_3 is yet another possible input function applied to the standardized algorithm by the HDR function generation unit 901, which can be used for example for daytime scenes with very bright content and some local darker areas (and main images that are not too high PB_C). ) Occurs, like, for example, entering a temple in an outdoor scene in India.

The unit 901 will determine an output function for any one of these three situations and all the other millions of situations. The nature of this algorithm is that this function will be similarly shaped, but closer to the diagonal (because if the original function is expressed in, for example, the Xnit PB_C image corresponds to (reasonably similar to the extent allowed by the ability) Reclassification between Y nits PB_C2 images (let's say 100 nits of images), and then reclassification from X to Z nits PB_C images with, for example, half of the Z between X and Y will involve a similar reclassification , But to a lesser degree; if mapped from X to X, there will be an identity transformation corresponding to the diagonal).

There are several ways, one of which can define this standardized algorithm to uniquely obtain the output F_H2hCI_1, F_H2hCI_2, and F_H2hCI_3 brightness mapping functions corresponding to the respective input functions, and the details of which do not really form the basis of the present invention Elements, except for the fact that some standardized algorithms must be available for such behavior. For example, some metrics can generally be defined (to quantify the PB_C_CH dependency on the maximum luminance of the selected PB_C_CH IDR image can be encoded), which can be used in a certain way (for example, large equal steps, or non-uniformly, etc.) to make The point y(x) of the input function of any normalized input luminance is offset toward the diagonal. Although it can also be offset vertically, the preferred embodiment described below, which works quite well, offsets such functions on the trajectory orthogonal to the diagonal from [0,0] to [1,1] point.

An advantageous embodiment of the high dynamic range video encoder (900) is characterized in that the standardized algorithm of the HDR function generation unit (901) applies a compression towards the diagonal of the main brightness mapping function (FL_50t1) To obtain the adjusted brightness mapping function (F_H2hCI), the compression involves scaling all output brightness values of the function by a scale factor that depends on the first maximum pixel brightness (PB_C_H50) and the second Maximum pixel brightness (PB_CH).

There can be various defined F_L50t1 functions (the definition of para below is an example), and they can be scaled in various ways by the standardized algorithm, but generally there will be scale adjustment involved, and this scale adjustment depends on At the start PB_C_H50 and target value PB_CH=PB_IDR. This can be done by different metrics, but the applicant has found over the years that it is easy to define the scale factor based on visually psychologically uniform values and by sending them through the ratio of the peak brightnesses of the v function, that is A scale factor is defined based on the v-function brightness output corresponding to the two peak brightness (and possibly the third PB_C of the SDR image).

An advantageous embodiment of the high dynamic range video encoder (900) includes: a limiter (1804) configured to target the normalized brightnesses including the brightest normalized brightness equal to 1.0 Re-determine a slope of the channel brightness mapping function (F_I2sCI) in a sub-range of. This is not necessary for many embodiments, but it is particularly a useful way to deal with the specific selection of the high-brightness gain encoding HG_COD of para, which is standardized in ETSI2, in order to fully comply with the owner of this specific embodiment.

A corresponding mirroring technology to the encoder (in fact, canceling all encoding by being able to re-derive all required information (even if such data is not actually communicated)) is a high dynamic range video decoder (1100), which It has an image input (1110) to receive an intermediate dynamic range image (IDR), the intermediate dynamic range image having a multiplication factor that is preferably 0.8 or less than a main high dynamic range image (MsterHDR) A second maximum pixel brightness (PB_CH) of the first maximum pixel brightness (PB_C_H50), the second maximum pixel brightness (PB_CH) is received via a second meta data input (1112), the decoder has a first meta data Input (1111) to receive a brightness mapping function (F_I2sCI) that defines all possible normalized brightness of the intermediate dynamic range image (IDR) to an LDR maximum pixel brightness low dynamic range image (Im_LDR ) Corresponds to the transformation of normalized brightness. The decoder is characterized in that it has a third meta data input (1113) to receive the first maximum pixel brightness (PB_C_H50), and the decoder includes: -A luminance function determination unit (1104) configured to apply a standardized algorithm to transform the luminance mapping function (F_I2sCI) into a decoded luminance mapping function (F_ENCINV_H2I), the decoded luminance mapping function for the intermediate dynamics The normalized brightness of any possible input of a pixel of the range image (IDR) designates a corresponding normalized HDR brightness of the main high dynamic range image (MsterHDR) as the output, and the normalized algorithm uses the first The equivalent values of the maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH); and -A color converter (1102) configured to continuously apply the decoded brightness mapping function (F_ENCINV_H2I) to the input normalized brightness of the intermediate dynamic range image (IDR) to obtain a reconstructed master The normalized reconstructed brightness (L_RHDR) of the pixels of the HDR image (REC_M_HDR); the decoder further has an image output (1120) to output the reconstructed main HDR image (REC_M_HDR). The LDR maximum brightness is again preferably the standardized 100 nit SDR brightness, although it is conceivable to deploy the low (ie, lowest) dynamic range (ie, maximum brightness) in which the reclassified image spectrum is deployed The image and its communication are future changes in similar operations such as 200 nit images.

Therefore, the MsterHDR image is not actually received as an image, but it is still uniquely defined by the received data (so, although formally, this MsterHDR image exists in the corresponding main image of the corresponding matching decoder location, and The decoder only reconstructs almost the same REC_M_HDR image from the IDR image it receives, and various functions even define the nature of the MsterHDR image at any decoding location). Different consumers can choose various values of PB_C_H50 and PB_IDR. The first one can be selected by the content creator for various reasons, such as, for example, because he purchased a 4000 nits rating monitor, or because he likes to give his main content some kind of best quality (for example, with no less than 10,000 nits The special PB_C establishes/defines everything), or because at least according to the creator, certain types of images require a certain quality, that is, PB_C_H50 (for example, spectacular fireworks or light shows or popular concerts are comparable, for example, A reasonably uniformly illuminated tennis match or news reading deserves a higher PB_C_H50).

The PB_IDR value can be selected based on different technical considerations, such as the pricing of general consumers of a video communication company, and as mentioned, the communication company can usually be different from the creative company.

Generally speaking, it is not reasonable to produce a reclassified IDR whose difference in PB_C is less than at least 20% (that is, factor 0.8, although in principle the value of PB_C can be closer, such as 0.9), it is usually more general The ground between PB_C will have a multiplication factor of 2 or more (for example, 2000 nits sent at a certain PB_CH that is lower than 1000 nits (for example, 800, 700, or 600 nits) and generally higher than 500 nits Main material). The PB_C_H50 at the decoding location is generally similar to other meta data, and especially the value of PB_CH, so it is generally received as meta data associated with video data, such as unrestricted SEI messages or special packets on video communication protocols Etc. (Whether it is in a logical data structure or several structures, according to what is best suited for each standardized or non-standard video communication protocol, this is a secondary detail of this new technology). Because the decoder uses a standardized algorithm to finally arrive at the IDR image and its ETSI2 compliant meta-data, the corresponding standardized algorithm can finally determine the F_ENCINV_H2I brightness mapping function required for reconstruction of the REC_M_HDR image pixel brightness. The decoder is designed or designed in the decoder (then further use this image to accomplish anything, showing that it is a typical application, but for example, it is another application that is stored on the hard disk memory).

An interesting embodiment of the high dynamic range video decoder (1100) is characterized in that the standardized algorithm of the luminance function determination unit (1104) calculates that it depends on the first maximum pixel luminance (PB_C_H50) and the second A scale factor of the maximum pixel brightness (PB_CH). As mentioned, this can be done in various ways, but the visually psychologically uniform v-function-based scale factor is practically easy to use for well-controlled HDR image processing, and meets various even critical artistic requirements , While keeping technical complexity under control.

A useful embodiment of the high dynamic range video decoder (1100) has the brightness mapping function (F_I2sCI) defined by a brightness map consisting of a normalized brightness range for a dark A first linear section with a first slope (SG_gr), a second linear section with a second slope (HG_gr) for a bright normalized brightness range, and a second linear section for the two ranges Between the brightness of a parabolic section. The corresponding mathematics involves, among other things, solving a second-order equation to obtain the channel-adjusted high brightness gain needed for the reconstruction. This is a useful first-order HDR reclassification method, which is suitable for markets with non-highest pixel color control requirements, such as, for example, real-life television broadcasts (as compared to, for example, detailed color control that sometimes involves, for example, blockbuster movies). As mentioned below, in some further divided embodiments, the F_L50t1 function and the individual components of all derivable functions (for example, the function communicated with IDR images: F_I2S) can be fully defined, but it can also be the A part of the definition of the reclassification function, for example, as illustrated in FIG. 4, defines all the reclassification functions together with the same customizable function.

A useful embodiment of the high dynamic range video decoder (1100) has its color converter (1102), which is configured to calculate a medium dynamic range image (MDR_300) with a maximum pixel intensity (PB_MDR) The maximum pixel brightness is not equal to the same value of 100 nits, the first maximum pixel brightness (PB_C_H50), and the second maximum pixel brightness (PB_CH), and the decoder has the medium dynamic range for outputting An image output (1122) of the image (MDR_300). Although the reconstruction of REC_M_HDR images can be everything required by some equipment in some submarkets (all types of other transformations may be applied to the reconstructed image), if our SLHDR2PLUS decoder only reconstructs the main HDR image, It is also advantageous to use other PB_C to calculate the corresponding image (for example, an MDR image that can be directly displayed on a certain display with any PB_C). This will also use the mathematical principle of the present invention, for example, in the manner illustrated in FIG. 16 or any equivalent manner.

Another useful embodiment of the high dynamic range video decoder (1100) has a post-data output (1121) for outputting a brightness mapping function (F_L_subsq) for the brightness mapping function All normalized brightness of the reconstructed main HDR image (REC_M_HDR) or alternatively the medium dynamic range image (MDR_300) defines the corresponding brightness of an image with another maximum pixel brightness, and the other maximum pixel brightness is preferably It is 100 nits, or a value higher or lower than the maximum luminance value of the reconstructed main HDR image (REC_M_HDR) or alternatively the medium dynamic range image (MDR_300). It may be that the received IDR image is reconstructed into a REC_M_HDR image that is not directly displayed on a basic monitor display, but is sent to a certain system for further colorimetric calculation. Then the decoder embodiment can also output a suitable brightness mapping function which is useful, that is, generally means a brightness mapping function associated with the image being output, such as REC_M_HDR image (generally with the defined The normalized brightness of the input of the function is the meaning of the normalized brightness of the images that are output together, and the output of the function is the normalized brightness of a reference image (usually an SDR image), when When it is standardized to have PB_C = 100 nits, it is generally the lowest quality expected in the HDR era. This does not exclude that someone may want to use it to define the system, such as a common transfer function of 80 or 50 nits. The PB_C of the output ordinate applies this teaching).

Anything formulated for a device (or a part or collection of a device) can be equivalently formulated into a signal, a memory product containing images (such as a Blu-ray disc), a method, etc., for example:

A method for high dynamic range video encoding of a received input high dynamic range image (MsterHDR) with a first maximum pixel brightness (PB_C_H50), which includes receiving a master brightness mapping function (FL_50t1), the brightness mapping function definition A relationship between the normalized brightness of the input high dynamic range image and the normalized brightness of a corresponding low dynamic range image (Im_LDR), the low dynamic range image preferably has a relationship equal to 100 nits The method is characterized in that the encoding further includes receiving a second maximum pixel luminance (PB_CH), and the encoding includes: -Apply a standardized algorithm to transform the main brightness mapping function (FL_50t1) into an adjusted brightness mapping function (F_H2hCI), the adjusted brightness mapping function correlates the normalized brightness of the input high dynamic range image Normalized luminance of an intermediate dynamic range image (IDR), the intermediate dynamic range image is characterized by having a maximum possible luminance equal to the second maximum pixel luminance (PB_CH); -Apply the adjusted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the brightness of the pixels of the intermediate dynamic range image (IDR); -Based on the main brightness mapping function (FL_50t1) and the adjusted brightness mapping function (F_H2hCI), a channel brightness mapping function (F_I2sCI) is derived, and the channel brightness mapping function is in the intermediate dynamic range image (IDR). ) When the respective normalized brightness of the low dynamic range image (Im_LDR) is given as the input, the respective normalized brightness of the low dynamic range image (Im_LDR) is defined as the output, and the brightness corresponds to the input high The individual brightness of dynamic range images (MsterHDR); -Output the intermediate dynamic range image (IDR); and -Output the second maximum pixel brightness (PB_CH), the channel brightness mapping function (F_I2sCI), and the first maximum pixel brightness (PB_C_H50).

Or, a method for decoding high dynamic range video once received an intermediate dynamic range image (IDR), the image having a multiplication factor that is preferably 0.8 or less than a main high dynamic range image (MsterHDR) A second maximum pixel brightness (PB_CH) of a first maximum pixel brightness (PB_C_H50), the second maximum pixel brightness (PB_CH) is received as the meta data of the intermediate dynamic range image, and the decoding method also receives meta data A brightness mapping function (F_I2sCI), which defines all possible normalized brightness of the intermediate dynamic range image (IDR) to an LDR maximum pixel brightness low dynamic range image (Im_LDR). The transformation of normalized brightness, and the decoding method is characterized in that it receives the first maximum pixel brightness (PB_C_H50), and the decoding method is characterized in that it includes: -Apply a standardized algorithm to transform the brightness mapping function (F_I2sCI) into a decoded brightness mapping function (F_ENCINV_H2I), the decoded brightness mapping function for any possible input of a pixel of the intermediate dynamic range image (IDR) The normalized brightness designates a corresponding normalized HDR brightness of the main high dynamic range image (MsterHDR) as the output, and the normalized algorithm uses the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH) equivalent value; -Apply the decoded brightness mapping function (F_ENCINV_H2I) to the normalized brightness of the intermediate dynamic range image (IDR) to obtain the normalized reconstructed brightness (L_RHDR) of the pixels of the reconstructed main HDR image (REC_M_HDR) );and -Output the reconstructed main HDR image (REC_M_HDR).

Figure 25 nicely shows the SLHDR2PLUS-encoded HDR image signal (2501) of this case, that is, for example, a 4kx2k pixel color matrix 2502 (YCbCr, or the preprocessor can be used by the known chromaticity equation [not shown] Calculate into any color representation required by YCbCr), and necessary post-data: the luminance mapping function F_I2sCI, and the two PB_C values. If this video signal HDR standard SLHDR2 communicated to the decoder by the decoder 2510 receives, as F_I2sCI based regular function, the decoder would be expected to have a peak brightness of the received image binning (PB_CH from which, the system in this example 800 nits to any lower peak brightness, which can be displayed by the display optimization calculation Im350 for, for example, a connected 350 nits medium dynamic range display optimization (which is not related to this application as described) The key aspect of this is only possible as one of the initial design criteria for the novel codec framework, and can be used for display optimization, such as the method disclosed in US20160307602, or the like). But it is now possible that anyone with SLHDR2PLUS decoder 2520 (for example, newly deployed by the cable TV operator who decided to introduce this service, and the like) can create other images with PB_C higher than the PB_CH value, for example, Reconstruction of an exemplary main HDR image of 5000 nits (ie, output image Im5000), or any displayed adapted image between PB_C_H50 and PB_CH, or even higher than PB_C_H50 (like Im1250 displayed adapted image) .

Figure 9 roughly shows the new SLHDR2PLUS encoder 900. When input, it gets the main HDR image (for example, 5000 nits PB_C image MsterHDR). It does not want to lose the reader can assume that it has been coded by human color graders using color grading software or around the universality, such as borrowing Start with HDR images captured by unprocessed cameras (MsterHDR images are aimed at, for example, the optimization of the TV viewing environment where the night is generally dim (that is, its average surround brightness, etc.); this technology can also be combined with other or variable environments Works, but it is more precisely a problem of display adaptation rather than new methods of creating or encoding HDR images). The grader has also established at least one good luminance degradation function FL_50t1 to convert the 5000-nit master HDR image into the corresponding good-looking SDR image (that is, usually 100 nits) that he has checked on his SDR reference monitor PB_C image), and he has completed this by filling in some of the 403, 404, and 405 parts according to the chrominance processing unit 451, and some good color adjustment F_C[Y]) (for example, in In real-life event broadcasting, other methods can calculate the applicable function shape in operation, and then some director may watch the result roughly, or even not, but the principle is that a good function FL_50t1 appears, regardless of whether it is only one from some units. Or the total function of all units together, etc.).

This function FL_50t1 must also be input as the start information of the novel encoder 900. Also input the peak brightness static (for the overall movie or broadcast) post data PB_C_H50, because it will be used, but it is also output by the encoder as the total IDR video coding signal (IDR+F_I2sCI+PB_CH+PB_C_H50, where the image is generally based on Some suitable video communication standards (for example, HEVC) are used to compress or decompress, and other meta-data can be communicated according to any available or configurable meta-data communication mechanism, ranging from MPEG SEI messages to private Internet packets, etc. ).

The HDR function generating unit 901 will calculate the HDR to IDR luminance mapping function F_H2hCI, which is required to calculate the IDR image from the Mster HDR image, and it will require the selection of the IDR PB_CH, assuming it is derived from some other input (for example, this It may have been selected by the cable TV operator and placed somewhere in the memory to be loaded by the configuration software; it will be assumed that PB_CH is equal to 1000 nits (for illustration purposes only; in general, this value will be SDR PB_C is several times high (for example, 4x high), based on the selected value, the technical aspect is somewhat different in the embodiment details).

FIG. 10 illustrates how the HDR function generating unit 901 can function.

Assuming that the classifier has defined a certain function (here, in the illustrated example, it is a linear-parabolic-linear function (abbreviated as para ), the applicant uses it according to the ETSI standardized codec principle to do the first setting of the brightness of the image area. Mostly good rebalancing (ie, it provides sufficient visibility for dark colors in SDR images at the cost of co-control compression, for example, in the brightest areas).

This function makes the input brightness of the darkest sub-region (L<Ld) of brightness by having a linear relationship with a controlled slope SG_gr optimally selected by the grader for this HDR image (in accordance with the above equation 1 And equation 2 transforms the visual psychological equal representation of pixel brightness) is related to the required output brightness: Ln_XDR= SG_gr *Ln_Mster_HDR if(Ln_Mster_HDR＜Ld) [Equation 4] (Ln_Mstr_HDR and Ln_XDR are respectively the brightness of the input main HDR image when pre-graded as the best starting image by the grader (that is, the uniform representation of the corresponding pixel brightness in visual psychology), and the Ln_XDR system has different dynamics The range and especially the summary of the output brightness of the image of the peak brightness PB_C are all shown on the same normalized vertical axis in order to explain the concept behind the present invention and its embodiments). In particular, when the grader starts to re-grade the corresponding best SDR image for his best graded Mster_HDR image, XDR will be the SDR of this type, and the corresponding brightness mapping function shape will be displayed as F_Mt1 [Use The shorthand symbol xty indicates that the function maps the brightness from the start PB_C x to the end PB_C y, and any one of x and y can roughly indicate the PB_C of the image, such as M indicating Master, or numerically indicating instance values , Where two zeros are dropped, for example, 50 means 5000 and 1 means 100 nits].

Similarly, for input brightness Ln_Mster_HDR higher than Lb, there is again a controllable linear relationship: Ln_SDR=HG_gr* Ln_Mster_HDR +(1-HG_gr) if(Ln_Mster_HDR＞Lb) [Equation 5]

The parabolic part of para extending between Ld=mx-WP and Lb=mx+WP has the function definition of L_XDR=a*x^2+b*x+c, and its coefficients a, b, and c can be calculated by The line tangent is calculated from the point of the curve where its poles intersect and its abscissa mx (as defined in the ETSI1 standard; mx=(1-HG)/(SG-HG)).

The general idea underlying the present invention is as follows (and it can be explained in a multiplicative view). Any master HDR brightness can be transformed into itself by applying an identity transformation (diagonal). If at the end of the spectrum of the reclassified image (that is, to establish the corresponding SDR brightness (XDR=SDR)), the output brightness L_SDR=F_Mt1(Ln_M) must be obtained, where Ln_M is a specific value of Ln_Mstr_HDR brightness, so Think of this as the multiplicative promotion of input luminance L_SDR=b_SDR(Ln_M)*Ln_M. If an intermediate function F_Mt1_ca can now be defined, the final process is to apply two functions continuously F_IDRt1(F_Mt1_ca(Ln_Mster_HDR)), where F_IDRt1 starts from the calculated IDR pixel brightness (derived from the main HDR brightness) towards any pixel ( (Or object) the final brightness mapping of the SDR brightness. In the form of multiplication terms, it can be said that L_SDR=b_IDR*b_ca*Ln_M, two of which correspond to the intermediate function (or channel adaptation function) and other related functions (which happen to be communicated with the IDR image to create an ETSI2 compliant HDR video Encoding the function). Note that these boost factors are themselves a function of Ln_Mster_HDR (or indeed any intermediate brightness associated with it).

Now it is convenient if you don't need to convey any additional functions (if the meta-data management is not perfect, it may be lost, for example).

Therefore, if the SLHDR2PLUS principle uses a predetermined fixed method to transform the grader’s F_Mt1 function (that is, the mechanism for any function shape he expects to use) to PB_IDR (according to the ETSI2 encoding method, this value is generally used as PB_CH The corresponding channel adaptation function can be useful. The displayable up-grading function F_H2h does not need to be shared in the meta data associated with the IDR image, because it is fixed and known to the decoder, so the inverse function F_?? may be calculated from the received F_I2s function, as shown It will be displayed (if PB_C_H50 is also transmitted to the receiver). The novelty of the decoder is the new method of deriving the image of PB_C> PB_IDR. Theoretically, any fixed method of deriving the F_Mt1_ca function from the main F_Mt1 can be carried out, provided that it is mathematically reversible or at least decodable according to demand, but it is desirable to choose to perform HDR to IDR reclassification (that is, to derive the F_Mt1_ca shape) such that It is used to derive a method compatible with the further deformation of MDR images and those produced by ETSI2 (in theory, only ETSI2 images are standardized between PB_C and 100 nits, so the dynamics between PB_IDR and 100 nits can be All images in the range require the image appearance (that is, the brightness and color of all pixels) to start with close equality, but it is also possible to try to impose technical limitations on the solution to be obtained. The technical limitation is from the received IDR to the main The HDR image upgraded image (that is, using F_?? to be calculated by the SLHDR2PLUS decoder) has the same appearance as that obtained by the display adjustment of ETSI2, which is used to receive, for example, 5000 nits PB_C Mster_HDR image, And the total brightness remapping function F_Mt1.

First, explain how this better channel adaptation (that is, the calculation of F_Mt1_ca, or F_H2hCI calculated in Figure 9; and the corresponding IDR image(s)) can be designed into several methods/embodiments of SLHDR2PLUS it works.

FIG. 12a shows the white level offset WLO_gr optimally selected by the grader (or automatically), and if available, the black level offset (BLO_gr) is also the same; corresponding to the unit 403 in FIG. 4.

At this time, it can be assumed that this is the only dynamic range adjustment, that is, the brightness mapping operation to obtain the SDR image from the Mster_HDR starting image (the white is on white and the black is on black (black-on-black)). Basic dynamic range conversion for poor-quality LDR images, which have neither the correct average brightness nor the average visual contrast, let alone the expected higher image quality descriptor of the resulting image, but as According to the first step of the reclassification chain of the applicant's method, it is a good step, and we must first explain this step and its channel adjustment). The idea is that if (despite the possibility of at most PB_HDR=5000 nits code brightness) the image to be mapped is actually currently (or if it is decided to use the same function for all the time-continuous images, the image in the video of the same scene If there is no pixel brightness higher than the value MXH in the lens), it is reasonable to map the highest MXH to the maximum brightness code in the SDR (ie, 1024, corresponding to a brightness of 100 nits). Any other mapping method (for example, HDR-white-on-SDR-white mapping) will make all the actual current brightness even darker, and it is not optimal, given the SDR brightness The range is small enough, and a large-range corresponding simulation that best contains HDR brightness is still needed.

Then, the question is whether this WLO value should be adjusted for the IDR image (as can be seen in Figure 12b, the brightest brightness in the intermediate image may have been reduced to be closer to PB_IDR, and there will still be a final offset for the SDR reclassified image Shift to map on 1.0; the map can also be equivalently displayed on the normalized luminance range of the HDR 5000 nit image, as indicated by ON). In the first method, it is not needed (because there is some freedom in how to design the algorithm used to calculate the F_Mt1_ca function), but if it is adjusted proportionally, the following method can be used.

The scale factor used for such level scaling needs to be determined in order to be able to scale the brightness mapping function, which in this case is its parameter WLO_ca, and similarly scaled BLO_gr (symbol BLO_ca). If this parameter is expected to be adjusted linearly and proportionally with PB_IDR, the restriction is that the action is fully performed, that is, when PB_IDR=PB_SDR, the offset has its maximum degree of BLO_gr. On the other hand, for HDR images, BLO or WLO should be zero, because there is an identity transform for mapping 5000 nits of Mster_HDR to Mster_HDR, so there is nothing to correct.

Therefore, this definition of the parameter can be formulated WLO_ca=scaleHor*WLO_gr (0＜=ScaleHor＜=1) BLO_ca=scaleHor*BLO_gr [Equation 6]

Then the question is how to define ScaleHor.

Figure 12b shows the spectrum of different dynamic ranges, more specifically, different PB_C images organized along the horizontal axis. They are located along the perceived position of the peak brightness PB_C of each image. Therefore, place them on the abscissa position of the system v(PB_C), whereby v is the function equation 1, where the value PB_C is used for the parameter L_in, and has the equation 2 for the peak brightness calculation of the graded Mster_HDR image Value RHO (ie, for example, for 5000 nits PB_C Mster_HDR images, RHO is 25). If the ordinate axis also makes its brightness L parameterized according to the v function (on the vertical axis), with the same RHO=25, then PB_C follows the straight line well, and the definition and calculation can be completed in this framework. For example, the brightness of the peak brightness PB_C of any intermediate image can be projected onto the main (5000 nits) brightness axis. The symbol used is "P_I1oI2", which means the brightness value corresponding to the application of the v function of the peak brightness of the image I1 (which is the normal brightness) when it is expressed in the brightness range of the image I2. So, for example, P_IoH is the peak brightness of the selected IDR image in the Mster_HDR brightness range, and P_SoH is the brightness of 100 nits (note that 1.0 in this range corresponds to the PB_C of the Mster_HDR image, so that, For example, the position of 100 nits (for example, 0.5) will change depending on the selected Mster-HDR image representation, which is why Equation 1 and Equation 2 are the RHO parameterized curve family).

Then the appropriate function for ScaleHor will start from 1-P_IoH. The more the PB_IDR decreases, that is, the more to the right, this function will indeed increase our IDR image representation of the MsterHDR image. And if P_IoH=1, it will produce 0, which occurs when the IDR image of 5000 nits is selected (purely used for the theoretical explanation of the scaleHor equation, because it is technically unreasonable). However, when IDR=SDR, this equation is not equal to 1.0, so it needs to be adjusted proportionally with a factor k.

If k=1-P_SoH (which is a fixed value compared with the variable P_IoH corresponding to various IDR positions), it can be verified that the normalization system is correct, so: ScaleHor=(1-P_IoH)/(1-P_SoH) [Equation 7]

The determination of the correct para for this channel conversion (FIG. 4, unit 404) is more complicated and is illustrated with FIG. 13 .

In this case, the inventor decided to perform the function transformation in the diagonal direction orthogonal to the identity diagonal ([0,0]-[1,1]). This must be converted with equivalent parameterization in the normal Mster_HDR/XDR coordinate system representation of the reclassification of all functions.

The basic proportional adjustment is defined in a 45-degree rotation axis system that changes the diagonal to the horizontal axis (Figure 13a). See that it is a function Fx that is rotated para, for example. It is reasonable to adjust any value dY in proportion to the points on the rotated diagonal (ie, the new x axis) by the factor La/K (the dX corresponds to a certain abscissa, that is, L_Mster_HDR in the original axis system) Brightness), the complete action of the K system function, that is, the complete dY value, and the proportionally adjusted dY_ca value will be (La/K)*dY in this rotating system.

Define sc_r=La/K, where La=1/P_IoH and K=1/P_SoH (It should be noted that the value of I2 brightness on the I1 axis can be reformulated to the value of I1 brightness on the I2 axis, especially for example 1. /P_IoH=P_HoI; for example, if P_IoH=0.7, this means that PB_Mstr_HDR will be nailed to 1/0.7 above PB_IDR).

Now it is necessary to calculate the equivalent vertical scale of the diagonal sc_r to adjust sc*.

This can be done by applying reverse rotation mathematics (actually by first defining K and La as 1.0 instead of 1.4), bringing the representation of Figure 13a to the diagonal of Figure 13b. This is generated by matrix rotation (rotation to any x_r, y_r in the diagonal system of the main representation, such as 1, dY): [x1,y1]=[cos(pi/4) -sin(pi/4); sin(pi/4) cos(pi/4)]*[1, P_HoI= 1/La] [x2,y2]=[cos(pi/4) -sin(pi/4); sin(pi/4) cos(pi/4)]*[1, P_HoS= 1/K] [Equation 8]

It should be noted that because the diagonal is adjusted proportionally, both the x-coordinate and the y-coordinate will change, but in any case, SG and HG (and any other proportional adjustment points) are defined as slopes rather than angles.

The rotation of the line from (0,0) in Figure 13b to the square representing the diagonally proportional adjustment point of the brightness mapping function from (0,0) to the circle of the original brightness mapping function point (or And vice versa) can be found by dividing the slope by any fixed abscissa value (for example, a) (with the angle change corresponding to the vertical change of the normalized scale factor sc*): sc* = (y2/x2)/(y1/x1)=[(1+1/K)/(1-1/K)]/[(1+1/La)/(1-1/La)] =[(K+1)/(K-1)]/[(La+1)/(La-1)] =[(La-1)*(K+1)]/[(La+1)*(K-1)] [Equation 8]

Subsequently, the actual ordinate distance n corresponding to all vertical scaling (sc*=1) must be calculated, and this can be achieved by making the midpoint of the scaled mip system due to the 45 degree angle in the diagonal ( It is completed with a distance Fd from below it to the diagonal and above it to the intersection point (mx, my) of two linear segments of para. Therefore, n=Fd is equal to half of the differential slope SG-1 at mx, that is, mx*(SG-1)/2.

Subsequently, the offset intersection point (mxca, myca) must be calculated as follows: mxca=mx+d= mx+ [mx*(SG-1)/2]*(1-sc*) myca=my-d=SG*mx-(mxca-mx)= -mxca+ mx*(SG+1) [Equation 9]

Using the position of the new point, the channel-adjusted shadow gain (SG_ca, see Figure 10) and the channel-adjusted high brightness gain HG_ca can be calculated finally: SG_ca = myca/mxca HG_ca=(myca-1)/(mxca-1) [Equation 10]

Finally, there are several methods/embodiments for the middle section of the parabola.

In a way to produce practically good visual results, take WP_ca=WP_gr, where WP_gr is the original parabolic section when optimized by the grader or automation of the content creator of the main HDR and main SDR images Width, and WP_ca is the width of the para function adjusted by the channel. Another method is to define WP_ca=v(abs(sc*), 100)*WP_gr, where the v function is again defined by the above equations 1 and 2.

Make it a usable technology that can be used to define a suitable IDR definition for SLHDR2PLUS.

Returning to FIG. 10, the above equation defines how the function F_Mt1_ca can be uniquely defined for the selected 1000 nit PB_IDR starting from, for example, a 5000 nit master HDR image. If this function is determined by the HDR function generating unit 901, it can be output as F_H2hCI and sent as the input of the IDR image calculation unit 902. This unit will apply this function to all the pixel intensities of the MsterHDR image received as the image input [L_IDR=F_H2hCI(L_MsterHDR)= F_Mt1_ca(L_MsterHDR)] to obtain the corresponding IDR image pixel intensity, and will output the IDR image.

The question is still which brightness mapping function to add to the IDR meta data to make it appear like a normal ETSI2 image (that is, so that any traditional ETSI2 decoder can decode it normally, and produce it as it should. SDR image or any MDR image that looks like it).

This secondary IDR luminance mapping function F_I2sCI (which will also be para) can be defined as follows (and will be calculated by the IDR mapping function generator 903). The shadow gain used for the IDR image SG_IDR can be regarded as the remaining multiplication (or slope) after going from Mster_HDR to the IDR image (that is, the remaining relative brightening from the IDR image to obtain the SDR image): Y_out (x_in)=SG_gr*x_in; = F_I2sCI(L_IDR=SG_ca*x_in)

It is also known to apply the same para linear segment mapping for the darkest pixel to the new IDR brightness input: Y_out=SG_IDR*L_IDR therefore: SG_gr= SG_IDR*SG_ca [Equation 11] (For example, take the input x_in=L_Mster_HDR=0.2, which maps from the diagonal to L_IDR=0.3=(0.3/0.2)*x_in, and finally maps to Y_out=0.4=k*0.3, where k=0.4/0.3; Y_out =SG_gr*0.2=(0.4)*0.2=(0.4/0.3)*(0.3/0.2)*0.2).

Therefore, from Equation 11, follow this method to calculate the required SG_IDR (in view of the fixed method used to determine SG_ca as described above): SG_IDR=SG_gr/SG_ca [Equation 12] Similarly: HG_IDR=HG_gr/HG_ca [Equation 13]

Among them, HG_gr is again the best high brightness gain determined by the content creator who correlates the main SDR image pattern with the main HDR image pattern (that is, its brightness distribution), and HG_ca corresponds to the original high brightness gain HG_gr. Channel adjustment is high brightness gain.

It should be noted that the basic shadow gain adjustment can be determined by the expected simple shadow gain from the difference in peak brightness between SDR and IDR images as: ShadBst=SG_IDR/P_IoS. As mentioned, when expressed on the normalized brightness axis of the SDR image, P_IoS is the maximum coded brightness of the IDR image, that is, for example, 7.0.

Note that there are some practical embodiments where the high-brightness gain cannot be greater than the predefined number (the high-brightness gain is programmed in the ETSI standard). In this case, a further recalculation of the high-brightness gain is required. See below, but this applies to all implementations. For example, it is not necessary. For example, this can be implemented as:

If HG_IDR>KLIM then HG_IDR_adj=KLIM [Equation 14], where KLIM is preferably equal to 0.5.

Indeed, assuming that the classifier has made HG_gr close to the maximum value of 0.5, and the corresponding HG_ca (which as a softer map should have HG_ca closer to the diagonal, that is, greater than HG_gr) is, for example, 0.75, the division system is found 0.67, which is higher than the maximum value that can be communicated based on a pure ETSI2 HDR video signal when standardized. One solution is to redefine the smaller HG_gr so that HG_IDR will not be higher than 0.5 (the standardized maximum value). This again requires that all reclassification aspects be taken into account in a considerable calculation, as will be shown below. Another option is to limit the HG_IDR to 0.5, so that the IDR + meta-data signal is compliant, and at the same time communicated as additional meta-data (HG_IDR is completely unrestricted). HG_gr will generally depend on the PB_C of the Mster_HDR image, but it also depends on what image object is in the image (for example, bright and colorful objects, which are important enough to not compress their brightness too much, an extreme case is close to the powerful sun The image of the bright planet, which uses many very high L_Mster_HDR brightness values and very few dark ones). HG_ca will generally depend on how close the selected PB_IDR is to PB_Mster_HDR among other things.

In addition, suppose WP_IDR=WP_gr [Equation 15]

As mentioned, other embodiments are possible, but the principles are explained in an easier way, and this hypothesis is now generated.

Use Equation 6 to calculate the appropriate channel-adjusted values for the black level offset and white level offset (if any such offset is defined by the content creator). Now the rest is how to calculate (by IDR video encoder) the corresponding values of BLO_IDR and WLO_IDR.

First, calculate in a better way to encode the value glim: glim= {log[1+(rhoSDR-1)*power((0.1/100);1/2.4)]/log(rhoSDR)}/{log[1+(rhoHDR-1)*power(1/PB_Mster_HDR; 1/2.4)]/log(rhoHDR)} [Equation 16] Where rhoSDR=1+32*power(100/10000;1/2.4), and rhoHDR=1+32*power(PB_Mster_HDR/10000; 1/ 2.4)

This will lead to a simple way to adapt BLO, because in fact, in the ETSI1 and ETSI2 standard methods of HDR encoding, there are also parallel to the luminance processing chain (units 402 to 406 in Figure 4 and 1502 to 1506 in Figure 15). A linear curve with an angle glim is applied to the perceived Y'HP and compared with the Y'GL value calculated by the explained unit, and the largest of the two values calculated in parallel is obtained (this, among other things, is for ETSI1 Reversibility is important to allow the reconstruction of the darkest HDR brightness) linear gain limiter. These diagrams are only used for easy understanding of part of the sequential reclassification steps of the described inventor's method.

It can now be shown that due to the action of this limiter, the BLO value can be easily adjusted to the channel by the following equation: BLO_IDR=BLO_gr*glim [Equation 17] glim depends on the specific selection of PB_Mster_HDR as shown above, and can be, for example, 0.6.

This is illustrated in Figure 17 . Figure 17b shows an enlargement of the darkest brightness of the full range brightness map shown in Figure 17a. The various functions are displayed on the standardized chart again, and these functions correspond to various input PB_C and output PB_C.

FL_gr is a function created by content creators to map, for example, 4000 nits Mster_HDR to SDR. The dotted curve FL_ca is used to generate channel adaptation such as 500 nit IDR from Mster_HDR. The dashed curve FL_IDR is a curve that maps the IDR brightness to the SDR brightness. In the enlarged chart of Figure 17b, it can be seen that the FL_gr curve has a sharp twist at the input of about 0.03, which is where the parallel gain limiter works (that is, its linear output y=glim*Y'HP is selected as the lower The output of the function of the brightness input, instead of the Y'GL value from the actions of all the units in the chain shown in Figure 4 (for full circuit description, see ETSI1 standard, Figure 4)).

The BLO value of any curve is the intersection with the horizontal axis that will occur if there is no gain limit, that is, for example, as shown by the dotted line by extending the local slope to BLO_gr higher than 0.3 of the FL_gr curve.

For this application, know that the FL_IDR curve can also be extended to obtain the BLO_IDR value (note that there is a glim_IDR value, which is used by the ETSI2 standard, which is different from glim_gr), and that this lower BLO_IDR value can be found as glim*BLO_gr (It should be noted that this glim (the only glim that needs to be calculated for SLHDR2PLUS) is what is shown as glim_gr in Figure 17b) is sufficient.

Subsequently, the following calculation is performed to obtain WLO_IDR.

Figure 17a also shows that there are three different WLOs, that is, the WLO_gr (also ON in Figure 12b) that was originally generated by the grader as his master HDR to SDR mapping strategy, and the FL_ca curve crosses the upper horizontal line. The channel-adapted WLO_ca and the mapping from WLO_gr brightness to IDR brightness axis (it can be imagined using a representation like Figure 12, where MXH is projected to MXI, and there is also WLO_IDR at the end, which is left WLO used to map IDR brightness down to SDR (when starting from the associated PB_C=5000 for WLO_gr and WLO_ca (because the input image used for reclassification using these functions is 5000 nits Mster_HDR) When PB_C=1000 nits for the IDR-related definition required for reclassification (because in the ETSI2 compliance point of view, the received initial image system from which other images are derived, for example, 1000 nits PB_C IDR image), It is different from the scaled WLO_ca because of the change in the normalized brightness abscissa definition).

Figure 17c zooms in on the upper corner of the function graph (close to [1,1]). As shown by the circular projection from the (normalized) ordinate position to the abscissa position, the WLO_IDR value follows the WLO_gr value that will be sent as input through the FL_ca curve. It can be seen in Figure 12b that the MXI position is indeed the normalized position mapped to the SDR brightness of 1.0 on the IDR brightness axis, so it is required by the definition of WLO_IDR.

On the surface, it can be considered that if the WLO value subsequently passes through the mapping curve para on the encoding side (see FIG. 4, the unit 404 is mapped after the unit 403), it will generally be the upper linear section of para.

However, because of how para is defined, any part of it can be involved (there is even a setting where the definition of a special value of SG of only para is theoretically moved to a very high intersection point higher than 1.0, so in the case of at most the brightest brightness The behavior is only determined by the shadow gain slope, resulting in a linear curve useful for regrading HDR images that mostly contain very bright brightness to SDR, such as, for example, a desert planet illuminated by 5 suns in a science fiction movie). Therefore, this becomes a bit involved in calculations, where it needs to test which of the three sub-parts of para can be applied, a better mathematical realization system: WLO_co=255*WLO_ca/510 BLO_co=255*BLO_ca/2040 Xh=(1-HG_ca)/(SG_ca-HG_ca)+WP_ca WW=(1-WLO_gr*255/510-BLO_co)/(1-WLO_co-BLO-co) IF WW＞=Xh THEN WLO_IDR=HG_ca*(1-WW)*510/255 [The upper linear section] ELSE { Xs=(1-HG_ca)/(SG_ca-HG_ca)-WP_ca IF WW＞Xs {[Input, that is, WLO_gr must be mapped through the parabolic sub-part of the parabolic channel adjustment] A= -0.5*(SG_ca-HG_ca/(2*WP_ca)) B=(1-HG_ca)/(2*WP_ca) + (SG_ca+HG_ca)/2 C= -[(SG_ca-HG_ca)*(2*WP_ca)-2*(1-HG_ca)]^2 / (8*(SG_ca-HG_ca)* 2*WP_ca) WLO_IDR=(1-(A*WW*WW+B*WW+C))*510/255 } ELSE [In the special case where the shadow gain sub-part of para is applied] WLO_IDR =(1-SG_ca*WW)*510/255 }

These parameters SG_IDR, HG_IDR, WP_IDR, BLO_IDR, WLO_IDR (and if necessary, similar additional parameters for the customizable curve) are the parameters of the characterization and therefore as the output parameters of the function F_I2sCI (in fact, whether to output the characterization of this The shape of the curve is required to display these parameters for adjustment, or whether to output the LUT characterizing the function is only an embodiment choice; the main one is the correct brightness mapping function shape F_I2sCI in the axis system normalized to 1.0 as the back Suppose the data is shared with (multiple) IDR images).

The encoder is now characterized according to the novel SLHDR2PLUS method. Then the question is how to design the decoder. It must be understood that this decoder will now only get the F_I2sCI function, so it must slightly calculate the function F_?? required to reconstruct the original Mster_HDR image from the received IDR image. In this SLHDR2PLUS encoding method, this will be the inverse function of the F_H2hCI function used in the encoder to generate IDR brightness, but such functions should still be computable.

As illustrated generally in FIG. 11, SLHDR2PLUS video decoder 1100, the luminance function determining unit 1104 must be calculated only F_ ?? function based on the information it receives, i.e. two peaks and brightness PB_CH F_I2sCI and PB_C_H50. Once the function is determined, it can be applied to reconstruct the original Mster_HDR brightness, by applying it (in the color converter 1102) to the received IDR brightness: L_REC_M_HDR=F_??(L_IDR), the corresponding HDR brightness can be obtained by applying The inverse functions of Equation 1 and Equation 2 are calculated from the brightness of the L_REC_M_HDR. Finally, the reconstructed main HDR image (REC_M_HDR) can be output by the color converter 1102 in any format as needed, for example, a PQ-based YCbCr color formula. The decoder 1100 can also be configured in the preferred embodiment to calculate any display-adapted image, for example, MDR_300, if 300 nits PB_D is connected to the display to be supplied with the best equivalent of the HDR image to be received, and this It can be done by SLHDR2PLUS mathematics, or only by the formal ETSI2 decoding, because the proper image (IDR) and the luminance mapping function (F_I2sCI) are already available as input in the color converter 1102).

Figure 14 shows those involved in para to reconstruct the REC_M_HDR image from the received IDR image (similar calculations will be completed for WLO and BLO, as well as the customizable curve shape points, where applicable (note that as discussed below, Some embodiments will not be between Mster_HDR and IDR, but just like SDR downgrade technology, that is, apply the principle of customizable curve between IDR and SDR).

Now, the new main HDR reconstruction shadow gain (SG_REC) and reconstruction high brightness gain (HG_REC) need to be calculated, and the inverse parabola equation of the parabolic section must be calculated to complete the required reconstruction para brightness mapping function shape F_L_RHDR (note that it is only used for For illustrative purposes, the inverse SDR to Mster_HDR luminance mapping function has also been shown as a dotted line on this normalized graph; it should be noted that due to the inverse nature of the SDR to HDR mapping, the shadow gain of the curve SG_RM is equal to 1/SG_gr, etc.).

Figure 15 first illustrates some aspects of the core computing topology of the general decoder 1502. As can be seen, it is roughly the same structure as the encoder, although it performs reclassification in the opposite direction (reconstruction of REC_M_HDR from IDR). When this computing topology can be easily reconfigured as required, it Department of convenience. If the luminance mapper 1501 obtains the total LUT (all parts of the continuous reclassification action), it will indeed operate in a similar manner (like a decoder).

Of course, some differences need to be configured to enable the decoder to perform the correct HDR reconstruction and reclassification. First of all, L_in will now be the IDR normalized luminance, and the output luminance Lh will be the normalized luminance that is scaled correctly for, for example, a 5000-nit PB_D display. Also see that the last multiplier that produced the REC_M_HDR image pixel color (Rs, Gs, Bs) is now multiplied by the PB_C_H50 value received in the meta data. In fact, the perceptual external calculation loop performed by the perceptualizer 1502 and linearizer 1506 applies the values of PB_CH and PB_C_H50 to Equation 1 and Equation 2 and the inverse equations of these equations, respectively. It should also be noted that the order of reclassification of various parts is reversed when they exist: firstly, the IDR brightness Y'IP is sensed by generating the reclassified IDR brightness Y'IPG in the fine classification unit 1503. The curve is refined and graded. After that, the first mapping to the HDR brightness axis (that is, the corresponding redistributed brightness for the corresponding correct HDR pattern (in fact, 5000 nits PB_C_H50 Mster_HDR pattern) is performed by the rough brightness mapping unit 1504, which Applying the reverse para of FIG. 14, it still needs to be calculated correctly, and it will produce the initial HDR brightness Y'HC. Finally, the anti-black and white shifter 1505 will establish the correct normalized REC_M_HDR brightness (Y'HR) to be used in further calculations with the chromaticity to reach the full three-dimensional color of each pixel. As explained, the unit 1504 will generally obtain the calculated SG_REC etc. (or the LUT version of the brightness mapping function to be applied to these three values accordingly). It should be noted that if the various PW values remain the same, WP_REC is WP_gr again. The unit 1505 will similarly obtain the black and white offsets (WLO_REC, BLO_REC) used for reconstruction of Mster_HDR. The lower part of the core unit for chroma processing (chroma processor 1550) will be similar to the encoder topology in Figure 4, except that the correct C_LUT F_C[Y] is loaded in the chroma processing determination unit 1551 (see the explanation below Its calculation).

The question now is whether and how to calculate the parameters of the function applied to the decoder programmed to reconstruct Mster_HDR from IDR (this is not the case before HDR video decoding).

For example, you can see a method for shadow gain.

Before calculating SG_REC, it can be asked whether the total shadow gain SG_RM from SDR to Mster_HDR can be determined, and then SG_REC can be determined by itself via the division of Equation 12. So SG_IDR=SG_gr/SG_ca It can also display SG_ca= (mx/mxca)*(SG_gr+1) -1

This can be seen because myca=SG_ca*mxca (according to the definition of the lower linear section of para adjusted by the channel), and also myca=my-d = mx*SG_gr+(mx-mxca).

The second relationship of mxca/mx is followed by dividing the upper formula of Equation 9 by mx.

Because by filling the first relationship into the second (removing the mx/mxca part), SG_ca can be written as SG_gr, and now the final relationship can be formed between SG_IDR and SG_gr: SG_ca= (SG_gr+1)/[(SG_gr-1)*(1-sc*)/2+1]-1 Since then: SG_IDR=SG_gr/{(SG_gr+1)/[(SG_gr-1)*(1-sc*)/2+1]-1} ［Equation 18］

Given that the known (received) SG_IDR (and sc* has been calculated only from the peak brightness (which is also known), because both PB_CH (ie, PB_IDR) and PB_C_H50 have been received and PB_SDR is usually 100 nits, but if not This can also be placed in the post data of the signal), this equation can now be solved for the unknown SG_gr.

Use SG_IDR = y and SG_gr = x to simplify notation, and then: y=[(x-1)*(1-sc*)*x/2+x]/[x-(x-1)*(1-sc*)/2] Therefore: x^2+x*(y-1)*[(sc*+1)/(sc*-1)]-y=0 [Equation 19] [These coefficients which are functions of y and sc* (hereinafter referred to as A', B', C') will be used in the overall system of the equation used to reconstruct the brightness of the Mster_HDR image to solve two Equation].

In order to determine all the parameters of the shape of a given reconstruction luminance mapping function, the following equation can generally be completed in one of the embodiments (this reconstruction is used to generate the inverse function of the IDR image on the encoder side). First, determine the correct para, the black and white offset can be calculated from it later.

Calculate rhoSDR as above again, and calculate rhoCH as: rhoCH=1+32*power(PB_CH/10000; 1/ 2.4) mu = log[1+(rhoSDR-1)*power(PB_CH/PB_SDR; 1/2.4)]/log(rhoSDR) K and La and sc* are calculated as above, where K=P_HoS and La=P_HoI A’=1 B’=(SG_IDR-1)*(sc*+1)/(sc*-1) C’=-SG_IDR

Once the necessary parameters of all required functions can be determined on the decoder side (note: the available parameters SG_IDR etc. are received from other channels), the rest of the decoding is due to the reversibility of the inverse curve(s) that have just been coded (for example, like The para in Figure 14 (appropriately shaped by calculating its appropriately defined parameter 1/SG_REC, etc.) will cancel the action of IDR encoding para as shown in Figure 10, that is, define IDR to Mster_HDR brightness Re-decoding etc.).

Follow SG_gr=[-B’+SQRT(B’ ^2-4*A’*C’)]/2*A’

Where ^2 indicates square. SG_REC = SG_gr/SG_IDR [Equation 20]

Therefore, the anti-channel adaptive shadow gain (1/SG_REC) is already known.

Similarly, the required high brightness gain can be calculated. A’’= (SG_REC*HG_IDR -SG_gr)*(SG_gr+1)/(SG_REC+1) B’’=SG_gr-HG_IDR-(SG_REC*HG_IDR-1)*(SG_gr+1)/(SG_REC+1) C’’=HG_IDR-1 MxRec=[-B’’+SQRT(B’’ ^2-4*A’’*C’’)]/2*A’’ IF MxRec =1 THEN HG_REC = 0 ELSE = HG_REC = max[0,(MxRec *SG_gr-1)/(MxRec -1)]

When the para function is defined from its parameters, once the parameters are calculated, the required para is defined.

To obtain BLO_REC and WLO_REC, execute the following equations: mx=(1-HG_gr)/(SG_gr-HG_gr) mxca=mx*(SG_gr-1)*(1-sc*)/2+mx myca=mx*(SG_gr+1)-mxca SG_ca=myca/mxca IF mxca=1 THEN HG_ca=0 ELSE HG_ca=max[0, (myca-1)/(mxca-1)] ScaleHor=(1-1/La)/(1-1/K) RHO=1+32*power(PB_C_H50/10000; 1/2,4) glim = {log[1 + (rhoSDR-1) * (0.1/100)^(1/2.4)] / log(rhoSDR)}/{log[1 + (RHO-1) * (1/PB_C_H50)^( 1/2.4)] / log(RHO)}; [As before; because of this fixed parallel bypass of the Im_PB_C_1 <> Im_PB_C_2 mechanism in the ETSI method, the glim is the same as that used by the encoder, and these two images It is defined as re-grading from the same PB_C_1, and in this particular SLHDR2PLUS method are Mster_HDR and IDR images respectively] BLO_gr=BLO_IDR/glim[The inverse equation of Equation 17, therefore, this is relatively easy to determine without the need for higher-order equations, and then only a fixed channel adaptation mechanism needs to be applied to obtain the required WLO_REC, which is equal to the WLO_ca used by the encoding , But now it’s the opposite, and addition becomes subtraction] BLO_REC=BLO_ca=BLO_REC*ScaleHor

Subsequently, WLO_REC is calculated by projecting it through para to be inverted later (such as the encoding principle). IF HG_ca=0 WLO_REC=0 ELSE { BLO_co=255*BLO_ca/2040 Xh=(1-HG_REC)/(SG_REC-HG_REC)+WP_REC Xh_REC=HG_REC*Xh+1-HG_REC WW_REC=1-WLO_IDR*255/510 IF WW_REC＞=Xh_REC THEN WCA=1-(1-WW_REC)/HG_REC ELSE Xs=(1-HG_REC)/(SG_REC-HG_REC)-WP_REC Xsca=SG_REC*Xs IF WW_REC＞Xsca { A’’’=-0.5*(SG_REC-HG_REC)/(2*WP_REC) B’’’=(1-HG_REC)/(2*WP_REC)+(SG_REC+HG_REC)/2 C’’’=-[(SG_REC-HG_REC)*(2*WP_REC)-2*(1-HG_REC)]^2 / [8*(SG_REC-HG_REC)*(2*WP_RE)] WCA=(-B’’’+SQRT(B’’’^2-4*A’’’*{C’’’-WW_REC})/(2*A’’’) WCA=min (WCA,1) } ELSE WCA=WW_REC/SG_REC WLO_REC=(1-WCA)*(1-BLO_co)/[(1-WCA*ScaleHor)*(510/255)]

It should be noted that although BLO is actually a pure additive contribution in terms of mapping, WLO is converted to a multiplication of the maximum value proportionally (for example, in Figure 4): Y’HPS= (Y’HP-BLO)/(1-BLO-WLO) [Equation 21]

All this information can generally be filled into a single luminance processing LUT, which relates Y'IP (for example, in the perceptual domain) to Y'HR (or better still, it is the total LUT that defines Lh for each L_in value) . This will reconstruct the REC_M_HDR image.

As mentioned above, it is also useful if the decoder can directly output the display-adapted image, for example, MDR_300.

Follow-up techniques may be used as illustrated in FIG. 16 (wherein the LUT using two parts, the most useful practice-based merely referred to as a LUT P_LUT loaded, because the luminance is calculated based on the rail in the preferred core calculation unit, e.g. , Each pixel color processor of the dedicated decoding IC is generally simply presented as LUT. Y_IDR brightness value is input (for example, generally PQ-based YCbCr encoding), and it is converted into normalized brightness by linearizer 1601 L_in. Perceptualizer 1602 operates as explained above (Equations 1 and 2), and uses the RHO value for IDR peak brightness PB_IDR, for example, 1000 nits. This produces the perceived IDR brightness Y'IP. Brightness The mapping unit 1603 reconstructs the main HDR image as explained above, that is, it obtains all the calculated parameters that define the IDR to MsterHDR reconstruction luminance mapping function F_L_REC, or generally obtains the LUT of the function shape. This generates the reconstructed Mster_HDR brightness Y 'HPR. This image forms a good basis for calculating the low dynamic range/peak brightness PB_C image. Basically, if the correct function is applied, this operates like the ETSI2 mechanism. These functions can be collectively communicated from the F_L_IDR button of the meta data Scale adjustment, or calculated from the reconstructed F_50t1 function, which is defined by the content creator as the best function to calculate the reconstruction of the main SDR image from the Mster_HDR image. Then it can be based on the principle defined in the ETSI2 standard (Readers of this standard, please refer to the standard for this detail) Calculate this F_50t1 function into an appropriate display adaptation function F_L_DA for, for example, 300 nits PB_D. Load this into the HDR to MDR brightness mapper 1604 (assuming one). Practice Above, a single P_LUT will contain all the actions of F_L_REC and subsequent F_L_DA.

Finally, the obtained MDR relative luminance is sent to the first multiplier 454 of FIG. 4 to perform the same processing (also using the correct accompanying F_C[Y]).

Finally, it is necessary to calculate the appropriate C_LUT (F_C[Y] in Figure 4 or Figure 15 respectively), which gives the brightness re-graded output color and other appropriate chromaticity (to have the appearance as close as possible to the Mster_HDR image, that is, The output image pixels and the chromaticity of the Mster_HDR image should be almost the same degree as possible to give different smaller dynamic ranges).

The C_LUT system used for Mster_HDR reconstruction is as follows (other reclassified C-LUT operations follow similar principles, for example, taking ETSI2 teaching into consideration).

First calculate the CP-LUT, which is the trans-form of the P_LUT that is applied in the encoder to map the Mster_HDR image to the IDR image (so in the decoder, this inverse chromaticity correction will be used from the received IDR The image chroma Cb and Cr are inversely converted to Mster_HDR (reconstructed chroma).

Then the C_LUT used for Mster_HDR reconstruction can be calculated as follows: XH=v(PB_M_HDR; 10000) XS=v(PB_SDR=100; 10000) XD=v(PB_D; 10000) XC=v(PB_CH; 10000)

Where v is again the function v(x, RHO) defined in the above equation 1 and equation 2. CfactCH=1-(XC-XS)/(XH-XS) CfactCA=1-(XD-XS)/(XH-XS) C_LUT[Y}=[1+CfactCA*power(CP_LUT[Y] ;2.4)]/[Y*{1+CfactCH* power(CP_LUT[Y] ;2.4)}] [Equation 22]

The display target PB_D can be set to PB_Mster_HDR for reconstruction, in which case only the divider remains as the C_LUT determiner. In actual embodiments, the power of 2.4 can also be included in the LUT as, for example, CPP_LUT = power(CP_LUT[Y]; 2.4), which can save some operations in some embodiments.

As mentioned above, some practical embodiments of the SLHDR2PLUS encoder (used in the current ETSI2 meta data definition compliance) recalculate HG_gr for compliance with the HG_IDR value. This can be done as described below.

For example, the meta data can be reserved for the 8-bit code of the HG of para, that is, in this case, because the IDR image + the meta data should be an ETSI2 compliant signal, the question is whether the required HG_IDR will fit in In the assigned code. The standard generally uses a code allocation function to transform the entity's HG_IDR into a certain HG_COD: HG_COD = F_COD[HG_IDR] in [0,255]. For example, FCOD can be 128*(4*HG_IDR), which means that the maximum value of 255 corresponds to the maximum HG_IDR of 0.5

To ensure that the IDR image is generated so that the HG_IDR fits within the code range, that is, a practical embodiment can achieve this by adjusting the HG_gr of the grader to some extent (making the IDR post-adjustment based on the fixed channel Suppose the data is judged to avoid overflow).

The calculation used for this (optional) embodiment can be, for example: Set HG_IDR=(254*2)/(255*4); Exposure=shadow/4+0.5 [where shadow is the ETSI2 code of shadow gain SG_gr] SG_gr=K*exposure A = SG_gr*(HG_IDR-1)-0.5*(SG_gr-1)*(1-sc*)*(HG_IDR+SG_gr) B=SG_gr-HG_IDR+1+0.5*(SG_gr-1)*(1-sc*)*(HG_IDR+1) C=HG_IDR-1 MxLM=[-B+sqrt(B*B-4*A*C)]/(2*A) IF MxLM = 1 THEN HG_gr_LM = 0 ELSE HG_gr_LM=max[0, (MxLM*SG_gr-1)/(MxLM-1)] Where HG_gr_LM is the adjusted HG_gr value. Then the rest of the algorithm will work as described above, as if the grader had selected the best HG_gr_LM value from the beginning.

This article details a method to deal with the design problems of the new codec of SLHDR2PLUS. Depends on the technical choice made, especially finding the critically important aspect over other aspects that can be relaxed, and there are alternative ways.

The above mathematical definition of a new way to implement an HDR decoder has at least the core calculation method consistent with the ETSI1 and ETSI2 methods: in particular, although the P-LUT and C_LUT functions of different shapes will be calculated as described above (although the figure 4 and Figure 15 detail the technical and physical principles behind how and why our HDR encoding method works. In practice, it is equivalent to the overall brightness processing of the brightness processing in the brightness processing track 401 and 1501 respectively [One-dimensional color In the aspect, the two-system image dependent function transformation through non-linear correlation] is performed by loading the correct shape of the total P_LUT brightness mapping function, and similarly referred to in units 451 and 1551 respectively The C-LUT of F_C[Y]), the computing topology can be reused, and its nature is highly useful to customers (it must be purchased once, for example, the IC in STB, and it can be processed by reprogramming meta-data But maintain the per-pixel color conversion engine and reconfigure into various new coding principles).

It is also possible to design IDR encoding techniques that deeply reuse the same ETSI2 decoding mathematics (ie, partially reclassify the 1503 to 1505 chain), and degrade the received image by only instructing the ETSI2 decoder to properly extrapolate instead of it. For normal operations, the display adapts it to a display with PB_D <PB_IDR. It should be emphasized that this is not a "blind" extrapolation method, which gives "exactly any" higher dynamic range image corresponding to the appearance of the IDR image (that is, especially the statistical distribution of the relative brightness or absolute brightness of the IDR pixel). But in fact, the encoding looks as close as possible to the original Mster_HDR image on the content creator's side (which has not actually been received in this type of embodiment, nor has it received the subsequent data, such as SG_gr) This method of HDR output image is "automatically" generated. This automatic is of course not so simple and involves the correct method on the content encoding side. For the decoder in the embodiment of this principle, the received PB_C_H50 secondary peak brightness is equivalently run in the programming of the core per-pixel decoder as if it is the desired display brightness PB_D (then it is, for example, 5x higher than PB_IDR).

Figure 18 illustrates this method (a block diagram of how encoder math works conceptually). In addition, for the sake of simplicity, it is assumed (although it is not necessary to connect these options as necessary for this example) to choose the freedom of choice of the fixed channel adaptation algorithm to only perform the para transformation that connects Mster_HDR and IDR, and any BLO and WLO ( If it can be applied to the current image or image lens) and the customizable curve is left for the secondary transformation, that is, IDR to SDR reclassification, and the meta data belonging to the ETSI2 compliant IDR signal is transmitted to the receiver (regardless of the traditional ETSI2 receiver or SLHDR2PLUS decoder receiver).

First, some introductory definitions are needed:

The inverse curve of the para curve shown in Figure 10 (that is, the parabola defined by the standardized shape of ETSI as defined in Equation 4 and Equation 5 above and defined by a*x^2+b*x+c Middle part) In this article, for the sake of brevity, it should be called abcara curve. According to ETSI1 section 7 (HDR signal reconstruction), it is defined as: L_out = 1/SG * L_in (if 0＜= L_in ＜= xS) L_out = -b/2a+ sqrt(b^2-4*a*(c -L_in))/2a (if xS＜L_in ＜ xH) L_out = 1/HG *(L_in-1)+1 (if xH＜= L_in) [Equation 23]

Among them, xS and xH are the points where the linear section is changed to the middle section of the parabola, which is consistent with how para is defined for coding (or any other purpose).

The basic principle of what the video encoder embodiment of FIG. 18 is trying to achieve is shown in FIG. 20 (in this example, an example of PB_C IDR of 500 nits has been selected. I don’t want to say that this method is somewhat limited or more limited. Suitable for lower PB_IDR).

If there is a fixed mechanism (in ETSI2 compatible or ETSI2 traditional decoder) to extrapolate from IDR to a higher PB_C than PB_IDR (using this type of PB_C setting, as if it is the peak brightness of the display), it can also be designed to reverse this The processed encoder, that is, by using the inverse function F_ENCINV_H2I of the appropriately adapted extrapolation luminance mapping function F_E_I2S (conformity received from the receiver by the IDR signal (ie, IDR image + post data including the F_I2S function) ETSI2 standard F_I2S function adjustment) to create IDR image, and then add the correct meta data, which will be F_I2S as described, which will be processed by content creators (for example, human graders) or in any intermediate real-time encoding process The total luminance mapping function F_H2S (for example, F_50t1) established by the automation in the derivation.

This relationship can also be formulated from the multiplicative viewpoint: L_SDR=m_F_I2S*m_F_ENCINV_H2I*L_HDR= m_F_I2S* L_IDR L_HDR=m_F_E_I2S*L_IDR

Among them, m_F_I2S or more precisely m_F_I2S(L_HDR) is the corresponding multiplier required to realize the brightness reclassification of any selected L_HDR value, corresponding to the F_I2S brightness mapping function shape, and similarly used for other multipliers.

Therefore, it must be resolved that the trans form of para from HDR to IDR (that is, abcara operating from IDR to HDR) has the same effect as a para extrapolated to PB_HDR (starting on any L_IDR).

For a better understanding, use Figure 21 . From the higher input image PB_C (that is, operating on any normalized input luminance L_in_X that corresponds to the actual luminance through PB_Ch higher than the normalized output image luminance PB_D) to the normal interpolation mode of lower PB_D , The original grader’s para F_H2S (such as received in the meta-data through the standard ESTI2 coded video communication chain) will be adjusted diagonally according to the arrow facing the diagonal [0,0]-[1,1], Generate F_ENCIV_H2I (which now corresponds to PB_IDR/PB_HDR vs. PB_SDR/PB_HDR (that is, for example, v(100/5000)/v(500;5000)=0.54/0.72 [where v(x;y) has the horizontal coordinate x The function of Equation 1, and RHO corresponds to the visual homogenization virtual logarithmic distance ratio of y]) via Equation 2. It is conceivable that through the identity process of mapping PB_HDR to PB_HDR and continuing from any higher to lower PB_D situation, the reclassification behavior will produce a curve that becomes a steep drop. In fact, for the brightness mapping curve of the para type, it will be in mathematics Up becomes abcara. In fact, the required function for extrapolating any received IDR image (based on the initial luminance mapping function F_H2S received in the meta-data, by using ETSI2 Chapter 7.3 Display Adaptation Mechanism) F_E_I2S will be used in F_ENCINV_H2I The mirror image function obtained by mirroring around the diagonal (and vice versa).

Thus, in view of the desired re-use standard ETSI2 SLHDR2PLUS calculation mechanism to implement the functionality, the rest of the system is defined corresponding to the encoder, as illustrated in FIG. 18.

For example, the SG of F_ENCINV_H2I is in the abcara definition 1/SG * L_in_X.

According to SG_COD (that is, the ETSI defined code of the above physical and mathematical shadow gain SG), (SG_COD=SGC*255/2 and the ETSI1 equation C23 exposure=SGC/4 +0.5 and C24 expgain=v(PB_HDR=5000/PB_target =500; PB_target) and equation C27 SG=expgain*exposure): 1/[(SGC/4+0.5)*v(5000/500;500)]=(X/4+0.5)* v(500/5000;500) [Equation 24]

To be solved for the unknown para shadow gain control X (that is, the SG of the X system F_ENCINV_H2I).

That is, the decoder defines how the F_E_I2S shape will look like for any grader's F_H2S selection (using the ETSI2 7.3 algorithm), but it needs to be interpreted as ETSI1 abcara, so that abcara can be related to the corresponding desired antipara F_ENCINV_H2I , In order to finally use the corresponding para in the new SLHDR2PLUS encoder to calculate the IDR image brightness (in the first preferred embodiment of this specific method of the general SLHDR2PLUS method, that is, the second peak brightness is used for the brightness mapping Derivative calculation of the function; white and black offsets in this category (at least in the HDR<>IDR sub-range) will be ignored, because they can be applied to the HDR<>SDR sub-ranges of different PB_C image spectra, As shown in Figure 7).

The encoder now operates in other order in practice (but obeys the same relationship to keep the system ETSI2 compliant). The channel adapter 1801 (from the received F50t1 function shape) calculates the required para to transform the L_HDR brightness into, for example, 500 nits PB_C L_IDR brightness (the channel adaptation mathematics of the previous embodiment described above can be used only by applying para, But then the WLO and BLO adjustments are ignored, that is, para only operates between the two 0 to 1.0 brightness representations without any offset involved). The inverter 1802 uses the inverse equation of Equation 24 to calculate the corresponding abcara (that is, has 1/X calculated on the left in view of the known SGC on the right side of the equation). This is a map that reconstructs the brightness of the L_HDR pixel from the received L_IDR brightness. Assuming that, for example, a constant WP is maintained in the codec definition chain, the inverter 1802 will therefore calculate the shadow gain SG_abc and the high brightness gain HG_abc of abcara. The lower track for meta data management will eventually need to calculate F_L_IDR (=F_I2S), so the adapter 1803 applies the ETSI2 7.3 algorithm in the opposite direction (if part of the brightness reclassification has been completed by using F_ENCINV_H2I to complete the brightness of the IDR image , Realize the remaining transformation F_I2S of the total transformation F_H2S) and determine the required mapping function F_I2S (especially its SG_IDR and HG_IDR).

As mentioned above, in some scenarios, it can happen that the HG_IDR value drops above what can be coded as HG_COD in ETSI2 compliance. What can be done in this scenario is to limit the value of HG_IDR to its maximum value and return to the intended thing through the chain, especially the counterpart of the F_H2S function of different original graders. Then all calculations can be restarted from this situation, and the optional units shown in dashed lines are performed in a continuous processing line.

FIG. 22 explains what the limiter 1804 performs when the brightness mapping curve is reshaped. The dotted line shows the starting F_H2S, and how the fixed channel adaptation algorithm can be used to derive the F_ENCINV_H2I function from this, and how the (original) remaining reclassification function F_I2S_or can be derived (provided there are no additional specific restrictions in the current ETSI2 The original F_IDR call now explains the specific embodiment method described in more detail). In view of this new method of HDR video coding, the HG_IDR_or of this function may not fit the HG_COD definition, that is, it needs to be higher than the 8-bit maximum value of 255 that can be conveyed in the ETSI2 compliant HDR video coding signal. Therefore, HG_IDR_or must be reduced to at most limited still codeable value HG_IDR_LIM (it is 2.0 in the current embodiment of ETSI2, but this is not a basic limitation of the method). This establishes a para with a high-brightness linear segment closer to the upper horizontal margin (L_out_X=1.0), which corresponds to a brighter IDR image, but there is no basic problem (as mentioned above, there are Some relaxation possibilities to design various changes). It will mean that the highest brightness area in the HDR scene image is represented by a smaller contrast IDR (although the original main HDR system can be fully restored, and the SDR appearance and all MDR reclassifications will also look good), but There is no practical problem because it is graded from the higher PB_C HDR main image, and this corresponds to, for example, things in the range of 3000 to 5000 nits, which are generally lamps and the like, which may suffer a little degradation (because of some degradation) The mapping system is always necessary anyway, and is somewhat expected for such ultra-bright areas). Then the second channel adapter 1805 will apply all the above mathematics again, but now it is in the case of finite HG_IDR (so the equivalent F_H2S can be calculated first, as mentioned, in this type of embodiment, it can be obtained by adding the finite F_I2S_LIM outside Insert into the PB_D=PB_Mster_HDR situation and execute, and then apply channel adjustment again).

The resulting F_H2I_LIM (that is, mapping L_HDR brightness to L_IDR brightness) can now be applied by the image pixel brightness mapper 1806 to determine all IDR brightness on a pixel-by-pixel basis (or in fact, also use ETSI2 chroma Processing, that is, the defined C_LUT corresponding to the shape of the F_H2I_LIM brightness mapping function, all IDR YCbCr colors). Finally, the IDR meta data determiner 1807 calculates a complete meta data set for reclassification based on meta data that is compliant with ETSI2 to reduce the PB_C image (for any display PB_D) to be lower than the PB_IDR ( Or by extrapolation higher than PB_IDR). Therefore, SG_IDR, HG_IDR, and WP_IDR are again determined based on any of the possible combinations that form the embodiment explained above. Now also determine BLO_IDR and WLO_IDR (as explained above, the specific brightness on the Mster_HDR brightness axis can be mapped to 1.0 on the SDR brightness axis, and this can be reformulated to an IDR brightness that is appropriately scaled Degree mapping, that is, define WLO_IDR, and similarly apply to BLO_IDR).

Finally, the customizable curve can be optimized for the new IDR post-data situation by the customizable curve optimizer 1808 (if the customizable curve is used, because some sub-market codec technology embodiments Variations (such as real-life broadcasts) may have chosen to never use a customizable curve, and then apply the former para+offset math).

Figure 19 illustrates how the adjustment of the customized curve works. It always contains two conceptual components (whether it is directly applied in a single direction (or reverse)). The first component can be understood by focusing attention on the object: suppose one of the control points of the customizable curve in the multi-linear segment corresponds to a pair of pants (so the specific L_in_S is normalized by the brightness xo1I system, for example , The average brightness of all pants pixels). Use transformation to according to human graders (or automated software), for example, to brighten the pants pixels (the surrounding or one of the particular control points) to output a normalized brightness that is better for the pants. It can also be seen in Figure 4 that in the ETSI method, this occurs as the last (optional) fine hierarchical coding step in the decoder (unit 405), and correspondingly the first step in the decoder. So in fact, this brightness transformation is actually defined in the SDR brightness domain (after the rough HDR to SDR brightness mapping of para+offset (if any)).

Therefore, it can be inferred that any brightness needs to be transformed (for the object!), which can be written multiplicatively as L_out=m(L_in_SDR)*L_in_SDR.

The required multiplicative brightness change (in percentage) can be different on any other image (for example, IDR image), but one thing you should be able to rely on is that the fine-graded correction corresponds to the specific “object” that needs to be regraded (even if the Customizable curve is used for another of its benefits. In addition to the fine grading of specific objects, for example, the improvement of the shape of the rough grading brightness mapping curve can still be physically interpreted as an object-based improvement. By defining a set of virtual objects corresponding to some sub-ranges of brightness). Therefore, if these objects are tracked to another DR brightness range, the normalized abscissa value can be changed, but it is not the core nature of the object (for example, a person on a locomotive has the same value as in SDR (ie, 5/100) Normalized brightness in different HDR (ie, 5/5000)). Therefore, it is necessary to recalculate the function for the new normalized brightness position distribution (this can be done for any amount of the reclassified brightness mapping function for the middle part, even no matter how complicated the various upward and downward partial tracks are, you will want to design HDR video coding embodiment). Therefore, Figure 19a roughly shows this: the original SDR object brightness (for example, the end point of the linear segment of the customizable curve) xo1I is moved to xo1N (this will be applied, for example, in Figure 20 The trans-abcara of F_I2S occurs). The same happens to other points, for example, the pentagonal segment point (generally it can be assumed to have a sufficiently good diffusion segment point (for example, 16), if the grader, for example, apply rough linearity and customize it again The relatively large sub-regions graded to darker brightness can be automatically set by the grading software (for example, 10)). Therefore, to offset all these points, it is now possible to post the original CC_gr curve of the data grader from the main content by applying the original CC_gr offset (that is, L_out_SDR=CC_gr[L_in_S]) (within the SDR brightness range) CC's F_H2S) defines the intermediate curve CC_XRM, where the L_in_S value is the original value xo1I, etc. (but now the L_out value is applied to xo1N and the IDR brightness position is remapped (produces a dashed curve). Of course, this will not be an appropriate HDR to IDR (Or more accurately, IDR to IDR) map the multiplier so that the correction is performed in step 2, as shown in Figure 19b.

As can be seen again in Figure 19b, the multiplicative fine correction can be interpreted as uncorrected (by definition, Mster_HDR pixel brightness) for the most extreme difference in the spectrum of the reclassified image (from Mster_HDR) PB_C image It has been correct, because the image is optimally graded by the content creator from its start) to full calibration, which can be adjusted in proportion, which is generally a 100 nit SDR image in the applicant’s method (where A complete correction system for a specific pixel, for example, mso1, which can be written as an absolute offset, but can also be written as a multiplicative correction yio1=mso1*xso1 (any brightness mapping curve shape yio1=F_L(xso1) can be specified as bright Degree-dependent multiplication value curve).

Because the multiplicative correction viewpoint can be formulated as the offset from the diagonal, where yio1=xso1, the vertical scale factor can be introduced: ScaleVer= max [(1-La)/(1-K); 0] [Equation 25] Use La and K as defined above.

Then find the required adjusted value of the customizable curve as: yiDA=Min[(yio1-xso1)*ScaleVer+xio1;1] ［Equation 26］ And this is to calculate all the values of xso1.

Fig. 27 shows another way to determine the end points of the segments of the customizable fine grading curve in a technically elegant way for the decoder. It has been described how to recalculate the parameters of the rough hierarchical para curve (and if there is a black and/or white offset, the explanation will be simplified by focusing on para). Assume that para is roughly graded from any starting dynamic range to the final dynamic range (for example, LDR dynamic range). The black and white offset can take into account the difference in the normalized range (if any of them is necessary), so the customizable curve only relates to the relative brightness of a specific area repositioned along the normalized axis. Therefore, the curves will start at (0, 0) and end at (1, 1) respectively, and in the example there are some segment connector points between 2 curve shape determination points (for example, (x1, y1)). It is also reasonable that the number of linear segments and points are equal in any representation and its re-grading, because the nature of the area does not change (the darkest area, such as the indoor color in a 200-nit PB_C image, can be compared with 1500 nits). In the special PB_C image, the different (generally perceptually uniform) normalized brightness ends, but the fact that there are two areas (indoor and outdoor) will not change when reclassified.

Therefore, for the re-determination of the shape of the multiple linear reclassification function, only the corresponding endpoints (xnew, ynew) need to be found.

Another property to be met (ideally) can be used, that is, regardless of the use of the total span function FL_50t1 (which in this case will consist of two continuous functions to be applied: total para 2710 and total multilinear function 2711) directly reclassify the main HDR images, or reclassification in two steps, first from the main image of 5000 nits to IDR of 700 nits (again by using two functions: IDR to generate para 2701 and IDR to generate multilinear function 2702), and then from it Downgrade to 100-nit LDR images (using channel para 2703 and channel multi-linear function 2704), the result must be the same: the same LDR image, because it is always the LDR image generated for the main HDR image, that is, the content creator has coded and The one conveyed (with the shape of the degraded luminance mapping function). That is, any one of all possible input HDR normalized brightness x1_MH is selected, and the final LDR output brightness should be the same. Therefore, the input brightness for exactly mapped (via the previous mapping) to the x coordinate of the channel multilinear: x1_CH_L will also be true. This one can be used to recalculate the line segment, because there is equality on the ordinate y, and only need to calculate x_new for the specific section of the corresponding multi-linear customized curve on the other dynamic range.

So on the encoding side, the channel-adapted Y_CHA can be calculated for any x1_MH input by applying a scaled standardized algorithm. This value Y_CHA will form the corresponding input x coordinate of the next block, which enters the channel PB_C and is determined para (the equation given above). The value of yi_CH is already known because it is equal to the value of y1_L used for the total reclassification of 5000 nits to 100 nits, which of course is directly known on the encoding side (relative to the decoding side) (for example, generated by a human grader) . This is done for all points of the multilinear function to obtain all its characteristic parameters in the video signal to be written (as part of F_I2sCI).

On the decoder side, the same principle can be used again to get a somewhat different algorithm, because now some unknown parameters must be calculated. So now it is necessary to calculate the x1_ML value corresponding to the received and therefore known x1_CH_L value, because the first step is to restore the total reclassification function(s). Generally there is the digital accuracy of the function, such as 256 quantized x values (that is, not specific, for example, points between two or three segments, but all points, so the points on the line in between are also the same ), so you can simply construct the LUT table for all points of the customizable curve when it is customized, that is, the y1_L of the known curve corresponds to the required x1_ML of x1_CH_L.

Mapping from LDR to IDR brightness, x1_CH is obtained for any yi_CH, and this value can be demapped via para 2703. If you know para 2701 and multilinear 2702, you can also determine which of all possible x1_MH values map to this Y_CHA value. From the above, we know how to calculate para 2701 from the function meta data received from the decoder side, as explained above. (Still) Don't know Multilinear 2702, but not needed at this time. Because it is known that the customized curve 2702 also follows the vertical scaling equation of the standardized algorithm. Any tested X1_MH can be converted into the corresponding X_CHA, and the Y_CHA value corresponding to it (and required) follows: Y_CHA=(y1_L-x1_ML)*scaleVer + X_CHA, and x1_ML can be calculated from x1_MH by applying a total para 2710 .

Therefore, exactly one corresponding to the x1_MH and x1_ML values will be found, which will restore the total multilinear function 2711. Since the total reclassification and channel part reclassification are known later, the remaining reclassification can also be determined (that is, between the main image of 5000 nits and the IDR of 700 nits), so everything is decoded, that is, the decision function, and all The processing of the IDR image pixel color can begin, as explained using Figure 26.

In order to ensure a better understanding of the readers, Figure 26 again conceptually illustrates in a summary manner. Readers with general knowledge can have found all the content in the above detailed explanation. The upper rail grid is related to the recalculation of the post data, that is, the various steps of determining various brightness mapping functions (the lower unit 2650 is the person who performs the actual pixel color processing). Now, it is good to refer to the two-step calculation, which respectively correspond to the standardized algorithm application in the HDR function unit 901 in the encoder (but from the SLHDR2PLUS video decoder side) and the function determination of the IDR mapping function generator 903. As explained, the decoder obtains the function F_I2sCI, which specifies the brightness reclassification behavior between the selected IDR image and the 100 nits classification with its channel peak luminance PB_CH. However, it is necessary to determine the larger span function that the original function calculator 2601 will perform (between 100 nits and the main HDR peak brightness of, for example, PB_C_H50=6000 nits), that is, the FL_50t1 function (or more precisely, in the coding The inverse shaping function of the one used by the side). But it has not yet arrived. I want to decode the IDR normalized brightness (or more accurately, in our general decoding topology, the perceptually uniform normalized pixel brightness) into the main HDR reconstructed brightness. Therefore, the originally received F_I2sCI and the function FL_50t1 cannot determine the reclassification between the PB_C_H50 nit main image and 100 nits (not the PB_CH nit IDR image and either of the other two), so it is necessary to determine the The received IDR pixel brightness function F_IDRt50 is used to obtain the (perceptually uniform) reconstructed main HDR image pixel brightness YpMstr, which is performed by the reconstruction function determiner 2602. The display adaptation possibility has been shown as a dotted line to show the optimization function calculation unit 2603, because as mentioned, although it will generally be implemented in our full-featured SLHDR2PLUS decoding IC, it is optional for the SLHDR2PLUS decoding system in principle. of. The channel peak brightness PB_CH will be used to convert the normal coded (for example, 10-bit YCbCr) IDR pixel brightness to the perceptually uniform IDR pixel brightness YpIDR, which will generally be performed on it in our better SLHDR2PLUS IC Reconstruct the luminance mapping (although those with ordinary knowledge know how to implement the principles of the present invention in alternative circuits or software that do not apply perceptual homogenization or other methods). The perceptual homogenizer 2650 applies Equation 1 and Equation 2 to it with PB_C_H=PB_CH. The brightness mapper 2651 simply applies the determined function, that is, YpMstr=F_IDRt50(YpIDR), to reconstruct the brightness of the main HDR image. If display adaptation is required to create a 350-nit PB_C image, for example, the display optimizer 2652 only applies the determined display optimization function to it to generate display optimized pixel brightness: Yglim=F_DO(YpMstr). These can be converted into the actual normalized pixel luminance L by the linearizer 2653, which applies the inverse equations 1 and 2, but now uses display-optimized, such as 350 nits PB_C_DO instead of PB_CH. Finally, there is generally an optional further brightness code generator 2654, which applies the perceptual quantizer EOTF of SMPTE 2084 to give the output brightness YPQ in the popular HDR10 format.

Although some embodiments/teachings are proposed to illustrate some aspects that can be changed individually or in combination, it is understood that several further changes can be formed along the same basic principle: conforming to ETSI2 HDR video communication or the like from receiving different intermediates After the dynamic range image is set up, the brightness mapping equation is re-derived to reconstruct the master HDR image that has been optimally graded at the content creation location. The algorithm components disclosed in this article can be implemented in practice (in whole or in part) as hardware (for example, a part of a specific application IC), or as software running on a special digital signal processor or a general-purpose processor.

Those with general knowledge in the technical field should be able to understand from our presentation that these ingredients can be optional improvements and can be combined with other ingredients to achieve, and how the (optional) steps of the method correspond to the various equipment Other components, and vice versa. The word "apparatus" in this application is used in its broadest sense, that is, a group of components that allow the realization of a specific purpose, and can therefore be, for example, IC (small circuit components), or Dedicated appliances (such as appliances with displays), or part of a network system, etc. "Arrangement" is also intended to be used in the broadest sense, so it can include, among other things, a single device, a part of a device, a (partial) collection of collaborative devices, etc.

The computer program product logo should be understood as covering the realization of general-purpose or special-purpose processors, in a series of loading steps (which may include intermediate conversion steps, such as translation to intermediate language, and the final processor language) to input commands to After that, the processor is implemented by any entity that executes any of the characteristic functions of the invention. In particular, computer program products can be implemented as data on a carrier (such as, for example, magnetic disks or tapes), data stored in memory, data travel via a network connection (wired or wireless), or program codes on paper. In addition to the code, the characteristic data required by the program can also be realized as a computer program product.

Some steps required for the operation of this method may already exist in the function of the processor instead of being described in the computer program product, such as data input and output steps.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention. When a person with ordinary knowledge can easily map the presented examples to other areas of the patent application, we do not mention all these options in depth for the sake of brevity. In addition to the combination of the elements of the present invention combined in the scope of the patent application, other combinations of elements are possible. Any combination of elements can be implemented in a single dedicated element.

Any reference signs between parentheses in the scope of patent application are not intended to limit the scope of patent application. The word "comprising" does not exclude the existence of elements or aspects not listed in the scope of the patent application. The word "一 (a/an)" before the element does not exclude the existence of multiple such elements. [Terms and abbreviations used]:

PB_C : The maximum encodable brightness of the image, usually indicated for any situation, C represents encoding (not to be confused with bit depth), for example, HDR images can have PB_C_HDR = 4000 nits (which also defines all the relative brightness below, because L_norm = L/PB_C, where L_norm (normalized brightness) is between 0.0 and 1.0

PB_D : The maximum displayable brightness (also known as peak brightness) of any display. For example, current HDR displays generally have a PB_D of 1000 nits (but down to 600 nits or at most 2000 nits and even 4000 nits It is also currently available for purchase, and may have a higher PB_D in the future).

IDR (Intermediate Dynamic Range): The image originally defined by PB_C1 (for example, 10,000 nits) (that is, the main image) is actually expressed as having PB_C2 <PB_C1 (for example, the general factor is 2 or lower, and PB_C2 is general> = 500 nits) of the secondary HDR image mechanism.

MDR (medium dynamic range; must not be confused with IDR): The image with PB_C_MDR is generally located between PB_C (PB_C_H) of the received HDR image and PB_C_SDR=100 nits (defined by the convention in the video field). The PB_C_MDR value is set equal to the PB_D of any display (in this way, incoming HDR images that have the wrong dynamic range, and therefore more importantly have the wrong relative statistical distribution of normalized brightness relative to each other, can be targeted for lower dynamics The specific available display of range capability is optimally reclassified, ie PB_D <PB_C_H)

Para : A function that is highly useful in programming to map the brightness defined on the first normalized brightness range corresponding to PB_C1 to be normalized by PB_C2, and this function is described by Equation 4 and Equation above 5 and the definition of the parabola in between, or formally defined by ETSI TS 103 433-1 V1.2.1 (2017-08) [in short ETSI1] p. 70 Equation C-20.

Abcara : The inverse function of any para (that is, a parameter that uniquely defines its shape), the opposite shape can also be found intuitively by swapping the axes (but sometimes requires mathematical calculation).

WLO (white level offset): the normalized brightness in the normalized brightness range of the first image (im1) mapped to 1.0 on the second normalized brightness range, Therefore, PB_C_im1>PB_C_im2. In this application, there are several different WLOs defined for various images of different PB_Cs along the encoding process. They are given a postscript, such as WLO_gr, so they can be easily distinguished.

BLO (black level offset): the normalized brightness in the normalized brightness range of the first image mapped to 0.0 on the second normalized brightness range, thereby PB_C_im1 >PB_C_im2. In this application, there are several different BLOs defined for various images of different PB_Cs along the encoding process. They are given a postscript such as BLO_IDR, so they can be easily distinguished.

P_LUT : The total mapping required to convert any possible normalized brightness of the first image into the corresponding normalized brightness of the second image (including part of the reclassification in our codec method, as explained in Figure 4 ), take this PB_C_im1 != PB_C_im2 (generally at least different in the multiplication factor 1.02). Because P_LUT[L] (which is generally image content-dependent and optimized by intelligent image analysis automation or humans, for example) changes the relative distribution of normalized brightness, that is, the histogram, which is a dynamic range transform Key aspects, such as those involved in the definition of IDR images that are key in the principles of this novel HDR codec

C_LUT : Pixel brightness dependent mapping of chromaticity of pixel color, together with P_LUT to complete the color conversion (YCbCr_out=T[Y_cbCr_in])

201: Camera 202: OOTF mapping 203: Grading Unit 204: Video communication connection 205: display 206: User Interface Control Unit 301: Image source 302: color changer 303: Video Compressor 304: Formatter 305: Communication Media 306: Unformatted 307: Video Decompressor 308: color changer 309: display adjustment unit 310: display 320: Video receiver 321: Broadcast station side encoder 401: EOTF applying unit/brightness processor/brightness processing track/brightness processing track 402: unit/luminance mapping 403: Black-white level shifter/unit/luminance mapping 404: rough dynamic range converter/unit 405: Curve Applicator/Unit 406: linearizer/unit 450: chroma processing track 451: Saturation processing determiner/unit 452: Multiplier 453: Matrix application unit 454: first multiplier 455: Multiplier 601: Shadow Man 602: Robot 900: encoder 901: HDR function generation unit/unit 902: IDR image calculation unit 903: IDR mapping function generator 920: video input 921: Second Metadata Input 922: First Metadata Input 923: Third Metadata Input 930: Image output 931: second meta data output 932: First meta data output 933: Third Metadata Output 1100: High dynamic range video decoder/decoder 1102: color changer 1104: Luminance function determination unit 1110: video input 1111: First meta data input 1112: Second meta data input 1113: Third Metadata Input 1120: Video output 1121: Metadata output 1122: Video output 1501: Luminance Mapper/Luminance Processing Track 1502: Perceptualizer/decoder/unit 1503: Fine grading unit/unit 1504: rough brightness mapping unit/unit 1505: Anti-black and white offset device/unit 1506: Linearizer/unit 1550: chroma processor 1551: Chroma processing judging unit/unit 1601: Linearizer 1602: Perceptualizer 1603: Luminance Mapping Unit 1604: HDR to MDR brightness mapper 1801: channel adapter 1802: inverter 1803: adapter 1804: limiter 1805: second channel adapter 1806: Image pixel brightness mapper 1807: IDR post data determiner 1808: Customizable curve optimizer 2301: content creation side 2302: Camera 2303: Grading equipment 2304: Grader 2305: Video Encoder 2306: Satellite antenna 2340: Communication Satellite 2351: Local satellite receiving dish 2352: final processing equipment 2353: HDR display 2370: Video Encoder 2381: decoder 2382: display optimizer 2501: HDR image signal encoded by SLHDR2PLUS 2502: pixel color matrix 2510: SLHDR2 decoder 2520: SLHDR2PLUS decoder 2601: Original function calculator 2602: Reconstruction function determiner 2603: Display optimization function calculation unit 2650: Perceptual homogenizer/unit 2651: Luminance Upper Mapper 2652: Display Optimizer 2653: Linearizer 2654: Luminance code generator 2701: para 2702: IDR generates multiple linear functions/multiple linearities/curves 2703: para 2704: Channel multiple linear function 2710: total para 2711: Total multilinear function BE: Main HDR image brightness range Be2: IDR brightness range BLO_gr: black level offset BLO_IDR: parameter BLO_REC: offset BN: Normalized brightness Bs: color component Cb: chroma Cb*: output color CC_XRM: Middle curve ColHDR: Brightness ColSDR: Brightness Cr: chroma Cr*: output chromaticity DR_1: LDR luminance dynamic range/first dynamic range DR_2: HDR brightness range / second dynamic range DRKSPST: Dark Space Station dX: abscissa EXP: Exposure F_??: function F_C: rough brightness extrusion function/rough mapping function F_ct: color transformation function/color mapping function F_E_I2S: Luminance mapping function F_ENCINV_H2I: decode brightness mapping function F_H2h: Brightness mapping function/upgrading function F_H2hCI: Adjusted brightness mapping function/luminance mapping function F_H2hCI_1: output F_H2hCI_2: output F_H2hCI_3: output F_H2I_LIM: Luminance mapping function shape F_H2S: Total luminance mapping function/initial luminance mapping function F_I2s: function F_I2S: mapping function F_I2S_or: remaining part reclassification function F_I2sCI: Channel brightness mapping function/luminance mapping function/luminance mapping function F_IDRt50: function F_L_CU: Fine graded luminance mapping function/function/curve F_L_DA: display adaptation function/luminance mapping function F_L_REC: MsterHDR reconstruction brightness mapping function F_L_RHDR: reconstruct the shape of the para luminance mapping function F_L_subsq: Brightness mapping function F_Mt1: function/total luminance remapping function F_Mt1_ca: Intermediate function F50t1: Function shape Fd: distance FL_50t1: Main brightness mapping function/total span function/luminance degradation function FL_ca: Curve FL_gr: Curve FL_IDR: Curve Fx: function GN: Normalized brightness Gs: color component HG: slope HG_abc: High brightness gain HG_ca: High brightness gain HG_gr: second slope/original high brightness gain HG_IDR: parameter HG_IDR_LIM: Codable value HG_REC: Rebuild high brightness gain IDR: Intermediate dynamic range Im1250: Display adjusted image Im5000: output image Im_COD: Compressed SDR image Im_LDR: output image/low dynamic range image/standard state range image Im_RHDR: Reconstructed HDR image/original main image Im_RLDR: SDR image ImDA:PB_C image ImSCN1: Sunny outdoor image ImSCN2: night scene ImSCN3: Scene image K: The complete action of the function L_HDR: Brightness L_IDR: Brightness L_in: input/parameter/normalized brightness L_in_X: Normalized input brightness L_Mster_HDR: Brightness L_REC_M_HDR: Brightness L_RHDR: normalized and reconstructed brightness L_SDR: Brightness Lh: Output brightness LIN_HDR: Linear luminance image Ln_Mster_HDR: Input brightness Ln_XDR: Brightness Ls: SDR brightness/luminance MAST_HDR: Main HDR image MB: minimum black level/lower endpoint MDR: Medium dynamic range MDR_300: Medium dynamic range image mip: midpoint mso1: absolute offset Mster_HDR: image Mster HDR: video MsterHDR: Main HDR video/input high dynamic range video/main high dynamic range video mx: abscissa n: distance OOTF: Optical-Optical Method P_IoH: Brightness P_SoH: Brightness PB_C: Image encoding peak brightness/maximum encoding brightness PB_C_H: Peak brightness PB_C_H50: The first maximum pixel brightness / second expected peak brightness / content peak brightness PB_CH: channel peak brightness / second largest pixel brightness PB_MDR: Maximum pixel brightness PQ: Perceptual quantizer REC_M_HDR: image/reconstructed main HDR image RecHDR: HDR image RN: normalized brightness Rs: color component RS: SDR luminance sub-range sc*: standardized scale factor SDR: Standard dynamic range SG: Shadow gain SG_abc: shadow gain SG_ca: shadow gain SG_gr: first slope SG_IDR: IDR image/parameter SG_REC: Shadow gain SG_RM: curve/total shadow gain WLO_ca: parameter WLO_gr: White level offset WLO_IDR: parameter WLO_REC: offset WP_IDR: parameter xo1I: Brightness Y: Brightness Y’CL: Output brightness Y’GL: Graded LDR brightness Y’HC: Initial HDR brightness Y’HP: Gray brightness Y’HPR: Reconstructed Mster_HDR brightness Y’HPS: Proportionally adjusted HDR brightness/normalized HDR input brightness Y’IP: IDR brightness Y’IPG: Reclassify IDR brightness Y_IDR: Brightness value Yglim: Optimized pixel brightness after display yio1: Brightness mapping curve shape YpIDR: Perceive uniform IDR pixel brightness YpMstr: Pixel brightness of the reconstructed main HDR image YPQ: output brightness

These and other aspects of the method and equipment according to the present invention will become apparent with reference to the embodiments and examples described below and with reference to the accompanying drawings, which are only non-limiting to illustrate more general concepts It is shown in detail, and the dotted line is used to indicate that the component is optional, and the non-dotted component is not necessarily necessary. Dotted lines can also be used to indicate elements that are interpreted as necessary, but are hidden inside the object, or used for intangible things such as, for example, the selection of objects/regions (and how they can be displayed on the display).

In the schema: [Figure 1] It schematically shows how to map the high dynamic range image to the corresponding optimal color grading and similar appearance (in view of the difference in the first dynamic range DR_1 and the second dynamic range DR_2, such as Many general color transformations that occur in lower dynamic range images (for example, standard dynamic range images with 100 nits peak brightness) that are similar to those expected and feasible) will also correspond in the case of reversibility (mode 2) In the mapping of the SDR image received by the receiver (decoder), the SDR image actually encodes the HDR scene; [Figure 2] Shows how the capture system for HDR images looks like; [Figure 3] Explains the possible ways to convey HDR images as some images with specific (different) peak brightness and the general definition of the common transmission in the meta-data is the brightness mapping function (for example, it is defined as the SDR image Im_LDR that can be used traditionally ) Brightness mapping function, for example, according to the applicant’s better method, start the description for new readers of the technology with a high conceptual level; [Figure 4] Shows various further details of encoding HDR images according to the applicant's specific method standardized in ETSI2, which is necessary to understand the various details of the various teachings of the novel SLHDR2PLUS codec method written below; [Figure 5] Explain how the color changes and especially re-classify the brightness conversion in the YCbCr color gamut; [Figure 6] Explain the concept of our customizable curve in some more details by explaining useful applications; [Figure 7] Explains the basic idea of the intermediate dynamic range (IDR) encoding and communication of the original master HDR image, as well as the not to be confused concept of the intermediate dynamic range image (MDR), which is generally calculated from any received image to target the current The display on any specific display with the peak brightness PB_D can be used to optimize it, such as a specific HDR tv purchased by any end consumer who attempts to watch the received HDR video; [Figure 8] It further explains how to start to deal with the IDR problem, especially by the decoder to solve it in a specific automatic calculation method, and especially if at least some of the decoders that can receive the content are ETSI2 compliant decoders on the market , And may not be easily upgraded with the new SLHDR2PLUS technology (for example, because the owner of tv or STB does not upgrade it); [Figure 9] Shows the general structure of the technical components generally required for the novel SLHDR2PLUS encoder of this application; [Figure 10] Explain some basic technical aspects involved in the continuous derivation of various corresponding brightness mapping functions by the encoder, especially the example of using the para brightness mapping function; [Figure 11] shows the possible typical high-level structure of the novel SLHDR2PLUS decoder, which follows the teachings of some embodiments of various possibilities described below to realize SLHDR2PLUS video communication; [Figure 12] Further explain some aspects of black and white shifts when he chooses from the main brightness mapping function that defines the reclassification requirements of his video content according to the content creator’s point of view; [Figure 13] Describes the technical principle of a better method for deriving the fixed algorithm of the channel-adapted version of para according to the principle of diagonal scaling; [Figure 14] Explain some aspects of the inverse curve of para (the so-called abcara); [Figure 15] Detailed description of some aspects of SLHDR2PLUS decoder; [Figure 16] Illustrates a useful way of implementing display adaptation to calculate the MDR image of any specific PB_D display integrated with the SLHDR2PLUS encoding principle of the new technology of this application; [Figure 17] Explain the channel adjustment of black and white offset (BLO & WLO) to accompany and simplify the following mathematics, and give some further aspects of the basic principles of general physical technology; [Figure 18] shows another embodiment of SLHDR2PLUS encoding (or a combination of several teachings of various embodiments illustrated in a diagram), which is particularly useful because the encoded image can be used by a standard deployed ETSI2 decoder Direct decoding [Figure 19] Explains how to determine the corresponding version of the original master customized curve for various dependent peak brightness image representations (whether as input or output images), such as IDR images with a common specification of customized curves, for example, movie creators The grader needs to fine-tune the rough-mapped IDR brightness to the precise final SDR brightness; [Figure 20] The basic technical principle of the method in Figure 18 is displayed on the frequency spectrum of the re-gradable image; [Figure 21] Explain the adjustment of the extrapolation of the brightness mapping function (for example, para) beyond the highest value of the initial image (corresponding to the consistent transformation or the diagonal line in the graph), and also explain it The relationship between the shape of a particular selected para function and the corresponding abcara; [Figure 22] A schematic illustration of how the specific embodiment of para’s high-brightness gain limitation works on the brightness reclassification between the input normalized brightness and the output normalized brightness; [Figure 23] Schematic diagram of the overall HDR communication and processing chain to clearly point out the difference between video encoding and display optimization (the former is the true definition of the image itself, while the latter is only about the ability to target in a specific dynamic range The optimization on any particular display of, loosely speaking, it can be customized, and it is better to take into account the reclassification requirements of various HDR scene images that may be quite different and quite challenging); [Figure 24] It further explains how the normalized function changes the algorithm will work. It will use any input function shape generated by, for example, the content creator’s grader to specify the normalized luminance response of the first peak luminance dynamic range. How to map to the normalized brightness of the second peak brightness image representation, and how can this input function be uniquely calculated for any given input function to map between a different particularly low third peak brightness representation and the second one function; [Figure 25] It schematically shows how the SLHDR2PLUS video encoding mechanism and signal that have been taught so far are well compatible with the traditional SLHDR2 decoder, but still provide additional possibilities for the deployed SLHDR2PLUS decoder; [Figure 26] Schematically summarize all the following decoding mathematics in a block to recover all the required luminance mapping functions for performing all required decoding luminance processing to obtain the required image from the received IDR image; and [Figure 27] A schematic illustration of a good practical method to obtain a customized version of the customizable curve (if it exists) numerically.

900: encoder

901: HDR function generation unit/unit

902: IDR image calculation unit

903: IDR mapping function generator

920: video input

921: Second Metadata Input

922: First Metadata Input

923: Third Metadata Input

930: Image output

931: second meta data output

932: First meta data output

933: Third Metadata Output

F_H2hCI: adapted brightness mapping function

F_I2sCI: Channel brightness mapping function

FL_50t1: main brightness mapping function

IDR: Intermediate dynamic range

MsterHDR: Main HDR video/input high dynamic range image

PB_C_H50: The first maximum pixel brightness

PB_CH: channel peak brightness / second largest pixel brightness

Claims (10)

A high dynamic range video encoder (900) configured to receive an input high dynamic range image (MsterHDR) via an image input (920), the input high dynamic range image having a first maximum pixel brightness (PB_C_H50), The encoder has a first meta data input (922) for the first maximum pixel brightness, and the high dynamic range video encoder is configured to receive a master brightness via a second meta data input (921) A mapping function (FL_50t1), which defines the relationship between the normalized brightness of the input high dynamic range image and the normalized brightness of a corresponding low dynamic range image (Im_LDR). The low dynamic range The image preferably has an LDR maximum pixel brightness equal to 100 nits, which is characterized in that the encoder further includes a third post data input (923) to receive a first maximum pixel brightness (PB_C_H50) lower than the first maximum pixel brightness (PB_C_H50). Two maximum pixel brightness (PB_CH), and the encoder is further characterized in that it includes: -An HDR function generating unit (901), which is configured to apply a standardized algorithm to transform the main brightness mapping function (FL_50t1) into an adjusted brightness mapping function (F_H2hCI), and the adjusted brightness mapping function makes The normalized brightness of the input high dynamic range image is related to the normalized brightness of an intermediate dynamic range image (IDR). The intermediate dynamic range image is characterized by having a maximum value equal to the second maximum pixel brightness (PB_CH). Possible brightness -An IDR image calculation unit (902), which is configured to apply the adjusted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the output of this unit The brightness of the pixels of the intermediate dynamic range image (IDR); and -An IDR mapping function generator (903), which is configured to derive a channel brightness mapping function (F_I2sCI) based on the main brightness mapping function (FL_50t1) and the adjusted brightness mapping function (F_H2hCI), The channel brightness mapping function, when the respective normalized brightness of the intermediate dynamic range image (IDR) is given as input, the respective normalized brightness of the low dynamic range image (Im_LDR) Defined as output, the brightness corresponds to the respective brightness of the input high dynamic range image (MsterHDR); further features of the encoder are: -An image output (930) to output the intermediate dynamic range image (IDR); -A first meta data output (932) to output the second maximum pixel brightness (PB_CH); -A second meta data output (931) to output the channel brightness mapping function (F_I2sCI); and -A third meta data output (933) to output the first maximum pixel brightness (PB_C_H50). For example, the high dynamic range video encoder (900) of claim 1, wherein the standardized algorithm of the HDR function generating unit (901) applies a compression toward the diagonal of the main brightness mapping function (FL_50t1) to obtain The adjusted brightness mapping function (F_H2hCI), the compression involves scaling all the output brightness values of the function by a scale factor that depends on the first maximum pixel brightness (PB_C_H50) and the second maximum pixel Brightness (PB_CH). For example, the high dynamic range video encoder (900) of claim 1 or 2, which includes: a limiter (1804), which is configured to target the normalized brightness that includes the brightest normalized brightness equal to 1.0 Change a sub-range of brightness to re-determine a slope of the channel brightness mapping function (F_I2sCI). A high dynamic range video decoder (1100), which has an image input (1110) to receive an intermediate dynamic range image (IDR), the intermediate dynamic range image has a multiplication factor preferably 0.8 or less A second maximum pixel brightness (PB_CH) that is smaller than a first maximum pixel brightness (PB_C_H50) of a main high dynamic range image (MsterHDR), and the second maximum pixel brightness (PB_CH) is input via a second meta data (1112 ) Receiving, the decoder has a first meta data input (1111) to receive a brightness mapping function (F_I2sCI), the brightness mapping function defines all possible normalized brightness of the intermediate dynamic range image (IDR) The transformation of the corresponding normalized brightness to an LDR maximum pixel brightness low dynamic range image (Im_LDR), the decoder is characterized in that it has a third post data input (1113) to receive the first maximum pixel brightness (PB_C_H50), and the decoder includes: -A luminance function determination unit (1104) configured to apply a standardized algorithm to transform the luminance mapping function (F_I2sCI) into a decoded luminance mapping function (F_ENCINV_H2I), the decoded luminance mapping function for the intermediate dynamics The normalized brightness of any possible input of a pixel of the range image (IDR) designates a corresponding normalized HDR brightness of the main high dynamic range image (MsterHDR) as the output, and the normalized algorithm uses the first The equivalent values of the maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH); and -A color converter (1102) configured to continuously apply the decoded brightness mapping function (F_ENCINV_H2I) to the input normalized brightness of the intermediate dynamic range image (IDR) to obtain a reconstructed master The normalized reconstructed brightness (L_RHDR) of the pixels of the HDR image (REC_M_HDR); the decoder further has an image output (1120) to output the reconstructed main HDR image (REC_M_HDR). For example, the high dynamic range video decoder (1100) of claim 4, wherein the standardized algorithm of the luminance function determination unit (1104) calculates that it depends on the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH) a scale factor. For example, the high dynamic range video decoder (1100) of claim 4 or 5, wherein the brightness mapping function (F_I2sCI) is defined by a brightness map, and the brightness map is defined by having a normalized brightness range for darkness A first linear section with a first slope (SG_gr) for a bright normalized brightness range, a second linear section with a second slope (HG_gr) for a bright normalized brightness range, and a second linear section for Consists of a parabolic segment of brightness between two ranges. Such as the high dynamic range video decoder (1100) of claim 4 or 5, wherein the color converter (1102) is configured to calculate the pixel brightness of a medium dynamic range image (MDR_300) with a maximum pixel brightness (PB_MDR) , The maximum pixel brightness is not equal to the values of the LDR maximum brightness, the first maximum pixel brightness (PB_C_H50), and the second maximum pixel brightness (PB_CH), and the decoder is capable of outputting the medium dynamic range image ( MDR_300) an image output (1122). For example, the high dynamic range video decoder (1100) of claim 4 or 5 has a post-data output (1121), the post-data output is used to output a brightness mapping function (F_L_subsq), the brightness mapping function For the reconstructed main HDR image (REC_M_HDR) or alternatively the normalized brightness of the medium dynamic range image (MDR_300), define the corresponding brightness of an image with another maximum pixel brightness, and the other maximum pixel brightness is lower than Preferably, it is 100 nits, or a value higher or lower than the maximum luminance value of the reconstructed main HDR image (REC_M_HDR) or alternatively the medium dynamic range image (MDR_300). A method for high dynamic range video encoding of a received input high dynamic range image (MsterHDR) with a first maximum pixel brightness (PB_C_H50), which includes receiving a master brightness mapping function (FL_50t1), the brightness mapping function definition A relationship between the normalized brightness of the input high dynamic range image and the normalized brightness of a corresponding low dynamic range image (Im_LDR), the low dynamic range image preferably has a relationship equal to 100 nits The method is characterized in that the encoding further includes receiving a second maximum pixel luminance (PB_CH), and the encoding includes: -Apply a standardized algorithm to transform the main brightness mapping function (FL_50t1) into an adjusted brightness mapping function (F_H2hCI), the adjusted brightness mapping function correlates the normalized brightness of the input high dynamic range image Normalized luminance of an intermediate dynamic range image (IDR), the intermediate dynamic range image is characterized by having a maximum possible luminance equal to the second maximum pixel luminance (PB_CH); -Apply the adjusted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the brightness of the pixels of the intermediate dynamic range image (IDR); -Based on the main brightness mapping function (FL_50t1) and the adjusted brightness mapping function (F_H2hCI), a channel brightness mapping function (F_I2sCI) is derived, and the channel brightness mapping function is in the intermediate dynamic range image (IDR ) When the respective normalized brightness of the low dynamic range image (Im_LDR) is given as the input, the respective normalized brightness of the low dynamic range image (Im_LDR) is defined as the output, and the brightness corresponds to the input high The individual brightness of dynamic range images (MsterHDR); -Output the intermediate dynamic range image (IDR); and -Output the second maximum pixel brightness (PB_CH), the channel brightness mapping function (F_I2sCI), and the first maximum pixel brightness (PB_C_H50). A method for decoding a high dynamic range video after receiving an intermediate dynamic range image (IDR), the image having a first high dynamic range image (MsterHDR) lower than a main high dynamic range image (MsterHDR) by a multiplication factor of preferably 0.8 or less A second maximum pixel brightness (PB_CH) of a maximum pixel brightness (PB_C_H50), the second maximum pixel brightness (PB_CH) is received as the meta data of the intermediate dynamic range image, and the decoding method also receives the meta data in the meta data A brightness mapping function (F_I2sCI), the brightness mapping function defines all possible normalized brightness of the intermediate dynamic range image (IDR) to an LDR maximum pixel brightness and the corresponding normalized low dynamic range image (Im_LDR) The conversion of brightness, and the decoding method is characterized in that it receives the first maximum pixel brightness (PB_C_H50), and the decoding method is characterized in that it includes: -Apply a standardized algorithm to transform the brightness mapping function (F_I2sCI) into a decoded brightness mapping function (F_ENCINV_H2I) for any possible input of a pixel of the intermediate dynamic range image (IDR) The normalized brightness designates a corresponding normalized HDR brightness of the main high dynamic range image (MsterHDR) as the output, and the normalized algorithm uses the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH) equivalent value; -Apply the decoded brightness mapping function (F_ENCINV_H2I) to the normalized brightness of the intermediate dynamic range image (IDR) to obtain the normalized reconstructed brightness (L_RHDR) of the pixels of the reconstructed main HDR image (REC_M_HDR) );and -Output the reconstructed main HDR image (REC_M_HDR).

Images