WO2023070582A1

WO2023070582A1 - A device and method for noise-adaptive high dynamic range image processing

Info

Publication number: WO2023070582A1
Application number: PCT/CN2021/127673
Authority: WO
Inventors: Hu Chen; Quanhe YU; Weiwei Xu
Original assignee: Huawei Technologies Co.,Ltd.
Priority date: 2021-10-29
Filing date: 2021-10-29
Publication date: 2023-05-04
Also published as: CN118103864A

Abstract

A device (200) and method (600) for noise-adaptive high dynamic range image processing are able to determine a plurality of image blocks (202) of the HDR image (201) (601), wherein each image block (202) comprises a plurality of pixels. Further, the device (200) and method (600) are able to determine a local contrast value (203) of the image block (202) of the HDR image (201) (602), and to add random noise (204) depending on the local contrast value (203) of the image block (202) (603). The device (200) and method (600) are thus able to adaptively add random noise (204) block-by-block to the image blocks (202) of the HDR image (201) according to their local contrast values (203).

Description

A DEVICE AND METHOD FOR NOISE-ADAPTIVE HIGH DYNAMIC RANGE IMAGE PROCESSING

TECHNICAL FIELD

The present disclosure relates toprocessing a high dynamic range (HDR) image, for example, converting the HDR image into a standard dynamic range (SDR) image. As part of the processing, random noise is added to the HDR image to counter banding artefacts. The disclosure provides a device, a method, and a computer program, for implementing the processing of the HDR image, wherein local contrast values of the HDR image are taken into account. The device, method, and computer program may be used for computer vision applications.

BACKGROUND

HDR images are typically transformed from a linear domain to a nonlinear domain, using a perceptual quantizer (PQ) curve or a hybrid log-gamma (HLG) curve, before being coded to 10-bit or 12-bit. Both the PQ curve and the HLG curve are standardized in R-REC-BT. 2100. For most natural images, coding to 10-bit is enough to ensure a high image quality, for example, without banding artefacts.

However, in many cases a HDR image has to be converted to an SDR image, in order to be shown on a SDR display. A conventional SDR display typically requiresa gamma curve for the nonlinear transformation, and supports image coding only to 8-bit. There are multiple variants of the gamma curve, and the most typical one is standardized in ITU-R Recommendation BT. 1886. The loss of precision, when coding the HDR image to an 8-bit SDR image, often causes banding artefacts. An example of banding artifacts 100 is illustrated in FIG. 1. Here in this example, the original HDR image is in high precision, more specifically in floating point precision, and the output SDR image ends up having visible banding artefacts after having been coded to 8-bit using a gamma curve.

Unfortunately, most middle-and low-end smartphones, and AR/VR headsets, i.e., head-mounted displays for augmented reality (AR) and virtual reality (VR) , include such a conventional SDR display, which means that the necessity to convert the HDR image into the 8-bit SDR image is rather common.

Recently, some high-end smartphones and televisions (TVs) have been able to support a 10-bit SDR image. The nonlinear curve, however, is still a gamma curve. In the conversion from PQ or HLG to gamma, errors are still introduced, which may again lead to banding artefacts.

One of the most typical methods for addressing banding artefacts is adding random noise to the HDR image during its processing and conversion into the SDR image. Popular image editors often provide such a feature. In parts of the HDR image that will have banding artefacts once converted to SDR, random noise is added. The strength of the random noise is usually adjusted by a user or colorists. Notably, the strength (level) of the random noise is important. If the level of the added random noise is strong enough, the banding artefacts are no more visible. If the level of the added random noise istoo low, then the banding artifacts are still visible. On the other hand, if the level of the added random noise is too strong, the random noise will be visible in the SDR image, which also degrades the image quality. Therefore, it is important to find a good trade-off.

In practice, due to the huge amount of images and videos, it is unrealistic that humans process all of the images or video frames manually. Therefore, automatic algorithmsfor adding the random noise have been proposed. The most classical approach uses pseudo-random noise for picture coding, which is simple and effective.

A desire to improve this approach, however, lies in the fact that the level of the added noise is fixed. For some images, the noise may thus be too weak to remove banding artefacts, while for some other images the noise may be too strong and thus disturbing. The random noise is conventionally added to the complete image, and as a result the noise is added also to regions without significant banding artefacts. The noise may thus become visible in the output SDR image, and may reduce its image quality, especially in these regions.

SUMMARY

In view of the above, an objective of this disclosure is to improve the removal of banding artefacts during the processing of a HDR image, for example, during its conversion into a SDR image. A goal is to remove the banding artefacts with just the right amount of random noise added to the HDR image. This should be true for any HDR image, and therefore the intention is to take into account the content of the HDR image. The present disclosure also intends to avoidadding random noise to regions of the HDR image, which would not show banding artifacts in the first place, when the HDR image is processed into the SDR image.

These and other objectives are achieved by the subject matter of the enclosed independent claims. Advantageous implementations are further defined in the dependent claims.

A first aspect of this disclosure provides a device for processing aHDR image, the device being configured to: determine a plurality of image blocks of the HDR image, each image block comprising a plurality of pixels; andfor each image block of the plurality of image blocks: determine a local contrast value of the image block; andadd random noise to the image block, wherein a level of the random noise added to the image block depends on the local contrast value of the image block.

The device of the first aspect is configured to add random noise to each image block separately. Thus, different regions of the HDR image are treated differently. For example, it may thereby be avoided that random noise is added to regions of the HDR image, which would not show banding artefacts in the first place. Further, the device of the first aspect takes into account the respective local contrast values of the image blocks. That means, that each image block may get a different level of random noise, specifically selected for that image block. The local contrast value reflects a content of the respective image block. Thus, the device of the first aspect is able to take into account the content of the HDR image as a whole, when adding the random noise. Overall, the removal of banding artefacts during the processing of the HDR image, for instance, conversion into a SDR image, is improved.

In an implementation form of the first aspect, the device is configured to adaptively add random noise block-by-block to all image blocks of the HDR image according to the local contrast values of the image blocks.

In the end, the random noise is added to the HDR image to avoid banding artefacts, wherein the right amount of random noise is added to each image block of the HDR image.

In an implementation form of the first aspect, the device is configured to, for each image block of the plurality of image blocks, determine a coding error value of the image block; wherein the level of the random noise added to the image block depends further on the coding error value of the image block.

This further improves the reduction of the banding artefacts with the right amount of random noise for each image block of the HDR image.

In an implementation form of the first aspect, the device is configured to, for each image block of the plurality of image blocks, calculate a ratio between the coding error value and the local contrast value of the image block; wherein the level of the random noise added to the image block depends on the ratio calculated for the image block.

In this way, the two influencing factors –coding error value and local contrast value –are taking into account in combination and in the most effective manner.

In an implementation form of the first aspect, the device is configured to, for each image block of the plurality of image blocks, determine whether the ratio calculated for the image block is below or above a first threshold value and a second threshold value, respectively; wherein the level of the random noise added to the image block is a minimum level or zero, if the ratio calculated for the image block is smaller than the first threshold value; wherein the level of the random noise added to the image block is a maximum level, if the ratio calculated for the image block is larger than the second threshold value; andwherein the level of the random noise added to the image block is between the minimum noise level and the maximum noise and depends on the ratio, if the ratio calculated for the image block is between the first threshold value and the second threshold value.

This avoids further that, on the one hand, not too much random noise is added to any image block, and, on the other hand, that no random noise or only little random noise is added where it is not absolutely necessary.

In an implementation form of the first aspect, if the ratio calculated for the image block is between the first threshold value and the second threshold value, the level of the random noise added to the image block is larger, if the local contrast value of the image block is smaller; and/orthe level of the random noise added to the image block is larger if the coding error value of the image block is larger.

In this way, the banding artefacts can be countered most effectively.

In an implementation form of the first aspect, the device is configured to, for each image block of the plurality of image blocks, determine the maximum noise level based on an average pixel value of the pixels of the image block.

Thus, the maximum noise level is determined based on the content or characteristics of the HDR image.

In an implementation form of the first aspect, the device is configured to convert the HDR image, which comprises the random noise added to each image block, to a SDR image.

In an implementation form of the first aspect, the device is configured to convert the HDR image from a PQ domain to a linear domain or to a gamma domain; andadd the random noise to each image block of the plurality of image blocks in the linear domain or in the gamma domain.

In an implementation form of the first aspect, the device is configured toconvert the HDR image from the linear domain to the gamma domain, after adding the random noise to each image block of the HDR image in the linear domain, and then generate the SDR image based on the HDR image converted to the gamma domain using quantization; orgenerate the SDR image based on the HDR image in the gamma domain using quantization, after adding the random noise to each image block of the HDR image in the gamma domain.

In an implementation form of the first aspect, the device is configured to, for each image block of the plurality of image blocks, determine the coding error of the image block in the gamma domain or in the linear domain.

In an implementation form of the first aspect, the device is configured to determine the coding error of the image block by: quantizing original pixel values of the pixels of the image block in the gamma domain to obtain quantized pixel values; converting the quantized pixel values from the gamma domain to the linear domain to obtain reconstructed pixel values; and calculating the coding error of the image block based on a mean square error between the original pixel values and the reconstructed pixel values.

In an implementation form of the first aspect, the device is configured to, for each image block of the plurality of image blocks, determine the local contrast value of the image block based on a pixel value of each pixel of the image block and an average pixel value of all the pixels in the image block.

Therefore, the solutions of the present disclosure can be implemented in either in the linear domain or in the gamma domain, thus enabling high flexibility and adaptability for different applications.

A second aspect of this disclosure provides a method for processing aHDR image, the method comprising: determining a plurality of image blocks of the HDR image, each image block comprising a plurality of pixels; andfor each image block of the plurality of image blocks: determining a local contrast value of the image block; andadding random noise to the image block, wherein a level of the random noise added to the image block depends on the local contrast value of the image block.

In an implementation form of the second aspect, the method comprises adaptively adding random noise block-by-block to all image blocks of the HDR image according to the local contrast values of the image blocks.

In an implementation form of the second aspect, the method comprises, for each image block of the plurality of image blocks, determining a coding error value of the image block; wherein the level of the random noise added to the image block depends further on the coding error value of the image block.

In an implementation form of the second aspect, the method comprises, for each image block of the plurality of image blocks, calculating a ratio between the coding error value and the local contrast value of the image block; wherein the level of the random noise added to the image block depends on the ratio calculated for the image block.

In an implementation form of the second aspect, the method comprises, for each image block of the plurality of image blocks, determining whether the ratio calculated for the image block is below or above a first threshold value and a second threshold value, respectively; wherein the level of the random noise added to the image block is a minimum level or zero, if the ratio calculated for the image block is smaller than the first threshold value; wherein the level of the random noise added to the image block is a maximum level, if the ratio calculated for the image block is larger than the second threshold value; andwherein the level of the random noise added to the image block is between the minimum noise level and the maximum noise and depends on the ratio, if the ratio calculated for the image block is between the first threshold value and the second threshold value.

In an implementation form of the second aspect, if the ratio calculated for the image block is between the first threshold value and the second threshold value, the level of the random noise added to the image block is larger, if the local contrast value of the image block is smaller; and/orthe level of the random noise added to the image block is larger if the coding error value of the image block is larger.

In an implementation form of the second aspect, the method comprises, for each image block of the plurality of image blocks, determining the maximum noise level based on an average pixel value of the pixels of the image block.

In an implementation form of the second aspect, the method comprises converting the HDR image, which comprises the random noise added to each image block, to a standard dynamic range, SDR, image.

In an implementation form of the second aspect, the method comprises converting the HDR image from a PQ domain to a linear domain or to a gamma domain; andadd the random noise to each image block of the plurality of image blocks in the linear domain or in the gamma domain.

In an implementation form of the second aspect, the method comprisesconverting the HDR image from the linear domain to the gamma domain, after adding the random noise to each image block of the HDR image in the linear domain, and then generate the SDR image based on the HDR image converted to the gamma domain using quantization; orgenerate the SDR image based on the HDR image in the gamma domain using quantization, after adding the random noise to each image block of the HDR image in the gamma domain.

In an implementation form of the second aspect, the method comprises, for each image block of the plurality of image blocks, determining the coding error of the image block in the gamma domain or in the linear domain.

In an implementation form of the second aspect, the method comprisesdetermining the coding error of the image block by: quantizing original pixel values of the pixels of the image block in the gamma domain to obtain quantized pixel values; converting the quantized pixel values from the gamma domain to the linear domain to obtain reconstructed pixel values; and calculating the coding error of the image block based on a mean square error between the original pixel values and the reconstructed pixel values.

In an implementation form of the second aspect, the method comprises, for each image block of the plurality of image blocks, determining the local contrast value of the image block based on a pixel value of each pixel of the image block and an average pixel value of all the pixels in the image block.

The method of the second aspect may be performed by the device of the first aspect. The method of the second aspect and its implementation forms achieve the same advantages as described above for the device of the first aspect and its respective implementation forms.

A third aspect of this disclosure provides a computer program comprising instructions which, when the program is executed by a computer, cause the computer to perform the method to the second aspect or any of its implementation forms.

A fourth aspect of this disclosure provides a non-transitory storage mediumstoring executable program code which, when executed by a processor, causes the method according to the second aspect or any of its implementation forms to be performed.

According to the aspects and implementation forms, this disclosure suggests adding random noise to the HDR image in animage-block-wise manner. The level of the random noise may be computed adaptively based on the local contrast value of each image block, and optionally further based on the coding error value of each image block.

There are two main advantages of this approach. Firstly, the HDR image quality is highly related to the local contrast values, and such local contrast values can thus be used to evaluate the HDR image quality. Therefore, computing the random noise level based on the local contrast values can be more effective than other metrics, for instance, histograms or gradients.

Secondly, the random noise level is computed block-by-block instead of pixel-by-pixel. Therefore, the computational complexity is significantly lower for the solutions of this disclosure. This also means higher speeds and lower power consumption. Especially, when it comes to hardware implementations in power-sensitive devices, for instance, smartphones or AR/VR headsets.

It has to be noted that all devices, elements, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof.

BRIEF DESCRIPTION OF DRAWINGS

The above described aspects and implementation forms will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which

FIG. 1 shows an example of banding artefacts in a conventional 8-bit SDR image. FIG. 2 shows a device according to an embodiment of this disclosure for processing a HDR image.

FIG. 3 shows a device according to an embodiment of this disclosure for processing a HDR image.

FIG. 4 shows an exemplary device according to an embodiment of this disclosure for converting a HDR image into a SDR image.

FIG. 5 shows an exemplary device according to an embodiment of this disclosure for converting a HDR image into SDR image.

FIG. 6 shows a method according to an embodiment of this disclosure for processing a HDR image.

DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 2shows a device 200 according to an embodiment of this disclosure. The device 200 is configured to process a HDR image 201. For instance, the device 200 may process the HDR image 201 for conversion into an SDR image, i.e., either to prepare the HDR image 201 for the conversion or to convert the HDR image 201 into the SDR image. The device 200 may be a part of an image processing pipeline. The device 200 may also be configured to implement an entire image processing pipeline. The device 200 may be a processing device, for instance, a processor or computer. For example, the device 200 may be a processor of an SDR display, or of a smartphone, or of an AR or VR headset.

The device 200 is configured to determine a plurality of image blocks of the HDR image 201, wherein each image block 202 comprises a plurality of pixels of the HDR image 201. Each image block 202 may comprise an equal number of pixels, and these pixels may be arranged likewise per each image block 202. However, this is not a requirement in this disclosure, and two or more image blocks 202 may also differ in their number of pixels or in the arrangement of their pixels. For instance, different image blocks 202 may have the same block size or different block sizes. The determination of the image blocks 202 may optionally be performed by a first processing sub-unit of the device 200, as indicated by a dashed box in FIG. 2.

For each image block 202 of the plurality of image blocks, the device 200 is further configured to determine a local contrast value 203 of the image block 202. That is, the device 200 may be configured to calculate a plurality of local contrast values 202 for the DR image 201, in a block-by-block manner. The determination of the local contrast value 203 per each image block 202 may optionally be performed by a second processing sub-unit of the device 200, as indicated by a dashed box in FIG. 2.

Then, the device 200 is configured to add random noise 204 to each image block 202. A level of the random noise 204 added to each image block 202 depends on the local contrast value 203 determined for that image block 202. Thus, the device 200 may be configured to generate the random noise 204 for the image block 202 based on its local contrast value 203, and then add the random noise 204 to the image block. The generating and adding of the random noise 204 to each image block 202 may optionally performed by respectively a third processing sub-unit and fourth processing sub-unit of the device 200, as indicated by dashed boxes in FIG. 2. The third and fourth processing sub-unit may be one sub-unit, the separation in FIG. 2 is mainly for reasons of illustration. According to the above, if the local contrast values 203 for twoimage blocks 202 are different, then the levels of the random noise 204 added to these two image blocks 202 may also be different.

The device 200 may in consequence be configured to adaptively add the random noise 204 block-by-block to all of the image blocks 202 determined for the HDR image 201. Thereby, the random noise 202 is addedin dependence ofthe respective local contrast values 203 of these image blocks 202.

The device 200 may optionally be further configured to output the HDR image 201 that has the random noise 204 added to each image block 200 (as indicated by the dashed arrow in FIG. 2) , for instance, as an input for a subsequent conversion into the SDR image. This could be performed by another device. Alternatively, the device 200 may convert the HDR image 201 with the added random noise 204 directly to the SDR image, and may output the SDR image.

FIG. 3 shows a device 200 according to an embodiment of this disclosure, which builds on the embodiment of the device 200 shown in FIG. 2. Same elements in FIG. 2 and FIG. 3 are labelled with the same reference signs, and are implemented and function likewise. The device 200 shown in FIG. 3 is further configured to determine a coding error value 301 for each image block 202. That is, the device 200 is configured to determine, for each of the image blocks 202 of the HDR image 201, the local contrast value 203 as well as the coding error value 301. The determination of the coding error value 301 per each image block 202 may optionally be performed by a fifth processing sub-unit of the device 200, as indicated by the additional dashed box in FIG. 3.

The device 200 is configured to add the determined and generated random noise 204 to each image block 202, wherein the level of the random noise 204 added to each image block 202 now depends on the local contrast value 203 and on the coding error value 301 of that image block 202. That is, if the local contrast values 203 and/or if the coding error value 301 for two image blocks 202 are different, then the levels of random noise 204 added to these two image blocks 202 may also be different. Only in some cases, if two blocks have different local contrast values 203 and different coding error values 301, the calculation of the random noise 204 may still lead to the same level of random noise 204 for two image blocks 202. Namely when the differences of the local contrast values 203 and the differences of the coding error values 301 compensate each other in determining the level of the random noise 204.

For example, the level of the random noise 204 added to each image block 202 may depend on a ratio between the coding error value 301 and the local contrast value 203. Generally, for each image block 202, the level of the random noise 204 added to the image block 202 may become larger with a smaller local contrast value 203 of the image block 202, and/ormay become larger with a larger coding error value 301 of the image block 202.

The device 200 of FIG. 2 or FIG. 3 may comprise a processor or processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the device 200 described herein. The processing circuitry may comprise hardware and/or the processing circuitry may be controlled by software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as application-specific integrated circuits (ASICs) , field-programmable arrays (FPGAs) , digital signal processors (DSPs) , or multi-purpose processors. The device 200may further comprise memory circuitry, which stores one or more instruction (s) that can be executed by the processor or by the processing circuitry, for example, under control of the software. For instance, the memory circuitry may comprise a non-transitory storage medium storing executable software code which, when executed by the processor or the processing circuitry, causes the various operations of the device 200 to be performed. In one embodiment, the processing circuitry comprises one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the device 200 to perform, conduct or initiate the operations or methods described herein.

According to the above devices 200 of FIG. 2 and FIG. 3, a main aspect of this disclosure lies in the computation of the strength of the random noise 204, locally adaptiveto each image block 202. For each image block 202, as described with respect to FIG. 3, influencing factors for the appearance of banding artefacts may be the coding error value 301 and the local contrast value 203. The ratio between the coding error value 301 (which indicates an amount of coding artefacts) and the local contrast value 201 (which indicates an image content) may be the determinative quantity for the random noise computation. If said ratio is small, e.g., if the coding error value 301 is relatively small and the local contrast value 203 is relatively large –which typically is the case in image regions with textures, where banding artifacts rarely occur –no or only little random noise 204 may be added in. If said ratio is high, e.g., if the coding error value 301 is relatively large and the local contrast value 203 is relatively small –which is typicallythe case in smootherimage areas where banding artifacts tend to show up –a higher level ofrandom noise 204 is added in, i.e., the added random noise 204 is stronger.

FIG. 4 shows a device 200 according to an embodiment of this disclosure, which builds on the embodiment of the device 200 shown in FIG. 3. Same elements in FIG. 3 and FIG. 4 are labelled with the same reference signs, and are implemented and function likewise. For the device 200 of FIG. 4, the coding error value 301, the local contrast value 203, and the random noise 204, are all computed in the linear domain.

The input of the device 200 shown in FIG. 4 is the HDR image 201 in the PQ domain, wherein the HDR image 201 may be coded with a bit-depth of N, wherein N is typically 10 or 12. Notably, the HDR image 201 can also be coded with another non-linear curve, e.g., HLG, and the following steps performed by the device 200 are still applicable.

The output of the device 200 shown in FIG. 4 is the SDR image 401 in the gamma domain, wherein the SDR image 401 has a bit-depth of M, wherein M is typically 8, but may also be 10. In general, N and M can take other values too, however, under the constraint that N ≥ M. The SDR image 401 can be generated from the HDR image with the added random noise 204 by conversion from the linear into the gamma domain and using quantization.

At the beginning of the processing performed by the device 200, the HDR image 201 in the PQ domain is converted to the linear domain. Then, it may be normalized to a range between 0 and 1. The normalization may be conducted by dividing a maximum image pixel value or a peak brightness value of a display, on which the SDR image 401 is to be displayed.

Next, the local contrast value 203 and the coding error value 301 (also referred to as “distortion” in FIG. 4) are computed in the linear domain. The

values

203 and 301 may be computed block-by-block by the device 200, i.e., for each image block 202 of the HDR image 201. Based on the two computed

values

203 and 301, alevel (or strength) of random noise 204 can be computed for each image block 202 in the linear domain. Then, the random noise 204 can be generated in the linear domain, according to the respective computed levels, and can be added to each image block 202 in the linear domain. Finally, the HDR image 201 (now with the added random noise 204) can be converted from the linear domain to the gamma domain, and can be coded to M bit.

There are different ways for computingthe local contrast value 203. However, one possible way is tocompute the localcontrast value 203 of each image block 202 based on a pixel value of each pixel of that image block 202 and an average pixel value of all the pixels in that image block 202. For instance, the local contrast value 203 of the image block 202 may be calculated according to:

Therein, Cis the local contrast value 203 of the image block 202, K x K is the pixel size of the image block 202, I (x, y) is the value of a pixel in the image block 202, and

is the average of all pixel values in the image block 202.

It is worth mentioning that the block size of each image block 202 may increase proportionally with the image and/or video resolution. For instance, in the above given formula for calculating the local contrast value 203, K=16 may be selected for an image block 202 of a video in HD resolution (i.e., 1920x1080) , and K=32 may be selected foran image block 202 of a video in 4K resolution (i.e., 3840x2160) .

As mentioned, there are also other ways of computing the local contrast value 203 for each image block 202, and also the selected values of K, or of other parameters, may be different. However, the steps performed by the device 200 are still applicable.

To calculate the coding error value 301 in the linear domain as mentioned above, the HDR image 201 in the linear domain can further be converted to the gamma domainasshown in FIG. 4. In the gamma domain, the HDR image 201 can be quantized to M bit. The quantized values in the gamma domain can beconverted back from the gamma domain to the linear domain, wherein these values may be referred to as reconstructed values. Then, a root mean square error between the original pixel values and the reconstructed pixel values can be taken as the coding error value 301. For instance, the device 200 may be configured to quantize original pixel values of the pixels of each image block 202 in the gamma domain, to convert the quantized pixel values from the gamma domain to the linear domain to obtain the reconstructed pixel values, and to calculate the coding error value 301 of the image block 202 based on the mean square error. As an example, the device 200 may calculate the coding error value 301 of the image block 202 according to:

Therein, D is the coding error value 301 of the image block 202, andI′ (x, y) is the reconstructed value of a pixel in the image block 202. The other variables have been described above with respect to the calculation of C (local contrast value 203) .

The level of the random noise 204 can be calculated as a next step of the device 200. There are different forms of random noise 204, and the most frequently used ones are Gaussian noise and uniform noise. The further description of the device 200 shown in FIG. 4 is based on the example of Gaussian noise. However, the steps performed by the device 200 are equally applicable to the case of uniform noise or other forms of noise as well, without loss of generality. Only default values or some selected parametersmay be different.

Firstly, the ratio r between the coding error value 301and the local contrast value 203 for an image block 202 may be computed, for example, according to:

Next, the ratio r may be scaled, and may be clipped to a range between 0 and 1 to obtain the ratio R, for example, according to:

Therein, r _L and r _H are two threshold values, for instance, the first threshold valuer _L=0 and the second threshold value r _H= 0.5may be a possible choice in case that Gaussian noise is used. If the ratio r is smaller than thefirst threshold r _L, there are no visible banding artefacts, and thus no random noise 204 may be added at all or only a minimum noise level. If the ratio r is larger than the second thresholdr _H, said ratio is clipped to 1, in order to limit the noise strength to a certain maximum noise level, and to thus avoid adding too much noise. In between, the ratio r is scaled to between 0 and 1 linearly, and accordingly also the noise level will depend linearly on the ratio.

For example, this may mean that the level of the random noise 204 is a minimum noise level or zero, if the ratio r is smaller than the first threshold value, and the level of the random noise 204 may be a maximum noise level, if the ratio r calculated for the image block 202 is larger than the second threshold value, and the level of the random noise 204 may between the minimum noise level and the maximum noise level and depends on the ratio, if the ratio r is between the first threshold value and the second threshold value.

The level of the random noise 202 –or mathematically speaking the noise standard deviation –may be computed by the device 200 using the following equation: σ=max (0, a ^R·D -C)

Therein, a is a constant that enables tunability, e.g., depending on the type of SDR display or the field of application. A possible value may be a=10 for the case of Gaussian noise. N may be 10, and M may be 8. In general, amay be a function of N and M, that is, a= a (N, M) .

Finally, the random noise level may be limited according to an average pixel value in the image block 202:

Therein, Imax is the maximum value of the pixel, and may be generally 1 because the image is normalized to the range between 0 and 1, and I _T is a threshold (apossible value of which may be I _T=Imax/2) . The constant amay also influence how much the noise level is limited, and may be set to 3 for Gaussian noise. Intuitively speaking, the level of the random noise 204 may be limited more for pixel values closer to zero, in order to avoid making too many of these pixels too dark, and may also be limited more for pixel values closer to the maximum, in order to avoid too many oversaturated pixels. The pixel values in the middle, i.e., those pixels closer to the threshold I _T, may be less limited.

FIG. 5 shows a device 200 according to an embodiment of this disclosure, which builds on the embodiment of the device 200 shown in FIG. 3. Same elements in FIG. 5 and FIG. 3 are labelled with the same reference signs, and are implemented and function likewise. For the device 200 shown in FIG. 5, the coding error value 301, the local contrast value 203, and the random noise 204, are all computed in the gamma domain.

The input of the HDR image 201, and the output of the SDR image 401, are the same as for the device 200 shown in FIG. 4. The major difference between the devices 200 of FIG. 4 and FIG. 5 is that the computation of the relevant local contrast values 203 and coding error values 301, and of the level of random noise 204, areperformed in the gamma domain in FIG. 5 not in the linear domain as in FIG. 4. As an example, no linear domain may be involved at all, as exemplarily shown in FIG. 5.

At the beginning of the processing of the device 200 of FIG. 5, the HDR image 201 is converted from the PQ domain to the gamma domain. Then, for each image block 202, the local contrast value 203, the coding error value 301, and the level of the random noise 204, are computed in the gamma domain. Specifically, these

values

203, 301 and 204 may be calculated based on the pixel values of the image blocks 202 in the gamma domain. For calculating the coding error value 301, the HDR image in the gamma domain may be quantized as described above with respect to Fig. 4. Then, the random noise 204 is added to the image blocks 202 also in the gamma domain. Finally, the HDR image with random noise 204 is coded to M bit, in order to produce the SDR image 401.

The above equations described regarding FIG. 4, respectivelyfor computing the local contrast values 203, the coding error values 301, and the level of random noise 204 for each image block 202, are also valid for the device 200 shown in FIG. 5. Some parameters may be tuned differently.

FIG. 6 shows a method 600 according to an embodiment of this disclosure. The method 600 is suitable for processing a HDR image 201, specifically, for converting the HDR image 201 into a SDR image 401. The method 600 may be performed by the device 200 of any one of FIGs. 2-5.

The method 600 comprises a step 601 of determining a plurality of image blocks 202 of the HDR image 201, wherein each image block 202 comprise a plurality of pixels. This step 601 may be performed by the first processing sub-unit of the device 200. Further, the method 6000 comprises a step 602 of determining, for each image block 202, a local contrast value 203 of the image block 202. Then, the method 600 comprises a step 601 of adding 603 random noise 204 to each image block 202, wherein a level of the random noise 204 added to the image block 202 depends on the local contrast value 203of the image block 202, i.e., determined for the image block 202.

The present disclosure has been described in conjunction with various embodiments as examples as well as aspects and implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed matter, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

A device (200) for processing a high dynamic range, HDR, image (201) , the device (200) being configured to:

determine a plurality of image blocks of the HDR image (201) , each image block (202) comprising a plurality of pixels; and

for each image block (202) of the plurality of image blocks:

determine a local contrast value (203) of the image block (202) ; and

add random noise (204) to the image block (202) , wherein a level of the random noise (204) added to the image block (202) depends on the local contrast value (203) of the image block (202) .
The device (200) according to claim 1, configured to adaptively add random noise (204) block-by-block to all image blocks (202) of the HDR image (201) according to the local contrast values (203) of the image blocks (202) .
The device (200) according to claim 1 or 2, further configured to, for each image block (202) of the plurality of image blocks:

determine a coding error value (301) of the image block (202) ;

wherein the level of the random noise (204) added to the image block (202) depends further on the coding error value (301) of the image block (202) .
The device (200) according to claim 3, further configured to, for each image block (202) of the plurality of image blocks:

calculate a ratio between the coding error value (301) and the local contrast value (203) of the image block (202) ;

wherein the level of the random noise (204) added to the image block (202) depends on the ratio calculated for the image block (202) .
The device (200) according to claim 4, further configured to, for each image block (202) of the plurality of image blocks:

determine whether the ratio calculated for the image block (202) is below or above a first threshold value and a second threshold value, respectively;

wherein the level of the random noise (204) added to the image block (202) is a minimum noise level or zero, if the ratio calculated for the image block (202) is smaller than the first threshold value;

wherein the level of the random noise (204) added to the image block (202) is a maximum noise level, if the ratio calculated for the image block (202) is larger than the second threshold value; and

wherein the level of the random noise (204) added to the image block (202) is between the minimum noise level and the maximum noise level and depends on the ratio, if the ratio calculated for the image block (202) is between the first threshold value and the second threshold value.
The device (200) according to claim 5, wherein, if the ratio calculated for the image block (202) is between the first threshold value and the second threshold value:

the level of the random noise (204) added to the image block (202) is larger, if the local contrast value (203) of the image block (202) is smaller; and/or

the level of the random noise (204) added to the image block (202) is larger if the coding error value (301) of the image block (202) is larger.
The device (200) according to claim 5 or 6, further configured to, for each image block (200) of the plurality of image blocks:

determine the maximum noise level based on an average pixel value of the pixels of the image block (202) .
The device (200) according to one of the claims 1 to 7, further configured to convert the HDR image (201) , which comprises the random noise (204) added to each image block (202) , to a standard dynamic range, SDR, image (401) .
The device (200) according to one of the claims 1 to 8, further configured to:

convert the HDR image (201) from a perceptual quantizer, PQ, domain to a linear domain or to a gamma domain; and

add the random noise (204) to each image block (202) of the plurality of image blocks in the linear domain or in the gamma domain.
The device (200) according to claim 8 and 9, further configured to:

convert the HDR image (201) from the linear domain to the gamma domain, after adding the random noise (204) to each image block (202) of the HDR image (201) in the linear domain, and then generate the SDR image (401) based on the HDR image (201) converted to the gamma domain using quantization; or

generate the SDR image (401) based on the HDR image (201) in the gamma domain using quantization, after adding the random noise (204) to each image block (202) of the HDR image (201) in the gamma domain.
The device (200) according to one of the claims 3 to 10, further configured to, for each image block (202) of the plurality of image blocks:

determine the coding error value (301) of the image block (202) in the gamma domain or in the linear domain.
The device (200) according to claim 11, further configured to determine the coding error value of the image block (202) by:

quantizing original pixel values of the pixels of the image block (202) in the gamma domain to obtain quantized pixel values;

converting the quantized pixel values from the gamma domain to the linear domain to obtain reconstructed pixel values; and

calculating the coding error value (301) of the image block (202) based on a mean square error between the original pixel values and the reconstructed pixel values.
The device (200) according to one of the claims 1 to 11, further configured to, for each image block (202) of the plurality of image blocks:

determine the localcontrast value (203) of the image block (202) based on a pixel value of each pixel of the image block (202) and an average pixel value of all the pixels in the image block (202) .
A method (600) for processing a high dynamic range, HDR, image (201) , the method (600) comprising:

determining (601) a plurality of image blocks of the HDR image (201) , each image block (202) comprising a plurality of pixels; and

for each image block (202) of the plurality of image blocks:

determining (602) a local contrast value (203) of the image block (202) ; and

adding (603) random noise (204) to the image block (202) , wherein a level of the random noise (204) added to the image block (202) depends on the local contrast value (203) of the image block (202) .
A computer program comprising instructions which, when the program is executed by a computer, cause the computer to perform the method (600) according to claim 14.