CN116167950B

CN116167950B - Image processing method, device, electronic equipment and storage medium

Info

Publication number: CN116167950B
Application number: CN202310457578.9A
Authority: CN
Inventors: 佩德拉姆·穆罕默德; 埃内斯托·安德拉德·内托; 李健; 楼剑
Original assignee: Rongming Microelectronics Shanghai Co ltd
Current assignee: Rongming Microelectronics Shanghai Co ltd
Priority date: 2023-04-26
Filing date: 2023-04-26
Publication date: 2023-08-04
Anticipated expiration: 2043-04-26
Also published as: CN116167950A

Abstract

The application relates to an image processing method, an image processing device, electronic equipment and a storage medium. The image processing method comprises the following steps: converting an input original image from an RGB color space to an HSL color space, and acquiring a first brightness value of a brightness channel; the original image is an SDR image; performing PQ conversion on the first brightness value to obtain a second brightness value; converting the second brightness value into a third brightness value according to an equation of the global mapping curve to obtain a first intermediate image; converting the first intermediate image into an RGB color space to obtain a second intermediate image; converting the second intermediate image from the original color gamut to the target color gamut to obtain a third intermediate image; the target color gamut is the color gamut of the target image, and the target image is an HDR image; and generating the target image according to the appointed format of the target image and the third intermediate image. According to the technical scheme, the whole SDR image can be converted into the HDR image, and high-quality visual quality output is provided for the SDR image with different brightness conditions.

Description

Image processing method, device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing device, an electronic device, and a storage medium.

Background

High Dynamic Range (HDR) imaging is considered the next revolution in digital media because it strives to follow the ability of the Human Visual System (HVS) to perceive the world, resulting in a viewing experience very close to the real world. HDR image and video technology can capture all light and color information in a scene, providing an effect that approximates our perception of eyes. This is in contrast to conventional Standard Dynamic Range (SDR) acquisitions, which can only capture and reproduce limited dynamic range and color information present in real scenes. Because the HDR value corresponds to an absolute light value, HDR can mimic the function of the HVS in terms of luminance and color ranges. HDR technology thoroughly changes the overall flow of digital media representations, affecting acquisition, compression, transmission, and display. HDR focuses on visual changes, allowing for the simultaneous capture of very dark and bright regions in a scene, solving the challenges of dark and bright regions present in SDR, and providing more detail of the entire scene. The pixels comprised by the HDR image (or video) may represent a larger range of colors and brightness levels than the pixels provided by the existing SDR image (or video). These pixels greatly improve the overall visual quality of the image (or video) content, making it more realistic to look more attractive to viewers.

The increasing popularity of HDR technology opens new market opportunities for content providers such as Netflix, hollywood movie companies, and the entire broadcast industry. However, the process of acquiring HDR remains challenging and costly, as the technology is still in an immature stage. It is therefore important to be able to make HDR content by converting existing SDR content into a new format and exploiting the functionality of the new HDR display. This can still significantly improve the visual quality of content previously captured in SDR format on an HDR display.

In the related art, some work has focused on converting an image in SDR format into HDR format, developing a so-called inverse tone mapping operator (itto) to achieve this conversion. In view of the fact that one tries to reconstruct the lost information, the design of an itto has proven to be very challenging. Thus, most work focuses only on a specific class of content, either on normal (average brightness) content in SDR format images, or on dark and/or bright content, with certain limitations. Therefore, how to overcome the above technical defect of converting an SDR format image into an HDR format image is a technical problem to be solved.

Disclosure of Invention

The invention aims to provide an image processing method, an image processing device, electronic equipment and a storage medium, which can convert an entire SDR image into an HDR image and provide high-quality visual quality output for SDR images with different brightness conditions.

According to a first aspect of embodiments of the present application, there is provided an image processing method, including:

converting an original image input in a light domain from an RGB color space to an HSL color space, and acquiring a first brightness value of a brightness channel; the original image is a standard dynamic range SDR image;

performing Perceptual Quantization (PQ) transformation on the first brightness value to obtain a second brightness value corresponding to the PQ domain;

converting the second brightness value into a third brightness value according to an equation of a designated global mapping curve to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image;

converting the first intermediate image from an HSL color space to an RGB color space to obtain a second intermediate image;

converting from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image to obtain a third intermediate image; the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of a target image, and the target image is a high dynamic range HDR image;

And generating the target image according to the appointed format of the target image and the third intermediate image.

In one embodiment, after the converting the first intermediate image from the HSL color space to the RGB color space to obtain the second intermediate image and before the converting from the original color gamut to the target color gamut to obtain the third intermediate image according to the color coordinates and the color conversion matrix of the second intermediate image, the method further includes:

for each color channel, performing color adjustment on the second intermediate image according to a designated color adjustment equation, and keeping the tone of the color unchanged and the brightness basically unchanged to obtain a fourth intermediate image;

after the fourth intermediate image is obtained, the third intermediate image is obtained by converting the original color gamut into the target color gamut according to the color coordinates of the fourth intermediate image and the color conversion matrix.

Since the color adjustment is performed according to the designated color adjustment equation, the hue and the brightness of the color are kept unchanged basically, and thus the color accuracy between the input SDR image and the output HDR image can be ensured.

In one embodiment, the color adjustment equation is as follows:

；

wherein ,for the color saturation of any color channel of said second intermediate image in the PQ domain,color saturation for any color channel of said fourth intermediate image in the PQ domain, +.>For the second luminance value, +.>For the value of the third luminance value,

αandβare two constants responsible for controlling the hue and saturation of the color.

In one embodiment, the global mapping curve is a piecewise linear curve, and comprises two endpoints, two demarcation points and three sections of linear curves, wherein the three sections of linear curves form a continuous curve, and the two endpoints and the two demarcation points divide the original image into a dark area, a normal area and a bright area; the three sections of linear curves are in one-to-one correspondence with the dark areas, the normal areas and the bright areas;

the converting the second luminance value into a third luminance value according to the equation of the specified global mapping curve includes:

determining the two endpoints and the two demarcation points according to the equation of the global mapping curve;

dividing the original image into a dark area, a normal area and a bright area according to the two endpoints and the two demarcation points;

and converting the second brightness value into a corresponding third brightness value according to an equation of a global mapping curve for each region of the original image.

In one embodiment, the equation for the global mapping curve is as follows:

；

wherein ,for the second luminance value, +.>For the third luminance value, +.>And->The slopes of the three linear curves, +.>And->The intercept of three linear curves, respectively +.>And->Is the abscissa of the two demarcation points, +.>For the minimum value of the second luminance value, and (2)>Is the maximum value of the second luminance value.

In one embodiment of the present invention, in one embodiment,is calculated by the following calculation formula:

；

is calculated by the following calculation formula:

；

wherein ,is the abscissa of the demarcation point of the dark and normal regions, +.>Is the abscissa of the demarcation point of the normal and bright areas.

In one embodiment of the present invention, in one embodiment,and->The value of (2) can be obtained by the following method:

acquiring a histogram of the original image in a PQ domain according to the second brightness value;

performing self-adaptive segmentation on the original image according to the histogram, and dividing the original image into a dark area, a normal area and a bright area;

obtaining according to the demarcation point of the dark area and the normal areaAccording to the value of the positiveThe demarcation point of the normal region and the bright region is +.>Is a value of (2).

In one embodiment of the present invention, in one embodiment, and />The calculation method of (2) is as follows:

Determination ofAnd->Respective corresponding values in the HDR domain->And->；

Acquiring coordinates of the two endpoints;

according toAnd the coordinates of the two endpoints are calculated to obtain and />Is a value of (2).

In one embodiment of the present invention, in one embodiment,and->The value of (2) is calculated by the following equation:

；

wherein ,and->Two constants.

In one embodiment of the present invention, in one embodiment,0.15%>0.4.

determining the respective pixel numbers in the dark area, the normal area and the bright area according to the histogram;

determining the weights of the dark area, the normal area and the bright area according to the respective pixel numbers in the dark area, the normal area and the bright area;

simulating according to the weights of the dark area, the normal area and the bright area to determineAnd->Is a value of (2).

In one embodiment, the global mapping curve is a gamma curve, and the equation for the gamma curve is as follows:

；

wherein ,for the second luminance value, +.>For the value of the third luminance value,c ₁ and (3) withc ₂ Respectively constant;

c ₂ The formula of (2) is as follows:

；

c ₁ the formula of (2) is as follows:

；

wherein ,for the minimum value of the second luminance value, and (2)>For the maximum value of said second luminance value, +.>Minimum luminance value for target HDR display in PQ domain,/->Is the maximum luminance value of the target HDR display in the PQ domain.

According to a second aspect of embodiments of the present application, there is provided an image processing apparatus including:

a first acquisition module configured to convert an original image input in a light domain from an RGB color space to an HSL color space and acquire a first luminance value of a luminance channel; the original image is a standard dynamic range SDR image;

a second acquisition module configured to perform a perceptual quantization, PQ, transformation on the first luminance value to obtain a second luminance value corresponding in a PQ domain;

the first conversion module is configured to convert the second brightness value into a third brightness value according to an equation of a designated global mapping curve to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image;

a second conversion module configured to convert the first intermediate image from an HSL color space to an RGB color space, resulting in a second intermediate image;

The third conversion module is configured to convert the original color gamut into the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image, so as to obtain a third intermediate image; the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of a target image, and the target image is a high dynamic range HDR image;

and the generation module is configured to generate the target image according to the designated format of the target image and the third intermediate image.

In one embodiment, the image processing apparatus further includes:

the color adjustment module is configured to perform color adjustment on the second intermediate image according to a specified color adjustment equation for each color channel, keep the tone of the color unchanged and the brightness basically unchanged, and obtain a fourth intermediate image;

the third conversion module is further configured to convert from an original color gamut to a target color gamut according to the color coordinates of the fourth intermediate image and the color conversion matrix after obtaining the fourth intermediate image, and obtain the third intermediate image.

According to a third aspect of embodiments of the present application, there is provided an electronic device comprising a memory and a processor, the memory being for storing a computer program executable by the processor; the processor is configured to execute the computer program in the memory to implement the method described above.

According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program, characterized in that the above-mentioned method is enabled when the executable computer program in the storage medium is executed by a processor.

Compared with the prior art, the beneficial effect of this application lies in: since an original image (SDR image) input in the light domain is converted from the RGB color space to the HSL color space and the luminance of the luminance channel is acquired to obtain a first luminance value, and then the first luminance value is subjected to Perceptual Quantization (PQ) conversion to obtain a corresponding second luminance value in the PQ domain, that is, the SDR image is converted to the perceptual domain, it helps to maintain the overall visual impression of the frame.

Then, according to the equation of the designated global mapping curve, the second luminance value is converted into the third luminance value, where the third luminance value is the luminance value of the luminance channel of the first intermediate image, on one hand, since the global mapping curve is used instead of the local mapping curve, the luminance of the whole image of the original image can be converted, that is, all the regions with different luminance levels in the SDR image are completely converted, instead of only the regions with a certain type of specific luminance level, so that the whole SDR image can be converted into the HDR image, and high-quality visual quality output is provided for the SDR image with different luminance conditions, and on the other hand, since the global mapping curve is used to adjust the luminance of the whole image of the original image as a whole, rapid change of the luminance which may cause flickering can be avoided, and further flickering artifacts can be avoided.

Then, converting the first intermediate image from the HSL color space to the RGB color space to obtain a second intermediate image, then converting the second intermediate image from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image to obtain a third intermediate image, and finally generating the target image according to the appointed format of the target image and the third intermediate image.

In summary, the technical scheme provided by the application can realize that the whole SDR image is converted into an HDR image, and provides high-quality visual quality output for the SDR image with different brightness conditions; flicker artifacts can also be avoided during the image format conversion process.

Drawings

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment.

FIG. 2 is a schematic diagram illustrating a gamma curve according to an exemplary embodiment.

Fig. 3 is a flowchart illustrating an image processing method according to another exemplary embodiment.

Fig. 4 is a flowchart illustrating an image processing method according to another exemplary embodiment.

FIG. 5 is a schematic diagram illustrating a piecewise linear curve in accordance with an exemplary embodiment.

Fig. 6 is a flowchart illustrating an image processing method according to another exemplary embodiment.

Fig. 7 is a flowchart illustrating an image processing method according to another exemplary embodiment.

Fig. 8 is a block diagram of an image processing apparatus according to an exemplary embodiment.

Fig. 9 is a block diagram of an image processing apparatus according to another exemplary embodiment.

Fig. 10 is a block diagram of an electronic device, according to an example embodiment.

Detailed Description

Unless defined otherwise, technical or scientific terms used in the specification and claims should be given the ordinary meaning as understood by one of ordinary skill in the art to which the invention pertains. In the following, specific embodiments of the present invention will be described with reference to the drawings, and it should be noted that in the course of the detailed description of these embodiments, it is not possible in the present specification to describe all features of an actual embodiment in detail for the sake of brevity. Modifications and substitutions of embodiments of the invention may be made by those skilled in the art without departing from the spirit and scope of the invention, and the resulting embodiments are also within the scope of the invention.

In the related art, in addition to the challenges of designing an ito due to attempting to reconstruct lost information, another important challenge of designing an ito stems from the fact that when we attempt to expand dynamic range, increase bit depth and brightness levels, we are faced with the opportunity to introduce and/or increase noise in different areas of the image, which is especially common in dark areas.

Furthermore, any change in brightness, regardless of the color space or color gamut processed, can result in a color shift. This variation is not noticeable when processing SDR content, but the higher luminance range in HDR results in noticeable visual differences. Therefore, maintaining color accuracy in the HDR case is most important when designing the itto.

In order to solve the technical problems, the application provides an image processing method, an image processing device, electronic equipment and a storage medium, which can convert a whole SDR image into an HDR image and provide high-quality visual quality output for SDR images with different brightness conditions; flicker artifacts can be avoided in the image format conversion process; and the color accuracy between the input SDR image and the output HDR image can be ensured.

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment. The image processing method can be applied to electronic equipment such as an image processor, a display chip and the like. Referring to fig. 1, the image processing method may include the steps of:

step 101, converting an original image input in a light domain from an RGB color space to an HSL color space, and acquiring a first brightness value of a brightness channel; the original image is an SDR image.

In this embodiment, the original image is in the RGB color space. In the RGB color space, an image is represented by three color channels, a red color channel, a green color channel, and a blue color channel. In the HSL color space, an image is represented by three channels, namely a hue channel, a saturation channel and a brightness channel. Therefore, before adjusting the brightness of the original image, the original image is converted from the RGB color space to the HSL color space, and the value of the brightness channel is extracted, resulting in a first brightness value.

In the present embodiment, the color gamut of the original image is bt.709 color gamut. Bt.709 color gamut was specifically designed for SDR images.

In this embodiment, the value of the luminance channel of the original image input in the optical domain may be calculated according to the bt.709 standard, to obtain the first luminance value.

Step 102, performing PQ conversion on the first luminance value to obtain a second luminance value corresponding to the first luminance value in the PQ domain.

In this embodiment, the PQ (Perceptual Quantizer) field is one way of encoding and compressing video signals. The PQ domain belongs to the perception domain, and is a perceptually uniform color space, intended to more closely match the way the human visual system perceives colors. The use of a logarithmic transfer function for the PQ domain allows for more efficient use of the available dynamic range. This makes it well suited for displaying High Dynamic Range (HDR) video content.

In this embodiment, perceptual Quantization (PQ) aims to optimize the light intensity distribution associated with the HVS attribute, converting the physical linearity values into perceptual linearity values. Perceptual linearity means that the human eye sees any intensity change at any brightness level in the same way (i.e. two consecutive brightness levels just below the threshold just noted by the HVS). Perceptual linearity is the basis of implementation because it is possible to identify what the human eye can see and map this information to a higher dynamic range in an optimal way. Therefore, first the first luminance value of the luminance channel of the input SDR image is transmitted to the PQ domain, so that the luminance value of the SDR image and the luminance value of the HDR image can be located in the same domain in the mapping process. This enables us to work in the perceptual domain, adjacent luminance values corresponding to differences that are perceivable by the human eye.

In this embodiment, the first luminance value of the luminance channel is subjected to PQ conversion to obtain the corresponding second luminance value in the PQ domain.

In this embodiment, the first luminance value may be PQ transformed using a perceptual quantizer. The perceptual quantizer is a transfer function that maps the linear light intensity of the display device to a non-linear signal to better match the characteristics of the Human Visual System (HVS). The function of the perceptual quantizer aims at optimizing the distribution of the light intensities in order to most efficiently utilize the available dynamic range and to provide an optimal reproduction of the original image.

In this embodiment, the perceptual quantizer is a SMPTE ST2084 transfer function. It is defined by a power function with an exponent of about 78124 and an output offset of 0.5. The transfer function is intended to provide a closer match to the nonlinear response of the HVS to light and to better utilize the available dynamic range. SMPTE ST2084 transfer functions are widely used to make HDR content for display devices such as televisions and projectors. The SMPTE ST2084 transfer function also uses a bit depth of 10 or 12 bits, as it may represent more than 8 bits to maintain high precision of the HDR content.

In this embodiment, the transfer function that can be used by the perceptual quantizer is as follows:

；

wherein ,is a first luminance value, ">Is the second luminance value. />Andare constant and can be set to 0.8359, 18.8515, 18.6875, 0.1593 and 78.8437, respectively.

In this embodiment, it achieves an acceptable visual quality level based on a preliminary subjective video quality assessment, since it seeks to simplify the complex equations of the PQ transformation using power functions. Thus, the itto of the present embodiment is hardware friendly and can be implemented in hardware with limited resources.

Step 103, converting the second brightness value into a third brightness value according to the equation of the designated global mapping curve to obtain a first intermediate image; the third luminance value is a luminance value of a luminance channel of the first intermediate image.

Flicker is a common problem that can occur during tone mapping, especially when using local tone mapping algorithms. This is because the local algorithm adjusts the brightness of individual pixels or a small group of pixels independently, resulting in rapid changes in the brightness of the entire image. These brightness variations may be particularly noticeable in high contrast areas and may be distracting and cause visual impact. To avoid this problem, the global mapping algorithm (equation of global mapping curve) in the present application uses different methods. It does not adjust the brightness of individual pixels, but rather averages the brightness values of the entire image. Since the brightness of the entire image is adjusted as a whole, this approach avoids rapid changes in brightness that may cause flicker.

In this embodiment, the global mapping curve may be determined according to the curve identifier input by the user. For example, the curve is identified as 00, the designated global map curve is a gamma curve, the curve is identified as 01, and the designated global map curve is a piecewise linear curve.

In this embodiment, the global mapping curve is a gamma curve, and the equation of the gamma curve is as follows:

；

wherein ,for the second brightness value, +. >For the value of the third luminance value,c ₁ and (3) withc ₂ Respectively, are constants.c ₁ And (3) withc ₂ The value of (2) may be determined from information entered by the user or may be calculated.

In the present embodiment, the following may be employed firstCalculation formula calculationc ₂ ：

；

Then, the following calculation formula is adopted for calculationc ₁ ：

；

wherein ,is the minimum value of the second luminance value, +.>Is the maximum value of the second luminance value,minimum luminance value for target HDR display in PQ domain,/->Is the maximum luminance value of the target HDR display in the PQ domain.

In the present embodiment, the gamma curve C1 is used as shown in fig. 2.

In this embodiment, the global mapping curve adopts a gamma curve, which has a simple mathematical form and low computational complexity, and is capable of generating a high-quality HDR image. Moreover, the contrast ratio of the generated HDR image is relatively high. The global mapping curve employs a gamma curve, which may also create a balance between the overall brightness and contrast of the generated HDR image.

In this embodiment, the mapping curve is constructed as the maximum and minimum luminance values of the target HDR display are taken into account. In so doing, the image processing method is enabled to generate content that appears pleasing on all types of HDR displays. Thus, the algorithms provided herein are display adaptive and can generate HDR image content that matches the functionality of any target HDR display.

Step 104, converting the first intermediate image from the HSL color space to the RGB color space to obtain a second intermediate image.

In this embodiment, the image data is subsequently manipulated in the RGB color space, and therefore the first intermediate image is converted from the HSL color space to the RGB color space, resulting in the second intermediate image.

Step 105, for each color channel, performing color adjustment on the second intermediate image according to the designated color adjustment equation, and keeping the hue and brightness of the color unchanged, thereby obtaining a fourth intermediate image.

An important consideration in designing an efficient ito is maintaining color accuracy between the input SDR image and the generated HDR image. This is a challenge because color shifting may result when extending the brightness level from an SDR image to an HDR image. Therefore, an effective color adjustment scheme is important in designing the itto.

Color adjustment refers to the process of adjusting the color in an image to make it look more natural or consistent with the intended visual appearance. This is typically done by adjusting the levels of red, green and blue (RGB) in the image. Color adjustment is an important step because it helps to maintain the desired color balance of the scene and to maintain the overall visual appeal of the image. Without proper color adjustment, colors in an HDR image may appear oversaturated, unnatural, or even distorted. In addition, it helps to avoid skin tone appearing too red or other inaccurate colors. Furthermore, in many cases, when the brightness of an image is modified, the color balance of the image may change, making it appear too warm or too cold. Color adjustment helps to restore the desired color balance of the image and creates a more pleasing visual experience for the viewer.

In this embodiment, in order to ensure the accuracy of the color of the generated image, the color adjustment equation is adopted as follows:

；

wherein ,for the color saturation of any color channel (red, green, or blue) of the second intermediate image in the PQ domain +.>Color saturation for any color channel of the fourth intermediate image in the PQ domain, +.>For the second brightness value, +.>For the value of the third luminance value,αandβare two constants responsible for controlling the hue and saturation of the color.

In this embodiment, since the color adjustment is performed according to the specified color adjustment equation, the hue and the brightness of the color are kept unchanged, so that the color accuracy between the input SDR image and the output HDR image can be ensured.

In addition, according to a broad simulation, the above color adjustment equation preserves the hue of the color while having negligible effect on its brightness when mapping an SDR image to an HDR image.

In the present embodiment, by performing color adjustment in the perceptual domain, the image processing method is caused to preserve the hue of the color of the SDR image and generate the color of the HDR image that is very close to the color of the SDR image.

Step 106, converting from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the fourth intermediate image to obtain a third intermediate image; the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is the HDR image.

In this embodiment, the target color gamut is bt.2020 color gamut. Because bt.709 gamut is designed for SDR images, rather than HDR images, after color adjustment, the gamut needs to be converted from bt.709 gamut to bt.2020 gamut. In this way, it is ensured that the generated content shows the correct color.

Wherein converting the color gamut from bt.709 color gamut to bt.2020 color gamut involves converting the color coordinates of a given image or video from one color space to another color space.

In one embodiment, gamut conversion may be performed using a 3*3 color conversion matrix that maps colors in a source color space (bt.709) to colors in a target color space (bt.2020). The process typically begins with converting an image or video from RGB to a native color representation of the color space. The color conversion matrix is then applied to the color coordinates of the image, converting them to a new color space. The matrix for converting from bt.709 gamut to bt.2020 gamut can be expressed as:

R2020 = 0.627404078626 × R709 + 0.329282097415 × G709 + 0.043313797587 × B709

G2020 = 0.069097233123 × R709 + 0.919541035593 × G709 + 0.011361189924 × B709

B2020 = 0.016391587664 × R709 + 0.088013255546 × G709 + 0.895595009604 × B709

wherein R, G and B represent red, green and blue channels, respectively. R709, G709, B709 are color coordinates of the fourth intermediate image in the bt.709 color gamut, and R2020, G2020, B2020 are color coordinates of the third intermediate image in the bt.2020 color gamut.

It should be noted that this step is quite complex, requires high accuracy, and this color conversion matrix is only an approximation, and if an accurate conversion is to be performed, a three-dimensional color look-up table (3D-CLUT) is required. It is also important that bt.2020 gamut be wider than bt.709, and the result of this conversion may be that some colors are out of range of the target color space.

Step 107, generating a target image according to the designated format of the target image and the third intermediate image.

The target image has two formats: HDR (high definition digital television) ₁₀ and HLG₁₀ . The difference between these two formats is their transfer function. HDR (high definition digital television) ₁₀ The format uses a so-called PQ function, while HLG ₁₀ The format uses mixed logarithmic gammaHybrid Log-Gamma, HLG for short). The PQ function is the function for performing PQ transformation as described above, and will not be described in detail herein.

In the present embodiment, the specified format of the target image may be determined from the identification of the image format input by the user. For example, when the identification of the image format is 01, the designated format of the target image is HDR ₁₀ When the identification of the image format is 11, the designated format of the target image is HLG ₁₀ 。

In the present embodiment, the designation format of the target image is HDR ₁₀ . Due to HDR ₁₀ The formatted image is already present in the PQ domain and therefore the target image can be directly output without performing any further conversion.

According to the technical scheme, the original image (SDR image) input in the light domain is converted from the RGB color space to the HSL color space, the brightness of the brightness channel is obtained, the first brightness value is obtained, then the first brightness value is subjected to Perceptual Quantization (PQ) conversion, and the corresponding second brightness value in the PQ domain is obtained, namely, the SDR image is converted to the perception domain, so that the whole visual impression of the frame is maintained. Then, the second luminance value is converted into the third luminance value according to the equation of the specified global mapping curve, on the one hand, since the global mapping curve is used instead of the local mapping curve, the luminance of the whole image of the original image can be converted, that is, all the regions with different luminance levels in the SDR image are subjected to luminance conversion, instead of only the regions with certain specific luminance levels, so that the whole SDR image can be converted into the HDR image, and high-quality visual quality output can be provided for the SDR image with different luminance conditions, and on the other hand, since the global mapping curve is used for adjusting the luminance of the whole image of the original image as a whole, the rapid change of the luminance which may cause flickering can be avoided, and the flickering artifact can be avoided. Then, converting the first intermediate image from the HSL color space to the RGB color space to obtain a second intermediate image, and then converting from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image to obtain a third intermediate image, so that the display of the correct color for the generated target image can be ensured. And finally, generating the target image according to the appointed format of the target image and the third intermediate image. In summary, the technical scheme provided by the application can realize that the whole SDR image is converted into an HDR image, and provides high-quality visual quality output for the SDR image with different brightness conditions; flicker artifacts can also be avoided during the image format conversion process.

It should be noted that, in other embodiments, as shown in fig. 3, the above step 105 may be omitted, and the conversion of the entire SDR image into the HDR image may be implemented, so as to provide a high-quality visual quality output for the SDR image with different brightness conditions. Steps 301 to 304, 306 are the same as steps 101 to 104, 107, and step 305 is similar to step 106, and will not be repeated here.

Fig. 4 is a flowchart illustrating an image processing method according to another exemplary embodiment. Referring to fig. 3, the image processing method may include the steps of:

(1) Extracting brightness of color channel

The present step is the same as step 101, and will not be described here again.

(2) PQ conversion

The present step is the same as step 102, and will not be described here again.

(3) Selecting a mapping curve

In this embodiment, the mapping curve is a global mapping curve, and the global mapping curve includes a gamma curve and a piecewise linear curve. The electronic device may select the mapping curve based on the curve identification entered by the user.

In this embodiment, the selected mapping curve is a piecewise linear curve. As shown in fig. 5, the global mapping curve C2 includes two endpoints D1, D2, two demarcation points D3, D4 and three sections of linear curves C21, C22, C23, wherein the three sections of linear curves C21, C22, C23 form a continuous curve, and the two endpoints D1, D2 and the two demarcation points D3, D4 divide the original image into a dark area, a normal area and a bright area; the three linear curves are in one-to-one correspondence with the dark area and the normal area and the bright area, specifically, the linear curve C21 is a curve corresponding to the dark area, the linear curve C22 is a curve corresponding to the normal area, and the linear curve C23 is a curve corresponding to the bright area. Is the abscissa of the two demarcation points D3, D4,

And->Is the ordinate of the two demarcation points D3, D4,>and->The slopes of the three linear curves C21, C22, C23, respectively.

In this embodiment, the equation of the global mapping curve is as follows:

；

wherein ,for the second brightness value, +.>For the third brightness value, +.>And->The intercepts of the three linear curves C21, C22, C23, respectively, +.>Is the minimum value of the second luminance value, +.>Is the maximum value of the second luminance value.

The method of selecting the mapping curve as the gamma curve and performing global mapping is described in the above embodiment, and will not be described herein.

(4) Method for determining segmentation

In the present embodiment, first, calculationAnd->The image is divided into dark, normal and bright regions.

wherein ,is calculated by the following calculation formula:

；

is calculated by the following calculation formula:

；

At the time of calculationAnd->After the value of->And->Respective corresponding values in the HDR domain->And->。/>And (3) withThe value can be calculated by the following formula:

；

wherein ,and->Two constants. />And->Can be obtained by simulation, but is not limited thereto.

In the present embodiment of the present invention, in the present embodiment,0.15%>0.4. When->0.15%>At 0.4, an HDR image of high visual quality can be obtained.

Next, coordinates of the two end points D1, D2 are acquired.

Next, according toAnd coordinates of the two endpoints D1 and D2, and calculating to obtain and />Is a value of (2).

(5) Generating piecewise linear curve and performing global mapping

In the present embodiment, according to and />And (3) generating a piecewise linear curve to obtain an equation of the global mapping curve C2.

Then, the second luminance value is converted into a third luminance value according to the piecewise linear curve, i.e. according to the equation of the global mapping curve C2. The specific method for carrying out global mapping comprises the following steps: firstly, according to an equation C2 of a global mapping curve, two endpoints D1 and D2 and two demarcation points D3 and D4 are determined, then, according to the two endpoints D1 and D2 and the two demarcation points D3 and D4, an original image is divided into a dark area, a normal area and a bright area, and then, for each area of the original image, a second brightness value is converted into a corresponding third brightness value according to the equation of the global mapping curve. The second luminance value is converted into a corresponding third luminance value according to a linear curve C21 for a dark area of the original image, is converted into a corresponding third luminance value according to a linear curve C22 for a normal area of the original image, and is converted into a corresponding third luminance value according to a linear curve C23 for a bright area of the original image.

In this embodiment, a piecewise linear curve is used, so that a good balance can be achieved between the overall brightness and contrast of the generated HDR image.

(6) Color adjustment

This step is the same as step 105 and will not be described again here.

(7) Color gamut conversion

This step is the same as step 106 and will not be described again here.

(8) Selecting HDR image format

As described above, the target image has two formats: HDR (high definition digital television) ₁₀ and HLG₁₀ 。

In the present embodiment, the designation format of the target image selected according to the identification of the image format input by the user is HLG ₁₀ 。HLG ₁₀ The format image may be converted using a mixed log gamma (HLG) function.

The hybrid logarithmic gamma function is a transfer function for High Dynamic Range (HDR) image and video encoding and decoding, aimed at achieving efficient transmission and storage of HDR video. The HLG function maps the input value of a pixel (also referred to as a "linear light" value) to the output value (also referred to as a "perceived light" value) that should be displayed on a display device. The function has two parts: for low input values, the function is a simple power function. This part of the function is used to encode and decode Standard Dynamic Range (SDR) images. For high input values, the function is a logarithmic function that is chosen to preserve the overall dynamic range of the image while compressing the high light values. The main advantage of the HLG function is that it is able to generate images that can be viewed on SDR and HDR displays. The image will have the same overall appearance on both types of displays, but the HDR display will be able to display the full dynamic range of the image, while the SDR display will display a tone mapped version of the image with a lower dynamic range. This allows the industry to transition more smoothly to HDR. Wherein, the HLG function is as follows:

；

wherein , is the luminance value of the color channel of the image in the light domain, is>Is its corresponding value in the HLG domain. a. b and c are equal to 0.17, 0.28 and 0.55, respectively.

(9) Reverse PQ conversion

In this embodiment, when HLG transformation is performed using the HLG function, the luminance value of the luminance channel of the image in the optical domain is used, and PQ transformation is performed before HLG transformation is performed, so that it is necessary to perform reverse PQ transformation first, and convert the generated HDR PQ value (third luminance value) into the optical domain, to obtain the corresponding fourth luminance value.

The inverse PQ transform is calculated as follows:

；

wherein ,for the third brightness value, +.>For a fourth luminance value for which the third luminance value corresponds in the light domain,/for the fourth luminance value>Are constant and set as 0.8359, 18.8515, 18.6875, 0.1593 and 78.8437, respectively.

(10) HLG transformation

In the present embodiment, the fourth luminance value in the optical domain is converted to the HLG domain using the HLG function, generating HLG ₁₀ And (5) formatting the image for output to a display for display.

In this embodiment, all the regions with different brightness levels in the SDR image can be subjected to brightness conversion, instead of performing local brightness conversion only on the regions with certain specific brightness levels, so that the whole SDR image can be converted into an HDR image, and high-quality visual quality output can be provided for the SDR image with different brightness conditions.

In this embodiment, the itto works in the perceptual domain to take into account the sensitivity of the human eye to luminance variations in different luminance areas. This helps the itto treat each region differently depending on how our eyes perceive them.

In this embodiment, in order to preserve the overall impression of the original SDR image, the itto adaptively divides the SDR image into dark, normal and bright regions using a novel segmentation method. This not only enables the overall brightness of each region to be maintained, but also enables the itto to handle all types of SDR images, a problem that most of the itto in the related art cannot solve.

The iTMO provided by the present application is content-adaptive, and can generate different outputs according to the conversion target, without any user intervention to set the value of the global mapping curve or to optimize the algorithm for a specific type of content. This is accomplished by implementing a gamma curve and a piecewise linear curve.

Another exemplary embodiment of the present application also provides an image processing method. As shown in fig. 6, unlike the above-described embodiment, in the present embodiment,and->The value of (2) can be obtained as follows.

Step 601, obtaining a histogram of the original image in the PQ domain according to the second luminance value.

In step 602, the original image is adaptively segmented according to the histogram, and the original image is divided into a dark area, a normal area and a bright area.

Step 603, obtaining according to the demarcation point of the dark area and the normal areaIs based on the demarcation point of the normal and bright areas>Is a value of (2).

In this embodiment, by using the histogram, the brightness distribution of the pixels in each frame of image can be better understood, so as to accurately partition the image, and further improve the calculationAnd->And ultimately helps achieve a higher visual quality output.

Another exemplary embodiment of the present application also provides an image processing method. As shown in fig. 7, unlike the above-described embodiment, in the present embodiment,and->The value of (2) can be obtained as follows.

Step 701, obtaining a histogram of the original image in the PQ domain according to the second luminance value.

Step 702, determining the respective pixel numbers in the dark area, the normal area and the bright area according to the histogram.

In step 703, the weights of the dark, normal and bright regions are determined according to the respective numbers of pixels in the dark, normal and bright regions.

In the present embodiment, the greater the number of pixels in one area, the greater the weight. For example, the weight of each region is proportional to the number of pixels in the region, but is not limited thereto.

Step 704, performing simulation according to the weights of the dark area, the normal area and the bright area to determineAnd->Is a value of (2).

In this embodiment, the number of pixels in each region in each frame of image can be better known by using the histogram, so as to assign an accurate weight to each region, so that the constructed global mapping curve is more accurate, and further higher visual quality output is realized.

To sum up, the technical scheme provided by the application has the following advantages: preserving an artistic impression of the input SDR content; generating a high visual quality output for various brightness conditions; the scintillation artifact is avoided by proposing a global mapping curve; a full-automatic method without manual intervention parameter setting; a display adaptation method capable of generating contents for the maximum brightness of all levels; color accuracy is maintained during the conversion process; hardware friendly implementation of real-time applications; the current image processing method can also be extended to non-real time applications to achieve better visual quality output by taking into account the histogram features of each frame.

Fig. 8 is a block diagram of an image processing apparatus according to an exemplary embodiment. As shown in fig. 8, in the present embodiment, the image processing apparatus includes:

A first obtaining module 81 configured to convert an original image input in a light domain from an RGB color space to an HSL color space and obtain a first luminance value of a luminance channel; the original image is a standard dynamic range SDR image;

a second acquisition module 82 configured to perform a perceptually quantized PQ transform on the first luminance value to obtain a corresponding second luminance value in the PQ domain;

a first conversion module 83 configured to convert the second luminance value into a third luminance value according to an equation of a specified global mapping curve, resulting in a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image;

a second conversion module 84 configured to convert the first intermediate image from the HSL color space to the RGB color space, resulting in a second intermediate image;

a third conversion module 85 configured to convert from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image, resulting in a third intermediate image; the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of a target image, and the target image is a high dynamic range HDR image;

A generating module 86 is configured to generate the target image according to the specified format of the target image and the third intermediate image.

In one embodiment, as shown in fig. 9, the image processing apparatus further includes:

a color adjustment module 87 configured to perform color adjustment on the second intermediate image according to a specified color adjustment equation for each color channel, keeping the hue and the brightness of the color substantially unchanged, resulting in a fourth intermediate image;

the third conversion module 85 is further configured to convert from an original color gamut to a target color gamut according to the color coordinates of the fourth intermediate image and the color conversion matrix after obtaining the fourth intermediate image, to obtain the third intermediate image.

The embodiment of the application also provides electronic equipment, which comprises a processor and a memory; the memory is used for storing a computer program executable by the processor; the processor is configured to execute the computer program in the memory to implement the image processing method according to any one of the foregoing embodiments.

Embodiments of the present application also provide a computer readable storage medium, which when executed by a processor, can implement the image processing method according to any of the above embodiments.

The specific manner in which the processor performs the operations in the apparatus of the above embodiments has been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 10 is a block diagram of an electronic device, according to an example embodiment. For example, the electronic device 900 may be provided as a server. Referring to FIG. 10, device 900 includes a processing component 922 that further includes one or more processors, and memory resources represented by memory 932, for storing instructions, such as application programs, executable by processing component 922. The application programs stored in memory 932 may include one or more modules that each correspond to a set of instructions. Further, processing component 922 is configured to execute instructions to perform the above-described methods for image processing.

The device 900 may also include a power component 926 configured to perform power management for the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input output (I/O) interface 958. The device 900 may operate based on an operating system stored in memory 932, such as Windows Server, macOS XTM, unixTM, linuxTM, freeBSDTM, or the like.

In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as memory 932, that includes instructions executable by processing component 922 of device 900 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In the present invention, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "plurality" refers to two or more, unless explicitly defined otherwise.

The embodiments are described above in order to facilitate the understanding and application of the present application by those of ordinary skill in the art. It will be apparent to those skilled in the art that various modifications can be made to these embodiments and that the general principles described herein may be applied to other embodiments without the use of inventive faculty. Accordingly, the present application is not limited to the embodiments herein, and those skilled in the art, based on the present disclosure, may make improvements and modifications without departing from the scope and spirit of the present application.

Claims

1. An image processing method, comprising:

after the fourth intermediate image is obtained, converting from an original color gamut to a target color gamut according to the color coordinates and the color conversion matrix of the fourth intermediate image to obtain a third intermediate image; the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of a target image, and the target image is a high dynamic range HDR image;

Generating the target image according to the appointed format of the target image and the third intermediate image;

wherein the color adjustment equation is as follows:

；

wherein ,for the color saturation of any color channel of said second intermediate image in the PQ domain,/L>For the color saturation of any color channel of said fourth intermediate image in the PQ domain,L _SDR,PQ for the value of the second luminance value,L _HDR,PQ for the value of the third luminance value,αandβare two constants responsible for controlling the hue and saturation of the color.

2. The image processing method according to claim 1, wherein the global map curve is a piecewise linear curve, and includes two end points, two demarcation points and three linear curves, the three linear curves forming a continuous curve, the two end points and the two demarcation points dividing the original image into a dark area, a normal area and a bright area; the three sections of linear curves are in one-to-one correspondence with the dark areas, the normal areas and the bright areas;

3. The image processing method of claim 2, wherein the equation of the global map curve is as follows:

；

wherein ,L _SDR,PQ for the value of the second luminance value,L _HDR,PQ for the value of the third luminance value,and->The slopes of the three linear curves, +.>And->The intercept of three linear curves, respectively +.>And->Is the abscissa of the two demarcation points,SDR _min,PQ for the minimum value of the second luminance value,SDR _max,PQ is the maximum value of the second luminance value.

4. The image processing method according to claim 3, wherein,is calculated by the following calculation formula:

；

is calculated by the following calculation formula:

；

5. The image processing method according to claim 3, wherein,and->The value of (2) can be obtained by the following method:

obtaining according to the demarcation point of the dark area and the normal areaIs based on the demarcation point of the normal area and the bright area>Is a value of (2).

6. The image processing method according to claim 4 or 5, wherein, and The calculation method of (2) is as follows:

Acquiring coordinates of the two endpoints;

7. The image processing method according to claim 6, wherein,and->The value of (2) is calculated by the following equation:

；

wherein ,and->Is a function of two constants, namely,HDR _min,PQ for the minimum luminance value of the target HDR display in the PQ domain,HDR _max,PQ maximum luminance value for target HDR display in PQ domain。

8. The image processing method according to claim 7, wherein,0.15%>0.4.

9. The image processing method according to claim 7, wherein,and->The value of (2) can be obtained by the following method:

10. The image processing method of claim 1, wherein the global mapping curve is a gamma curve, and an equation of the gamma curve is as follows:

；

wherein ,L _SDR,PQ for the value of the second luminance value,L _HDR,PQ for the value of the third luminance value,c ₁ and (3) withc ₂ Respectively constant;

the formula of (2) is as follows:

；

the formula of (2) is as follows:

；

wherein ,SDR _min,PQ for the minimum value of the second luminance value,SDR _max,PQ for the maximum value of the second luminance value,HDR _min,PQ for the minimum luminance value of the target HDR display in the PQ domain,HDR _max,PQ is the maximum luminance value of the target HDR display in the PQ domain.

11. An image processing apparatus, comprising:

the third conversion module is configured to convert the original color gamut into the target color gamut according to the color coordinates and the color conversion matrix of the fourth intermediate image, so as to obtain a third intermediate image; the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of a target image, and the target image is a high dynamic range HDR image;

A generation module configured to generate the target image according to a specified format of the target image and the third intermediate image;

wherein the color adjustment equation is as follows:

；

wherein ,for the color saturation of any color channel of said second intermediate image in the PQ domain,/L>For the color saturation of any color channel of said fourth intermediate image in the PQ domain,L _SDR,PQ for the value of the second luminance value,L _HDR,PQ for the value of the third luminance value,αandβto take charge of controlling hue and saturation of colourIs a constant of (a) and (b).

12. An electronic device comprising a memory and a processor, the memory for storing a computer program executable by the processor; the processor is configured to execute a computer program in the memory to implement the method of any one of claims 1-10.

13. A computer readable storage medium having stored thereon a computer program, characterized in that the method according to any of claims 1-10 is enabled when the executable computer program in the storage medium is executed by a processor.