CN117994179A - Image processing method, apparatus, electronic device, and computer-readable storage medium - Google Patents

Abstract

The application relates to an image processing method, an image processing device, an electronic device and a computer scale storage medium. The method comprises the following steps: tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth; performing gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth; compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth. By adopting the method, the accuracy of image processing can be improved, and the target image with more natural color transition between different areas can be obtained.

Description

Image processing method, apparatus, electronic device, and computer-readable storage medium

Technical Field

The present application relates to the field of image technology, and in particular, to an image processing method, an image processing device, an electronic device, and a computer readable storage medium.

Background

In the image processing flow, it is generally required to perform processes such as tone mapping, black level compensation (black level compensation), lens correction (LENS SHADING correction), bad pixel correction (bad pixel correction), color interpolation (demosaic), bayer noise removal, white balance correction, color correction (color correction), and the like, and then transmit the processed result to a CPU (central processing unit ) through an I/O interface, so as to output a final image.

However, in the conventional image processing method, color transition between different areas in the processed image is not natural enough, and there is a problem that accuracy of image processing is low.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, electronic equipment, a computer readable storage medium and a computer program product, which can improve the accuracy of image processing and obtain a target image with more natural color transition between different areas.

In a first aspect, the present application provides an image processing method. The method comprises the following steps:

Tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

performing gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

In a second aspect, the present application also provides an image processing apparatus. The device comprises:

The tone mapping module is used for tone mapping the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

the gamma module is used for carrying out gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

the compression module is used for compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

In a third aspect, the application further provides electronic equipment. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the following steps:

Tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

performing gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

Tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

performing gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:

Tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

performing gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

The image processing method, the image processing device, the electronic equipment, the computer readable storage medium and the computer program product are used for performing tone mapping on the first image to obtain a second image, and performing gamma processing on the second image to obtain a third image; the bit depth of the first image, the second image and the third image is higher than the preset bit depth, namely, the images with higher bit depth are processed in the tone mapping and gamma processing process, the brightness of the images is improved, then the compression processing is carried out, the problem that the color transition of different areas of the images is unnatural due to the fact that the image information loss is caused by the compression processing is carried out on the images while the tone mapping and gamma processing is avoided, the target images with more natural color transition of different areas can be obtained, and the accuracy of the image processing is improved. Meanwhile, the electronic equipment compresses the third image corresponding to the higher bit depth, so that the storage space of the target image can be reduced.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of an image processing method in one embodiment;

FIG. 2 is a graph of a first gamma parameter according to one embodiment;

FIG. 3 is a flow diagram of gamma processing and compression processing in one embodiment;

FIG. 4 is a schematic diagram of a triangle scale relationship in one embodiment;

FIG. 5 is an effect diagram of global tone mapping and local tone mapping in one embodiment;

FIG. 6 is a flow diagram of global tone mapping in one embodiment;

FIG. 7 is a flow diagram of local tone mapping in one embodiment;

FIG. 8 is a flow diagram of image processing in one embodiment;

FIG. 9 is a block diagram showing the structure of an image processing apparatus in one embodiment;

fig. 10 is an internal structural diagram of an electronic device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

In one embodiment, as shown in fig. 1, an image processing method is provided, where the method is applied to an electronic device for illustration, and the electronic device may be a terminal or a server; it will be appreciated that the method may also be applied to a system comprising a terminal and a server and implemented by interaction of the terminal and the server. The terminal can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things equipment and portable wearable equipment, and the internet of things equipment can be smart speakers, smart televisions, smart air conditioners, smart vehicle-mounted equipment, smart automobiles and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.

In this embodiment, the image processing method includes the steps of:

step S102, tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the second image are both higher than a predetermined bit depth.

Tone Mapping (Tone Mapping), a computer graphics technique that approximates high dynamic range images on a limited dynamic range medium. Essentially, tone mapping is to solve the problem of performing a large contrast attenuation to transform the scene brightness into a displayable range, while keeping the image details and colors, etc. very important information for representing the original scene.

Wherein the tone mapping may comprise at least one of global tone mapping and local tone mapping.

The bit depth refers to the bit depth that the computer actually needs to represent for each pixel when recording the color of the digital image. The computer is capable of displaying colors, and a unit of a number called "Bit" (Bit) is used to record data of the color represented. When these data are recorded in a computer in a certain arrangement, a computer file of a digital image is formed. The "Bit" (Bit) is the smallest unit in computer memory that is used to record the value of each pixel color. The more colorful the image, the more "bits". This number of bits used in the computer for each pixel is the "bit depth".

Illustratively, the Bit depth of the first image and the second image may be 16Bit, 24Bit, 32Bit, or the like.

The preset bit depth may be set as desired. For example, the preset Bit depth may be 10Bit, and the Bit depths of the first image and the second image are 16Bit, and the Bit depths of the first image and the second image are higher than the preset Bit depth, which means that the first image and the second image are both high-order images. Wherein an image with a Bit depth of 10Bit refers to one of the ranges of values of each pixel in the image that may be 0 to 1023.

Optionally, the electronic device sequentially performs global tone mapping and local tone mapping on the first image to obtain a second image.

Step S104, gamma processing is carried out on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth.

Illustratively, the Bit depth of the third image may be 16Bit, 24Bit, 32Bit, or the like.

The gamma processing is mainly used for correcting images, correcting images with over-high gray level or under-low gray level, and enhancing contrast.

Optionally, the electronic device performs gamma processing on the second image in the gamma module to obtain a third image; the bit depth of the third image is higher than the preset bit depth, and the image brightness of the third image is higher than the image brightness of the second image.

It will be appreciated that the bit depth of the first, second and third images are the same, i.e. the same bit depth image is used for processing both during tone mapping and during gamma processing.

For example, the preset Bit depth is 10Bit, the Bit depth of the third image is 16Bit, and the electronic device may perform gamma processing on the second image with the Bit depth of 16Bit by using the following formulas (1), (2), (3) to obtain a third image with 16 Bit.

Routput[i]＝Gamma16Bit[Rinput[i]] (1)

Goutput[i]＝Gamma16Bit[Ginput[i]] (2)

Boutput[i]＝Gamma16Bit[Binput[i]] (3)

Step 106, compressing the third image to obtain a target image; the bit depth of the target image is less than or equal to the predetermined bit depth.

For example, the preset Bit depth is 10Bit, the Bit depth of the third image is 16Bit, and the electronic device may compress the third image with the Bit depth of 16Bit by using the following formulas (4), (5), (6) to obtain a target image of 10 Bit.

Routput[i]＝Gamma16Bit[Rinput[i]]＞＞6 (4)

Goutput[i]＝Gamma16Bit[Ginput[i]]＞＞6 (5)

Boutput[i]＝Gamma16Bit[Binput[i]]＞＞6 (6)

Wherein Routput [ i ] is the pixel value of R (Red ) channel in the target image, gamma16Bit [ Rinput [ i ] is the pixel value of R channel in the third image with Bit depth of 16Bit after Gamma processing, goutput [ i ] is the pixel value of G (Green ) channel in the target image, gamma16Bit [ Ginput [ i ] is the pixel value of G channel in the third image with Bit depth of 16Bit after Gamma processing, boutput [ i ] is the pixel value of B (Blue ) channel in the target image, gamma16Bit [ Binput [ i ] is the pixel value of B channel in the third image with Bit depth of 16Bit after Gamma processing; 6 in Gamma16Bit [ Rinput [ i ] ] > 6 is an integer shift operation, where > 6 represents shifting 6 bits right from 16Bit to 10Bit, i.e., equal to dividing by 64, gamma16Bit [ Rinput [ i ] > 6 is equal to Gamma16Bit [ Rinput [ i ] ] divided by 64.

It can be appreciated that in the conventional technology, the image is compressed during the tone mapping or gamma processing stage, and darker areas of the image easily cause the problem of quantization of the color level, i.e. color transition between different areas in the processed image is not natural enough.

The image processing method comprises the steps of performing tone mapping on a first image to obtain a second image, and performing gamma processing on the second image to obtain a third image; the bit depth of the first image, the second image and the third image is higher than the preset bit depth, namely, the images with higher bit depth are processed in the tone mapping and gamma processing process, the brightness of the images is improved, then the compression processing is carried out, the problem that the color transition of different areas of the images is unnatural due to the fact that the image information loss is caused by the compression processing of the images in the tone mapping and gamma processing process is avoided, the quantization information loss can be reduced, the target images with more natural color transition of different areas are obtained, and the accuracy of the image processing is improved. Meanwhile, the electronic equipment compresses the third image corresponding to the higher bit depth, so that the storage space of the target image can be reduced.

In one embodiment, the method further comprises: acquiring a first gamma parameter corresponding to a preset first bit depth in an image processing flow; the first gamma parameter is used for processing image data with a first bit depth, and the first bit depth is lower than or equal to a preset bit depth; interpolation is carried out on the first gamma parameter, and a second gamma parameter corresponding to the second bit depth is obtained; the second bit depth is higher than the preset bit depth; performing gamma processing on the second image to obtain a third image, including: and performing gamma processing on the second image by adopting the second gamma parameter to obtain a third image.

The gamma parameter is a parameter used in gamma processing. The first gamma parameter is preset in the image processing flow and is used for processing the image data of the first bit depth, and the second gamma parameter is preset in the image processing flow and is used for processing the image data of the second bit depth. By way of example, the first Bit depth may be 10Bit, 8Bit, etc., and the second Bit depth may be 16Bit, 24Bit, 32Bit, etc.

The first gamma parameter in the image processing flow is a discrete gamma array that inputs elements within 10Bit range and outputs elements within 10Bit range. Fig. 2 is a graph of a first gamma parameter, where the first gamma parameter is output=gamma [ i ], i is a first input element (ranging from 0 to 1023), and output is a first output element (ranging from 0 to 1023). For example ,0＝gamma[0],2＝gamma[1],4＝gamma[2],7＝gamma[3],10＝gamma[4],15＝gamma[5],.....1023＝gamma[1022],1023＝gamma[1023].

It is understood that, in the image Processing process (ISP, image Signal Processing), the bit depth of the image data is typically compressed to the first bit depth in the tone mapping process, so that the first gamma parameter corresponding to the first bit depth is preset for performing gamma Processing on the image data of the first bit depth.

Optionally, the electronic device acquires a first gamma parameter corresponding to a preset first bit depth in the image processing flow; interpolation is carried out on the first gamma parameter, and a second gamma parameter corresponding to the second bit depth is obtained; wherein the second bit depth is a bit depth of the second image; and performing gamma processing on the second image by adopting the second gamma parameter to obtain a third image. It will be appreciated that the first image and the third image may both be of the second bit depth.

The electronic device obtains a first gamma parameter of 10Bit preset in an image processing flow, interpolates the first gamma parameter of 10Bit, and obtains a second gamma parameter of 16 Bit; and performing gamma processing on the second image of the 16Bit by adopting the second gamma parameter of the 16Bit to obtain a third image of the 16 Bit. Wherein the Bit depth 10Bit is less than or equal to the predetermined Bit depth and the Bit depth 16Bit is greater than the predetermined Bit depth.

In this embodiment, the electronic device obtains a first gamma parameter corresponding to a first bit depth preset in an image processing flow, interpolates the first gamma parameter to obtain a second gamma parameter corresponding to a second bit depth, and may perform gamma processing on a second image with the same higher bit depth by using the second gamma parameter with the higher bit depth to obtain a third image, thereby improving accuracy of image processing.

In one embodiment, as shown in fig. 3, the electronic device performs steps S302 to S310:

Step S302, a second image with a Bit depth of 16 Bit; step S304, a first gamma parameter corresponding to 10Bit is obtained; step S306, the first gamma parameter is interpolated to 16Bit to obtain a second gamma parameter; step S308, performing gamma processing on the second image by using the second gamma parameter to obtain a third image; in step S310, the third image is compressed to obtain a target image with a Bit depth of 10 Bit.

In one embodiment, the first bit depth range corresponding to the first gamma parameter includes at least two pairs of first elements, each pair of first elements including a first input element and a first output element; interpolation is carried out on the first gamma parameter to obtain a second gamma parameter corresponding to the second bit depth, and the interpolation comprises the following steps: multiplying the first input element and the first output element of each pair of first elements by a second bit depth to obtain each pair of second elements including a second input element and a second output element; acquiring interpolation input elements except for a second input element in a second bit depth range, and determining an interpolation output element of the interpolation input element based on a second element pair adjacent to the interpolation input element for each interpolation input element; the second bit depth range is larger than the first bit depth range, and the interpolation input element and the interpolation output element form a pair of interpolation element pairs; and forming a second gamma parameter corresponding to the second bit depth based on each pair of second element pairs and each pair of interpolation element pairs.

The bit depth range is a range of pixel values corresponding to the bit depth. The first bit depth range is a pixel value range corresponding to the first bit depth, and the second bit depth range is a pixel value range corresponding to the second bit depth. Illustratively, the first Bit depth is 10Bit, then the first Bit depth range is (0-1023); the second Bit depth is 16Bit and the second Bit depth range is (0-65535).

The electronic device multiplies the first input element and the first output element of each pair of first elements by a second bit depth to obtain each pair of second element pairs including a second input element and a second output element. Taking a first Bit depth of 10Bit and a second Bit depth of 16Bit as an example, the first Bit depth range corresponding to the first gamma parameter is (0-1023), and the second Bit depth range corresponding to the second gamma parameter is (0-65535); the electronic device multiplies the first input element and the first output element of each pair of first elements in a first Bit depth range (0-1023) corresponding to a first gamma parameter output=gamma [ i ] by a second Bit depth to obtain each pair of second element pairs including a second input element and a second output element in a second gamma parameter output×16=gamma 16Bit [ i×16].

Exemplary, the second element pair may be ：0＝gamma16Bit[0],2x16＝gamma16Bit[1x16],4x16＝gamma16Bit[2x16],7x16＝gamma[3x16],10x16＝gamma16Bit[4x16],15x16＝gamma16Bit[5x16],……1023x16＝gamma16Bit[1022x16],1023x16＝gamma16Bit[1023x16].

It can be understood that the interval size of the second bit depth range is greater than the interval size of the first bit depth range, so after the electronic device acquires the second element pair, there are some values in the second bit depth range where the corresponding data is not acquired, and therefore interpolation processing is required to obtain each data in the second bit depth range to form the second gamma parameter.

Optionally, for each interpolation input element, the electronic device determines at least one second element pair adjacent to the interpolation input element, and determines an interpolation output element of the interpolation input element based on the at least one second element pair.

In an alternative embodiment, the electronic device may randomly determine a second output element from the at least one second element pair as the interpolated output element of the interpolated input element.

In another alternative embodiment, the electronic device may average the 2 second output elements of the adjacent 2 second element pairs, and take the average as the interpolated output element of the interpolated input element.

In another alternative embodiment, the electronic device determines 2 adjacent pairs of second elements, and determines the interpolation output element of the pair of interpolation elements to be generated based on a triangular proportional relationship between the pair of interpolation elements to be generated and the 2 adjacent pairs of second elements.

Referring to fig. 4, the triangle proportional relationship is as follows formula (7) (8):

where output is the interpolation output element of the interpolation element pair to be generated, i is the interpolation input element, output0 is the second output element of the second element pair of the preamble, i0 is the second input element of the second element pair of the preamble, output1 is the second output element of the second element pair of the postamble, and i1 is the second input element of the second element pair of the postamble.

For example, for the interpolation input element 70, the adjacent 2 second elements are 160=gamma 16Bit [64], 240=gamma 16Bit [80], and the interpolation output element corresponding to the interpolation input element 70 can be obtained by adopting the triangular proportional relationship:

In this embodiment, the electronic device multiplies the first input element and the first output element in each pair of first elements by the second bit depth to obtain each pair of second elements including the second input element and the second output element, and interpolates to obtain the interpolation element pair, so that a second gamma parameter corresponding to the second bit depth can be accurately formed, gamma processing can be accurately performed on the second image to obtain the third image, and accuracy of image processing is improved.

In one embodiment, the method further comprises: acquiring at least two exposure images obtained by exposure with different exposure parameters, and fusing the at least two exposure images to obtain a first image; the bit depth of the exposure image is lower than or equal to the preset bit depth; tone mapping the first image to obtain a second image, including: performing global tone mapping on the first image to obtain an intermediate image; the bit depth of the intermediate image is higher than a preset bit depth; and carrying out local tone mapping on the intermediate image to obtain a second image.

The exposure parameters may include, but are not limited to, exposure time period, aperture value, sensitivity, and the like. It will be appreciated that the electronic device may be exposed with different exposure parameters to obtain exposure images of different brightness.

Optionally, the electronic device exposes with different exposure parameters to obtain at least two exposure images with different brightness; performing HDR (HIGH DYNAMIC RANGE IMAGING ) fusion on at least two exposure images with different brightness to obtain a first image; global tone mapping (Global Tone Mapping) is performed on the first image to obtain an intermediate image; the intermediate image is locally tone mapped (Local Tone Mapping) to obtain a second image. The first image may be an HDR image and be an RGB color space.

For example, the Bit depth of the exposure images is 10Bit, which is lower than or equal to the preset Bit depth, and the electronic device performs HDR fusion on at least two exposure images with different brightness to obtain 1 first image with Bit depth of 16 Bit.

As shown in fig. 5, after performing global tone mapping on the first image, the electronic device enhances the contrast of the first image to obtain an intermediate image, and then performs local tone mapping on the intermediate image to continuously enhance the contrast of the image to obtain a second image; the brightness of the first image, the intermediate image, and the second image increases in order.

In this embodiment, the electronic device fuses at least two exposure images with lower bit depth to obtain a first image with higher bit depth, global tone mapping is performed on the first image with higher bit depth to obtain an intermediate image with higher bit depth, and local tone mapping is performed on the intermediate image with higher bit depth to obtain a second image with higher bit depth, that is, the images with higher bit depth are processed in the global tone mapping and local tone mapping processes, so that the problem of unnatural color transition in different areas of the image due to image information loss caused by compression processing of the images can be avoided, and the accuracy of image processing is improved.

In other optional embodiments, the electronic device may further perform local tone mapping on the first image, and then perform global tone mapping on the locally tone mapped image to obtain the second image.

In one embodiment, global tone mapping is performed on a first image to obtain an intermediate image, including: converting the first image from RGB color space to gray space, and performing global tone mapping on the first image in the gray space to obtain a global tone mapping image in the gray space; dividing the global tone mapping image of the gray space by the first image of the gray space to obtain a global tone mapping gain; the first image of the RGB color space is processed based on the global tone mapping gain to obtain an intermediate image of the RGB color space.

The electronic device converts the first image from the RGB color space to a gray space using the following equation (9):

Gray[i]＝0.299×R[i]+0.587×B[i]+0.114×G[i] (9)

wherein, R [ i ] is the pixel value of R (Red) channel in the first image of RGB color space, B [ i ] is the pixel value of B channel in the first image of RGB color space, G [ i ] is the pixel value of G channel in the first image of RGB color space, gray [ i ] is the pixel value of the first image of Gray space.

The bit depth of the first image of the RGB color space, the first image of the gray space, the global tone mapped image of the gray space, and the intermediate image of the RGB color space are all higher than a preset bit depth.

The electronic device performs global tone mapping on the first image in the gray space by using the following formula (10) to obtain a global tone mapped image in the gray space:

Where Input [ i ] is the pixel value of the first image in the gray space, output [ i ] is the pixel value of the global tone-mapped image in the gray space, max is the maximum pixel value in the first image in the gray space, min is the minimum pixel value in the first image in the gray space, and 1000 is the global tone-mapped parameter.

Illustratively, the Bit depth of the first image in RGB color space, the first image in gray space, the global tone mapped image in gray space, and the intermediate image in RGB color space are all 16Bit, then the range of Input [ i ] is (0-65535), the range of Output [ i ] is (0-65535), max is typically 65535, and min is typically 0 or a smaller value.

Optionally, the electronic device divides each pixel value in the global tone mapping image in the gray space by a pixel value in a corresponding position of the first image in the gray space to obtain a global tone mapping gain corresponding to each pixel value, and multiplies the global tone mapping gain corresponding to each pixel value by the pixel value in the corresponding position in the first image in the RGB color space to obtain an intermediate image in the RGB color space.

Further, the electronic device may also multiply the global tone mapping gain by R, G, and B pixel values for corresponding locations in the first image of the RGB color space.

In order to maintain data accuracy, the global tone mapping gain may be stored as high-accuracy floating point data.

The electronic device processes the first image of the RGB color space based on the global tone mapping gain using the following formulas (11) (12) (13) to obtain an intermediate image of the RGB color space:

where Input [ i ] is the pixel value of the first image in the gray space, output [ i ] is the pixel value of the global tone-mapped image in the gray space, rinput is the pixel value of the R channel in the first image in the RGB color space, routput [ i ] is the pixel value of the R channel in the intermediate image in the RGB color space, ginput is the pixel value of the G channel in the first image in the RGB color space, goutput [ i ] is the pixel value of the G channel in the intermediate image in the RGB color space, binput is the pixel value of the B channel in the first image in the RGB color space, boutput [ i ] is the pixel value of the B channel in the intermediate image in the RGB color space.

Illustratively, the Bit depth of the first image in RGB color space, the first image in gray space, the global tone mapped image in gray space, and the intermediate image in RGB color space are all 16Bit, then the range of Input [ i ] is (0-65535), and the range of Output [ i ] is (0-65535).

In this embodiment, the electronic device converts the first image from the RGB color space to the gray space, performs global tone mapping on the first image in the gray space to obtain a global tone mapped image in the gray space, divides the global tone mapped image in the gray space by the first image in the gray space, and may obtain a global tone mapped gain, where the gain value acts on the first image in the RGB color space, and does not change the relative relationship between the R channel pixel value, the G channel pixel value, and the B channel pixel value in the first image in the RGB color space, so as to ensure that the intermediate image in the RGB color space obtained after the processing is not color-shifted, and improve the accuracy of image processing. At the same time, the method can save the calculation amount by performing global tone mapping.

In one embodiment, as shown in fig. 6, the electronic device acquires 10Bit exposure images of at least two RGB color spaces and inputs the 10Bit exposure images into an HDR fusion module, and the HDR fusion module fuses the 10Bit exposure images of the at least two RGB color spaces to obtain a 16Bit first image of the RGB color spaces; converting the first image of the 16Bit from an RGB color space to a gray space to obtain a first image of the 16Bit of the gray space, and performing global tone mapping on the first image of the 16Bit of the gray space to obtain a global tone mapping image of the 16Bit of the gray space; dividing the 16Bit global tone mapping image of the gray space by the 16Bit first image of the gray space to obtain global tone mapping gain; and applying the global tone mapping gain to R, G and B in the 16Bit first image of the RGB color space to obtain a 16Bit intermediate image of the RGB color space.

In one embodiment, performing local tone mapping on the intermediate image to obtain a second image includes: converting the intermediate image from RGB color space to gray space, and performing local tone mapping on the intermediate image in the gray space to obtain a local tone mapping image in the gray space; dividing the local tone mapping image of the gray space by the intermediate image of the gray space to obtain local tone mapping gain; and processing the intermediate image of the RGB color space based on the local tone mapping gain to obtain a second image of the RGB color space.

The electronic device converts the intermediate image from the RGB color space to a gray space using the following equation (14):

Gray[i]＝0.299×R[i]+0.587×B[i]+0.114×G[i] (14)

Wherein, R [ i ] is the pixel value of R (Red) channel in the intermediate image of RGB color space, B [ i ] is the pixel value of B channel in the intermediate image of RGB color space, G [ i ] is the pixel value of G channel in the intermediate image of RGB color space, gray [ i ] is the pixel value of the intermediate image of Gray space.

The bit depth of the intermediate image of the RGB color space, the intermediate image of the gray space, the local tone mapped image of the gray space, and the second image of the RGB color space are all higher than the preset bit depth.

Optionally, the electronic device performs local tone mapping on the intermediate image in the gray space by adopting a block histogram equalization mode, that is, a local histogram equalization mode, so as to obtain a local tone mapping image in the gray space.

Optionally, the electronic device divides each pixel value of the local tone mapping image in the gray space by a pixel value in a corresponding position of the intermediate image in the gray space to obtain a local tone mapping gain corresponding to each pixel value, and multiplies the local tone mapping gain corresponding to each pixel value by the pixel value in the corresponding position of the intermediate image in the RGB color space to obtain the second image in the RGB color space.

Further, the electronic device may also multiply the local tone mapping gain by R, G, and B pixel values at corresponding locations in the intermediate image of the RGB color space.

In order to maintain data accuracy, the local tone mapping gain may be stored as high-accuracy floating point data.

The electronic device processes the intermediate image of the RGB color space based on the local tone mapping gain using the following formulas (15) (16) (17) to obtain a second image of the RGB color space:

where Input [ i ] is the pixel value of the intermediate image in the gray space, output [ i ] is the pixel value of the local tone mapped image in the gray space, rinput is the pixel value of the R channel in the intermediate image in the RGB color space, routput [ i ] is the pixel value of the R channel in the second image in the RGB color space, ginput is the pixel value of the G channel in the intermediate image in the RGB color space, goutput [ i ] is the pixel value of the G channel in the second image in the RGB color space, binput is the pixel value of the B channel in the intermediate image in the RGB color space, boutput [ i ] is the pixel value of the B channel in the second image in the RGB color space.

Illustratively, the Bit depth of the intermediate image in RGB color space, the intermediate image in gray space, the local tone mapped image in gray space, and the second image in RGB color space are all 16Bit, then the range of Input [ i ] is (0-65535), and the range of Output [ i ] is (0-65535).

In this embodiment, the electronic device converts the intermediate image from the RGB color space to the gray space, and performs local tone mapping on the intermediate image in the gray space to obtain a local tone mapped image in the gray space, and divides the local tone mapped image in the gray space by the intermediate image in the gray space to obtain a local tone mapped gain; the gain value acts on the intermediate image of the RGB color space, and the relative relation among the R channel pixel value, the G channel pixel value and the B channel pixel value in the intermediate image of the RGB color space is not changed, so that the second image of the RGB color space obtained after processing is ensured to be unbiased, and the accuracy of image processing is improved. At the same time, the local tone mapping is performed in this way, so that the calculation amount can be saved.

In one embodiment, as shown in fig. 7, the electronic device converts the 16Bit intermediate image of the RGB color space from the RGB color space to the gray space to obtain the 16Bit intermediate image of the gray space, and performs local tone mapping on the 16Bit intermediate image of the gray space to obtain the 16Bit local tone mapped image of the gray space; dividing the 16Bit local tone mapping image of the gray space by the 16Bit intermediate image of the gray space to obtain local tone mapping gain; and applying the local tone mapping gain to R, G and B in the 16Bit intermediate image of the RGB color space to obtain a 16Bit second image of the RGB color space.

In one embodiment, performing local tone mapping on an intermediate image of a gray space to obtain a local tone mapped image of the gray space, includes: performing gamma processing on the intermediate image in the gray space to obtain an intermediate image in the gray space after gamma processing; performing local tone mapping on the intermediate image subjected to the gamma processing in the gray space to obtain a local tone mapping image subjected to the gamma processing in the gray space; and performing anti-gamma processing on the local tone mapping image subjected to the gamma processing in the gray space to obtain the local tone mapping image in the gray space.

The bit depth of the intermediate image in the gray space, the intermediate image in the gray space after gamma processing, the local tone mapping image and the local tone mapping image in the gray space are all higher than the preset bit depth.

It can be understood that the electronic device performs gamma processing on the intermediate image in the gray space, so that the image brightness can be improved, that is, the image brightness of the intermediate image in the gray space after gamma processing is higher than the image brightness of the intermediate image in the gray space, and then performs local tone mapping on the intermediate image in the gray space after gamma processing, so that the brightness of the image is improved, then the brightness of the final display is improved, and the local tone mapping is performed after the image brightness is improved, so that the local contrast enhancement effect in the local tone mapping process can be improved.

In one embodiment, compressing the third image to obtain the target image includes: compressing the third image in the gamma module to obtain a target image; or after converting the third image from the RGB color space to the YUV color space, compressing the third image in the YUV color space to obtain the target image.

Optionally, performing gamma processing on the second image in the gamma module to obtain a third image, and performing compression processing on the third image in the gamma module to obtain a target image.

Optionally, after the electronic device converts the third image from the RGB color space to the YUV color space, the following formulas (18), (19), and (20) are used to compress the third image in the YUV color space to obtain the target image.

Y＝(0.299×R+0.587×G+0.114×B)＞＞6 (18)

U＝(-0.169×R-0.331×G+0.5×B)＞＞6 (19)

V＝(0.5×R-0.419×G-0.081×B)＞＞6 (20)

In the YUV color space, Y represents brightness (Luminance or Luma), that is, gray scale values, and U and V represent chromaticity (Chrominance or Chroma), which are used to describe image colors and saturation, and are used to designate the colors of pixels.

In this embodiment, the gamma module performs the compression processing after performing the gamma processing, so that the target image can be obtained more quickly and the problem of quantization color level of the image can be avoided; or after the third image is converted from the RGB color space to the YUV color space, the compression processing is carried out on the third image in the YUV color space, so that the target image can be obtained more accurately.

In one embodiment, as shown in fig. 8, an electronic device acquires at least two exposure images obtained by exposing with different exposure parameters, and performs HDR fusion on the at least two exposure images through an HDR fusion module to obtain a first image of 16 Bit; performing global tone mapping on the first image of the 16Bit to obtain an intermediate image of the 16 Bit; performing local tone mapping on the 16Bit intermediate image to obtain a 16Bit second image; performing gamma processing on the 16Bit second image through a gamma module to obtain a third image, and performing compression processing on the third image through the gamma module to obtain a 10Bit target image; wherein 16Bit is higher than the preset Bit depth and 10Bit is lower than or equal to the preset Bit depth.

In one embodiment, there is also provided an image processing method including the steps of:

a1, acquiring at least two exposure images obtained by exposure with different exposure parameters, and fusing the at least two exposure images to obtain a first image; the bit depth of the exposure image is less than or equal to the predetermined bit depth.

A2, converting the first image from RGB color space to gray space, and performing global tone mapping on the first image in the gray space to obtain a global tone mapping image in the gray space; dividing the global tone mapping image of the gray space by the first image of the gray space to obtain a global tone mapping gain; the first image of the RGB color space is processed based on the global tone mapping gain to obtain an intermediate image of the RGB color space.

Step A3, converting the intermediate image from RGB color space to gray space; performing gamma processing on the intermediate image in the gray space to obtain an intermediate image in the gray space after gamma processing; performing local tone mapping on the intermediate image subjected to the gamma processing in the gray space to obtain a local tone mapping image subjected to the gamma processing in the gray space; performing anti-gamma processing on the local tone mapping image subjected to the gamma processing in the gray space to obtain a local tone mapping image in the gray space; dividing the local tone mapping image of the gray space by the intermediate image of the gray space to obtain local tone mapping gain; processing the intermediate image of the RGB color space based on the local tone mapping gain to obtain a second image of the RGB color space; the bit depth of each first image, each intermediate image and each second image is higher than a predetermined bit depth.

Step A4, obtaining a first gamma parameter corresponding to a preset first bit depth in an image processing flow; the first gamma parameter is used for processing image data with a first bit depth, and the first bit depth is lower than or equal to a preset bit depth; the first bit depth range corresponding to the first gamma parameter comprises at least two pairs of first elements, and each pair of first elements comprises a first input element and a first output element.

Step A5, multiplying the first input element and the first output element of each pair of first elements by a second bit depth to obtain each pair of second elements comprising a second input element and a second output element; acquiring interpolation input elements except for a second input element in a second bit depth range, and determining an interpolation output element of the interpolation input element based on a second element pair adjacent to the interpolation input element for each interpolation input element; the second bit depth range is larger than the first bit depth range, and the interpolation input element and the interpolation output element form a pair of interpolation element pairs; forming a second gamma parameter corresponding to the second bit depth based on each pair of second element pairs and each pair of interpolation element pairs; the second bit depth is higher than the predetermined bit depth.

Step A6, performing gamma processing on the second image by adopting a second gamma parameter to obtain a third image; the bit depth of the third image is higher than the preset bit depth.

Step A7, compressing the third image in the gamma module to obtain a target image; or after converting the third image from the RGB color space to the YUV color space, compressing the third image in the YUV color space to obtain the target image.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides an image processing device for realizing the above-mentioned image processing method. The implementation of the solution provided by the apparatus is similar to the implementation described in the above method, so the specific limitation of one or more embodiments of the image processing apparatus provided below may refer to the limitation of the image processing method hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 9, there is provided an image processing apparatus including: a tone mapping module 902, a gamma module 904, and a compression module 906, wherein:

a tone mapping module 902, configured to tone map the first image to obtain a second image; the bit depth of the first image and the second image are both higher than a predetermined bit depth.

The gamma module 904 is configured to perform gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth.

A compression module 906, configured to perform compression processing on the third image to obtain a target image; the bit depth of the target image is less than or equal to the predetermined bit depth.

The image processing device performs tone mapping on the first image to obtain a second image, and performs gamma processing on the second image to obtain a third image; the bit depth of the first image, the second image and the third image is higher than the preset bit depth, namely, the images with higher bit depth are processed in the tone mapping and gamma processing process, the brightness of the images is improved, then the compression processing is carried out, the problem that the color transition of different areas of the images is unnatural due to the fact that the image information loss is caused by the compression processing is carried out on the images while the tone mapping and gamma processing is avoided, the target images with more natural color transition of different areas can be obtained, and the accuracy of the image processing is improved. Meanwhile, the electronic equipment compresses the third image corresponding to the higher bit depth, so that the storage space of the target image can be reduced.

In one embodiment, the apparatus further comprises an interpolation module; the interpolation module is used for acquiring a first gamma parameter corresponding to a first bit depth preset in an image processing flow; the first gamma parameter is used for processing image data with a first bit depth, and the first bit depth is lower than or equal to a preset bit depth; interpolation is carried out on the first gamma parameter, and a second gamma parameter corresponding to the second bit depth is obtained; the second bit depth is higher than the preset bit depth; the gamma module 904 is further configured to perform gamma processing on the second image by using the second gamma parameter to obtain a third image.

In one embodiment, the first bit depth range corresponding to the first gamma parameter includes at least two pairs of first elements, each pair of first elements including a first input element and a first output element; the interpolation module is further configured to multiply the first input element and the first output element of each pair of first elements by a second bit depth to obtain each pair of second element pairs including the second input element and the second output element; acquiring interpolation input elements except for a second input element in a second bit depth range, and determining an interpolation output element of the interpolation input element based on a second element pair adjacent to the interpolation input element for each interpolation input element; the second bit depth range is larger than the first bit depth range, and the interpolation input element and the interpolation output element form a pair of interpolation element pairs; and forming a second gamma parameter corresponding to the second bit depth based on each pair of second element pairs and each pair of interpolation element pairs.

In one embodiment, the apparatus further comprises a fusion module; the fusion module is used for acquiring at least two exposure images obtained by exposure with different exposure parameters, and fusing the at least two exposure images to obtain a first image; the bit depth of the exposure image is lower than or equal to the preset bit depth; the tone mapping module 902 is further configured to perform global tone mapping on the first image to obtain an intermediate image; the bit depth of the intermediate image is higher than a preset bit depth; and carrying out local tone mapping on the intermediate image to obtain a second image.

In one embodiment, the tone mapping module 902 is further configured to convert the first image from an RGB color space to a gray space, and perform global tone mapping on the first image in the gray space to obtain a global tone mapped image in the gray space; dividing the global tone mapping image of the gray space by the first image of the gray space to obtain a global tone mapping gain; the first image of the RGB color space is processed based on the global tone mapping gain to obtain an intermediate image of the RGB color space.

In one embodiment, the tone mapping module 902 is further configured to convert the intermediate image from the RGB color space to the gray space, and perform local tone mapping on the intermediate image in the gray space to obtain a local tone mapped image in the gray space; dividing the local tone mapping image of the gray space by the intermediate image of the gray space to obtain local tone mapping gain; and processing the intermediate image of the RGB color space based on the local tone mapping gain to obtain a second image of the RGB color space.

In one embodiment, the tone mapping module 902 is further configured to perform gamma processing on the intermediate image in the gray space, so as to obtain an intermediate image in the gray space after gamma processing; performing local tone mapping on the intermediate image subjected to the gamma processing in the gray space to obtain a local tone mapping image subjected to the gamma processing in the gray space; and performing anti-gamma processing on the local tone mapping image subjected to the gamma processing in the gray space to obtain the local tone mapping image in the gray space.

In one embodiment, the compressing module 906 is further configured to compress the third image in the gamma module to obtain a target image; or after converting the third image from the RGB color space to the YUV color space, compressing the third image in the YUV color space to obtain the target image.

The respective modules in the above-described image processing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or independent of a processor in the electronic device, or may be stored in software in a memory in the electronic device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, an electronic device, which may be a terminal, is provided, and an internal structure thereof may be as shown in fig. 10. The electronic device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the electronic device is used to exchange information between the processor and the external device. The communication interface of the electronic device is used for conducting wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an image processing method. The display unit of the electronic device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the electronic equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 10 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the electronic device to which the present inventive arrangements are applied, and that a particular electronic device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

The embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform steps of an image processing method.

Embodiments of the present application also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform an image processing method.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims (11)

1. An image processing method, comprising:

Tone mapping is carried out on the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

performing gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

2. The method according to claim 1, wherein the method further comprises:

Acquiring a first gamma parameter corresponding to a preset first bit depth in an image processing flow; the first gamma parameter is used for processing the image data with the first bit depth, and the first bit depth is lower than or equal to the preset bit depth;

Interpolation is carried out on the first gamma parameter, and a second gamma parameter corresponding to the second bit depth is obtained; the second bit depth is higher than the preset bit depth;

and performing gamma processing on the second image to obtain a third image, wherein the gamma processing comprises the following steps:

And performing gamma processing on the second image by adopting the second gamma parameter to obtain a third image.

3. The method of claim 2, wherein the first bit depth range corresponding to the first gamma parameter includes at least two pairs of first elements, each pair of first elements including a first input element and a first output element; the interpolating the first gamma parameter to obtain a second gamma parameter corresponding to a second bit depth includes:

Multiplying the first input element and the first output element of each pair of the first element pairs by a second bit depth to obtain each pair of second element pairs comprising a second input element and a second output element;

acquiring interpolation input elements except the second input element in a second bit depth range, and determining an interpolation output element of the interpolation input element based on a second element pair adjacent to the interpolation input element for each interpolation input element; the second bit depth range is larger than the first bit depth range, and the interpolation input element and the interpolation output element form a pair of interpolation element pairs;

And forming a second gamma parameter corresponding to the second bit depth based on each pair of second element pairs and each pair of interpolation element pairs.

4. The method according to claim 1, wherein the method further comprises:

acquiring at least two exposure images obtained by exposure with different exposure parameters, and fusing the at least two exposure images to obtain a first image; the bit depth of the exposure image is lower than or equal to a preset bit depth;

the tone mapping of the first image to obtain a second image includes:

Performing global tone mapping on the first image to obtain an intermediate image; the bit depth of the intermediate image is higher than a preset bit depth;

And carrying out local tone mapping on the intermediate image to obtain a second image.

5. The method of claim 4, wherein said globally tone mapping the first image to obtain an intermediate image comprises:

Converting the first image from RGB color space to gray space, and performing global tone mapping on the first image in the gray space to obtain a global tone mapping image in the gray space;

dividing the global tone mapping image of the gray space by the first image of the gray space to obtain a global tone mapping gain;

and processing the first image of the RGB color space based on the global tone mapping gain to obtain an intermediate image of the RGB color space.

6. The method of claim 4, wherein said locally tone mapping said intermediate image to obtain a second image comprises:

Converting the intermediate image from RGB color space to gray space, and performing local tone mapping on the intermediate image in the gray space to obtain a local tone mapping image in the gray space;

dividing the local tone mapping image of the gray space by the intermediate image of the gray space to obtain local tone mapping gain;

And processing the intermediate image of the RGB color space based on the local tone mapping gain to obtain a second image of the RGB color space.

7. The method of claim 6, wherein said locally tone mapping said intermediate image in gray space to obtain a locally tone mapped image in gray space, comprising:

Performing gamma processing on the intermediate image in the gray space to obtain an intermediate image in the gray space after gamma processing;

Performing local tone mapping on the intermediate image subjected to the gamma processing in the gray space to obtain a local tone mapping image subjected to the gamma processing in the gray space;

And performing anti-gamma processing on the local tone mapping image subjected to gamma processing in the gray space to obtain the local tone mapping image in the gray space.

8. The method of claim 1, wherein compressing the third image to obtain a target image comprises:

Compressing the third image in a gamma module to obtain a target image; or alternatively

And converting the third image from the RGB color space to the YUV color space, and then compressing the third image in the YUV color space to obtain a target image.

9. An image processing apparatus, comprising:

The tone mapping module is used for tone mapping the first image to obtain a second image; the bit depth of the first image and the bit depth of the second image are higher than a preset bit depth;

the gamma module is used for carrying out gamma processing on the second image to obtain a third image; the bit depth of the third image is higher than the preset bit depth;

the compression module is used for compressing the third image to obtain a target image; the bit depth of the target image is lower than or equal to the preset bit depth.

10. An electronic device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to perform the steps of the image processing method according to any of claims 1 to 8.

11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 8.