CN116681598A

CN116681598A - Image processing method, chip, electronic device, and computer-readable storage medium

Info

Publication number: CN116681598A
Application number: CN202210168885.0A
Authority: CN
Inventors: 杨浩
Original assignee: Shenzhen Goodix Technology Co Ltd
Current assignee: Shenzhen Goodix Technology Co Ltd
Priority date: 2022-02-23
Filing date: 2022-02-23
Publication date: 2023-09-01

Abstract

The embodiment of the application relates to the technical field of image processing, and discloses an image processing method, a chip, electronic equipment and a computer readable storage medium. The image processing method comprises the following steps: acquiring a tone mapping curve corresponding to a high dynamic range HDR image to be processed; selecting a control point in a coordinate system where a tone mapping curve is located according to the dynamic range of the HDR image; determining a first ordinate corresponding to the abscissa of the control point on the tone mapping curve; if the first ordinate is not in the preset range, adjusting the tone mapping curve so that the second ordinate corresponding to the abscissa of the control point on the adjusted tone mapping curve is in the preset range; according to the adjusted tone mapping curve, a low dynamic range LDR image corresponding to the HDR image is obtained, so that the LDR image obtained after compression can be prevented from being distorted while the HDR image is compressed into the LDR image, and the LDR image obtained after compression accords with the perception of human eyes.

Description

Image processing method, chip, electronic device, and computer-readable storage medium

Technical Field

Embodiments of the present application relate to the field of image processing technologies, and in particular, to an image processing method, a chip, an electronic device, and a computer readable storage medium.

Background

Most display devices currently cannot display or process high dynamic range (High Dynamic Range, HDR) images at high speed, limited by physical hardware and economic costs. Thus, HDR is typically compressed into low dynamic range (Low Dynamic Range, LDR) images by tone mapping techniques to accommodate the display capabilities and processing capabilities of most display devices. However, in the conventional tone mapping technology, although an HDR image can be compressed into an LDR image, the LDR image obtained after compression is easily distorted, and does not conform to the perception of human eyes.

Disclosure of Invention

An object of an embodiment of the present application is to provide an image processing method, a chip, an electronic device, and a computer readable storage medium, so that an LDR image obtained after compression can be prevented from being distorted while an HDR image is compressed into an LDR image, so that the LDR image obtained after compression conforms to the perception of human eyes.

In a first aspect, an embodiment of the present application provides an image processing method, including: acquiring a tone mapping curve corresponding to a high dynamic range HDR image to be processed; selecting a control point in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image; determining a first ordinate of the abscissa of the control point corresponding to the tone mapping curve; if the first ordinate is not in the preset range, adjusting the tone mapping curve so that a second ordinate corresponding to the abscissa of the control point on the adjusted tone mapping curve is in the preset range; the preset range is a pixel value range conforming to the perception of human eyes; and obtaining a low dynamic range LDR image corresponding to the HDR image according to the adjusted tone mapping curve.

As a possible implementation, the tone mapping curve is generated based on a target luminance histogram of the HDR image, and the adjusting the tone mapping curve includes: determining a target frequency to be cut according to the first ordinate and the preset range; determining a region to be cut and a region to be allocated in the target brightness histogram; according to the target frequency, frequency clipping is carried out on the area to be clipped, and frequency distribution is carried out on the area to be distributed, so that an updated brightness histogram is obtained; and calculating an accumulated probability distribution curve according to the updated brightness histogram, and taking the accumulated probability distribution curve as an adjusted tone mapping curve.

As a possible implementation manner, the determining, according to the first ordinate and the preset range, the target frequency number to be cut includes: if the first ordinate is greater than the upper limit value of the preset range, calculating the target frequency to be cut through the following formula: num_clip= (y_orig-y_up) N; if the first ordinate is smaller than the lower limit value of the preset range, calculating the target frequency to be cut through the following formula: num_clip= (y_down-y_orig) ×n; wherein num_clip is the target frequency, y_orig is the first ordinate, y_up is the upper limit, y_down is the lower limit, and N is the total frequency of the target luminance histogram.

As a possible implementation manner, the performing frequency clipping on the to-be-clipped area according to the target frequency number, and performing frequency distribution on the to-be-distributed area to obtain an updated brightness histogram, where the method includes: determining a clipping height according to the target frequency; according to the clipping height, frequency clipping is carried out on each interval Bin in the to-be-clipped area, frequency distribution is carried out on each interval Bin in the to-be-distributed area, and an updated brightness histogram is obtained; the clipping height and the number of the intervals Bin in the region to be clipped satisfy the following relation:

(K-1)*n≤K*n≤(K+1)*n

wherein K is the clipping height, and n is the number of intervals Bin in the region to be clipped.

As a possible implementation manner, the reduced value of the histogram height corresponding to each Bin in the region to be clipped after frequency clipping is equal to the clipping height; and the sum of the added values of the histogram heights corresponding to the bins in the area to be allocated after the frequency allocation is equal to the target frequency.

As a possible implementation manner, the control points include m control points arranged from small to large according to an abscissa, and the determining the target brightness The region to be clipped and the region to be allocated in the histogram comprise: when the first ordinate corresponding to the ith control point on the tone mapping curve is greater than the upper limit value of the preset range, determining that the abscissa range of the region to be cut in the target brightness histogram is (a, b), and determining that the abscissa range of the region to be allocated in the target brightness histogram is (c, d); wherein b is the abscissa x of the ith control point _i C is x _i Adding 1; when i=1, a is 0, d is the abscissa x of the 2 nd control point ₂ The method comprises the steps of carrying out a first treatment on the surface of the When i=m, a is the abscissa x of the m-1 th control point _m-1 Adding 1 and d to the maximum abscissa x of the target brightness histogram _end The method comprises the steps of carrying out a first treatment on the surface of the When 1 < i < m, a is the abscissa x of the i-1 th control point _i-1 Add 1, d is the abscissa x of the (i+1) th control point _i+1 The method comprises the steps of carrying out a first treatment on the surface of the Or when the first ordinate corresponding to the ith control point on the tone mapping curve is smaller than the lower limit value of the preset range, determining that the abscissa range of the region to be clipped in the target brightness histogram is (e, f), and determining that the abscissa range of the region to be allocated in the target brightness histogram is (g, h); wherein e is the abscissa x of the ith control point _i Adding 1, h is x _i The method comprises the steps of carrying out a first treatment on the surface of the When i=1, f is the abscissa x of the 2 nd control point ₂ G is 0; when i=m, f is the maximum abscissa x of the target luminance histogram _end G is the abscissa x of the m-1 th control point _m-1 Adding 1; when 1 < i < m, f is the abscissa x of the (i+1) -th control point _i+1 G is the abscissa x of the i-1 th control point _i-1 1 is added.

As a possible implementation manner, the frequency allocation for each Bin in the area to be allocated includes: calculating the frequency of average allocation of each interval Bin in the area to be allocated by the following formula:wherein (1)>For the average allocated frequency number, num_clip isThe target frequency, M, is the total number of intervals Bin in the area to be allocated; and increasing the frequency number of the average allocation for the frequency number corresponding to each interval Bin in the area to be allocated.

As a possible implementation manner, the frequency allocation for each Bin in the area to be allocated includes: calculating the frequency k allocated to the u-th interval Bin in the area to be allocated by the following formula _u ：

Wherein num_clip is the target frequency, M is the total number of intervals Bin in the area to be allocated, and d _u For the width, d, of the u-th interval Bin in the region to be allocated _j For the width of the jth interval Bin in the area to be allocated, u is more than or equal to 1 and less than or equal to M, and j is more than or equal to 1 and less than or equal to M; increasing the frequency corresponding to the ith interval Bin in the area to be allocated by the k _u 。

As a possible implementation manner, the HDR image is synthesized based on an LDR image with an R Zhang Bao light ratio of Ex, and the dynamic range of the LDR image with the exposure ratio of Ex is D _L The dynamic range of the HDR image is based on Ex, R, D _L Determining, according to the dynamic range of the HDR image, selecting a control point in a coordinate system where the tone mapping curve is located, including: selecting m control points in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image; wherein, m=r-1, when the m control points are arranged from small to large according to the abscissa, the abscissa of the 1 st control point is D _L The abscissa of the ith control point is D _L *Ex ⁱ ，1＜i≤m。

As a possible implementation, the preset range is determined based on a target dynamic range, which is a specified dynamic range to which the HDR image needs to be compressed.

As a possible implementation, the target luminance histogram of the HDR image is obtained by: acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image; performing low-pass filtering on the original brightness histogram to obtain a filtered brightness histogram; and acquiring a target luminance histogram of the HDR image according to the filtered luminance histogram.

As a possible implementation manner, the obtaining a target luminance histogram of the HDR image according to the filtered luminance histogram includes: obtaining a target luminance histogram of the HDR image by:

Hist_new＝Hist_LPF*α+Hist*(1-α)；

wherein hist_new is the target luminance histogram, hist_lpf is the filtered luminance histogram, hist is the original luminance histogram, and α is a preset filter intensity coefficient.

As a possible implementation manner, the obtaining an original luminance histogram of the HDR image according to the luminance value of each pixel in the HDR image includes: acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image and a preset interval Bin with unequal width; wherein the width of the interval Bin representing the larger brightness value is larger.

As a possible implementation, the tone mapping curve includes: a global tone mapping curve and/or a local tone mapping curve for each pixel block in the HDR image; the first ordinate includes: an ordinate of the control point corresponding to the abscissa on the global tone mapping curve and/or an ordinate of the control point corresponding to the abscissa on the local tone mapping curve; the adjusted tone mapping curve includes: an adjusted global tone mapping curve and/or an adjusted local tone mapping curve.

As a possible implementation manner, the tone mapping curve after the adjustment includes: under the condition of the adjusted global tone mapping curve and the adjusted local tone mapping curve, obtaining a low dynamic range LDR image corresponding to the HDR image according to the adjusted tone mapping curve, including: obtaining a global brightness image according to the adjusted global tone mapping curve; obtaining a local brightness image according to the adjusted local tone mapping curve; and fusing the global brightness image and the local brightness image to obtain a low dynamic range LDR image corresponding to the HDR image.

As a possible implementation manner, the obtaining a local brightness image according to the adjusted local tone mapping curve includes: for an L-th pixel block in the HDR image, determining a pixel block in a neighborhood centered on the L-th pixel block; wherein 1.ltoreq.L.ltoreq.T, T being the total number of pixel blocks in the HDR image; determining weights respectively corresponding to the pixel blocks in the neighborhood according to the brightness difference value of the pixel blocks in the neighborhood and the L-th pixel block; according to the weights respectively corresponding to the pixel blocks in the neighborhood, performing smoothing treatment on the adjusted local tone mapping curve of the L-th pixel block to obtain a smoothed local tone mapping curve of the L-th pixel block; and obtaining a local brightness image according to the local tone mapping curve after the smoothing treatment of the T pixel blocks in the HDR image.

As a possible implementation manner, the smoothing processing is performed on the adjusted local tone mapping curve of the L-th pixel block according to weights corresponding to the pixel blocks in the neighborhood, to obtain a smoothed local tone mapping curve of the L-th pixel block, where the smoothing processing includes: the smoothed local tone mapping curve of the L pixel block is obtained by the following formula:

wherein LUT _new LUT for the smoothed local tone mapping curve of the L-th pixel block _q An adjusted local tone mapping curve, w, for the qth pixel block in said neighborhood _q And Q is the total number of pixel blocks in the neighborhood, and the weight of the Q-th pixel block is the weight of the Q-th pixel block.

As a possible implementation manner, after the obtaining the low dynamic range LDR image corresponding to the HDR image according to the adjusted tone mapping curve, the method further includes: and carrying out contrast enhancement on the LDR image corresponding to the HDR image according to a preset contrast enhancement curve to obtain the LDR image with enhanced contrast.

As a possible implementation manner, the performing contrast enhancement on the LDR image corresponding to the HDR image according to a preset contrast enhancement curve to obtain a LDR image with enhanced contrast, includes: normalizing brightness values of all pixel points in the LDR image; mapping the normalized brightness value of each pixel point of the LDR image according to a preset contrast enhancement curve, and outputting a brightness mapping image; and restoring the dynamic range of the brightness mapping image to the dynamic range before normalization to obtain the LDR image with enhanced contrast.

As a possible implementation manner, the contrast enhancement curve is a Gamma curve, and the parameter Gamma of the Gamma curve is determined by the following manner:

wherein avgL is an average brightness value of the LDR image, th1_avgl is a first preset threshold, th2_avgl is a second preset threshold, and th2 > th1 is greater than or equal to 1.

As a possible embodiment, the slope of the contrast enhancement curve at a position where the luminance value is smaller than the preset luminance value is smaller than 1.

In a second aspect, an embodiment of the present application further provides a chip, where the chip is located in an electronic device and connected to a memory in the electronic device, where the memory stores instructions executable by the chip, and where the instructions are executed by the chip, so that the chip can perform the image processing method described above.

In a third aspect, an embodiment of the present application further provides an electronic device, including: the chip and the memory connected with the chip.

In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described image processing method.

In the embodiment of the application, through adjusting the tone mapping curve corresponding to the HDR image, the corresponding ordinate of the abscissa of the control point on the adjusted tone mapping curve can be ensured to be in a preset range, and because the preset range is a pixel value range conforming to the perception of human eyes, the obtained LDR image corresponding to the HDR image can conform to the perception of human eyes based on the adjusted tone mapping curve, and the LDR image observed by human eyes is ensured to present a better effect, so that the HDR image is compressed into the LDR image, and meanwhile, the distortion of the LDR image obtained after compression is avoided, and the LDR image obtained after compression conforms to the perception of human eyes.

Drawings

One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings.

Fig. 1 is a flowchart of an image processing method according to the present embodiment of the application;

FIG. 2 is a flowchart of a method for obtaining a target luminance histogram of an HDR image according to the present embodiment of the present application;

FIG. 3 is a schematic diagram of an original luminance histogram obtained based on bins of equal width according to the present embodiment of the application;

FIG. 4 is a diagram of an original luminance histogram obtained based on bins of unequal widths according to the present embodiment of the application;

FIG. 5 is a diagram showing an original luminance histogram and a filtered luminance histogram according to the present embodiment of the present application;

FIG. 6 is a schematic diagram of tone mapping curves generated based on a target luminance histogram according to the present embodiment of the application;

FIG. 7 is a schematic illustration of an adjusted tone mapping curve generated based on an updated luminance histogram according to the present embodiment of the application;

FIG. 8 is a flow chart of one implementation of the tone mapping curve adjustment mentioned in this embodiment of the present application;

FIG. 9 is a schematic diagram of curve adjustment according to control point cp1 according to the present embodiment of the application;

FIG. 10 is a schematic diagram of curve adjustment according to control point cp2 according to the present embodiment of the application;

FIG. 11 is a flow chart of one implementation of step 105 as referred to in this embodiment of the present application;

FIG. 12 is a flow chart of one implementation of step 1052 mentioned in this embodiment of the application;

fig. 13 is a schematic view of a pixel block according to the present embodiment of the application;

FIG. 14 is a diagram showing a relationship between brightness difference and weight according to the present embodiment of the application;

FIG. 15 is a schematic diagram of Gamma curve according to the present embodiment of the application;

FIG. 16 is a schematic illustration of a spline according to the present embodiment of the application;

FIG. 17 is a schematic diagram of a sigmoid curve referred to in this embodiment of the application;

FIG. 18 is a flowchart of the LDR image contrast enhancement according to the preset contrast enhancement curve according to the present embodiment of the present application, to obtain a contrast enhanced LDR image;

fig. 19 is a process diagram of an image processing method according to the present embodiment of the application;

fig. 20 is a schematic structural view of an electronic device according to the present embodiment of the application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the following detailed description of the embodiments of the present application will be given with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in various embodiments of the present application, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the claimed application may be practiced without these specific details and with various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present application, and the embodiments can be mutually combined and referred to without contradiction.

The embodiment of the application relates to an image processing method which is applied to electronic equipment, wherein the electronic equipment can be image processing equipment. The image processing of the present embodiment mainly refers to compression of an HDR image into an LDR image by using a tone mapping technique, so as to preserve details in the HDR image and avoid distortion of the LDR image while performing image compression. The pixel value range of the HDR image is higher than the pixel value range that can be processed by the specific device, and the value range of the HDR image is generally 16 bits (0-65535), 14 bits (0-16383), 12 bits (0-4095), and the like. The pixel value range of the LDR image is lower than that of the HDR image, and the pixel value range of the LDR image is typically 10 bits (0-1023), 8 bits (0-255), or the like.

The flowchart of the image processing method of the present embodiment may refer to fig. 1, including:

step 101: and acquiring a tone mapping curve corresponding to the HDR image to be processed.

Step 102: and selecting a control point in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image.

Step 103: a first ordinate of the control point corresponding to the abscissa on the tone-mapping curve is determined.

Step 104: and if the first ordinate is not in the preset range, adjusting the tone mapping curve so that the second ordinate corresponding to the abscissa of the control point on the adjusted tone mapping curve is in the preset range.

Step 105: and obtaining an LDR image corresponding to the HDR image according to the adjusted tone mapping curve.

In this embodiment, by adjusting the tone mapping curve corresponding to the HDR image, it is ensured that the ordinate corresponding to the abscissa of the control point on the adjusted tone mapping curve can be in a preset range, and because the preset range is a pixel value range conforming to the perception of human eyes, the LDR image corresponding to the HDR image obtained based on the adjusted tone mapping curve can conform to the perception of human eyes, and the LDR image observed by human eyes is ensured to exhibit a better effect, so that the LDR image obtained after compression can be compressed into the LDR image while avoiding distortion of the LDR image obtained after compression, thereby conforming the perception of human eyes.

Implementation details of the image processing method of the present embodiment are specifically described below, and the following is merely provided for facilitating understanding, and is not necessary to implement the present embodiment.

In step 101, the HDR image to be processed acquired by the electronic device may be a Raw Bayer image or an RGB image. The electronic device may first obtain a target luminance histogram of the HDR image, and then generate a tone mapping curve according to the target luminance histogram, and specifically may obtain the tone mapping curve according to a histogram equalization principle.

The target luminance histogram is used for representing luminance value distribution of each pixel in the HDR image, specifically, the electronic device may count pixels in a preset luminance value range according to the luminance value of each pixel point in the HDR image, obtain luminance value range-frequency information, and draw an original luminance histogram based on the luminance value range-frequency information. A luminance value range can be understood as a Bin, which is a luminance value range.

In this embodiment, a target luminance histogram may be obtained from the original luminance histogram. The target luminance histogram may be an original luminance histogram, or may be a luminance histogram obtained by further processing the original luminance histogram.

In some embodiments, the method for obtaining the target luminance histogram of the HDR image may refer to the flowchart in fig. 2, including:

step 201: and acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image.

Step 202: and performing low-pass filtering on the original brightness histogram to obtain a filtered brightness histogram.

Step 203: and obtaining a target brightness histogram of the HDR image according to the filtered brightness histogram.

In step 201, the electronic device may count pixels within a preset luminance value range according to the luminance value of each pixel point in the HDR image, obtain luminance value range-frequency information, and draw an original luminance histogram based on the luminance value range-frequency information. The preset luminance value range may be set according to actual needs, so that the number of pixels in different luminance value ranges in the HDR image may be reflected based on the original luminance histogram, and the preset luminance value range may be understood as a preset luminance level.

In some embodiments, the implementation of step 201 may be: and acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image and a preset interval Bin with equal width. For example, a schematic diagram of an original luminance histogram obtained based on bins of equal width can be referred to fig. 3. The abscissa of the original luminance histogram is the luminance value, and the ordinate is the frequency number. Fig. 3 reflects the number of pixels distributed in different luminance value intervals, and the widths of the different luminance value intervals are equal.

In some embodiments, the implementation of step 201 may be: acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image and a preset interval Bin with unequal width; wherein the width of the interval Bin representing the larger brightness value is larger. For example, referring to fig. 4, a schematic diagram of an original luminance histogram obtained based on bins Bin with different widths may be shown, where the widths of the bins with different luminance values are not equal, and as the luminance values increase, the widths of the bins with luminance values also increase. For example, the widths of the intervals Bin of unequal widths may be, in order: 2 ⁰ ，2 ¹ ，2 ² ,...,2 ⁿ However, the width of the section Bin of the unequal width is not particularly limited in the present embodiment, and other unequal widths may be employed.

In this embodiment, it is considered that the human eye is nonlinear to the change in luminance according to weber's law. The conventional luminance histogram is obtained by converting the gray level of the image using a nonlinear function (for example, log function conversion), and then performing histogram statistics on the converted image. In this embodiment, a histogram statistics method is provided to replace nonlinear conversion of image gray scale based on bins with unequal widths, so that time-consuming nonlinear conversion operation is avoided. Therefore, the method of acquiring the original luminance histogram based on the bins with different widths in the embodiment can not only effectively enhance the image, but also shorten the image processing time.

In step 202, the electronic device performs Low-pass filter (LPF) on the original luminance histogram, which may be understood as performing histogram smoothing on the original luminance histogram, and the filtered luminance histogram may be understood as a smoothed luminance histogram. The histogram smoothing is performed by filtering the original luminance histogram with an LPF, and specifically, the filtered luminance histogram may be obtained by performing weighted average according to the frequency number corresponding to each Bin and the frequency numbers corresponding to the surrounding bins, and the weight of each Bin may be determined by an LPF kernel. For example, referring to fig. 5, the comparison schematic diagram of the original luminance histogram and the filtered luminance histogram may be shown in fig. 5, where weights of 5 bins Bin in fig. 5 are respectively: 1. 2, 1.

In step 203, the electronic device may directly take the filtered luminance histogram as a target luminance histogram of the HDR image. The filtered luminance histogram may also be further processed to obtain a target luminance histogram. For example, the filtered luminance histogram may be processed according to the original luminance histogram and a preset filtering intensity coefficient to obtain the target luminance histogram.

In this embodiment, the low-pass filtering, that is, the histogram smoothing, is performed on the original luminance histogram, so that the problem of image distortion caused by the overlarge slope of the tone mapping curve generated based on the original luminance histogram is advantageously avoided.

In some embodiments, the implementation of step 203 may be: the target luminance histogram of the HDR image is obtained by the following formula:

Hist_new＝Hist_LPF*α+Hist*(1_α)；

wherein hist_new is a target luminance histogram, hist_lpf is a filtered luminance histogram, hist is an original luminance histogram, and α is a preset filter intensity coefficient. The specific size of alpha belongs to the interval [0,1], the specific size of alpha can be set according to actual needs, the higher the alpha is, the higher the LPF strength is, the contrast is reduced, otherwise, the higher the contrast is, but the more obvious the distortion problem is.

In this embodiment, the intensity of the LPF may be controlled by the above formula, so that the problems of image contrast and image distortion may be considered, so that the target luminance histogram may not only maintain a certain contrast, but also be not significantly distorted.

In step 102, the electronic device selects a control point in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image. Wherein the selected control point is used for determining whether adjustment of the tone mapping curve is required. The abscissa of the control point can be determined according to the dynamic range of the HDR image, and the ordinate of the control point can be manually set by a person skilled in the art according to actual needs so as to accord with the perception of human eyes.

In some embodiments, the HDR image is synthesized based on an LDR image with an R Zhang Bao light ratio of Ex, with a dynamic range of D for the LDR image with an exposure ratio of Ex _L The dynamic range of HDR images is based on Ex, R, D _L The implementation manner of the step 102 may be determined as follows: selecting m control points in a coordinate system where a tone mapping curve is located according to the dynamic range of the HDR image; wherein, when m=r-1 and m control points are arranged from small to large according to the abscissa, the abscissa of the 1 st control point is D _L The abscissa of the ith control point is D _L *Ex ⁱ ，1＜i ≤m。

In a specific implementation, an image Sensor sets exposure parameters according to brightness of a photographed scene, so as to obtain R LDR images with low dynamic range, wherein the exposure ratio of the R LDR images is Ex, the exposure ratio refers to the ratio of exposure time corresponding to different LDR images, and the Sensor can set the dynamic range of the R images to D according to the exposure ratio _L Is combined into a high dynamic range image. Thus Ex, R, D _L The dynamic range of the high dynamic range HDR image is determined. The implementation isThe purpose of the example is: the resulting high dynamic range HDR image is compressed into a low dynamic range LDR image. When Ex, R, D are as described above _L After the three parameters are determined, control points can be selected by combining the three parameters, and the selection method comprises the following steps: the number of control points is equal to R-1, and assuming that r=3, the number of control points is 2. The abscissa of the control points is based on Ex and D _L The range of the ordinate is calculated as being set for a person. For example, the abscissa x of the 1 st control point ₁ ＝D _L Abscissa x of control point 2 ₂ ＝D _L *Ex ² The abscissa of the mth control point is x _m ＝D _L *Ex ^m 。

Assuming that the ordinate ranges of the 1 st control point and the 2 nd control point are respectively denoted by [ y1_down, y1_up ], [ y2_down, y2_up ], the ordinate ranges of the different control points may be the same or different, and specifically, the ordinate ranges may be manually set by a person skilled in the art according to actual needs, and the manually set ordinate ranges may be pixel value ranges conforming to the human eyes. For example, [ y1_down, y1_up ] may be [0.4×d,0.7×d ], [ y2_down, y2_up ] may be [0.8×d,0.9×d ], D is a specified dynamic range to which an HDR image needs to be compressed. The manually set ordinate range can be obtained through priori data experiments, and the priori data measurement can be carried out by adjusting the ordinate of the control point, specifically, the coefficient before D in the ordinate range can be adjusted, so that the optimal effect observed by human eyes is ensured, and the image observed by the human eyes is not distorted.

For example, referring to fig. 6, it can be seen from fig. 6 that, based on the target luminance histogram, a tone mapping curve is generated, and control points selected in the coordinate system where the tone mapping curve is located are cp1 and cp2. It should be noted that, in fig. 6, only for the sake of illustrating the relationship between the target luminance histogram and the tone mapping curve, the shape of the target luminance histogram is also drawn in the coordinate system where the tone mapping curve is located. It will be appreciated that the target luminance histogram has the same meaning as the abscissa of the tone mapping curve, the ordinate of the target luminance histogram has the meaning of a frequency number, and the ordinate of the tone mapping curve has the meaning of the pixel value of the mapped LDR.

In step 103, the electronic device may calculate a first ordinate of the abscissa of the control point corresponding to the tone mapping curve, for example, the tone mapping curve may be represented as a tone mapping function, and the value of the abscissa of the control point may be substituted into the tone mapping function, so as to calculate a mapping value, i.e. the first ordinate.

In step 104, the electronic device may determine whether the first ordinate is in a preset range, where the preset range is a range of pixel values that accords with the human eye perception. The preset range may be set by those skilled in the art according to actual needs on the basis of satisfying the condition of the pixel value range conforming to the human eye's sense. The preset range may be understood as an ordinate range preset for the control point, and preset ranges set for different control points may be the same or different.

In some embodiments, the preset range is determined based on a target dynamic range, which is a specified dynamic range to which the HDR image needs to be compressed. Referring to fig. 6, it is assumed that the ordinate ranges of control points cp1 and cp2 are denoted as [ y1_down, y1_up ], [ y2_down, y2_up ], respectively. Alternatively, [ y1_down, y1_up ] may be [0.4×d,0.7×d ], [ y2_down, y2_up ] may be [0.8×d,0.9×d ], D is a specified dynamic range to which an HDR image needs to be compressed. The manually set ordinate range can be obtained through priori data experiments, the priori data can be measured by adjusting the ordinate of the control point, and particularly the coefficient before D in the ordinate range can be adjusted, so that the optimal effect observed by human eyes is ensured, and the image observed by the human eyes cannot be distorted.

And when the first ordinate is not in the preset range, adjusting the tone mapping curve so that the corresponding second ordinate of the abscissa of the control point on the adjusted tone mapping curve is in the preset range. The adjusted tone mapping curve may be represented as an adjusted tone mapping function, and the abscissa of the control point is substituted into the adjusted tone mapping function, so as to calculate a mapping value, i.e. a second ordinate, which is within a preset range.

The embodiment can obtain an updated luminance histogram by performing frequency clipping and frequency distribution on the target luminance histogram, and generate an adjusted tone mapping curve based on the updated luminance histogram. For example, a schematic diagram of an adjusted tone mapping curve generated based on an updated luminance histogram may be referred to as fig. 7, where a dotted line is a tone mapping curve before adjustment and a solid line is a tone mapping curve after adjustment. In the tone-mapping curve of fig. 7, where the control points cp1 and cp2 are adjusted, it can be considered that the second ordinate of the abscissa of cp1 and cp2 on the adjusted tone-mapping curve is in the preset range. As can also be seen in conjunction with fig. 6 and 7, the updated luminance histogram in fig. 7 is cut off for the frequency numbers corresponding to the first 7 bins Bin, and the frequency numbers corresponding to the 8 th to 12 th bins Bin are increased, as compared with the target luminance histogram in fig. 6, with the increased portions shown as dashed boxes.

In some embodiments, an implementation of adjusting the tone mapping curve may refer to fig. 8, including:

step 1041: and determining the target frequency to be cut according to the first ordinate and the preset range.

Step 1042: and determining the region to be cut and the region to be allocated in the target brightness histogram.

Step 1043: and performing frequency clipping on the area to be clipped according to the target frequency, and performing frequency distribution on the area to be distributed to obtain an updated brightness histogram.

Step 1044: and calculating an accumulated probability distribution curve according to the updated brightness histogram, and taking the accumulated probability distribution curve as an adjusted tone mapping curve.

In step 1041, the preset range has an upper limit value and a lower limit value, and when the first ordinate is greater than the upper limit value, the target frequency to be cut may be calculated according to the difference between the first ordinate and the upper limit value. When the first ordinate is smaller than the lower limit value, the target frequency to be cut can be calculated according to the difference value between the first ordinate and the lower limit value.

In some embodiments, if the first ordinate is greater than the upper limit value of the preset range, the target frequency to be cut may be calculated by a formula:

num_clip＝(y_orig-y_up)*N；

if the first ordinate is smaller than the lower limit value of the preset range, the target frequency to be cut can be calculated through the following formula:

num_clip＝(y_down-y_orig)*N；

wherein num_clip is the target frequency, y_orig is the first ordinate, y_up is the upper limit, y_down is the lower limit, and N is the total frequency of the target luminance histogram.

In step 1042, the electronic device can determine a region to be clipped and a region to be allocated in the target luminance histogram according to the abscissa of the control point. The region to be clipped can be understood as a region in the target luminance histogram, which needs to be reduced in height, and the reduction in height can also be understood as a reduction in frequency. The area to be allocated may be understood as an area in the target luminance histogram where an increase in height is required, and an increase in height may be understood as an increase in frequency.

In some embodiments, the control points include m control points arranged from small to large in the abscissa, and the implementation of step 1042 may be as follows:

when the first ordinate of the ith control point corresponding to the tone mapping curve is greater than the upper limit value of the preset range, determining that the abscissa range of the region to be clipped in the target brightness histogram is (a, b), and determining that the abscissa range of the region to be allocated in the target brightness histogram is (c, d). Wherein b is the abscissa x of the ith control point _i C is x _i 1 is added.

When i=1, a is 0, d is the abscissa x of the 2 nd control point ₂ 。

When i=m, a is the abscissa x of the m-1 th control point _m-1 Adding 1 and d to the maximum abscissa x of the target brightness histogram _end 。

When 1 < i < m, a is the abscissa x of the i-1 th control point _i-1 Add 1, d is the abscissa x of the (i+1) th control point _i+1 。

Or alternatively, the process may be performed,

when the ith control point corresponds to the first vertical direction on the tone mapping curveAnd if the coordinates are smaller than the lower limit value of the preset range, determining that the abscissa range of the region to be cut in the target brightness histogram is (e, f), and determining that the abscissa range of the region to be allocated in the target brightness histogram is (g, h). Wherein e is the abscissa x of the ith control point _i Adding 1, h is x _i 。

When i=1, f is the abscissa x of the 2 nd control point ₂ G is 0;

when i=m, f is the maximum abscissa x of the target luminance histogram _end G is the abscissa x of the m-1 th control point _m-1 Adding 1;

when 1 < i < m, f is the abscissa x of the (i+1) -th control point _i+1 G is the abscissa x of the i-1 th control point _i-1 1 is added.

To facilitate an understanding of the above embodiments, the following is illustrated by a specific example:

assuming that m=2, 2 control points arranged from small to large on the abscissa are cp1 and cp2, respectively. cp1 abscissa x ₁ The ordinate range is set to [ y1_down, y1_up ]]Cp2 abscissa is x ₂ The ordinate range is set to [ y2_down, y2_up ]]。

Referring to FIG. 9, for control point cp1 in FIG. 9, if the first ordinate y1_orig of cp1 on the tone mapping curve is greater than y1_up, the abscissa range of the region to be clipped is (0, x) ₁ ) The abscissa range of the area to be allocated is (x ₁ +1,x ₂ ) Target frequency num_clip= (y2_orig-y2_up) N. If the first ordinate y1_orig of the abscissa of cp1 on the tone mapping curve is smaller than y1_down, the abscissa range of the region to be clipped is (x) ₁ +1,x ₂ ) The abscissa range of the area to be allocated is (0, x ₁ ) Target frequency num_clip= (y1_down-y1_orig) ×n. Wherein x is ₁ Can represent the upper limit value, x of the current interval Bin in which the abscissa of cp1 is located ₁ +1 represents the next Bin of the current Bin.

Referring to FIG. 10, for control point cp2 in FIG. 10, if the first ordinate y2_orig of the abscissa of cp2 on the tone mapping curve is less than y2_Down, then the abscissa of the region to be croppedThe coordinate range is (x ₂ +1， x _end ) The abscissa range of the area to be allocated is (x ₁ +1,x ₂ ) Target frequency num_clip= (y2_down-y2_orig) ×n. If the first ordinate y2_orig of the abscissa of cp2 on the tone mapping curve is greater than y2_up, the abscissa range of the region to be clipped is (x ₁ +1,x ₂ ) The abscissa range of the area to be allocated is (x ₂ +1，x _end ) Target frequency num_clip= (y2_orig-y2_up) N. Wherein x is ₂ Can represent the upper limit value, x of the current interval Bin in which the abscissa of cp2 is located ₂ +1 represents the next Bin of the current Bin.

In step 1043, the frequency clipping may be understood as: after the region to be cut is cut, the sum of the frequencies of which the frequencies corresponding to the intervals Bin in the region to be cut are reduced is the target frequency. The frequency allocation can be understood as: the sum of the frequency numbers to which the frequency numbers corresponding to the respective bins in the area to be allocated are increased is the target frequency number.

In some embodiments, the implementation of step 1043 may be: and determining the clipping height according to the target frequency. And according to the clipping height, performing frequency clipping on each Bin in the area to be clipped, and performing frequency distribution on each Bin in the area to be distributed to obtain an updated brightness histogram. The clipping height and the number of the intervals Bin in the area to be clipped satisfy the following relation:

(K-1)*n≤K*n≤(K+1)*n

wherein K is the clipping height, and n is the number of intervals Bin in the area to be clipped.

In some embodiments, the clipping height K for clipping each Bin in the region to be clipped may be determined according to the target frequency and a binary search algorithm. The frequency of clipping each Bin in the region to be clipped is the clipping height, that is, the clipping height K may be understood as the frequency of clipping. The sum of clipping heights of all the intervals Bin in the region to be clipped may be equal to the target frequency or may not be exactly equal to the target frequency. For example, the sum of the clipping heights of all the intervals Bin in the region to be clipped may be close to, but not completely equal to, the target frequency, and if the determined K satisfies the inequality relationship, the frequency clipping and the allocation may be performed according to the K value, so that even if the sum of the clipping heights of all the intervals Bin in the region to be clipped is not equal to the calculated target frequency, the frequency clipping and the allocation satisfying the requirement may be completed.

In some embodiments, the reduced value of the histogram height corresponding to each Bin in the region to be clipped after the frequency clipping is equal to the clipping height. And the sum of the added values of the histogram heights corresponding to the intervals Bin in the area to be allocated after the frequency allocation is carried out is equal to the target frequency.

In some embodiments, the step 1043 mentioned: frequency allocation is carried out on each interval Bin in the area to be allocated, and the frequency allocation comprises the following steps: the frequency number of average allocation of each Bin in the area to be allocated is calculated by the following formula:wherein (1)>For the average allocated frequency number, num_clip is a target frequency number, and M is the total number of intervals Bin in the area to be allocated; the frequency number of average allocation is increased for each interval Bin in the area to be allocated. That is, in the present embodiment, the target frequency number is taken as the total frequency number to be allocated, and the total frequency number is equally allocated to each section Bin in the area to be allocated.

For example, referring to fig. 9 or 10, the Bin in the area to be allocated includes 8 th to 12 th bins, and the frequency numbers corresponding to 8 th to 12 th bins are increasedThe increased frequency is shown in dashed boxes. Referring to fig. 9, the Bin in the region to be cut includes 1 st to 7 th bins, and the frequency numbers corresponding to 1 st to 7 th bins are each reduced by K/7.

In some embodiments, the step 1043 mentioned: in the area to be allocatedEach interval Bin is subjected to frequency distribution, and the frequency distribution comprises the following steps: calculating the frequency number k allocated to the ith interval Bin in the area to be allocated by the following formula _u ：

Wherein num_clip is the target frequency, M is the total number of intervals Bin in the area to be allocated, d _u For the width, d, of the u-th interval Bin in the area to be allocated _j The width of the jth interval Bin in the area to be allocated is equal to or more than 1 and equal to or less than or equal to M, and j is equal to or less than 1 and equal to or less than M; increasing the frequency corresponding to the ith interval Bin in the area to be allocated by k _u . That is, in the present embodiment, the target frequency number is taken as the total frequency number to be allocated, and the total frequency number is weighted to be allocated to each Bin in the area to be allocated.

In step 1044, the updated luminance histogram is the luminance histogram obtained after the frequency clipping and the frequency allocation. Based on the updated luminance histogram, a cumulative probability distribution curve may be calculated, and finally the cumulative probability distribution curve may be used as an adjusted tone mapping curve.

In some embodiments, the plurality of selected control points may be sequentially adjusted according to each control point, so that the abscissas of the plurality of control points on the adjusted tone mapping curve are all within a preset range. Referring to fig. 9 and 10, in fig. 9, the tone mapping curve (denoted as curve 1) is first adjusted according to the control point cp1, and the tone mapping curve (denoted as curve 2) passing through the control point cp1 after the adjustment is obtained. Referring to fig. 10, with reference to curve 2, if the abscissa of cp2 is not within the preset range on the first ordinate corresponding to curve 2, curve 2 is adjusted to obtain a tone mapping curve (denoted as curve 3) passing through cp 2. And the curve 3 is used as a tone mapping curve finally obtained by adjustment, and the corresponding second ordinate of the abscissa of the control points cp1 and cp2 on the curve 3 is in a preset range, namely, the control points cp1 and cp2 are both positioned on the curve 3.

In step 105, the electronic device performs tone mapping on the HDR image according to the adjusted tone mapping curve, so as to compress the HDR image, and obtain an LDR image corresponding to the HDR image. By tone mapping techniques, it is possible to achieve preservation of image details in an HDR image while image compression. By adjusting the tone mapping curve, the corresponding ordinate of the abscissa of the control point on the adjusted tone mapping curve is in a preset range, so that the LDR image distortion obtained by compression is avoided, and the LDR image obtained by compression accords with the perception of human eyes.

In some embodiments, the tone mapping curve includes: global tone mapping curves and/or local tone mapping curves for blocks of pixels in an HDR image. The first ordinate includes: an ordinate of the control point corresponding to an abscissa on the global tone mapping curve and/or an ordinate of the control point corresponding to an abscissa on the local tone mapping curve; the adjusted tone mapping curve includes: an adjusted global tone mapping curve and/or an adjusted local tone mapping curve.

In some embodiments, the tone mapping curve obtained in step 101 is a global tone mapping curve, which is obtained based on a global target luminance histogram of the HDR image, which may be obtained based on a global original luminance histogram. Steps 102 to 105 are all processes based on the global tone mapping curve. In step 105, global tone mapping is performed on the HDR image according to the adjusted global tone mapping curve, so as to obtain a low dynamic range LDR image corresponding to the HDR image. The global tone mapping refers to mapping all pixel points in an image by using a space invariant mapping function. Because each pixel point of the HDR image uses the same mapping function, the mapping function is a one-to-one mapping relation, the algorithm is simple, the running speed is high, and the image processing time is saved.

In some embodiments, the tone mapping curve obtained in step 101 is a local tone mapping curve for each pixel block in the HDR image. Wherein each pixel block may comprise a number of pixel points, and the local tone mapping curve is obtained based on a local target luminance histogram of each pixel block of the HDR image. The local target luminance histogram may be obtained based on the local original luminance histogram. Steps 102 to 105 are all processes based on the local tone mapping curve. In step 105, a low dynamic range LDR image corresponding to the HDR image is obtained according to the adjusted local tone mapping curve. Namely, according to the adjusted local tone mapping curve, performing local tone mapping on the HDR image to obtain a low dynamic range LDR image corresponding to the HDR image. Wherein local tone mapping refers to adjusting the mapping function based on local pixel statistics and local pixel context. The local tone mapping performs different transformations for different areas where each pixel of the image is located, so that it is possible that two pixels with the same brightness value are mapped to obtain different new values or two pixels with different brightness values are mapped to obtain the same new value because of different positions in the same image. The local tone mapping is favorable for mapping to obtain an image with better effect.

In some embodiments, the tone mapping curve obtained in step 101 includes: global tone mapping curves and local tone mapping curves for pixel blocks in an HDR image. In step 104, the local tone mapping curve and the global tone mapping curve may be adjusted in an adjustment manner as shown in fig. 8, to obtain an adjusted local tone mapping curve and an adjusted global tone mapping curve. The implementation of step 105 may refer to fig. 11, including:

step 1051: and obtaining a global brightness image according to the adjusted global tone mapping curve.

Step 1052: and obtaining a local brightness image according to the adjusted local tone mapping curve.

Step 1053: and fusing the global brightness image and the local brightness image to obtain an LDR image corresponding to the HDR image.

In step 1051, the electronic device may perform global tone mapping on the HDR image according to the adjusted global tone mapping curve to obtain a mapped global luminance image.

In step 1052, the electronic device may perform local tone mapping on each pixel block according to the adjusted local tone mapping curve corresponding to each pixel block in the HDR image, to obtain a mapped local luminance image corresponding to each pixel block.

In step 1053, the electronic device may fuse the global luminance image and the local luminance image according to a preset fusion ratio, obtain a fused luminance image, and use the fused luminance image as an LDR image corresponding to the obtained HDR image.

In some embodiments, the fused luminance image pic_new may be obtained by the following formula:

Pic_new＝Pic_1*ε+Pic_2*(1-ε)；

wherein pic_1 is a global luminance image, pic_2 is a local luminance image, epsilon is a preset fusion proportion, and the size of the fusion proportion can be set according to actual needs, which is not particularly limited in this embodiment.

In this embodiment, global tone mapping and local tone mapping are performed on the HDR image to be processed, and the global luminance image obtained by the global tone mapping and the local luminance image obtained by the local tone mapping are fused, so that the advantages of the global tone mapping and the local tone mapping are combined, and an LDR image with better effect is obtained. The HDR image is compressed into the LDR image, and meanwhile, the distortion of the LDR image obtained after compression is greatly avoided, so that the LDR image obtained after compression is more in line with the perception of human eyes.

In some embodiments, the implementation of step 1052 may refer to fig. 12, including:

Step 301: for an L-th pixel block in the HDR image, a pixel block in a neighborhood centered on the L-th pixel block is determined.

Step 302: and determining weights respectively corresponding to the pixel blocks in the adjacent region according to the brightness difference value of the pixel blocks in the adjacent region and the L-th pixel block.

Step 303: and carrying out smoothing treatment on the adjusted local tone mapping curve of the L-th pixel block according to the weights respectively corresponding to the pixel blocks in the adjacent region to obtain a smoothed local tone mapping curve of the L-th pixel block.

Step 304: and obtaining a local brightness image according to the local tone mapping curve after the smoothing processing of the T pixel blocks in the HDR image.

In step 301, 1.ltoreq.L.ltoreq.T, T being the total number of pixel blocks in the HDR image. For example, referring to fig. 13, the L-th pixel block is the pixel block denoted by L in fig. 13, and the pixel blocks in the neighborhood centered on the L-th pixel block are the 9 pixel blocks centered on the L-th pixel block in fig. 13, which are respectively: pixel blocks marked L and pixel blocks marked L1 to L8. The local tone mapping curves for each of the 9 pixel blocks in fig. 13 are also labeled.

In step 302, a luminance value for each pixel block within the neighborhood may be calculated, which may be an average of the luminance of a plurality of pixel points in each pixel block. Then, a luminance difference value of the average luminance value of each pixel block and the average luminance value of the L-th pixel block is calculated. The brightness difference value and the weight can have a preset corresponding relation, so that the electronic equipment can obtain the weights respectively corresponding to the pixel blocks in the adjacent areas according to the corresponding relation.

In some embodiments, the correspondence between the luminance difference value and the weight may be a relationship curve as shown in fig. 14, and the electronic device may determine the weight corresponding to each pixel block in the neighborhood with reference to the relationship curve shown in fig. 14. It should be noted that fig. 14 only shows an exemplary relationship between the luminance difference and the weight, and in a specific implementation, the exemplary relationship between the luminance difference and the weight may be set according to actual needs, and is not limited to the relationship shown in fig. 14.

In step 303, the greater the weight, the higher the smoothness corresponding to the pixel block, where the smoothing process can also be understood as: and carrying out weighted filtering on the local tone mapping curve of the L pixel block according to the weights respectively corresponding to the pixel blocks in the adjacent region to obtain the smoothed local tone mapping curve of the L pixel block. That is, the tone mapping curve of the L-th pixel block and the tone mapping curves of other pixel blocks in the neighborhood are weighted and mixed according to a certain weight, and the smoothed local tone mapping curve of the L-th pixel block is obtained.

In some embodiments, the implementation of step 303 may be: the smoothed local tone mapping curve of the L-th pixel block is obtained by the following formula:

Wherein LUT _new LUT for the smoothed local tone mapping curve of the L-th pixel block _q An adjusted local tone mapping curve, w, for the q-th pixel block in the neighborhood _q The Q-th pixel block is weighted, and Q is the total number of pixel blocks in the neighborhood. The local tone mapping curve of each pixel block may be stored in a Look-Up-Table (LUT) in which coordinate information of several points on the tone mapping curve may be stored in detail.

Referring to fig. 13, the weights corresponding to the pixel blocks marked with L and the pixel blocks marked with L1 to L8 are respectively: 1. w1, w2...w 8, the local tone mapping curves of the pixel blocks labeled L and the pixel blocks labeled L1 to L8, respectively, may be represented as LUTs _L 、LUT _L1 、LUT _L2 ....LUT _L8 The expression of the smoothed local tone mapping curve of the L-th pixel block may be as follows:

according to the above manner, the smoothed local tone mapping curve of each pixel block in the HDR image can be obtained, and finally the smoothed local tone mapping curve of all the pixel blocks, i.e., T pixel blocks, in the HDR image is obtained.

In step 304, according to the smoothed local tone mapping curves of the T pixel blocks in the HDR image, local luminance images corresponding to the T pixel blocks are obtained.

In this embodiment, by performing smoothing processing on each pixel block in the HDR image, it is advantageous to suppress artifacts existing in the low dynamic range LDR image that is finally obtained and to suppress the brightness of halation in the LDR image.

In some embodiments, after step 105, a process of contrast enhancement of the LDR image is also included. In this embodiment, the contrast enhancement may be performed on the LDR image corresponding to the HDR image according to a preset contrast enhancement curve, to obtain the LDR image after the contrast enhancement. The input of the contrast enhancement curve may be an LDR image after the HDR image is compressed by the tone mapping curve, and the output may be an LDR image after the contrast enhancement.

In some embodiments, the contrast enhancement curve may be selected according to actual needs, such as a Gamma curve as shown in fig. 15, a spline curve as shown in fig. 16, a sigmoid curve as shown in fig. 17, and so on.

In some embodiments, the parameters of the contrast enhancement curve may be manually set according to actual needs, for example, when the contrast enhancement curve is a Gamma curve, the parameter γ of the Gamma curve may be greater than or equal to 1, so as to compress the dark brightness of the LDR image.

In some embodiments, the parameters of the contrast enhancement curve may also be determined from the average brightness of the LDR image. For example, the contrast enhancement curve is a Gamma curve, and the parameter γ of the Gamma curve is determined as follows:

Wherein avgL is an average brightness value of the LDR image, th1_avgl is a first preset threshold, th2_avgl is a second preset threshold, and th2 > th1 is greater than or equal to 1. The first preset threshold and the second preset threshold can take values between 0 and 1 according to actual needs. For example, th1=1, th2=2, th1_avgl=0.2, th2_avgl=0.5, but this embodiment is not particularly limited thereto.

In some embodiments, the contrast enhancement curve may be a spline curve as shown in fig. 16, for example, a spline curve obtained by segmenting three Hermite interpolation, selecting four reference points, namely (0, 0), c1 (x 1, y 1), c2 (x 2, y 2), and c3 (1, 1), interpolating the curve by the reference points, and performing contrast enhancement processing on the LDR image according to the interpolated contrast enhancement curve. (x 1, y 1), (x 2, y 2) can be understood as parameters of a spline curve, the size of which can be determined by the average luminance of the LDR image. In one possible embodiment, x1=0.3, x2=0.7, y1=0.1, y2=0.8.

In some embodiments, according to a preset contrast enhancement curve, contrast enhancement is performed on an LDR image, and a flowchart of obtaining an LDR image after contrast enhancement may refer to fig. 18, which includes:

Step 401: and normalizing the brightness value of each pixel point in the LDR image.

Step 402: and mapping the normalized brightness value of each pixel point of the LDR image according to a preset contrast enhancement curve, and outputting a brightness mapping image.

Step 403: and restoring the dynamic range of the brightness mapping image to the dynamic range before normalization to obtain the LDR image with enhanced contrast.

In step 401, normalization may be understood as normalizing the luminance value of each pixel in the LDR image from the current dynamic range [0, d ] of the LDR image to [0,1], to obtain a normalized luminance image.

In step 402, each pixel point in the normalized luminance image may be mapped using the selected contrast enhancement curve, outputting a luminance mapped image. The contrast enhancement curve may be represented by a contrast enhancement function, the normalized luminance image may be an input to the contrast enhancement function, and the luminance map image may be an output of the contrast enhancement function.

In step 403, the current dynamic range [0,1] of the brightness map image may be restored to the dynamic range [0, d ] before normalization, to obtain the LDR image after contrast enhancement.

In this embodiment, considering that the LDR image mapped by the tone mapping curve may enhance the brightness of the dark portion, the overall brightness of the image is higher, and the contrast is reduced (contrast refers to the difference range between the dark portion and the bright portion in the image). Therefore, in this embodiment, the preset contrast enhancement curve is used to enhance the contrast of the LDR image, which is favorable for suppressing the brightness of the dark portion, avoiding the overall brightness of the image from being too high, and is favorable for enhancing the contrast of the LDR image. Namely, the image processing method of the embodiment is beneficial to avoiding the distortion of the LDR image obtained after compression, so that the LDR image obtained after compression accords with the perception of human eyes and the contrast of the LDR image is enhanced.

In some embodiments, the slope of the contrast enhancement curve at the position where the brightness value is smaller than the preset brightness value is smaller than 1, where the preset brightness value may be set according to the actual requirement, which is intended to indicate that the brightness value is in a smaller range. The slope of the contrast enhancement curve is controlled to be smaller than 1 at the position where the brightness value is smaller than the preset brightness value, so that the aim of noise suppression is fulfilled.

In some embodiments, a process schematic of the image processing method may refer to fig. 19, where the input image is an HDR image to be processed, and specifically may be a Raw Bayer image or an RGB image. The image processing process mainly comprises the following steps:

S1, generating brightness, namely generating a brightness image according to the Raw Bayer image or the RGB image.

S2, counting the histogram, namely counting pixels in a preset brightness range according to the brightness image to obtain brightness value range-frequency information. Specifically, a global original luminance histogram and a local original luminance histogram may be statistically generated.

S3, carrying out histogram smoothing filtering, namely respectively carrying out low-pass filtering on the statistical global original brightness histogram and the local original brightness histogram. Specifically, the luminance information is unchanged, and the statistical frequency information is smoothed according to the weight specified by the LPFkernel.

S4, generating a tone mapping curve, namely generating the global tone mapping curve according to the global target brightness histogram obtained after the smoothing filtering. And generating a local tone mapping curve according to the local target brightness histogram obtained after the smoothing filtering.

S4, adjusting the tone mapping curve, namely adjusting the local tone mapping curve and the global tone mapping curve according to the designated control point.

S5, smoothing the local curve, namely smoothing the adjusted local tone mapping curve of each pixel block according to the brightness information of each pixel block and the pixel blocks in the neighborhood of the pixel block.

S6, fusing the local brightness image and the global brightness image, namely mixing the local brightness image and the global brightness image according to a certain proportion.

S7, contrast enhancement, namely performing further contrast enhancement processing on the output LDR image according to a preset contrast enhancement curve to obtain an LDR image with enhanced contrast.

The image processing method in this embodiment can compress the high dynamic range HDR image into the low dynamic range LDR image, and simultaneously make the LDR image retain details in the HDR image as much as possible, so as to avoid distortion of the LDR image obtained after compression, and thus make the LDR image obtained after compression conform to the perception of human eyes. It is also possible to suppress artifacts and halation problems existing in an LDR image, and to enhance the contrast of the LDR image while suppressing noise in the LDR image.

The above examples in this embodiment are all examples for easy understanding, and do not limit the technical configuration of the present application.

The above steps of the methods are divided, for clarity of description, and may be combined into one step or split into multiple steps when implemented, so long as they include the same logic relationship, and they are all within the protection scope of this patent; it is within the scope of this patent to add insignificant modifications to the algorithm or flow or introduce insignificant designs, but not to alter the core design of its algorithm and flow.

The embodiment of the present application also relates to a chip, and referring to fig. 20, a chip 501 is connected to a memory 505, and the memory stores instructions executable by the chip, so that the chip can perform the image processing method in the above embodiment.

Where memory 505 and memory 505 are connected by way of a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together various circuits of one or more of the chips 501 and memory 505. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the chip 501 is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the chip 501.

The chip 501 is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 505 may be used to store data used by chip 501 in performing operations.

The embodiment of the application also relates to an electronic device, referring to fig. 20, including: the chip 501 and a memory 502 connected to the chip 501.

The embodiment of the application also relates to a computer readable storage medium which stores a computer program. The computer program implements the above-described method embodiments when executed by a processor.

That is, it will be understood by those skilled in the art that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, where the program includes several instructions for causing a device (which may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps in the methods of the embodiments of the application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples of carrying out the application and that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. An image processing method, comprising:

acquiring a tone mapping curve corresponding to a high dynamic range HDR image to be processed;

selecting a control point in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image;

determining a first ordinate of the abscissa of the control point corresponding to the tone mapping curve;

if the first ordinate is not in the preset range, adjusting the tone mapping curve so that a second ordinate corresponding to the abscissa of the control point on the adjusted tone mapping curve is in the preset range; the preset range is a pixel value range conforming to the perception of human eyes;

and obtaining a low dynamic range LDR image corresponding to the HDR image according to the adjusted tone mapping curve.

2. The image processing method of claim 1, wherein the tone mapping curve is generated based on a target luminance histogram of the HDR image, the adjusting the tone mapping curve comprising:

determining a target frequency to be cut according to the first ordinate and the preset range;

determining a region to be cut and a region to be allocated in the target brightness histogram;

According to the target frequency, frequency clipping is carried out on the area to be clipped, and frequency distribution is carried out on the area to be distributed, so that an updated brightness histogram is obtained;

and calculating an accumulated probability distribution curve according to the updated brightness histogram, and taking the accumulated probability distribution curve as an adjusted tone mapping curve.

3. The image processing method according to claim 2, wherein the determining the frequency of the object to be cut according to the first ordinate and the preset range includes:

if the first ordinate is greater than the upper limit value of the preset range, calculating the target frequency to be cut through the following formula:

num_clip＝(y_orig-y_up)*N；

if the first ordinate is smaller than the lower limit value of the preset range, calculating the target frequency to be cut through the following formula:

num_clip＝(y_down-y_orig)*N；

4. The image processing method according to claim 2, wherein the performing frequency clipping on the region to be clipped according to the target frequency number and performing frequency distribution on the region to be distributed to obtain an updated luminance histogram includes:

Determining a clipping height according to the target frequency;

according to the clipping height, frequency clipping is carried out on each interval Bin in the to-be-clipped area, frequency distribution is carried out on each interval Bin in the to-be-distributed area, and an updated brightness histogram is obtained;

the clipping height and the number of the intervals Bin in the region to be clipped satisfy the following relation:

(K-1)*n≤K*n≤(K+1)*n

5. The image processing method according to claim 4, wherein a reduced value of a histogram height corresponding to each Bin in the region to be clipped after frequency clipping is equal to the clipping height; and the sum of the added values of the histogram heights corresponding to the bins in the area to be allocated after the frequency allocation is equal to the target frequency.

6. The image processing method according to claim 2, wherein the control points include m control points arranged from small to large in abscissa, and the determining the region to be clipped and the region to be allocated in the target luminance histogram includes:

when the first ordinate corresponding to the ith control point on the tone mapping curve is greater than the upper limit value of the preset range, determining that the abscissa range of the region to be cut in the target brightness histogram is (a, b), and determining that the abscissa range of the region to be allocated in the target brightness histogram is (c, d); wherein b is the abscissa x of the ith control point _i C is x _i Adding 1;

when i=1, a is 0, d is the abscissa x of the 2 nd control point ₂ ；

When i=m, a is the abscissa x of the m-1 th control point _m-1 Adding 1 and d to the maximum abscissa x of the target brightness histogram _end ；

When 1 < i < m, a is the abscissa x of the i-1 th control point _i-1 Add 1, d is the abscissa x of the (i+1) th control point _i+1 ；

Or alternatively, the process may be performed,

when the first ordinate corresponding to the ith control point on the tone mapping curve is smaller than the lower limit value of the preset range, determining that the abscissa range of the region to be cut in the target brightness histogram is (e, f), and determining that the abscissa range of the region to be allocated in the target brightness histogram is (g, h); wherein e is the abscissa x of the ith control point _i Adding 1, h is x _i ；

When i=1, f is the abscissa x of the 2 nd control point ₂ G is 0;

7. The image processing method according to claim 2, wherein said performing frequency distribution on each Bin in the area to be distributed comprises:

Calculating the frequency of average allocation of each interval Bin in the area to be allocated by the following formula:

wherein (1)>For the average allocated frequency number, num_clip is the target frequency number, and M is the total number of intervals Bin in the area to be allocated;

and increasing the frequency number of the average allocation for the frequency number corresponding to each interval Bin in the area to be allocated.

8. The image processing method according to claim 2, wherein said performing frequency distribution on each Bin in the area to be distributed comprises:

calculating the frequency k allocated to the u-th interval Bin in the area to be allocated by the following formula _u ：

Wherein num_clip is the target frequency, M is the total number of intervals Bin in the area to be allocated, and d _u For the width, d, of the u-th interval Bin in the region to be allocated _j For the width of the jth interval Bin in the area to be allocated, u is 1-M,1≤j≤M；

increasing the frequency corresponding to the ith interval Bin in the area to be allocated by the k _u 。

9. The image processing method according to any one of claims 1 to 8, wherein the HDR image is synthesized based on an LDR image having an R Zhang Bao light ratio Ex, and the dynamic range of the LDR image having the exposure ratio Ex is D _L The dynamic range of the HDR image is based on Ex, R, D _L Determining, according to the dynamic range of the HDR image, selecting a control point in a coordinate system where the tone mapping curve is located, including:

selecting m control points in a coordinate system where the tone mapping curve is located according to the dynamic range of the HDR image; wherein, m=r-1, when the m control points are arranged from small to large according to the abscissa, the abscissa of the 1 st control point is D _L The abscissa of the ith control point is D _L *Ex ⁱ ，1＜i≤m。

10. The image processing method according to any one of claims 1 to 8, wherein the preset range is determined based on a target dynamic range, which is a specified dynamic range to which the HDR image needs to be compressed.

11. The image processing method according to any one of claims 2 to 8, wherein the target luminance histogram of the HDR image is obtained by:

acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image;

performing low-pass filtering on the original brightness histogram to obtain a filtered brightness histogram;

and acquiring a target luminance histogram of the HDR image according to the filtered luminance histogram.

12. The image processing method of claim 11, wherein the obtaining the target luminance histogram of the HDR image from the filtered luminance histogram comprises:

obtaining a target luminance histogram of the HDR image by:

Hist_new＝Hist_LPF*α+Hist*(1-α)；

13. The image processing method according to claim 11, wherein the obtaining the original luminance histogram of the HDR image from the luminance value of each pixel point in the HDR image includes:

acquiring an original luminance histogram of the HDR image according to the luminance value of each pixel point in the HDR image and a preset interval Bin with unequal width; wherein the width of the interval Bin representing the larger brightness value is larger.

14. The image processing method according to any one of claims 1 to 8, wherein the tone mapping curve includes: a global tone mapping curve and/or a local tone mapping curve for each pixel block in the HDR image; the first ordinate includes: an ordinate of the control point corresponding to the abscissa on the global tone mapping curve and/or an ordinate of the control point corresponding to the abscissa on the local tone mapping curve; the adjusted tone mapping curve includes: an adjusted global tone mapping curve and/or an adjusted local tone mapping curve.

15. The image processing method of claim 14, wherein the tone mapping curve after the adjustment comprises: under the condition of the adjusted global tone mapping curve and the adjusted local tone mapping curve, obtaining a low dynamic range LDR image corresponding to the HDR image according to the adjusted tone mapping curve, including:

obtaining a global brightness image according to the adjusted global tone mapping curve;

obtaining a local brightness image according to the adjusted local tone mapping curve;

and fusing the global brightness image and the local brightness image to obtain a low dynamic range LDR image corresponding to the HDR image.

16. The image processing method according to claim 15, wherein the obtaining the local luminance image according to the adjusted local tone mapping curve includes:

for an L-th pixel block in the HDR image, determining a pixel block in a neighborhood centered on the L-th pixel block; wherein 1.ltoreq.L.ltoreq.T, T being the total number of pixel blocks in the HDR image;

determining weights respectively corresponding to the pixel blocks in the neighborhood according to the brightness difference value of the pixel blocks in the neighborhood and the L-th pixel block;

According to the weights respectively corresponding to the pixel blocks in the neighborhood, performing smoothing treatment on the adjusted local tone mapping curve of the L-th pixel block to obtain a smoothed local tone mapping curve of the L-th pixel block;

and obtaining a local brightness image according to the local tone mapping curve after the smoothing treatment of the T pixel blocks in the HDR image.

17. The image processing method according to claim 16, wherein the smoothing the adjusted local tone mapping curve of the L-th pixel block according to the weights respectively corresponding to the pixel blocks in the neighborhood to obtain the smoothed local tone mapping curve of the L-th pixel block includes:

the smoothed local tone mapping curve of the L pixel block is obtained by the following formula:

18. The image processing method according to any one of claims 1 to 8, further comprising, after the obtaining the low dynamic range LDR image corresponding to the HDR image according to the adjusted tone mapping curve:

And carrying out contrast enhancement on the LDR image corresponding to the HDR image according to a preset contrast enhancement curve to obtain the LDR image with enhanced contrast.

19. The image processing method according to claim 18, wherein the performing contrast enhancement on the LDR image corresponding to the HDR image according to a preset contrast enhancement curve to obtain a LDR image with enhanced contrast, includes:

normalizing brightness values of all pixel points in the LDR image;

mapping the normalized brightness value of each pixel point of the LDR image according to a preset contrast enhancement curve, and outputting a brightness mapping image;

and restoring the dynamic range of the brightness mapping image to the dynamic range before normalization to obtain the LDR image with enhanced contrast.

20. The image processing method according to claim 18, wherein the contrast enhancement curve is a Gamma curve, and the parameter γ of the Gamma curve is determined by:

Th1_AVGL.ltoreq.avgL.ltoreq.Th2_AVGL whereinavgL is an average brightness value of the LDR image, th1_avgl is a first preset threshold, th2_avgl is a second preset threshold, and th2 > th1 is greater than or equal to 1.

21. The image processing method according to claim 18, wherein a slope of the contrast enhancement curve at a position where a luminance value is smaller than a preset luminance value is smaller than 1.

22. A chip, wherein the chip is located in an electronic device and is connected to a memory in the electronic device, the memory storing instructions executable by the chip to enable the chip to perform the image processing method of any one of claims 1 to 21.

23. An electronic device, comprising: the chip of claim 22, and a memory coupled to said chip.

24. A computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the image processing method of any one of claims 1 to 21.