WO2022041287A1

WO2022041287A1 - Image acquisition method and apparatus, device, and computer-readable storage medium

Info

Publication number: WO2022041287A1
Application number: PCT/CN2020/112740
Authority: WO
Inventors: 程敏; 胡彬林; 邓志鹏; 刘俊
Original assignee: 华为技术有限公司
Priority date: 2020-08-31
Filing date: 2020-08-31
Publication date: 2022-03-03
Also published as: CN114450934B; CN114450934A

Abstract

A processor of an electronic device acquires pixel values of at least two types of pixels in an image pixel array. The pixel values of the different types of pixels are generated by means of performing photography according to different brightness control parameters, and exposure times of the pixels are simultaneously ended in the same frame, thereby preserving both highlight details and shadow details. The processor acquires images corresponding to the respective brightness control parameters and fuses same, so as to acquire an improved dynamic range, and effectively reduce motion blur.

Description

Method, apparatus, device, and computer-readable storage medium for acquiring images

technical field

The present application relates to the technical field of image capturing, and in particular, to a method, apparatus, device, and computer-readable storage medium for acquiring an image.

Background technique

With the development of image capturing technology, the image quality is also getting higher and higher. When using a camera to shoot a scene with a wide dynamic range, there are often problems of overexposed highlights and overexposed dark areas, resulting in loss of details. Therefore, how to obtain an image in which highlights and dark details are simultaneously displayed to improve image quality is an urgent problem to be solved.

In the related art, in order to obtain an image in which highlights and dark details are simultaneously displayed, multiple exposures between frames are used, that is, a long exposure of the previous frame is used to obtain the details of the dark parts, and a short exposure of the next frame is used to obtain the highlight details. After that, the long-exposure image and the short-exposure image are fused through image processing technology to obtain an ultra-wide dynamic image, so that the final obtained image retains both highlight details and dark details.

However, the use of multiple exposures between frames will result in a difference in exposure time between frames. When shooting moving objects, there will be movement displacement between long and short exposures, resulting in the occurrence of motion trailing. Therefore, related technologies are used. The quality of the acquired images is still not good.

SUMMARY OF THE INVENTION

The present application provides a method, apparatus, device and computer-readable storage medium for acquiring an image, which can improve the quality of the acquired image and solve the problem of motion smearing.

In a first aspect, a method for acquiring an image is provided, and the method is applied to a processor of an electronic device, where the electronic device includes but is not limited to a device capable of acquiring images, such as a camera and a video camera. When acquiring an image, first, the processor acquires the pixel values of at least two types of pixel points in the image pixel array. In the image, the pixel values of different types of pixel points are generated according to different brightness control parameters. The class pixels are all black and white pixels, or the at least two types of pixels are all color pixels. Afterwards, the processor obtains images corresponding to each brightness control parameter according to the pixel values of at least two types of pixel points; and fuses the images corresponding to each brightness control parameter.

Based on the method for acquiring an image, since the same type of pixel in the image pixel array is divided into at least two types, each type is photographed with different brightness control parameters, so that pixels at different positions in the same frame use different Shooting with brightness control parameters can retain both highlight details and dark details to achieve a higher dynamic range. In addition, although the pixels at different positions are shot with different brightness control parameters, they all end the exposure at the same time in the same frame, and the images corresponding to the fused different brightness control parameters are also obtained based on the same frame of image, so it can be Minimize motion smearing.

Exemplarily, the brightness control parameters include but are not limited to exposure duration and gain, and different brightness control parameters refer to the same exposure duration and different gains; or, different brightness control parameters refer to different exposure durations and the same gain; or, different brightness control parameters. The brightness control parameter means that the exposure time is different, and the gain is also different. In the case of different exposure durations, it is necessary to end the exposure at the same time. That is to say, when shooting according to different brightness control parameters, the exposure start time can be different, but the exposure end time needs to be the same, so as to ensure that the images captured according to different brightness control parameters are the same frame image.

In a possible implementation manner, the at least two types of pixel points include a first type of pixel point and a second type of pixel point, the pixel value of the first type of pixel point is generated according to the first brightness control parameter, and the second type of pixel point The pixel value is photographed and generated according to the second brightness control parameter;

Obtaining an image corresponding to each brightness control parameter according to the pixel values of at least two types of pixels includes: obtaining a first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel, and obtaining a first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel; Acquiring a second image corresponding to the second brightness control parameter; fusing the images corresponding to each brightness control parameter includes: fusing the first image and the second image.

In a possible implementation manner, acquiring the first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel in an interpolation manner includes: determining the second type of pixel according to the pixel value of the first target pixel The first cross-correlation interpolation of the point under the first brightness control parameter, the first image is obtained according to the pixel value of the first type of pixel point and the first cross-correlation interpolation, and the first target pixel point includes pixels adjacent to the second type of pixel points. The first type of pixel point;

Acquiring the second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel includes: determining the second cross-correlation of the first type of pixel under the second brightness control parameter according to the pixel value of the second target pixel Interpolation, obtaining a second image according to the pixel value of the second type of pixel point and the second cross-correlation interpolation, and the second target pixel point includes the second type of pixel point adjacent to the first type of pixel point.

In a possible implementation manner, determining the first cross-correlation interpolation value of the second type of pixel point under the first brightness control parameter according to the pixel value of the first target pixel point includes: determining according to the pixel value of the second type of pixel point The weighted weight of the first target pixel point, the weighted weight of the first target pixel point is positively correlated with the difference between the second type of pixel point and the first target pixel point; the first cross-correlation interpolation of the pixel-like point under the first brightness control parameter;

Determining the second cross-correlation interpolation of the first type of pixels under the second brightness control parameter according to the pixel value of the second target pixel includes: determining the weighting weight of the second target pixel according to the pixel value of the first type of pixel, The weighted weight of the second target pixel is positively correlated with the difference between the first type of pixel and the second target pixel; according to the pixel value of the second target pixel and the weighted weight, the first type of pixel is determined in the second brightness control parameter The second cross-correlation interpolation under .

In a possible implementation manner, determining the first cross-correlation interpolation value of the second type of pixel point under the first brightness control parameter according to the pixel value of the first target pixel point and the weighted weight, including:

For any first target pixel, multiply the pixel value and weighted weight of any first target pixel to obtain a product result corresponding to any first target pixel; multiply the product result corresponding to each first target pixel Accumulate to obtain the first accumulated sum; accumulate the weighted weights of each first target pixel point to obtain the second accumulated sum; take the quotient of the first accumulated sum and the second accumulated sum as the second type of pixel point in the first brightness first cross-correlation interpolation under control parameters;

Taking α _2i as the weighted weight of the first target pixel point, W _2i as the pixel value of the first target pixel point, and N ₁ as the number of the first target pixel point as an example, according to the pixel value of the first target pixel point and the weighted weight Determining the first cross-correlation interpolation of the second type of pixel point under the first brightness control parameter includes: determining the second type of pixel point under the first brightness control parameter according to the pixel value of the first target pixel point and the weighted weight according to the following formula. The first cross-correlation interpolation W′ ₂ of :

The second cross-correlation interpolation of the first type of pixels under the second brightness control parameter is determined according to the pixel value of the second target pixel and the weighted weight, including:

For any second target pixel point, multiply the pixel value and weighted weight of any second target pixel point to obtain the product result corresponding to any second target pixel point; multiply the product result corresponding to each second target pixel point Accumulate to obtain the third cumulative sum; accumulate the weighted weights of each second target pixel point to obtain the fourth cumulative sum; take the quotient of the third cumulative sum and the fourth cumulative sum as the first type of pixel point in the second brightness The second cross-correlation interpolation under the control parameter.

For example, according to the pixel value of the second target pixel and the weighted weight, the second cross-correlation interpolation W′ ₁ of the first type of pixel under the second brightness control parameter is determined according to the following formula:

Wherein, α _1i is the weighted weight of the second target pixel, W _1i is the pixel value of the second target pixel, and N ₂ is the number of the second target pixel.

In a possible implementation manner, the second brightness control parameter is smaller than the first brightness control parameter, and fusing the first image and the second image includes:

Perform brightness alignment on the pixel value of the first type of pixel point and the second cross-correlation interpolation according to the first brightness control parameter to obtain a first brightness alignment result, and fuse the pixel value of the first type of pixel point and the first brightness alignment result;

Perform luminance alignment on the pixel values of the second type of pixel points and the first cross-correlation interpolation according to the first luminance control parameter to obtain a second luminance alignment result; and fuse the first cross-correlation interpolation and the second luminance alignment result.

Through the process of brightness alignment, the brightness smoothing algorithm is used to achieve the imaging target that the highlight area is not exposed and the dark area is not too dark, so as to keep the clarity of the final image obtained without loss.

In a possible implementation manner, the pixel value of the first type of pixel point and the second cross-correlation interpolation value are aligned according to the first luminance control parameter to obtain the first luminance alignment result, including:

According to the difference multiple between the first brightness control parameter and the second brightness control parameter, the pixel value of the first type of pixel point and the second cross-correlation interpolation are brightness aligned according to the first brightness control parameter, and the first brightness alignment result is obtained, The difference multiple is obtained based on the comprehensive difference between the first brightness control parameter and the second brightness control parameter;

Exemplarily, according to the following formula, according to the difference multiple between the first brightness control parameter and the second brightness control parameter, the pixel value of the first type of pixel point and the second cross-correlation interpolation are brightness aligned according to the first brightness control parameter, Get the first brightness alignment result

Wherein, K is the difference multiple of the first brightness control parameter and the second brightness control parameter, BL is the value of the black level, and W′ ₂ is the second cross-correlation interpolation.

Perform luminance alignment on the pixel values of the second type of pixel points and the first cross-correlation interpolation according to the first luminance control parameter to obtain a second luminance alignment result, including: according to the difference between the first luminance control parameter and the second luminance control parameter Multiples, the pixel values of the second type of pixels and the first cross-correlation interpolation are brightness aligned according to the first brightness control parameter to obtain a second brightness alignment result.

Exemplarily, according to the following formula, the pixel value of the second type of pixel point and the first cross-correlation interpolation are brightness aligned according to the first brightness control parameter, and the second brightness alignment result is obtained:

Among them, W ₂ is the pixel value of the second type of pixel point.

In a possible implementation manner, fusing the pixel value of the first type of pixel point and the first brightness alignment result includes:

Multiply the pixel value of the first type of pixel point and the corresponding weighted weight to obtain the first product result, multiply the first brightness alignment result and the corresponding weighted weight to obtain the second product result; Sum the results of the square product to obtain the first sum value; sum the weighted weights corresponding to the first type of pixels and the weighted weights corresponding to the first brightness alignment result to obtain the second sum value; add the first sum value to the second sum value The quotient of the values is used as the fusion result of the pixel value of the first type of pixel point and the first luminance alignment result.

Exemplarily, according to the pixel value of the first type of pixel point and the corresponding weighted weight, as well as the first brightness alignment result and the corresponding weighted weight, the pixel value of the first type of pixel point and the first brightness alignment result are fused according to the following formula:

Among them, W _F1 is the pixel value W ₁ of the first type of pixel point and the first brightness alignment result

The fusion result of β ₁₁ is the first weighting weight, and β ₁₂ is the second weighting weight.

Fusion of the first cross-correlation interpolation and the second luminance alignment results, including:

Multiply the first cross-correlation interpolation and the corresponding weighted weight to obtain the third product result, multiply the second luminance alignment result and the corresponding weighted weight to obtain the fourth product result; multiply the third product result and the fourth product result Sum up to get the third sum value; sum the weighted weight corresponding to the second type of pixel point and the weighted weight corresponding to the second brightness alignment result to get the fourth sum value; calculate the quotient of the third sum value and the fourth sum value As the fusion result of the first cross-correlation interpolation and the second luminance alignment result.

For example, taking the weighting weight corresponding to the first cross-correlation interpolation as the third weighting weight and the weighting weight corresponding to the second luminance alignment result being the fourth weighting weight as an example, the first cross-correlation interpolation and the second luminance alignment result are fused according to the following formula :

Among them, W _F2 is the first cross-correlation interpolation W′ ₁ and the second luminance alignment result

, β ₂₁ is the third weighting weight, and β ₂₂ is the fourth weighting weight.

In a possible implementation manner, before fusing the pixel values of the first type of pixel points and the first brightness alignment result, the method further includes: based on the fact that W ₁ is less than the first excessive dark threshold, determining β ₁₁ as the first value, and β ₁₂ as the first value the second value, where the first value is greater than the second value; or, based on the fact that W ₁ is greater than or equal to the first over-dark threshold, determine that β ₁₁ is the second value and β ₁₂ is the first value;

Before fusing the first cross-correlation interpolation and the second luminance alignment result, the method further includes: determining β ₂₁ as the second value and β ₂₂ as the first value based on W′ ₁ being greater than the first overexposure threshold; or, based on W′ ₁ being less than the first overexposure threshold Equal to the first overexposure threshold, determine β ₂₁ as the first value and β ₂₂ as the second value.

In a possible implementation manner, before fusing the pixel values of the first type of pixels and the first brightness alignment result, the method further includes: based on the fact that W ₁ is less than the second dark threshold, determining that β ₁₁ is the first value, and β ₁₂ is the first value. the second value, the first value is greater than the second value; or, based on W ₁ greater than the second overexposure threshold, determine β ₁₁ to be the second value, and β ₁₂ to be the first value; or, based on W ₁ between the second overexposure threshold Between the threshold and the second overexposure threshold, it is determined that β ₁₁ starts from the first value and decreases as W ₁ increases, and β ₁₂ starts from the second value and increases as W ₁ increases;

Before fusing the results of the first cross-correlation interpolation and the second brightness alignment, the method further includes: based on the fact that W′ ₁ is smaller than the second over-darkness threshold, determining that β ₂₁ is a first value, β ₂₂ is a second value, and the first value is greater than the second value ; or, based on W′ ₁ being greater than the second overexposure threshold, determine the second value of β ₂₁ , and β ₂₂ as the first value; or, based on W′ ₁ being between the second overexposure threshold and the second overexposure threshold, It is determined that β ₂₁ decreases from the first value as W' ₁ increases, and β ₂₂ increases from the second value as W' ₁ increases.

In a possible implementation, the method is applied to a processor of an electronic device, and the electronic device further includes an image sensor and at least two control circuits, and the at least two control circuits are used to control the image sensor to shoot according to different brightness control parameters Generate pixel values in different classes of pixels.

In a second aspect, an apparatus for acquiring an image is provided, the apparatus comprising:

The first acquisition module is used to acquire the pixel values of at least two types of pixel points in the image pixel array. In the image, the pixel values of different types of pixel points are generated according to different brightness control parameters, and at least two types of pixel points are generated. All are black and white pixels, or at least two types of pixels are color pixels;

a second acquisition module, configured to acquire an image corresponding to each brightness control parameter according to pixel values of at least two types of pixel points;

The fusion module is used to fuse the images corresponding to each brightness control parameter.

a second obtaining module, configured to obtain a first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel, and obtain a second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel;

The fusion module is used to fuse the first image and the second image.

In a possible implementation manner, the second obtaining module is configured to determine the first cross-correlation interpolation value of the second type of pixel under the first brightness control parameter according to the pixel value of the first target pixel, and according to the first type of pixel The first image is obtained by interpolating the pixel value of the point and the first cross-correlation, and the first target pixel includes the first-type pixel point adjacent to the second-type pixel point;

The second acquisition module is configured to determine the second cross-correlation interpolation value of the first type of pixel point under the second brightness control parameter according to the pixel value of the second target pixel point, according to the pixel value of the second type of pixel point and the second cross-correlation value The second image is obtained by interpolation, and the second target pixel point includes a second type of pixel point adjacent to the first type of pixel point.

In a possible implementation manner, the second acquisition module is configured to determine the weighted weight of the first target pixel point according to the pixel value of the second type of pixel point, and the weighted weight of the first target pixel point is related to the second type of pixel point and The difference of the first target pixel is positively correlated; the first cross-correlation interpolation of the second type of pixel under the first brightness control parameter is determined according to the pixel value of the first target pixel and the weighted weight;

The second acquisition module is configured to determine the weighted weight of the second target pixel point according to the pixel value of the first type of pixel point, and the weighted weight of the second target pixel point is positively related to the difference between the first type of pixel point and the second target pixel point Correlation; determining the second cross-correlation interpolation value of the first type of pixels under the second brightness control parameter according to the pixel value of the second target pixel and the weighted weight.

In a possible implementation manner, the second obtaining module is configured to multiply the pixel value of any first target pixel and the weighted weight for any first target pixel to obtain any first target pixel Corresponding product results; the product results corresponding to each first target pixel are accumulated to obtain a first cumulative sum; the weighted weights of each first target pixel are accumulated to obtain a second cumulative sum; the first cumulative sum and The quotient of the second accumulated sum is used as the first cross-correlation interpolation of the second type of pixels under the first brightness control parameter;

The second obtaining module is used for multiplying the pixel value of any second target pixel and the weighted weight for any second target pixel to obtain a product result corresponding to any second target pixel; The product results corresponding to the target pixels are accumulated to obtain the third accumulated sum; the weighted weights of each second target pixel are accumulated to obtain the fourth accumulated sum; the quotient of the third accumulated sum and the fourth accumulated sum is taken as the first The second cross-correlation interpolation of the pixel-like points under the second luminance control parameter.

In a possible implementation manner, the second brightness control parameter is smaller than the first brightness control parameter, and the fusion module is configured to perform brightness alignment between the pixel value of the first type of pixel and the second cross-correlation interpolation value according to the first brightness control parameter , obtain the first brightness alignment result, fuse the pixel value of the first type of pixel point and the first brightness alignment result; align the pixel value of the second type of pixel point and the first cross-correlation interpolation according to the first brightness control parameter. The second luminance alignment result; fusion of the first cross-correlation interpolation and the second luminance alignment result.

In a possible implementation manner, the fusion module is configured to interpolate the pixel value of the first type of pixel and the second cross-correlation interpolation according to the difference multiple between the first brightness control parameter and the second brightness control parameter according to the first The brightness control parameters are brightness aligned to obtain a first brightness alignment result, and the difference multiple is obtained based on the comprehensive difference between the first brightness control parameter and the second brightness control parameter;

The fusion module is configured to perform brightness alignment on the pixel value of the second type of pixel point and the first cross-correlation interpolation according to the first brightness control parameter according to the difference multiple between the first brightness control parameter and the second brightness control parameter, to obtain the first brightness control parameter. Two luminance alignment results.

In a possible implementation manner, the fusion module is used to multiply the pixel values of the first type of pixel points and the corresponding weighted weights to obtain a first product result, and multiply the first brightness alignment result and the corresponding weighted weights , obtain the second product result; sum the first product result and the second product result to obtain the first sum value; sum the weighting weight corresponding to the first type of pixel point and the weighting weight corresponding to the first brightness alignment result to obtain The second sum value; the quotient of the first sum value and the second sum value is used as the fusion result of the pixel value of the first type of pixel point and the first brightness alignment result;

The fusion module is used to multiply the first cross-correlation interpolation and the corresponding weighted weight to obtain the third product result, and multiply the second brightness alignment result and the corresponding weighted weight to obtain the fourth product result; Sum the result of the fourth product to obtain the third sum value; sum the weighted weight corresponding to the second type of pixel point and the weighted weight corresponding to the second brightness alignment result to obtain the fourth sum value; combine the third sum value with the first The quotient of the four-sum value is used as the fusion result of the first cross-correlation interpolation and the second luminance alignment result.

In a possible implementation manner, the fusion module is further configured to determine that β ₁₁ is a first value, β ₁₂ is a second value, and the first value is greater than the second value based on the fact that W ₁ is smaller than the first over-dark threshold; or, Based on W ₁ being greater than or equal to the first over-dark threshold, determine that β ₁₁ is the second value, and β ₁₂ is the first value;

The fusion module is further configured to determine that β ₂₁ is the second value and β ₂₂ is the first value based on the fact that W′ ₁ is greater than the first overexposure threshold; or, based on the fact that W′ ₁ is less than or equal to the first overexposure threshold, determine that β ₂₁ is The first value, β ₂₂ is the second value.

In a possible implementation manner, the fusion module is further configured to determine that β ₁₁ is the first value, β ₁₂ is the second value, and the first value is greater than the second value based on the fact that W ₁ is less than the second over-dark threshold; or, Based _on W ₁ being greater than the second overexposure threshold, it is determined that β ₁₁ is the second value and β ₁₂ is the first value _; The first value starts to decrease with the increase of W ₁ , and β ₁₂ starts to increase with the increase of W ₁ from the second value;

The fusion module is further configured to determine that β ₂₁ is the first value, β ₂₂ is the second value, and the first value is greater than the second value based on W′ ₁ being less than the second over-dark threshold; or, based on W′ ₁ being greater than the second over-dark threshold exposure threshold, determine the second value of β ₂₁ and β ₂₂ as the first value; or, based on W′ ₁ being between the second over-dark threshold and the second over-exposure threshold, determine that β ₂₁ starts from the first value and increases with W′ ₁ decreases and β ₂₂ increases from the second value as W′ ₁ increases.

In a third aspect, an electronic device is provided, the electronic device includes a processor, an image sensor, and at least two control circuits, where the at least two control circuits are used to control the image sensor to generate different types of pixels according to different brightness control parameters. The pixel value in ;

The processor is configured to perform the first aspect and the method for acquiring an image in any one of the first aspects.

In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising at least one instruction, the instruction being executed by a processor to implement the first aspect and any possible implementation manner of the first aspect of acquiring an image. method.

A computer program (product) is provided, the computer program (product) comprising: computer program code which, when executed by a computer, causes the computer to perform the methods in the above-mentioned aspects.

A chip is provided that includes a processor for invoking and executing instructions stored in the memory from a memory to cause a communication device on which the chip is mounted to perform the methods of the above aspects.

Description of drawings

1 is a structural diagram of an image pixel array provided by an embodiment of the present application;

2 is a structural diagram of an image pixel array provided by an embodiment of the present application;

3 is a structural diagram of an image pixel array provided by an embodiment of the present application;

4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 5 is a flowchart of a method for acquiring an image provided by an embodiment of the present application;

6 is a flowchart of a method for acquiring an image provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a process of acquiring an image according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a 3*3 pixel array provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of the relationship between the weighted weights corresponding to the first type of pixel points and the weighted weights corresponding to the first brightness alignment result provided by the embodiment of the present application;

FIG. 10 is a schematic structural diagram of an apparatus for acquiring an image provided by an embodiment of the present application.

detailed description

In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

With the development of image capture technology, more and more scenes with wider dynamic range are captured using cameras. When the camera shoots a scene with a wide dynamic range, it is prone to the problem of overexposure of highlights and overexposed dark areas, resulting in loss of details. For example, backlit face, flashlight effect in infrared fill light environment, strong light suppression in night road monitoring scene, overexposure of traffic lights in electric police scene, too dark face in mixed traffic scene, overexposed license plate, etc. Therefore, an effective WDR technology is needed to solve such problems and achieve the purpose of simultaneously revealing the details of highlights and shadows.

In this regard, an embodiment of the present application proposes a method for acquiring an image, which adopts multiple exposures within a frame, that is, different brightness control parameters are used for pixels in the same frame at different positions to achieve higher dynamic Scope. Since pixels at different positions are exposed simultaneously, motion smearing is minimized.

In order to implement different brightness control parameters for pixels at different positions in the frame, the embodiments of the present application improve the image sensor. Taking the image pixel array included in the color filter array (color filter array, CFA) image sensor shown in FIG. 1 as an example, the image pixel array includes two types of pixel points, one is a color pixel point, and the pixels of the color pixel point are Value dimension color value; one is black and white pixel point, black and white pixel point is also called grayscale pixel point, the pixel value of this black and white pixel point is gray value. R, G, and B shown in FIG. 1 are color pixels, and color information can be obtained by restoring the color pixels. W is a black and white pixel point, through which a black and white image can be obtained. Combining color and black and white results in a high-quality color image.

In an exemplary embodiment, based on the method provided by the embodiment of the present application, the same type of pixel in the image pixel array is divided into at least two types, and each type is photographed with different brightness control parameters, so as to achieve, within the same frame, Pixels at different positions are shot with different brightness control parameters to achieve a higher dynamic range. For example, black and white pixels are divided into at least two categories, or color pixels are divided into at least two categories.

Taking the image pixel array shown in FIG. 2 as an example, the black and white pixels are divided into two categories, one is the black and white pixels in odd rows, and the other is black and white pixels in even rows. Mark the black and white pixels of odd rows as W1, mark the black and white pixels of even rows as W2, and use the black and white pixels of odd rows (W1) and the black and white pixels of even rows (W2) to be controlled by different circuits to achieve the same configuration. Or shoot with different brightness control parameters. For example, W1 is controlled by control circuit 1, and W2 is controlled by control circuit 2. When the brightness control parameters of W1 and W2 are the same, it is a linear exposure mode, which is suitable for scenes with relatively uniform ambient lighting conditions. In this linear exposure mode, the brightness changes of odd and even rows of pixels are smooth. However, when the ambient lighting conditions are uneven and there is a relatively strong contrast between light and dark, it is necessary to configure different brightness control parameters for W1 and W2 to shoot to achieve wide dynamic imaging technology.

In addition to using the CFA array sensor and the control method shown in FIG. 2 , the method provided in this embodiment of the present application can also be applied to a bayer array sensor or other CFA array sensors, and it is not limited to setting different brightness by row. Control parameters, you can also set different brightness control parameters by row and column. Exemplarily, FIG. 3 shows a pixel array included in a CFA array sensor in which different brightness control parameters are set in rows and columns respectively. For example, the black and white pixels in the pixel array are divided into four categories, marked as W1, W2, W3 and W4 respectively. Among them, W1 is controlled by control circuit 1, W2 is controlled by control circuit 2, W3 is controlled by control circuit 3, and W4 is controlled by control circuit 4.

It should be noted that FIG. 2 and FIG. 3 are only for classifying black and white pixels and configuring different brightness control parameters as an example to illustrate, but are not used to limit the present application. The color pixels are divided into at least two types, and different brightness control parameters are configured for each type of color pixels. The process of applying the method provided by the embodiment of the present application for classifying based on color pixels is consistent with the process of applying the method provided by the embodiment of the present application for classifying based on black and white pixels. Be explained. In addition, the embodiments of the present application do not limit brightness control parameters, including but not limited to exposure parameters and gain, and exposure parameters include but are not limited to exposure duration.

Next, with reference to the description of the image pixel array shown in FIG. 2 or FIG. 3 , the method for acquiring an image provided by the embodiment of the present application will be described. Exemplarily, the execution body of the method may be an electronic device such as a camera, a video camera, etc. that can acquire images. As shown in FIG. 4 , the electronic device includes a lens 401 , an image sensor 402 , a control circuit 403 , a processor 404 and an encoder 405 . , the processor 404 is connected with the control circuit 403 and the image sensor 402 .

Wherein, the control circuit 403 is used to receive the feedback from the processor 404, and use at least two brightness control parameters to control the image sensor 402; the image sensor 402 is used to perform photoelectric conversion according to the control of the control circuit 403 to generate an image pixel array, Wherein, each brightness control parameter of the control circuit 403 corresponds to a type of pixel point.

There are at least two control circuits 403 , and the embodiment of the present application does not limit the number of the control circuits 403 . In a possible implementation manner, the electronic device includes at least two control circuits 403, and the at least two control circuits are used to control the image sensor 402 to generate pixel values in different types of pixel points by photographing according to different brightness control parameters. For example, when acquiring an image, light enters the lens 401, and the lens 401 transmits the light to the image sensor 402; the image sensor 402 performs photoelectric conversion to generate an original image. Since the pixels of the image sensor 402 are controlled by the control circuit 403, corresponding image pixels, that is, the pixel values of the pixels, are generated based on different brightness control parameters. The image directly generated by the image sensor 402 is in RAW format. According to the design of the image sensor 402, the format of the RAW image is also different. For example, it can be Bayer RGGB, RYYB, RCCC, RCCB, RGBW, CMYW and other formats.

The image sensor 402 sends the initial image to the processor 404, which may have an ISP function, and the processor 404 performs image signal processing. For example, convert RAW images of various formats to RGB format. You can also convert RAW format to YUV format, or HSV format, Lab format, CMY format, YCbCr format. These formats may be referred to as raw media data formats. In addition, the processor 404 applies the method provided by the embodiment of the present application to perform processing such as interpolation, and controls the control circuit 403 .

The encoder 405 encodes (that is, compresses) the image, and the format of the generated image or video may be: jpeg format, bmp format, tga format, png format, and gif format. Video media formats such as: MPEG format, AVI format, nAVI format, ASF format, MOV format, WMV format, 3GP format, RM format, RMVB format, FLV/F4V format, H.264 format, H.265 format.

In an exemplary embodiment, the processor 404 and the encoder 405 may be integrated together. For example, HiSilicon

chip

It is a chip that integrates the functions of the processor 404 and the encoder 405 .

Based on the above-mentioned electronic device shown in FIG. 4 , the method is applied to the processor of the electronic device. Referring to FIG. 5 , the method provided by the embodiment of the present application includes the following processes.

In step 501, the processor of the electronic device obtains the pixel values of at least two types of pixel points in the image pixel array. In the image, the pixel values of different types of pixel points are generated according to different brightness control parameters. The dots are all black and white pixels, or the at least two types of pixels are all color pixels.

In the embodiment of the present application, in order to implement different brightness control parameters for pixels at different positions in the frame, the same type of pixel in the image pixel array is divided into at least two types, and the at least two types of pixels may be black and white pixels. , or, the at least two types of pixel points are all color pixel points. In addition, the pixel values of the at least two types of pixel points are generated according to different brightness control parameters, and the brightness control parameters can be controlled by the control circuit.

Since the electronic device includes an image sensor, a control circuit and a processor, and the processor is connected to the image sensor and the control circuit respectively, the processor can not only read the pixel values of the pixel points in the pixel array included in the image sensor, but also pass the control circuit Determine the brightness control parameters corresponding to the pixels.

The embodiments of the present application do not limit the number of at least two types of pixel points into which the same type of pixel point is divided, which may be determined based on the configuration of the electronic device. For example, if the black and white pixels are divided into two types of pixel points W1 and W2 as shown in FIG. 2, the processor obtains the pixel values of the two types of black and white pixel points W1 and W2. If the black and white pixels are divided into four types of pixels W1, W2, W3 and W4 as shown in FIG. 3, the processor obtains the pixel values of the four types of black and white pixels W1, W2, W3 and W4.

In addition, the brightness control parameters include but are not limited to exposure duration and gain. Different brightness control parameters refer to the same exposure duration and different gains; or, different brightness control parameters refer to different exposure durations and the same gain; or, different brightness control parameters The parameter means that the exposure time is different, and the gain is also different.

In the case of different exposure durations, it is necessary to end the exposure at the same time. That is to say, when shooting according to different brightness control parameters, the exposure start time can be different, but the exposure end time needs to be the same, so as to ensure that the images captured according to different brightness control parameters are the same frame of image.

Step 502, the processor of the electronic device obtains images corresponding to each brightness control parameter according to the pixel values of the at least two types of pixel points.

Since the pixel values of the at least two types of pixel points are generated according to different brightness control parameters, an image can be obtained from the pixel values of the pixel points generated according to each brightness control parameter. Exemplarily, taking the brightness control parameter including the exposure parameter, and the exposure parameter being the exposure duration as an example, images with different display effects can be obtained with different exposure durations. For example, taking the two brightness control parameters as long exposure and short exposure as an example, long exposure can obtain dark detail information, and short exposure can obtain highlight detail information. Therefore, the electronic device acquires an image with dark detail information and an image with highlight detail information.

It should be noted that, although the at least two types of pixel points are not all the pixel points in the image pixel array, but one of the pixel points, the embodiment of the present application obtains the pixel value according to the pixel value of the at least two types of pixel points. The image corresponding to each brightness control parameter is a complete image, and the number of pixels in the image is consistent with the number of pixels in the image pixel array. Since the pixel value of each type of pixel point in the at least two types of pixel points is generated according to the corresponding brightness control parameters, it does not have pixel values generated according to other brightness control parameters. For example, taking the image pixel array shown in FIG. 3 as an example, the pixel value of W1 controlled by the control circuit 1 is generated according to the brightness control parameter 1 corresponding to W1, and the W1 does not have the brightness control parameter corresponding to W3. 3 Generated pixel values. Similarly, the pixel value of W3 controlled by the control circuit 3 is generated according to the brightness control parameter 3 corresponding to W3, and the W3 does not have the pixel value generated according to the brightness control parameter 1 corresponding to W1.

Therefore, in order to obtain a complete image corresponding to each brightness control parameter, when obtaining the image corresponding to any brightness control parameter, it can be determined according to the pixel value of the pixel corresponding to any brightness control parameter that no brightness corresponds to any one of the brightness control parameters. Controls the pixel values of other pixels of the parameter. The embodiment of the present application does not limit the manner of determining the pixel values of other pixel points that do not correspond to any brightness control parameter according to the pixel value of the pixel point corresponding to any brightness control parameter. For example, it can be implemented by means of interpolation or by means of mean value.

Still taking the brightness control parameter corresponding to W1 in FIG. 3 as brightness control parameter 1 as an example, when acquiring the image corresponding to the brightness control parameter 1, since the pixel value of W1 is generated according to the brightness control parameter 1, and W2, W3 and W4 do not have pixel values generated according to the brightness control parameter 1, so the pixel values of W2, W3 and W4 under the brightness control parameter 1 can be determined according to the pixel value of W1. Therefore, each black and white pixel in the image pixel array has a pixel value corresponding to the brightness control parameter 1, thereby generating a black and white image corresponding to the brightness control parameter 1.

Similarly, for the brightness control parameter 3 corresponding to W3, when the image corresponding to the brightness control parameter 3 is obtained, since the pixel value of W3 is generated according to the brightness control parameter 3, and W1, W2 and W4 do not have the basis of The pixel value generated by the brightness control parameter 3, therefore, the pixel values of W1, W2 and W4 under the brightness control parameter 3 can be determined according to the pixel value of W3. Therefore, each black and white pixel in the image pixel array has a pixel value corresponding to the brightness control parameter 3 , thereby generating a black and white image corresponding to the brightness control parameter 3 .

Step 503 , the processor of the electronic device fuses the images corresponding to each brightness control parameter.

After the images corresponding to each brightness control parameter are obtained, the method of fusing the images corresponding to each brightness control parameter is not limited in this embodiment of the present application.

Based on the method for acquiring an image provided by the embodiment of the present application, based on the method for acquiring an image, since the same type of pixel in the image pixel array is divided into at least two types, each type is shot with different brightness control parameters, so as to achieve the same The pixels in the frame and at different positions are shot with different brightness control parameters, which can retain both highlight details and dark details to achieve a higher dynamic range. In addition, although the pixels at different positions are shot with different brightness control parameters, the exposure is ended at the same time in the frame, and the images corresponding to the different brightness control parameters fused are also obtained based on the same frame image, so the maximum Minimize motion smearing.

For ease of understanding, the method provided by this embodiment of the present application is illustrated by taking the same pixel point in the image pixel array as an example of being divided into two types of pixel points, in conjunction with the image pixel array shown in FIG. 2 . Still taking the execution body of the method as an example of a processor of an electronic device, referring to FIG. 6 , the method provided by the embodiment of the present application includes the following processes.

In step 601, the processor of the electronic device obtains the pixel value of the first type of pixel point and the pixel value of the second type of pixel point in the image pixel array, the pixel value of the first type of pixel point is captured and generated according to the first brightness control parameter, and the first type of pixel point is photographed and generated. The pixel values of the second type of pixel points are photographed and generated according to the second brightness control parameter.

In an exemplary embodiment, the at least two control circuits included in the electronic device are a first control circuit and a second control circuit; the first control circuit is used to control the image sensor to convert light into an electrical signal according to the first brightness control parameter to capture the image The pixel value of the first type of pixel point; the second control circuit is used to control the image sensor to convert light into an electrical signal according to the second brightness control parameter to obtain the pixel value of the second type of pixel point.

Wherein, the first type of pixel points and the second type of pixel points are both black and white pixel points, or, the first type of pixel points and the second type of pixel points are both color pixel points. The values of the first brightness control parameter and the second brightness control parameter are different, and the embodiments of the present application do not limit the values of the first brightness control parameter and the second brightness control parameter.

Both the first brightness control parameter and the second brightness control parameter include, but are not limited to, exposure duration and gain. The exposure duration of the first brightness control parameter and the second brightness control parameter are the same, but the gain is different; The exposure duration of the brightness control parameter is different, and the gain is the same; or, the exposure duration of the first brightness control parameter and the second brightness control parameter are different, and the gain is also different. In the case of different exposure durations, it is necessary to end the exposure at the same time. That is to say, when shooting according to the first brightness control parameter and the second brightness control parameter, the exposure start time can be different, but the exposure end time needs to be the same, so as to ensure the image obtained according to the first brightness control parameter and the second brightness control parameter. is the same frame image.

Step 602: The processor of the electronic device obtains a first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel, and obtains a second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel.

Although the first type of pixel point and the second type of pixel point are not all the pixel points in the image pixel array, but one of the pixel points, the first type of pixel point is obtained according to the pixel value of the first type of pixel point in the embodiment of the present application. The first image corresponding to a brightness control parameter and the second image corresponding to the second brightness control parameter obtained according to the pixel values of the second type of pixels are both complete images. The number of pixels in the first image and the second image All are consistent with the number of pixels in the image pixel array. Part of the values of the pixels in the first image are obtained directly from the image pixel array, and the remaining part is calculated from the directly obtained values of this part of the pixels. Similarly, a part of the values of the pixel points in the second image is obtained directly from the image pixel array, and the remaining part is calculated from the directly obtained values of this part of the pixel points. That is, the first image and the second image are initial images, and the number of pixels in the initial image is the same as the number of pixels in the image pixel array. The image pixel array is an image in RAW format, and the first and second images are also in the same format. Since the pixel value of the first type of pixel point is generated by shooting according to the first brightness control parameter, the first type of pixel point does not have the pixel value generated by shooting according to the second brightness control parameter. The pixel value of the second type of pixel point is generated by shooting according to the second brightness control parameter, and the second type of pixel point does not have the pixel value generated by shooting according to the first brightness control parameter.

Therefore, in order to obtain the first image corresponding to the first brightness control parameter, when obtaining the first image corresponding to the first brightness control parameter, the pixel value of the first type of pixel corresponding to the first brightness control parameter can be used to determine the The pixel value of the second type of pixel point corresponding to the first brightness control parameter. The embodiment of the present application does not limit the manner of determining the pixel value of the second type of pixel point not corresponding to the first brightness control parameter according to the pixel value of the first type of pixel point corresponding to the first brightness control parameter. For example, it can be implemented by means of mean value or by means of interpolation.

Taking the mean value method as an example, when determining the pixel value of the second type pixel point under the first brightness control parameter, determine the average value of the pixel value of each first type pixel point adjacent to the second type pixel point, and use The average value is used as the pixel value of the second type of pixel point under the first brightness control parameter. As shown in Figure 2, the pixel value of any W1 is generated according to the first brightness control parameter, and the pixel value of any W2 is generated according to the second brightness control parameter. For a pixel value under a brightness control parameter, the pixel values of four W1 adjacent to the W2 are averaged, and the obtained average value is used as the pixel value of the W2 under the first brightness control parameter.

Optionally, in addition to determining the average value of the pixel values of each pixel point of the first type adjacent to the pixel point of the second type, the average value is taken as the pixel value of the pixel point of the second type under the first brightness control parameter. In addition, the average value of the pixel values of all the pixel points of the first type can also be determined, and the pixel value of each pixel point of the second type under the first brightness control parameter is set as the average value.

For the interpolation method, in an exemplary embodiment, acquiring the first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel point includes: determining the second type of pixel according to the pixel value of the first target pixel point The first cross-correlation interpolation of the point under the first brightness control parameter; the first image is obtained according to the pixel value of the first type of pixel point and the first cross-correlation interpolation, and the first target pixel point includes pixels adjacent to the second type of pixel point. The first type of pixels.

Taking the image acquisition process shown in Figure 7 as an example, W1 is the first type of pixel point. For ease of understanding, the pixel value of each first type of pixel point is recorded as W1, but it does not represent the value of each first type of pixel point. The pixel values are the same, and the pixel values of each first-type pixel point may be determined based on the current environment, and may be different pixel values, which are only for ease of understanding. Similarly, W2 is the second type of pixel point, and for ease of understanding, the pixel value of each second type of pixel point is denoted as W2. W2' is the pixel value of the second type of pixel by interpolation of the pixel value of the first target pixel under the first brightness control parameter, and the first target pixel is the first type of pixel adjacent to the second type of pixel. point.

For further understanding, please refer to the 3*3 pixel array shown in FIG. 8, in which the pixel points corresponding to each number 1 are the first type of pixel points, and the pixel value of the pixel point corresponding to each number 1 is based on the first brightness control parameter generate. The pixels corresponding to the number 2 in FIG. 8 are the second type of pixels, and the pixels corresponding to each number 1 are the first type of pixels adjacent to the pixels corresponding to the number 2, so the pixels corresponding to each number 1 are of the first type. The points are all first target pixel points, and the pixel value of the pixel point corresponding to the number 2 under the first brightness control parameter is obtained by interpolating the pixel value of the pixel point corresponding to each number 1.

The manner of obtaining the interpolation value is not limited in this embodiment of the present application. In an exemplary embodiment, determining the first cross-correlation interpolation value of the second type of pixel point under the first brightness control parameter according to the pixel value of the first target pixel point includes: determining the first cross-correlation interpolation value according to the pixel value of the second type of pixel point The weighted weight of the target pixel point, the weighted weight of the first target pixel point is positively correlated with the difference between the second type of pixel point and the first target pixel point; the second type of pixel is determined according to the pixel value of the first target pixel point and the weighted weight. Click the first cross-correlation interpolation under the first brightness control parameter.

The embodiment of the present application does not limit the method of determining the weighted weight of the first target pixel point according to the pixel value of the second type of pixel point. The weighted weight of the first target pixel point is related to the second type of pixel point and the first target pixel The difference size of the points is positively correlated, including but not limited to mapping the difference between the second type of pixel point and the first target pixel point as the weighted weight of the first target pixel point, the greater the difference between the second type of pixel point and the first target pixel point. is smaller, the smaller the weighted weight of the first target pixel point, the greater the difference between the second type of pixel point and the first target pixel point, the greater the weighted weight of the first target pixel point.

For example, still taking the 3*3 pixel array shown in FIG. 7 as an example, the pixel point corresponding to each number 1 is used as the first target pixel point, and the weighted weight of the pixel point corresponding to each number 1 passes through the pixel point corresponding to each number 1 The size of the difference between the pixels corresponding to the number 2 is determined.

Exemplarily, determining the first cross-correlation interpolation of the second type of pixel under the first brightness control parameter according to the pixel value of the first target pixel and the weighted weight, including: for any first target pixel, determining the The pixel value of a first target pixel is multiplied by the weighted weight of the first target pixel to obtain a product result corresponding to any one of the first target pixels. After that, the product results corresponding to each first target pixel are accumulated to obtain a first accumulated sum. The weighted weights of each first target pixel are accumulated to obtain a second accumulated sum. The quotient of the first accumulated sum and the second accumulated sum is used as the first cross-correlation interpolation of the second type of pixels under the first brightness control parameter.

For example, taking α _2i as the weighted weight of the first target pixel, W _2i as the pixel value of the first target pixel, and N ₁ as the number of the first target pixel, according to the pixel value of the first target pixel and The weighted weight determines the first cross-correlation interpolation of the second type of pixel point under the first brightness control parameter, including: according to the pixel value of the first target pixel point and the weighted weight, according to the following formula, determine the second type of pixel point in the first brightness control The first cross-correlation interpolation W′ ₂ under the parameter:

It should be noted that the method of acquiring the second image is the same in principle as the method of acquiring the first image, and reference may be made to the above-mentioned process of acquiring the first image. In an exemplary embodiment, acquiring the second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel includes: determining, according to the pixel value of the second target pixel, whether the first type of pixel is in the second brightness control For the second cross-correlation interpolation under the parameters, a second image is obtained according to the pixel value of the second-type pixel point and the second cross-correlation interpolation, and the second target pixel point includes the second-type pixel point adjacent to the first-type pixel point.

Wherein, determining the second cross-correlation interpolation of the first type of pixel under the second brightness control parameter according to the pixel value of the second target pixel includes but not limited to: determining the second target pixel according to the pixel value of the first type of pixel The weighted weight of the point, the weighted weight of the second target pixel point is positively correlated with the difference between the first type of pixel point and the second target pixel point; The second cross-correlation interpolation under the second brightness control parameter.

Exemplarily, determining the second cross-correlation interpolation of the first type of pixel under the second brightness control parameter according to the pixel value of the second target pixel and the weighted weight, including: for any second target pixel, adding any Multiply the pixel value of the second target pixel and the weighted weight to obtain a product result corresponding to any second target pixel; accumulate the product results corresponding to each second target pixel to obtain a third cumulative sum; The weighted weights of the two target pixels are accumulated to obtain a fourth accumulated sum; the quotient of the third accumulated sum and the fourth accumulated sum is used as the second cross-correlation interpolation of the first type of pixels under the second brightness control parameter.

Step 603, the processor of the electronic device fuses the first image and the second image.

Since the first image is an image corresponding to the first brightness control parameter, and the second image is an image corresponding to the second brightness control parameter, in order to obtain the same frame of image, in an exemplary embodiment, the first image and the second image are fused. For example, the pixel value of each pixel point of the first image is fused with the pixel value of each pixel point corresponding to the second image. The embodiment of the present application does not limit the manner of fusing the first image and the second image. In an exemplary embodiment, the method provided by the embodiment of the present application further includes a process of aligning the brightness of the first image and the second image, so as to Through the brightness smoothing algorithm, the imaging target of not exposing the bright area and not being too dark in the dark area is realized, so as to keep the clarity of the final image obtained without loss. Exemplarily, taking the second brightness control parameter smaller than the first brightness control parameter as an example, the method for fusing the first image and the second image provided by the embodiments of the present application includes but is not limited to the following processes 603A1 to 603A4.

603A1. Perform luminance alignment on the pixel value of the first type of pixel point and the second cross-correlation interpolation value according to the first luminance control parameter, to obtain a first luminance alignment result.

Because the pixel value of the first type of pixel is generated according to the first brightness control parameter, and the second cross-correlation interpolation is also the value corresponding to the first type of pixel, but the second cross-correlation interpolation is under the second brightness control parameter. value of . That is to say, for the same first type of pixel point, there is not only a pixel value generated according to the first brightness control parameter, but also a second mutual interpolation corresponding to the second brightness control parameter. Therefore, it is necessary to align the brightness of the two values of the same first-type pixel point. Since the smaller the brightness control parameter is, the darker the obtained image will be. In the case that the second brightness control parameter is smaller than the first brightness control parameter, the pixel value of the first type of pixel and the second cross-correlation according to the first brightness control parameter are adopted. Interpolate for luminance alignment.

In an exemplary embodiment, performing luminance alignment on the pixel value of the first type of pixel point and the second cross-correlation interpolation value according to the first luminance control parameter to obtain the first luminance alignment result, comprising: according to the first luminance control parameter and the second luminance alignment The difference multiple between the brightness control parameters, the pixel value of the first type of pixel point and the second cross-correlation interpolation are brightness aligned according to the first brightness control parameter, and the first brightness alignment result is obtained, and the difference multiple is based on the first brightness control parameter and The comprehensive difference between the second brightness control parameters is obtained.

Taking the brightness parameter including the gain and the exposure duration as an example, the first brightness control parameter includes the first gain and the first exposure duration, the second brightness control parameter includes the second gain and the second exposure duration, then the first brightness control parameter and the second The difference multiple between the brightness control parameters is the difference between the first product result and the second product result. The first product result is the product result of the first gain and the first exposure duration, and the second product result is the product result of the second gain and the second exposure duration.

603A2, fuse the pixel value of the first type of pixel point and the first brightness alignment result.

In an exemplary embodiment, fusing the pixel value of the first type of pixel point and the first brightness alignment result includes: multiplying the pixel value of the first type of pixel point and the corresponding weighted weight to obtain a first product result, A luminance alignment result and the corresponding weighted weight are multiplied to obtain a second product result. After that, sum the first product result and the second product result to obtain the first sum value. The weighted weight corresponding to the first type of pixel point and the weighted weight corresponding to the first brightness alignment result are summed to obtain a second sum value. The quotient of the first sum value and the second sum value is used as the fusion result of the pixel value of the first type of pixel point and the first brightness alignment result.

In an exemplary embodiment, before fusing the first image and the second image, it further includes a process of determining the weighting weight corresponding to the pixel points of the first type and determining the weighting weight corresponding to the first luminance alignment result. Including but not limited to the following two determination methods:

Determining Mode 1: Based on W ₁ being less than the first over-dark threshold, determine that β ₁₁ is the first value, β ₁₂ is the second value, and the first value is greater than the second value; or, based on W ₁ being greater than or equal to the first over-dark threshold, Determine β ₁₁ as the second value and β ₁₂ as the first value.

Illustratively, the first value is 1 and the second value is 0. If the pixel value of the first type of pixel is less than the first dark threshold, it means that the pixel value of the first type of pixel is not too dark. The weighted weight of is set to 0, then the pixel value of the pixel point at this position is based on the pixel value of the first type of pixel point. Similarly, if the pixel value of the first type of pixel point is greater than or equal to the first dark threshold, it means that the pixel value of the first type of pixel point is too dark, the weighted weight corresponding to the first type of pixel point is set to 0, and the first type of pixel point is set to 0. The weighted weight corresponding to the luminance alignment result is set to 1, and the pixel value of the pixel at this position is subject to the first luminance alignment result. The method of determining the weighting weight corresponding to the first type of pixel point and determining the weighting weight corresponding to the first brightness alignment result makes the image after fusing the pixel value of the first type pixel point and the first brightness alignment result except for overexposure and overexposure. Outside of dark areas, the sharpness is consistent with the linear exposure mode.

The second determination method is based on that W ₁ is smaller than the second over-dark threshold, and β ₁₁ is determined to be the first value, β ₁₂ is the second value, and the first value is greater than the second value; or, based on W ₁ greater than the second over-exposure threshold, determine β ₁₁ is the second value and β ₁₂ is the first value; or, based on the fact that W ₁ is between the second overdark threshold and the second overexposure threshold, it is determined that β ₁₁ decreases from the first value as W ₁ increases. small, β ₁₂ increases from the second value as W ₁ increases.

Illustratively, the first value is 1 and the second value is 0. In the second method, the relationship between the weighted weights corresponding to the first type of pixel points and the weighted weights corresponding to the first luminance alignment result is shown in FIG. 9 . If the pixel value of the first type of pixel point is less than the second dark threshold (corresponding to th1 in Figure 9), it means that the pixel value of the first type of pixel point is not too dark, and the weighting weight corresponding to the first type of pixel point is set to 1 , and the weighting weight corresponding to the first luminance alignment result is set to 0, then the pixel value of the pixel point at this position is based on the pixel value of the first type of pixel point. Similarly, if the pixel value of the first type of pixel point is greater than the second overexposure threshold (corresponding to th2 in Figure 9), it means that the pixel value of the first type of pixel point is overexposed, and the weighted weight corresponding to the first type of pixel point is is set to 0, and the weighting weight corresponding to the first luminance alignment result is set to 1, then the pixel value of the pixel at this position is subject to the first luminance alignment result. If the pixel value of the first type of pixel is between the second over-dark threshold and the second over-exposure threshold, it is determined that the weighting weight corresponding to the first type of pixel starts from 1 and decreases as the first cross-correlation interpolation increases, The weighting weight corresponding to the first luminance alignment result starts from 0 and increases as the first cross-correlation interpolation increases.

The weighting weight corresponding to the first type of pixel points and the weighting weight corresponding to the first brightness alignment result are determined by this determination method 2, so that the image after fusing the pixel value of the first type pixel point and the first brightness alignment result is slightly lower than linear The sharpness of the exposure mode, but it can effectively prevent the unsmooth problem caused by the poor linearity of the image sensor in the first method.

603A3: Perform luminance alignment on the pixel value of the second type of pixel point and the first cross-correlation interpolation value according to the first luminance control parameter, to obtain a second luminance alignment result.

Like the first type of pixels, since the pixel value of the second type of pixel is generated according to the second brightness control parameter, and the first cross-correlation interpolation is also the value corresponding to the second type of pixel, but the first cross-correlation Interpolation is the value under the first brightness control parameter. That is, for the same second type of pixel point, there is not only a pixel value generated according to the second brightness control parameter, but also a first mutual interpolation corresponding to the first brightness control parameter. Therefore, it is necessary to align the brightness of the two values of the same second-type pixel point. Since the smaller the brightness control parameter is, the darker the obtained image will be. When the second brightness control parameter is smaller than the first brightness control parameter, the first cross-correlation between the pixel value of the second type of pixel point and the first brightness control parameter is adopted according to the first brightness control parameter. Interpolate for luminance alignment.

In an exemplary embodiment, performing luminance alignment on the pixel values of the second type of pixel points and the first cross-correlation interpolation value according to the first luminance control parameter, to obtain a second luminance alignment result, includes: according to the first luminance control parameter and the second luminance alignment result. The difference multiple between the brightness control parameters, the pixel value of the second type of pixel point and the first cross-correlation interpolation are brightness aligned according to the first brightness control parameter, and the second brightness alignment result is obtained, and the difference multiple is based on the first brightness control parameter and The comprehensive difference between the second brightness control parameters is obtained.

Exemplarily, according to the following formula, the pixel value of the second type of pixel point and the first cross-correlation interpolation are brightness aligned according to the first brightness control parameter to obtain the second brightness alignment result:

Among them, W ₂ is the pixel value of the second type of pixel point.

603A4, fuse the first cross-correlation interpolation and the second luminance alignment result.

In an exemplary embodiment, fusing the first cross-correlation interpolation and the second luminance alignment result includes: multiplying the first cross-correlation interpolation and the corresponding weighted weight to obtain a third product result, and combining the second luminance alignment result with the corresponding The weighted weights are multiplied to obtain the fourth product result. After that, the third product result is summed with the fourth product result to obtain the third sum value. The weighted weight corresponding to the second type of pixel point and the weighted weight corresponding to the second brightness alignment result are summed to obtain a fourth sum value. The quotient of the third sum value and the fourth sum value is used as the fusion result of the first cross-correlation interpolation and the second luminance alignment result.

In an exemplary embodiment, before fusing the first cross-correlation interpolation and the second luminance alignment result, the method further includes: a process of determining a third weighting weight and determining a fourth weighting weight. Including but not limited to the following two determination methods:

Determining Mode 1: Based on W′ ₁ being greater than the first overexposure threshold, determine β ₂₁ as the second value and β ₂₂ as the first value; or, based on W′ ₁ being less than or equal to the first overexposure threshold, determine β ₂₁ as the first value value, β ₂₂ is the second value.

Illustratively, the first value is 1 and the second value is 0. If the first cross-correlation interpolation value is smaller than the first too dark threshold, it means that the first cross-correlation interpolation value is not too dark, and the third weighting weight corresponding to the first cross-correlation interpolation value is set to 1, and the weighting weight corresponding to the second luminance alignment result is set to If it is 0, the pixel value of the pixel at this position is subject to the first cross-correlation interpolation. Similarly, if the first cross-correlation interpolation is greater than or equal to the first overdark threshold, it means that the first cross-correlation interpolation is too dark, the weighting weight corresponding to the first cross-correlation interpolation is set to 0, and the weighting corresponding to the second luminance alignment result is set to 0. If the weight is set to 1, the pixel value of the pixel at this position is subject to the second luminance alignment result. The method of determining the third weighting weight corresponding to the first cross-correlation interpolation and determining the fourth weighting weight corresponding to the second luminance alignment result makes the image after fusing the first cross-correlation interpolation and the second luminance alignment result except for the overexposure and Consistent with the sharpness of the linear exposure mode, except in overly dark areas.

The second determination method is based on that W′ ₁ is less than the second over-dark threshold, and β ₂₁ is determined to be the first value, β ₂₂ is the second value, and the first value is greater than the second value; or, based on W′ ₁ greater than the second over-exposure threshold , determine the second value of β ₂₁ and β ₂₂ as the first value; or, based on the fact that W′ ₁ is between the second over-dark threshold and the second over-exposure threshold, determine that β ₂₁ increases with W′ ₁ from the first value β ₂₂ increases from the second value as W′ ₁ increases.

Illustratively, the first value is 1 and the second value is 0. In the second determination method, the relationship between the third weighting weight corresponding to the first cross-correlation interpolation and the fourth weighting weight corresponding to the second luminance alignment result may also be as shown in FIG. 9 . The second determination method is used to determine the third weighted weight and the fourth weighted weight, so that the image after the fusion of the first cross-correlation interpolation and the second brightness alignment result is slightly lower than the sharpness of the linear exposure mode, but the determination method can be effectively prevented. A non-smoothing problem due to poor linearity of the image sensor.

An embodiment of the present application provides an apparatus for acquiring an image, and the apparatus is configured to execute the method for acquiring an image shown in FIG. 5 and FIG. 6 through the modules shown in FIG. 10 . Referring to Figure 10, the device includes:

The first acquisition module 1001 is used to acquire the pixel values of at least two types of pixel points in the image pixel array. In the image, the pixel values of different types of pixel points are generated by shooting according to different brightness control parameters, and at least two types of pixels are generated. The points are all black and white pixels, or at least two types of pixels are color pixels; the functions performed by the first acquisition module 1001 can refer to 501 shown in FIG. 5 , and details are not repeated here.

The second acquisition module 1002 is configured to acquire images corresponding to each brightness control parameter according to the pixel values of at least two types of pixel points; the functions performed by the second acquisition module 1002 can refer to 502 shown in FIG. 5 , which will not be repeated here. .

The fusion module 1003 is used to fuse images corresponding to each brightness control parameter. For the functions performed by the fusion module 1003, reference may be made to 503 shown in FIG. 5, and details are not repeated here.

In an exemplary embodiment, at least two types of pixel points include a first type of pixel point and a second type of pixel point, the pixel value of the first type of pixel point is photographed and generated according to the first brightness control parameter, and the pixel value of the second type of pixel point is Shooting and generating according to the second brightness control parameter;

A second obtaining module 1002, configured to obtain a first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel, and obtain a second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel; For example, refer to the related description of 602 shown in FIG. 6 , which will not be repeated here.

The fusion module 1003 is used to fuse the first image and the second image. For example, refer to the related description of 603 shown in FIG. 6 , which will not be repeated here.

In an exemplary embodiment, the second obtaining module 1002 is configured to determine, according to the pixel value of the first target pixel, a first cross-correlation interpolation value of the second type of pixel under the first brightness control parameter, and according to the first type of pixel The pixel value and the first cross-correlation interpolation obtain the first image, and the first target pixel point includes the first type pixel point adjacent to the second type pixel point;

In an exemplary embodiment, the second obtaining module 1002 determines, according to the pixel value of the second target pixel point, a second cross-correlation interpolation value of the first type of pixel point under the second brightness control parameter, according to the pixel value of the second type of pixel point The second image is obtained by interpolating the value and the second cross-correlation, and the second target pixel point includes a second type of pixel point adjacent to the first type of pixel point.

In an exemplary embodiment, the second obtaining module 1002 is configured to determine the weighted weight of the first target pixel point according to the pixel value of the second type of pixel point, and the weighted weight of the first target pixel point is related to the second type of pixel point and the first target pixel point. The difference size of a target pixel is positively correlated; the first cross-correlation interpolation of the second type of pixel under the first brightness control parameter is determined according to the pixel value of the first target pixel and the weighted weight;

The second acquisition module 1002 is configured to determine the weighted weight of the second target pixel point according to the pixel value of the first type of pixel point, and the difference between the weighted weight of the second target pixel point and the first type of pixel point and the second target pixel point Positive correlation; according to the pixel value of the second target pixel point and the weighted weight, determine the second cross-correlation interpolation value of the first type of pixel point under the second brightness control parameter.

In an exemplary embodiment, the second obtaining module 1002 is configured to, for any first target pixel, multiply the pixel value of any first target pixel and the weighted weight to obtain the corresponding value of any first target pixel. Accumulate the product results corresponding to each first target pixel to obtain the first cumulative sum; accumulate the weighted weights of each first target pixel to obtain the second cumulative sum; combine the first cumulative sum with the first cumulative sum The quotient of the two accumulated sums is used as the first cross-correlation interpolation of the second type of pixels under the first brightness control parameter;

The second obtaining module 1002 is configured to multiply the pixel value and the weighted weight of any second target pixel for any second target pixel to obtain a product result corresponding to any second target pixel; The product results corresponding to the two target pixels are accumulated to obtain the third accumulated sum; the weighted weights of each second target pixel are accumulated to obtain the fourth accumulated sum; the quotient of the third accumulated sum and the fourth accumulated sum is taken as the third accumulated sum. The second cross-correlation interpolation of a class of pixels under the second luminance control parameter.

In an exemplary embodiment, the brightness control parameter is smaller than the first brightness control parameter, and the fusion module 1003 is configured to perform brightness alignment between the pixel value of the first type of pixel point and the second cross-correlation interpolation according to the first brightness control parameter to obtain the first brightness control parameter. A brightness alignment result, which fuses the pixel values of the first type of pixel points and the first brightness alignment result; performs brightness alignment on the pixel values of the second type of pixel points and the first cross-correlation interpolation according to the first brightness control parameter to obtain the second brightness Alignment results; fuse the first cross-correlation interpolation and the second luminance alignment results.

In an exemplary embodiment, the fusion module 1003 is configured to interpolate the pixel value of the first type of pixel point and the second cross-correlation value according to the first brightness control parameter according to the difference multiple between the first brightness control parameter and the second brightness control parameter. Perform brightness alignment on the control parameters to obtain a first brightness alignment result, and the difference multiple is obtained based on the comprehensive difference between the first brightness control parameter and the second brightness control parameter;

The fusion module 1003 is configured to perform brightness alignment on the pixel value of the second type of pixel point and the first cross-correlation interpolation according to the first brightness control parameter according to the difference multiple between the first brightness control parameter and the second brightness control parameter, to obtain The second luminance alignment result.

In an exemplary embodiment, the fusion module 1003 is configured to multiply the pixel values of the first type of pixel points and the corresponding weighted weights to obtain a first product result, and multiply the first brightness alignment result and the corresponding weighted weights, Obtain the second product result; sum the first product result and the second product result to obtain the first sum value; sum the weighted weight corresponding to the first type of pixel point and the weighted weight corresponding to the first brightness alignment result to obtain the first Two-sum value; take the quotient of the first sum value and the second sum value as the fusion result of the pixel value of the first type of pixel point and the first brightness alignment result;

The fusion module 1003 is used to multiply the first cross-correlation interpolation and the corresponding weighted weight to obtain the third product result, multiply the second brightness alignment result and the corresponding weighted weight to obtain the fourth product result; The result and the fourth product result are summed to obtain the third sum value; the weighted weight corresponding to the second type of pixel point and the weighted weight corresponding to the second brightness alignment result are summed to obtain the fourth sum value; the third sum value is combined with The quotient of the fourth sum is used as the fusion result of the first cross-correlation interpolation and the second luminance alignment result.

In an exemplary embodiment, the fusion module 1003 is further configured to determine that β ₁₁ is a first value, β ₁₂ is a second value, and the first value is greater than the second value based on the fact that W ₁ is less than the first over-dark threshold; or, based on W ₁ is greater than or equal to the first over-dark threshold, determine that β ₁₁ is the second value, and β ₁₂ is the first value;

The fusion module 1003 is further configured to determine β ₂₁ as the second value and β ₂₂ as the first value based on W′ ₁ being greater than the first overexposure threshold; or, based on W′ ₁ being less than or equal to the first overexposure threshold, determine β ₂₁ is the first value, and β ₂₂ is the second value.

In an exemplary embodiment, the fusion module 1003 is further configured to determine that β ₁₁ is the first value, β ₁₂ is the second value, and the first value is greater than the second value based on the fact that W ₁ is less than the second over-dark threshold; or, based on If W ₁ is greater than the second overexposure threshold, it is determined that β ₁₁ is the second value and β ₁₂ is the first value; or, based on the fact that W ₁ is between the second over-dark threshold and the second over-exposure threshold, it is determined that β ₁₁ is from the first The first value begins to decrease with the increase of W ₁ , and β ₁₂ starts to increase with the increase of W ₁ from the second value;

The fusion module 1003 is further configured to determine that β ₂₁ is the first value, β ₂₂ is the second value, and the first value is greater than the second value based on W′ ₁ being less than the second dark threshold; or, based on W′ ₁ being greater than the second value Overexposure threshold, determine the second value of β ₂₁ , and β ₂₂ as the first value; or, based on W′ ₁ being between the second over-dark threshold and the second over-exposure threshold, determine that β ₂₁ starts from the first value and increases with W ' ₁ increases and decreases, and β ₂₂ increases from the second value as W' ₁ increases.

It should be understood that, when the device provided in FIG. 10 realizes its functions, it is only illustrated by the division of the above-mentioned functional modules. The internal structure is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.

An embodiment of the present application also provides an electronic device, the electronic device includes a processor, an image sensor, and at least two control circuits, where the at least two control circuits are used to control the image sensor to shoot and generate different types of pixels according to different brightness control parameters pixel value in point;

The processor is configured to perform the method of acquiring an image in any one of FIG. 5 or FIG. 6 .

An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes at least one instruction, and the instruction is executed by a processor to implement any one of the methods for acquiring an image in FIG. 5 or FIG. 6 .

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer program instructions, when loaded and executed on a computer, result in whole or in part of the processes or functions described herein. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk), and the like.

In this application, the terms "first", "second" and other words are used to distinguish the same or similar items with basically the same function and function, and it should be understood that between "first", "second" and "nth" There are no logical or timing dependencies, and no restrictions on the number and execution order. It will also be understood that, although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first image may be referred to as a second image, and, similarly, a second image may be referred to as a first image, without departing from the scope of various described examples. Both the first image and the second image may be images, and in some cases, may be separate and distinct images.

It should also be understood that, in each embodiment of the present application, the size of the sequence number of each process does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not be used in the embodiment of the present application. Implementation constitutes any limitation.

In this application, the meaning of the term "at least one" refers to one or more, and the meaning of the term "plurality" in this application refers to two or more. For example, a plurality of second messages refers to two or more more than one second message. The terms "system" and "network" are often used interchangeably herein.

It is to be understood that the terminology used in describing the various described examples herein is for the purpose of describing particular examples and is not intended to be limiting. As used in the description of the various described examples and the appended claims, the singular forms "a", "an")" and "the" are intended to include the plural forms as well, unless the context dictates otherwise. clearly instructed.

It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "and/or" is an association relationship that describes an associated object, indicating that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, A and B exist simultaneously, and B exists alone. a situation. In addition, the character "/" in this application generally indicates that the related objects are an "or" relationship.

It will also be understood that the term "includes" (also referred to as "includes", "including", "comprises" and/or "comprising") when used in this specification designates the presence of stated features, integers, steps, operations, elements , and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groupings thereof.

It should also be understood that the terms "if" and "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting." Similarly, depending on the context, the phrases "if it is determined..." or "if the [stated condition or event] is detected" can be interpreted to mean "when determining..." or "in response to determining... ” or “on detection of [recited condition or event]” or “in response to detection of [recited condition or event]”.

It should be understood that determining B according to A does not mean that B is only determined according to A, and B may also be determined according to A and/or other information.

It should also be understood that references throughout the specification to "one embodiment," "an embodiment," and "one possible implementation" mean that a particular feature, structure, or characteristic associated with the embodiment or implementation is included herein. in at least one embodiment of the application. Thus, appearances of "in one embodiment" or "in an embodiment" or "one possible implementation" in various places throughout this specification are not necessarily necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

The above descriptions are only examples of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. Inside.

Claims

A method for acquiring an image, characterized in that the method comprises:

Obtain the pixel values of at least two types of pixel points in the image pixel array, in the image, the pixel values of different types of the pixel points are generated according to different brightness control parameters, and the at least two types of pixel points are are black and white pixels, or, the at least two types of pixels are color pixels;

Acquiring images corresponding to each brightness control parameter according to the pixel values of the at least two types of pixel points;

The images corresponding to the respective brightness control parameters are fused.
The method according to claim 1, wherein the at least two types of pixel points include a first type of pixel point and a second type of pixel point, and the pixel value of the first type of pixel point is captured according to a first brightness control parameter generating, and the pixel values of the second type of pixel points are photographed and generated according to the second brightness control parameter;

The obtaining the image corresponding to each brightness control parameter according to the pixel values of the at least two types of pixel points includes:

Obtain a first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel point, and obtain a second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel point;

The image fusion corresponding to each brightness control parameter includes:

The first image and the second image are fused.
The method according to claim 2, wherein the obtaining the first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel point comprises:

Determine the first cross-correlation interpolation value of the second type of pixel under the first brightness control parameter according to the pixel value of the first target pixel, and according to the pixel value of the first type of pixel and the first cross-correlation interpolation Obtaining a first image by correlation interpolation, and the first target pixel point includes a first type pixel point adjacent to the second type pixel point;

The obtaining of the second image corresponding to the second brightness control parameter according to the pixel value of the second type of pixel includes:

The second cross-correlation interpolation of the first type of pixels under the second brightness control parameter is determined according to the pixel value of the second target pixel, and according to the pixel value of the second type of pixel and the second cross-correlation interpolation A second image is obtained by correlation interpolation, and the second target pixel point includes a second type of pixel point adjacent to the first type of pixel point.
The method according to claim 3, wherein the determining, according to the pixel value of the first target pixel, the first cross-correlation interpolation of the second type of pixel under the first brightness control parameter, comprising: The weighted weight of the first target pixel is determined according to the pixel value of the second type of pixel, and the weighted weight of the first target pixel is the same as the weight of the second type of pixel and the first target pixel. The difference is positively correlated; according to the pixel value of the first target pixel point and the weighted weight, determine the first cross-correlation interpolation value of the second type of pixel point under the first brightness control parameter;

The determining, according to the pixel value of the second target pixel, the second cross-correlation interpolation of the first type of pixel under the second brightness control parameter includes: determining the first type of pixel according to the pixel value of the first type of pixel. The weighted weight of the second target pixel point, the weighted weight of the second target pixel point is positively correlated with the difference between the first type of pixel point and the second target pixel point; according to the second target pixel point The pixel value of and the weighting weight determine the second cross-correlation interpolation of the first type of pixel point under the second brightness control parameter.
The method according to claim 4, wherein the first mutual relationship between the second type of pixels under the first brightness control parameter is determined according to the pixel value of the first target pixel and the weighted weight. Relevant interpolation, including:

For any first target pixel, multiply the pixel value and the weighted weight of the any first target pixel to obtain a product result corresponding to the any first target pixel; The corresponding product results are accumulated to obtain the first accumulated sum; the weighted weights of each first target pixel point are accumulated to obtain the second accumulated sum; the quotient of the first accumulated sum and the second accumulated sum is used as the second type. the first cross-correlation interpolation of the pixel point under the first brightness control parameter;

The determining of the second cross-correlation interpolation of the first type of pixels under the second brightness control parameter according to the pixel value of the second target pixel and the weighted weight includes:

For any second target pixel point, multiply the pixel value and weighted weight of the any second target pixel point to obtain the product result corresponding to the any second target pixel point; The corresponding product results are accumulated to obtain the third accumulated sum; the weighted weights of each second target pixel are accumulated to obtain the fourth accumulated sum; the quotient of the third accumulated sum and the fourth accumulated sum is taken as the first type The second cross-correlation interpolation of the pixel point under the second luminance control parameter.
The method according to any one of claims 3-5, wherein the second brightness control parameter is smaller than the first brightness control parameter, and the fusion of the first image and the second image comprises:

Perform brightness alignment on the pixel value of the first type of pixel point and the second cross-correlation interpolation according to the first brightness control parameter to obtain a first brightness alignment result, and fuse the pixel value of the first type of pixel point and the first luminance alignment result;

Perform luminance alignment on the pixel values of the second type of pixel points and the first cross-correlation interpolation according to the first luminance control parameter to obtain a second luminance alignment result; fuse the first cross-correlation interpolation and the first cross-correlation interpolation. Two luminance alignment results.
The method according to claim 6, wherein the pixel value of the first type of pixel point and the second cross-correlation interpolation are luminance aligned according to the first luminance control parameter to obtain the first luminance Align results, including:

According to the difference multiple between the first brightness control parameter and the second brightness control parameter, the pixel value of the first type of pixel point and the second cross-correlation interpolation are brightness aligned according to the first brightness control parameter to obtain the first brightness control parameter. Brightness alignment result, the difference multiple is obtained based on the comprehensive difference between the first brightness control parameter and the second brightness control parameter;

The performing luminance alignment on the pixel value of the second type of pixel point and the first cross-correlation interpolation value according to the first luminance control parameter to obtain a second luminance alignment result, including:

According to the difference multiple between the first brightness control parameter and the second brightness control parameter, perform the pixel value of the second type of pixel point and the first cross-correlation interpolation according to the first brightness control parameter Align the luminance to obtain the second luminance alignment result.
The method according to claim 6 or 7, wherein the fusing the pixel values of the first type of pixel points and the first brightness alignment result comprises:

Multiply the pixel value of the first type of pixel point and the corresponding weighted weight to obtain a first product result, and multiply the first brightness alignment result and the corresponding weighted weight to obtain a second product result; The first product result and the second product result are summed to obtain a first sum value; the weighted weight corresponding to the first type of pixel point and the weighted weight corresponding to the first brightness alignment result are summed to obtain the second Sum value; take the quotient of the first sum value and the second sum value as the fusion result of the pixel value of the first type of pixel point and the first brightness alignment result;

The fusion of the first cross-correlation interpolation and the second luminance alignment result includes:

Multiply the first cross-correlation interpolation value and the corresponding weighted weight to obtain a third product result, multiply the second luminance alignment result and the corresponding weighted weight to obtain a fourth product result; multiply the third product The result is summed with the result of the fourth product to obtain a third sum value; the weighted weight corresponding to the second type of pixel point and the weighted weight corresponding to the second brightness alignment result are summed to obtain a fourth sum value; The quotient of the third sum value and the fourth sum value is used as a fusion result of the first cross-correlation interpolation and the second luminance alignment result.
The method according to claim 8, wherein before the fusion of the pixel values of the first type of pixel points and the first luminance alignment result, the method further comprises:

Based on the fact that the W 1 is smaller than the first over-dark threshold, it is determined that the β 11 is a first value, the β 12 is a second value, and the first value is greater than the second value; or, based on the W 1 being greater than or equal to For the first over-dark threshold, determine that β 11 is the second value and β 12 is the first value;

Before the fusion of the first cross-correlation interpolation and the second luminance alignment result, the method further includes:

Based on the fact that the W 1 ′ is greater than the first overexposure threshold, it is determined that the β 21 is the second value and the β 22 is the first value; or, based on the W 1 ′ being less than or equal to the first overexposure threshold, The β 21 is determined to be the first value, and the β 22 is determined to be the second value.
The method according to claim 8, wherein before the fusion of the pixel values of the first type of pixel points and the first luminance alignment result, the method further comprises:

Based on the fact that the W 1 is smaller than the second over-dark threshold, it is determined that the β 11 is a first value, the β 12 is a second value, and the first value is greater than the second value; or, based on the W 1 greater than the first value Two overexposure thresholds, determine that the β 11 is the second value and the β 12 is the first value; or, based on the W 1 , is between the second overexposure threshold and the second overexposure threshold , it is determined that the β 11 decreases with the increase of the W 1 from the first value, and the β 12 increases with the increase of the W 1 from the second value;

Before the fusion of the first cross-correlation interpolation and the second luminance alignment result, the method further includes:

Based on the W 1 ′ being less than the second over-dark threshold, it is determined that the β 21 is a first value, the β 22 is a second value, and the first value is greater than the second value; or, based on the W 1 ′ greater than the second overexposure threshold, determine the second value of β 21 , and the β 22 is the first value; or, based on the W 1 ′ between the second over-dark threshold and the second over-exposure threshold In between, it is determined that the β 21 decreases from the first value as the W 1 ′ increases, and the β 22 increases from the second value as the W 1 ′ increases.
The method according to any one of claims 1-10, wherein the method is applied to a processor of an electronic device, and the electronic device further comprises an image sensor and a control circuit:

the control circuit, configured to accept the feedback from the processor and use at least two brightness control parameters to control the image sensor;

The image sensor is configured to perform photoelectric conversion according to the control of the control circuit to generate the image array, wherein each control parameter of the control circuit corresponds to a type of pixel point.
A device for acquiring images, characterized in that the device comprises:

The first acquisition module is used to acquire the pixel values of at least two types of pixel points in the image pixel array. In the image, the pixel values of different types of the pixel points are generated according to different brightness control parameters. The at least two types of pixels are both black and white pixels, or the at least two types of pixels are color pixels;

a second acquisition module, configured to acquire images corresponding to respective brightness control parameters according to the pixel values of the at least two types of pixel points;

The fusion module is used to fuse the images corresponding to the respective brightness control parameters.
The device according to claim 12, wherein the at least two types of pixel points include a first type of pixel point and a second type of pixel point, and the pixel value of the first type of pixel point is captured according to a first brightness control parameter generating, and the pixel values of the second type of pixel points are photographed and generated according to the second brightness control parameter;

The second obtaining module is configured to obtain the first image corresponding to the first brightness control parameter according to the pixel value of the first type of pixel, and obtain the corresponding second brightness control parameter according to the pixel value of the second type of pixel. the second image;

The fusion module is used to fuse the first image and the second image.
The device according to claim 13, wherein the second obtaining module is configured to determine the first pixel value of the second type of pixel under the first brightness control parameter according to the pixel value of the first target pixel. a cross-correlation interpolation, obtaining a first image according to the pixel values of the first-type pixel points and the first cross-correlation interpolation, and the first target pixel point includes a first target pixel point adjacent to the second-type pixel point class pixel;

The second acquisition module is configured to determine the second cross-correlation interpolation value of the first type of pixel point under the second brightness control parameter according to the pixel value of the second target pixel point, and according to the second type of pixel point A second image is obtained by interpolating the pixel value of , and the second cross-correlation, and the second target pixel includes a second type of pixel adjacent to the first type of pixel.
The device according to claim 14, wherein the second obtaining module is configured to determine the weighting weight of the first target pixel point according to the pixel value of the second type of pixel point, the first target pixel point The weighted weight of the pixel point is positively related to the difference between the second type of pixel point and the first target pixel point; according to the pixel value of the first target pixel point and the weighted weight, it is determined that the second type of pixel point is in the the first cross-correlation interpolation under the first brightness control parameter;

The second acquisition module is configured to determine the weighted weight of the second target pixel point according to the pixel value of the first type of pixel point, and the weighted weight of the second target pixel point is the same as the first type of pixel point. and the difference of the second target pixel point is positively correlated; according to the pixel value of the second target pixel point and the weighted weight, determine the second cross-correlation of the first type pixel point under the second brightness control parameter interpolation.
The device according to claim 15, wherein the second obtaining module is configured to multiply the pixel value and the weighted weight of any first target pixel for any first target pixel, Obtain the product result corresponding to any of the first target pixel points; accumulate the product results corresponding to each first target pixel point to obtain a first cumulative sum; accumulate the weighted weights of each first target pixel point to obtain the first Two cumulative sums; taking the quotient of the first cumulative sum and the second cumulative sum as the first cross-correlation interpolation of the second type of pixel point under the first brightness control parameter;

The second obtaining module is used for multiplying the pixel value and the weighted weight of any second target pixel to obtain the product corresponding to the any second target pixel. Result: Accumulate the product results corresponding to each second target pixel point to obtain a third cumulative sum; Accumulate the weighted weights of each second target pixel point to obtain a fourth cumulative sum; Combine the third cumulative sum and the fourth cumulative sum The quotient of the sum is used as the second cross-correlation interpolation of the first type of pixels under the second brightness control parameter.
The device according to any one of claims 14-16, wherein the second brightness control parameter is smaller than the first brightness control parameter, and the fusion module is configured to combine the pixels of the first type of pixels The value and the second cross-correlation interpolation are luminance aligned according to the first luminance control parameter to obtain a first luminance alignment result, and the pixel values of the first type of pixel points and the first luminance alignment result are fused; The pixel values of the second type of pixel points and the first cross-correlation interpolation are luminance aligned according to the first luminance control parameter to obtain a second luminance alignment result; the first cross-correlation interpolation and the second luminance are fused Align the results.
The device according to claim 17, wherein the fusion module is configured to combine the pixels of the first type of pixels according to the difference multiple between the first brightness control parameter and the second brightness control parameter. The value and the second cross-correlation interpolation are luminance aligned according to the first luminance control parameter to obtain a first luminance alignment result, and the difference multiple is obtained based on the comprehensive difference between the first luminance control parameter and the second luminance control parameter ;

The fusion module is configured to interpolate the pixel value of the second type of pixel and the first cross-correlation interpolation according to the difference multiple between the first brightness control parameter and the second brightness control parameter. The brightness alignment is performed using the first brightness control parameter to obtain a second brightness alignment result.
The device according to claim 17 or 18, wherein the fusion module is configured to multiply the pixel values of the first type of pixel points and the corresponding weighted weights to obtain a first product result, and the Multiplying the first brightness alignment result and the corresponding weighted weight to obtain a second product result; summing the first product result and the second product result to obtain a first sum value; adding the first type of pixel point The corresponding weighted weight is summed with the weighted weight corresponding to the first brightness alignment result to obtain a second sum value; the quotient of the first sum value and the second sum value is used as the first type of pixel point. the fusion result of the pixel value and the first luminance alignment result;

The fusion module is used to multiply the first cross-correlation interpolation value and the corresponding weighted weight to obtain a third product result, and multiply the second brightness alignment result and the corresponding weighted weight to obtain a fourth product result. ; Summing the third product result and the fourth product result to obtain the third sum value; Summing the weighting weight corresponding to the second type of pixel point and the weighting weight corresponding to the second brightness alignment result , to obtain a fourth sum; the quotient of the third sum and the fourth sum is used as a fusion result of the first cross-correlation interpolation and the second luminance alignment result.
The apparatus according to claim 19, wherein the fusion module is further configured to determine that the β 11 is a first value and the β 12 is a second value based on the W 1 being less than a first over-dark threshold value value, the first value is greater than the second value; or, based on the W 1 being greater than or equal to the first over-dark threshold, it is determined that the β 11 is the second value and the β 12 is the first value;

The fusion module is further configured to determine that the β 21 is the second value and the β 22 is the first value based on the W 1 ' being greater than the first overexposure threshold; or, based on the W 1 ' being less than or equal to For the first overexposure threshold, it is determined that the β 21 is the first value, and the β 22 is the second value.
The apparatus according to claim 19, wherein the fusion module is further configured to determine that the β 11 is a first value and the β 12 is a second value based on the W 1 being less than a second over-dark threshold value, the first value is greater than the second value; or, based on the W 1 greater than the second overexposure threshold, it is determined that the β 11 is the second value, and the β 12 is the first value; or, based on the W 1 is between the second overdark threshold and the second overexposure threshold, it is determined that the β 11 decreases from the first value as the W 1 increases, and the β 12 decreases from the first value the second value begins to increase as the Wi increases ;

The fusion module is further configured to determine that the β 21 is a first value, the β 22 is a second value, and the first value is greater than the second value based on the W 1 ′ being less than a second dark threshold; Or, based on the fact that the W 1 ′ is greater than the second overexposure threshold, the second value of β 21 is determined, and the β 22 is the first value; or, based on the W 1 ′ being between the second over-dark threshold value Between the second overexposure threshold, it is determined that the β 21 decreases as the W 1 ′ increases from the first value, and the β 22 decreases with the W 1 ′ from the second value 1 ' increases and increases.
An electronic device, characterized in that the electronic device includes a processor, an image sensor and at least two control circuits, the at least two control circuits are used to control the image sensor to generate different types of images according to different brightness control parameters. The pixel value in the pixel point of ;

The control circuit is configured to receive feedback from the image sensor, generate at least two brightness control parameters and send them to the image sensor;

the image sensor, configured to generate the generated image pixel array according to the at least two brightness control parameters, and each brightness control parameter corresponds to a type of pixel point;

The processor is configured to execute the method for acquiring an image according to any one of claims 1-10.
A computer-readable storage medium, characterized in that the computer-readable storage medium comprises at least one instruction, and the instruction is executed by a processor to implement the method for acquiring an image according to any one of claims 1-10.