WO2019183813A1

WO2019183813A1 - Image capture method and device

Info

Publication number: WO2019183813A1
Application number: PCT/CN2018/080734
Authority: WO
Inventors: 孙涛; 朱聪超
Original assignee: 华为技术有限公司
Priority date: 2018-03-27
Filing date: 2018-03-27
Publication date: 2019-10-03
Also published as: CN111418201A; CN111418201B

Abstract

Embodiments of the present application disclose an image capture method, device, and system, and relate to the technical field of communications, to improve the quality of captured images. The method is applied to an electronic device comprising a camera and an ISP, and comprises: calculating a short exposure value and a captured frame number M according to a preview image, sending the short exposure value and the captured frame number M to the camera via the ISP, and controlling, according to the short exposure value and the number of captured frames M, the camera to acquire M frames of short exposure images; performing multi-frame noise reduction processing and local brightness adjustment on the M frames of short exposure images to obtain a RAW image; sending the RAW image to the ISP for processing to obtain a YUV image; and performing compression encoding on the YUV image to obtain a target image. The image capture method provided by the present application is used for capturing high dynamic range images.

Description

Shooting method and device

Technical field

The embodiments of the present invention relate to the field of communications technologies, and in particular, to a shooting method and device.

Background technique

With the continuous development of science and technology, electronic devices such as smart phones and personal digital assistants (PADs) are becoming more and more powerful, and more and more users like to take images through electronic devices such as smart phones. However, limited by the hardware conditions of the electronic device, when shooting a high dynamic image, the usually acquired image can only collect a part of the information of the high dynamic range scene, or there is a problem of overexposure in the bright area, or there is a problem of insufficient brightness in the dark area. The overall shooting effect is poor.

In order to improve the shooting effect, the existing high dynamic image shooting method is as shown in FIG. 1 , and a long exposure image, a short exposure image, and a normal exposure image are used as a reference to synthesize an image by an exposure fusion algorithm. The dark area in the image is brightened by long exposure, the bright area is restored by short exposure, and the entire image is bright and dark. However, this shooting method has the following problems: the feature points between different brightness images are inconsistent, resulting in difficulty in registration, residual motion between images; long exposure images may be blurred due to long time; long exposure images due to overall brightness enhancement, images The contrast ratio is lowered, etc., resulting in poor quality of the captured image.

Summary of the invention

The embodiment of the present application provides a photographing method, device and system to solve the problem that the image quality of the captured image is poor when the high dynamic image is taken.

To achieve the above objective, the embodiment of the present application adopts the following technical solutions:

A first aspect of the embodiments of the present application provides a photographing method, which is applied to an electronic device including a camera and an image signal processor (ISP), including: the electronic device calculates a preview image according to a high dynamic range image. The short exposure amount and the number of shooting frames M, the short exposure amount and the number of shooting frames M are sent to the camera through the ISP, and the camera is controlled to acquire the M frame short exposure image according to the short exposure amount and the number of shooting frames M, and the M frame short exposure image. Multi-frame noise reduction processing and local brightness adjustment are performed to obtain a frame of RAW image, and the RAW image is sent to the ISP for processing to obtain a YUV image, and the YUV image is compression-encoded to obtain a target image. That is, in the technical solution provided by the embodiment, the shooting parameters such as the short exposure amount and the shooting frame number M are calculated based on the currently captured preview image, and the multi-frame short exposure image is captured by using the calculated shooting parameters, and the multi-frame short exposure image is lowered. The RAW image is obtained by processing such as noise and local brightness adjustment, and the RAW is processed by the ISP to obtain the target image. In this way, not only is it possible to control the exposure for each shooting scene, but also, through the noise reduction process, the captured image has better noise performance, and the highlight detail of the image is preserved by local brightness adjustment, in addition, the RAW image is recharged. To the ISP processing, because the ISP's processing is faster, the shooting efficiency is improved.

In a possible design, in combination with the first aspect, calculating the short exposure amount based on the preview image comprises: calculating a luminance average value of the highlight region composed of the overexposed pixels in the preview image; and comparing the brightness average value of the highlight region to the preview image The exposure value, and the target brightness value that the user desires to achieve, determine the short exposure amount. In this way, the short exposure amount can be determined according to the brightness of the image desired by the user, so that the brightness of the captured image fits the user's request.

It should be noted that, as long as the overexposed pixels exist in the preview image, the short exposure amount can be calculated according to the preview image, and the shooting scheme of the multi-frame short exposure image described in the first aspect is executed.

In yet another possible design, in combination with the first aspect, in order to highlight the brightness of the subject (such as a face, a flower, and a feature area where the subject is located) that the user desires to photograph, the short exposure amount calculated according to the preview image may be lengthened. , that is, increase the exposure time to make the brightness of the captured image stronger. Specifically, if the preview image includes a feature area in which the image capturing body that the user desires to capture, calculating the short exposure amount according to the preview image includes: calculating a brightness average value of the feature area of the preview image, according to the brightness average value of the feature area and the target The brightness value calculates a first brightness reduction ratio of the feature area, and determines a second brightness reduction ratio of the feature area according to the calculated first brightness reduction ratio and a preset minimum reduction ratio for performing brightness compensation on the feature area, according to the preview image The exposure value and the second brightness reduction ratio determine a short exposure amount, wherein the second brightness reduction ratio is greater than or equal to the minimum reduction ratio. Thus, the brightness reduction ratio of the compensation feature area is always greater than the preset minimum reduction ratio, ensuring the amount of exposure required to increase the brightness of the feature area.

In another possible design, in combination with the first aspect or any of the above possible designs, in order to avoid serious exposure of the preview image, the shooting scheme of the multi-frame short exposure image provided by the embodiment of the present application cannot display the image well. The problem of the details of the dark area, when the ratio of the underexposed pixels in the preview image is greater than the preset threshold, the multi-frame noise reduction processing and the local brightness adjustment are performed on the M-frame short exposure image to obtain a frame of RAW image. The method further includes: calculating a brightness average value of the over dark area of the preview image, determining a long exposure amount according to the brightness average value of the over dark area, the exposure value of the preview image, and the target brightness value, and controlling the camera to acquire a long exposure image of one frame. The M frame short exposure image is subjected to multi-frame noise reduction processing, and the local brightness adjusted image and the long exposure image are subjected to exposure fusion to obtain a frame of RAW image, and the RAW image is input into the ISP for processing. In this way, the processing scheme of the multi-frame short exposure image + one frame long exposure image can be improved to improve the noise detail of the dark area in the image.

It should be noted that, in practical applications, the number of frames of the long exposure image can be dynamically set in combination with the degree of underexposure of the preview image (not only can be set to one frame, but also can be set to multiple frames), and multi-frame multi-exposure images are used. A multi-frame long exposure image processing scheme to improve the noise detail of dark areas in the image.

In still another possible design, in combination with the first aspect or any of the above possible designs, the multi-frame noise reduction processing includes: multi-frame time domain noise reduction processing, or multi-frame time domain noise reduction processing and spatial domain noise reduction. deal with.

A second aspect of the embodiments of the present application provides a photographing method, which is applied to an electronic device including a camera and an ISP, including: the electronic device calculates a long exposure amount and a photographing frame number N according to a preview image of the high dynamic range image, The long exposure amount and the number of shooting frames N are sent to the camera through the ISP, and the camera is controlled to acquire N frames of long exposure images according to the long exposure amount and the number of shooting frames N, and multi-frame noise reduction processing and partial brightness adjustment for the N frame long exposure images. A RAW image is obtained, and the RAW image is sent to the ISP for processing, and the YUV image is obtained, and the YUV image is compression-encoded to obtain a target image; wherein the scheme can be applied to the case where the preview image is underexposed severely. In this way, based on the technical solution provided by the embodiment, not only the details of the dark area of the image can be improved, but also the obtained RAW image is recharged to the ISP processing, and the processing is faster, and the shooting efficiency is improved.

A third aspect of the embodiments of the present application provides an electronic device including a camera and an ISP, the electronic device further comprising: a calculating unit, a shooting control unit, and an image processing unit; wherein the calculating unit is configured to use the preview image Calculating a short exposure amount, and a shooting frame number M; a shooting control unit, configured to send the short exposure amount and the shooting frame number M to the camera through the ISP, and control the camera according to the short exposure amount Obtaining an M frame short exposure image with the shooting frame number M; and an image processing unit configured to perform multi-frame noise reduction processing and local brightness adjustment on the M frame short exposure image to obtain a frame RAW image, and the RAW image Sending to the ISP for processing, obtaining a YUV image, and compressing and encoding the YUV image to obtain a target image.

For a specific implementation manner of the electronic device, reference may be made to the behavior of the electronic device in the shooting method provided by the first aspect or any of the foregoing aspects, and details are not repeatedly described herein. Thus, the electronic device provided by this aspect can achieve the same benefits as any of the possible aspects of the first aspect or the first aspect.

A fourth aspect of an embodiment of the present application provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled to one or more processors for storing computer program code, the computer program code comprising computer instructions for causing the electronic device to perform when the one or more processors execute the computer instructions A method of photographing in any of the possible designs of any of the above aspects.

A fifth aspect of the embodiments of the present application provides a computer storage medium comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform a photographing method in any of the possible designs of any of the above aspects.

In a sixth aspect of the embodiments of the present application, there is provided a computer program product, when the computer program product is run on a computer, causing the computer to perform a photographing method in any of the possible designs of any of the above aspects.

These and other aspects of the present application will be more readily apparent from the following description of the embodiments.

DRAWINGS

1 is a schematic flow chart of a conventional high-motion image;

Figure 2 is a schematic block diagram of an embodiment of the present application;

FIG. 3 is a structural diagram of a mobile phone according to an embodiment of the present application;

FIG. 4 is a flowchart of a shooting method according to an embodiment of the present application;

FIG. 5 is a flowchart of a shooting method according to an embodiment of the present application;

FIG. 5a is a gray histogram of a preview image according to an embodiment of the present application;

FIG. 5b is a schematic diagram of highlight recovery according to an embodiment of the present application;

FIG. 5c is a schematic diagram of local enhancement of an image according to an embodiment of the present application;

FIG. 5 is a schematic diagram of image highlighting provided by an embodiment of the present application; FIG.

FIG. 5e is a schematic diagram of a detailed back-up according to an embodiment of the present application; FIG.

FIG. 6 is a schematic diagram of a photographing effect provided by an embodiment of the present application; FIG.

FIG. 6b is a schematic diagram of an effect of photographing a person according to an embodiment of the present application; FIG.

7 is a flowchart of a method for photographing a multi-frame short exposure image + a long exposure image according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a shooting effect of a multi-frame short exposure image + a long exposure image according to an embodiment of the present application; FIG.

FIG. 7b is a schematic diagram of a weight curve provided by an embodiment of the present application; FIG.

FIG. 7c is a schematic diagram of exposure fusion provided by an embodiment of the present application; FIG.

FIG. 8 is a structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The principle block diagram of the embodiment of the present application is as shown in FIG. 2, after the user opens the camera application, the camera preview function is activated, and the shooting parameters (short exposure amount, and the number of shooting frames M) are determined according to the preview image, and the ISP is set by the shooting device. Transfer the determined shooting parameters to the camera, control the camera to capture M (M is an integer greater than or equal to 2) frame short exposure; subsequently, the M frame short exposure frame is processed in the original (RAW) domain (such as multi-frame time domain drop Noise, single-frame time domain noise reduction, local brightness enhancement, etc.) Obtain a RAW picture with high dynamic range and low noise, and send the RAW picture to ISP, and process the YUV image by ISP; finally, combine YUV image The Joint Photographic Experts Group (JPEG) encodes the target image. In this way, you can quickly get a high-quality JPEG image to improve the shooting in a variety of application scenarios. Improve the noise, dynamic range and other effects of shooting. Specifically, the implementation manner of the following may refer to the scheme shown in FIG. 4 or FIG. 5 below.

For ease of understanding, the description of some of the concepts related to the embodiments of the present application is given for reference. As follows:

YUV image: It can refer to the image obtained by YUV coding method. YUV is a color coding method adopted by European television system. Generally, a three-tube color camera can be used for image acquisition, and then the obtained color image signal is subjected to color separation, separately amplified and corrected to obtain RGB, and then subjected to a matrix conversion circuit to obtain a luminance signal Y and two color difference signals R-Y (ie, U ), B-Y (ie V), the last sender encodes the three signals of luminance and color difference respectively to obtain a YUV image.

The dynamic range refers to the ability of the camera to adapt to the illumination of the scene in the scene, specifically the range of brightness. Generally, an image with a large range of brightness variation may be referred to as a High Dynamic Range (HDR) image, and an image with a small range of luminance variation may be referred to as a Low Dynamic Range (LDR) image.

Exposure: refers to the time interval from when the camera shutter is opened to closed. During this time, the object can leave an image on the film. The exposure time depends on the need. The longer the exposure time, the brighter the photo generated on the film. The darker the opposite, the longer the exposure time is generally required to extend the exposure time, and the short exposure time is suitable for light. Normally, the short exposure amount and the long exposure amount are defined by the normal exposure amount, the exposure time smaller than the normal exposure amount is referred to as a short exposure amount, and the exposure time larger than the normal exposure amount is referred to as a long exposure amount.

The normal exposure can be the exposure when the camera previews the image. Generally, the average value of the Y value of the current image can be calculated in the YUV space, and various exposure parameter settings can be adjusted (automatically or manually). When the average value falls near a target value, the exposure amount in the exposure parameter is considered It is the normal exposure.

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise stated, "/" means the meaning of or, for example, A/B may represent A or B; "and/or" herein is merely a description of the associated object. The association relationship indicates that there may be three relationships, for example, A and/or B, which may indicate that there are three cases in which A exists separately, A and B exist at the same time, and B exists separately. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.

The photographing method provided by the embodiment of the present application can be applied to an electronic device provided with a camera, wherein the electronic device can be a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a palm. Computer (Personal Digital Assistant, PDA), camera, digital camera, monitoring and other equipment. Specifically, in the embodiment of the present application, the electronic device is used as the mobile phone 100 shown in FIG. 3 as an example, and the shooting method provided by the present application is introduced. It should be understood that the illustrated mobile phone 100 is only one example of an electronic device, and the mobile phone 100 may have more or fewer components than those shown in the figures, two or more components may be combined, or Has a different component configuration. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

As shown in FIG. 3, the mobile phone 100 may include components such as a processor 101, a memory 102, an ISP 103, a camera 104, a touch screen 105, and the like, which may pass through one or more communication buses or signal lines (not shown in FIG. 3). Communicate. It will be understood by those skilled in the art that the hardware structure shown in FIG. 3 does not constitute a limitation on the mobile phone 100, and the mobile phone 100 may include more or less components than those illustrated, or combine some components, or different component arrangements. .

The various components of the mobile phone 100 will be specifically described below with reference to FIG. 3:

The processor 101 is a control center of the mobile phone 100, and connects various parts of the mobile phone 100 using various interfaces and lines, by running or executing an application (Application, App) stored in the memory 102, and calling data stored in the memory 102. And instructions to perform various functions and processing data of the mobile phone 100. In some embodiments, the processor 101 may include one or more processing units; the processor 101 may also integrate an application processor (application server), a modem processor, and a digital signal processor (Digital Signal Processor, DSP); wherein the application processor mainly processes an operating system, a user interface, an application, etc., the modem processor mainly processes wireless communication, and the DSP is mainly used to convert an analog signal into a digital signal, and filter the noise of the digital signal. Wait. It can be understood that the above-mentioned modem processor and image processor may not be integrated into the processor 101. For example, the processor 101 may be a Kirin 960 chip manufactured by Huawei Technologies Co., Ltd.

The memory 102 is used to store applications and data, and the processor 101 performs various functions and data processing of the mobile phone 100 by running applications and data stored in the memory 102. The memory 102 mainly includes a storage program area and a storage data area, wherein the storage program area can store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.); the storage data area can be stored according to the use of the mobile phone. Data created at 100 o'clock (such as audio data, phone book, etc.). Further, the memory 102 may include a high speed random access memory, and may also include a nonvolatile memory such as a magnetic disk storage device, a flash memory device, or other volatile solid state storage device. The memory 102 can store various operating systems, such as the IOS operating system developed by Apple Inc., the ANDROID operating system developed by Google Inc., and the like.

The ISP 103 is configured to perform processing such as dead pixel repair, white balance, gamma correction, sharpness, color interpolation, and the like on the image output by the DSP in the processor 101, and output an image required by the user application. ISP 103 is a determining factor in the performance of an imaging device. The ISP 103 can be integrated in the AP, or it can be a separate chip, and is not limited.

The touch screen 104 can be referred to as a touch display panel for implementing the input and output functions of the mobile phone 100, and can collect touch operations on or near the user (such as the user using any suitable object or accessory such as a finger, a stylus, etc. on the touch screen 104. The operation on or near the touch screen 104) (such as the user pressing the operation of the shooting button), and driving the corresponding connection device according to a preset program, can also be used to display information input by the user or information provided to the user (eg, by The images captured by the camera) and various menus of the phone. Optionally, the touch screen 104 may include two parts: a touch detection device and a touch controller, wherein the touch detection device detects a touch orientation of the user, and detects a signal brought by the touch operation, and transmits a signal to the touch controller; the touch controller The touch information is received from the touch detection device and converted into contact coordinates, sent to the processor 101, and can receive commands from the processor 101 and execute them.

The camera 105 can be referred to as a camera, and is a component having basic functions such as video capturing/propagating and still image capturing, and is mainly used for image capturing. Specifically, the camera 105 may include a camera and an image sensor, and the image sensor may be a Charge Coupled Device (CCD), or may be a Metal Oxide Semiconductor (CMOS) image sensor or the like. Any type of image sensor.

In a possible design, when the mobile phone 100 captures the function of the high dynamic image, the processor 101 obtains the short exposure amount and the number of shooting frames M, and transmits the terminal exposure amount and M to the camera 105 through the ISP 103; After the instruction, the processor 101 controls the camera 105 to acquire the M frame short exposure image according to the short exposure amount and M; then, the processor 101 performs multi-frame time domain noise reduction, spatial domain noise reduction, local brightness adjustment, etc. on the M frame short exposure image. The RAW map processing operation obtains a frame of RAW image; the processor 101 transmits the RAW image to the ISP 103, the ISP 103 converts the RAW image into a YUV image, and the processor 101 converts the YUV image into a target image, such as a JPEG image. Specifically, the possible design can refer to the scheme shown in FIG. 4 or FIG. 5.

Moreover, in various embodiments of the present application, the handset 100 may also include a light sensor 106. Specifically, the light sensor 106 may include an ambient light sensor and a proximity sensor. The ambient light sensor may sense the brightness of ambient light around the mobile phone 100, so that the mobile phone 100 adjusts the brightness of the display of the touch screen 104 according to the brightness of the ambient light. The proximity sensor can sense the proximity of the handset 100 to the human ear, and the handset 100 can turn off the power of the display as it moves to the ear. In addition, the mobile phone 100 can also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, and will not be described herein.

The mobile phone 100 may further include a power supply device 107 (such as a battery and a power management chip) that supplies power to the various components. The battery may be logically connected to the processor 101 through the power management chip to manage charging, discharging, and power management through the power supply device 107. And other functions.

Although not shown in FIG. 3, the mobile phone 100 may further include a Bluetooth device, a positioning device, an audio circuit, a speaker, a microphone, a WI-FI device, a near field communication (NFC) device, and the like, and details are not described herein.

The following embodiments can all be implemented in an electronic device (e.g., mobile phone 100) having the above hardware.

FIG. 4 or FIG. 5 is a flowchart of a photographing method provided by an embodiment of the present application, wherein the method can be performed by the mobile phone 100 shown in FIG. 3 to capture a high dynamic image. Taking FIG. 5 as an example, the method may include S501 to S506.

S501: After detecting that the mobile phone is turned on, the processor of the mobile phone automatically turns on the mobile phone to perform the function for capturing high dynamic images provided by the application; or the processor of the mobile phone receives the shooting function provided by the user to open the application provided by the application. After the operation, the function of capturing high dynamic images provided by the present application is performed in the mobile phone.

For example, in the night monitoring, the shooting method provided by the embodiment of the present application can be turned on to implement monitoring.

S502: The processor of the mobile phone receives the request for opening the camera application sent by the user, opens the camera application, and controls the camera to start the image preview function to obtain the preview image.

The preview image may refer to an image displayed on the display screen of the mobile phone before the image to be captured by the user is not imaged. For example, when the processor of the mobile phone detects that the user requests the boot camera application by clicking the desktop icon or sliding the camera shortcut icon on the unlock interface, the processor of the mobile phone controls the camera to capture, focus, etc. the image, and obtain a preview image. Further, the captured preview image can also be displayed on the display screen of the mobile phone for preview by the user.

The preview image may include a plurality of pixels, each pixel corresponding to a gray value, the gray value may be used to indicate the brightness of the pixel, and the value may range from 0 to 255, and the gray value of the pixel is larger. Indicates that the brighter the pixel, the smaller the gray value of the pixel, indicating that the pixel is darker.

S503: If the processor of the mobile phone determines that the preview image is a high dynamic image, the processor of the mobile phone calculates a short exposure amount according to the preview image, and a shooting frame number M, where M is an integer greater than or equal to 2.

Specifically, the gray histogram of the preview image may be obtained, and according to the gray histogram of the preview image, the proportion value of the overexposed pixel and the ratio of the underexposed pixel in the preview image are calculated, and if the ratio of the overexposed pixel is greater than the first The preset value and/or the scale value of the underexposed pixel is greater than the second preset value, and then the preview image is determined to be a high dynamic range image.

Wherein, the ratio of the overexposure pixel is greater than the first preset value and/or the ratio of the underexposure pixel is greater than the second preset value, which may mean that the ratio of the overexposed pixel is greater than the first preset value, or the underexposure pixel The ratio value is greater than the second preset value, or the ratio value of the overexposed pixel is greater than the first preset value and the ratio value of the underexposed pixel is greater than the second preset value.

The gray histogram is a statistic of the brightness level distribution of the pixels in the preview image, and all the pixels in the preview image are counted according to the grayscale value of the grayscale value, and the probability is represented by the horizontal axis. The brightness and vertical axis of the image represent the relative number of pixels in the preview image that are within the brightness range to form a grayscale histogram. Usually, the horizontal axis is from left to right, and the gray value is from small to large.

The overexposed pixel may refer to a pixel whose gray value is greater than the overexposure threshold; the underexposed pixel may refer to a pixel whose gray value is smaller than the underexposure threshold, and the overexposure threshold and the underexposure threshold may be set as needed, and are not limited. Generally, for the range of gray value values [0-255], the overexposure threshold Tover=235 can be defined, and the underexposure threshold Tunder=35, that is, the pixels with gray values of 235 to 255 are overexposed pixels, gray Pixels with a value of 0 to 35 are underexposed pixels.

The scale value of the overexposed pixel may refer to: the ratio of pixels between the [overexposed threshold Tover, 255] in the gray histogram to all the pixels included in the preview image, which is recorded as Ratio_over; the ratio of underexposed pixels may refer to: gray The ratio of pixels between [0, underexposed threshold] in the histogram to all pixels included in the preview image is recorded as Ratio_under. Normally, in order to improve the shooting effect, if Ratio_over exceeds the first preset value (for example, set to 5%), it means that the preview image is overexposed, and the overall brightness contrast of the preview image is relatively large; otherwise, Ratio_under exceeds the second pre- Setting the value (such as 40%) means that the preview image is underexposed, and the overall brightness contrast of the preview image is relatively large. For example, as shown in FIG. 5a, a gray histogram of a preview image provided by an embodiment of the present application, wherein a pixel having a gray value lower than 30 has a scale value of about 0.6+0.18=0.78, which is significantly higher than a preset value. (40%), so the image is a high dynamic image.

Specifically, the brightness average value of the highlight region of the preview image may be calculated, and the short exposure amount is determined according to the brightness average value of the highlight region, the exposure value of the preview image, and the target brightness value. Wherein, the average value of the brightness of the highlight region may refer to the ratio of the sum of the gray values of all the pixels in the highlight region to the number of all the pixels in the highlight region, and the target luminance value may refer to the brightness that the user desires to achieve. The highlight area may refer to an area formed by overexposed pixels in the preview image, and the area may also be referred to as an overexposed area of the preview image, and the brightness average of the highlight area is greater than the target brightness average.

Exemplarily, the ratio of the target luminance value to the luminance average value of the highlight region may be calculated, and the product of the exposure value of the preview image and the ratio is used as the short exposure amount. For example, as shown in the following formula, assuming that the luminance average value of the highlight region is Mover and the target luminance value is 210, the ratio Rover of the target luminance value to the average value Mover is used as the falling ratio of the luminance compensation to be performed on the highlight region. exposure value on the basis of the preview image on E _1, obtained by multiplying short exposure Rover E _target:

E _target =E ₁ *R _over .

Specifically, the average value of the brightness of the over dark area in the preview image may be calculated, and the number M of shot frames is determined according to the correspondence between the average value of the brightness of the over dark area and the number of shot frames. The average value of the brightness of the over dark area may refer to the ratio of the sum of the gray values of all the pixels in the dark area to the number of all the pixels in the dark area, and the dark area may refer to the underexposure pixels in the preview image. The area, which may also be referred to as the underexposed area of the preview image. The correspondence may be a preset functional relationship.

For example, suppose that the area of the pixel whose gray value is in the [0, 64] interval is an over dark area. At this time, the luminance average value M _under of the pixel whose gray value is in the [0, 64] interval can be calculated, and then, according to The following formula is used to set M. Optionally, when the average value M _{under is} less than or equal to the threshold 40, M is 6. When M _{under is} greater than the threshold 20 and less than or equal to 40, M is 4; when M _{under is} greater than the threshold 40, M Is 2.

The shooting frame number may also be determined by referring to the shooting scene, and the shooting scene may include any shooting scenes such as day shooting, night shooting, backlight shooting, night scene shooting, dark light shooting, and point light shooting, and the current shooting scene is brighter, setting The smaller the number of shot frames M is, the larger the M is due to the high noise level in the image. Typically, M can be set to 2 frames or 4 frames or 6 frames, etc., for example, M is 6 frames for night scene shooting, 2 frames for M during daytime shooting, and 4 frames for large backlighting.

Specifically, according to the ratio of the exposure time ET of the current preview image and the sensitivity ISO, and the preview image, it is determined which shooting scene the current shooting is in, for example, when the ratio R of the exposure time ET and the ISO is less than a certain preset ratio ( For example, when 0.9), it indicates that the current shooting scene is shooting during the day, and vice versa, for night scene shooting; if it is shooting during the day, and the ratio of the underexposed pixels in the preview image exceeds a certain threshold (large threshold), then the current shooting scene may be For backlighting scenes; if it is night scene shooting, and the scale value of overexposed pixels in the preview image exceeds a certain threshold T ₁ (threshold is small) and less than a certain threshold T ₂ (large threshold), the current shooting scene may be shot by point light source If it is a night scene shooting, and the scale value of the overexposed pixels in the preview image is smaller than the above T ₁ , it can be considered as a dark light shot. The exposure time ET is generally the reciprocal of the real time. For example, if the current exposure amount is 80 ms, the exposure time ET=1000 ms/80 ms=12.

S504: The controller of the mobile phone sends the short exposure amount and the number of shooting frames M through the ISP, and when the controller of the mobile phone receives the photographing instruction issued by the user, the processor of the mobile phone controls the camera according to the short exposure amount and the number of shooting frames M. , shooting M frame short exposure images.

Specifically, when the user presses the camera shutter in the mobile phone, the processor of the mobile phone controls the camera to acquire the M frame short exposure image based on the short exposure amount and other parameters (sensitivity, aperture coefficient, etc.).

S505: The processor of the mobile phone performs multi-frame noise reduction processing and local brightness adjustment on the M frame short exposure image to obtain a frame of RAW image.

Specifically, each process in this step can be referred to below.

S506: The processor of the mobile phone transmits the RAW image to the ISP of the mobile phone, and the ISP of the mobile phone converts the RAW image into a YUV image, and the processor of the mobile phone performs JPEG encoding on the IPS processed YUV image to obtain a target image.

Among them, YUV is an encoding format, Y stands for brightness, and U and V are chromaticity. The ISP converts the RAW image to obtain the YUV image. The ISP can first generate the RGB three-channel data through demosaic, and then convert the RGB to YUV according to the following formula to obtain the YUV image: Y=0.299R+0.587G+0.115b. U = -0.147R - 0.289G + 0.436B, V = 0.615R - 0.515G - 0.100B.

Wherein, the JPEG encoding of the YUV image may include processing such as detail folding, edge cropping, and the like. Specifically, the process can refer to the prior art, and details are not described herein.

Compared with the prior art, in the solution shown in FIG. 5, when shooting a high dynamic image, the short exposure amount and the number of shooting frames that need to be compensated for the highlight region can be determined in real time according to the preview image, based on the determined short exposure amount. And shooting a frame number to collect a multi-frame short exposure image, and multi-frame short exposure in the RAW domain for multi-frame noise reduction processing and local brightness adjustment and other processing to obtain a frame of RAW map; then, the resulting RAW map is recharged to The ISP processes the YUV image and processes the ISP-processed YUV image to obtain a JPEG image. First, in the technical solution provided by the embodiment, the short exposure amount and the frame number M are calculated based on the currently captured preview image to ensure that each shooting scene can control the exposure, and secondly, the multi-frame short exposure frame is performed in the RAW domain. Noise reduction, local brightness adjustment and other processes not only have better noise performance, but also preserve high-light detail through local brightness adjustment; in addition, in this scheme, the processed RAW image is recharged to ISP processing, due to ISP processing Faster, greatly improving the shooting efficiency.

For example, as shown in FIG. 6a, the left image is an effect diagram of the preview image, and the right image is an effect diagram of the image captured after the scheme shown in FIG. 5 is executed. As can be seen from FIG. 6a, after the scheme shown in FIG. 5 is executed, The details of the highlight area and the over dark area are improved.

Specifically, in the solution shown in FIG. 5, S505 may include the following processes (1) to (6), wherein (1) to (3) are multi-frame noise reduction processes, and (4) to (6) are local. Brightness adjustment process.

(1) A frame reference image R is selected from the M frame short exposure images.

Exemplarily, the contrast of each frame of the M frame short exposure image can be obtained, and the image with the highest contrast is used as the reference image R in order to improve the sharpness after image fusion.

Among them, the contrast of the image can be used to characterize the sharpness of the image, the higher the contrast, the clearer the image. In general, the average of the laplacian gradient of the image can be calculated and used as the contrast of the image. Specifically, the average value of the laplacian gradient of the image may be calculated by referring to the prior art, and details are not described herein again.

(2) performing pixel value difference between the second frame image and the reference image R, and performing point-by-point averaging on the matched pixel points to obtain a new reference image R1, and simultaneously recording the position of the unmatched pixel and storing it in the mask;

(3) performing the pixel value difference between the third frame image and R1, and performing point-by-point averaging on the matched pixel points to obtain a new reference image R2, and simultaneously recording the position of the unmatched pixel and storing it in the mask;

(4) Repeat the above action (3) on the remaining frames until all the frames are fused, and the image R and mask after the time domain noise reduction are obtained.

The second frame image may be any image other than the reference image R except for the M frame short exposure image, and the third frame image may be any image of the M frame image that is not subjected to pixel value difference with the reference image. .

Exemplarily, the reference image is R, the second frame image is M, the feature points in R are first detected, and the feature points in the M image are matched, and the feature points in R and M are matched; then, the warp matrix is calculated, M is transformed by the warp matrix to obtain the registered M', and the feature points in R and M' are matched. The feature point may be a pixel point in which the gray value of the R changes drastically or a point where the curvature is large on the edge of the image. Typically, the feature points can be SURF feature points. The Warp matrix can be called a transformation matrix, or other matrix that can transform, rotate, scale, etc. the image to deform the image.

In the process of matching the feature points in R and M', if there are moving objects in the image shooting, the positions of the moving objects are different in the images of different frames. At this time, it is necessary to detect the motion by the difference of the pixel points between the images. The area, forming a ghost mask, first calculates the brightness difference value diff of pixel by pixel:

Diff _xy =abs(R _xy -M' _xy )

If the diff exceeds the set threshold (such as 10), it is considered that the two points in the same position (x, v) in R and M' do not match, and the position is recorded to generate a template mask, which is greater than 0 in the mask. The location is a mismatched location. Then average the rest of R and M':

O _xy =(R _xy +M' _xy )/2

In this way, the difference detection is performed before the fusion, and the area with small difference is averaged to reduce the noise; for the area with large difference (the unmatched area), the average noise reduction is not performed, where the noise is large, and the next step is required. Noise reduction performs noise filtering, that is, the mask obtained in step (2) can be used to guide the intensity of the subsequent airspace noise reduction processing.

(3) Perform spatial denoising on a single frame image to obtain an image after spatial denoising.

Specifically, the spatial denoising method of the time domain denoising image may be spatially denoised using a commonly used spatial denoising method such as Non-Local Means (NLM), and will not be described again.

For the ghost mask in 2), since there is no time domain noise reduction residual noise, it is necessary to increase the noise reduction intensity of the region, and the noise reduction intensity can be set as needed, and is not limited.

It should be noted that when there is a motion area in the preview image, the above step (3) may be performed to remove ghosts. When there is no motion area in the preview image, step (3) may not be performed, and only steps are performed ( 1) ~ (2) to achieve a variety of noise reduction processing. In the embodiment of the present application, only the motion region exists in the preview image is taken as an example for description.

(4) The image after the noise reduction in the airspace is subjected to highlight recovery processing to obtain a highlight recovery image.

Specifically, as shown in FIG. 5b, the correction matrix C can be obtained, and the image I after the spatial domain noise reduction is corrected according to the correction matrix C (ie, Lens Shading Correction (LSC) processing), and the corrected image I is obtained. '; Image I and image I' are subjected to exposure fusion to obtain a highlight recovery image O.

Wherein, each element in C corresponds to a correction coefficient r (r>1, the farther away from the center, the larger the r value), the image I after the spatial domain noise reduction is corrected according to the correction matrix C, and the corrected image I′ can be obtained. It is meant that each corrected pixel I _{ij is} multiplied by a correction coefficient r of C corresponding point C _ij to obtain a corrected image I′.

Exposing the image I to the image I′ to obtain the highlight recovery image O may include: calculating the gray value of each pixel in the highlight recovery image O according to O _f =W _ij *I _ij +W _ij *I′ _ij . Where W _ij is the fusion weight, and the calculation process can refer to the following calculation weights.

The process is simply that when the weight W _ij is calculated, its weight center is set to 210.

Thus, when the brightness of a certain point in the image I is high, and the corresponding point in I' may be overexposed, the unexposed details of I may be preserved by the method of highlight recovery, or the details of the overexposed area in I' may be retained. .

(5) Perform local brightness enhancement processing on the image after the spatial noise reduction to obtain an enhanced image.

Since the embodiment of the present application reduces the exposure during the exposure calculation in order to preserve the details of the overexposed portion, the dark portion detail information in the image after the spatial domain noise reduction may be insufficient. In view of this, the gamma curve can be used for brightening, the Retinex method, and the like. The brightening method brightens the image after the airborne noise reduction, and combines the highlighted image with the highlight recovery image to obtain an image with darkness and rich brightness details. Specifically, the processing is as shown in FIG. 5c, including:

The spatially denoised image is converted into a grayscale Gray according to the conversion formula Gray=R*0.299+G*0.587+B*0.114. Where R\G\B refers to the red/green/blue channel of the image, respectively.

The detail of the grayscale Gray is separated to obtain the base layer (base layer) and the detail layer D; for example, the grayscale gray can be separated into the base and the detail layer D by the Gaussian filtering method.

Dynamic range compression (or brightness enhancement) is performed on the Base layer to obtain an enhanced grayscale image. Specifically, the enhanced grayscale image can be enhanced by the Retinex method.

For example, according to the mathematical model: r (x, y) = logS (x, y) - log (F (x, y) * S (x, y)) to obtain an enhanced grayscale, where S is the spatial domain noise reduction After the image, r is the enhanced result; F is the center surround function, generally defined as a Gaussian operator, such as:

The brightening gain (Gain) is determined according to the average brightness of the image of the enhanced grayscale image and the preset brightening amplitude. Among them, the preset brightness range can be set as needed, and is not limited. M _G is assumed that the average image brightness, brighten the preset amplitude is 128 In this case, M _G and 128 may be compared, the value of Gain obtained, such as: Gain = 128 / M _G.

The enhanced grayscale image is brightened according to the Gain value to obtain a brightened image. Specifically, each pixel in the enhanced grayscale image may be multiplied by a Gain value, or a Gamma curve may be directly converted according to the Gain value to perform a point-by-point lookup table (the gamma curve is preset according to experience), and each pixel in the image is highlighted. Get the highlighted image G_bright.

The Base layer, the brightened image G_bright, and the highlight restored image O are subjected to exposure fusion to obtain a local brightness enhanced image B_enhance.

The detail enhancement layer B is used to enhance the detail of the local brightness enhancement image B_enhance (ie, image addition) to obtain the final enhanced grayscale image Genhance. Wherein, the image addition may refer to adding the gray values of the pixels at the same position of the two images.

According to Gray and Genhance, the gain coefficient of each pixel is calculated, and the corresponding gain coefficient is multiplied by four channels of each RGGB format in the noise-reduced image to obtain a final enhanced image Ienhance. For example, as shown in FIG. 5d, each of the R\G\B channels constitutes a group, and a single Gray can be calculated; after the grayscale enhancement calculation is performed, a gain coefficient Ratio=Genhance/Gray is obtained, and then, Ratio is obtained. Multiply on each channel of R, G, and B.

Among them, the exposure fusion technique in this step can be referred to the following.

(6) Perform a detail overlay on the enhanced image Ienhance.

The core of the detail overlay is to represent the original image as the sum of the base layer and the detail layer. On this basis, the detail component is separately enhanced and the enhanced image is obtained, such as multiplying the detail component by a coefficient. After back-stacking onto the original image, the key is to acquire the basic components. Specifically, as shown in Figure 5e:

Performing the Gaussian filtering of the Ienhance obtained in (5) above to obtain a low frequency image I;

Obtaining a detailed image according to the formula Idetail=Ienhance-I;

Set the back-off coefficient α; get the detail back-up image Ienhance+I_detail*α

Wherein, the back-off coefficient α is an empirical value, generally greater than 1. In general, the lower the sharpness of the image, the higher the rebound coefficient α; the higher the sharpness of the image, the lower the back-off coefficient α.

It should be noted that each operation in FIG. 5e is a point-by-point operation for each pixel of the image.

Further, in the solution shown in FIG. 5, if there are feature areas in the preview image, such as: face area, portrait area, flower area, food, etc., the image capturing subject that the user desires to capture, in order to brighten the feature area when imaging The brightness needs to be increased by a short exposure amount, that is, the exposure time is longer, so that the exposure of the preview image is higher, and the object in the feature area is brighter.

Taking the feature area as a human face as an example, the average brightness value M _{face of the} face area can be calculated;

Calculate the ratio of the target brightness value to the brightness average of the highlight area:

The reset R _over R _over calculated, and the minimum drop rate R _min to compensate the brightness of the feature region, and then, in the exposure value E of the preview image on the basis of _1, Rover multiplied by the short exposure amount provided to give E _target . The average luminance value M _{face of the} face region may be: a ratio of the total of the gray values of all the pixels of the face region to the number of all the pixels of the face region; generally, the average luminance value of the face region is lower. , R _over is larger (R _{over is} generally less than 1). For example, the final result of R _over will be determined according to the following R _over and the set R _min , that is, whether R _over is smaller than the minimum value R _min , and if it is small, it is set to the minimum value R _min , which can no longer be Small; if it is larger than the minimum value R _min , then it is the calculated R _over .

Where R _min can be set according to the average brightness value M _{face of} different face regions, as shown below, x represents the average brightness value of the face:

For example, as shown in FIG. 6b, in order to photograph a person, the left image in FIG. 6b is a preview image, and the image on the right is an image obtained by highlighting the exposure value of the human face, which is obviously brightened a lot.

Further, in the solution shown in FIG. 5, if the ISP includes a spatial domain noise reduction module, a lens distortion correction LSC module, and a DRC module for adjusting brightness, when the ISP performs YUV format conversion on the RAW image, it also needs to be turned off. These modules are used to avoid ISP's repeated processing of images.

Further, if the ratio of the underexposed pixels in the preview image is greater than the preset threshold, it indicates that the over dark area in the preview image is relatively large. At this time, the long exposure amount can be determined, and the camera is controlled to capture a long exposure according to the long exposure amount. The image is blended with the M-frame short exposure image and the long exposure image to improve dark area detail. For example, as shown in FIG. 7, it is a flow chart of a multi-frame short exposure image and a long exposure image.

Specifically, the average value of the brightness of the over dark area of the preview image may be calculated, and the long exposure amount may be determined according to the average value of the brightness of the over dark area, the exposure value of the preview image, and the target brightness value. The brightness average value of the over dark region may refer to a ratio of a sum of gray values of all pixels in the over dark region to a number of all pixels in the over dark region, and an average brightness value of the over dark region is smaller than the target luminance value.

For example, the ratio of the target brightness value to the brightness average value of the over dark area can be calculated, and the product of the exposure value of the preview image and the ratio is used as the long exposure amount. For example, if the average brightness value of the over dark area is Munder and the target brightness value is 210, the ratio Runder of the target brightness value to the average value Munder is used as the compensation ratio for the brightness increase of the underexposure area. You can multiply the exposure value E _{1 of the} preview image by Runder to get the long exposure amount E _targer .

E _target =E ₁ *R _under

For example, as shown in FIG. 7a, the left picture is a simple multi-frame short exposure effect picture, and the right picture is a multi-frame short exposure image and a frame long exposure image fusion effect diagram. As can be seen from FIG. 7a, one frame length is added. The details of the over dark areas are improved after the image is exposed.

In the scheme shown in FIG. 7, the process of multi-frame time domain noise reduction and spatial domain noise reduction of the long exposure image may refer to the multi-frame time domain noise reduction and spatial domain noise reduction process of the short exposure image in FIG. 5, Let me repeat.

Specifically, the exposure fusion mentioned above, and the exposure fusion of the image after the spatial attenuation of the long exposure image and the local brightness adjustment in the scheme shown in FIG. 7 can be referred to the following:

Define the weight center. Preferably, c = 128 can be set.

Calculate the weight of each pixel according to the Gaussian weight formula and the gray value of the pixels included in each image

Among them, the Gaussian weight formula (the pixel value in the formula is normalized using the grayscale maximum value of 255, and the normalized by 128 is 0.5) is:

For example, as shown in FIG. 7b, the closer the gray value is to 128, the closer the weight is to 1, and the farther away from the 128 area (such as overexposed 255 or dead black area 0), the smaller the weight, the smaller the sharing of the final result.

A plurality of images are weighted and summed according to weights, where ij represents a pixel point position and k represents an image of a different brightness.

For example, as shown in FIG. 7c, the upper left three columns are a short exposure input map, a medium exposure input map, and a long exposure input map; the lower left three columns are a short exposure weight map, a medium exposure weight map, and a long exposure weight map; The large image is the result of exposure fusion of different exposure images on the left side according to their respective weighted weight maps. It can be seen from Fig. 7c that after the image and its weight-weighted image are subjected to exposure fusion, the highlights and shadows of the image are well received. Show.

Further, in the process of merging the multi-frame short exposure image and the one-frame long exposure image, when the long-exposure image has problems such as blur or ghosting, the long-exposure image is discarded, and only the multi-frame short-exposure image is fused (ie, The scheme shown in Fig. 5 is performed to fuse the poor quality long exposure image to affect the quality of the entire image.

It should be noted that, in the preview image, once there is overexposed pixels, or there are typical overexposed pixels, the multi-frame short exposure fusion scheme shown in FIG. 4 or FIG. 5 may be selected to improve the highlight portion of the image. Details; conversely, if the image does not have any overexposed pixels, but the entire image of the preview image is underexposed (such as a dark night), then a long exposure can be determined to perform the fusion scheme of the long exposure image described below. In other words, when the preview image is too bright, the multi-frame short exposure fusion shooting scheme shown in FIG. 4 or FIG. 5 is adopted. When the preview image is too dark, the multi-frame long exposure fusion shooting scheme is adopted; the preview image is bright and black, and The multi-frame short exposure fusion + one-frame long exposure image fusion shooting scheme shown in FIG. 7 is adopted.

The multi-frame long exposure fusion shooting scheme is: after the user opens the photographing application, the camera preview function is activated, and the shooting parameters (long exposure amount, and the number of shooting frames N) are determined according to the preview image, and the image set by the shooting device is set. The signal processor (ISP) transmits the determined shooting parameters to the camera, and the control camera captures the N-frame long exposure image; subsequently, the N-frame long exposure frame is processed in the RAW domain (eg multi-frame noise reduction, local brightness) Enhance, etc.) Get a RAW image with high dynamic range and low noise, send the RAW image to ISP, and process the YUV image by ISP; finally, Joint Photographic Experts Group (JPEG) for YUV image The code gets the target image. In this way, you can quickly get a high-quality JPEG image to improve the shooting in a variety of application scenarios. Improve the noise, dynamic range and other effects of shooting. Specifically, the process of multi-frame noise reduction, local brightness enhancement, and the like in the solution may be referred to the above, and details are not described herein.

Here, the process of determining the shooting parameters (the long exposure amount and the number N of shots) based on the preview image can be referred to the above S503. For example, for the case where the image is underexposed as a whole (such as when there is no light source in the night scene), the brightness average value M _over of the pixel whose brightness range is between [210, 225] may be less than the preset target value 210. At this time, a new one can be set. The target value Tnew, the ratio Rover of the target value Tnew and the luminance average value Mover is used as the rising ratio R _over for the luminance compensation to be performed on the highlight region, R _over may be greater than 1, that is, the set exposure will be larger than the exposure of the preview image. The time is longer, and then, based on the exposure value E ₁ of the preview image, multiplying Rover to obtain a long exposure E _target :

E _target =E ₁ *R _over

It can be understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art will readily appreciate that the present application can be implemented in a combination of hardware or hardware and computer software in combination with the algorithmic steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

The embodiments of the present application may divide the functional modules of the electronic device and the server according to the foregoing method. For example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

FIG. 8 is a schematic diagram showing a possible composition of the electronic device involved in the foregoing embodiment. As shown in FIG. 8 , the electronic device 80 may include: a computing unit 801. The photographing control unit 802 and the image processing unit 803.

The calculation unit 801 is configured to calculate a short exposure amount according to the preview image, and capture the frame number M to support the electronic device to execute S503.

The shooting control unit 802 is configured to send the short exposure amount and the shooting frame number M to the camera through the ISP, and control the camera to acquire the M frame according to the short exposure amount and the shooting frame number M. Short exposure image; execute S504 with the supporting electronic device.

The image processing unit 803 is configured to perform multi-frame noise reduction processing and local brightness adjustment on the M-frame short exposure image to obtain a frame of RAW image, and send the RAW image to the ISP for processing to obtain a YUV image, The YUV image is compression-encoded to obtain a target image. S505 and S506 are executed in support electronic devices.

It should be noted that all the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again. The electronic device provided by the embodiment of the present application is configured to perform the above-described photographing method, and thus the same effect as the above-described photographing method can be achieved.

In the case of an integrated unit, the above-mentioned computing unit 801, the shooting control unit 802 and the image processing unit 803 can be integrated into a processing module, wherein the processing module is used for controlling and managing the actions of the electronic device, for example, for the processing module. Steps S501-S506 in FIG. 5, and/or other processes for the techniques described herein, are performed on the supporting electronic device. In addition, the above electronic device may further include a display module and a storage module.

The storage module is used to store program code of the electronic device, recorded video, and various parameter information of the video.

The processing module may be a processor or a controller, such as a CPU, a graphics processing unit (GPU), a general purpose processor, a digital signal processor (DSP), and an application specific integrated circuit (application- Specific integrated circuit (ASIC), FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor can also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like.

The display module can be a display, and can be used to display information input by the user, information provided to the user, and various menus of the terminal. Specifically, the display can be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. In addition, a touch panel can be integrated on the display for collecting touch events on or near the display, and transmitting the collected touch information to other devices (such as a processor, etc.).

The memory module may be a memory, which may include a high speed RAM, and may also include a nonvolatile memory such as a magnetic disk storage device, a flash memory device, or other volatile solid state storage device.

In addition, the electronic device can also include a communication module that can be used to support communication of the electronic device with other network entities, such as communication with a server. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, and a WIFI chip.

In a specific implementation, when the processing module is a processor, the display module is a touch screen, and the storage module is a memory, the electronic device in the embodiment of the present application may specifically be the mobile phone shown in FIG. 3 .

The embodiment of the present application further provides a computer storage medium, where the computer storage medium stores computer instructions. When the computer instruction is run on the electronic device, the electronic device performs the foregoing method steps to implement the shooting method in the foregoing embodiment.

The embodiment of the present application further provides a computer program product, when the computer program product is run on a computer, causing the computer to perform the above related method steps to implement the photographing method in the above embodiment.

In addition, the embodiment of the present application further provides a device, which may be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store a computer to execute instructions, when the device is running, The processor may execute a computer-executable instruction stored in the memory to cause the chip to perform the photographing method in each of the above method embodiments.

The electronic device, the computer storage medium, the computer program product, or the chip provided by the embodiment of the present application are all used to perform the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the corresponding correspondence provided above. The beneficial effects of the method are not repeated here.

Through the description of the above embodiments, those skilled in the art can understand that for the convenience and brevity of the description, only the division of the above functional modules is illustrated. In practical applications, the above functions may be assigned differently according to needs. The function module is completed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or It can be integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

An integrated unit can be stored in a readable storage medium if it is implemented as a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the embodiments of the present application may be embodied in the form of a software product in the form of a software product in essence or in the form of a contribution to the prior art, and the software product is stored in a storage medium. A number of instructions are included to cause a device (which may be a microcontroller, chip, etc.) or a processor to perform all or part of the steps of the various embodiments of the present application. The foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The above content is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application. It is covered by the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

A photographing method is applied to an electronic device including a camera and an image signal processor ISP, wherein the method comprises:

Calculating a short exposure amount according to the preview image, and shooting the frame number M, wherein the preview image is a high dynamic range image, and the M is an integer greater than or equal to 2;

And sending the short exposure amount and the shooting frame number M to the camera through the ISP, and controlling the camera to acquire an M frame short exposure image according to the short exposure amount and the shooting frame number M;

Performing multi-frame noise reduction processing and local brightness adjustment on the M-frame short exposure image to obtain a frame of RAW image;

Sending the RAW image to the ISP for processing to obtain a YUV image;

The YUV image is compression-encoded to obtain a target image.
The method according to claim 1, wherein said calculating a short exposure amount based on the preview image comprises:

Calculating a brightness average value of the highlight region of the preview image; wherein the highlight region refers to an area formed by overexposed pixels in the preview image;

Determining the short exposure amount according to an average value of brightness of the highlight region, an exposure value of the preview image, and a target brightness value; wherein an average value of brightness of the highlight region is gray of all pixels in the highlight region A ratio of the sum of the degrees to the number of all pixels in the highlight region, the target luminance being the brightness desired by the user, the luminance average of the highlight region being greater than the target luminance average.
The method according to claim 1, wherein if the preview image includes a feature area, the calculating a short exposure amount according to the preview image comprises:

Calculating a brightness average value of the feature area of the preview image; wherein the feature area refers to an image capturing subject that the preview image user desires to capture;

Calculating a first brightness reduction ratio of the feature area according to a brightness average value of the feature area and a target brightness value; wherein the target brightness value is a brightness that the user desires to achieve;

Determining a second brightness reduction ratio of the feature area according to the calculated first brightness reduction ratio and a preset minimum reduction ratio for performing brightness compensation on the feature area; the second brightness reduction ratio is greater than or equal to The minimum drop ratio;

The short exposure amount is determined according to an exposure value of the preview image and the second brightness reduction ratio.
The method according to any one of claims 1-3, wherein when the ratio value of the underexposed pixels in the preview image is greater than a preset threshold, multi-frame noise reduction is performed on the short exposure image of the M frame. Processing, local brightness adjustment, before obtaining a frame of RAW image, the method further includes:

Calculating an average value of brightness of an over dark area of the preview image;

Determining a long exposure amount according to the average value of the brightness of the over dark area, the exposure value of the preview image, and the target brightness value; wherein the brightness average value of the over dark area refers to all pixels in the over dark area a ratio of a sum of gray values to a number of all pixels in the over dark region, the average brightness of the over dark regions being less than the target brightness value;

Controlling the camera to acquire a long exposure image of one frame;

Performing multi-frame noise reduction processing and local brightness adjustment on the short exposure image of the M frame to obtain a frame of RAW image, including:

The M frame short exposure image is subjected to multi-frame noise reduction processing, and the local brightness adjustment image is subjected to exposure fusion with the long exposure image to obtain a frame RAW image.
An electronic device, comprising: a camera and an image signal processor ISP, wherein the electronic device further comprises:

a calculating unit, configured to calculate a short exposure amount according to the preview image, and a shooting frame number M, wherein the preview image is a high dynamic range image, and the M is an integer greater than or equal to 2;

a shooting control unit, configured to send the short exposure amount and the shooting frame number M to the camera through the ISP, and control the camera to acquire an M frame short according to the short exposure amount and the shooting frame number M Exposure image

An image processing unit, configured to perform multi-frame noise reduction processing and local brightness adjustment on the M-frame short exposure image to obtain a frame of RAW image; and send the RAW image to the ISP for processing to obtain a YUV image; The YUV image is compression-encoded to obtain a target image.
The electronic device according to claim 5, wherein the calculating unit is specifically configured to:

Calculating a brightness average value of the highlight region of the preview image; wherein the highlight region refers to an area formed by overexposed pixels in the preview image;

Determining the short exposure amount according to an average value of brightness of the highlight region, an exposure value of the preview image, and a target brightness value; wherein an average value of brightness of the highlight region is gray of all pixels in the highlight region A ratio of the sum of the degrees to the number of all pixels in the highlight region, the target luminance being the brightness desired by the user, the luminance average of the highlight region being greater than the target luminance average.
The electronic device according to claim 5, wherein the calculating unit is specifically configured to:

If the preview image includes a feature area, calculate a brightness average value of the feature area of the preview image; wherein the feature area refers to an image capturing subject that the preview image user desires to capture;

Calculating a first brightness reduction ratio of the feature area according to a brightness average value of the feature area and a target brightness value; wherein the target brightness value is a brightness that the user desires to achieve;

Determining a second brightness reduction ratio of the feature area according to the calculated first brightness reduction ratio and a preset minimum reduction ratio for performing brightness compensation on the feature area; the second brightness reduction ratio is greater than or equal to The minimum drop ratio;

The short exposure amount is determined according to an exposure value of the preview image and the second brightness reduction ratio.
The electronic device according to any one of claims 5 to 7, wherein

The calculating unit is further configured to perform multi-frame noise reduction processing and local brightness adjustment on the M-frame short exposure image in the image processing unit when a ratio value of the under-exposed pixels in the preview image is greater than a preset threshold. Calculating an average value of the brightness of the over dark area of the preview image before obtaining a frame of the RAW image;

The photographing control unit is further configured to determine a long exposure amount according to an average value of a brightness of the over dark area, an exposure value of the preview image, and the target brightness value; wherein an average brightness of the over dark area The value refers to the ratio of the sum of the gray values of all the pixels in the over dark area to the number of all the pixels in the dark area, the brightness average of the over dark area is smaller than the target brightness value; controlling the camera to collect one frame long Exposure image

The image processing unit is specifically configured to perform exposure fusion on the multi-frame noise reduction processing, the partial brightness adjustment image, and the long exposure image on the M-frame short exposure image to obtain a frame RAW image.
An electronic device comprising one or more processors and one or more memories;

The one or more memories are coupled to the one or more processors, the one or more memories for storing computer program code, the computer program code comprising computer instructions, when the one or more processors The electronic device performs the photographing method according to any one of claims 1 to 4 when the computer instruction is executed.
A computer storage medium, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the photographing method of any of claims 1-4.
A computer program product, characterized in that when the computer program product is run on a computer, the computer is caused to perform the photographing method according to any one of claims 1-4.