CN113382169B - A photographing method and electronic device - Google Patents

Abstract

The application provides a photographing method and electronic equipment, and relates to the field of image processing. The method can shorten the photographing time and improve the user experience. The method comprises the following steps: in a photographing preview mode, collecting an image; after a first operation of a user for photographing is received, generating a composite image by using M frames of images containing a first acquisition image; the first collected image is an image collected in a photographing preview mode, the first collected image comprises at least one image with exposure parameters, M frames of images comprise N images with exposure parameters, and M and N are positive integers larger than 1 respectively. The method and the device are applied to photographing.

Description

A photographing method and electronic device

technical field

The present application relates to the field of image processing, in particular to a photographing method and electronic equipment.

Background technique

In the actual environment, the observable brightness difference, that is, the ratio of the brightness of the brightest object to the brightness of the darkest object is about 10 ⁸ , and the maximum brightness difference that human eyes can see is about 10 ⁵ , while the current image There are only 256 different brightnesses that can be represented by display devices and image acquisition devices such as monitors and cameras. Therefore, when shooting with a camera, on the one hand, you can reduce the exposure of the camera to make the photos show more details in dark places, but this will sacrifice the details of the brighter picture parts; Increase the exposure to make the photo show more details in bright places, but this will sacrifice the details of the parts of the picture with less brightness.

In order to enable photos to show more dynamic range and picture details, high-dynamic range (high-dynamic range, HDR) technology came into being. Specifically, HDR takes multiple photos continuously, the exposure of these photos increases (or decreases) in turn, and then these photos are fused, so that you can get photos that include both bright details and dark details. .

However, because HDR needs to collect multiple photos and also need to fuse multiple photos, the entire process takes a lot of time, which leads to a long time for taking pictures and affects the user experience.

Contents of the invention

Embodiments of the present application provide a photographing method and an electronic device for shortening the photographing time.

In the first aspect, a photographing method is provided, including: capturing an image in the photographing preview mode; after receiving a first user operation for photographing, using M frames of images containing the first captured image to generate a composite image; wherein , the first captured image is an image captured in the photo preview mode, M frames of images include images of N kinds of exposure parameters, and M and N are positive integers greater than 1, respectively. In the above method of the present application, after receiving the first operation of the user, the composite image can be generated by using the first collected image of at least one exposure parameter collected before the first operation. In this way, it can avoid that the electronic device needs to spend time collecting images of the at least one exposure parameter (ie, the exposure parameter of the first collected image) after receiving the first operation, thereby shortening the drawing time of the electronic device and improving the user experience. Use experience.

In a possible design, the method further includes: acquiring a second acquired image after receiving the first operation. Using M frames of images including the first captured image to generate a composite image includes: using the first captured image and the second captured image to generate a composite image; the first captured image and the second captured image include images of N kinds of exposure parameters. In the above design, by using the first captured image captured before receiving the first operation and the second captured image captured after receiving the first operation to generate a composite image, on the one hand, it is possible to avoid spending a lot of money after receiving the first operation. Acquire the image of the above-mentioned at least one exposure parameter (that is, the exposure parameter of the first acquired image) in time, shortening the drawing time of the electronic device; on the other hand, after receiving the first operation, according to the current photographing demand, acquire In addition to the above-mentioned at least one exposure parameter (ie, the exposure parameter of the first captured image), the second captured image of other exposure parameters, and use the first captured image and the second captured image to generate a composite image, which can improve the quality of the final composite image Imaging effect.

In a possible design, the first captured image is an image of a first exposure parameter; the second captured image includes images of exposure parameters of N kinds of exposure parameters except the first exposure parameter. In the above design, considering that in some application scenarios, the electronic device only captures an image with one exposure parameter (namely the first exposure parameter) in the photo preview mode, at this time, the second captured image can be captured after receiving the first operation (that is, images of exposure parameters other than the first exposure parameter among N kinds of exposure parameters), collect other images required to generate a composite image except the first captured image, and then generate a composite image, thereby achieving shortening The drawing time of electronic equipment improves the effect of user experience.

In a possible design, the first exposure parameter is the first exposure parameter under normal exposure of the image.

In a possible design, the first collected image includes: images of W kinds of exposure parameters; the second collected image includes images of exposure parameters other than the W kinds of exposure parameters among the N kinds of exposure parameters. In the above design, considering some application scenarios, the electronic device can collect images of various exposure parameters (that is, W kinds of exposure parameters) in the photo preview mode. At this time, the second collection image can be collected after receiving the first operation ( That is, the image of the exposure parameters other than the W exposure parameters in the N exposure parameters) is used to collect other images required to generate the composite image except the first captured image, and then generate the composite image, thereby shortening the electronic The drawing time of the device can improve the effect of user experience.

In a possible design, in the photo preview mode, collecting images includes: in the photo preview mode, collecting images of N kinds of exposure parameters. Using M frames of images including the first collected image to generate a composite image includes: fusing the first collected image to generate a composite image; the first collected image includes images of N kinds of exposure parameters. In the above design, it is considered that in some application scenarios, the electronic device can collect various exposure parameters in the photo preview mode, so it is possible to collect all the images enough to generate a composite image in the photo preview mode, so that after receiving the first After the operation, it only needs to use the collected images to generate the contract images without spending time collecting images, thereby further shortening the drawing time of the electronic device and improving the user experience.

In a possible design, the method further includes: displaying a preview image in the photo preview mode; the preview image is an image generated by fusing Q kinds of exposure parameters; wherein, Q is less than N. In the above design, since the preview image displayed in the photo preview mode is an image generated by fusing Q kinds of exposure parameters, compared with the image generated by fusing N kinds of exposure parameters, the above-mentioned preview image has the need Occupying fewer system resources and faster generation speed, etc., so the above design can achieve the effect of saving system resources in the photo preview mode and ensuring the smoothness of preview image display.

In a possible design, the first collected image is an image collected by using an interleaved high dynamic range Stagger HDR technology. Through the above design, it can be implemented on electronic devices using Stagger HDR technology to shorten the drawing time of electronic devices and improve user experience.

In a possible design, the first acquired image is an image acquired by using a dual conversion gain DCG technique. Through the above design, it can be implemented on the electronic equipment adopting the DCG technology to shorten the drawing time of the electronic equipment and improve the user experience.

In a possible design, the method further includes: determining whether the first captured image satisfies a preset condition; the preset condition includes at least one of the following: the moving speed of the object in the image is less than a first threshold, and the shaking of the electronic device when the image is taken If the amplitude is smaller than the second threshold value, the autofocus function converges when the image is captured and the auto exposure function converges when the image is captured. Using the M frames of images including the first captured image to generate a composite image includes: after determining that the first captured image satisfies a preset condition, using the M frames of images including the first captured image to generate a composite image. Through the above design, the image quality of the first captured image used to generate the composite image can be guaranteed, thereby reducing the possibility of ghosting, jitter and the like in the finally generated composite image.

In a possible design, the first captured image is the last image stored in the zero-second delay buffer of the ZSL buffer before receiving the user's first operation for taking pictures. In the above design, by selecting the closest image in the ZSLbuffer as the first captured image used to generate the composite image, the interval between each image used to generate the composite image can be reduced, and the final generated composite image can be reduced. The possibility of ghosting, shaking and other phenomena will increase the success rate of generating composite images.

In a possible design, the method further includes: determining a first captured image satisfying a preset condition from the P frame images. Wherein, the P frames of images are images collected in the photo preview mode, and the preset conditions include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device when the image is taken is less than the second threshold, and the image is taken The autofocus function converges when the image is captured and the autoexposure function converges when the image is captured. Through the above design, the image quality of the first captured image used to generate the composite image can be guaranteed, thereby reducing the possibility of ghosting, jitter and the like in the finally generated composite image.

In a possible design, the P-frame image is the image that is recently stored in the ZSL buffer zero-second delay cache after receiving the user's first operation for taking pictures. In this way, the time interval between images used to generate the composite image can be reduced, the possibility of phenomena such as ghosting and shaking in the finally generated composite image can be reduced, and the success rate of generating the composite image can be improved.

In the second aspect, an electronic device is improved, including: an image acquisition unit, configured to acquire images in the photo preview mode; an image processing unit, configured to use the first operation including the first Collect M frames of images of the image to generate a composite image; wherein, the first captured image is an image collected in the photo preview mode, and the M frames of images include images of N kinds of exposure parameters, and M and N are positive integers greater than 1 respectively.

In a possible design, the image acquisition unit is further configured to acquire a second acquired image after receiving the first operation; the image processing unit is configured to, after receiving the user's first operation for taking pictures, use the M frames of images of the image to generate a composite image, including: an image processing unit, specifically configured to generate a composite image using the first collected image and the second collected image; the first collected image and the second collected image include N kinds of exposure parameters image.

In a possible design, the first captured image is an image of a first exposure parameter; the second captured image includes images of exposure parameters of N kinds of exposure parameters except the first exposure parameter.

In a possible design, the first exposure parameter is the first exposure parameter under normal exposure of the image.

In a possible design, the first collected image includes: images of W kinds of exposure parameters; the second collected image includes images of exposure parameters other than the W kinds of exposure parameters among the N kinds of exposure parameters.

In a possible design, the image acquisition unit is used to acquire images in the photo preview mode, including: the image acquisition unit is specifically used to acquire images of N exposure parameters in the photo preview mode; the image processing unit, After receiving the user's first operation for taking pictures, using M frames of images including the first captured image to generate a composite image, including: an image processing unit, specifically used to fuse the first captured image to generate a composite image; A collected image includes images of N kinds of exposure parameters.

In a possible design, the electronic device further includes: a display unit, configured to display a preview image in the photo preview mode; the preview image is an image generated by fusing Q exposure parameters; wherein, Q is less than N.

In a possible design, the first collected image is an image collected by using an interleaved high dynamic range Stagger HDR technology.

In a possible design, the first acquired image is an image acquired by using a dual conversion gain DCG technique.

In a possible design, the image processing unit is further configured to determine whether the first captured image satisfies a preset condition; the preset condition includes at least one of the following: the moving speed of the object in the image is less than the first threshold, and the electronic The vibration amplitude of the device is smaller than the second threshold, the auto-focus function converges when the image is captured, and the auto-exposure function converges when the image is captured. The image processing unit is configured to generate a composite image using M frames of images containing the first captured image after receiving the user's first operation for taking pictures, including: an image processing unit configured to determine that the first captured image satisfies the predetermined After the conditions are set, a composite image is generated by using M frames of images including the first captured image.

In a possible design, the first captured image is the last image stored in the zero-second delay buffer of the ZSL buffer before receiving the user's first operation for taking pictures.

In a possible design, the image processing unit is also used to determine the first captured image that meets the preset condition from the P frame images; wherein, the P frame image is an image captured in the photo preview mode, and the preset conditions include At least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is captured, the autofocus function converges when the image is captured, and the auto exposure function converges when the image is captured.

In a possible design, the P-frame image is the image that is recently stored in the ZSL buffer zero-second delay cache after receiving the user's first operation for taking pictures.

In a third aspect, an electronic device is provided, including: one or more processors, one or more processors coupled with one or more memories, one or more memories storing computer programs; when one or more processors When the computer program is executed, the electronic device is made to execute the photographing method provided by the first aspect or any one of the designs of the first aspect.

In the fourth aspect, there is provided a computer-readable storage medium, including: computer software instructions; when the computer software instructions are run in the computer, the computer is made to perform the photographing as provided in the first aspect or any one of the designs of the first aspect method.

In the fifth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer is made to execute the photographing method provided in the first aspect or any one of the designs of the first aspect.

For the effect descriptions of the second aspect to the fifth aspect above, reference may be made to the effect descriptions of the first aspect, and details are not repeated here.

Description of drawings

Fig. 1 is a schematic diagram of the effect of taking pictures with HDR;

Fig. 2 is a schematic flow chart of using HDR to synthesize images;

FIG. 3 is one of the timing diagrams of image acquisition by an electronic device;

Fig. 4 is the second schematic diagram of the sequence of image acquisition by an electronic device;

Fig. 5 is a third schematic diagram of the sequence of image acquisition by an electronic device;

6 is a schematic structural diagram of a pixel element circuit;

Fig. 7 is a fourth schematic diagram of the sequence of image acquisition by an electronic device;

FIG. 8 is one of the structural schematic diagrams of an electronic device provided in the embodiment of the present application;

FIG. 9 is one of the schematic flow charts of a photographing method provided in an embodiment of the present application;

FIG. 10 is one of the timing schematic diagrams of an electronic device collecting images provided by an embodiment of the present application;

FIG. 11 is the second schematic diagram of the sequence of image acquisition by an electronic device provided in the embodiment of the present application;

FIG. 12 is the second schematic flow diagram of a photographing method provided in the embodiment of the present application;

Fig. 13 is the third schematic flow diagram of a photographing method provided by the embodiment of the present application;

FIG. 14 is the third schematic diagram of the sequence of image acquisition by an electronic device provided in the embodiment of the present application;

FIG. 15 is the fourth schematic flow diagram of a photographing method provided in the embodiment of the present application;

FIG. 16 is the fourth schematic diagram of the sequence of capturing images by an electronic device provided in the embodiment of the present application;

Fig. 17 is the fifth schematic diagram of the sequence of capturing images by an electronic device provided in the embodiment of the present application;

Fig. 18 is the fifth schematic flow diagram of a photographing method provided by the embodiment of the present application;

Fig. 19 is the sixth schematic flow diagram of a photographing method provided by the embodiment of the present application;

FIG. 20 is the sixth schematic diagram of the sequence of capturing images by an electronic device according to the embodiment of the present application;

Fig. 21 is the seventh schematic flow diagram of a photographing method provided in the embodiment of the present application;

Fig. 22 is the seventh schematic diagram of the sequence of image acquisition by an electronic device provided in the embodiment of the present application;

FIG. 23 is the eighth schematic diagram of the sequence of image acquisition by an electronic device provided in the embodiment of the present application;

Fig. 24 is the eighth schematic flow diagram of a photographing method provided by the embodiment of the present application;

Fig. 25 is the ninth schematic flow diagram of a photographing method provided by the embodiment of the present application;

FIG. 26 is the ninth schematic diagram of the sequence of capturing images by an electronic device according to the embodiment of the present application;

FIG. 27 is the tenth schematic diagram of the sequence of image acquisition by an electronic device provided in the embodiment of the present application;

Fig. 28 is the tenth schematic flow diagram of a photographing method provided by the embodiment of the present application;

Fig. 29 is the eleventh schematic flow diagram of a photographing method provided in the embodiment of the present application;

FIG. 30 is a second schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Among them, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations, and the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way , which is easy to understand.

First, the relevant technologies involved in the embodiments of the present application are introduced:

1. High dynamic range (high-dynamic range, HDR)

HDR can be understood as a technology that fuses multiple frames of low-dynamic range (LDR) images of various exposure brightnesses into one frame of HDR images. More picture details can be included in the HDR image.

Exemplarily, FIG. 1 is a schematic diagram of photographs taken of objects on a table, wherein (a) of FIG. 1 is an LDR image with a relatively high exposure brightness, and FIG. 1 (b) is an LDR image with a relatively low exposure brightness. It can be seen that in (a) of Figure 1, due to the high exposure brightness, on the one hand, the details of the picture in the box with low brightness can be clearly displayed; on the other hand, due to the high brightness of the basketball on the table, there is overexposure Therefore, only the outline of the basketball on the table can be displayed; in (b) of Figure 1, due to the low exposure brightness, the details of the basketball part can be clearly displayed on the one hand, and the box with lower brightness cannot be displayed on the other hand The picture details inside. Furthermore, (a) and (b) in Figure 1 can be merged into (c) in Figure 1 through HDR, and it can be seen that (c) in Figure 1 can display not only the details of the basketball part but also the picture inside the box detail.

Specifically, the general HDR photographing process is shown in Figure 2, including:

S101. The electronic device starts a camera application, enters a photo preview mode, and displays a photo preview image.

S102. The electronic device receives a user's photographing operation.

S103. The electronic device sets exposure parameters in response to the photographing operation.

For example, if four frames of EVO images, one frame of EV-2 images, and one frame of EV-4 images are required for HDR. Then the electronic device sets exposure parameters, so that the camera sequentially captures four frames of EV0 images, one frame of EV-2 images, and one frame of EV-4 images according to the set exposure parameters.

It should be noted that, for the sake of brevity, in the embodiment of the present application, the exposure brightness of the normal exposure is expressed as EV0, and the exposure brightness EV0×2 ⁿ is expressed as EVn. For example, EV-1 means that the exposure brightness is half of EV0, and EV-2 means that the exposure brightness is half of EV-1; for another example, EV1 means that the exposure brightness is twice that of EV0, and EV2 means that the exposure brightness is twice that of EV1.

S104. The camera collects images according to the exposure parameters.

Continuing with the above example, the camera captures four frames of EV0 images, one frame of EV-2 images, and one frame of EV-4 images in sequence.

S105. The electronic device uses the collected images to synthesize images to obtain an HDR image.

Specifically, FIG. 3 is a sequence diagram of a photographing process provided by an embodiment of the present application. Wherein, the timing diagram includes two time axes, the upper time axis is used to reflect the image exposure time of the photosensitive element in the camera, and the lower time axis is used to reflect the time when the camera reads the photosensitive data.

Wherein, the electronic device receives the user's photographing operation at time point t1. After the electronic device receives the photographing operation, it takes a period of time to generate the exposure parameters, send the exposure parameters to the camera, and make the camera execute the exposure parameters. Specifically, assuming that the time taken by the electronic device to generate the exposure parameters, send the exposure parameters to the camera, and make the camera execute the exposure parameters is three acquisition periods T, then the six acquisition periods of the electronic device in the dotted line box as shown in Figure 3 Collect 4 frames of EV0 images, one frame of EV-2 images, and one frame of EV-4 images. Afterwards, the electronic device synthesizes an HDR image based on the collected 6 frames of images.

2. Stagger high-dynamic range (stagger HDR)

Stagger HDR is a technology that can collect multiple frames of images with different exposure brightness in one acquisition cycle by increasing the frame rate of the sensor. For example, stagger HDR can generate long-exposure images and short-exposure images within one acquisition period. For example, as shown in Figure 4, the electronic device can acquire EV0 and EV-4 two frames of images within one acquisition period T. For another example, staggerHDR can generate long-exposure images, medium-exposure images, and short-exposure images within one acquisition period. For example, as shown in Figure 5, the electronic device can acquire three frames of EV0, EV-2 and EV-4 within one acquisition period T image. In comparison, in the stagger HDR scene, due to the acquisition of images of various exposure parameters, the blank frame interval between two frames of images (the frame interval can be called VB) compared to the non-stagger HDR scene The lower frame interval VB will be shorter.

At present, stagger HDR is mainly used in preview mode and video shooting mode to increase the HDR effect of video or preview screen. Since the frame interval VB of stagger HDR is short, it can well reduce the ghosting in the video and preview process.

3. Dual conversion gain (DCG)

DCG can be understood as having two capacitors that store photon energy in the photosensitive unit corresponding to a pixel, or it can be understood as the ability to read twice in a pixel element circuit.

Exemplarily, as shown in (a) of FIG. 6, it is a traditional complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS) pixel element circuit structure, wherein PD is a photodiode, TX is a transfer transistor, and RST is a reset Transistor, SF is the source follower, RS is the row selection transistor, V _AA _PIX is the analog pixel power supply voltage, V _OUT is the pixel output voltage node, FD is the floating diffusion node, and C _FD is the capacitance at the FD node. (b) in FIG. 6 shows the circuit structure of a pixel element using DCG. Compared with (a) in FIG. 6, the circuit structure shown in (b) in FIG. 6 adds a capacitor and transistor DCG at the dotted box, so that it has the ability to read twice in the circuit.

Therefore, in the scene where DCG is used, in one exposure, the reading result similar to stagger HDR can be achieved. Unlike stagger HDR, what DCG gets is the superposition of exposures, and what stagger HDR gets is new exposures.

Exemplarily, FIG. 7 is a timing diagram of traditional HDR, stagger HDR and DCG, three kinds of HDR in the process of acquiring images. It can be seen that DCG takes less time to acquire the images of EV0, EV-2 and EV-4 with the same three exposure parameters, so DCG can further reduce ghosting and increase the time of long exposure.

The following is an introduction to the photographing method provided by this application in conjunction with examples:

It can be seen from the scheme described in S101-S105 above that, in this scheme, at least 9 acquisition cycles are required from the time when the user performs the photographing operation to when the electronic device acquires 6 frames of images to be fused. In addition, coupled with the time required for the electronic device to synthesize the HDR image, the entire photographing process takes a long time, which affects the user experience.

In view of the above technical problems, an embodiment of the present application provides a photographing method, in which a synthetic image can be generated by using an image before the user triggers photographing (referred to as the first captured image for short). In this way, it can be avoided that the electronic device spends time collecting the image of the exposure parameter of the first collected image after receiving the user's operation for triggering the photographing.

In this way, it is possible to save the time required for image acquisition after receiving the photographing operation. Thereby speeding up the speed of taking pictures and improving the user experience.

The following is an introduction to the solution provided by the embodiment of the present application in combination with specific examples:

An embodiment of the present application provides a photographing method, which can be applied to an electronic device. Wherein, after detecting that the user inputs an operation at a specific position on the touch screen, the electronic device may display a corresponding interface and information according to the operation.

Exemplary, the electronic device in the embodiment of the present application may be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, and Cellular phones, personal digital assistants (personal digital assistant, PDA), augmented reality (augmented reality, AR)\virtual reality (virtual reality, VR) equipment and other electronic equipment with a touch screen, the embodiment of the present application does not make any reference to the specific form of the equipment. special restrictions.

Taking the electronic device as a mobile phone as an example, as shown in FIG. , power management module 341, battery 342, antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, audio module 370, speaker 370A, receiver 370B, microphone 370C, earphone jack 370D, sensor module 380, button 390, motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (subscriber identification module, SIM) card interface 395, etc.

Wherein, the sensor module 380 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

It can be understood that, the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 300 . In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 310 may include one or more processing units, for example: the processor 310 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor ( image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor neural network processor (neural-network processing unit, NPU), and/or microcontroller Control unit (micro controller unit, MCU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

Wherein, the controller may be the nerve center and command center of the electronic device 300 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 310 for storing instructions and data. In some embodiments, the memory in processor 310 is a cache memory. The memory may hold instructions or data that the processor 310 has just used or recycled. If the processor 310 needs to use the instruction or data again, it can be directly recalled from the memory. Repeated access is avoided, and the waiting time of the processor 310 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 310 may include one or more interfaces. The interface can include integrated circuit (inter-integrated circuit, I2C) interface, serial peripheral interface (serial peripheral interface, SPI), integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, pulse code modulation (pulse code modulation) , PCM) interface, universal asynchronous receiver/transmitter (UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input/output (general-purpose input/output, GPIO) interface, user identification A module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serialbus, USB) interface, etc.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 300 . In other embodiments of the present application, the electronic device 300 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 340 is configured to receive charging input from the charger. The power management module 341 is used for connecting the battery 342 , the charging management module 340 and the processor 310 . The power management module 341 receives the input of the battery 342 and/or the charging management module 340, and supplies power for the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, and the wireless communication module 360, etc. In some other embodiments, the power management module 341 and the charging management module 340 may also be set in the same device.

The wireless communication function of the electronic device 300 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 350 , the wireless communication module 360 , a modem processor, a baseband processor, and the like. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 300 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 350 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 300 . The wireless communication module 360 can provide applications on the electronic device 300 including wireless local area networks (wireless local area networks, WLAN) (such as Wi-Fi network), Bluetooth (blue tooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), NFC, infrared technology (infrared, IR) and other wireless communication solutions.

The electronic device 300 implements a display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 394 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 310 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 394 is used to display images, videos and the like. The display can be a touch screen. In some embodiments, the electronic device 300 may include 1 or N display screens 394, where N is a positive integer greater than 1.

The electronic device 300 can realize the shooting function through an ISP, a camera 393 , a video codec, a GPU, a display screen 394 , and an application processor. The ISP is used for processing the data fed back by the camera 393 . Camera 393 is used to capture still images or video. In some embodiments, the electronic device 300 may include 1 or N cameras 393, where N is a positive integer greater than 1.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 300 can be implemented through the NPU, for example: recognition of the film state, image restoration, image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 321 may be used to store computer-executable program codes including instructions. The processor 310 executes various functional applications and data processing of the electronic device 300 by executing instructions stored in the internal memory 321 . The internal memory 321 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 300 . In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The electronic device 300 may implement audio functions through an audio module 370 , a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, and an application processor. Such as music playback, recording, etc.

Touch sensor, also known as "touch panel (TP)". The touch sensor can be arranged on the display screen 394, and the touch sensor and the display screen 394 form a touch screen, also called “touch screen”. The touch sensor is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations can be provided through the display screen 394 . In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device 300 , which is different from the position of the display screen 394 .

The keys 390 include a power key, a volume key and the like. The motor 391 can generate a vibrating prompt. The indicator 392 can be an indicator light, which can be used to indicate the charging status, the change of the battery capacity, and can also be used to indicate messages, missed calls, notifications and the like. The SIM card interface 395 is used for connecting a SIM card.

In order to save the time required to collect images after receiving the photographing operation during the photographing process, thereby speeding up the photographing speed and improving the user experience, the method provided in the embodiment of the present application can be implemented in two ways:

Method 1: A composite image may be generated by using the first captured image captured in the photo preview mode and the second captured image collected after receiving the user's first operation for taking pictures.

Method 2: Collect all the images required to generate the composite image in the photo preview mode. In this method, the composite image can be generated without collecting images after receiving the first operation.

The following is a detailed introduction to the implementation process of the above two methods with examples:

method one:

In the first way, on the one hand, when compositing an HDR image, it is necessary to obtain multiple frames of images with various exposure parameters, and then determine the HDR image based on the multiple frames of images. On the other hand, considering that in some scenarios, when the electronic device is in the photo preview mode, it will collect images according to the preset exposure parameters, and start to generate the composite HDR image after receiving the user's first operation for taking pictures. The exposure parameters of the desired image.

For example, in the photo preview mode, the electronic device can first estimate the exposure parameter of the image under normal exposure (called the first exposure parameter), and then collect an image with a brightness of EV0 according to the first exposure parameter, as shown in Figure 3 Before receiving the user's first operation (that is, the photographing operation in the figure), the electronic device collects the image of EVO; after receiving the user's first operation, the electronic device starts to generate the exposure parameters of the image required for the composite HDR image, The required images are collected according to the generated exposure parameters, that is, four frames of EV0 images, one frame of EV-2 images, and one frame of EV-4 images are collected after three acquisition cycles of the photographing operation in FIG. 3 .

Therefore, the photographing method provided in the embodiment of the present application can use two types of images to synthesize an HDR image. Among them, one type of image is the image collected according to the preset exposure parameters in the photo preview mode (called the first collected image), and the other type of image is after receiving the user's first operation for taking pictures, the electronic device generates Images acquired with exposure parameters. In this way, since it is not necessary to spend time collecting images with preset exposure parameters after receiving the user's first operation, the time spent on collecting such images is saved, so the speed of taking pictures can be accelerated and the user experience can be improved.

Specifically, as shown in Figure 9, the photographing method provided by this application may include the following steps:

S401. The electronic device starts a camera application and enters a photo preview mode.

Specifically, the user may click the camera icon on the display interface of the electronic device to trigger the electronic device to start the camera application in response to the click operation and enter the photo preview mode.

In this application, the photo preview mode can be understood as: before receiving the user's first operation for taking pictures, the electronic device uses the camera to capture images, so as to display the captured images on the interface for the user to preview. In the actual application process, the photo preview mode can also be called other names, such as camera preview mode, browsing mode, etc., as long as the electronic device in this mode has the ability to use the camera to collect information before receiving the user's first operation for taking pictures. image function, then this mode can be understood as the photo preview mode in this application.

S402. The electronic device estimates a first exposure parameter of the image under normal exposure.

Wherein, the exposure parameter may be various parameters reflecting the exposure of the image. For example, the exposure parameter may include one or more items of light transmission time, shutter speed, light transmission area, and aperture size. Wherein, one exposure parameter may correspond to one exposure brightness of an image, which may also be understood as: when the electronic device collects an image according to one exposure parameter, it may collect an image of the exposure brightness corresponding to the exposure parameter.

In addition, in the embodiment of the present application, normal exposure can be understood as the exposure performed to make the display effect of the image meet the preset conditions. In the case of normal exposure, the first exposure parameter of the image can be understood as making the display effect of the image meet the preset condition. The exposure parameters corresponding to the conditions. Wherein, the preset condition may be set according to an actual application scenario.

S403. The electronic device collects an image of the first exposure parameter.

It should be noted that, in this embodiment, the first exposure parameter in the case of normal exposure is taken as an example for illustration. In some implementations, the electronic device can collect images using other exposure parameters in the photo preview mode. For example, the electronic device can collect images according to preset exposure parameters. In these implementations, the electronic device can also collect other exposure parameters in the photo preview mode. parameter without acquiring an image of the first exposure parameter.

Exemplarily, as shown in FIG. 10 , before receiving a photographing operation, the electronic device captures an image of EVO according to a first exposure parameter in a photographing preview mode. In a specific implementation process, the collected image may also be an image of other brightness, for example, it may be an image of EV1, EV2, EV-1 or EV-2 and so on.

Specifically, after the electronic device acquires the image of the first exposure parameter, it may cache the acquired image in a zero-second delay buffer (zero shutter lag buffer, ZSL buffer) for subsequent processing. Among them, the ZSL buffer can adopt the first-in-first-out storage method. When a new image is collected, the ZSL buffer deletes the old image and saves the new image.

S404. The electronic device receives a first user operation for taking pictures.

For example, in the photo preview mode, the electronic device may display a preset control on the interface, and the preset control is used to take a picture after the user clicks the preset control, and the first operation at this time may be the operation of the user clicking the preset control .

S405. The electronic device collects a second collected image.

In the actual implementation process, after the electronic device receives the first operation, it can store the image of the first exposure parameter stored in the ZSL buffer into the memory, and then clear the ZSL buffer to start collecting the second image and save the second image. The image is stored in the ZSL buffer.

Wherein, the second collected image includes images of other exposure parameters in the N types of exposure parameters except the first exposure parameter. Wherein, N represents the number of exposure parameters of the image required to synthesize the HDR image.

Specifically, since the EVO image of the first exposure parameter has been collected in S403, only images of exposure parameters other than the first exposure parameter among the N types of exposure parameters can be collected here, such as collecting images of EV-2 and Image of EV-4. Therefore, the second captured image may only include images of exposure parameters other than the first exposure parameter among the N types of exposure parameters. Of course, in order to achieve a better image effect, the second captured image may also include an image of the first exposure parameter, which is not limited in this embodiment of the present application.

In an implementation manner, S405 may include:

S4051. The electronic device determines an exposure sequence.

Wherein, the exposure sequence is used to reflect the number and sequence of the second collected images.

S4052. The electronic device collects a second collected image according to the above exposure sequence.

Exemplarily, as shown in FIG. 10 , the electronic device collects images of EV-2 and EV-4 in the fourth collection cycle after receiving the first operation. Wherein, it should be noted that, in the example shown in FIG. 10 , since the electronic device receives the first operation, it first executes determining the exposure sequence, generating the exposure parameters corresponding to each image in the exposure sequence, and the electronic device according to the exposure parameters. Actions such as adjusting the aperture of the camera, therefore, there are 3 acquisition periods between when the electronic device receives the first operation and when the electronic device acquires the second acquired image.

S406. The electronic device generates a composite image according to the first collected image and the second collected image.

Wherein, the first captured image is an image captured in the photo preview mode, that is, the first captured image is an image of the first exposure parameter captured in S403 above. Specifically, the first captured image may be an image recently stored in the ZSL buffer before receiving the user's first operation. For example, in FIG. 10 , the first captured image includes the 4 frames of E0 images that the electronic device recently stored in the ZSL buffer before receiving the photographing operation.

Specifically, the electronic device can fuse the first captured image and the second captured image according to various HDR technologies to obtain a composite image (for example, fusing 4 frames of EVO, 1 frame of EV-2, and 1 frame of EV-4 in FIG. 10 to obtain Composite image). In the embodiment of the present application, no limitation may be imposed on the technology used by the electronic device to fuse the first collected image and the second collected image.

In addition, before the electronic device collects the second captured image, it takes some time to perform actions such as determining the exposure sequence, generating the exposure parameters corresponding to each image in the exposure sequence, and adjusting the aperture of the camera according to the exposure parameters by the electronic device, as shown in Fig. In 10, there are 3 collection periods between when the electronic device receives the first operation and when the electronic device collects the second collected image. Therefore, before the electronic device receives the first operation and collects the second captured image, the electronic device can also obtain some images, for example, in FIG. 10 , three frames of the first exposure parameter can be collected after the electronic device receives the first operation. Image of EV0.

Therefore, in an implementation manner, the above S406 may include:

S4061. The electronic device generates a composite image according to the first collected image, the second collected image, and the third collected image.

Wherein, the third captured image is an image captured after the electronic device receives the first operation and before the electronic device captures the second captured image. Exemplarily, as shown in FIG. 11 , the third captured image may include images of three frames of EVO captured after the electronic device receives the first operation.

In addition, on the one hand, in the process of capturing images by the electronic device, there may be a ghost phenomenon. Wherein, the ghosting phenomenon can be understood as a phenomenon in which the position of the object in the image changes too much during the process of collecting the image, resulting in the phenomenon of ghosting of the object in the image. On the other hand, in the process of capturing images by the electronic device, there may also be a phenomenon that the auto-focus function does not converge or the auto-exposure function does not converge. Among them, the auto-focus function does not converge, which can be understood as the phenomenon that the electronic device adjusts the focal length before the focus is successful; the auto-exposure function does not converge, which can be understood as the electronic device adjusts the exposure parameters of the camera before determining the exposure parameters corresponding to the normal exposure. The phenomenon.

Therefore, in order to improve the success rate of image synthesis, in an implementation manner, after receiving the user's first operation at S404, as shown in FIG. 12 , the method may further include:

S407. Determine whether each image in the first collected image satisfies a preset condition.

Wherein, the preset condition may include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is taken, the autofocus function converges when the image is taken, and the automatic exposure function when the image is taken convergence. Wherein, the first threshold and the second threshold may be determined according to actual needs.

Exemplarily, in the actual implementation process: for the above preset condition "the moving speed of the object in the image is less than the first threshold", it can be realized in any of the following ways:

Method 1: Use a sensor in the electronic device for detecting the moving speed of the object to detect the moving speed of the object in the image, and then judge whether the moving speed is smaller than the first threshold;

Method 2: Detect whether there is a ghost in the image, and if there is no ghost, determine that the moving speed of the object in the image is less than a first threshold.

For the above preset condition "the shaking amplitude of the electronic device is smaller than the second threshold when taking an image", it can be realized by any of the following methods:

Method 1: Use a sensor for detecting vibration in the electronic device to detect the vibration amplitude of the electronic equipment, and then determine whether the vibration amplitude is smaller than the second threshold;

Method 2: Detect whether there is a ghost image in the image, and if there is no ghost image, determine that the vibration amplitude of the electronic device when the image is taken is smaller than a second threshold.

In other words, in some scenarios, by detecting whether there is a ghost in the image, it can be determined whether the moving speed of the object in the image and the shaking amplitude of the electronic device meet the conditions, that is, whether the moving speed is less than the first threshold and whether the shaking amplitude is less than the second threshold.

Regarding the "autofocus function converges when shooting images" in the above preset condition, it can be determined by judging whether the focal length of the camera changes within the preset time period, or whether the objects in the image are clear or not.

Aiming at the "automatic exposure function converges when shooting images" in the above preset conditions, it can be judged whether there are changes in the parameters such as the light transmission time, shutter speed, light transmission area and aperture size of the camera within the preset time period, or the exposure brightness of the image Whether it is normal and so on to determine.

Further, on the one hand, after determining that each image in the first captured image satisfies the preset condition through S407, the electronic device then performs S406, that is, generates a composite image according to the first captured image and the second captured image.

On the other hand, if it is determined through S407 that the first captured image includes an image that does not meet the preset condition, as shown in FIG. 12 , the method further includes:

S408. The electronic device collects a fourth collected image.

Wherein, the fourth collected image includes images of N kinds of exposure parameters required for synthesizing the HDR image. Exemplarily, if compositing an HDR image requires images of three exposure parameters, that is, images of EVO, EV-2 and EV-4. Then, as shown in FIG. 3 , the fourth captured image may include 4 frames of EVO images, 1 frame of EV-2 images, and 1 frame of EV-4 images in the dashed box in the figure.

S409. The electronic device generates a composite image according to the fourth collected image.

Similar to S406, the electronic device may generate a composite image according to the fourth collected image according to various HDR technologies. In the embodiment of the present application, there may be no limitation on the HDR technology adopted by the electronic device.

In the above implementation, on the one hand, after determining that the first captured image satisfies the preset condition, a composite image can be generated according to the first captured image and the second captured image, so as to ensure the quality of the composite image; After the first captured image includes an image that does not satisfy the preset condition, the fourth captured image may be re-captured, and a composite image may be generated based on the fourth captured image, so as to ensure smooth generation of the composite image.

In another implementation manner, after the user's first operation is received at S404, as shown in FIG. 13 , the method may further include:

S410. The electronic device determines a first captured image that satisfies a preset condition from the P frames of images.

Wherein, the P frames of images are images collected in the shooting preview mode, that is, the P frames of images are the images of the first exposure parameters collected in S403 above. Specifically, the P-frame image can be the P-frame image that was recently stored in the ZSL buffer before receiving the user's first operation, as shown in Figure 14, the electronic device reads the most recently stored ZSL before receiving the first operation (that is, the photographing operation in the figure). The 6-frame EV0 image of the buffer.

Wherein, the preset condition may include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is taken, the autofocus function converges when the image is taken, and the automatic exposure function when the image is taken convergence. Wherein, the first threshold and the second threshold may be determined according to actual needs. Wherein, the specific implementation process of judging whether the image satisfies the above-mentioned preset conditions can refer to the corresponding description in S407 above, which will not be repeated here.

Exemplarily, after receiving the first operation, the electronic device first reads the P frames of images before receiving the first operation from the ZSL buffer, for example, in FIG. Operation) 6 frames of E0 images collected before. Then select 4 frames of images satisfying the preset conditions from the 6 frames of images as the first collected images.

After the first captured image is determined through S410, the electronic device then executes S406, that is, generates a composite image according to the first captured image and the second captured image.

In the above implementation, by first acquiring P frames of images with a large number of frames, and then selecting the first captured image that meets the preset conditions from the P frames of images, the quality of the images included in the first captured image is guaranteed, and the composite Image quality.

It can be understood that, in the embodiment of the present application, the electronic device may perform some or all of the steps in the embodiment of the present application, and these steps or operations are only examples. In the embodiment of the present application, other operations or variations of various operations may also be performed. . In addition, each step may be performed in a different order presented in the embodiment of the present application, and it may not be necessary to perform all operations in the embodiment of the present application. For example, the above S407 may be performed before S405 or after S405; for another example, the above S410 may be performed before S405 or after S405, and the method provided in this application may not limit this.

When the electronic device supports the collection of images with multiple exposure parameters in the photo preview mode, for example, in a scenario where the electronic device supports stagger HDR or DCG, as shown in Figure 15, the photo taking method provided by this application may include the following step:

S501. The electronic device starts a camera application and enters a photo preview mode.

S502. The electronic device estimates a first exposure parameter of the image under normal exposure.

For the implementation process of S501 and S502 above, reference may be made to the corresponding descriptions in S401 and S402 above, and details will not be repeated here.

S503. The electronic device collects images of W exposure parameters according to the first exposure parameter.

Specifically, after determining the first exposure parameter (that is, the exposure parameter of the EVO image), the electronic device may also sequentially determine the exposure parameters of images such as EV-1, EV-2, EV-4, etc., and then according to the determined exposure parameter , using stagger HDR technology to collect images of W exposure parameters in photo preview mode.

Among them, the number of W can be determined according to the number of exposures in the same acquisition cycle supported by stagger HDR. For example, the adopted stagger HDR technology supports two exposures in the same acquisition period, that is, images with two exposure parameters can be acquired in the same acquisition period, then W can be 2. For another example, the adopted stagger HDR technology supports three exposures in the same acquisition period, that is, images with three exposure parameters can be acquired in the same acquisition period, then W can be 2 or 3. For another example, the adopted stagger HDR technology supports four exposures in the same acquisition period, that is, images with four exposure parameters can be acquired in the same acquisition period, then W can be 2, 3 or 4, and so on.

It should be noted that this embodiment mainly uses stagger HDR as an example to introduce this method. It can be understood that this method can also be applied to scenarios of other technologies such as DCG, and this application does not need to limit this.

Exemplarily, as shown in FIG. 16 , before receiving the photographing operation, the electronic device collects images of two exposure parameters, ie images of EVO and EV-4, in the photographing preview mode.

Specifically, after the electronic device collects images of W kinds of exposure parameters, it can cache the collected images in the ZSL buffer for subsequent processing.

S504. The electronic device receives a first user operation for taking pictures.

The specific implementation process of S504 may refer to the content of S404 above, which will not be repeated here.

S505. The device collects a second collected image.

Wherein, after the electronic device receives the first operation, the image stored in the ZSL buffer (that is, the image of the W exposure parameters described in S503) can be stored in the memory, and then the ZSL buffer is cleared to start collecting the second collection. image and store the second captured image into the ZSL buffer.

Wherein, the second collected image may include images of exposure parameters other than the W exposure parameters among the N exposure parameters. Wherein, N represents the number of exposure parameters of the image required to synthesize the HDR image.

Specifically, since images of W types of exposure parameters have been collected in S403, it is only necessary to collect images of other exposure parameters among the N types of exposure parameters except W types of exposure parameters. Therefore, the second captured image may only include images of exposure parameters other than the W exposure parameters among the N exposure parameters. Of course, in order to achieve a better image effect, the second captured image may also include images of W exposure parameters, which is not limited in this embodiment of the present application.

In an implementation manner, S505 may include:

S5051. The electronic device determines an exposure sequence.

Wherein, the exposure sequence is used to reflect the number and sequence of the second collected images.

S5052. The electronic device collects a second collected image according to the above exposure sequence.

Exemplarily, as shown in FIG. 16 , the electronic device collects the image of EV-2 in the fourth collection cycle after receiving the first operation (ie, the photographing operation in the figure). In addition, since images of two exposure parameters can be collected in one acquisition cycle in the stagger HDR scene, one frame of EV-4 images can also be collected while collecting EV-2 images, so the second captured image Can include one frame of EV-2 image and one frame of EV-4 image.

In addition, similar to the above example, in the example shown in Figure 16, the electronic device needs to take a certain amount of time (three acquisition cycles in the figure) between receiving the first operation and starting to acquire the second acquired image to perform the determination of the exposure sequence, generating the exposure parameters corresponding to each image in the exposure sequence, and the electronic device adjusting the aperture of the camera according to the exposure parameters.

S506. The electronic device generates a composite image according to the first collected image and the second collected image.

Wherein, the first captured image is an image captured in the photographing preview mode, that is, the first captured image is an image of W kinds of exposure parameters collected in S503 above. Specifically, the first captured image may be an image recently stored in the ZSL buffer before receiving the user's first operation. For example, in FIG. 16 , the first captured image includes 4 frames of EVO images and 4 frames of EV-4 images collected by the electronic device before receiving the photographing operation. The second collected image includes 1 frame of EV-2 image and 1 frame of EV-4 image.

Specifically, the electronic device may fuse the first collected image and the second collected image according to various HDR technologies to obtain a composite image. In the embodiment of the present application, no limitation may be imposed on the technology used by the electronic device to fuse the first collected image and the second collected image.

In addition, in an implementation manner, the above S506 may include:

S5061. The electronic device generates a composite image according to the first collected image, the second collected image, and the third collected image.

Wherein, the third captured image is an image captured after the electronic device receives the first operation and before the electronic device captures the second captured image. Exemplarily, as shown in FIG. 17 , the third captured image may include 3 frames of EVO images and 3 frames of EV-2 images captured after the electronic device receives the first operation.

Wherein, for the specific implementation process and beneficial effects of S5061, reference may be made to the corresponding description of S4061 above, which will not be repeated here.

In addition, in order to improve the success rate of image synthesis, in an implementation manner, after receiving the user's first operation at S504, as shown in FIG. 18 , the method further includes:

S507. Determine whether each image in the first collected image satisfies a preset condition.

Wherein, the preset condition may include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is taken, the autofocus function converges when the image is taken, and the automatic exposure function when the image is taken convergence. Wherein, the first threshold and the second threshold may be determined according to actual needs.

Further, on the one hand, after determining that each image in the first captured image satisfies the preset condition through S507, the electronic device then performs S506, that is, generates a composite image according to the first captured image and the second captured image.

On the other hand, if it is determined through S507 that the first captured image includes an image that does not meet the preset condition, as shown in FIG. 18 , the method further includes:

S508. The electronic device collects a fourth collected image.

Wherein, the fourth collected image includes images of N kinds of exposure parameters required for synthesizing the HDR image.

S509. The electronic device generates a composite image according to the fourth collected image.

For the specific implementation process of S507-S509 and the beneficial effects achieved, reference may be made to the corresponding descriptions of S407-S409 above, which will not be repeated here.

In another implementation manner, after receiving the user's first operation at S404, as shown in FIG. 19 , the method further includes:

S510. The electronic device determines a first captured image that satisfies a preset condition from the P frame images.

Wherein, the P frames of images are images collected in the shooting preview mode, that is, the P frames of images are the P frames of the images of the N types of exposure parameters collected in S503 above. Specifically, the P-frame image may be the P-frame image recently stored in the ZSL buffer before receiving the user's first operation.

Wherein, the preset condition may include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is taken, the autofocus function converges when the image is taken, and the automatic exposure function when the image is taken convergence. Wherein, the first threshold and the second threshold may be determined according to actual needs. Wherein, the specific implementation process of judging whether the image satisfies the above-mentioned preset conditions can refer to the corresponding description in S407 above, which will not be repeated here.

Exemplarily, after receiving the first operation, the electronic device first reads the P frames of images before receiving the first operation from the ZSL buffer, for example, in FIG. Operation) 12 frames of images (including 6 frames of E0 images and 6 frames of EV-4 images) collected in the previous 6 acquisition cycles. Then select 4 frames of EVO images and 1 frame of EV-4 images that meet the preset conditions from the 12 frames of images as the first captured image, so as to synthesize an HDR image with 1 frame of EV-2 included in the second captured image.

For the beneficial effect achieved by S510, reference may be made to the corresponding description of S410 above, and details will not be repeated here.

Method 2:

In this method 2, considering that the electronic device can collect all the images required for the composite image in the photo preview mode when the electronic device supports the collection of images with multiple exposure parameters in the photo preview mode, in this way After receiving the first operation, there may be no need to collect images.

Specifically, as shown in Figure 21, the method may include the following steps:

S601. The electronic device starts a camera application and enters a photo preview mode.

S602. The electronic device estimates a first exposure parameter of the EVO image under normal exposure.

For the implementation process of S601 and S602 above, reference may be made to the corresponding descriptions in S401 and S402 above, and details will not be repeated here.

S603. The electronic device collects images of N kinds of exposure parameters according to the first exposure parameter.

Wherein, the number of N may be determined according to the number of exposure parameters required for synthesizing the HDR image and the HDR technology (such as stagger HDR or DCG) adopted by the electronic device.

Wherein, similar to S503 above, the electronic device may first determine N types of exposure parameters according to the first exposure parameter, for example, the N types of exposure parameters include exposure parameters of E0 image, EV-2 image and EV-4 image, three types of images. Then images of these N kinds of exposure parameters are collected.

For example, taking stagger HDR as an example, as shown in Figure 22, before receiving the camera operation, the electronic device collects images of three exposure parameters, EV0, EV-2 and EV-4, in each collection cycle.

For another example, taking DCG as an example, as shown in FIG. 23 , before receiving the photographing operation, the electronic device collects images of three exposure parameters, E0, EV-2 and EV-4, in each collection cycle.

S604. The electronic device receives a first user operation for taking pictures.

The specific implementation process of S604 may refer to the content of S404 above, and will not be repeated here.

S605. The electronic device fuses the first collected image to generate a composite image.

Wherein, the first captured image is an image captured in the photo preview mode, that is, the first captured image is an image of the N kinds of exposure parameters collected in S603. Specifically, the first captured image may be an image recently stored in the ZSL buffer before receiving the user's first operation. For example, in FIG. 22, the first captured image includes 4 frames of E0 images, 4 frames of EV-2 images and 4 frames of EV-4 images that the electronic device recently stored in the ZSL buffer before receiving the camera operation. For another example, in FIG. 23 , the first captured image includes 4 frames of EVO images, 4 frames of EV-2 images, and 4 frames of EV-4 images that the electronic device recently stored in the ZSL buffer before receiving the photographing operation.

Specifically, the electronic device may fuse the first collected images according to various HDR technologies to obtain a composite image. In this embodiment of the present application, no limitation may be imposed on the technology used by the electronic device to fuse the first captured image.

In addition, in an implementation manner, as shown in FIG. 24, the method further includes:

S606. Determine whether each image in the first collected image satisfies a preset condition.

Wherein, the preset condition may include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is taken, the autofocus function converges when the image is taken, and the automatic exposure function when the image is taken convergence. Wherein, the first threshold and the second threshold may be determined according to actual needs.

Further, on the one hand, after determining that each image in the first captured image satisfies the preset condition through S606, the electronic device then performs S605, that is, fuses the first captured image to generate a composite image.

On the other hand, if it is determined through S606 that the first captured image includes an image that does not meet the preset condition, as shown in FIG. 24 , the method further includes:

S607. The electronic device collects a fourth collected image.

Wherein, the fourth collected image includes images of N kinds of exposure parameters required for synthesizing the HDR image.

S608. The electronic device generates a composite image according to the fourth collected image.

For the specific implementation process of S606-S608 and the beneficial effects achieved, reference may be made to the corresponding descriptions of S407-S409 above, which will not be repeated here.

In another implementation, as shown in Figure 25, the method further includes:

S609. The electronic device determines a first captured image that meets a preset condition from the P frame images.

Wherein, the P frames of images are images collected in the shooting preview mode, that is, the P frames of images are the P frames of images among the images of N kinds of exposure parameters collected in S603. Specifically, the P-frame image may be the P-frame image recently stored in the ZSL buffer before receiving the user's first operation.

Wherein, the preset condition may include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device is less than the second threshold when the image is taken, the autofocus function converges when the image is taken, and the automatic exposure function when the image is taken convergence. Wherein, the first threshold and the second threshold may be determined according to actual needs. Wherein, the specific implementation process of judging whether the image satisfies the above-mentioned preset conditions can refer to the corresponding description in S407 above, which will not be repeated here.

Exemplarily, after receiving the first operation, the electronic device first reads the P frames of images before receiving the first operation from the ZSL buffer, for example, in FIG. Operation) 18 frames of images (including 6 frames of E0 images, 6 frames of EV-2 images and 6 frames of EV-4 images) collected in the previous 6 acquisition cycles. Then select 4 frames of EVO image, 1 frame of EV-2 image and 1 frame of EV-4 image that meet the preset conditions from the 18 frames of images as the first captured image, so as to fuse the first captured image to generate an HDR image.

As another example, as shown in Figure 27, the electronic device reads 18 frames of images (including 6 frames of EVO images, 6 frames of EV -2 images and 6 frames of EV-4 images). Then select 4 frames of EVO image, 1 frame of EV-2 image and 1 frame of EV-4 image that meet the preset conditions from the 18 frames of images as the first captured image, so as to fuse the first captured image to generate an HDR image.

In addition, in an implementation manner, as shown in FIG. 28, the method further includes:

S610. In the photo preview mode, display an image generated by fusing the images of Q kinds of exposure parameters. Among them, Q is smaller than N.

In a possible design, the Q types of exposure parameters may include exposure parameters with the smallest exposure brightness and the largest exposure brightness among the N types of exposure parameters.

Exemplarily, as shown in FIG. 26 , before receiving the photographing operation, the electronic device may collect images with three exposure parameters (that is, N is 3) in the photographing preview mode. In this case, if the image obtained by fusing the images of the three exposure parameters is displayed on the interface as a preview image, it needs to occupy more hardware resources and may cause the problem of not smooth display.

Therefore, in this design, when the preview image is displayed in the photo preview mode, the preview image is generated by fusing images with fewer types of exposure parameters, and the preview image is displayed to achieve more efficient and Smoothly display the effect of the preview image.

For example, continuing the above example, in the case that the electronic device collects images of three kinds of exposure parameters in the photo preview mode, using the above-mentioned design of the present application, the two kinds of exposure parameters collected can be fused to generate a preview image, and the preview image can be displayed . Specifically, the two exposure parameters used for fusion may be the two exposure parameters with the minimum exposure brightness and the maximum exposure brightness among the three exposure parameters (ie, the exposure parameters of the EVO image and the EV-4 image).

In the method provided in the embodiment of the present application, in the process of synthesizing the HDR image, as shown in FIG. 29 , first, the electronic device captures the first captured image in the photo preview mode (ie, S201 in the figure). Then, after receiving the user's first operation for taking pictures, use the first captured image to generate a composite image (ie S202 in the figure). In this way, it is possible to save the time required for image acquisition after receiving the photographing operation. Thereby speeding up the speed of taking pictures and improving the user experience.

It can be understood that, in order to realize corresponding functions, the above-mentioned electronic device includes hardware structures and/or software modules corresponding to each function. In this embodiment of the present application, the electronic device is divided into functional modules according to the foregoing method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. Optionally, the division of modules in this embodiment of the present application is schematic, and is only a logical function division, and there may be another division manner in actual implementation.

As shown in FIG. 30 , it is a schematic composition diagram of an electronic device provided in the embodiment of the present application. The electronic device 40 may be a chip or a system on a chip. The electronic equipment 70 includes:

An image acquisition unit 701, configured to acquire images in the photo preview mode;

The image processing unit 702 is configured to generate a composite image by using M frames of images including the first captured image after receiving a first user operation for taking pictures.

Wherein, the first captured image is an image captured in the photo preview mode, the first captured image includes images of at least one exposure parameter, and M frames of images include images of N kinds of exposure parameters, and M and N are positive values greater than 1, respectively. integer.

Optionally, the image acquisition unit 701 is further configured to acquire a second acquired image after receiving the first operation.

The image processing unit 702 is configured to, after receiving the user's first operation for taking pictures, use M frames of images containing the first captured image to generate a composite image, including: an image processing unit, specifically configured to use the first captured image and The second collected image generates a composite image; the first collected image and the second collected image include images of N kinds of exposure parameters.

Optionally, the first captured image is an image of a first exposure parameter; the second captured image includes images of exposure parameters of N types of exposure parameters except the first exposure parameter.

Optionally, the first exposure parameter is the first exposure parameter under normal exposure of the image.

Optionally, the first collected image includes: images of W types of exposure parameters; the second collected image includes images of exposure parameters other than W types of exposure parameters among the N types of exposure parameters.

Optionally, the image acquisition unit 701, configured to acquire images in the photo preview mode, includes: an image acquisition unit specifically configured to acquire images of N exposure parameters in the photo preview mode.

The image processing unit 702 is configured to generate a composite image using M frames of images including the first captured image after receiving the user's first operation for taking pictures, including: an image processing unit specifically configured to fuse the first captured image, A composite image is generated; the first collected image includes images of N kinds of exposure parameters.

Optionally, the electronic device 70 further includes: a display unit 703, configured to display a preview image in the photo preview mode. The preview image is an image generated by fusing images of Q kinds of exposure parameters. Among them, Q is smaller than N.

Optionally, the first collected image is an image collected using an interleaved high dynamic range Stagger HDR technology.

Optionally, the first collected image is an image collected by using a dual conversion gain DCG technique.

Optionally, the image processing unit 702 is also configured to determine whether the first captured image satisfies a preset condition; the preset condition includes at least one of the following: the moving speed of the object in the image is less than the first threshold, and the shaking of the electronic device when the image is taken If the amplitude is smaller than the second threshold value, the autofocus function converges when the image is captured and the auto exposure function converges when the image is captured.

The image processing unit 702 is configured to generate a composite image using M frames of images containing the first captured image after receiving the user's first operation for taking pictures, including: an image processing unit configured to determine that the first captured image satisfies After preset conditions, a composite image is generated by using M frames of images including the first captured image.

Optionally, the first captured image is the last image stored in the ZSLbuffer zero-second delay cache before receiving the user's first operation for taking pictures.

Optionally, the image processing unit 702 is further configured to determine a first captured image that satisfies a preset condition from the P frame images.

Wherein, the P frames of images are images collected in the photo preview mode, and the preset conditions include at least one of the following: the moving speed of the object in the image is less than the first threshold, the shaking amplitude of the electronic device when the image is taken is less than the second threshold, and the image is taken The autofocus function converges when the image is captured and the autoexposure function converges when the image is captured.

Optionally, the P frame image is the image recently stored in the zero-second delay cache of the ZSL buffer after receiving the user's first operation for taking pictures. The embodiment of the present application also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instruction is executed, the method provided in the embodiment of the present application is executed.

The embodiment of the present application also provides a computer program product including instructions. When it runs on a computer, the computer can execute the method provided by the embodiment of the present application.

The functions or actions or operations or steps in the above-mentioned embodiments may be fully or partially implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or may include one or more data storage devices such as servers and data centers that can be integrated with the medium. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely illustrative of the application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of this application. Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the application fall within the scope of the claims of the application and their equivalent technologies, the application also intends to include these modifications and variations.

Claims (8)

1. A photographing method, comprising:

in a photographing preview mode, collecting an image;

determining a first acquisition image meeting preset conditions from the P frame images; the first acquired image is an image stored in a ZSL buffer zero second delay buffer before receiving a first operation of photographing by a user, the P frame image is an image acquired in the photographing preview mode, and the preset condition comprises at least one of the following: the motion speed of an object in the image is smaller than a first threshold value, the jitter amplitude of the electronic equipment is smaller than a second threshold value when the image is shot, the automatic focusing function is converged when the image is shot, and the automatic exposure function is converged when the image is shot;

After the first operation of photographing by a user is received, storing the first acquired image stored in the ZSL buffer into a memory, and emptying the ZSL buffer;

collecting a second collected image, and storing the second collected image into the ZSL buffer;

generating a composite image using the first acquired image and the second acquired image; the first acquired image and the second acquired image comprise images with N exposure parameters, and N is a positive integer greater than 1.

2. The method of claim 1, wherein the first acquired image is an image of a first exposure parameter; the second acquired image includes an image of an exposure parameter other than the first exposure parameter of the N exposure parameters.

3. The method of claim 2, wherein the first exposure parameter is a first exposure parameter under normal exposure of the image.

4. The method of claim 1, wherein the first acquired image comprises: an image of W exposure parameters; the second acquired image includes images of exposure parameters other than the W exposure parameters among the N exposure parameters.

5. The method of claim 4, wherein the first acquired image is an image acquired using an interleaved high dynamic range stabger HDR technique.

6. The method of claim 4, wherein the first acquired image is an image acquired using a dual conversion gain DCG technique.

7. An electronic device, comprising: one or more processors coupled with one or more memories, the one or more memories storing a computer program;

the computer program, when executed by the one or more processors, causes the electronic device to perform the photographing method as provided in any of claims 1-6.

8. A computer-readable storage medium, comprising: computer software instructions;

the computer software instructions, when run in a computer, cause the computer to perform the photographing method as provided in any of claims 1-6.

Images