CN110198418B - Image processing method, device, storage medium and electronic device - Google Patents

Abstract

The application discloses image processing method is applied to electronic equipment, the electronic equipment comprises a first camera module and a second camera module, wherein the second camera module is a long-focus camera module, and the method comprises the following steps: determining a first target camera module and a second target camera module from the first camera module and the second camera module; acquiring a first RAW image through the first target camera module, and acquiring a second RAW image and a third RAW image through the second target camera module, wherein the exposure levels of the first RAW image, the second RAW image and the third RAW image are sequentially reduced; taking the second RAW image as a reference image, and performing synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW synthesis image with a high dynamic range; the RAW composite image is used to perform image preview, photographing or recording operation. The image processed by the image processing method provided by the application can be suitable for previewing, photographing and recording.

Description

Image processing method, device, storage medium and electronic device

technical field

The present application belongs to the field of image technology, and in particular, relates to an image processing method, device, storage medium and electronic device.

Background technique

Compared with ordinary images, High-Dynamic Range (HDR) images can provide more dynamic range and image details. The electronic device can shoot multiple frames of images with different exposure degrees in the same scene, and synthesize the dark part details of the overexposed image, the intermediate details of the normally exposed image, and the bright part details of the underexposed image to obtain an HDR image. However, the images processed by the related HDR technology are difficult to be used for preview, photography and video recording at the same time.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image processing method, device, storage medium, and electronic device, and the image obtained by the processing can be suitable for previewing, photographing, and video recording.

An embodiment of the present application provides an image processing method, which is applied to an electronic device, where the electronic device includes a first camera module and a second camera module, wherein the second camera module is a telephoto camera module, and the Methods include:

Determine a first target camera module and a second target camera module from the first camera module and the second camera module;

A first RAW image is acquired through the first target camera module, and a second RAW image and a third RAW image are acquired through the second target camera module, wherein the first RAW image, the second RAW image and the The exposure of the third RAW image decreases sequentially;

Taking the second RAW image as a reference image, performing synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW composite image with a high dynamic range;

Using the RAW composite image, image preview or photographing or video recording operations are performed.

An embodiment of the present application provides an image processing apparatus, which is applied to an electronic device, where the electronic device includes a first camera module and a second camera module, wherein the second camera module is a telephoto camera module, and the The device includes:

a determining module, for determining a first target camera module and a second target camera module from the first camera module and the second camera module;

An acquisition module, configured to acquire a first RAW image through the first target camera module, and acquire a second RAW image and a third RAW image through the second target camera module, wherein the first RAW image, The exposures of the second RAW image and the third RAW image are sequentially decreased;

a synthesis module, configured to use the second RAW image as a reference image to perform synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW synthesized image with a high dynamic range;

The processing module is used for synthesizing the image by using the RAW, and performing image preview or photographing or video recording operations.

The embodiments of the present application provide a storage medium on which a computer program is stored, and when the computer program is executed on a computer, the computer is caused to execute the image processing method provided by the embodiments of the present application.

Embodiments of the present application further provide an electronic device, including a memory and a processor, where the processor is configured to execute the image processing method provided by the embodiments of the present application by invoking a computer program stored in the memory.

In this embodiment, since the first RAW image, the second RAW image and the third RAW image are all fully exposed RAW images, the first RAW image, the second RAW image and the third RAW image are used to perform high dynamic range The image obtained after the synthesis processing will not lose its resolution, and its signal-to-noise ratio is also high, that is, the RAW composite image with high dynamic range obtained in this embodiment has the advantages of high resolution and high signal-to-noise ratio. This high-resolution, high signal-to-noise ratio RAW composite image with high dynamic range can be directly used for image preview, photography and video recording.

Description of drawings

The technical solutions of the present application and the beneficial effects thereof will be apparent through the detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings.

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present application.

FIG. 2 is a first synthesis schematic diagram of the image processing method provided by the embodiment of the present application.

FIG. 3 is another schematic flowchart of an image processing method provided by an embodiment of the present application.

FIG. 4 is a second synthesis schematic diagram of the image processing method provided by the embodiment of the present application.

5 is a schematic diagram of image synthesis when the first camera module is determined as the first target camera module, and the second camera module (ie, the telephoto camera module) is determined as the second target camera module.

6 is a schematic diagram of image synthesis when the second camera module (ie the telephoto camera module) is determined as the first target camera module, and the first camera module is determined as the second target camera module.

7 to 11 are schematic diagrams of scenarios of an image processing method provided by an embodiment of the present application.

FIG. 12 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.

FIG. 13 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

FIG. 14 is a schematic structural diagram of an image processing circuit provided by an embodiment of the present application.

Detailed ways

Please refer to the drawings, wherein the same component symbols represent the same components, and the principles of the present application are exemplified by being implemented in a suitable computing environment. The following description is based on illustrated specific embodiments of the present application and should not be construed as limiting other specific embodiments of the present application not detailed herein.

It can be understood that, the executive body of the embodiment of the present application may be an electronic device such as a smart phone or a tablet computer.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method can be applied to an electronic device, and the electronic device can have a first camera module and a second camera module, wherein the second camera module can be a telephoto camera module. The flow of the image processing method may include:

101. Determine a first target camera module and a second target camera module from the first camera module and the second camera module.

102. Obtain a first RAW image through a first target camera module, and obtain a second RAW image and a third RAW image through a second target camera module, wherein the first RAW image, the second RAW image and the third RAW image The exposure of the image decreases sequentially.

Compared with ordinary images, High-Dynamic Range (HDR) images can provide more dynamic range and image details. The electronic device can shoot multiple frames of images with different exposure degrees in the same scene, and synthesize the dark part details of the overexposed image, the intermediate details of the normally exposed image, and the bright part details of the underexposed image to obtain an HDR image. However, the images processed by the related HDR technology are difficult to be used for preview, photography and video recording at the same time.

For example, when using zzHDR technology for image processing, because zzHDR technology has both long-exposure pixels and short-exposure pixels in the same frame of image, when long-exposure pixels and short-exposure pixels are combined to generate a high dynamic range image, the reduction will be reduced by at least half the resolution. Therefore, the high dynamic range images obtained by using zzHDR technology cannot be used for photography or video recording, otherwise the user will obviously see the image display effect with reduced resolution. That is, zzHDR technology cannot be applied to the scene of taking pictures and videos. Other HDR technologies in the related art also have problems such as reduced resolution or low signal-to-noise ratio, so that the images processed by these HDR technologies cannot be used for previewing, photographing and video recording at the same time.

In the processes of 101 and 102 in this embodiment of the present application, for example, the electronic device may first determine the first target from two camera modules, that is, the first camera module and the second camera module (telephoto camera module). a camera module and a second target camera module.

After that, the electronic device can obtain the first RAW image through the determined first target camera module, and obtain the second RAW image and the third RAW image through the determined second target camera module. Wherein, the exposure of the first RAW image, the second RAW image and the third RAW image decreases sequentially. That is, the exposure of the first RAW image may be greater than that of the second RAW image, and the exposure of the second RAW image may be greater than that of the third RAW image.

It should be noted that the camera module of the electronic device is composed of a lens and an image sensor. The lens is used to collect external light source signals and provide them to the image sensor. The image sensor senses the light source signal from the lens and converts it into a digitized original image. data, that is, RAW image data. RAW is an unprocessed, uncompressed format that can be aptly called a "digital negative".

Exposure refers to the degree of exposure of an image. Exposure levels can include overexposure, normal exposure, and underexposure. Exposure, also known as exposure value, represents all camera aperture-shutter combinations that give the same exposure.

In this embodiment, the sequentially decreasing exposure of the first RAW image, the second RAW image, and the third RAW image may include the following situations: for example, the first RAW image may be an overexposed image, and the second RAW image may be a normal exposure image, the third RAW image may be an underexposed image. For another example, the first RAW image, the second RAW image, and the third RAW image may be images with successively decreasing exposure times. For example, in comparison, the first RAW image may be a long-exposure image with a longer exposure time, and the second RAW image may be The image may be a medium exposure image with an intermediate exposure time, the third RAW image may be a short exposure image with a shorter exposure time, and so on.

That is, in 101 and 102 of this embodiment, the electronic device may first select a first target camera module for overexposure or long exposure from the first camera module and the second camera module, and select a first target camera module for performing overexposure or long exposure. A second target camera module for normal exposure and underexposure or medium exposure and short exposure. After selecting the first target camera module and the second target camera module, the electronic device can obtain the first RAW image through the first target camera module, and obtain the second RAW image and the first RAW image through the second target camera module Three RAW images.

103. Using the second RAW image as a reference image, perform compositing processing on the first RAW image, the second RAW image, and the third RAW image, to obtain a RAW composite image with a high dynamic range.

For example, after acquiring the first RAW image, the second RAW image and the third RAW image, the electronic device may use the second RAW image as a reference image, and perform HDR on the first RAW image, the second RAW image and the third RAW image Composite processing, resulting in a RAW composite image with high dynamic range.

That is, since the exposure of the first RAW image is the highest and the exposure of the third RAW image is the lowest, the electronic device can obtain the details of the dark parts from the first RAW image, obtain the details of the bright parts from the third RAW image, and obtain the The dark details and highlights of the RAW image are HDR composited with the second RAW image, resulting in a RAW composite image with high dynamic range.

104. Use RAW to synthesize images, and perform image preview or photographing or video recording operations.

For example, after obtaining a RAW composite image with a high dynamic range, the electronic device can use the RAW composite image to perform image preview or photographing or video recording operations. For example, the electronic device may display the preview interface of the camera application of the electronic device for the user to preview after performing certain processing on the RAW composite image. Alternatively, when the electronic device receives a photographing instruction, for example, when the user presses the photographing button, the electronic device may perform certain processing on the RAW composite image and display it on the display screen as a photo output for the user to view. Or, when the electronic device receives the video recording instruction, the electronic device may use the RAW composite image as one of the frames of the video obtained by video recording after performing certain processing on the RAW composite image.

Please refer to FIG. 2 at the same time. FIG. 2 is a first synthesis schematic diagram of the image processing method provided in this embodiment.

It should be noted that, in this embodiment, since the first RAW image, the second RAW image and the third RAW image are all fully exposed RAW images, the first RAW image, the second RAW image and the third RAW image are used The image obtained after high dynamic range synthesis processing will not lose its resolution, and its signal-to-noise ratio is also high, that is, the RAW composite image with high dynamic range obtained in this embodiment has high resolution and signal-to-noise ratio. higher advantage. This high-resolution, high signal-to-noise ratio RAW composite image with high dynamic range can be directly used for image preview, photography and video recording.

Please refer to FIG. 3 , which is another schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method can be applied to electronic equipment. The electronic device may include a first camera module and a second camera module, wherein the second camera module may be a telephoto camera module. The flow of the image processing method may include:

201. The electronic device acquires the depth of field of the captured image through the first camera module and the second camera module.

For example, in the embodiment of the present application, the first camera module is an ordinary camera module, and the second camera module is a telephoto camera module. The electronic device may first obtain the depth of field of the images captured by the first camera module and the second camera module.

It should be noted that there is a permissible circle of confusion before and after the far focus and near focus. The range between this range is called the depth of field, that is to say, before and after the focus point, the image still has a clear range, and the front and back of the subject is. Depth, controls the blur of the image on the sensor. Aperture, lens and subject distance are important factors that affect depth of field. The depth of field information can be calculated by using the dual camera module. For example, after the user clicks the camera application to enter the preview screen, the electronic device can use the first camera module and the second camera module to capture an image, and then use the first camera module and the second camera module to capture an image. The depth of field information of the image can be calculated from the two images. The method in the prior art may be adopted for calculating the depth of field information by using the dual camera modules, which will not be repeated in this embodiment.

202. If the depth of field is greater than or equal to the preset threshold, the electronic device determines the first camera module as the first target camera module, and determines the second camera module as the second target camera module.

For example, after acquiring the depth of field, the electronic device can detect whether the depth of field is greater than or equal to a preset threshold.

If it is detected that the depth of field is greater than or equal to the preset threshold, it can be considered that the current shooting scene is mostly distant. At this time, the electronic device may determine the first camera module as the first target camera module, and determine the second camera module (ie, the telephoto camera module) as the second target camera module.

It should be noted that, in this embodiment, the first target camera module may be a camera module for capturing a RAW image with maximum exposure (ie, the first RAW image). The second target camera module may be a camera module for capturing a RAW image with an intermediate exposure (ie, a second RAW image) and a RAW image with a minimum exposure (ie, a third RAW image). Since this embodiment can use a second RAW image with an intermediate exposure as a reference image when performing HDR synthesis, this embodiment uses a telephoto camera module to shoot the second RAW image in a shooting scene with a large number of distant views. The image can make the details (especially the distant view details) of the second RAW image clearer, that is, the second RAW image captured by the telephoto camera module has better clarity. Then, use the second RAW image as the reference image to perform HDR composition processing (for example, obtain details of dark parts from the first RAW image and obtain details of bright parts from the third RAW image during HDR composition processing, etc., and obtain the The image quality of the resulting RAW composite image will also be better.

203. The electronic device obtains the first RAW image through the first target camera module, and obtains the second RAW image and the third RAW image through the second target camera module, wherein the first RAW image, the second RAW image and the third RAW image are The exposure of the RAW image decreases sequentially, the second target camera module obtains the second RAW image and the third RAW image by means of alternate exposure, and the frame rates of the first target camera module and the second module camera module are the same.

For example, after determining the first target camera module and the second target camera module, the electronic device can obtain the first RAW image through the first target camera module, and obtain the second RAW image through the second target camera module and the third RAW image.

The frame rates of the first target camera module and the second target camera module may be the same. For example, the frame rates of the first target camera module and the second target camera module are both 30fps. Of course, in some other embodiments, the frame rates of the first target camera module and the second target camera module can also be other values. Moreover, the frame rate of the first target camera module and the second target camera module may be a value not lower than 20fps. When the frame rate of the first target camera module and the second target camera module is not lower than 20fps, the RAW composite image of the image composite obtained by the first target camera module and the second target camera module is subsequently used. When previewing or recording images, the screen will be smoother and will not be stuck.

And, the exposure degrees of the first RAW image, the second RAW image, and the third RAW image are sequentially decreased. That is, the exposure of the first RAW image may be greater than that of the second RAW image, and the exposure of the second RAW image may be greater than that of the third RAW image. In some embodiments, the first RAW image may be an overexposed image, the second RAW image may be a normally exposed image, and the third RAW image may be an underexposed image. Alternatively, under the condition that other exposure parameters are the same, the exposure time of the first RAW image may be longer than that of the second RAW image, and the exposure time of the second RAW image may be longer than that of the third RAW image. That is, the first RAW image may be a long exposure image, the second RAW image may be a medium exposure image, and the third RAW image may be a short exposure image.

In this embodiment, the second target camera module acquires the second RAW image and the third RAW image by means of alternate exposure. For example, the second target camera module may alternately perform medium exposure and short exposure to obtain corresponding images. That is, in this embodiment, the first target camera module can continuously acquire long-exposure images, and the second target camera module can continuously and alternately acquire medium-exposure images and short-exposure images.

In this embodiment, the acquired overexposed images (or long exposure images) may be collectively recorded as the first RAW image, the normal exposure images (or medium exposure images) may be collectively recorded as the second RAW image, and the underexposed images may be collectively recorded as the second RAW image. (or short exposure images) are collectively recorded as the third RAW image.

204. The electronic device uses the second RAW image as a reference image, and obtains the Nth frame image obtained by the first target camera module, the Nth frame image obtained by the second target camera module, and the second target camera module. The N+1-th frame image is synthesized, and a RAW synthesized image with high dynamic range is obtained.

For example, in the process that the first target camera module continuously acquires long-exposure images, and the second target camera module continuously and alternately acquires medium-exposure images and short-exposure images, the electronic device may The N-th frame image obtained by the group, the N-th frame image obtained by the second target camera module, and the N+1-th frame image obtained by the second target camera module are synthesized and processed to obtain RAW synthesis with high dynamic range. image. Wherein, when performing HDR composition, the electronic device may use the second RAW image as a reference image for HDR composition. That is, the electronic device can obtain the details of the dark parts from the first RAW image, obtain the details of the bright parts from the third RAW image, and perform HDR synthesis on the obtained details of the dark parts and the bright parts with the second RAW image, so as to obtain a high-quality image with high RAW composite images with dynamic range.

Please refer to FIG. 4 at the same time. FIG. 4 is a second synthesizing schematic diagram of the image processing method provided in this embodiment. For example, the first target camera module performs long exposures in chronological order to obtain the following images: L1, L2, L3, L4 (that is, L1 is the first frame image captured by the first target camera module, and L2 is the first target camera module. L3 is the third frame image captured by the first target camera module, L4 is the fourth frame image captured by the first target camera module) and so on. It can be understood that L1 , L2 , L3 and L4 have the same exposure degree, for example, under the condition that other exposure parameters are the same, L1 , L2 , L3 and L4 have the same exposure time t1 . The second target camera module alternately performs medium exposure and short exposure in chronological order to obtain the following images: M1, S1, M2, S2, M3 (that is, M1 is the first frame image captured by the second target camera module, and S1 is the first frame of image captured by the second target camera module. The second frame image captured by the two-target camera module, M2 is the third frame image captured by the second target camera module, S2 is the fourth frame image captured by the second target camera module, and M3 is the second target camera module. The fifth frame image taken) and so on. It can be understood that M1, M2, and M3 have the same exposure. For example, when other exposure parameters are the same, M1, M2, and M3 have the same exposure time t2. S1 and S2 have the same exposure, for example, in the case of other exposure parameters being the same, S1 and S2 have the same exposure time t3. Then, during HDR synthesis, the electronic device synthesizes L1, M1 and S1 to obtain a P1 image (with M1 as the reference image for HDR synthesis), and synthesizes L2, S1 and M2 to obtain a P2 image (with M2 as the HDR synthesis) Baseline image), combine L3, M2, and S2 to obtain a P3 image (use M2 as the base image for HDR synthesis), and synthesize L4, S2, and M3 to obtain a P4 image (use M3 as the base image for HDR synthesis). That is, the electronic device can sequentially obtain P1, P2, P3 and P4 images with high dynamic range.

205. The electronic device converts the RAW composite image into a YUV composite image.

206. After the electronic device performs preset processing on the YUV composite image, preview the image.

For example, 205 and 206 could include:

After obtaining the RAW composite image, the electronic device may convert the RAW composite image from the RAW format to the YUV format, thereby obtaining the YUV composite image.

After that, the electronic device can perform preset processing such as image sharpening, image noise reduction, etc. on the YUV composite image, and after the preset processing, use the image obtained after the processing to perform an image preview operation. For example, the electronic device may display the image obtained by the preset processing of the YUV composite image on the preview interface of the camera application of the electronic device for the user to preview.

See also Figure 5 and Figure 6. 5 is a schematic diagram of image synthesis when the electronic device determines the first camera module as the first target camera module and the second camera module (ie the telephoto camera module) as the second target camera module. 6 is a schematic diagram of image synthesis when the electronic device determines the second camera module (ie the telephoto camera module) as the first target camera module and the first camera module as the second target camera module.

207. The electronic device stores the preset processed YUV composite image into a preset image cache queue.

208. When performing a photographing operation, the electronic device acquires a frame of YUV composite image from a preset image buffer queue, and displays the obtained YUV composite image as a photo.

For example, 207 and 208 could include:

After performing preset processing such as image sharpening and image noise reduction on the YUV composite image, the electronic device may store the processed image in a preset image buffer queue.

For example, after previewing the image, the user clicks the camera button of the camera application of the electronic device to take a photo, then the electronic device can obtain a frame of the pre-processed YUV composite image from the preset image cache queue, and will obtain the The obtained image is displayed as a photo, for example, displayed on the display screen of the electronic device for the user to check the photo effect.

In an embodiment, after 201, the following process may be further included: if it is detected that the depth of field is smaller than a preset threshold, it may be considered that the current shooting scene is a close-up scene, and there is not much distant view. At this time, the electronic device may determine the second camera module (ie, the telephoto camera module) as the first target camera module, and the first camera module as the second target camera module. After the first target camera module and the second target camera module are determined, the electronic device may obtain the first RAW image through the first target camera module, and obtain the second RAW image and the third target camera module through the second target camera module RAW images, wherein the exposure of the first RAW image, the second RAW image, and the third RAW image decreases sequentially, and the second target camera module acquires the second RAW image and the third RAW image by means of alternate exposure, and the first target The frame rates of the camera module and the camera module of the second module are the same. After that, the electronic device synthesizes the Nth frame image obtained by the first target camera module, the Nth frame image obtained by the second target camera module, and the N+1th frame image obtained by the second target camera module processing, resulting in a RAW composite image with high dynamic range. After obtaining the RAW composite image, the electronic device can use the RAW composite image to perform image preview or photographing or video recording operations.

In one embodiment, the electronic device performs composite processing on the first RAW image, the second RAW image, and the third RAW image to obtain a RAW composite image with a high dynamic range, which may include:

The electronic device determines the second RAW image as a reference image for image alignment processing, where the image alignment processing at least includes image brightness alignment processing and image position alignment processing;

After the images are aligned, the electronic device performs composite processing on the first RAW image, the second RAW image, and the third RAW image to obtain a RAW composite image with a high dynamic range.

For example, the first RAW image is an overexposed image, the second RAW image is a normal exposure image, and the third RAW image is an underexposed image. Then, the electronic device may determine the normally exposed second RAW image as a reference image for image alignment processing, where the image alignment processing at least includes image brightness alignment processing and image position alignment processing. That is, before performing HDR fusion, the electronic device needs to perform image brightness alignment and image position alignment on the first RAW image, the second RAW image, and the third RAW image. In this embodiment of the present application, the electronic device may first determine a normally exposed second RAW image as a reference image for image alignment processing, and then perform image brightness alignment on the first RAW image and the third RAW image using the second RAW image as a reference and position alignment. After the images are aligned, the electronic device can perform HDR composition on the first RAW image, the second RAW image and the third RAW image (the second RAW image can still be used as a composite reference image during HDR composition), so as to obtain a high dynamic Range of RAW composite images.

It should be noted that using the second RAW image with moderate exposure as the reference image for the image alignment process can make the image alignment process more accurate, thereby making the final synthesized image have a better imaging effect.

In this embodiment, the first RAW image, the second RAW image, and the third RAW image used for HDR composition may be images whose shooting time interval is shorter than a preset interval. For example, the long-exposure image, the medium-exposure image, and the short-exposure image used for HDR composition may be images whose shooting time interval is shorter than a preset interval. The electronic device may generate a time stamp corresponding to each image when the image is captured. Based on the time stamp of shooting, the electronic device can detect whether the corresponding long-exposure image, medium-exposure image, and short-exposure image are images whose shooting time interval is less than a preset interval during HDR synthesis. If so, the electronic device may perform HDR composition. If not, the electronic device may not perform HDR composition on these images, but perform frame dropping processing on these images, such as deleting these images.

Please refer to FIG. 7 to FIG. 11 . FIGS. 7 to 11 are schematic diagrams of scenarios of the image processing method provided by the embodiments of the present application.

For example, as shown in FIG. 7, the electronic device has a first camera module 301 and a second camera module 302, wherein the second camera module 302 is a telephoto camera module, and the first camera module 301 can be Ordinary camera module. For example, the first camera module 301 may be a common camera module that is neither a telephoto camera module nor a wide-angle camera module. Moreover, the first camera module 301 and the second camera module 302 have the same frame rate, for example, their frame rates are both 30fps or 60fps, and so on.

For example, as shown in Figure 8, the user clicks the icon of the camera application and points the camera at a certain scene. At this time, the electronic device enters the preview interface of the camera application. When it is detected that the preview interface is entered, the first camera module can capture an image, and the second camera module can also capture an image. After that, the electronic device can obtain the corresponding depth of field through the images captured by the first camera module and the second camera module. For example, the electronic device may synthesize the image captured by the first camera module and the image captured by the second camera module, and calculate the depth of field information of the synthesized image.

After acquiring the depth of field, the electronic device can detect whether the depth of field is greater than or equal to a preset threshold. For example, in this embodiment, if the electronic device detects that the depth of field is greater than or equal to the preset threshold, it can be considered that the current shooting scene is mostly distant. At this time, the electronic device may determine the first camera module as the first target camera module, and determine the second camera module (ie, the telephoto camera module) as the second target camera module.

For another example, if it is detected that the depth of field is smaller than the preset threshold, it can be considered that the current shooting scene is mainly close-range, and there is not much distant view. At this time, the electronic device may determine the second camera module (ie, the telephoto camera module) as the first target camera module, and the first camera module as the second target camera module.

After the first target camera module and the second target camera module are determined, the electronic device may enter the HDR mode. For example, in the HDR mode, the electronic device can continuously acquire RAW images according to the shooting interval through the image sensor of the first target camera module. For example, this embodiment is recorded as the first RAW image, and the exposure time of the first RAW image is T1. At the same time, the electronic device can alternately acquire RAW images with exposure times T2 and T3 through the image sensor of the second target camera module. For example, the RAW images with exposure time T2 are collectively recorded as the second RAW images, and The RAW image of T3 is collectively recorded as the third RAW image. It can be understood that the first RAW image, the second RAW image and the third RAW image are the images of the current scene on which the camera is aimed. Since the shooting time interval of the first RAW image, the second RAW image, and the third RAW image is very short, it can be considered that they are images obtained in the same scene. In one embodiment, the exposure parameters of the first RAW image, the second RAW image and the third RAW image may be the same except for different exposure times.

In this embodiment, the exposure time T1 is greater than T2, and T2 is greater than T3. That is, the first target camera module performs long exposure to obtain a long-exposure RAW image. The second target camera module alternately performs medium and short exposures to obtain RAW images of medium exposure and short exposure. For example, as shown in FIG. 4 , in chronological order, the electronic device can obtain long-exposure RAW images L1 , L2 , L3 and L4 through the image sensor of the first target camera module, and so on. Moreover, in a chronological order, the electronic device can alternately acquire medium and short exposure RAW images M1 , S1 , M2 , S2 , M3 and so on through the image sensor of the second target camera module.

After acquiring the long-exposure image, the medium-exposure image, and the short-exposure image, the electronic device may perform HDR composite processing on the long-exposure image, the medium-exposure image, and the short-exposure image, to obtain a RAW composite image with a high dynamic range. The images used for HDR synthesis processing may be the Nth frame image obtained by the first target camera module, the Nth frame image obtained by the second target camera module, and the Nth frame image obtained by the second target camera module. N+1 frames of images. Moreover, when performing HDR composition, the electronic device can use the medium exposure image as a reference image for HDR composition.

For example, as shown in FIG. 4 , after acquiring the images L1, M1 and S1, the electronic device can perform HDR synthesis processing on the first images L1, M1 and S1 (with M1 as the reference image for HDR synthesis) to obtain an image with high dynamic range. Image P1. After acquiring the image P1, the electronic device can convert the image P1 into a YUV format image, and perform preset processing such as image sharpening and image noise reduction on the format-converted image to obtain a processed image. Afterwards, the electronic device may display the processed image on the display screen, that is, display it for the user to view in the preview interface.

After acquiring the images L2 and M2, the electronic device may perform HDR synthesis processing on the first images L2, M2 and S1 (using M2 as the reference image for HDR synthesis) to obtain an image P2 with a high dynamic range. After acquiring the images L3 and S2, the electronic device may perform HDR synthesis processing on the first images L3, M2 and S2 (with M2 as the reference image for HDR synthesis) to obtain an image P3 with a high dynamic range. After acquiring the images L4 and M3, the electronic device may perform HDR synthesis processing on the first images L4, M3 and S2 (using M3 as the reference image for HDR synthesis) to obtain an image P4 with a high dynamic range, and so on.

After obtaining the images P2, P3 and P4 in turn, the electronic device can also convert the images P2, P3 and P4 into images in YUV format, and perform preset processing such as image sharpening and image noise reduction on the format-converted images. Get the processed image. Afterwards, the electronic device may display the processed image on the display screen, that is, display it for the user to view in the preview interface. That is, the images P1, P2, P3, and P4 after format conversion and preset processing are sequentially displayed on the preview interface.

It can be understood that, in this embodiment, the image displayed on the preview interface may be recorded as a preview frame image. Each time the electronic device obtains a preview frame image, the corresponding preview frame image may be stored in the preset image buffer queue. The preset image buffer queue may be a fixed-length queue or a non-fixed-length queue.

For example, in this embodiment, the preset image buffer queue is a fixed-length queue, and the queue length is 30 frames. For example, the electronic device sequentially obtains images Y1 , Y2 , Y3 , Y4 and so on for preview. Then, the electronic device may store the obtained images Y1, Y2, Y3, and Y4 in the preset image buffer queue in sequence. It can be understood that, the preset image cache queue always stores the latest 30 frames of preview frame images, as shown in FIG. 9 .

Afterwards, for example, as shown in FIG. 10, the user clicks the camera button in the camera application interface. At this time, the electronic device can obtain a preview frame from the preset image cache queue, and display the obtained preview frame as a photo on the display. On the screen for users to view the photo effect. For example, the electronic device acquires the image Y60 from a preset image buffer queue. Then, the electronic device can display the image Y60 as a photo for the user to view, as shown in FIG. 11 .

It can be understood that, in this embodiment, when taking pictures, the electronic device can directly acquire images with high dynamic range from the preset image buffer queue, and display them to the user for viewing, so that the effect of taking pictures with zero delay can be achieved.

Please refer to FIG. 12 , which is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application. The image processing apparatus can be applied to electronic equipment, and the electronic equipment can include a first camera module and a second camera module, wherein the second camera module is a telephoto camera module. The image processing apparatus 400 may include: a determination module 401 , an acquisition module 402 , a synthesis module 403 , and a processing module 404 .

The determining module 401 is configured to determine a first target camera module and a second target camera module from the first camera module and the second camera module.

The acquiring module 402 is configured to acquire a first RAW image through the first target camera module, and acquire a second RAW image and a third RAW image through the second target camera module, wherein the first RAW image , the second RAW image, and the third RAW image are sequentially decreased in exposure.

The synthesis module 403 is configured to use the second RAW image as a reference image to perform synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW synthesized image with a high dynamic range.

The processing module 404 is configured to use the RAW composite image to perform image preview or photographing or video recording operations.

In one embodiment, the determining module 401 may be used to:

Obtain the depth of field of the captured image through the first camera module and the second camera module;

According to the depth of field, a first target camera module and a second target camera module are determined from the first camera module and the second camera module.

In one embodiment, the determining module 401 may be used to:

If the depth of field is greater than or equal to a preset threshold, the first camera module is determined as the first target camera module, and the second camera module is determined as the second target camera module;

Alternatively, if the depth of field is smaller than a preset threshold, the second camera module is determined as the first target camera module, and the first camera module is determined as the second target camera module.

In one embodiment, the frame rates of the first camera module and the second camera module are the same.

In one embodiment, the exposure times of the first RAW image, the second RAW image and the third RAW image are sequentially decreased.

In one embodiment, the second target camera module acquires the second RAW image and the third RAW image by means of alternate exposure.

The synthesizing module 403 can be used for: the Nth frame image obtained by the first target camera module, the Nth frame image obtained by the second target camera module, and the second target camera module. The acquired N+1 th frame image is synthesized to obtain a RAW synthesized image with high dynamic range.

In one embodiment, the processing module 404 may be used to:

converting the RAW composite image to a YUV composite image;

After the preset processing is performed on the YUV composite image, a preview of the image is performed.

In one embodiment, the processing module 404 may be used to:

Store the pre-processed YUV composite image in the preset image cache queue;

During the photographing operation, a frame of YUV composite image is obtained from the preset image buffer queue, and the obtained YUV composite image is displayed as a photo.

An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed on a computer, causes the computer to execute the process in the image processing method provided in this embodiment.

An embodiment of the present application further provides an electronic device, including a memory and a processor, where the processor is configured to execute the process in the image processing method provided by the present embodiment by invoking a computer program stored in the memory.

For example, the above-mentioned electronic device may be a mobile terminal such as a tablet computer or a smart phone. Please refer to FIG. 13 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

The electronic device 500 may include a camera module 501 , a memory 502 , a processor 503 and other components. Those skilled in the art can understand that the structure of the electronic device shown in FIG. 13 does not constitute a limitation to the electronic device, and may include more or less components than the one shown, or combine some components, or arrange different components.

The camera module 501 may be a dual camera module installed on an electronic device or the like. The camera module 501 may include a first camera module and a second camera module. The second camera module may be a telephoto camera module.

Memory 502 may be used to store applications and data. The application program stored in the memory 502 contains executable code. Applications can be composed of various functional modules. The processor 503 executes various functional applications and data processing by running the application programs stored in the memory 502 .

The processor 503 is the control center of the electronic device, uses various interfaces and lines to connect various parts of the entire electronic device, and executes the electronic device by running or executing the application program stored in the memory 502 and calling the data stored in the memory 502. The various functions and processing data of the device are used to monitor the electronic equipment as a whole.

In this embodiment, the processor 503 in the electronic device loads the executable code corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 503 executes and stores it in the memory 502 in the application, which executes:

Determine a first target camera module and a second target camera module from the first camera module and the second camera module;

A first RAW image is acquired through the first target camera module, and a second RAW image and a third RAW image are acquired through the second target camera module, wherein the first RAW image, the second RAW image and the The exposure of the third RAW image decreases sequentially;

Taking the second RAW image as a reference image, performing synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW composite image with a high dynamic range;

Using the RAW composite image, image preview or photographing or video recording operations are performed.

Embodiments of the present invention also provide an electronic device. The above electronic device includes an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units that define an image signal processing (Image Signal Processing) pipeline. The image processing circuit may at least include: a camera, an image signal processor (Image Signal Processor, ISP processor), a control logic, an image memory, a display, and the like. The camera may at least include one or more lenses and an image sensor.

The image sensor may include an array of color filters (eg, Bayer filters). The image sensor can acquire light intensity and wavelength information captured with each imaging pixel of the image sensor and provide a set of raw image data that can be processed by an image signal processor.

Image signal processors can process raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the image signal processor may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision. The raw image data can be stored in the image memory after being processed by the image signal processor. The image signal processor may also receive image data from the image memory.

The image memory may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, Direct Memory Access) feature.

When receiving image data from the image memory, the image signal processor may perform one or more image processing operations, such as temporal filtering. The processed image data can be sent to the image memory for additional processing before being displayed. The image signal processor may also receive processed data from the image memory and process the processed data as image data in the raw domain and in the RGB and YCbCr color spaces. The processed image data may be output to a display for viewing by a user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the image signal processor can also be sent to an image memory, and the display can read image data from the image memory. In one embodiment, the image memory may be configured to implement one or more frame buffers.

Statistics determined by the image signal processor may be sent to the control logic. For example, the statistics may include statistics of the image sensor for auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens shading correction, etc.

Control logic may include a processor and/or microcontroller executing one or more routines (eg, firmware). One or more routines may determine camera control parameters and ISP control parameters based on the received statistics. For example, camera control parameters may include camera flash control parameters, lens control parameters (eg, focal length for focusing or zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices, etc. for automatic white balance and color adjustment (eg, during RGB processing).

Please refer to FIG. 14 , which is a schematic structural diagram of an image processing circuit in this embodiment. As shown in FIG. 14 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present invention are shown.

The image processing circuit may include: a first camera module 510 , a second camera module 520 , a first image signal processor 530 , a second image signal processor 540 , a control logic 550 , an image memory 560 , and a display 570 . The first camera module 510 may include one or more first lenses 511 and a first image sensor 512 . The second camera module 520 may include one or more second lenses 521 and a second image sensor 522 .

The first image collected by the first camera module 510 is transmitted to the first image signal processor 530 for processing. After the first image signal processor 530 processes the first image, the statistical data of the first image (such as the brightness of the image, the contrast value of the image, the color of the image, etc.) may be sent to the control logic 550 . The control logic 550 can determine the control parameters of the first camera module 510 according to the statistical data, so that the first camera module 510 can perform automatic focusing, automatic exposure and other operations according to the control parameters. The first image may be stored in the image memory 560 after being processed by the first image signal processor 530 . The first image signal processor 530 may also read images stored in the image memory 560 for processing. In addition, the first image can be directly sent to the display 570 for display after being processed by the image signal processor 530 . Display 570 may also read images in image memory 560 for display.

The second image collected by the second camera module 520 is transmitted to the second image signal processor 540 for processing. After the second image signal processor 540 processes the second image, the statistical data of the second image (such as the brightness of the image, the contrast value of the image, the color of the image, etc.) may be sent to the control logic 550 . The control logic 550 can determine the control parameters of the second camera module 520 according to the statistical data, so that the second camera module 520 can perform automatic focusing, automatic exposure and other operations according to the control parameters. The second image may be stored in the image memory 560 after being processed by the second image signal processor 540 . The second image signal processor 540 may also read images stored in the image memory 560 for processing. In addition, the second image can be directly sent to the display 570 for display after being processed by the image signal processor 540 . Display 570 may also read images in image memory 560 for display.

In other embodiments, the first image signal processor and the second image signal processor can also be combined into a unified image signal processor, which respectively processes the data of the first image sensor and the second image sensor.

In addition, not shown in the figures, the electronic device may further include a CPU and a power supply module. The CPU is connected to the logic controller, the first image signal processor, the second image signal processor, the image memory, and the display, and the CPU is used to implement global control. The power supply module is used to supply power to each module.

Generally, for mobile phones with dual camera modules, in some camera modes, the dual camera modules work. At this time, the CPU controls the power supply module to supply power to the first camera module and the second camera module. The image sensor in the first camera module is powered on, and the image sensor in the second camera module is powered on, so that image acquisition and conversion can be implemented. In some camera modes, one camera in the dual camera module can work. For example, only the telephoto camera works. In this case, the CPU controls the power supply module to supply power to the image sensor of the corresponding camera.

The following is the process of implementing the image processing method provided by this embodiment using the image processing technology in FIG. 14 :

Determine a first target camera module and a second target camera module from the first camera module and the second camera module;

A first RAW image is acquired through the first target camera module, and a second RAW image and a third RAW image are acquired through the second target camera module, wherein the first RAW image, the second RAW image and the The exposure of the third RAW image decreases sequentially;

Taking the second RAW image as a reference image, performing synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW composite image with a high dynamic range;

Using the RAW composite image, image preview or photographing or video recording operations are performed.

In one embodiment, when the electronic device determines the first target camera module and the second target camera module from the first camera module and the second camera module, it may perform: through the first camera module and the second target camera module. A camera module and the second camera module obtain the depth of field of the captured image; according to the depth of field, determine the first target camera module and the second camera module from the first camera module and the second camera module The second target camera module.

In one embodiment, when the electronic device determines the first target camera module and the second target camera module from the first camera module and the second camera module according to the depth of field information, it can Execute: if the depth of field is greater than or equal to a preset threshold, then determine the first camera module as the first target camera module, and determine the second camera module as the second target camera module; or , if the depth of field is smaller than the preset threshold, the second camera module is determined as the first target camera module, and the first camera module is determined as the second target camera module.

In one embodiment, the frame rates of the first camera module and the second camera module are the same.

In one embodiment, the exposure times of the first RAW image, the second RAW image and the third RAW image are sequentially decreased.

In one embodiment, the second target camera module acquires the second RAW image and the third RAW image by means of alternate exposure.

Then, when the electronic device performs synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW composite image with a high dynamic range, the electronic device may perform: acquiring the first target camera module The Nth frame image obtained, the Nth frame image obtained by the second target camera module, and the N+1 frame image obtained by the second target camera module are synthesized and processed to obtain a high dynamic range. RAW composite image.

In one embodiment, when the electronic device performs the preview of the image by using the RAW composite image, it may perform: converting the RAW composite image into a YUV composite image; after performing preset processing on the YUV composite image, Make a preview of the image.

In one embodiment, the electronic device may also perform: storing the pre-processed YUV composite image into a preset image buffer queue; when performing a photographing operation, obtain a frame of YUV composite image from the preset image buffer queue image, and the obtained YUV composite image is displayed as a photo.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the detailed description of the image processing method above, and details are not repeated here.

The image processing apparatus provided in the embodiments of the present application and the image processing methods in the above embodiments belong to the same concept, and any method provided in the image processing method embodiments can be executed on the image processing apparatus. For the implementation process, please refer to the embodiment of the image processing method, which will not be repeated here.

It should be noted that, for the image processing methods described in the embodiments of the present application, those of ordinary skill in the art can understand that all or part of the process for implementing the image processing methods described in the embodiments of the present application can be controlled by computer programs. To complete, the computer program can be stored in a computer-readable storage medium, such as a memory, and executed by at least one processor, and the execution process can include the flow of the embodiment of the image processing method . The storage medium may be a magnetic disk, an optical disk, a read only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), and the like.

For the image processing apparatus of the embodiments of the present application, each functional module may be integrated into one processing chip, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disk, etc. .

The image processing method, device, storage medium, and electronic device provided by the embodiments of the present application are described in detail above. The principles and implementations of the present application are described with specific examples. The descriptions of the above embodiments are only It is used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there will be changes in the specific embodiments and application scope. In summary, this specification The content should not be construed as a limitation on this application.

Claims (9)

1. An image processing method is applied to electronic equipment, and is characterized in that the electronic equipment comprises a first camera module and a second camera module, wherein the second camera module is a long-focus camera module, and the method comprises the following steps:

acquiring the depth of field of the shot image through the first camera module and the second camera module;

if the depth of field is greater than or equal to a preset threshold value, determining the first camera module as a first target camera module, and determining the second camera module as a second target camera module; if the depth of field is smaller than a preset threshold value, determining the second camera module as a first target camera module, and determining the first camera module as a second target camera module;

acquiring a first RAW image through the first target camera module, and acquiring a second RAW image and a third RAW image through the second target camera module, wherein the exposure levels of the first RAW image, the second RAW image and the third RAW image are sequentially reduced;

taking the second RAW image as a reference image, and performing synthesis processing on the first RAW image, the second RAW image and the third RAW image to obtain a RAW synthesis image with a high dynamic range;

and previewing or photographing or recording the image by using the RAW composite image.

2. The image processing method according to claim 1, wherein frame rates of the first camera module and the second camera module are the same.

3. The image processing method according to claim 2, wherein the sequentially decreasing exposure levels of the first, second, and third RAW images includes:

the exposure time of the first RAW image, the second RAW image, and the third RAW image is sequentially decreased.

4. The image processing method according to claim 2, wherein the second target camera module acquires a second RAW image and a third RAW image by means of alternate exposure;

synthesizing the first RAW image, the second RAW image and the third RAW image to obtain a RAW synthesized image with a high dynamic range, including: and synthesizing the Nth frame of image acquired by the first target camera module, the Nth frame of image acquired by the second target camera module and the (N + 1) th frame of image acquired by the second target camera module to obtain a RAW synthesized image with a high dynamic range.

5. The image processing method according to claim 1, wherein performing a preview of an image using the RAW composite image comprises:

converting the RAW composite image into a YUV composite image;

and previewing the image after performing preset processing on the YUV synthetic image.

6. The image processing method according to claim 5, characterized in that the method further comprises:

storing the YUV synthetic image after the preset processing into a preset image cache queue;

and when the photographing operation is carried out, acquiring a frame of YUV synthetic image from the preset image cache queue, and displaying the acquired YUV synthetic image as a photo.

7. The utility model provides an image processing device, is applied to electronic equipment, its characterized in that, electronic equipment includes first module and the second module of making a video recording, wherein the second module of making a video recording is the long focus module of making a video recording, the device includes:

the determining module is used for acquiring the depth of field of the shot image through the first camera module and the second camera module; if the depth of field is greater than or equal to a preset threshold value, determining the first camera module as a first target camera module, and determining the second camera module as a second target camera module; if the depth of field is smaller than a preset threshold value, determining the second camera module as a first target camera module, and determining the first camera module as a second target camera module;

the acquisition module is used for acquiring a first RAW image through the first target camera module and acquiring a second RAW image and a third RAW image through the second target camera module, wherein the exposure levels of the first RAW image, the second RAW image and the third RAW image are sequentially reduced;

the synthesis module is used for synthesizing the first RAW image, the second RAW image and the third RAW image by taking the second RAW image as a reference image to obtain a RAW synthesis image with a high dynamic range;

and the processing module is used for previewing or photographing or recording the images by using the RAW composite image.

8. A storage medium having stored thereon a computer program, characterized in that the computer program, when executed on a computer, causes the computer to execute the method according to any of claims 1 to 6.

9. An electronic device comprising a memory, a processor, wherein the processor is configured to perform the method of any of claims 1 to 6 by invoking a computer program stored in the memory.

Images