WO2023137956A1

WO2023137956A1 - Image processing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2023137956A1
Application number: PCT/CN2022/098717
Authority: WO
Inventors: 范文明
Original assignee: 上海闻泰信息技术有限公司
Priority date: 2022-01-18
Filing date: 2022-06-14
Publication date: 2023-07-27
Also published as: CN114418914A

Abstract

Disclosed in the embodiments of the present disclosure are an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: detecting a shooting scene where the electronic device is located; if detecting that the electronic device is in a backlit shooting scene, collecting, by means of a camera, a first image focused on a backlit area, and collecting, by means of the camera, a second image focused on a non-backlit area; separately performing enhancement processing on the first image and the second image; and fusing an enhanced first image and an enhanced second image so as to obtain a third image. By implementing the embodiments of the present disclosure, the image quality of an image shot in a backlit shooting scene can be improved.

Description

Image processing method, device, electronic device and storage medium

related cross reference

This disclosure claims the priority of the Chinese patent application with the application number 2022100575855 and the title of the invention "image processing method, device, electronic equipment and storage medium" submitted to the China Patent Office on January 18, 2022, the entire contents of which are incorporated in this disclosure by reference.

technical field

The present disclosure relates to an image processing method, device, electronic equipment and storage medium.

Background technique

With the advancement of science and technology and the ultimate pursuit of electronic devices by users, the requirements for the quality of images taken by electronic devices (such as mobile phones, etc.) are getting higher and higher, and the shooting functions of electronic devices are also becoming more and more abundant.

Contents of the invention

(1) Technical problems to be solved

In the prior art, when an electronic device shoots under strong light interference or a backlight environment, there is a problem of underexposure or overexposure, and the quality of the captured image is low.

(2) Technical solution

According to various embodiments of the present disclosure, an image processing method, device, electronic device, and storage medium are provided.

An image processing method applied to electronic equipment, the method comprising:

Detecting the shooting scene where the electronic device is located;

If it is detected that the electronic device is in a backlight shooting scene, the first image focused on the backlight area is collected by the camera, and the second image focused on the non-backlight area is collected by the camera;

performing enhancement processing on the first image and the second image respectively;

The enhanced first image and the enhanced second image are fused to obtain a third image.

As an optional implementation manner of the embodiment of the present disclosure, before said fusing the enhanced first image and the enhanced second image to obtain the third image, the method further includes: performing registration processing on the enhanced first image and the enhanced second image, so as to make the spatial position information of the enhanced first image consistent with the spatial position information of the enhanced second image; said fusing the enhanced first image and the enhanced second image to obtain the third image includes: The processed second image is fused to obtain a third image.

As an optional implementation manner of an embodiment of the present disclosure, the fusing the enhanced first image and the enhanced second image to obtain a third image includes: decomposing the enhanced first image into a first high-frequency component and a first low-frequency component, and decomposing the enhanced second image into a second high-frequency component and a second low-frequency component; wherein, the first high-frequency component corresponds to a first high-frequency region in the enhanced first image, and the first low-frequency component corresponds to a first low-frequency region in the enhanced first image; In the second high-frequency region in the second image, the second low-frequency component corresponds to the second low-frequency region in the enhanced second image; fusing the first high-frequency component and the second high-frequency component into a third high-frequency component, and fusing the first low-frequency component and the second low-frequency component into a third low-frequency component; fusing the third high-frequency component and the third low-frequency component to obtain a third image.

As an optional implementation manner of an embodiment of the present disclosure, the fusing the first high-frequency component and the second high-frequency component into a third high-frequency component includes: calculating a first modulus value corresponding to the first high-frequency component and a second modulus value corresponding to the second high-frequency component; comparing the first modulus value with the second modulus value, and determining the largest modulus value among the first modulus value and the second modulus value; if the largest modulus value is the first modulus value, the first high-frequency component is used as the third high-frequency component; the third high frequency component.

As an optional implementation manner of an embodiment of the present disclosure, the fusing the first low-frequency component and the second low-frequency component into a third low-frequency component includes: determining an average low-frequency component according to the first low-frequency component and the second low-frequency component, and using the average low-frequency component as the third low-frequency component.

As an optional implementation manner of the embodiment of the present disclosure, the decomposing the enhanced first image into a first high-frequency component and a first low-frequency component, and decomposing the enhanced second image into a second high-frequency component and a second low-frequency component include: performing non-subsampling contourlet transformation or fast non-subsampling contourlet transformation on the enhanced first image to obtain a first high-frequency component and a first low-frequency component; performing non-subsampling contourlet transformation or fast non-subsampling contourlet transformation on an enhanced second image to obtain a second high-frequency component and a second low-frequency component.

As an optional implementation manner of an embodiment of the present disclosure, the detecting the shooting scene where the electronic device is located includes: acquiring a preview image collected by a camera, and calculating a grayscale value of the preview image; if the grayscale value of the preview image is greater than a preset grayscale threshold, determining that the electronic device is in a backlight shooting scene; if the grayscale value of the preview image is less than or equal to the preset grayscale threshold, determining that the electronic device is not in a backlight shooting scene.

As an optional implementation manner of the embodiment of the present disclosure, performing enhancement processing on the first image and the second image respectively includes: performing first enhancement processing on the first image, and performing second enhancement processing on the second image; wherein, the first enhancement processing includes a multi-scale retinal enhancement algorithm, and the second enhancement processing includes a homomorphic filtering algorithm.

As an optional implementation of this embodiment of the present disclosure, the method further includes: dividing the preview image captured by the camera into multiple image areas according to a preset area size; calculating the gray value corresponding to each of the image areas; comparing the gray value corresponding to each of the image areas with a regional gray threshold to divide the preview image into a backlit area and a non-backlit area, wherein the backlit area refers to an area in the preview image whose gray value is greater than the area gray threshold, and the non-backlit area refers to the area in which the gray value of the preview image is less than or an area equal to the gray threshold of the area.

An image processing device, comprising:

A detection module, configured to detect the shooting scene where the electronic device is located;

A focus acquisition module, configuring the focus acquisition module as a module that collects a first image focused on a backlight area through the camera and acquires a second image focused on a non-backlight area through the camera if it is detected that the electronic device is in a backlight shooting scene;

An enhancement module, configuring the focus acquisition module as a module for respectively performing enhancement processing on the first image and the second image;

A fusion module, configured to fuse the enhanced first image with the enhanced second image to obtain a third image.

As an optional implementation manner of the embodiment of the present disclosure, the device further includes: an alignment module configured to perform registration processing on the enhanced first image and the enhanced second image, so as to make the spatial position information of the enhanced first image consistent with the spatial position information of the enhanced second image; and further configure the fusion module to fuse the registered first image and the registered second image to obtain a third image.

As an optional implementation manner of the embodiment of the present disclosure, the fusion module is further configured to decompose the enhanced first image into a first high-frequency component and a first low-frequency component, and decompose the enhanced second image into a second high-frequency component and a second low-frequency component; wherein, the first high-frequency component corresponds to a first high-frequency region in the enhanced first image, and the first low-frequency component corresponds to a first low-frequency region in the enhanced first image; the second high-frequency component corresponds to a second high-frequency region in the enhanced second image, and the second low-frequency component corresponds to the enhanced second image. The second low-frequency region in the image; the first high-frequency component and the second high-frequency component are fused into a third high-frequency component, and the first low-frequency component and the second low-frequency component are fused into a third low-frequency component; the third high-frequency component and the third low-frequency component are fused to obtain a module of a third image.

As an optional implementation manner of the embodiment of the present disclosure, the fusion module is further configured to calculate a first modulus value corresponding to the first high-frequency component and a second modulus value corresponding to the second high-frequency component; compare the first modulus value with the second modulus value, and determine the largest modulus value among the first modulus value and the second modulus value; if the largest modulus value is the first modulus value, use the first high-frequency component as the third high-frequency component;

As an optional implementation manner of the embodiment of the present disclosure, the fusion module is further configured to determine an average low frequency component according to the first low frequency component and the second low frequency component, and use the average low frequency component as a module of the third low frequency component.

As an optional implementation manner of the embodiment of the present disclosure, the fusion module is further configured to perform non-subsampling contourlet transformation or fast non-subsampling contourlet transformation on the enhanced first image to obtain the first high-frequency component and the first low-frequency component; perform the non-subsampling contourlet transformation or the fast non-subsampling contourlet transformation on the enhanced second image to obtain the second high-frequency component and the second low-frequency component.

As an optional implementation of the embodiment of the present disclosure, the detection module is further configured to acquire a preview image collected by the camera, and calculate a gray value of the preview image; if the gray value of the preview image is greater than a preset gray threshold, then determine that the electronic device is in a backlight shooting scene; if the gray value of the preview image is less than or equal to the preset gray threshold, then determine that the electronic device is not in a backlight shooting scene.

As an optional implementation manner of the embodiment of the present disclosure, the enhancement module is further configured to perform a first enhancement process on the first image, and perform a second enhancement process on the second image; wherein, the first enhancement process includes a multi-scale retinal enhancement algorithm, and the second enhancement process includes a module of a homomorphic filtering algorithm.

As an optional implementation of the embodiment of the present disclosure, the focus acquisition module is further configured to divide the preview image captured by the camera into multiple image areas according to the preset area size; calculate the gray value corresponding to each of the image areas; compare the gray value corresponding to each of the image areas with the area gray threshold, and divide the preview image into a backlight area and a non-backlight area; wherein, the backlight area refers to an area in the preview image whose gray value is greater than the area gray threshold, and the non-backlight area refers to the gray scale in the preview image Areas with values less than or equal to the area grayscale threshold.

An electronic device, comprising a memory and one or more processors, the memory is configured as a module storing computer-readable instructions; when the computer-readable instructions are executed by the processor, the one or more processors execute the steps of the image processing method described in any one of the above.

One or more non-volatile storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more processors are made to execute the steps of any one of the image processing methods described above.

Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the disclosure. The objectives and other advantages of the disclosure will be realized and attained by the structure particularly pointed out in the written description, claims hereof as well as the accompanying drawings, the details of one or more embodiments of the disclosure being set forth in the accompanying drawings and the description below.

In order to make the above objects, features and advantages of the present disclosure more comprehensible, optional embodiments are given below and described in detail in conjunction with the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative labor.

FIG. 1 is an application scene diagram of an image processing method provided by one or more embodiments of the present disclosure;

FIG. 2 is a flowchart of steps of an image processing method provided by one or more embodiments of the present disclosure;

FIG. 3 is a flowchart of steps of an image processing method provided by one or more embodiments of the present disclosure;

FIG. 4 is a flow chart of steps for fusing the enhanced first image with the enhanced second image provided by one or more embodiments of the present disclosure;

Fig. 5 is a structural block diagram of an image processing device in one or more embodiments of the present disclosure;

Fig. 6 is a structural block diagram of an electronic device in one or more embodiments of the present disclosure.

Detailed ways

In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, in the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

In the following description, a lot of specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways than described here; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, not all of them.

The terms "first" and "second" and the like in the specification and claims of the present disclosure are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first image and the second image are for distinguishing different images, not for describing a specific sequence of images.

In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure shall not be construed as being preferred or advantageous over other embodiments or designs. To be precise, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "plurality" refers to two or more.

In related technologies, there are two solutions to solve the problem of underexposure or overexposure of images captured in backlit shooting scenarios. One solution includes using a flash for exposure compensation and subject brightness compensation through a flash. Since the brightness of the flash of an electronic device is insufficient, brightness compensation can only be performed on a subject that is shot at a close distance. When the subject is farther away from the camera, the effect of this solution is worse. Another solution is to increase the dynamic range of the image through HDR (High-Dynamic Range, high-dynamic range image), reduce the bright or dark areas in the image captured by the camera, and increase the dynamic range of the image through HDR. Since this solution only selects over-exposed, over-dark and normal exposure images for fusion, the dynamic range of the obtained image is limited, and it cannot fully meet various complex backlight environments. In some complex backlight shooting scenes, the effect of this solution is not ideal.

When the electronic device is in the backlight shooting scene for image capture, the camera of the electronic device will be directly or indirectly reflected by the light source, and the images collected by the camera will be different when the focus position of the camera is different. When the focus is in an area disturbed by strong light, that is, when focusing on the backlight area, the image collected by the camera has a high-brightness area, and this high-brightness area covers up part of the information in the image; image effects.

The image processing method provided in the present disclosure can be applied to the application environment shown in FIG. 1 , and the image processing method is applied to an image processing system. The image processing system may include an electronic device 10, wherein the electronic device 10 may include but not limited to a mobile phone, a tablet computer, a wearable device, a notebook computer, a PC (Personal Computer, personal computer), a video camera, and the like. In addition, the operating system of the above-mentioned electronic device 10 may include but not limited to Android (Android) operating system, IOS operating system, Symbian (Symbian) operating system, Black Berry (Blackberry) operating system, Windows Phone8 operating system, etc., the embodiments of the present disclosure are not limited.

After the electronic device 10 detects the shooting operation, it may collect images through a camera, and the camera may be a camera of the electronic device 10 . Optionally, the electronic device 10 may also collect images through cameras of other electronic devices establishing a communication connection with the electronic device 10, which is not limited here. The electronic device 10 may analyze the image collected by the camera to determine that it is currently in a backlight shooting scene. As shown in FIG. 1 , the backlight shooting scene may be a scene where the direction of the camera faces a light source. In this backlight shooting scene, the electronic device 10 uses the camera to shoot, and the captured image may be overexposed in some areas and/or underexposed in some areas.

As an optional implementation manner, the electronic device 10 may detect the shooting scene where the electronic device 10 is currently located before detecting the shooting operation. Wherein, the method of detecting the current shooting scene may include but not limited to detection by a light sensor, detection of images collected by the camera after the camera is turned on and before a shooting operation is detected, etc., which is not limited here. The electronic device 10 may detect the current light intensity through the light sensor, and determine the current shooting scene according to the light intensity. For example, if the light intensity is greater than the intensity threshold, the electronic device 10 determines that the current shooting scene is a backlight shooting scene; if the light intensity is less than or equal to the intensity threshold, the electronic device 10 determines that it is not currently in a backlight shooting scene.

When the electronic device 10 detects that it is currently in a backlight shooting scene, it can automatically switch to a processing mode for processing images collected in the backlight shooting scene, or output a prompt message to prompt the user to switch to the processing mode corresponding to the switching operation. After detecting the switching operation, the electronic device 10 switches to a processing mode for processing images collected in the backlight shooting scene. The specific content of the image processing method disclosed in the embodiments of the present disclosure will be described in the following embodiments, and will not be explained too much here.

Referring to FIG. 2, FIG. 2 is a flow chart of the steps of an image processing method provided by one or more embodiments of the present disclosure. The image processing method can be applied to the above-mentioned electronic device, and may include the following steps:

Step 210, detecting the shooting scene where the electronic device is located.

In one embodiment, after the electronic device collects the preview image through the camera, it can detect the shooting scene where the electronic device is located according to the collected preview image, that is, detect whether the electronic device is in a backlit shooting scene. Wherein, the detection method may include detecting the gray value of the image pixel, detecting the RGB value of the image pixel, etc., which are not limited here.

In one embodiment, the electronic device may acquire a preview image collected by the camera, and calculate the grayscale value of the preview image. If the grayscale value is greater than a preset grayscale threshold, it is determined that the electronic device is in a backlight shooting scene. The electronic device may continue to perform steps 220 to 240. If the grayscale value is less than or equal to the preset grayscale threshold, it is determined that the electronic device is not in a backlight shooting scene. The electronic device may perform image enhancement processing and/or filtering processing on the collected images. Wherein, the image enhancement processing may include contrast enhancement, Gamma (γ, gamma) correction, histogram equalization, histogram regulation, color image enhancement method based on HSV space, etc., and the filtering processing includes mean filtering, median filtering, Gaussian filtering, bilateral filtering, etc., which are not limited in the embodiments of the present disclosure.

As an optional implementation manner, the electronic device may calculate an average gray value of all pixels in the image collected by the camera, and compare the average gray value of all pixels in the image with a preset gray threshold. If the average grayscale value is greater than the preset grayscale threshold, it is determined that the electronic device is in a backlight shooting scene; if the average grayscale value is less than or equal to the preset grayscale threshold, it is determined that the electronic device is not in a backlight shooting scene.

As an optional implementation manner, the electronic device may calculate the grayscale value of each pixel in the image collected by the camera, and compare the grayscale value of each pixel in the image with a preset grayscale threshold. Calculate the number of pixels whose grayscale value is greater than the preset grayscale threshold. If the number of pixels is greater than the preset number threshold, it is determined that the electronic device is in a backlight shooting scene. If the number of pixels is less than or equal to the preset number threshold, it is determined that the electronic device is not in a backlight shooting scene.

Step 220, if it is detected that the electronic device is in a backlit shooting scene, capture a first image focused on the backlit area through the camera, and capture a second image focused on the non-backlit area through the camera.

If the electronic device detects that the electronic device is in a backlit shooting scene, the first image focused on the backlit area is collected through the camera, and the second image focused on the non-backlit area is collected through the camera. There is no sequence relationship between the first image and the second image, and they can also be executed simultaneously. Wherein, the backlight area refers to an area that is affected by light and has a backlight effect, and the non-backlight area refers to an area that does not have a backlight effect due to the light.

In one embodiment, the electronic device can divide the preview image into multiple image areas according to the preset area size, for example, each image area includes 64×64 pixels, and calculate the gray value corresponding to each image area in the preview image, respectively compare the gray value corresponding to each image area with the area gray threshold, so as to divide the preview image into a backlight area and a non-backlight area. The area refers to the area in the preview image whose gray value is less than or equal to the gray threshold of the area, that is, the area where the backlight effect does not appear due to the influence of light. The electronic device then collects the first image focused on the backlit area and the second image focused on the non-backlit area through the camera.

As an optional implementation manner, when the electronic device detects that the electronic device is in a backlight shooting scene, the camera can automatically collect the first image focused on the backlight area, and the camera can collect the second image focused on the non-backlight area, without requiring the user to find the focus area, simplifying the user's operation, and improving the convenience of the image processing method.

As an optional implementation, when the electronic device detects that the electronic device is in a backlit shooting scene, it can send a prompt message to prompt the user to manually select the focus on the backlit area and the non-backlit area. When the focus selection operation for the backlit area is detected, the camera focuses on the backlit area and collects the first image according to the focus selection operation for the backlit area. The area that needs to be focused, and then collect the first image and the second image, so as to improve the accuracy of the focus of the collected image.

Step 230, performing enhancement processing on the first image and the second image respectively.

Since the image quality of the first image and the second image collected by the camera focusing on different areas is not enough, the electronic device may respectively perform enhancement processing on the first image and the second image to improve the image quality of the first image and the second image.

In one embodiment, the first enhancement process can be performed on the first image, and the second enhancement process can be performed on the second image; wherein, the first enhancement process includes a multi-scale retinal enhancement algorithm (MSR, Multi-Scale Retinex), and the second enhancement process includes a homomorphic filtering algorithm.

The first image is collected when the camera focuses on the backlit area, and focusing on the backlit area will affect the clarity and contrast of the first image. Therefore, the electronic device can use a multi-scale retinal enhancement algorithm to enhance the contrast and clarity of the first image. The multi-scale retinal enhancement algorithm determines multiple scales of the first image, performs Gaussian blur on the first image at multiple scales, and obtains multiple blurred images after blurring. The enhanced first image is obtained based on the first image and the multiple blurred images, which can improve the image quality of the first image.

The second image is collected when the camera focuses on the non-backlit area, and focusing on the non-backlit area will cause more image noise in the second image. Therefore, the electronic device can use a homomorphic filtering algorithm to remove the noise in the second image. The homomorphic filtering algorithm transforms the illumination-reflection model corresponding to the second image, and then passes the transformed result through a frequency domain filter to obtain a filtered result, and inversely transforms the filtered result to obtain an enhanced second image, which can improve the image quality of the second image.

Step 240, fusing the enhanced first image and the enhanced second image to obtain a third image.

The electronic device may fuse the enhanced first image and the enhanced second image to obtain a third image. The third image may be an image finally presented on a display screen for viewing by a user, or may be an image stored in a memory.

As an optional implementation, the electronic device may divide the first image and the second image into image areas according to the number of pixels. For example, each image area includes 64×64 pixels. There is a one-to-one correspondence between each image area in the first image and each image area in the second image. For every two image areas corresponding to the first image and the second image, select the image area with higher image quality in the two image areas. , as the image area corresponding to the position in the third image, the same operation is performed on other image areas, and the third image is obtained after the fusion of all areas is completed.

In the embodiment of the present disclosure, the electronic device can detect the current environment. When it detects that the electronic device is in a backlight shooting scene, the camera captures the first image focused on the backlight area and the second image focused on the non-backlight area. However, underexposure or overexposure in the backlight shooting scene will cause the first image and the second image to lose part of the effective information. The sharpness and contrast of the second image improve the image quality of the image captured in the backlight shooting scene. .

Referring to FIG. 3, FIG. 3 is a flow chart of the steps of the image processing method provided by one or more embodiments of the present disclosure. The image processing method can also be applied to the above-mentioned electronic device, and may include the following steps:

Step 310, detecting the shooting scene where the electronic device is located.

Step 320, if it is detected that the electronic device is in a backlit shooting scene, capture a first image focused on the backlit area through the camera, and capture a second image focused on the non-backlit area through the camera.

Step 330, performing enhancement processing on the first image and the second image respectively.

Steps 310-330 are the same as steps 210-230, and will not be repeated here.

Step 340, performing registration processing on the enhanced first image and the enhanced second image, so that the spatial position information of the first image is consistent with the spatial position information of the second image.

Since the electronic device may move when collecting the first image and the second image, there is a difference between the spatial position information of the first image and the spatial position information of the second image. Therefore, the electronic device may perform registration processing on the enhanced first image and the enhanced second image, for example, register key points in the enhanced first image and the enhanced second image, register image features in the enhanced first image and the enhanced second image, etc., so that the spatial position information of the enhanced first image is consistent with the spatial position information of the enhanced second image, so that the subsequent image fusion step can be performed.

As an optional implementation manner, the electronic device may perform affine transformation registration on the enhanced first image and the enhanced second image, first use feature matching between the enhanced first image and the enhanced second image as data to obtain a predicted affine transformation matrix, and register the enhanced first image with the enhanced second image according to the predicted affine transformation matrix, so that the spatial position information of the enhanced first image and the spatial position information of the enhanced second image are consistent. For example, the electronic device may perform feature matching between the enhanced first image and the enhanced second image, obtain a predicted radial transformation matrix for transforming the enhanced first image into the enhanced second image, and transform the enhanced first image according to the predicted affine transformation matrix, thereby obtaining a first image that is registered with the enhanced second image.

Step 350, merging the registered first image and the registered second image to obtain a third image.

The electronic device may fuse the registered first image and the registered second image to obtain a third image. The first image after the registration processing and the second image after the registration processing are obtained after registration processing is performed on the basis of the first image after the enhancement processing and the second image after the enhancement processing, and the method of fusing the first image after the registration processing and the second image after the registration processing in step 350 can be the same as the method of fusing the first image after the enhancement processing and the second image after the enhancement processing in step 240, and will not be repeated here.

As an optional implementation manner, since there is a one-to-one correspondence between each pixel point in the first image after registration processing and each pixel point in the second image after registration processing, the electronic device can weight and calculate the pixel values (such as grayscale values or RGB (Red, Green, Blue, red, green, blue) values, etc.) After adding, the obtained target pixel value can be used as the pixel value of the corresponding pixel point in the third image, and the above-mentioned fusion operation can be performed on all corresponding pixel points in the registered first image and the registered second image to obtain the third image.

In the embodiment of the present disclosure, the electronic device may also perform registration processing on the first image and the second image, so as to facilitate the fusion of the first image and the second image, and improve the accuracy of image fusion.

Referring to FIG. 4 , FIG. 4 is a flow chart of steps for fusing the enhanced first image with the enhanced second image provided by one or more embodiments of the present disclosure. The step of fusing the enhanced first image with the enhanced second image may include the following steps:

Step 410, decomposing the enhanced first image into a first high-frequency component and a first low-frequency component, and decomposing the enhanced second image into a second high-frequency component and a second low-frequency component; wherein, the first high-frequency component corresponds to the first high-frequency region in the enhanced first image, and the first low-frequency component corresponds to the first low-frequency region in the enhanced first image; the second high-frequency component corresponds to the second high-frequency region in the enhanced second image, and the second low-frequency component corresponds to the second low-frequency region in the enhanced second image.

The electronic device may decompose the enhanced first image into a first high frequency component and a first low frequency component, and decompose the enhanced second image into a second high frequency component and a second low frequency component.

Wherein, the electronic device may calculate the gray value of each pixel in the enhanced first image, so as to obtain the gray value change speed between each pixel in the enhanced first image and surrounding pixels, and decompose the enhanced first image into a first high frequency component and a first low frequency component according to the gray value change speed, the first high frequency component includes a plurality of images having the same image size as the enhanced first image, and the first low frequency component includes an image having the same image size as the enhanced first image. The first high-frequency component corresponds to the first high-frequency region in the enhanced first image, that is, the multiple images in the first high-frequency component correspond to different first high-frequency regions in the enhanced first image, and the first high-frequency region is an image region in the enhanced first image whose gray value change speed is greater than the first change speed threshold. For example, the first high frequency component may correspond to the edge region of the enhanced first image, because the gray value change speed of the edge region of the image is usually greater than the gray value change speed of the middle region. The first low-frequency component corresponds to the first low-frequency region in the enhanced first image, that is, the image in the first low-frequency component corresponds to the first low-frequency region in the enhanced first image, and the first low-frequency region is an image region in the enhanced first image whose grayscale value change speed is less than or equal to the first change speed threshold. For example, the first low-frequency component may correspond to the middle region of the enhanced first image, because the grayscale value change speed of the middle region of the image is usually smaller than the grayscale value change speed of the edge region.

The electronic device may calculate the gray value of each pixel in the enhanced second image, thereby obtaining the gray value change speed between each pixel in the enhanced second image and surrounding pixels, and decompose the enhanced second image into a second high frequency component and a second low frequency component according to the gray value change speed. The second high-frequency component corresponds to the second high-frequency region in the enhanced second image, that is, the multiple images in the second high-frequency component correspond to different second high-frequency regions in the enhanced second image, and the second high-frequency region is an image region in the enhanced second image whose gray value change speed is greater than the second change speed threshold. For example, the second high frequency component may correspond to the edge region of the enhanced second image, because the gray value change speed of the edge region of the image is usually greater than the gray value change speed of the middle region. The second low-frequency component corresponds to the second low-frequency region in the enhanced second image, that is, the image in the second low-frequency component corresponds to the second low-frequency region in the enhanced second image, and the second low-frequency region is an image region whose grayscale value change speed in the enhanced second image is less than or equal to the second change speed threshold. For example, the second low-frequency component may correspond to the middle region of the enhanced second image, because the grayscale value change speed of the middle region of the image is usually smaller than the grayscale value change speed of the edge region. Wherein, the first change speed threshold and the second change speed threshold may be equal in magnitude or unequal in magnitude.

As an optional implementation manner, the electronic device may perform NSCT (Nonsubsampled contourlet transform, non-subsampled contourlet transform) forward transform or FNSCT (FastNonsubsampled contourlet transform, fast non-subsampled contourlet transform) forward transform on the enhanced first image, thereby obtaining the first high-frequency component and the first low-frequency component, and the first high-frequency transformation coefficient corresponding to the first high-frequency component and the first low-frequency transformation coefficient corresponding to the first low-frequency component. The electronic device converts the enhanced first image into a first spectrogram, the first spectrogram can illustrate the gray value change speed between each pixel in the enhanced first image and the surrounding pixels, and perform multiple NSP (Nonsubsampled Pyramid, non-subsampled tower filter) decomposition on the enhanced first image according to the first spectrogram, thereby obtaining one or more first bandpass subband images and a first lowpass subband image. The image size of the first bandpass subband image is the same as that of the enhanced first image, and each first bandpass subband image corresponds to a different first high frequency region in the enhanced first image. The image size of the first low-pass sub-band image is the same as that of the enhanced first image, the first low-pass sub-band image corresponds to the first low-frequency region in the enhanced first image, and the first low-pass sub-band image can be used as the first low-frequency component. The electronic device then performs NSDFB (Nonsubsampled directional filter bank, non-subsampled directional filter bank) decomposition on each first bandpass subband image, and can decompose each first bandpass subband image into a plurality of first multi-directional subband images. The image size of the first multi-directional subband image is the same as the image size of the enhanced first image, and the plurality of first direction subband images can be jointly used as the first high frequency component.

In addition, when performing NSP decomposition on the enhanced first image to obtain the first low-pass sub-band image, the first low-frequency transform coefficient corresponding to the first low-pass sub-band image is obtained. When performing NSDFB decomposition on the first bandpass sub-band image to obtain the first multi-directional sub-band image, and obtain the first high-frequency transformation coefficient corresponding to the first multi-directional sub-band image, since multiple first multi-directional sub-band images can be obtained, that is, the first high-frequency transformation coefficient can include multiple values, and the multiple values correspond to multiple first multi-directional sub-band images.

The electronic device may also perform NSCT (Nonsubsampled contourlet transform, non-subsampled contourlet transform) forward transform or FNSCT (FastNonsubsampled contourlet transform, fast non-subsampled contourlet transform) forward transform on the enhanced second image, thereby obtaining a second high-frequency component and a second low-frequency component, and a second high-frequency transformation coefficient corresponding to the second high-frequency component and a second low-frequency transformation coefficient corresponding to the second low-frequency component. The electronic device converts the enhanced second image into a second spectrogram, and the second spectrogram can illustrate the gray value change speed between each pixel in the enhanced second image and surrounding pixels, and performs multiple NSP (Nonsubsampled Pyramid, non-subsampled tower filter) decomposition on the enhanced second image according to the second spectrogram, thereby obtaining one or more second bandpass subband images and a second lowpass subband image. The image size of the second bandpass subband image is the same as that of the enhanced second image, and each second bandpass subband image corresponds to a different second high frequency region in the enhanced second image. The image size of the second low-pass sub-band image is the same as that of the enhanced second image, the second low-pass sub-band image corresponds to the second low-frequency area in the enhanced second image, and the second low-pass sub-band image can be used as the second low-frequency component. The electronic device then performs NSDFB (Nonsubsampled directional filter bank) decomposition on each second bandpass subband image, and can decompose each second bandpass subband image into a plurality of second multi-directional subband images. The image size of the second multidirectional subband image is the same as the image size of the enhanced second image, and the plurality of second direction subband images can be jointly used as the second high frequency component.

In addition, when performing NSP decomposition on the enhanced second image to obtain the second low-pass sub-band image, the second low-frequency transformation coefficient corresponding to the second low-pass sub-band image is obtained. When NSDFB decomposition is performed on the second bandpass sub-band image to obtain the second multi-directional sub-band image, and the second high-frequency transformation coefficient corresponding to the second multi-directional sub-band image is obtained, since a plurality of second multi-directional sub-band images can be obtained, that is, the second high-frequency transformation coefficient can include multiple values, and the multiple values correspond to multiple second multi-directional sub-band images.

Step 420, fusing the first high-frequency component and the second high-frequency component into a third high-frequency component, and fusing the first low-frequency component and the second low-frequency component into a third low-frequency component.

In one embodiment, the step of fusing the first high-frequency component and the second high-frequency component into a third high-frequency component may include: calculating a first modulus value corresponding to the first high-frequency component and a second modulus value corresponding to the second high-frequency component; comparing the first modulus value with the second modulus value, and determining the largest modulus value among the first modulus value and the second modulus value; if the largest modulus value is the first modulus value, the first high-frequency component is used as the third high-frequency component;

Wherein, because a plurality of images in the first high-frequency component can correspond to different first high-frequency regions in the first image after the enhancement process, so the first high-frequency component can correspond to a plurality of first modulus values, and the plurality of first modulus values can correspond to a plurality of images in the first high-frequency component respectively. Since the first high-frequency region in the first image corresponds to the second high-frequency region in the second image, the plurality of first modulus values corresponding to the first high-frequency component corresponds to the plurality of second modulus values corresponding to the second high-frequency component. The electronic device compares every two corresponding first modulus values with the second modulus value, and determines the largest modulus value among the first modulus value and the second modulus value corresponding to each two. If the largest modulus value is the first modulus value, the image of the first high-frequency component corresponding to the first modulus value is fused to the image of the third high-frequency component;

As an optional embodiment, the electronic equipment can obtain the first high -frequency transformation coefficient and the second high -frequency transformation coefficient on the enhanced first image and the enhanced secondary image. The direction of the direction is the image, so the fusion of the first high -frequency transformation coefficient and the second high -frequency variable change coefficient can be equivalent to integrating the first high frequency component and the second high frequency component. The electronic device respectively takes absolute values of multiple values in the first high-frequency transformation coefficient to obtain multiple first modular values, respectively takes absolute values of multiple values in the second high-frequency transformation coefficient to obtain multiple second modular values, there is a one-to-one correspondence between the first and second modular values, compares every two corresponding first and second modular values, and determines the largest modulus value among the first and second corresponding first and second modulus values, and if the largest modulus value is the first modulus value, then use the value of the first high-frequency transformation coefficient corresponding to the first modulus value as the value of the third high-frequency component transformation coefficient , if the largest modulus value is the second modulus value, the value of the second high-frequency transform coefficient corresponding to the second modulus value is used as the value of the third high-frequency component transform coefficient, and all the first modulus values are compared with the second modulus value to obtain the third high-frequency component transform coefficient, thereby determining the third high-frequency component corresponding to the third high-frequency transform coefficient.

In one embodiment, the step of fusing the first low-frequency component and the second low-frequency component into a third low-frequency component may include: determining an average low-frequency component according to the first low-frequency component and the second low-frequency component, and using the average low-frequency component as the third low-frequency component.

Where there is a corresponding relationship between the image area of the first low-frequency component and the image area of the second low-frequency component, the electronic device may calculate the average low-frequency component of the first low-frequency component and the second low-frequency component, that is, calculate an average value of data corresponding to the first low-frequency component and data corresponding to the second low-frequency component, thereby obtaining the average low-frequency component, and using the average low-frequency component as the third low-frequency component.

As an optional implementation manner, the electronic device performs NSCT forward transform or FNSCT forward transform on the enhanced first image and the enhanced second image to obtain the first low-frequency transform coefficient and the second low-frequency transform coefficient, because the first low-frequency transform coefficient can correspond to the first low-pass subband image in the first low-frequency component, and the second low-frequency transform coefficient can correspond to the second low-pass sub-band image in the second low-frequency component, so fusing the first low-frequency transform coefficient and the second low-frequency transform coefficient can be equivalent to fusing the first low-frequency component and the second low-frequency component to obtain The third low-frequency transform coefficient of corresponds to the third low-frequency component. The electronic device may fuse the first low-frequency transformation coefficient representing the first low-frequency component and the second low-frequency transformation coefficient representing the second low-frequency component obtained after NSCT forward transformation or FNSCT forward transformation of the first image and the second image, to obtain a third low-frequency transformation coefficient, and use the third low-frequency transformation coefficient to represent the third low-frequency component. The electronic device calculates an average value of the first low-frequency transformation coefficient and the second low-frequency transformation coefficient, and uses the average value as a third low-frequency transformation coefficient, thereby determining a third low-frequency component corresponding to the third low-frequency transformation coefficient.

Step 430, fusing the third high-frequency component and the third low-frequency component into a third image.

The electronic device may fuse a third high-frequency component obtained by fusing the first high-frequency component and the second high-frequency component with a third low-frequency component obtained by fusing the first low-frequency component and the second low-frequency component to obtain a third image. As an optional implementation manner, NSCT inverse transformation may be performed on the third high-frequency component and the third low-frequency component respectively, and the obtained result is a third image.

In an embodiment of the present disclosure, the electronic device may decompose the enhanced first image into the first high-frequency component and the first low-frequency component, and decompose the enhanced second image into the second high-frequency component and the second low-frequency component, then fuse the first high-frequency component and the second high-frequency component to obtain the third high-frequency component, and fuse the first low-frequency component and the second low-frequency component to obtain the third low-frequency component, and finally fuse the third high-frequency component and the third low-frequency component to obtain the third image, and can complement the effective information in the enhanced first image and the enhanced second image to solve the problem Underexposure or overexposure in backlit scenes.

It should be understood that although the various steps in the flow charts in FIGS. 2-4 are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in FIGS. 2-4 may include a plurality of sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily performed sequentially, but may be performed in turn or alternately with at least a part of other steps or sub-steps or stages of other steps.

Based on the same inventive concept, as an implementation of the above-mentioned method, the embodiment of the present disclosure also provides an image processing device. The embodiment of the device corresponds to the embodiment of the method described above. For the convenience of reading, this embodiment of the device does not repeat the details in the embodiment of the method described above one by one, but it should be clear that the device in the embodiment of the present disclosure can correspondingly implement all the content of the method embodiment described above.

FIG. 5 is a structural block diagram of an image processing device provided in an embodiment of the present disclosure. As shown in FIG. 5 , the image processing device 500 provided in an embodiment of the present disclosure includes:

The detection module 510 is configured to detect the shooting scene where the electronic device is located.

The focus collection module 520 is configured to configure the focus collection module 520 as a module for collecting the first image focused on the backlight area through the camera and the second image focused on the non-backlight area through the camera if it is detected that the electronic device is in a backlight shooting scene.

The enhancement module 530, the enhancement module 530 is configured as a module for performing enhancement processing on the first image and the second image respectively.

The fusion module 540 is a module that configures the fusion module 540 to fuse the enhanced first image and the enhanced second image to obtain a third image.

As an optional implementation manner of this embodiment of the present disclosure, the image processing apparatus 500 further includes:

The alignment module is configured to perform registration processing on the enhanced first image and the enhanced second image, so that the spatial position information of the enhanced first image is consistent with the spatial position information of the enhanced second image.

The fusion module 540 is also configured to fuse the registered first image and the registered second image to obtain a third image.

As an optional implementation of the embodiment of the present disclosure, the fusion module 540 is further configured to decompose the enhanced first image into a first high-frequency component and a first low-frequency component, and decompose the enhanced second image into a second high-frequency component and a second low-frequency component; wherein, the first high-frequency component corresponds to the first high-frequency region in the enhanced first image, and the first low-frequency component corresponds to the first low-frequency region in the enhanced first image; the second high-frequency component corresponds to the second high-frequency region in the enhanced second image, and the second low-frequency component corresponds to the second low-frequency component in the enhanced second image. frequency region; the first high-frequency component and the second high-frequency component are fused into a third high-frequency component, and the first low-frequency component and the second low-frequency component are fused into a third low-frequency component; the third high-frequency component and the third low-frequency component are fused to obtain a module of a third image.

As an optional implementation manner of the embodiment of the present disclosure, the fusion module 540 is further configured to calculate a first modulus value corresponding to the first high-frequency component and a second modulus value corresponding to the second high-frequency component; compare the first modulus value with the second modulus value, and determine the largest modulus value among the first modulus value and the second modulus value; if the largest modulus value is the first modulus value, then use the first high-frequency component as the third high-frequency component;

As an optional implementation manner of the embodiment of the present disclosure, the fusion module 540 is further configured to determine an average low frequency component according to the first low frequency component and the second low frequency component, and use the average low frequency component as a module of the third low frequency component.

As an optional implementation of the embodiment of the present disclosure, the detection module 510 is also configured to obtain a preview image collected by the camera, and calculate the gray value of the preview image; if the gray value of the preview image is greater than the preset gray threshold, it is determined that the electronic device is in a backlight shooting scene; if the gray value of the preview image is less than or equal to the preset gray threshold, it is determined that the electronic device is not in a backlight shooting scene.

As an optional implementation of the embodiment of the present disclosure, the enhancement module 530 is further configured to perform a first enhancement process on the first image, and perform a second enhancement process on the second image; wherein, the first enhancement process includes a multi-scale retinal enhancement algorithm, and the second enhancement process includes a module of a homomorphic filtering algorithm.

As an optional implementation of the embodiment of the present disclosure, the focus acquisition module 520 is also configured to divide the preview image captured by the camera into multiple image areas according to the preset area size; calculate the gray value corresponding to each image area; compare the gray value corresponding to each image area with the area gray threshold value, and divide the preview image into a backlight area and a non-backlight area; wherein, the backlight area refers to the area in the preview image whose gray value is greater than the area gray threshold value, and the non-backlight area refers to the area in which the gray value in the preview image is less than or equal to the area gray value The region of the degree threshold.

The image processing apparatus provided in the embodiments of the present disclosure can execute the image processing methods provided in the foregoing method embodiments, and its implementation principle and technical effect are similar, and will not be repeated here. Each module in the above-mentioned image processing device may be fully or partially realized by software, hardware or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, and can also be stored in the memory of the electronic device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

In one embodiment, an electronic device is provided. The electronic device may be a terminal device, and its internal structure may be as shown in FIG. 6 . The electronic device includes a processor, a memory, a communication interface, a database, a display screen and an input device connected through a system bus. Wherein, the processor of the electronic device is configured as a module providing calculation and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer readable instructions. The internal memory provides an environment for the execution of the operating system and computer readable instructions in the non-volatile storage medium. The communication interface of the electronic device is configured as a wired or wireless communication module with an external terminal, and the wireless mode can be realized through WIFI, operator network, near field communication (NFC) or other technologies. When the computer-readable instructions are executed by the processor, the image processing method provided by the above-mentioned embodiments can be implemented. The display screen of the electronic device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the housing of the electronic device, or an external keyboard, touch pad or mouse.

Those skilled in the art can understand that the structure shown in FIG. 6 is only a block diagram of a partial structure related to the disclosed solution, and does not constitute a limitation to the electronic device to which the disclosed solution is applied. The specific electronic device may include more or less components than those shown in the figure, or combine certain components, or have a different component arrangement.

In one embodiment, the image processing apparatus provided in the present disclosure may be implemented in the form of computer-readable instructions, and the computer-readable instructions may be run on a computer device as shown in FIG. 6 . Various program modules constituting the electronic device can be stored in the memory of the computer device, for example, the detection module 510 and the focus acquisition module 520 shown in FIG. 5 . The computer-readable instructions constituted by the various program modules cause the processor to execute the steps in the image processing methods of the various embodiments of the present disclosure described in this specification.

In one embodiment, an electronic device is provided, including a memory and one or more processors, the memory is configured as a module storing computer-readable instructions; when the computer-readable instructions are executed by the processor, the one or more processors execute the steps of the image processing method described in the above method embodiments.

The electronic device provided by the embodiment of the present disclosure can implement the image processing method provided by the above method embodiment, and its implementation principle and technical effect are similar, and will not be repeated here.

One or more non-volatile storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more processors are made to perform the image processing steps described in any one of the above.

The computer-readable instructions stored on the computer-readable storage medium provided by the embodiments of the present disclosure can implement the image processing method provided by the above-mentioned method embodiments, and its implementation principle and technical effect are similar, and will not be repeated here.

Those of ordinary skill in the art can understand that the implementation of all or part of the processes in the above method embodiments can be completed by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Wherein, any reference to storage, database or other media used in the various embodiments provided by the present disclosure may include at least one of non-volatile and volatile storage. Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as Static Random Access Memory (SRAM) and Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered as within the scope of this specification.

The above examples only express several implementations of the present disclosure, and the descriptions thereof are more specific and detailed, but should not be construed as limiting the scope of the patent for the invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the scope of protection of the disclosed patent should be based on the appended claims.

Industrial Applicability

The image processing method provided by the present disclosure can effectively solve the problem of underexposure or overexposure when an electronic device is shooting in a strong light interference or backlight environment, improve the image quality of an image shot in a backlight shooting scene, and has strong industrial applicability.

Claims

An image processing method applied to electronic equipment, the method comprising:

Detecting the shooting scene where the electronic device is located;

If it is detected that the electronic device is in a backlight shooting scene, the first image focused on the backlight area is collected by the camera, and the second image focused on the non-backlight area is collected by the camera;

performing enhancement processing on the first image and the second image respectively;

The enhanced first image and the enhanced second image are fused to obtain a third image.
The method according to claim 1, wherein, before said fusing the enhanced first image and the enhanced second image to obtain the third image, said method further comprises:

performing registration processing on the enhanced first image and the enhanced second image, so that the spatial position information of the enhanced first image and the spatial position information of the enhanced second image are consistent;

Said merging the enhanced first image and the enhanced second image to obtain a third image includes:

The first image after registration processing and the second image after registration processing are fused to obtain a third image.
The method according to claim 1 or 2, wherein said fusing the enhanced first image and the enhanced second image to obtain a third image comprises:

decomposing the enhanced first image into a first high-frequency component and a first low-frequency component, and decomposing the enhanced second image into a second high-frequency component and a second low-frequency component; wherein, the first high-frequency component corresponds to a first high-frequency region in the enhanced first image, and the first low-frequency component corresponds to a first low-frequency region in the enhanced first image; the second high-frequency component corresponds to a second high-frequency region in the enhanced second image, and the second low-frequency component corresponds to a second low-frequency region in the enhanced second image;

fusing the first high frequency component and the second high frequency component into a third high frequency component, and fusing the first low frequency component and the second low frequency component into a third low frequency component;

Fusing the third high-frequency component and the third low-frequency component to obtain a third image.
The method according to claim 3, wherein said fusing said first high frequency component and said second high frequency component into a third high frequency component comprises:

calculating a first modulus value corresponding to the first high-frequency component and a second modulus value corresponding to the second high-frequency component;

Comparing the first modulus value with the second modulus value, and determining the largest modulus value among the first modulus value and the second modulus value, if the largest modulus value is the first modulus value, the first high frequency component is used as the third high frequency component, and if the largest modulus value is the second modulus value, the second high frequency component is used as the third high frequency component.
The method according to claim 3, wherein said fusing said first low frequency component and said second low frequency component into a third low frequency component comprises:

An average low frequency component is determined according to the first low frequency component and the second low frequency component, and the average low frequency component is used as a third low frequency component.
The method according to claim 3, wherein said decomposing the enhanced first image into a first high-frequency component and a first low-frequency component, and decomposing the enhanced second image into a second high-frequency component and a second low-frequency component comprises:

performing non-subsampling contourlet transformation or fast non-subsampling contourlet transformation on the enhanced first image to obtain a first high-frequency component and a first low-frequency component;

The non-subsampling contourlet transformation or the fast non-subsampling contourlet transformation is performed on the enhanced second image to obtain a second high frequency component and a second low frequency component.
The method according to claim 1, wherein the detecting the shooting scene where the electronic device is located comprises:

Obtain a preview image collected by the camera, and calculate the gray value of the preview image;

If the grayscale value of the preview image is greater than a preset grayscale threshold, it is determined that the electronic device is in a backlight shooting scene;

If the grayscale value of the preview image is less than or equal to the preset grayscale threshold, it is determined that the electronic device is not in a backlight shooting scene.
The method according to claim 1, wherein said respectively performing enhancement processing on said first image and said second image comprises:

Performing a first enhancement process on the first image, and performing a second enhancement process on the second image; wherein, the first enhancement process includes a multi-scale retinal enhancement algorithm, and the second enhancement process includes a homomorphic filtering algorithm.
The method according to claim 1, wherein the method further comprises:

dividing the preview image collected by the camera into multiple image areas according to a preset area size;

Calculating the gray value corresponding to each of the image regions;

Comparing the grayscale value corresponding to each of the image regions with the grayscale threshold of the region to divide the preview image into a backlit region and a non-backlit region, wherein the backlit region refers to an area in the preview image whose grayscale value is greater than the regional grayscale threshold, and the non-backlit region refers to an area in the preview image whose grayscale value is less than or equal to the regional grayscale threshold.
An image processing device, comprising:

A detection module, configured to detect the shooting scene where the electronic device is located;

A focus acquisition module, configuring the focus acquisition module as a module that collects a first image focused on a backlight area through the camera and acquires a second image focused on a non-backlight area through the camera if it is detected that the electronic device is in a backlight shooting scene;

An enhancement module, configuring the enhancement module as a module for performing enhancement processing on the first image and the second image respectively;

A fusion module, configured to fuse the enhanced first image with the enhanced second image to obtain a third image.
The device according to claim 10, wherein the device further comprises:

An alignment module, configured to perform registration processing on the enhanced first image and the enhanced second image, so that the spatial position information of the enhanced first image and the spatial position information of the enhanced second image are consistent;

The fusion module is further configured to fuse the registered first image and the registered second image to obtain a third image.
Apparatus according to claim 10 or 11, wherein:

The fusion module is further configured to decompose the enhanced first image into a first high-frequency component and a first low-frequency component, and decompose the enhanced second image into a second high-frequency component and a second low-frequency component; wherein, the first high-frequency component corresponds to a first high-frequency region in the enhanced first image, and the first low-frequency component corresponds to a first low-frequency region in the enhanced first image; the second high-frequency component corresponds to a second high-frequency region in the enhanced second image, and the second low-frequency component corresponds to a second low-frequency region in the enhanced second image; The first high-frequency component and the second high-frequency component are fused into a third high-frequency component, the first low-frequency component and the second low-frequency component are fused into a third low-frequency component; the third high-frequency component is fused with the third low-frequency component to obtain a module of a third image.
The apparatus of claim 12, wherein:

The fusion module is also configured to calculate a first modulus value corresponding to the first high-frequency component and a second modulus value corresponding to the second high-frequency component; compare the first modulus value with the second modulus value, and determine the largest modulus value among the first modulus value and the second modulus value; if the largest modulus value is the first modulus value, then use the first high-frequency component as the third high-frequency component; if the largest modulus value is the second modulus value, use the second high-frequency component as the module of the third high-frequency component.
The apparatus of claim 12, wherein:

The fusion module is further configured to determine an average low frequency component according to the first low frequency component and the second low frequency component, and use the average low frequency component as a module of a third low frequency component.
The apparatus of claim 12, wherein:

The fusion module is also configured to perform non-subsampling contourlet transformation or fast non-subsampling contourlet transformation on the enhanced first image to obtain the first high-frequency component and the first low-frequency component; perform the non-subsampling contourlet transformation or the fast non-subsampling contourlet transformation on the enhanced second image to obtain the second high-frequency component and the second low-frequency component.
The apparatus of claim 10, wherein:

The detection module is also configured to obtain a preview image collected by the camera, and calculate the grayscale value of the preview image; if the grayscale value of the preview image is greater than a preset grayscale threshold, then determine that the electronic device is in a backlight shooting scene; if the grayscale value of the preview image is less than or equal to the preset grayscale threshold, then determine that the electronic device is not in a backlight shooting scene.
The apparatus of claim 10, wherein:

The enhancement module is further configured to perform a first enhancement process on the first image, and a second enhancement process on the second image; wherein the first enhancement process includes a multi-scale retinal enhancement algorithm, and the second enhancement process includes a module of a homomorphic filtering algorithm.
The apparatus of claim 10, wherein:

The focus acquisition module is also configured to divide the preview image collected by the camera into a plurality of image areas according to the preset area size; calculate the gray value corresponding to each of the image areas; compare the gray value corresponding to each of the image areas with a regional gray threshold, and divide the preview image into a backlit area and a non-backlit area; wherein, the backlit area refers to an area in the preview image whose gray value is greater than the regional gray threshold, and the non-backlit area refers to a gray value in the preview image that is less than or equal to the regional gray threshold. area.
An electronic device, comprising: a memory and one or more processors, wherein computer-readable instructions are stored in the memory; when the computer-readable instructions are executed by the one or more processors, the one or more processors execute the steps of the image processing method according to any one of claims 1-9.
One or more non-volatile computer-readable storage media storing computer-readable instructions, when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image processing method according to any one of claims 1-9.