CN114972091A

CN114972091A - Image processing method and device, electronic equipment and storage medium

Info

Publication number: CN114972091A
Application number: CN202210573976.2A
Authority: CN
Inventors: 许泳哲; 严琼
Original assignee: Shenzhen TetrasAI Technology Co Ltd
Current assignee: Shenzhen TetrasAI Technology Co Ltd
Priority date: 2022-05-25
Filing date: 2022-05-25
Publication date: 2022-08-30

Abstract

The disclosure relates to an image processing method, an image processing device, an electronic device and a storage medium, wherein the method comprises the steps of obtaining an original format to-be-processed image and an object area image corresponding to a target object in the to-be-processed image; carrying out brightening processing on the object region image based on first brightness information corresponding to the object region image to obtain a first brightening image; based on second brightness information corresponding to the background image in the image to be processed, carrying out brightening processing on the background image in the image to be processed to obtain a second brightening image; performing object repairing processing on the first brightening image to obtain an object repairing image; performing noise reduction processing on the second brightening image to obtain a first noise reduction image; and generating a target image corresponding to the image to be processed based on the object repairing image and the first noise reduction image. By utilizing the embodiment of the disclosure, the image noise can be effectively reduced, the image definition can be improved, the resource consumption in the image processing process can be reduced, and the equipment performance can be improved.

Description

Image processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.

Background

With the development of computer technology, intelligent hardware devices such as mobile phones and the like are rapidly popularized. When an intelligent hardware device such as a mobile phone is used for shooting an object image such as a human face, the shot object image has the problems of high noise, low image definition and the like due to dark light and the like.

In the related art, when processing an image shot in a dark scene such as night, some intelligent hardware devices often beautify the image to improve the image quality. However, in the related art, although the beauty treatment can reduce noise to a certain extent and improve the aesthetic feeling of objects such as human faces in images, the texture details cannot be recovered, the noise is still large, the Image quality definition is still poor, and the beauty treatment is often performed on images subjected to Image Signal Processing (ISP) such as JPG, and therefore, the problems of large resource consumption in the Image Processing process, poor equipment performance and the like exist.

Disclosure of Invention

The present disclosure provides an image processing method, an image processing apparatus, an electronic device, and a storage medium, which can effectively reduce image noise, improve image definition and quality, and simultaneously reduce resource consumption in an image processing process and improve device performance. The technical scheme of the disclosure is as follows:

according to an aspect of the embodiments of the present disclosure, there is provided an image processing method including:

acquiring an original format image to be processed and an object area image corresponding to a target object in the image to be processed;

carrying out brightening processing on the object area image based on first brightness information corresponding to the object area image to obtain a first brightening image;

based on second brightness information corresponding to the background image in the image to be processed, carrying out brightening processing on the background image in the image to be processed to obtain a second brightening image;

performing object repairing processing on the first brightening image to obtain an object repairing image, wherein the definition of the object repairing image is greater than that of the first brightening image;

performing noise reduction processing on the second brightening image to obtain a first noise reduction image;

and generating a target image corresponding to the image to be processed based on the object repairing image and the first noise reduction image.

In the above embodiment, the object area image corresponding to the target object in the original format image to be processed and the background image in the image to be processed are respectively subjected to brightening treatment, so that the target object shot in a dark scene is more prominent, the brightness better conforms to the light and shadow effect of an actual image, the brightness and the contrast of the image are improved, and the image noise is effectively reduced; and then, the second brightening image corresponding to the image of the object area is subjected to object restoration processing, the second brightening image corresponding to the image to be processed is subjected to noise reduction processing respectively, and the target image corresponding to the image to be processed is generated based on the object restoration image and the first noise reduction image, so that the noise of the target image for imaging is effectively reduced, the image definition and quality are improved, the image processing is reduced before the image imaging, the resource consumption in the image noise reduction process can be greatly reduced, and the equipment performance is improved.

In an optional embodiment, the performing, based on the first luminance information corresponding to the object region image, a brightening process on the object region image to obtain a first brightening image includes:

acquiring a first coded image corresponding to the object region image, wherein the first coded image is an image containing brightness information;

and performing brightening processing on the first coded image based on the first brightness information to obtain the first brightening image.

In the above embodiment, the first coded image including the luminance information corresponding to the object region image is subjected to the brightening treatment by combining the first luminance information, so that the effectiveness and pertinence of the brightening treatment can be greatly improved, the target object shot in a dark scene can be more prominent, the luminance better conforms to the light and shadow effect of an actual image, the luminance and the contrast of a target image obtained subsequently are improved, and the image noise is effectively reduced.

In an optional embodiment, the acquiring the first encoded image corresponding to the object region image includes:

carrying out format conversion processing on the object area image to obtain a first area coded image in a first color coding format;

carrying out format conversion processing on the first area coding image to obtain a second area coding image in a second color coding format;

and taking the brightness channel image in the second area coding image as the first coding image.

In the above embodiment, the luminance channel image is selected as the first encoded image, so that the image processing data amount can be reduced and the image processing efficiency can be improved on the basis of effectively extracting the luminance information of the target area image.

In an optional embodiment, the first brightness information includes a first target average brightness, and the first target average brightness is used for controlling the image quality of the object region image in a preset dimension; performing, by the first encoding apparatus, a first luminance information on the first encoded image, and obtaining the first luminance information includes:

acquiring the first target average brightness and a first initial average brightness corresponding to the first coded image;

constructing first mapping information based on the first initial average brightness and the first target average brightness;

and updating the brightness of the first coding image based on the first mapping information to obtain the first brightening image.

In the above embodiment, the first mapping information is constructed by combining the first target average luminance for controlling the image quality of the object region image in the preset dimension and the first initial average luminance corresponding to the first encoded image, and the luminance of the first encoded image is updated based on the first mapping information, so that the target object in the dark scene is more prominent and the luminance better conforms to the light and shadow effect of the actual image on the basis of improving the image quality of the first brightening image.

In an optional embodiment, the performing, based on second luminance information corresponding to a background image in the image to be processed, a brightening process on the background image in the image to be processed to obtain a second brightening image includes:

acquiring a second coded image corresponding to the image to be processed, wherein the second coded image is an image containing brightness information;

and based on the second brightness information, carrying out brightening processing on a background coded image in the second coded image to obtain the second brightening image.

In the above embodiment, the second coded image containing the luminance information corresponding to the image to be processed is brightened by combining the second luminance information, so that the effectiveness and pertinence of the brightening treatment can be greatly improved, a target object shot in a dark scene can be more prominent, the luminance better conforms to the shadow effect of an actual image, the luminance and the contrast of a target image obtained subsequently are improved, and the image noise is effectively reduced.

In an optional embodiment, the second brightening image is an image in the original format, the first brightness information includes a first target average brightness, and the first target average brightness is used for controlling the image quality of the object region image in a preset dimension; the second brightness information comprises a second target average brightness, and the second target average brightness is used for controlling the image quality of the background image in the preset dimension; performing, on the basis of the second luminance information, a brightening process on a background encoded image in the second encoded image, to obtain the second brightening image, includes:

acquiring the first target average brightness and a second initial average brightness corresponding to the background coding image;

determining the second target average brightness based on the first target average brightness;

constructing second mapping information based on the second initial average brightness and the second target average brightness;

updating the brightness of the background coded image in the second coded image based on the second mapping information to obtain a brightness updated image;

and carrying out format reduction processing on the brightness updating image to obtain the second brightening image.

In the above embodiment, a second target average luminance for controlling the image quality of the background image in the preset dimension is generated by combining the first target average luminance for controlling the image quality of the target area image in the preset dimension, and second mapping information is constructed by combining the second target average luminance and a second initial average luminance corresponding to a background encoded image in the second encoded image, and based on the second mapping information, luminance update is performed on the background encoded image in the second encoded image alone, so that the image quality of the background encoded image in the second encoded image can be improved, and the quality of the entire image can be improved better.

In an optional embodiment, the performing noise reduction processing on the second highlight image to obtain a first noise-reduced image includes:

acquiring a target color channel image corresponding to the second brightening image;

performing low-frequency noise reduction processing on the target color channel image based on an image noise reduction network to obtain a second noise reduction image;

carrying out format reduction processing on the second noise reduction image to obtain a third noise reduction image in the original format;

and carrying out high-frequency noise reduction processing on the third noise reduction image based on a preset frequency domain noise reduction algorithm to obtain the first noise reduction image.

In the above embodiment, the low-frequency noise in the second brightening image can be effectively removed by combining the image denoising network, and the high-frequency noise is removed by combining the traditional frequency domain denoising algorithm, so that the parameter adjusting speed can be effectively increased, the better balance between denoising and image texture detail retaining can be obtained, and the image denoising effect and the definition can be improved.

In an optional embodiment, the performing, by the image noise reduction network, low-frequency noise reduction processing on the target color channel image to obtain a second noise-reduced image includes:

performing down-sampling processing on the target color channel image to obtain a down-sampled image;

inputting the down-sampling image into the image noise reduction network to carry out low-frequency noise reduction processing to obtain a fourth noise reduction image;

respectively performing upsampling processing on the downsampled image and the fourth noise-reduced image to obtain a first upsampled image corresponding to the downsampled image and a second upsampled image corresponding to the fourth noise-reduced image;

determining a high-frequency object image according to the first up-sampling image and the target color channel image;

determining the second noise-reduced image based on the high-frequency object image and a second up-sampled image.

In the above embodiment, before the network performs noise reduction processing, the down-sampling is performed on the target color channel image, so that a high-frequency signal in the target color channel image can be removed, only low-frequency information is retained, which is beneficial for the image noise reduction network to remove low-frequency noise in the image, and the down-sampling processing can compress the image, and also can greatly reduce the amount of calculation in the noise reduction processing process performed by the image noise reduction network, and improve the image noise reduction processing efficiency; and the second up-sampling image obtained by the network after the up-sampling processing of the fourth noise reduction image is fused with the high-frequency object image, so that high-frequency information can be recovered, and high-frequency noise reduction can be performed in the subsequent process.

In an optional embodiment, the first brightened image is an image including luminance information, and performing the object repairing process on the first brightened image to obtain the object repaired image includes:

acquiring a preset image corresponding to the target object, wherein the definition of the preset image is greater than that of the first brightening image;

carrying out object posture transformation analysis on the first brightening image and the preset image to obtain object posture transformation information between the first brightening image and the preset image;

and performing object repairing processing on the first brightening image based on the object posture transformation information and the preset image to obtain the object repairing image.

In the above embodiment, the preset image and the object posture change information between the first brightening image and the preset image are combined to perform object repairing processing on the first brightening image containing the brightness information, so that the definition of the image in a dark scene can be greatly improved.

In an alternative embodiment, the object region image comprises an image obtained by:

carrying out object detection on the image to be processed to obtain key point information of the target object;

carrying out noise reduction processing on the image to be processed to obtain a fifth noise reduction image;

and determining the object region image from the fifth noise reduction image based on the key point information.

In the above embodiment, after the noise of the image to be processed is reduced, the object region image corresponding to the target object is determined from the noise-reduced fifth noise-reduced image in combination with the key point information of the target object, so that the noise introduced from the short and medium exposure frame images with shorter exposure time can be effectively removed when the object images with multiple exposure times are fused, and the image quality of the object region image is further improved.

In an optional embodiment, the acquiring the image to be processed includes:

acquiring a first exposure image in the original format, a second exposure image in the original format and a third exposure image in the original format in response to an image acquisition instruction for the target object; the exposure time of the first exposure image is longer than that of the second exposure image, and the exposure time of the second exposure image is longer than that of the third exposure image;

carrying out object alignment processing on the first exposure image, the second exposure image and the third exposure image to obtain a first alignment image corresponding to the first exposure image, a second alignment image corresponding to the second exposure image and a third alignment image corresponding to the third exposure image;

generating the image to be processed based on the first, second, and third alignment images.

In the above embodiment, under the condition that an image acquisition instruction for a target object is triggered, an alignment image after object alignment processing is performed in combination with exposure images in an original format corresponding to a plurality of exposure times to generate an image to be processed, so that brightness complementation can be realized based on different exposure images, and the comprehensiveness of image brightness information in a dim light scene is improved.

In an optional embodiment, the object repairing image is an image containing luminance information, the first noise-reduced image is an image in the original format, and generating a target image corresponding to the image to be processed based on the object repairing image and the first noise-reduced image includes:

acquiring a third coded image corresponding to the first noise-reduced image, wherein the third coded image is an image containing brightness information;

updating a region image corresponding to the target object in the third encoded image based on the object restoration image to obtain a fourth encoded image;

and carrying out format reduction processing on the fourth coded image to obtain the target image.

In the above embodiment, based on the object repair image including the luminance information, the region image corresponding to the target object in the third encoded image including the luminance information and corresponding to the first noise-reduced image is updated to obtain the fourth encoded image, and the format reduction processing is performed on the fourth encoded image, so that the image quality and the definition of the target image for imaging can be greatly improved.

According to another aspect of the embodiments of the present disclosure, there is provided an image processing apparatus including:

the image acquisition module is configured to acquire an object area image corresponding to a target object in an original format image to be processed;

the first brightening processing module is configured to execute brightening processing on the object area image based on first brightness information corresponding to the object area image to obtain a first brightening image;

the second brightening processing module is configured to carry out brightening processing on the background image in the image to be processed based on second brightness information corresponding to the background image in the image to be processed to obtain a second brightening image;

the object repairing processing module is configured to perform object repairing processing on the first brightening image to obtain an object repairing image, and the definition of the object repairing image is greater than that of the first brightening image;

the first noise reduction processing module is configured to perform noise reduction processing on the second brightening image to obtain a first noise reduction image;

and the target image generation module is configured to generate a target image corresponding to the image to be processed based on the object repairing image and the first noise reduction image.

In an optional embodiment, the first brightening processing module comprises:

a first encoded image acquisition unit configured to perform acquisition of a first encoded image corresponding to the object region image, the first encoded image being an image containing luminance information;

a first brightening processing unit configured to perform brightening processing on the first encoded image based on the first luminance information, resulting in the first brightening image.

In an alternative embodiment, the first encoded image acquisition unit includes:

a first format conversion processing unit configured to perform format conversion processing on the object region image, resulting in a first region encoded image in a first color encoding format;

a second format conversion processing unit configured to perform format conversion processing on the first area encoded image, resulting in a second area encoded image in a second color encoding format;

an encoded image determination unit configured to perform encoding of a luminance channel image in the second region encoded image as the first encoded image.

In an optional embodiment, the first brightness information includes a first target average brightness, and the first target average brightness is used for controlling the image quality of the object region image in a preset dimension; the first brightening processing unit includes:

a first average luminance obtaining unit configured to perform obtaining the first target average luminance and a first initial average luminance corresponding to the first encoded image;

a first mapping information construction unit configured to perform construction of first mapping information based on the first initial average brightness and the first target average brightness;

a first luminance updating unit configured to perform luminance updating on the first encoded image based on the first mapping information, resulting in the first luma image.

In an optional embodiment, the second brightening processing module comprises:

a second coded image acquisition unit configured to perform acquisition of a second coded image corresponding to the image to be processed, the second coded image being an image including luminance information;

and the second brightening processing unit is configured to carry out brightening processing on a background coded image in the second coded image based on the second brightness information to obtain the second brightening image.

In an optional embodiment, the second brightening image is an image in the original format, the first brightness information includes a first target average brightness, and the first target average brightness is used for controlling the image quality of the object region image in a preset dimension; the second brightness information comprises a second target average brightness, and the second target average brightness is used for controlling the image quality of the background image in the preset dimension; the second brightening processing unit includes:

a second average brightness acquiring unit configured to perform acquiring the first target average brightness and a second initial average brightness corresponding to the background encoded image;

an average luminance determination unit configured to perform determination of the second target average luminance based on the first target average luminance;

a second mapping information construction unit configured to perform construction of second mapping information based on the second initial average brightness and the second target average brightness;

a second brightness updating unit configured to perform brightness updating on a background encoded image in the second encoded image based on the second mapping information to obtain a brightness updated image;

and the first format reduction processing unit is configured to perform format reduction processing on the brightness updating image to obtain the second brightening image.

In an optional embodiment, the first denoising processing module includes:

a target color channel image acquisition unit configured to perform acquisition of a target color channel image corresponding to the second highlight image;

the first noise reduction processing unit is configured to perform low-frequency noise reduction processing on the target color channel image based on an image noise reduction network to obtain a second noise reduction image;

the second format reduction processing unit is configured to perform format reduction processing on the second noise reduction image to obtain a third noise reduction image in the original format;

and the second noise reduction processing unit is configured to execute high-frequency noise reduction processing on the third noise reduction image based on a preset frequency domain noise reduction algorithm to obtain the first noise reduction image.

In an optional embodiment, the first denoising processing unit includes:

a down-sampling processing unit configured to perform down-sampling processing on the target color channel image to obtain a down-sampled image;

a third noise reduction processing unit configured to input the downsampled image into the image noise reduction network for low-frequency noise reduction processing to obtain a fourth noise reduction image;

an upsampling processing unit configured to perform upsampling processing on the downsampled image and the fourth noise-reduced image respectively to obtain a first upsampled image corresponding to the downsampled image and a second upsampled image corresponding to the fourth noise-reduced image;

a high frequency object image determination unit configured to perform determining a high frequency object image from the first upsampled image and the target color channel image;

a second noise-reduced image determination unit configured to perform determination of the second noise-reduced image based on the high-frequency object image and a second up-sampled image.

In an optional embodiment, the first brightened image is an image including brightness information, and the object repairing processing module includes:

a preset image acquisition unit configured to perform acquisition of a preset image corresponding to the target object, the preset image having a higher definition than the first highlight image;

an object posture transformation analysis unit configured to perform object posture transformation analysis on the first highlight image and the preset image to obtain object posture transformation information between the first highlight image and the preset image;

and the object repairing processing unit is configured to execute object repairing processing on the first brightening image based on the object posture transformation information and the preset image to obtain the object repairing image.

In an alternative embodiment, the image acquisition module comprises:

the object detection unit is configured to perform object detection on the image to be processed to obtain key point information of the target object;

the noise reduction processing unit is configured to perform noise reduction processing on the image to be processed to obtain a fifth noise reduction image;

an object region image determination unit configured to perform determination of the object region image from the fifth noise-reduced image based on the key point information.

In an optional embodiment, the to-be-processed image acquiring unit includes:

an exposure image acquisition unit configured to execute acquisition of a first exposure image in the original format, a second exposure image in the original format, and a third exposure image in the original format in response to an image capture instruction for the target object; the exposure time of the first exposure image is longer than that of the second exposure image, and the exposure time of the second exposure image is longer than that of the third exposure image;

an object alignment processing unit, configured to perform object alignment processing on the first exposure image, the second exposure image and the third exposure image to obtain a first alignment image corresponding to the first exposure image, a second alignment image corresponding to the second exposure image and a third alignment image corresponding to the third exposure image;

a to-be-processed image generation unit configured to perform generation of the to-be-processed image based on the first, second, and third alignment images.

In an optional embodiment, the object restoration image is an image including luminance information, the first noise reduction image is an image in the original format, and the target image generation module includes:

an object code image acquisition unit configured to perform acquisition of a third code image corresponding to the first noise reduction image, the third code image being an image containing luminance information;

an object region image updating unit configured to update a region image corresponding to the target object in the third encoded image based on the object repair image, resulting in a fourth encoded image;

and the second format reduction processing unit is configured to perform format reduction processing on the fourth coded image to obtain the target image.

According to another aspect of the embodiments of the present disclosure, there is provided an electronic device including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method of any one of the above.

According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions of the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform any one of the methods of the embodiments of the present disclosure.

According to another aspect of the embodiments of the present disclosure, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the methods of the embodiments of the present disclosure described above.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a flow diagram illustrating an image processing method according to an exemplary embodiment;

FIG. 2 is a flow diagram illustrating an acquisition of an image to be processed according to an exemplary embodiment;

FIG. 3 is a flowchart illustrating a process for brightening a first encoded image based on first luminance information to obtain a first brightened image, according to an example embodiment;

FIG. 4 is a flowchart illustrating a process of performing a brightening process on a background encoded image in a second encoded image based on second luminance information, resulting in a second brightened image, according to an example embodiment;

FIG. 5 is a schematic diagram of a denoising process performed on a second brightened image according to an exemplary embodiment;

FIG. 6 is a schematic illustration of an image imaging process provided in accordance with an exemplary embodiment;

FIG. 7 is a block diagram of an image processing apparatus according to an exemplary embodiment;

FIG. 8 is a block diagram illustrating an electronic device for image processing in accordance with an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the disclosure, as detailed in the appended claims.

The present disclosure provides an image processing method, which can be applied to a terminal. Specifically, the terminal may include, but is not limited to, a smart phone, a desktop computer, a tablet computer, a notebook computer, a smart speaker, a digital assistant, an Augmented Reality (AR)/Virtual Reality (VR) device, a smart wearable device, a vehicle-mounted terminal, a smart television, and other types of entity devices; software running on the physical device, such as an application, an applet, etc., is also possible.

In practical applications, the terminal may be provided with a camera device, and optionally, the camera device may be a camera device integrated with the terminal, or may be a split camera device connected in a wired or wireless manner.

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment, which is used in an electronic device such as a terminal, as shown in fig. 1, and includes the following steps.

S101: acquiring an object area image corresponding to a target object in an original format image to be processed;

in a specific embodiment, the raw format may be an image format directly output by an image sensor in the camera device. Specifically, the target object may be an object to be photographed, such as a human face, a hand, and the like.

In a specific embodiment, the image to be processed may be an image in an original format obtained by fusing object images corresponding to a plurality of exposure times. Optionally, the object images corresponding to the multiple exposure times may be images of a target object captured in a dark scene.

In an alternative embodiment, as shown in fig. 2, the acquiring the to-be-processed image may include:

s201: in response to an image capture instruction for a target object, a first exposure image in a native format, a second exposure image in the native format, and a third exposure image in the native format are acquired.

S203: and carrying out object alignment processing on the first exposure image, the second exposure image and the third exposure image to obtain a first alignment image corresponding to the first exposure image, a second alignment image corresponding to the second exposure image and a third alignment image corresponding to the third exposure image.

S205: and generating an image to be processed based on the first alignment image, the second alignment image and the third alignment image.

In practical application, the image acquisition instruction can be triggered by pressing a hardware button of the camera device, clicking a preset control and the like. Correspondingly, the camera device can collect a first exposure image in an original format, a second exposure image in the original format and a third exposure image in the original format.

In an alternative embodiment, the first exposure image in the original format, the second exposure image in the original format, and the third exposure image in the original format may be images acquired by the image sensor with different exposure times.

In an alternative embodiment, the first exposure image in the original format, the second exposure image in the original format, and the third exposure image in the original format may be images obtained by performing dead pixel correction on images acquired by the image sensor at different exposure times. Optionally, the dead pixel correction may include static dead pixel correction, dynamic dead pixel correction, and the like.

In a specific embodiment, the first exposure image, the second exposure image and the third exposure image may be images of the target object acquired at different exposure times. Specifically, the exposure time of the first exposure image is longer than that of the second exposure image, and the exposure time of the second exposure image is longer than that of the third exposure image. Alternatively, the first exposure image may include one or more frames of images with the same exposure time.

In this embodiment, the first exposure image may be at least one frame of image including the target object, and optionally, in the case where the first exposure image is one frame of image including the target object, the object alignment process may be performed using the frame of image (first exposure image) as a reference image, and accordingly, the first alignment image may be the first exposure image. The above performing the object alignment processing on the first exposure image, the second exposure image and the third exposure image to obtain the first alignment image corresponding to the first exposure image, the second alignment image corresponding to the second exposure image and the third alignment image corresponding to the third exposure image may include:

and taking the first exposure image as a reference image, and carrying out object alignment processing on the second exposure image and the third exposure image to obtain a second alignment image corresponding to the second exposure image and a third alignment image corresponding to the third exposure image.

In a specific embodiment, the second alignment image corresponding to the second exposure image may be an image obtained by performing object alignment processing on the second exposure image with the first exposure image as a reference image, and the third alignment image corresponding to the third exposure image may be an image obtained by performing object alignment processing on the third exposure image with the first exposure image as a reference image.

In an alternative embodiment, the first exposure image may include a plurality of frames of images with the same exposure time, and correspondingly, the first alignment image may include other alignment images corresponding to the preset image in the first exposure image and other images in the first exposure image; the preset image may be any one of the first exposure images, and optionally, the preset image may be an image with the earliest acquisition time in the multi-frame images. The other images may be images other than the preset image in the first exposure image; correspondingly, the performing the object alignment processing on the first exposure image, the second exposure image and the third exposure image to obtain the first alignment image corresponding to the first exposure image, the second alignment image corresponding to the second exposure image and the third alignment image corresponding to the third exposure image may include:

and taking a preset image in the first exposure image as a reference image, and carrying out object alignment processing on the second exposure image, the third exposure image and other images to obtain other alignment images, a second alignment image and a third alignment image.

In an optional embodiment, the second alignment image corresponding to the second exposure image may be an image obtained by performing object alignment processing on the second exposure image with a preset image as a reference image, and the third alignment image corresponding to the third exposure image may be an image obtained by performing object alignment processing on the third exposure image with the preset image as a reference image.

In a specific embodiment, in the object alignment processing, an optical flow method, a homography matrix mapping method, and the like may be combined, and the embodiments of the present disclosure are not limited to the above.

In an optional embodiment, in a case that the first alignment image includes other alignment images corresponding to the preset image in the first exposure image and other images in the first exposure image, the generating the to-be-processed image based on the first alignment image, the second alignment image, and the third alignment image may include:

fusing the preset image and other aligned images to obtain a first fused image; carrying out image correction processing on the first fusion image, the second alignment image and the third alignment image to obtain a first correction image corresponding to the first fusion image, a second correction image corresponding to the second alignment image and a third correction image corresponding to the third alignment image; and performing fusion processing on the first correction image, the second correction image and the third correction image to obtain an image to be processed.

In a specific embodiment, the preset image and the other object images may be fused by using a multi-frame weighted average, a frequency domain fusion, or the like, so as to obtain the first fused image.

In a specific embodiment, the image correction processing performed on the first fused image, the second aligned image and the third aligned image may include, but is not limited to, performing white balance processing or lens shading correction processing on the first fused image, the second aligned image and the third aligned image.

In a specific embodiment, the first corrected image, the second corrected image, and the third corrected image may be fused by using a multi-frame weighted average, a frequency domain fusion, and the like, so as to obtain an image to be processed.

In the above embodiment, the image noise reduction effect can be improved by fusing the preset image with a longer exposure time with other object images, and the brightness and color of the image can be improved on the basis of brightness complementation by combining the images with different exposure times by firstly carrying out image correction on the first fused image, the second aligned image and the third aligned image and then carrying out image fusion.

In an alternative embodiment, in the case that the first alignment image is the first exposure image, the generating the image to be processed based on the first alignment image, the second alignment image, and the third alignment image may include:

carrying out image correction processing on the first alignment image, the second alignment image and the third alignment image to obtain calibrated images corresponding to the first alignment image, the second alignment image and the third alignment image respectively; and fusing the calibrated images corresponding to the first alignment image, the second alignment image and the third alignment image to obtain the image to be processed.

In a specific embodiment, the image correction process may include, but is not limited to, a white balance process, a lens shading correction process, and the like. The fusion process may include, but is not limited to, multi-frame weighted averaging, frequency domain fusion, and the like.

In the above embodiment, under the condition of triggering an image acquisition instruction for a target object, an alignment image after object alignment processing is performed in combination with exposure images in an original format corresponding to a plurality of exposure times to generate an image to be processed, so that brightness complementation can be realized based on different exposure images, and the comprehensiveness of image brightness information in a dim light scene is improved.

In an alternative embodiment, the object region image may be an image of a region in which the target object is located in the image to be processed. Optionally, in order to improve the quality in the image, after the noise of the image to be processed is reduced, an object region image may be extracted from the noise-reduced image, and accordingly, the object region image may be obtained in the following manner:

carrying out object detection on an image to be processed to obtain key point information of a target object; carrying out noise reduction processing on the image to be processed to obtain a fifth noise reduction image; and determining the object region image from the fifth noise-reduced image based on the key point information.

In a specific embodiment, format conversion processing may be performed on an image to be processed in an original format to obtain an image to be processed in a first color coding format; further, object detection may be performed on the image to be processed in the first color coding format to obtain key point information of the target object in the image to be processed.

In an alternative embodiment, the object detection process may be performed in conjunction with a preset object detection network. The first color encoding format may be an RGB format.

In a specific embodiment, the denoising processing performed on the image to be processed may adopt a preset denoising network, or may adopt a traditional frequency domain denoising algorithm. Correspondingly, the fifth noise-reduced image may be an image subjected to noise reduction processing on the image to be processed.

In a specific embodiment, the object region image may be an original format image. Specifically, the image of the region corresponding to the key point information in the fifth noise-reduced image may be taken as the target region image in combination with the key point information.

S103: and carrying out brightening processing on the target area image based on the first brightness information corresponding to the target area image to obtain a first brightening image.

In a specific embodiment, the first brightening image corresponding to the object region image may be an image obtained by brightening the object region image; the first luminance information may be used to control a shadow effect of the object region image.

In an optional embodiment, performing a brightening process on the object region image based on the first luminance information corresponding to the object region image, to obtain a first brightening image may include:

acquiring a first coded image corresponding to the object region image; and performing brightening processing on the first coded image based on the first brightness information to obtain a first brightening image.

In a specific embodiment, the first encoded image may be an image including luminance information; alternatively, the first encoded image may be an image in YUV format, and accordingly, the object region image may be converted into an image in first color encoding format (RGB) and converted into an image in second color encoding format (YUV) from the image in first color encoding format (RGB).

In an alternative embodiment, the first encoded image may be a luminance channel image in an image in YUV format; correspondingly, the acquiring of the first encoded image corresponding to the object region image may include:

carrying out format conversion processing on the object area image to obtain a first area coding image in a first color coding format; carrying out format conversion processing on the first area coded image to obtain a second area coded image in a second color coding format; and taking the brightness channel image in the second area coded image as the first coded image.

In a specific embodiment, the second region-coded image may include a Y-channel image, a U-channel image, and a V-channel image corresponding to the object region image. The Y-channel image (luminance channel image in the second area encoded image) corresponding to the object area image may be used as the first encoded image.

In an alternative embodiment, the first luminance information may include a first target average luminance; as shown in fig. 3, the performing a brightening process on the first encoded image based on the first luminance information to obtain a first brightening image may include:

s301: acquiring first target average brightness and first initial average brightness corresponding to a first coded image;

s303: constructing first mapping information based on the first initial average brightness and the first target average brightness;

s305: and updating the brightness of the first coding image based on the first mapping information to obtain a first brightening image.

In a specific embodiment, the first initial average luminance may be an average luminance of the first encoded image. The first target average brightness may be used to control the image quality of the object region image in a preset dimension; specifically, the preset dimension corresponding to the image quality may be a preset image quality dimension for measuring the image shadow effect. Optionally, the preset dimension may be an image exposure dimension, an image brightness dimension, or the like, and optionally, taking the preset dimension as the image exposure dimension as an example, the first target average brightness may be determined by combining with the image quality requirement that the highlight of the object is not overexposed, so as to effectively control the image shadow effect of the object region. Optionally, taking a preset dimension as an image brightness dimension as an example, the first target average brightness may be determined in combination with an image quality requirement that the average brightness corresponding to the first encoded image needs to reach seventy percent of the average brightness of the full image (the second encoded image), so as to effectively control the image shadow effect of the object region image.

In a specific embodiment, a reference function of the luminance distribution map may be preset, and the reference function is fitted based on the first initial average luminance and the first target average luminance to obtain the first mapping information;

optionally, the reference function may be a linear function or a non-linear function, the first mapping information may be a mapping curve about a luminance distribution graph, and optionally, the luminance distribution graph may be a luminance histogram, an abscissa of the zero degree histogram may be a pixel value (0 to 255 pixels), and an ordinate may be a number of pixels of the corresponding pixel value in the first encoded image.

In a specific embodiment, the performing luminance update on the first encoded image based on the first mapping information to obtain the first enhanced image may include:

generating a first brightness distribution graph based on the first mapping information and an initial region brightness distribution graph corresponding to the first coded image; carrying out equalization processing on the first brightness distribution map to obtain a second brightness distribution map; generating first brightness mapping information based on the second brightness distribution map and the initial region brightness distribution map; and updating the brightness of the first coding image based on the first brightness mapping information to obtain a first brightening image.

In a specific embodiment, the first luminance profile may be obtained by applying the first mapping information to the initial region luminance profile corresponding to the first encoded image. Optionally, there may be an overexposed region in the first luminance distribution map, and accordingly, the first luminance distribution map may be equalized to obtain a second luminance distribution map. Then, first luminance mapping information between the initial region luminance distribution map before the brightening and the second luminance distribution map can be determined according to the cumulative distribution function, and the first luminance mapping information can be a mapping relation curve. The abscissa of the mapping relationship curve is the pixel value of the first encoded image, and the ordinate is the mapped pixel value. Accordingly, the initial brightness in the first encoded image may be updated in a brightness manner in combination with the first brightness mapping information, so as to obtain an image (first brightened image) after the first encoded image is brightened. Specifically, the image format of the first enhanced image is consistent with the image format of the first encoded image, and the first enhanced image may be an image in a YUV format or a luminance channel image in an image in a YUV format.

In the above embodiment, the first target average brightness used for controlling the image quality of the object region image in the preset dimension and the first initial average brightness corresponding to the first encoded image are combined to construct the first mapping information, and based on the first mapping information, the brightness of the first encoded image is updated separately, so that the target object in the dark scene is more prominent and the brightness better conforms to the shadow effect of the actual image on the basis of improving the image quality of the first brightening image.

S105: and carrying out brightening treatment on the background image in the image to be processed based on second brightness information corresponding to the background image in the image to be processed to obtain a second brightening image.

In a specific embodiment, the second brightening image corresponding to the image to be processed may be an image obtained by brightening a background image in the image to be processed. The second brightness information corresponding to the background image in the image to be processed may be used to control a shadow effect of the background image in the image to be processed.

In an optional embodiment, the performing, based on the second luminance information corresponding to the background image in the image to be processed, a brightening process on the background image in the image to be processed to obtain a second brightening image may include:

acquiring a second coded image corresponding to the image to be processed; and based on the second brightness information, carrying out brightening processing on the background coded image in the second coded image to obtain a second brightening image.

In a particular embodiment, the second encoded image may be an image that includes luminance information. Optionally, the original format image to be processed may be subjected to down-sampling processing to obtain an image (i.e., a second encoded image) containing luminance information.

In an optional embodiment, the second enhanced image is an image in an original format, and the second luminance information includes a second target average luminance corresponding to the background encoded image; as shown in fig. 4, the performing a brightening process on the background coded image in the second coded image based on the second luminance information to obtain a second brightening image packet may include:

s401: acquiring first target average brightness and second initial average brightness corresponding to a background coded image;

s403: determining a second target average brightness based on the first target average brightness;

s405: constructing second mapping information based on the second initial average brightness and the second target average brightness;

s407: based on the second mapping information, carrying out brightness updating on a background coding image in the second coding image to obtain a brightness updating image;

s409: and carrying out format reduction processing on the brightness updating image to obtain a second brightening image.

In a specific embodiment, the second initial average luminance may be an average luminance of the background encoded image. The two-target average brightness can be used to control the image quality of the background image in a preset dimension. Optionally, in the process of determining the average brightness of the second target, in order to improve the shadow effect of the overall image, the average brightness of the second target may be determined by combining the average brightness of the first target, taking a preset dimension as an image exposure dimension as an example, the average brightness of the first target may be combined to determine that the background highlight is not overexposed, and then the average brightness of the second target may be determined by combining the background highlight not overexposed, so as to effectively control the image shadow effect of the background image.

In a specific embodiment, for the specific refinement for constructing the second mapping information based on the second initial average brightness and the second target average brightness, reference may be made to the specific refinement for constructing the first mapping information based on the first initial average brightness and the first target average brightness, which is not described herein again.

In a specific embodiment, the performing, based on the second mapping information, luminance update on the background encoded image in the second encoded image to obtain a luminance updated image may include:

generating a third brightness distribution graph based on the second mapping information and the initial background brightness distribution graph corresponding to the background coding image; carrying out equalization processing on the third brightness distribution map to obtain a fourth brightness distribution map; generating second brightness mapping information based on the fourth brightness distribution map and the initial background brightness distribution map; and updating the brightness of the background coded image in the second coded image based on the second brightness mapping information to obtain a brightened second coded image.

In a specific embodiment, the luminance updating is performed on the background encoded image in the second encoded image based on the second mapping information to obtain the specific refinement of the luminance updated image, and the luminance updating is performed on the first encoded image based on the first mapping information to obtain the specific refinement of the first enhanced image, which is not described herein again.

In a specific embodiment, the luminance update image may be a brightening image corresponding to the second encoded image after brightening the background encoded image in the second encoded image. Optionally, performing format reduction processing on the luminance update image to obtain the second brightening image may include converting the luminance update image into the second brightening image in the original format.

In the above embodiment, the second coded image containing the luminance information corresponding to the image to be processed is brightened by combining the second luminance information, so that the effectiveness and pertinence of the brightening treatment can be greatly improved, a target object shot in a dark scene can be more prominent, the luminance better conforms to the shadow effect of an actual image, the luminance and the contrast of a target image obtained subsequently are improved, and the image noise is effectively reduced. And combining with a first target average brightness for controlling the image quality of the object region image in a preset dimension to generate a second target average brightness for controlling the image quality of the background image in the preset dimension, and combining with the second target average brightness and a second initial average brightness corresponding to the background encoded image in the second encoded image to construct second mapping information, and based on the second mapping information, performing brightness update on the background encoded image in the second encoded image alone, so as to improve the image quality of the background encoded image in the second encoded image and better improve the quality of the whole image.

S107: performing object repairing processing on the first brightening image to obtain an object repairing image;

in a specific embodiment, the sharpness of the object restoration image is greater than the sharpness of the first highlight image.

In an optional embodiment, the first enhanced image may be an image including luminance information, and accordingly, the performing the object repairing process on the first enhanced image to obtain the object repaired image may include:

acquiring a preset image corresponding to a target object; carrying out object posture transformation analysis on the first brightening image and the preset image to obtain object posture transformation information between the first brightening image and the preset image; and performing object repairing processing on the first brightening image based on the object posture transformation information and the preset image to obtain an object repairing image.

In a specific embodiment, the preset image may be an image including the same type of object as the target object, for example, the target object is a female face; accordingly, the preset image may include a female face as an object. Specifically, the definition of the preset image is greater than that of the first brightening image. Optionally, the definition of the preset image is greater than or equal to a preset threshold, and the preset threshold may be an image definition measurement threshold set in combination with an image quality requirement in practical application.

In a specific embodiment, the object posture transformation information may be posture transformation information between the target object in the first highlight image and the object in the preset image.

In a specific embodiment, the first brightening image is subjected to object repairing processing based on the object posture transformation information and the preset image, and a pre-trained object repairing network can be combined in the process of obtaining the object repairing image; optionally, the object posture transformation information, the preset image and the first brightening image may be input to an object repairing network for object repairing processing, so as to obtain the object repairing image; or performing posture transformation on the object in the preset image by combining the object posture transformation information so as to align the posture of the object in the preset image with that of the target object, and then inputting the object reference image and the first highlight image after the posture transformation into the object repairing network for object repairing processing to obtain the object repairing image. Specifically, the image format of the object restoration image is consistent with the image format of the first brightening image, and the object restoration image may be an image in a YUV format or a luminance channel image in an image in a YUV format.

S109: performing noise reduction processing on the second brightening image to obtain a first noise reduction image;

in an optional embodiment, the performing noise reduction processing on the second highlight image to obtain a first noise-reduced image may include:

acquiring a target color channel image corresponding to the second brightening image; carrying out low-frequency noise reduction processing on the target color channel image based on an image noise reduction network to obtain a second noise reduction image; carrying out format reduction processing on the second noise-reduced image to obtain a third noise-reduced image in an original format; and carrying out high-frequency noise reduction processing on the third noise reduction image based on a preset frequency domain noise reduction algorithm to obtain a first noise reduction image.

In a specific embodiment, format conversion is performed on the second brightening image in the original format to obtain a multi-channel color image, and a plurality of pixels (for example, the multi-channel color image is a bggr image, and the plurality of pixels on each unit may be four pixels corresponding to the bggr) on each unit in the multi-channel color image are stacked to a channel dimension to obtain a target color channel image, so that different color channels can be processed independently, interference is avoided, and the noise reduction effect is improved better.

In an optional embodiment, the performing, by the image noise reduction network, low-frequency noise reduction processing on the target color channel image to obtain a second noise-reduced image may include:

carrying out down-sampling processing on the target color channel image to obtain a down-sampled image; inputting the down-sampled image into an image noise reduction network to carry out low-frequency noise reduction processing to obtain a fourth noise reduction image; respectively performing up-sampling processing on the down-sampled image and the fourth noise-reduced image to obtain a first up-sampled image corresponding to the down-sampled image and a second up-sampled image corresponding to the fourth noise-reduced image; determining a high-frequency object image according to the first up-sampling image and the target color channel image; and determining a second noise reduction image based on the high-frequency object image and the second up-sampled image.

In a specific embodiment, downsampling processing may be performed on the target color channel image by combining with a downsampling algorithm such as a gaussian pyramid to obtain a downsampled image after the downsampling processing of the target color channel image, and then the downsampled image is input into a pre-trained image denoising network to perform low-frequency denoising processing, so as to obtain a fourth denoising image. Specifically, before the network performs noise reduction processing, the down-sampling is performed on the target color channel image, so that a high-frequency signal in the target color channel image can be removed, only low-frequency information is retained, the image noise reduction network is facilitated to remove low-frequency noise in the image, meanwhile, the down-sampling processing can compress the image, and the calculation amount in the noise reduction processing process performed by the image noise reduction network can be greatly reduced; then, in order to restore the high-frequency information, the downsampled image may be upsampled to obtain a first upsampled image corresponding to the downsampled image, and the first upsampled image is removed from the target color channel image to obtain a high-frequency object image; and performing fusion processing on the high-frequency object image and the second up-sampled image subjected to the up-sampling processing on the fourth noise-reduced image to obtain the second noise-reduced image.

In the above embodiment, before the network performs noise reduction processing, the down-sampling is performed on the target color channel image, so that a high-frequency signal in the target color channel image can be removed, only low-frequency information is retained, which is beneficial for the image noise reduction network to remove low-frequency noise in the image, and the down-sampling processing can compress the image, and also can greatly reduce the amount of calculation in the process of performing noise reduction processing on the image noise reduction network, thereby improving the efficiency of image noise reduction processing; and the second up-sampling image obtained by the network after the up-sampling processing of the fourth noise reduction image is fused with the high-frequency object image, so that high-frequency information can be recovered, and high-frequency noise reduction can be performed in the subsequent process.

After the low-frequency noise in the image is effectively removed based on the image noise reduction network, the high-frequency noise in the third noise reduction image can be effectively removed by combining a preset frequency domain noise reduction algorithm. Optionally, the preset frequency domain denoising algorithm may include, but is not limited to, wavelet transform, fourier transform, and other algorithms, and accordingly, after the third denoising image is converted into the frequency domain by combining the preset frequency denoising algorithm, the high frequency noise may be removed, and the frequency domain image from which the high frequency noise is removed is converted into the time domain, so as to obtain the first denoising image.

In an alternative embodiment, as shown in fig. 5, fig. 5 is a schematic diagram of a noise reduction process for a second highlight image according to an exemplary embodiment. Specifically, a target color channel image corresponding to the second brightening image can be obtained, downsampling processing is performed on the target color channel image by combining downsampling algorithms such as a gaussian pyramid, downsampled images of the target color channel image after downsampling processing are obtained, and then the downsampled images are input into a pre-trained image denoising network, so that a fourth denoising image is obtained. In addition, the downsampled image may be upsampled to obtain a first upsampled image corresponding to the downsampled image, and the first upsampled image may be removed from the target color channel image to obtain a high-frequency object image; and performing fusion processing on the high-frequency object image and the second up-sampled image subjected to the up-sampling processing on the fourth noise-reduced image to obtain the second noise-reduced image. And then, carrying out format reduction processing on the second noise reduction image to obtain a third noise reduction image in an original format, and removing high-frequency noise in the third noise reduction image by combining a preset frequency domain noise reduction algorithm to obtain the first noise reduction image.

In the above embodiment, the low-frequency noise in the second brightening image can be effectively removed by combining the image denoising network, and then the high-frequency noise is removed by combining the traditional frequency domain denoising algorithm, so that the parameter adjusting speed can be effectively increased, the better balance between denoising and image texture detail retaining can be obtained, and the image denoising effect and definition can be improved.

S111: and generating a target image corresponding to the image to be processed based on the object repairing image and the first noise reduction image.

In an optional embodiment, the object repairing image may be an image including luminance information, and the first noise-reduced image may be an image in an original format, and optionally, the generating a target image corresponding to the image to be processed based on the object repairing image and the first noise-reduced image may include:

acquiring a third coded image corresponding to the first noise-reduced image; updating the region image corresponding to the target object in the third coded image based on the object restoration image to obtain a fourth coded image; and carrying out format reduction processing on the fourth coded image to obtain a target image.

In a specific embodiment, the third encoded image may be an image containing luminance information, and optionally, the third encoded image may be an image in YUV format. Optionally, the first denoising object may be demosaic and black and white level subtraction operations are performed to obtain an image in RGB format, and then the image in RGB format is converted into an image in YUV format to obtain a third encoded image.

In an alternative embodiment, in order to better improve the image quality, the first noise-reduced image may be subjected to denoising and sharpening operations, and then converted into a corresponding image in YUV format.

In an optional embodiment, assuming that the object repairing image is a luminance channel image in an image in a YUV format, the object repairing image may be attached to a Y channel of an area image corresponding to a target object in the third encoded image according to the object mask information, so as to obtain a fourth encoded image.

In an optional embodiment, assuming that the object restoration image is an image in YUV format, the object restoration image may be attached to the region image corresponding to the target object in the third encoded image according to the object mask information, so as to obtain a fourth encoded image.

In a specific embodiment, the fourth encoded image may be an image in YUV format, and the fourth encoded image may be converted from YUV format back to RGB format and then converted back to the target image in the original format using double mosaic, plus black and white level operation, and the like.

In a specific embodiment, the target Image may be an Image in an original format for imaging, and in the case of obtaining the target Image, the target Image in the original format may be sent to an ISP (Image Signal Processing) unit for performing mild noise reduction, super-resolution up-sampling, demosaicing, color correction matrix, gamma conversion, and the like to obtain an imaged Image (e.g., an Image in an imaging format such as JPG). The ISP unit may be a unit for processing the output signal of the front-end image sensor.

According to the technical scheme provided by the embodiment of the specification, the specification respectively brightens the object area image corresponding to the target object in the image to be processed in the original format and the background image in the image to be processed, so that the target object shot in a dark scene is more prominent, the brightness better conforms to the shadow effect of the actual image, the brightness and the contrast of the image are improved, and the image noise is effectively reduced; and then, the second brightening image corresponding to the image of the object area is subjected to object restoration processing, the second brightening image corresponding to the image to be processed is subjected to noise reduction processing respectively, and the target image corresponding to the image to be processed is generated based on the object restoration image and the first noise reduction image, so that the noise of the target image for imaging is effectively reduced, the image definition and quality are improved, the image processing is reduced before the image imaging, the resource consumption in the image noise reduction process can be greatly reduced, and the equipment performance is improved.

In an alternative embodiment, including with the image to be processed is shown in fig. 6, where fig. 6 is a schematic diagram of an image imaging process provided in accordance with an exemplary embodiment. Specifically, a first exposure image in an original format, a second exposure image in the original format and a third exposure image in the original format can be collected by combining the camera device; then, carrying out object alignment processing on the first exposure image, the second exposure image and the third exposure image, and fusing the aligned images to obtain an image to be processed in an original format; then, extracting an object area image in an original format from the to-be-processed image in the original format, and carrying out brightening processing on the object area image by combining first brightness information corresponding to the object area image to obtain a first brightening image; meanwhile, carrying out brightening treatment on the background image in the image to be processed by combining second brightness information corresponding to the background image in the image to be processed to obtain a second brightening image; then, the first brightening image can be input into an object repairing network for object repairing processing, and an object repairing image containing brightness information is obtained; successively carrying out high-frequency and low-frequency noise reduction processing on the second brightening image by combining an image noise reduction network and a traditional preset frequency domain noise reduction algorithm to obtain a first noise reduction image; then, combining the object repairing image and the first noise reduction image to generate a target image in an original format; and then, sending the target image in the original format to an ISP unit to generate a final imaging image.

Fig. 7 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. Referring to fig. 7, the apparatus includes:

an image obtaining module 710 configured to perform obtaining an object area image corresponding to a target object in an original format image to be processed;

a first brightening processing module 720, configured to perform brightening processing on the object region image based on the first luminance information corresponding to the object region image, to obtain a first brightening image;

the second brightening processing module 730 is configured to perform brightening processing on the background image in the image to be processed based on second brightness information corresponding to the background image in the image to be processed, so as to obtain a second brightening image;

the object repairing processing module 740 is configured to perform object repairing processing on the first highlight image to obtain an object repaired image, wherein the definition of the object repaired image is greater than that of the first highlight image;

a first denoising module 750 configured to perform denoising processing on the second highlighted image, resulting in a first denoised image;

and a target image generating module 760 configured to generate a target image corresponding to the image to be processed based on the object restored image and the first noise-reduced image.

In an alternative embodiment, the first brightening process module 720 includes:

a first encoded image acquisition unit configured to perform acquisition of a first encoded image corresponding to the target area image, the first encoded image being an image containing luminance information;

and the first brightening processing unit is configured to carry out brightening processing on the first coded image based on the first brightness information to obtain a first brightening image.

and the coded image determining unit is configured to execute the brightness channel image in the second area coded image as the first coded image.

In an alternative embodiment, the first luminance information includes a first target average luminance for controlling image quality of the object region image in a preset dimension; the first brightening processing unit includes:

a first average brightness acquiring unit configured to perform acquiring a first target average brightness and a first initial average brightness corresponding to a first encoded image;

and the first brightness updating unit is configured to perform brightness updating on the first coding image based on the first mapping information to obtain a first brightening image.

In an alternative embodiment, the second brightening process module 730 includes:

a second coded image acquisition unit configured to perform acquisition of a second coded image corresponding to the image to be processed, the second coded image being an image containing luminance information;

and the second brightening processing unit is configured to carry out brightening processing on the background coded image in the second coded image based on the second brightness information to obtain a second brightening image.

In an optional embodiment, the second highlight image is an image in an original format, the first brightness information includes a first target average brightness, and the first target average brightness is used for controlling the image quality of the object region image in a preset dimension; the second brightness information comprises a second target average brightness, and the second target average brightness is used for controlling the image quality of the background image in a preset dimension; the second brightening processing unit includes:

a second average brightness acquiring unit configured to perform acquiring a first target average brightness and a second initial average brightness corresponding to the background encoded image;

an average luminance determination unit configured to perform determination of a second target average luminance based on the first target average luminance;

the second brightness updating unit is configured to perform brightness updating on the background coding image in the second coding image based on the second mapping information to obtain a brightness updating image;

and the first format reduction processing unit is configured to perform format reduction processing on the brightness updating image to obtain a second brightening image.

In an alternative embodiment, the first denoising processing module 750 includes:

the second format reduction processing unit is configured to perform format reduction processing on the second noise reduction image to obtain a third noise reduction image in an original format;

and the second noise reduction processing unit is configured to execute high-frequency noise reduction processing on the third noise reduction image based on a preset frequency domain noise reduction algorithm to obtain a first noise reduction image.

In an alternative embodiment, the first denoising processing unit includes:

the third noise reduction processing unit is configured to execute low-frequency noise reduction processing of a downsampled image input image noise reduction network to obtain a fourth noise reduction image;

the up-sampling processing unit is configured to perform up-sampling processing on the down-sampled image and the fourth noise-reduced image respectively to obtain a first up-sampled image corresponding to the down-sampled image and a second up-sampled image corresponding to the fourth noise-reduced image;

a high-frequency object image determination unit configured to perform determining a high-frequency object image from the first up-sampled image and the target color channel image;

a second noise-reduced image determination unit configured to perform determination of a second noise-reduced image based on the high-frequency object image and the second up-sampled image.

In an alternative embodiment, the first highlighted image is an image containing luminance information, and the object repairing process module 740 includes:

the preset image acquisition unit is configured to acquire a preset image corresponding to the target object, and the definition of the preset image is greater than that of the first brightening image;

the object posture transformation analysis unit is configured to perform object posture transformation analysis on the first brightening image and the preset image to obtain object posture transformation information between the first brightening image and the preset image;

and the object repairing processing unit is configured to execute object repairing processing on the first brightening image based on the object posture transformation information and the preset image to obtain an object repairing image.

In an alternative embodiment, image acquisition module 710 includes:

and an object region image determination unit configured to perform determination of an object region image from the fifth noise-reduced image based on the key point information.

In an alternative embodiment, the image-to-be-processed acquiring unit includes:

an exposure image acquisition unit configured to execute acquisition of a first exposure image in an original format, a second exposure image in the original format, and a third exposure image in the original format in response to an image capture instruction for a target object; the exposure time of the first exposure image is longer than that of the second exposure image, and the exposure time of the second exposure image is longer than that of the third exposure image;

the object alignment processing unit is configured to perform object alignment processing on the first exposure image, the second exposure image and the third exposure image to obtain a first alignment image corresponding to the first exposure image, a second alignment image corresponding to the second exposure image and a third alignment image corresponding to the third exposure image;

a to-be-processed image generation unit configured to perform generation of an image to be processed based on the first alignment image, the second alignment image, and the third alignment image.

In an alternative embodiment, the object restoration image is an image containing luminance information, the first noise reduction image is an image in an original format, and the target image generation module 760 includes:

an object region image updating unit configured to update a region image corresponding to the target object in the third encoded image based on the object restored image, resulting in a fourth encoded image;

and the second format reduction processing unit is configured to execute format reduction processing on the fourth coded image to obtain a target image.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 8 is a block diagram illustrating an electronic device for image processing, which may be a terminal, according to an exemplary embodiment, and an internal structure thereof may be as shown in fig. 8. The electronic device comprises a processor, a memory, a network interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a method of image processing. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the electronic equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and does not constitute a limitation on the electronic devices to which the disclosed aspects apply, as a particular electronic device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.

In an exemplary embodiment, there is also provided an electronic device including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the image processing method as in the embodiments of the present disclosure.

In an exemplary embodiment, there is also provided a computer-readable storage medium in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform an image processing method in an embodiment of the present disclosure.

In an exemplary embodiment, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the image processing method in the embodiments of the present disclosure.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, comprising:

2. The image processing method according to claim 1, wherein the performing a brightening process on the target area image based on the first luminance information corresponding to the target area image to obtain a first brightening image comprises:

3. The image processing method according to claim 2, wherein the acquiring the first encoded image corresponding to the object region image comprises:

carrying out format conversion processing on the object region image to obtain a first region coding image in a first color coding format;

4. The image processing method according to claim 2, wherein the first luminance information includes a first target average luminance for controlling image quality of the object region image in a preset dimension; performing, by the first encoding apparatus, a first luminance information on the first encoded image, and obtaining the first luminance information includes:

5. The image processing method according to claim 1, wherein the brightening the background image in the image to be processed based on the second luminance information corresponding to the background image in the image to be processed to obtain a second brightening image comprises:

6. The image processing method according to claim 5, wherein the second highlight image is an image in the original format, the first brightness information includes a first target average brightness, and the first target average brightness is used for controlling image quality of the object region image in a preset dimension; the second brightness information comprises a second target average brightness, and the second target average brightness is used for controlling the image quality of the background image in the preset dimension; performing, on the basis of the second luminance information, a brightening process on a background encoded image in the second encoded image, to obtain the second brightening image, includes:

acquiring the first target average brightness and a second initial average brightness corresponding to the background coded image;

7. The image processing method according to claim 1, wherein the performing noise reduction processing on the second highlight image to obtain a first noise-reduced image comprises:

8. The image processing method according to claim 7, wherein the performing low-frequency noise reduction processing on the target color channel image based on the image noise reduction network to obtain a second noise-reduced image comprises:

determining the second noise-reduced image based on the high-frequency object image and the second up-sampled image.

9. The image processing method according to any one of claims 1 to 8, wherein the first highlight image is an image including luminance information, and performing the object restoration process on the first highlight image to obtain the object restoration image includes:

10. The image processing method according to any one of claims 1 to 8, wherein the object region image is acquired by:

11. The image processing method according to any one of claims 1 to 8, wherein the acquiring the image to be processed includes:

12. The image processing method according to any one of claims 1 to 8, wherein the object restoration image is an image including luminance information, the first noise-reduced image is an image in the original format, and the generating the target image corresponding to the image to be processed based on the object restoration image and the first noise-reduced image includes:

updating a region image corresponding to the target object in the third coded image based on the object restoration image to obtain a fourth coded image;

13. An image processing apparatus characterized by comprising:

the second brightening processing module is configured to execute brightening processing on the background image in the image to be processed based on second brightness information corresponding to the background image in the image to be processed to obtain a second brightening image;

14. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the image processing method of any one of claims 1 to 12.

15. A computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable an image processing device to perform the image processing method of any one of claims 1 to 12.