CN116167924A

CN116167924A - Image processing method, device and storage medium

Info

Publication number: CN116167924A
Application number: CN202111394200.6A
Authority: CN
Inventors: 冯超禹; 杨越麒; 张玉倩; 宋小鸿; 雷磊
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2021-11-23
Filing date: 2021-11-23
Publication date: 2023-05-26

Abstract

The disclosure relates to an image processing method, an image processing device and a storage medium. The method comprises the following steps: acquiring a first image to be processed and first image information of the first image; wherein the first image information includes: image area information divided according to image content and/or image brightness distribution information determined according to image brightness; denoising different image areas of the first image with different intensities based on the image area information by using a target neural network; and/or, performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information; and determining a second image according to the output image of the target neural network.

Description

Image processing method, device and storage medium

Technical Field

The disclosure relates to the technical field of terminals, and in particular relates to an image processing method, an image processing device and a storage medium.

Background

With the development of the terminal technical field, the configuration of the terminal equipment is higher and higher, and the shooting function becomes a necessary function of the terminal equipment. The requirements for shooting by users using terminal devices are increasing, and the requirements for the quality of images obtained by shooting by the terminal devices are also increasing.

When the terminal equipment is used for image shooting, complicated ambient illumination and shooting sensor noise often cause insufficient dynamic range of the shot image and obvious dark noise, and finally influence the image quality effect. In the related art, aiming at the problem of insufficient dynamic range of an image, a method for surrounding exposure is generally collected, and a high dynamic range (High Dynamic Range, HDR) image is obtained by collecting and fusing multiple frames of images with different exposure degrees; however, this method is prone to ghosting, color cast or unnatural transitions.

Aiming at the problem of obvious noise at the dark part of an image, a single-frame image space domain denoising or multi-frame image time domain denoising method is generally adopted to denoise the image, but the single-frame image space domain denoising method cannot give consideration to the detail information and the image noise of the image, and the multi-frame image time domain denoising method is easy to generate ghost, deformation or blurring conditions, seriously affects the quality of the image and reduces the use experience of a user.

Disclosure of Invention

The present disclosure provides an image processing method, apparatus, and storage medium.

According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method including:

acquiring a first image to be processed and first image information of the first image; wherein the first image information includes: image area information divided according to image content and/or image brightness distribution information determined according to image brightness;

Denoising different image areas of the first image with different intensities based on the image area information by using a target neural network; and/or, performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information;

and determining a second image according to the output image of the target neural network.

Optionally, the determining a second image according to the output image of the target neural network includes:

if the first image information comprises the image area information, determining a denoising image output by the target neural network as the second image;

or alternatively, the process may be performed,

if the first image information comprises the image brightness distribution information, determining a brightness restoration image output by the target neural network as the second image;

or alternatively, the process may be performed,

and if the first image information comprises the image area information and the image brightness distribution information, performing image fusion on the denoising image and the brightness restoration image output by the target neural network to obtain the second image.

Optionally, the acquiring the first image to be processed and the first image information of the first image includes:

Acquiring a first image to be processed, and performing image segmentation processing on the first image to obtain region distribution information of the first image; the region distribution information is used for indicating the position distribution of a plurality of different types of image regions in the first image;

performing edge detection processing on the first image to obtain edge distribution information of the first image; the edge distribution information is used for indicating the position distribution of a plurality of first subareas and second subareas in the first image, and the pixel change intensity of the first subareas is larger than that of the second subareas;

determining the image region information according to the region distribution information and the edge distribution information; the image area information is used for indicating the position distribution of the subareas contained in one image area.

Optionally, the denoising processing of different image areas of the first image with different intensities based on the image area information by using a target neural network includes:

and carrying out denoising processing with different intensities on the image areas of different categories in the first image and the first subarea and the second subarea contained in the image areas of different categories based on the image area information by using the first subnetwork of the target neural network.

Optionally, the denoising processing of the image area of the different category and the first sub-area and the second sub-area included in the image area of the different category in the first image based on the image area information by using the first sub-network includes:

extracting, by a first extraction module of the first sub-network, the first sub-region and the second sub-region included in a plurality of different categories of image regions from the first image based on the image region information;

performing first noise reduction processing on the first sub-areas in the image areas of the different categories by using a first noise reduction module of the first sub-network;

performing second noise reduction processing on the second sub-areas in the image areas of the different categories by using a second noise reduction module of the first sub-network; the noise reduction intensity of the first noise reduction process is smaller than that of the second noise reduction process;

and carrying out image fusion on the plurality of first sub-areas subjected to the first noise reduction treatment and the plurality of second sub-areas subjected to the second noise reduction treatment by utilizing a fusion module of the first sub-network to obtain a denoising image.

Optionally, the first noise reduction module is configured to:

and carrying out texture enhancement processing on the first subarea in the plurality of different types of image areas after the first noise reduction processing.

Optionally, the fusion module is configured to:

and carrying out image fusion on the first subareas subjected to texture enhancement processing and the second subareas subjected to second noise reduction processing to obtain the denoising image.

acquiring a first image to be processed and brightness values of all pixel points in the first image;

determining the image brightness distribution information of the first image based on brightness values of all pixel points in the first image; the image brightness distribution information is at least used for indicating the position distribution of the exposure abnormal area in the first image.

Optionally, the performing, by using the target neural network, a luminance repair process on the exposure abnormal region of the first image based on the image luminance distribution information includes:

performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information by using a second sub-network of the target neural network; the average brightness value of the exposure abnormal region is larger than a first brightness threshold value, or the average brightness value of the exposure abnormal region is smaller than a second brightness threshold value, and the first brightness threshold value is larger than the second brightness threshold value.

Optionally, the performing, by using the second sub-network of the target neural network, luminance repair processing on the exposure abnormal region of the first image based on the image luminance distribution information includes:

determining an exposure abnormal region from the first image based on the image brightness distribution information by using a second extraction module of the second sub-network, and extracting a first semantic feature of the first image and a second semantic feature of the exposure abnormal region;

performing contour restoration on the exposure abnormal region in the first image based on the first semantic features by using a first restoration module of the second subnetwork;

and performing texture restoration on the exposure abnormal region in the first image output by the first restoration module based on the second semantic features by using a second restoration module of the second subnetwork to obtain a brightness restoration image.

According to a second aspect of embodiments of the present disclosure, there is provided an image processing apparatus including:

the acquisition module is used for acquiring a first image to be processed and first image information of the first image; wherein the first image information includes: image area information divided according to image content and/or image brightness distribution information determined according to image brightness;

The processing module is used for carrying out denoising processing with different intensities on different image areas of the first image based on the image area information by utilizing a target neural network; and/or, performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information;

and the determining module is used for determining a second image according to the output image of the target neural network.

Optionally, the determining module is configured to:

or alternatively, the process may be performed,

Optionally, the acquiring module is configured to:

Optionally, the processing module is configured to:

and carrying out denoising processing with different intensities on the image areas of different categories in the first image and the first subarea and the second subarea contained in the image areas of different categories based on the image area information by using a first subarea of the target neural network.

Optionally, the processing module is configured to:

Optionally, the acquiring module is configured to:

Optionally, the processing module is configured to:

According to a third aspect of the embodiments of the present disclosure, there is provided an image processing apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: the executable instructions, when executed, implement the steps in the method according to the first aspect of the embodiments of the present disclosure.

According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer readable storage medium, which when executed by a processor of an image processing apparatus, causes the image processing apparatus to perform the steps of the method according to the first aspect of embodiments of the present disclosure.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

according to the image processing method provided by the embodiment of the disclosure, on one hand, the image area information and/or the image brightness distribution information of the first image to be processed are obtained, the target neural network is utilized to perform denoising processing of different intensities on different image areas in the first image based on the image area information, meanwhile, detail information of the image and denoising effect of the image are taken into consideration, and denoising processing is performed on the image more pertinently; and/or, based on the image brightness distribution information, the brightness restoration is carried out on the exposure abnormal region in the first image, so that not only the lost image information of the exposure abnormal region can be restored, but also the situations of local deformation and unnatural transition can be effectively avoided, and a high-dynamic image can be obtained.

On the other hand, the denoising and/or brightness restoration processing is directly carried out on the first image of the single frame, so that the conditions of ghosting, deformation and the like caused by inconsistent positions of target objects in the images in the multi-frame image fusion process can be reduced, the complexity of image processing is reduced, the image processing efficiency is improved, and the user experience is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a schematic diagram illustrating an image processing method based on bracketing exposure according to an exemplary embodiment.

Fig. 2 is a schematic diagram illustrating a single frame denoising method according to an exemplary embodiment.

Fig. 3 is a schematic diagram illustrating a multi-frame denoising method according to an exemplary embodiment.

Fig. 4 is a flowchart illustrating an image processing method according to an exemplary embodiment.

Fig. 5 is a flowchart diagram two of an image processing method according to an exemplary embodiment.

Fig. 6 is a flow chart illustrating an image denoising method according to an exemplary embodiment.

Fig. 7 is a flowchart three of an image processing method according to an exemplary embodiment.

Fig. 8 is a flow chart illustrating an image brightness restoration method according to an exemplary embodiment.

Fig. 9 is a flowchart three of an image processing method according to an exemplary embodiment.

FIG. 10 is a schematic diagram illustrating a joint training according to an example embodiment.

FIG. 11 is a schematic workflow diagram of an electronic device, according to an example embodiment.

Fig. 12 is a schematic structural view of an image processing apparatus according to an exemplary embodiment.

Fig. 13 is a block diagram of an image processing apparatus according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the invention. Rather, they are merely examples of apparatus and methods consistent with aspects of the invention as detailed in the accompanying claims.

Currently, for the problem of insufficient dynamic range of an image, a bracketing exposure penalty is generally adopted, as shown in fig. 1, and fig. 1 is a schematic diagram illustrating an image processing method based on bracketing exposure according to an exemplary embodiment. The terminal equipment is utilized to continuously shoot a plurality of frames of images with different exposure degrees, the images with different exposure degrees are aligned and fused in brightness, the underexposure images are utilized to supplement the image information of the overexposed region in the normal exposure image, and the overexposed images are utilized to supplement the image information of the underexposed region in the normal exposure image, so that the image with a single frame and a high dynamic range is obtained.

However, in the method, because the information of the images with different exposure degrees is unequal, the images with different exposure degrees are fused, so that on one hand, the difficulty is high, and the self-adaption capability is poor; on the other hand, when the target object is a moving object, the problem of ghosting and color cast easily occur in the fused image; for a scene with gradually changed brightness (such as sky), the fused image is easy to generate the phenomena of unnatural transition and fault.

Aiming at the problem of dark noise of an image, a single-frame denoising method and a multi-frame denoising method are generally adopted; wherein, as shown in fig. 2, fig. 2 is a schematic diagram illustrating a single frame denoising method according to an exemplary embodiment. And performing spatial domain denoising on the single frame image through a filter to obtain a denoised image. However, the single-frame denoising method cannot give consideration to image details and image noise, and when the denoising effect is improved, the phenomenon that the image detail information is lost seriously occurs; when more image detail information is retained, a phenomenon that noise remains too serious occurs.

As shown in fig. 3, fig. 3 is a schematic diagram illustrating a multi-frame denoising method according to an exemplary embodiment. And (3) obtaining images with the same exposure degree by a plurality of frames, and performing time domain denoising on the images by utilizing an image alignment and superposition fusion technology to obtain denoised images. However, the multi-frame denoising method can utilize time domain information, obtain better denoising effect and simultaneously keep image detail information, but has higher requirements on image alignment and superposition fusion technology, and the obtained denoising image is easy to have ghost, deformation and blurring; and when the handheld terminal equipment of the user is unstable, multi-frame denoising is performed based on the acquired multi-frame images, and the denoising effect of the obtained denoising image is even obviously deteriorated.

The embodiment of the disclosure provides an image processing method. Fig. 4 is a flowchart of an image processing method according to an exemplary embodiment, as shown in fig. 4, the method includes the steps of:

step S101, a first image to be processed and first image information of the first image are acquired; wherein the first image information includes: image area information divided according to image content and/or image brightness distribution information determined according to image brightness;

Step S102, denoising processing with different intensities is carried out on different image areas of the first image based on the image area information by utilizing a target neural network; and/or, performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information;

step S103, determining a second image according to the output image of the target neural network.

In the embodiment of the present disclosure, the image processing method may be applied to a mobile terminal having an image capturing device, and the mobile terminal may be: smart phones, tablet computers, or wearable electronic devices, etc.

The image acquisition device generally refers to a device capable of completing a photographing function in a mobile terminal, and comprises a camera, a processing module and a storage module, wherein the processing module and the storage module are required to complete the acquisition and the transmission of images, and the device can also comprise some processing function modules.

The image capturing device may be a camera or video camera or the like.

In step S101, the first image may be an unprocessed single frame image; it can be generally understood as an original image containing noise, such as a single frame RAW image.

The RAW is in an uncompressed format, and can be conceptualized as "original image encoded data".

In some embodiments, the first image may also be a single frame RGB format image or a single frame YUV format image.

Here, the first image may be an image obtained by shooting by the mobile terminal, or the first image may also be an image obtained by transmission by other devices; alternatively, the first image may be an image downloaded from a cloud server through a network, and the source of the first image is not limited in this disclosure.

First image information for a first image may be acquired based on the first image. Here, the first image information includes: image area information divided according to image content and/or image brightness distribution information determined according to image brightness.

The image region information can be used for describing the position distribution of different regions in the first image, the image contents of different image regions are different, and the denoising requirements corresponding to different image regions are different. The image brightness distribution information may be used to describe a position distribution of different brightness areas within the first image.

The method comprises the steps that semantic features of a first image are obtained, and the first image is classified according to the semantic features to obtain a plurality of image areas with different categories; and determining image area information of the first image according to the position distribution of the plurality of different types of image areas in the first image. Here, the image area information of the first image is used for the target neural network to perform denoising processing with different intensities on a plurality of different types of image areas in the first image based on the image area information.

The brightness information of each pixel point in the first image can be counted to obtain the image brightness distribution information of the first image; here, the image brightness distribution information is used for the target neural network to perform brightness restoration processing on the exposure abnormal region in the first image based on the image brightness distribution information.

In step S102, the first image and the image area information of the first image may be input into a target neural network, and a plurality of different types of image areas are determined from the first image based on the image area information by using the target neural network, and denoising processing with different intensities is performed on the plurality of different types of image areas in a targeted manner based on denoising requirements of the plurality of different types of image areas. At this time, the target neural network may be used to denoise the first image.

It should be noted that, the image captured by the mobile terminal will generally have noise, where the noise includes noise generated during the photoelectric conversion process in the image sensor, and the darker the environment, the greater the noise of the image. This is because the darker the environment, the smaller the amount of light entering, resulting in a smaller signal-to-noise ratio of the image.

For a common photographing device, noise in a photographed image at night is obviously larger than that in a photographed image at daytime, and although some noise can be reduced by prolonging exposure time in a scene photographed at night, the extension of the exposure time has a certain limit, and too long exposure time can reduce the frame rate and can cause overexposure of a highlight area in the image. Based on this, in order to effectively suppress noise in an image, the image needs to be subjected to denoising processing.

However, considering that the importance degree of the image information contained in different image areas in the image is different, if denoising processing with the same intensity is adopted for a plurality of image areas in the image, although noise in the image can be effectively suppressed, detail information of the image is also seriously lost, so in the embodiment of the disclosure, the target neural network is utilized, denoising processing is carried out on the first image based on the image area information of the first image, dynamic adjustment of denoising intensity of different image areas is realized, and meanwhile, detail information of the image and denoising effect of the image are simultaneously considered.

Alternatively, the first image and the image brightness distribution information of the first image may be input into a target neural network, and the abnormal exposure area may be determined from the first image based on the image brightness distribution information by using the target neural network, and the abnormal exposure area may be subjected to brightness restoration processing. At this time, the target neural network may be used to perform luminance restoration processing on the first image.

In order to increase the dynamic range of the image, it is generally necessary to continuously capture multiple frames of images with different exposure degrees, and fuse the multiple frames of images with different exposure degrees to obtain an image with a high dynamic range.

Based on the above, according to the embodiment of the disclosure, the brightness restoration processing is performed on the exposure abnormal region in the acquired single frame image, so that the restored image not only can contain a plurality of image regions with different exposure degrees, but also has a high dynamic range; the problems of ghosting, color cast and the like caused by image fusion can be reduced, the complexity of image processing is reduced, and the image processing efficiency is improved.

Alternatively, the first image, the image region information of the first image, and the image luminance distribution information may be input into the target neural network, and the denoising process and the luminance repair process may be performed on the first image in parallel based on the image region information and the image luminance distribution information using the target neural network. At this time, the target neural network may be used to perform denoising processing and luminance restoration processing on the first image.

Thus, according to the embodiment of the disclosure, through parallel execution of the strategy, the denoising process and the brightness restoration process are simultaneously performed on the first image of the single frame, so that the image processing time can be greatly shortened, and the image processing efficiency can be improved.

In step S103, a second image may be determined from at least one image output by the target neural network.

According to the embodiment of the disclosure, on one hand, through directly carrying out denoising processing and/or brightness restoration processing on a single frame image, the conditions of ghosting, deformation and the like caused by inconsistent positions of target objects in the image in the multi-frame image fusion process can be reduced, on the other hand, different image areas in the single frame image are subjected to denoising processing with different intensities by utilizing a target neural network based on image area information, and meanwhile, detail information of the image and denoising effect of the image are taken into consideration, so that the denoising processing is carried out on the image more specifically; and/or, based on the image brightness distribution information, the brightness restoration is carried out on the exposure abnormal region in the single frame image, so that not only the lost image information of the exposure abnormal region can be restored, but also the situations of local deformation and unnatural transition can be effectively avoided, and the high-dynamic image can be obtained.

or alternatively, the process may be performed,

In an embodiment of the present disclosure, if the first image information includes only image area information; at this time, the target neural network performs image denoising processing on only the first image.

Inputting a first image and image area information of the first image into a target neural network, determining different image areas from the first image by the target neural network based on the image area information, and denoising the different image areas with different intensities to obtain a denoised image; the denoised image may be determined directly as the final output second image.

If the first image information only comprises the image brightness distribution information; at this time, the target neural network performs the luminance repair process only on the first image.

Inputting a first image and image brightness distribution information of the first image into a target neural network, determining an exposure abnormal region from the first image by the target neural network based on the image brightness distribution information, and carrying out brightness restoration on the exposure abnormal region to obtain a brightness restoration image; the luminance repair image may be directly determined as the finally output second image.

If the first image information includes image area information and image brightness distribution information, the target neural network may perform image denoising processing and brightness restoration processing on the first image in parallel.

The first image, the image area information and the image brightness distribution information of the first image are input into a target neural network, and the target neural network respectively performs image denoising processing and brightness restoration processing on the first image based on the image area information and the image brightness distribution information to obtain and output a denoising image and a brightness restoration image.

And the denoising image and the brightness restoration image can be weighted and fused according to the weights corresponding to the denoising image and the brightness restoration image, so that a finally output second image is obtained.

Here, the weights corresponding to the denoising image and the luminance restoration image may be set according to actual demands. For example, when the image acquisition environment is darker and the noise of the dark portion in the acquired first image is more obvious, the weight corresponding to the denoising image may be greater than the weight corresponding to the luminance repairing image.

For example, in the case that the image acquisition environment is a high light ratio scene with obvious contrast, the weight corresponding to the denoising image can be smaller than the weight corresponding to the brightness restoration image.

The pixel value of each pixel point in the denoising image and the pixel value of each pixel point in the brightness restoration image can be obtained; and carrying out weighted fusion on the pixel values of all the pixel points in the denoising image and the brightness restoration image based on the weight values corresponding to the denoising image and the brightness restoration image to obtain the pixel values of all the pixel points in the second image.

In some embodiments, a first image feature of the denoised image and a second image feature of the luminance restoration image may be obtained; based on the weights of the denoising image and the brightness restoration image, carrying out weighted fusion on the first image characteristic of the denoising image and the second image characteristic of the brightness restoration image to obtain a fused image characteristic; and carrying out image reconstruction based on the first image feature, the second image feature and the fusion image feature to obtain a second image.

Here, image fusion between the denoising image and the luminance repair image may be accomplished using an image fusion network.

In the embodiment of the disclosure, a plurality of image areas of different categories in a first image can be determined by performing image segmentation processing on the first image; and determining the region distribution information of the first image according to the position distribution of the image regions of a plurality of different categories in the first image. Here, the categories of the plurality of image areas within the first image may be determined based on the image subjects within the image areas; such as face areas, sky areas, building areas, etc. The categories of a plurality of image areas in the first image can be determined based on the distance between the image main body and the image acquisition device in the image area; for example, a foreground region and a background region.

There may be various ways of performing the image segmentation processing on the first image. In some embodiments, the image segmentation process may be performed based on depth information of each pixel point in the first image; generally, if the depth information indicates that the image subject of a certain image area is closer to the plane where the image acquisition device is located, the image area can be determined as a foreground area when the depth value is smaller; if the depth information indicates that the image main body of a certain image area is far away from the plane where the image acquisition device is located, and the depth value is large, the image area can be determined as a background area.

In other embodiments, the first image may be divided into a plurality of image areas based on the depth information and the color information of each pixel point in the first image, and the image subjects in the plurality of image areas may be identified based on the image identification network, so as to obtain the category to which the image subjects in the plurality of image areas belong, and the plurality of image areas in the same category may be combined, so as to obtain a plurality of image areas in different categories.

It will be appreciated that the intensity of the denoising process corresponding to the different classes of image regions within the first image may be different; for example, the intensity of the denoising process of the foreground region is smaller than that of the background region; alternatively, the intensity of the denoising process of the face region is smaller than that of the sky region.

In an embodiment of the disclosure, the first sub-region may be a detail region within the image and the second sub-region may be a flat region within the image. It will be appreciated that within the image, the intensity of the pixel variation between the individual pixels within the detail region is greater than the intensity of the pixel variation between the individual pixels within the flat region.

The plurality of detail areas and the plurality of flat areas in the first image may be determined by performing an edge detection process on the first image.

Here, edge detection refers to a detection method for extracting texture features in an image, and the purpose of edge detection is to determine pixel points with obvious brightness changes in the image. The edge detection may be performed on the first image, for example, by a method of edge-preserving filtering in combination with an edge detection operator (such as Sobel operator, kirsch operator or Canny operator, etc.). Of course, the edge detection may be performed on the first image by other edge detection methods, which is not limited in this disclosure.

The first sub-region and the second sub-region contained in the image regions of different categories can be determined according to the position distribution of the image regions of different categories indicated by the region distribution information and the position distribution of the first sub-region (namely the detail region) and the second sub-region (namely the flat region) indicated by the edge distribution information, so that the image region information of the first image is obtained.

It will be appreciated that since the respective image areas can be further divided into a detail area and a flat area; for example, for a face region, the detail region may be a sub-region of the human eye within the face region, and the flat region may be a sub-region of the cheek within the face region; for sky regions, the detail region may be a cloudy subregion within a sky region and the flat region may be a blue sky subregion within the sky region. The denoising intensities corresponding to the first and second sub-regions within each image region may also be different.

Therefore, when the image area information of the first image is determined so that the target neural network is used for denoising the first image later, different intensities of denoising processing can be performed on different areas in the first image according to different types of image areas indicated by the image area information, and the detail areas and the flat areas in the image areas.

In the embodiment of the disclosure, the target neural network includes two sub-network modules for performing denoising processing and luminance repairing processing, respectively.

The first sub-network of the target neural network determines different types of image areas in the first image and first sub-areas (namely detail areas) and second sub-areas (namely flat areas) contained in the different types of image areas according to the image area information; and denoising the first subarea and the second subarea in the image areas of different categories with different intensities.

The denoising intensity of the denoising process for the image region can be determined according to the importance degree of the image information contained in the image regions of different categories. For example, the first image includes a foreground region and a background region, and since the importance of the image information included in the foreground region is greater than that included in the background region, the denoising intensity of the denoising process of the foreground region is smaller than that of the background region.

In an embodiment of the disclosure, the first subnetwork includes: the device comprises a first extraction module, a first noise reduction module, a second noise reduction module and a fusion module.

The first image and the image area information of the first image are input into the first sub-network, and the first extraction module is utilized to extract the first sub-area and the second sub-area contained in the image areas of a plurality of different categories from the first image according to the image areas of different categories indicated by the image area information and the position distribution of the first sub-area and the second sub-area of the image area.

And the first noise reduction module performs first noise reduction processing on a first sub-region in the image regions of the different categories output by the first extraction module.

Here, the noise reduction intensity of the first noise reduction process of the first sub-region within the plurality of different categories of image regions is the same; alternatively, the noise reduction intensities of the first noise reduction processes of the first sub-regions within the plurality of different categories of image regions are different.

It should be noted that the noise reduction intensity of the first noise reduction process of the first sub-region in the plurality of different types of image regions may be determined according to the importance degree of the detail information contained in the image region to which the first sub-region belongs.

For example, the noise reduction intensity of the first noise reduction process of the first sub-region of the face region in the first image is smaller than the noise reduction intensity of the first noise reduction process of the first sub-region of the sky region in the first image.

For another example, the noise reduction intensity of the first noise reduction process of the first sub-region of the foreground region in the first image is smaller than the noise reduction intensity of the first noise reduction process of the first sub-region of the background region in the first image.

And the second noise reduction module performs second noise reduction processing on a second sub-region in the plurality of different types of image regions output by the first extraction module.

Here, the noise reduction intensity of the second noise reduction process of the second sub-region within the plurality of different-category image regions is the same; or, the noise reduction intensities of the second noise reduction processes of the second sub-regions in the plurality of different category image regions are different; the noise reduction intensity of the second noise reduction process of each of the image areas is greater than the noise reduction intensity of the first noise reduction process of each of the image areas.

The noise reduction intensity of the second noise reduction process of the second sub-region in the plurality of different types of image regions may be determined according to the importance degree of the image information included in the image region to which the second sub-region belongs.

For example, the noise reduction intensity of the second noise reduction process of the second sub-region of the face region in the first image is smaller than the noise reduction intensity of the second noise reduction process of the second sub-region of the sky region in the first image.

For another example, the noise reduction intensity of the second noise reduction process of the second sub-region of the foreground region in the first image is smaller than the noise reduction intensity of the second noise reduction process of the second sub-region of the background region in the first image.

And carrying out image fusion on the plurality of first sub-areas subjected to the first noise reduction output by the first noise reduction module and the plurality of second sub-areas subjected to the second noise reduction output by the second noise reduction module according to the position distribution of the plurality of first sub-areas and the plurality of second sub-areas by utilizing a fusion module, so as to obtain a denoising image.

It can be understood that, because the first sub-region is a detail region in the image region, and the second sub-region is a flat region in the image region, in order to obtain better denoising effect and retain more image detail information, the first extraction module determines a plurality of first sub-regions and a plurality of second sub-regions from the first image based on the image region information, and the first denoising module and the second denoising module respectively adopt denoising processes with different intensities for the first sub-region (namely the detail region) and the second sub-region (namely the flat region) in the image region, thereby realizing dynamic adjustment of denoising effects of different regions and simultaneously considering image details and image denoising effects.

Optionally, the first noise reduction module is configured to:

In an embodiment of the present disclosure, in order to reduce an influence of the first noise reduction process on detailed information of a plurality of first sub-regions in the first image, the first noise reduction module may perform a texture enhancement process on the plurality of sub-regions after performing the first noise reduction process on the first sub-regions in the plurality of different types of image regions.

The first sub-region may be subjected to texture enhancement processing in a plurality of ways. In some embodiments, the texture features of the plurality of first sub-regions may be obtained by fusing the texture features with the plurality of first sub-regions after the first noise reduction process to obtain a plurality of first sub-regions after the texture enhancement process.

In other embodiments, the texture enhancement processing may be performed on the plurality of first sub-regions based on a preset operator (such as a second order differential operator, such as a laplace texture enhancement operator), to obtain a plurality of first sub-regions after the texture enhancement processing.

It should be noted that, in the process of performing the first noise reduction processing on the plurality of first sub-regions, original texture details in the first sub-regions may also be removed, which may cause texture blurring on visual effects; based on the above, in the embodiment of the present disclosure, by performing texture enhancement processing on the first sub-region after the first noise reduction processing, more edge detail information may be retained in the first sub-region obtained through the texture enhancement processing, and the texture detail of the first sub-region is clearer and more obvious; the image details of the first sub-region are not blurred due to the first noise reduction processing while the noise reduction effect of the first noise reduction processing on the first sub-region is ensured.

Optionally, the fusion module is configured to:

In the embodiment of the disclosure, a fusion module is utilized to fuse images of a plurality of first subregions subjected to texture enhancement processing and a plurality of second subregions subjected to second noise reduction processing and output by a first noise reduction module according to the position distribution of the plurality of first subregions and the plurality of second subregions, so as to obtain a noise-removed image.

In the embodiment of the disclosure, the brightness value of each pixel point in the first image can be obtained, and the brightness distribution of the first image is determined based on the brightness value of each pixel point; determining a plurality of different brightness areas from a first image according to the brightness distribution of the first image; and determining image brightness distribution information describing a position distribution of a plurality of different brightness regions within the first image.

It should be noted that, the brightness value of the pixel is between 0 and 255, the brightness of the pixel with the brightness value close to 255 is higher, the brightness of the pixel with the brightness value close to 0 is lower, and the rest part belongs to the middle tone. The luminance values of each pixel point in the first image may be counted and the distribution of the luminance values of the pixels of the first image may be determined based on the position distribution of each pixel point.

In some embodiments, the exposure anomaly region may be determined from a plurality of different Liu Angdu regions based on an average luminance value of the plurality of different luminance regions. Here, the exposure abnormality region includes: underexposed areas and/or overexposed areas.

The first image and the image brightness distribution information of the first image can be input into a target neural network, a second sub-network of the target neural network determines an exposure abnormal region in the first image according to the image brightness distribution information, and brightness restoration is carried out on the exposure abnormal region, so that a brightness restoration image with a high dynamic range is obtained.

Here, performing luminance repair on the exposure abnormal region may include: luminance repair is performed on underexposed areas within the first image (i.e., areas having an average luminance value less than the second luminance threshold) and/or luminance repair is performed on overexposed areas within the first image (i.e., areas having an average luminance value greater than the first luminance threshold).

In an embodiment of the disclosure, the second subnetwork includes: the device comprises a second extraction module, a first repair module and a second repair module.

The first image and the image brightness distribution information of the first image are input into a second sub-network, and the second extraction module is utilized to determine an exposure abnormal region from the first image according to the image brightness distribution information; extracting features of the first image to obtain first semantic features of the first image; and extracting features of the edge pixel points of the exposure abnormal region to obtain second semantic features of the exposure abnormal region.

It should be noted that the first semantic feature may be a global semantic feature of the first image, and is used to describe information such as an outline and an area of the first image. The second semantic feature may be a local semantic feature of an exposure anomaly region, and is used for describing texture, color, and other information of the exposure anomaly region.

And the first restoration module performs contour restoration on the exposure abnormal region in the first image according to the first semantic features output by the second extraction module.

And the second restoration module carries out texture restoration on the exposure abnormal region output by the first restoration module according to the second semantic features output by the second extraction module to obtain a brightness restoration image.

In the embodiment of the disclosure, the first semantic features of the first image and the second semantic features of the exposure abnormal region are acquired through the second sub-network, and the exposure abnormal region in the first image is repaired by contours and textures based on the first semantic features and the second semantic features which are complementary in information, so that not only can the lost image information of the exposure abnormal region be repaired, but also the local deformation and the unnatural transition condition can be effectively reduced, the image subjected to the brightness repair processing not only contains the image information of different dynamic ranges, but also can the natural transition between the image information of a plurality of different dynamic ranges.

In some embodiments, the area of abnormal exposure may include: overexposed and/or underexposed areas; and after the second restoration module carries out texture restoration on the exposure abnormal region, carrying out tone mapping processing on the image after the texture restoration.

It will be appreciated that for the overexposed region, the brightness of the overexposed region within the first image is suitably reduced by the tone mapping process; for the underexposed region, the brightness of the underexposed region in the first image is appropriately increased by tone mapping processing. Tone mapping is a process of performing a large contrast attenuation to change the brightness of a scene to a displayable range, and at the same time, maintaining information important for representing the original scene, such as image detail information and color.

In the embodiment of the disclosure, a preset tone mapping operator corresponding to the shooting scene may be determined according to an actual shooting scene of the first image, and tone mapping processing may be performed on an exposure abnormal region in the first image after texture repair based on the preset tone mapping operator, so as to obtain a brightness repair image. In this way, the first image after contour and texture restoration is subjected to tone mapping processing, and brightness fine adjustment is performed on the exposure abnormal region in the first image, so that the situations of brightness inversion and unnatural transition of image information with a plurality of different dynamic ranges are reduced.

The present disclosure also provides the following embodiments:

fig. 5 is a flowchart two of an image processing method according to an exemplary embodiment, as shown in fig. 5, the method includes:

Step S201, a first image to be processed is obtained, and image segmentation processing is carried out on the first image to obtain region distribution information of the first image;

in this example, the first image is a single frame RAW image; the region distribution information is used to indicate a position distribution of a plurality of different categories of image regions within the first image. For example, the first image includes a face and a sky background; and carrying out image segmentation processing on the first image to obtain region distribution information of the first image, wherein the region distribution information is used for indicating the position distribution of the face region and the sky region in the first image.

It will be appreciated that different classes of image regions may employ different intensities of denoising processes; for example, the intensity of the denoising process of the face region in the first image is smaller than the intensity of the denoising process of the sky region.

Step S202, performing edge detection processing on the first image to obtain edge distribution information of the first image; the edge distribution information is used for indicating the position distribution of a plurality of first subareas and second subareas in the first image;

here, the pixel variation intensity of the first sub-region is greater than the pixel variation intensity of the second sub-region. In this example, the first sub-region may be a detail region within the first image and the second sub-region may be a flat region within the first image.

It will be appreciated that the intensity of the denoising process corresponding to the different sub-regions within the first image may be different. For example, the intensity of the denoising process for the detail region within the first image is smaller than the intensity of the denoising process for the flat region.

Step S203, determining the image area information according to the area distribution information and the edge distribution information; the image area information is used for indicating the position distribution of the subareas contained in one image area;

in this example, the position distribution (i.e., image area information) of the detail area and the flat area contained in each of the image areas in the first image is determined based on the position distribution of the image areas of different categories in the first image indicated by the area distribution information and the position distribution of the plurality of detail areas and the plurality of flat areas in the first image indicated by the edge distribution information.

It should be noted that, each image area may include a detail area and a flat area; for example, the sky area within the first image includes a white cloud area (i.e., a detail area) and a blue sky area (i.e., a flat area); the face region in the first image includes a human eye region (i.e., a detail region) and a cheek region (i.e., a flat region).

Step S204, using a first neural network, based on the image region information, denoising processing with different intensities is performed on different types of image regions in the first image and the first sub-region and the second sub-region contained in the different types of image regions;

in this example, the first neural network may be a region-based image noise reduction network; the first image and the image area information of the first image are input into the first neural network, dynamic adjustment of denoising effects of different areas in the first image is achieved through the first neural network, and meanwhile image details and image noise are considered.

In some embodiments, the denoising processing of the image region of the different category and the first sub-region and the second sub-region included in the image region of the different category in the first image with the first neural network based on the image region information includes:

extracting, by a first extraction module of the first neural network, the first sub-region and the second sub-region included in a plurality of different categories of image regions from the first image based on the image region information;

Performing first noise reduction processing on the first subareas in the plurality of different types of image areas by using a first noise reduction module of the first neural network, and performing texture enhancement processing on the first subareas in the plurality of different types of image areas after the first noise reduction processing;

performing second noise reduction processing on the second sub-regions in the image regions of the different categories by using a second noise reduction module of the first neural network; the noise reduction intensity of the first noise reduction process is smaller than that of the second noise reduction process;

and carrying out image fusion on the plurality of first subareas subjected to texture enhancement processing and the plurality of second subareas subjected to second noise reduction processing by utilizing a fusion module of the first neural network to obtain the denoising image.

In the example, the first noise reduction module and the second noise reduction module are utilized to carry out noise reduction processing with different intensities on detail areas and flat areas in different types of image areas, and the first noise reduction module is utilized to carry out texture enhancement processing on the detail areas after the first noise reduction processing, so that the noise reduction effect is ensured, and the detail areas in the image cannot be blurred; and carrying out high-intensity second noise reduction processing on the flat area in the image, and improving the processing efficiency.

In step S205, the denoised image output by the first neural network is determined as the second image.

In this example, the first neural network is utilized to denoise different areas of the first image to different degrees based on the image area information, balance between image details and noise is considered, and more image detail information is reserved while denoising.

Illustratively, as shown in fig. 6, fig. 6 is a flow chart illustrating an image denoising method according to an exemplary embodiment.

Fig. 7 is a flowchart three of an image processing method according to an exemplary embodiment, as shown in fig. 7, the method includes:

step S301, obtaining a first image to be processed and brightness values of all pixel points in the first image; determining the image brightness distribution information of the first image based on brightness values of all pixel points in the first image; the image brightness distribution information is at least used for indicating the position distribution of the exposure abnormal area in the first image;

in this example, the first image is a single frame RAW image; the abnormal exposure area can be determined from the first image based on the brightness value of each pixel point in the first image by acquiring the brightness value of each pixel point in the first image, and the image brightness distribution information can be obtained.

Here, the average luminance value of the exposure abnormal region is greater than a first luminance threshold value, or the average luminance value of the exposure abnormal region is less than a second luminance threshold value, the first luminance threshold value being greater than the second luminance threshold value. In this example, the exposure abnormal region may be an overexposed region and an underexposed region.

Step S302, performing brightness restoration processing on the exposure abnormal region of the first image based on image brightness distribution information by using a second neural network;

in this example, the second neural network may be a feature-based luminance repair network. And carrying out brightness restoration processing on the exposure abnormal region in the first image through the first image and the image brightness distribution information of the first image to obtain a brightness restoration image.

In some embodiments, the performing, by using a second neural network, a luminance repair process on an exposure anomaly region of the first image based on image luminance distribution information includes:

determining an exposure abnormal region from the first image based on the image brightness distribution information by using a second extraction module of the second neural network, and extracting a first semantic feature of the first image and a second semantic feature of the exposure abnormal region;

Performing contour restoration on the exposure abnormal region in the first image based on the first semantic features by using a first restoration module of the second neural network;

and performing texture repair on the exposure abnormal region in the first image output by the first repair module based on the second semantic features by using a second repair module of the second neural network to obtain a brightness repair image.

In this example, the first semantic feature may be a global semantic feature of a first image for describing image content of the first image; the second semantic features may be local semantic features of the exposure anomaly region within the first image, indicative of texture features and color features used to describe the exposure anomaly region.

And step S303, determining the brightness restoration image output by the second neural network as a second image.

In the example, the global semantic features of the first image and the local semantic features of the exposure abnormal region are utilized to carry out information complementation through the second neural network, and the first image is subjected to targeted brightness restoration from the aspects of macroscopic scenes and microscopic details, so that the lost image content of the exposure abnormal region can be restored, the situations of local deformation and unnatural transition can be effectively reduced, the restored image is more natural, and the image with a high dynamic range is obtained.

Illustratively, as shown in fig. 8, fig. 8 is a flow chart illustrating an image brightness restoration method according to an exemplary embodiment.

Fig. 9 is a flowchart three of an image processing method according to an exemplary embodiment, as shown in fig. 9, the method includes:

step S401, a first image to be processed and first image information of the first image are acquired; wherein the first image information includes: image area information divided according to image content and image brightness distribution information determined according to image brightness;

in order to satisfy timeliness of image processing of the mobile terminal, the present example shortens image processing time by performing image denoising processing and luminance restoration processing on the first image in parallel.

The method comprises the steps of obtaining a RAW image (namely a first image) of a single frame, and carrying out image segmentation processing and edge detection processing on the first image to obtain region distribution information and edge distribution information of the first image respectively; and obtaining image region information according to the region distribution information and the edge distribution information. Meanwhile, image brightness distribution information of the first image is determined based on brightness values of all pixel points in the first image.

Step S402, denoising processing with different intensities is carried out on different image areas of the first image based on the image area information by utilizing a target neural network, so as to obtain a denoised image;

in this example, the first neural network and the second neural network may be two sub-networks in the target neural network, and the denoising processing with different intensities is performed on the image areas of different types in the first image and the first sub-area and the second sub-area included in the image areas of different types based on the image area information by using the first sub-network (i.e., the first neural network) in the target neural network, so as to obtain a denoised image.

Step S403, performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information by using the target neural network to obtain a brightness restoration image;

in this example, a second sub-network (i.e., a second neural network) within the target neural network is utilized, based on different intensities of denoising processing for different categories of image regions within the first image, and the first and second sub-regions contained within the different categories of image regions.

In order to implement the image denoising process and the brightness restoration process on the first image in parallel through the target neural network, the target neural network may be obtained by performing joint training on the first neural network and the second neural network.

Here, as shown in fig. 10, fig. 10 is a schematic diagram illustrating a joint training according to an exemplary embodiment. The region-based image noise reduction network (i.e., the first neural network) and the feature-based luminance restoration network (i.e., the second neural network) are trained simultaneously, cooperatively constraining the loss functions of the two networks.

In this example, the loss functions of the target neural network may be obtained by setting weight values for the loss functions of the two networks, and weighting and fusing the loss functions of the two networks based on the weight values; therefore, the robustness and generalization capability of the two networks are enhanced, and the brightness restoration network/image denoising network has the effect adjustment capability even if the target neural network is poor in image denoising effect/brightness restoration effect.

And step S404, performing image fusion on the denoising image and the brightness restoration image output by the target neural network to obtain the second image.

In this example, based on the denoising image output by the first sub-network and the luminance restoration image output by the second sub-network, the denoising image and the luminance restoration image are weighted and fused according to the weight values corresponding to the first sub-network and the second sub-network, so as to obtain a second image with high quality and high dynamic range.

Illustratively, as shown in fig. 11, fig. 11 is a schematic workflow diagram of an electronic device including a sensor module, an image generation module, and an image post-processing module, according to an exemplary embodiment.

The electronic equipment acquires a single-frame RAW image through the sensor module and transmits the single-frame RAW image to the image generation module; the image generation module performs image denoising and brightness restoration on the single-frame RAW image to obtain a second image after denoising and brightness restoration, and transmits the second image to the image post-processing module; and performing ISP processing on the second image by an image post-processing module to obtain an image in a JPG format, and taking the image in the JPG format as a final output image.

Here, by processing a single frame RAW image, it is possible to reduce the dependence on image alignment and fusion effects while shortening the photographing time. Considering that for a single frame image, the denoising effect and the brightness restoration effect of the image are both poor, in this example, the image generation module utilizes a target neural network comprising a first neural network and a second neural network, combines image area information and image brightness distribution information of the image, realizes adaptive noise removal and brightness restoration of the image, and effectively improves the dynamic range and image quality of the image shot by the electronic device.

The embodiment of the disclosure also provides an image processing device. Fig. 12 is a schematic structural view of an image processing apparatus according to an exemplary embodiment, and as shown in fig. 12, the image processing apparatus 100 includes:

an acquiring module 101, configured to acquire a first image to be processed and first image information of the first image; wherein the first image information includes: image area information divided according to image content and/or image brightness distribution information determined according to image brightness;

the processing module 102 is configured to perform denoising processing with different intensities on different image areas of the first image based on the image area information by using a target neural network; and/or, performing brightness restoration processing on the exposure abnormal region of the first image based on the image brightness distribution information;

a determining module 103, configured to determine a second image according to the output image of the target neural network.

Optionally, the determining module 103 is configured to:

or alternatively, the process may be performed,

Or alternatively, the process may be performed,

Optionally, the acquiring module 101 is configured to:

Optionally, the processing module 102 is configured to:

Optionally, the acquiring module 101 is configured to:

Optionally, the processing module 102 is configured to:

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 13 is a block diagram of an image processing apparatus according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a mobile computer, or the like.

Referring to fig. 13, the apparatus 200 may include one or more of the following components: a processing component 202, a memory 204, a power supply component 206, a multimedia component 208, an audio component 210, an input/output (I/O) interface 212, a sensor component 214, and a communication component 216.

The processing component 202 generally controls overall operation of the apparatus 200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 202 can include one or more modules that facilitate interactions between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

The memory 204 is configured to store various types of data to support operations at the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phonebook data, messages, pictures, videos, and the like. The memory 204 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 206 provides power to the various components of the device 200. The power supply components 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

The multimedia component 208 includes a screen between the device 200 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front-facing camera and/or a rear-facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 200 is in an operational mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 210 is configured to output and/or input audio signals. For example, the audio component 210 includes a Microphone (MIC) configured to receive external audio signals when the device 200 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, audio component 210 further includes a speaker for outputting audio signals.

The I/O interface 212 provides an interface between the processing assembly 202 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 214 includes one or more sensors for providing status assessment of various aspects of the apparatus 200. For example, the sensor assembly 214 may detect the on/off state of the appliance 200, the relative positioning of the components, such as the display and keypad of the device 200, the sensor assembly 214 may also detect a change in position of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration/deceleration of the device 200, and a change in temperature of the device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 216 is configured to facilitate communication between the apparatus 200 and other devices in a wired or wireless manner. The device 200 may access a wireless network based on a communication standard, such as Wi-Fi,4G, or 5G, or a combination thereof. In one exemplary embodiment, the communication component 216 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 204, including instructions executable by processor 220 of apparatus 200 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It is to be understood that the invention is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. An image processing method, comprising:

2. The method of claim 1, wherein the determining a second image from the output image of the target neural network comprises:

or alternatively, the process may be performed,

3. The method of claim 1, wherein the acquiring the first image to be processed and the first image information of the first image comprises:

4. A method according to claim 3, wherein using the target neural network to denoise different image regions of the first image with different intensities based on the image region information comprises:

5. The method of claim 4, wherein using the first subnetwork to denoise the image regions of different categories within the first image and the first sub-region and the second sub-region included within the image regions of different categories based on the image region information includes:

6. The method of claim 5, wherein the first noise reduction module is configured to:

7. The method of claim 6, wherein the fusion module is configured to:

8. The method of claim 1, wherein the acquiring the first image to be processed and the first image information of the first image comprises:

9. The method according to claim 8, wherein the performing, with the target neural network, the luminance repair process on the exposure abnormal region of the first image based on the image luminance distribution information, includes:

10. The method according to claim 9, wherein the performing, with the second sub-network of the target neural network, the luminance repair process on the exposure abnormal region of the first image based on the image luminance distribution information, includes:

11. An image processing apparatus, comprising:

12. An image processing apparatus, characterized by comprising:

a processor;

a memory for storing executable instructions;

wherein the processor is configured to: the image processing method of any of claims 1-10, when executed by executable instructions stored in the memory.

13. A non-transitory computer readable storage medium, which when executed by a processor of an image processing apparatus, causes the image processing apparatus to perform the image processing method of any one of claims 1-10.