WO2020140986A1

WO2020140986A1 - Image denoising method and apparatus, storage medium and terminal

Info

Publication number: WO2020140986A1
Application number: PCT/CN2020/070337
Authority: WO
Inventors: 张弓
Original assignee: Oppo广东移动通信有限公司
Priority date: 2019-01-04
Filing date: 2020-01-03
Publication date: 2020-07-09
Also published as: CN109639982B; CN109639982A

Abstract

Disclosed are an image denoising method and apparatus, a storage medium and a terminal. The method comprises: determining brightness information of a skin color area in a target image; determining, in the skin color area, a target sub-area with the brightness lower than a pre-set brightness threshold value, and determining a target denoising intensity according to the brightness of the target sub-area and the brightness information; and performing, based on the target denoising intensity, denoising processing on the target sub-area to obtain a target image subjected to denoising processing. By using the technical solution, by means of determining, based on the brightness, a target sub-area to be subjected to denoising processing, determining the denoising intensity according to the brightness of each pixel point in the target sub-area and brightness information of a skin color area, and performing, based on the denoising intensity, denoising processing on a corresponding pixel point, the effect of locally denoising the skin color area based on the brightness is achieved, such that noises in the skin color area are distributed more uniformly.

Description

Image noise reduction method, device, storage medium and terminal

Technical field

Embodiments of the present application relate to the technical field of terminals, and in particular, to an image noise reduction method, device, storage medium, and terminal.

Background technique

With the rapid development of terminal technology, electronic devices such as mobile phones and tablet computers are equipped with image collection functions, and users have higher and higher requirements for the quality of the images collected by the terminal.

In actual use, users usually use their mobile phones to take pictures. However, under different environments, the noise of the face image obtained by the shooting changes greatly. For example, under backlighting, side light, or point light sources, the noise of the neck shadows and nostrils of the captured face images is more noisy, which affects the final rendering of the photos.

Summary of the invention

Embodiments of the present application provide an image noise reduction method, device, storage medium, and terminal, which can optimize a noise reduction scheme in related technologies.

In a first aspect, an embodiment of the present application provides an image noise reduction method, including:

Determine the brightness information of the skin color area in the target image;

Determining a target sub-area in the skin-color area whose brightness is lower than a preset brightness threshold, and determining a target noise reduction intensity according to the brightness of the target sub-area and the brightness information;

Perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.

In a second aspect, an embodiment of the present application further provides an image noise reduction device, which includes:

The information determination module is used to determine the brightness information of the skin color area in the target image;

A noise reduction intensity determination module, configured to determine a target sub-region with a brightness lower than a preset brightness threshold in the skin-color region, and determine a target noise reduction intensity according to the brightness of the target sub-region and the brightness information;

The noise reduction processing module is configured to perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.

In a third aspect, embodiments of the present application also provide embodiments of the present application to provide a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements image reduction as provided by any embodiment of the present application Noise method.

According to a fourth aspect, an embodiment of the present application provides a terminal, including a memory, a processor, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, it can be implemented as arbitrarily implemented in the present application. Example provides image noise reduction method.

An embodiment of the present application provides an image noise reduction solution, by determining the brightness information of the skin color area in the target image; determining the target sub-area in the skin color area whose brightness is lower than the preset brightness threshold, according to the brightness and brightness information of the target sub-area Determine the target noise reduction intensity; perform noise reduction on the target sub-region based on the target noise reduction intensity to obtain the target image after noise reduction processing. By adopting the above technical solution, the target sub-region to be subjected to noise reduction processing is determined based on the brightness, the noise reduction intensity is determined according to the brightness of each pixel in the target sub-region and the brightness information of the skin color area, and the corresponding pixels are determined based on the noise reduction intensity Perform noise reduction on the points to achieve the local noise reduction effect on the skin color area based on brightness, making the noise distribution of the skin area more uniform.

BRIEF DESCRIPTION

1 is a flowchart of an image noise reduction method provided by an embodiment of the present application;

2 is a flowchart of another image noise reduction method provided by an embodiment of the present application;

3 is a schematic diagram of an image noise distribution provided by an embodiment of the present application;

4 is a flowchart of another image noise reduction method provided by the present application;

5 is a structural block diagram of an image noise reduction device according to an embodiment of the present application;

6 is a schematic structural diagram of a terminal according to an embodiment of the present application;

7 is a structural block diagram of a smartphone provided by an embodiment of the present application.

detailed description

The present application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described here are only used for explaining the present application, rather than limiting the present application. In addition, it should be noted that, for ease of description, the drawings only show parts related to the present application but not all structures.

Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowchart describes the steps as sequential processing, many of the steps can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the steps can be rearranged. The processing may be terminated when its operation is completed, but may also have additional steps not included in the drawings. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and so on.

The noise reduction scheme in the related art regards the human face as a whole, and uses a similar noise reduction intensity to perform overall noise reduction on the human face. Since the actual distribution of noise in the face image is not consistent, the overall noise reduction scheme will result in uneven distribution of noise in different areas, and the noise reduction effect is not ideal.

FIG. 1 is a flowchart of an image noise reduction method provided by an embodiment of the present application. The method may be suitable for photographing scenes, including but not limited to shooting videos or photos. The method may be performed by an image noise reduction device, which may be The software and/or hardware implementation can generally be integrated in the terminal. As shown in Figure 1, the method includes:

Step 110: Determine the brightness information of the skin color region in the target image.

It should be noted that the terminal in the embodiment of the present application may include mobile phones, tablet computers, notebook computers, computers, and other electronic devices that display images. An operating system is integrated in the terminal in the embodiment of the present application, and the type of the operating system is not limited in the embodiment of the present application, for example, it may include an Android operating system, a Windows operating system, and an Apple (ios) operating system and many more.

It should be noted that the brightness information may be brightness related information of each pixel in the skin color area of the target image. For example, the brightness information may be the average brightness value of the skin area; the brightness information may be the brightness weighted value of the skin area; the brightness information may also be the maximum brightness of the skin area; and the brightness information may also be the minimum brightness of the skin area, etc. .

It should be noted that the target image may be an image obtained by shooting a target scene through a terminal with a shooting function, an image acquired from an album of the terminal, or an image acquired from an Internet platform, and so on. The target image may be an image of RGB color mode, YUV color mode, HSV color mode, or Lab color mode. Among them, color is usually described by three relatively independent attributes. The combined effect of three independent variables naturally forms a spatial coordinate. This is the color mode. The color mode can be divided into a primary color mode and a color separation color mode. Among them, the primary color mode includes but not limited to RGB color mode, and the color separation mode includes but not limited to YUV color mode, Lab color mode and HSV color mode. The color light separation color mode is a color mode for indicating the separation of color and brightness. In the YUV color model, the Y component represents brightness, the U component represents color, and the V component represents density, where the U component and V component together represent the color of the image. In the Lab color model, the L component represents brightness, and a and b jointly represent color. In the HSV color model, the H component represents hue, the S component represents saturation, and the V component represents lightness, where hue is the basic attribute of color, saturation refers to the purity of color, and lightness is also brightness. In an image with a color-separated color mode, the brightness component and the color component can be extracted separately, and the image can be processed in any aspect of brightness and color. Exemplarily, when the brightness is processed, the color of the image will not be processed. The weight has no effect.

In the embodiment of the present application, the brightness of each pixel in the skin color area is obtained, so that the mean value of the brightness mean_lux of the skin color area can be calculated. The maximum brightness max_lux and the minimum brightness min_lux can be determined by comparing the brightness of each pixel in the skin color area.

Step 120: Determine a target sub-region whose brightness in the skin-color region is lower than a preset brightness threshold, and determine a target noise reduction intensity according to the brightness of the target sub-region and the brightness information.

Exemplarily, each pixel in the skin color area of the target image in the color separation mode is obtained, the brightness of each pixel in the skin color area is compared with a preset brightness threshold, and a target whose brightness is lower than the preset brightness threshold is marked Pixel points, clustering the target pixel points into at least one target sub-region; according to the brightness of the target pixel point in each of the target sub-regions, the average brightness value of the skin color area, the maximum brightness value of the skin color area and the The minimum brightness value of the skin color area determines the target noise reduction intensity based on the brightness.

It should be noted that the preset brightness threshold is a threshold value used to filter the target sub-regions in the skin-color area that needs to be subjected to local noise reduction, including but not limited to the average brightness of the skin-color area and the brightness weighted value of the skin-color area (weighted weight Is the ratio of the number of pixels corresponding to each brightness to the total number of pixels contained in the skin color area), the target brightness value in the skin color area where the brightness is lower than the average brightness and the corresponding pixel is the most, or the brightness in the skin area is lower than the brightness weight Value and the corresponding target brightness value with the most pixels.

For example, the brightness of each pixel in the skin color area is compared with the average brightness to determine the target sub-area with brightness lower than the average brightness. For the area composed of pixels whose brightness is higher than the average brightness value, no additional local noise reduction process is performed.

In another example, the brightness of each pixel in the skin color area is compared with the brightness weighted value to determine the target sub-area whose brightness is lower than the brightness weighted value. For the area composed of pixels whose brightness is higher than the brightness weighted value, no additional local noise reduction process is performed.

Step 130: Perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.

Exemplarily, based on the target noise reduction intensity based on brightness, any pixel in the target sub-region is adjusted according to the target noise reduction intensity, and the remaining pixel points in the target sub-region are separately adjusted according to the target noise reduction intensity, Therefore, the local noise reduction processing of the target image is realized, and the amount of face noise under undesirable light such as backlight, side light, direct light source, and dark light is effectively suppressed. The image noise reduction method provided in the embodiment of the present application can make the face area present a natural and clear picture effect, avoid the appearance of the traditional face noise reduction scheme, regard the face as a whole, and perform overall noise reduction processing on the entire face area, The phenomenon of uneven noise in different areas of the processed image will affect the consistency of the photos. In addition, the embodiments of the present application perform noise reduction processing on the face area on the basis of a single frame image, which has a high processing speed, which can avoid the multi-frame noise reduction scheme taking a long time and affecting the success rate and photo output of the photo. The problem of film speed.

In the technical solution of the embodiment of the present application, by determining the brightness information of the skin color region in the target image; determining the target sub-region in the skin color region whose brightness is lower than the preset brightness threshold, the target noise reduction is determined according to the brightness and brightness information of the target sub-region Intensity; based on the target noise reduction intensity, the target sub-region is subjected to noise reduction processing to obtain the noise-reduced target image. By adopting the above technical solution, the target sub-region to be subjected to noise reduction processing is determined based on the brightness, the noise reduction intensity is determined according to the brightness of each pixel in the target sub-region and the brightness information of the skin color area, and the corresponding pixels are determined based on the noise reduction intensity Perform noise reduction on the points to achieve the local noise reduction effect on the skin color area based on brightness, making the noise distribution of the skin area more uniform.

FIG. 2 is a flowchart of another image noise reduction method provided by an embodiment of the present application. The method includes:

Step 201: Acquire a target image in a color-separated color mode, perform face recognition on the target image, and determine face information contained in the target image.

Exemplarily, perform face detection on the entire target image, mark facial feature points, and determine facial contour information, eyebrow contour information, eye contour information, nose contour information, and mouth contour by using the facial feature points information. Optionally, a face frame can be used to identify the face area.

In the embodiment of the present application, the target image in the color separation mode can be an image captured by the camera according to the shooting instruction, or it can be collected by the camera and displayed on the electronic device screen for the user before the shooting instruction is executed. Preview image information.

It should be noted that, when the acquired image is not a color separation mode, a setting algorithm may be used to convert the image to a color separation mode. Taking a mobile phone as an example, when acquiring images based on an image acquisition device in a mobile phone, a method of generating an image in the YUV color mode includes: converting the raw data into an RGB color mode image based on the raw data acquired by the image sensor; The RGB color mode image described above generates the YUV color mode image. The image acquisition device may be, for example, a camera, and the camera may include a charge-coupled device (CCD, Charge-coupled Device) image sensor or a complementary metal oxide semiconductor (CMOS, Complementary Metal, Oxide, Semiconductor) image sensor, based on the CCD image sensor or The CMOS image sensor converts the captured light source signal into RAW raw data of a digital signal, converts the RGB raw color image data based on the RAW raw data, and further converts it into YUV color mode image data. In the image acquisition device of the mobile phone, the JPG format image can be formed by the YUV color mode image.

It should be noted that the color in the RGB color mode image data formed by the conversion of RAW raw data is not the true color of the image, and the RGB color mode image data formed here cannot be processed in any way. The YUV color mode image data The formed color is the true color of the image, and the image data of the YUV color mode can be processed. In common image processing, RGB data is usually processed, and the raw data collected by the image sensor is converted into the following color mode during the processing process: RAW raw data-RGB color mode image-YUV color mode image Among them, the RGB color mode image is processed, and then the processed RGB color mode image is converted to the YUV color mode image, and the JPG format image can be output. Correspondingly, when processing images in other color modes, they need to be obtained by converting images in YUV color mode, and converting the processed images to images in YUV color mode to obtain images in JPG format.

Step 202: Determine the facial skin area of the target image according to the contour information in the human face information.

Exemplarily, the facial skin area of the target image may be determined according to the contour information in the human face information. For example, the facial skin area on the human face is determined according to the position coordinates of the facial features.

In the embodiments of the present application, face recognition technology and key points (representing the characteristic points of eyebrows, eyes, mouth, and face contours) are used to identify the number, size, and pose of faces in the image, and also recognize the face Regions, facial features position, so as to segment the facial skin area in the facial image. Optionally, a face recognition technology may also be used to identify the position of the nose head, a face frame with a fixed length and width is taken based on the nose head coordinates, and a facial skin area is determined based on the area selected by the face frame.

Step 203: Obtain the brightness and color of each pixel in the facial skin area separately, and determine the investigation area from the target image according to the brightness and color.

Exemplarily, the average brightness value of the face skin area is calculated according to the brightness of pixels included in the face skin area. And, the color average value of the facial skin area is calculated according to the color of the pixels included in the facial skin area. Using the facial skin area as a reference, the similar target image area is expanded to obtain areas with similar brightness and color like facial skin, such as neck, ears, and shoulders. For example, calculate the brightness deviation between the brightness of the pixels in the neighborhood of the face skin area and the above average brightness, and the color deviation between the colors of the pixels in the neighborhood of the face skin area and the above average color, if the brightness deviation is less than the setting If the brightness threshold is set, and the color deviation is less than the set color threshold, it is determined that the pixel is similar to the pixel in the skin area of the face. An area composed of pixels similar to pixels in the skin area of the face is marked as the investigation area.

As another example, using the face frame as a reference, increase the area of the face frame according to a set ratio. Obtain the brightness and color of each new pixel in the new area of the face frame before and after the change. Determine the brightness deviation of the brightness of each newly added pixel from the above average brightness value, and the color deviation of the color of each newly added pixel from the above average color value. When the brightness deviation is less than the set brightness threshold and the color deviation is less than the set color threshold, it is determined that the newly added pixel belongs to the investigation area.

It should be noted that there are many ways to increase the face frame according to the set ratio, and the embodiments of the present application are not specifically limited. For example, a new face frame can be obtained by extending 10% along the long and short sides of the face frame away from the centroid. For another example, the extension of 5% along a certain direction may be determined according to the difference between the brightness of the pixel and the average brightness of the facial skin area and the difference between the color and the average color.

Step 204, a skin color area is formed by the investigation area and the facial skin area.

Step 205: Perform noise statistics on the target image, and determine the noise levels of the skin color area and the background area based on the statistical results.

After the skin color area is determined, the remaining area in the target image except the skin color area is recorded as the background area, including hair, clothes, and accessories. Perform noise statistics on the target image, and determine the noise levels of the skin color area and the background area based on the statistical results. For example, an image noise estimation algorithm is used to estimate the noise of the target image. For example, the target image is divided into blocks, based on the domain correlation (the difference between all pixels in an image block area and its neighbors is calculated by calculating the difference between the pixels in the image block to reflect the correlation between the pixels in the block, referred to as the block Intra-domain correlation, use the intra-block domain correlation to determine the smoothness of the image block) to filter the smooth image blocks; then use the SVD (K-SVD algorithm, K-singular value decomposition) for the filtered smooth image blocks (Algorithm) to perform noise estimation, and finally compare the noise estimation values of each smooth image block to determine the maximum noise estimation value and the minimum noise estimation value. The noise interval formed by the maximum noise estimate value and the minimum noise estimate value is divided into N noise levels, and N is a positive integer, which is the system default value. The higher the noise level, the greater the noise. A similar method is used to estimate the noise of the skin color area and the background area respectively to obtain the skin color noise value and the background noise value. The skin color noise value and the background noise value are respectively matched with the noise level, and the noise levels of the skin color area and the background area are determined respectively. Based on the above noise estimation result, the overall noise reduction intensity of the overall noise reduction processing of the target image is determined, and the overall noise reduction processing of the target image is performed according to the overall noise reduction intensity.

FIG. 3 is a schematic diagram of an image noise distribution provided by an embodiment of the present application. As shown in FIG. 3, black areas indicate non-noise areas and gray areas indicate noise areas. As shown in FIG. 3, noise is concentrated on the hair edge 310 and eyebrows. Edge 320, facial contour 330 and neck shadow 340 and other areas.

In the embodiment of the present application, after determining the noise levels of the skin area and the background area, if the noise level of the skin area is higher than the noise level of the background area, that is, the noise of the skin area is greater than the noise of the background area, a local noise reduction event is triggered To perform additional noise reduction on the skin color area. If the noise of the skin color area is smaller than the noise of the background area, then the details of the face area are protected.

Step 206: Determine an overall noise reduction intensity for performing overall noise reduction processing on the target image according to the noise level, and perform overall noise reduction processing on the target image based on the overall noise reduction intensity.

It should be noted that the facial features such as eyes, eyebrows, and mouth contain more image detail information, so it is not appropriate to reduce noise.

Step 207: Determine whether the noise level of the skin color area is higher than the noise level of the background area. If yes, perform step 208; otherwise, perform step 219.

Step 208: Trigger the local noise reduction event.

When the noise level of the skin color area of the target image is higher than the noise level of the background area, a local noise reduction event is triggered. Wherein, the local noise reduction event is used to instruct to perform the operation of determining the brightness information of the skin color region in the target image.

Step 209: It is detected that a local noise reduction event is triggered.

Step 210: Determine the average brightness value, the maximum brightness value, and the minimum brightness value of the skin color region in the target image.

Step 211: Compare the brightness of each pixel in the skin color area with the average brightness, mark target pixels whose brightness is lower than the average brightness, and cluster the target pixels into at least one target sub-area.

In the embodiment of the present application, the brightness of each pixel in the skin color area is compared with the average brightness of the skin color area, the target pixel with brightness lower than the average brightness is marked, and the target pixel is clustered into at least one target sub-area . A target pixel in each target sub-region is sequentially obtained, and the target noise reduction intensity based on the brightness is calculated according to the brightness in_lux of the target pixel, the average brightness of the skin area, the maximum brightness, and the minimum brightness. For example, the following formula can be used to calculate the target noise reduction intensity L_nr based on brightness:

L_nr=(max_lux-min_lux)*(mean_lux-in_lux)/mean_lux (1)

Among them, max_lux represents the maximum brightness of the skin area, min_lux represents the minimum brightness of the skin area, mean_lux represents the average brightness of the skin area, and in_lux represents the brightness of a target pixel in the target sub-region.

It should be noted that the corresponding target pixel may be subjected to noise reduction processing according to the brightness-based target noise reduction intensity. The target noise reduction intensity corresponding to the target pixel in each target sub-region can be calculated in the above manner.

For example, the target sub-region may be a region with low brightness such as a shadow on the neck or a contour of the face.

Step 212: Determine a target noise reduction intensity based on the brightness according to the brightness of the target pixel in each target sub-region, the average brightness, the maximum brightness, and the minimum brightness.

Step 213: Determine whether the number of faces in the target image is greater than 1, if yes, perform step 214, otherwise perform step 218.

Step 214: When the target image contains at least two human faces, obtain the color of the skin color area, and determine the first color mean, maximum color value, and minimum color value of the skin color area in the target image according to the color of the skin color area.

Exemplarily, when the target image contains at least two faces, the color components of the pixels included in each face are respectively obtained, and the colors of the pixels in the skin color area are calculated based on the weighted summation. Taking the YUV color mode as an example, the color C of each pixel can be expressed as:

Among them, (m,n) represents a pixel of the facial skin color area of each face, which belongs to the coordinate range (0,0) to (x,y), and is the set weight, which can be the system default value, U _mn and V _mn respectively represent the color component of each pixel in the facial skin color area of each face.

The first color mean, maximum color value and minimum color value of the skin color area in the target image are determined according to the color of each pixel in the facial skin color area of each face.

Step 215: Calculate the second color average of the skin color area corresponding to each face separately.

Exemplarily, the color average value of the skin color area of each face is determined based on the color of each pixel in the facial skin color area of each face, and is recorded as the second color average.

Step 216: For the target skin color area where the second color average value is less than the first color average value, determine based on the color according to the color of the target skin color area, the first color average value, the maximum color value and the minimum color value Target noise reduction intensity.

Exemplarily, the second color mean is compared with the first color mean to determine the target skin color area where the second color mean is less than the first color mean. Obtain a survey pixel in the target skin color area in sequence, and calculate the color-based target noise reduction intensity according to the color in_col of the survey pixel, the first color mean mean1_col, the maximum color value max_col, and the minimum color value min_col. For example, the following formula can be used to calculate the color-based target noise reduction intensity C_nr:

C_nr=(max_col-min_col)*(mean1_col-in_col)/mean1_col (3)

Optionally, the color-based target noise reduction intensity can also be calculated according to the second color mean mean2_col, the first color mean mean1_col, the maximum color value max_col, and the minimum color value min_col of the skin color area corresponding to each face in the target skin color area . For example, the following formula can be used to calculate the color-based target noise reduction intensity C_nr:

C_nr=(max_col-min_col)*(mean1_col-mean2_col)/mean1_col (4)

Step 217: Determine a weighted operation result of the target noise reduction intensity based on brightness and the target noise reduction intensity based on color, and use the weighted operation result as the target noise reduction intensity.

Exemplarily, a set weighting coefficient is adopted to perform weighted operation on the target noise reduction intensity based on brightness and the color-based target noise reduction intensity, and use the weighted operation result as the target noise reduction intensity, that is, Nr=a*L_nr+b*C_nr , Where a and b are weighting coefficients, which are system default values.

Step 218: Perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.

Exemplarily, when the number of faces in the target image is one, the target noise reduction intensity L_nr based on the brightness is used to perform noise reduction processing on the pixels in the target sub-region to obtain the noise-reduced target image.

If the number of faces in the target image is at least 2, the target noise reduction intensity Nr after the weighting operation is used to perform noise reduction processing on the pixels in the target sub-region to obtain the noise-reduced target image.

Step 219: Output the target image.

According to the technical solution of the embodiment of the present application, when the target image contains at least two faces, the color range of each face is counted separately to calculate the average face color of the entire target, the average face color of each face, and the color The maximum value and the minimum color value, thereby determining the color-based target noise reduction intensity based on the face color mean, color maximum and color minimum values; determining the weighted operation of the brightness-based target noise reduction intensity and the color-based target noise reduction intensity As a result, the weighted calculation result is used as the target noise reduction intensity. Using the above technical solution, the skin color region can be divided into different sub-regions according to the skin tone and skin brightness of each face, and different target noise reduction intensities can be determined for different sub-regions. The area uses a larger target noise reduction intensity, and the lighter skin color, the brighter the area uses a smaller target noise reduction intensity, thereby effectively reducing the noise of dark skin areas, neck shadows, face contours and other dark light areas .

In some embodiments, after performing noise reduction processing on the target sub-region based on the target noise reduction intensity, the method further includes: performing noise statistics on the noise-reduced target image, and determining the skin color region based on the statistical results Noise level; determine whether the noise level belongs to a preset noise interval; if it is, output the target image after noise reduction processing; otherwise, determine the mixing weight according to the noise level, based on the mixing weight on the target image and noise reduction processing The target image after the mixing process is processed, and the target image after the mixing process is output.

FIG. 4 is a flowchart of another image noise reduction method provided by the present application. After acquiring the original target image origin_pic, a noise reduction area is selected. For example, the original target image origin_pic is subjected to face detection, keypoints, edge markers, and skin color area selection to determine the skin color area of the face, as well as the investigation area including ears, shoulders, and necks that are similar to the skin color of the face. The skin color area and the investigation area are marked as skin color areas, and the skin color area is the noise reduction area. Perform noise estimation on the target image to determine the overall noise reduction intensity. Noise reduction is performed on the entire face based on the overall noise reduction intensity. The target image NR_pic after noise reduction is obtained by performing local noise reduction on the skin color area based on the brightness of the skin and the color of the skin. Determine the noise level of the skin color region in NR_pic, and when the noise level does not belong to the preset noise interval, acquire the original target image origin_pic and the target image NR_pic after noise reduction processing. The mixed weight blend_percent (0≤blend_percent≤100) is determined based on the noise level of the skin color region of NR_pic, and each pixel in the final output target image is a mixture value of origin_pic and NR_pic based on blend_percent (ie blend_percent*NR_pic+(1-blend_percent) *origin_pic). The advantage of this design is that when the noise level after the local noise reduction process in the skin color area is not in the preset noise interval (it may be that the noise reduction transition has lost some details), the mixing is determined according to the noise level of the skin color area after the noise reduction process Weights, and based on the mixed weights, the original target image and the noise-reduced target image are mixed to dynamically adjust the noise distribution of the target image, so that the noise distribution in the final target image is more uniform, more natural and clear Target image.

It should be noted that the technical solutions of the embodiments of the present application may be added to the intermediate or final process of ISP (Image Signal Processing) to optimize the photo shooting effect. Optionally, the technical solutions of the embodiments of the present application can also be used in combination with a multi-frame noise reduction technology to achieve better noise reduction effect in random noise and dark noise reduction scenes.

FIG. 5 is a structural block diagram of an image noise reduction device provided by an embodiment of the present application. The device can be implemented by software and/or hardware, and is generally integrated in a terminal, which can effectively suppress the backlight and side light by performing the image noise reduction method. , Face light under direct light, dark light and other unfavorable face noise, presents a clearer and natural face image. As shown in Figure 5, the device includes:

The information determination module 510 is used to determine the brightness information of the skin color area in the target image;

The noise reduction intensity determination module 520 is configured to determine a target sub-region whose brightness is lower than a preset brightness threshold in the skin-color region, and determine a target noise reduction intensity according to the brightness of the target sub-region and the brightness information;

The noise reduction processing module 530 is configured to perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.

An embodiment of the present application provides an image noise reduction device, which determines a target noise reduction intensity according to the brightness and brightness information of the target subregion by determining a target subregion in the skin color region whose brightness is lower than a preset brightness threshold; based on the target reduction The noise intensity performs noise reduction processing on the target sub-region to obtain the target image after noise reduction processing. By adopting the above technical solution, the target sub-region to be subjected to noise reduction processing is determined based on the brightness, the noise reduction intensity is determined according to the brightness of each pixel in the target sub-region and the brightness information of the skin color area, and the corresponding pixels are determined based on the noise reduction intensity Perform noise reduction on the points to achieve the local noise reduction effect on the skin color area based on brightness, making the noise distribution of the skin area more uniform.

Optionally, it also includes a skin tone area, which is used for:

Before determining the brightness information of the skin color region in the target image, obtain the target image in the color separation mode, perform face recognition on the target image, and determine the face information contained in the target image, where the face information Including the number of faces, and the outline information of the face, eyebrows, eyes, nose and mouth;

Determine the facial skin area of the target image according to the contour information in the human face information;

Obtain the brightness and color of each pixel in the face skin area separately, and determine the investigation area from the target image according to the brightness and color, wherein the pixels in the investigation area and the face The deviation of pixels in the skin area in brightness and color is less than the set threshold;

The skin area is composed of the investigation area and the facial skin area.

Optionally, an event trigger module is also included. The event trigger module is used to:

After the skin color area is composed of the investigation area and the facial skin area, perform noise statistics on the target image, and determine the noise levels of the skin color area and the background area based on the statistical results;

When the noise level of the skin color area is higher than the noise level of the background area, a local noise reduction event is triggered, where the local noise reduction event is used to indicate the execution of the operation to determine the brightness information of the skin color area in the target image.

Optionally, an overall noise reduction module is also included. The overall noise reduction module is used for:

After performing noise statistics on the target image and determining the noise levels of the skin-color area and the background area based on the statistical results, the overall noise reduction intensity of the overall noise reduction processing on the target image is determined according to the noise level, based on The overall noise reduction intensity performs overall noise reduction processing on the target image.

Optionally, the noise reduction intensity determination module 520 is specifically used to:

Compare the brightness of each pixel in the skin color area with a preset brightness threshold, mark target pixels whose brightness is lower than the preset brightness threshold, and cluster the target pixels into at least one target sub-region;

The target noise reduction intensity based on brightness is determined according to the brightness of the target pixel in each target sub-region, the average brightness, the maximum brightness, and the minimum brightness.

Optionally, it also includes a color information determination module, which is used to:

Before performing noise reduction processing on the target sub-region based on the target noise reduction intensity, when the target image contains at least two human faces, the color of the skin color region is acquired, and the skin color region in the target image is determined according to the color The first color mean, maximum color value and minimum color value of

Calculate the second color average of the skin color area corresponding to each face separately;

For the target skin color area where the second color average value is less than the first color average value, a color-based target drop is determined according to the color of the target skin color area, the first color average value, the maximum color value, and the minimum color value Noise strength.

Optionally, the device further includes:

The weighting operation module is used to determine the target noise reduction intensity based on the brightness after determining the target noise reduction intensity based on the color according to the color of the target skin color area, the first color mean, the maximum color value and the minimum color value And the weighted calculation result of the target noise reduction intensity based on color, and using the weighted calculation result as the target noise reduction intensity.

Optionally, an image mixing module is also included. The image mixing module is used to:

After performing noise reduction processing on the target sub-region based on the target noise reduction intensity, performing noise statistics on the noise-reduced target image, and determining the noise level of the skin color region based on the statistical results;

Determine whether the noise level belongs to a preset noise interval;

If yes, output the target image after noise reduction processing;

Otherwise, a mixing weight is determined according to the noise level, and the target image and the noise-reduced target image are mixed based on the mixing weight, and the mixed-processed target image is output.

Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform an image noise reduction method, the method includes:

A local noise reduction event is detected and triggered;

Determine the brightness information of the skin color region in the target image, where the brightness information includes the average brightness, the maximum brightness, and the minimum brightness;

Determining a target sub-region in the skin color region whose brightness is lower than the average brightness, and determining a target noise reduction intensity according to the brightness of the target sub-region and the brightness information;

Storage medium-any kind of memory device or storage device of various types. The term "storage media" is intended to include: installation media such as CD-ROM, floppy disk or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc. ; Non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements. The storage medium may also include other types of memory or a combination thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system through a network such as the Internet. The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (eg, in different computer systems connected through a network). The storage medium may store program instructions executable by one or more processors (eg, embodied as a computer program).

Of course, a storage medium containing computer-executable instructions provided by the embodiments of the present application is not limited to the image noise reduction operation described above, but can also perform the image noise reduction provided by any embodiment of the present application Related operations in the method.

An embodiment of the present application provides a terminal, and an image noise reduction device provided by an embodiment of the present application may be integrated in the terminal. 6 is a schematic structural diagram of a terminal according to an embodiment of the present application. As shown in FIG. 6, the terminal includes a memory 610 and a processor 620. The memory 610 is used to store a computer program; the processor 620 reads and executes the computer program stored in the memory 610. The processor 620 implements the following steps when executing the computer program: determining the brightness information of the skin color area in the target image; determining the target sub-area in the skin color area whose brightness is lower than a preset brightness domain value, according to the target sub The brightness of the area and the brightness information determine a target noise reduction intensity; perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.

The memory and processor listed in the above examples are all components of the terminal, and the terminal may also include other components. Take a smart phone as an example to illustrate the possible structure of the above terminal. 7 is a structural block diagram of a smartphone provided by an embodiment of the present application. As shown in FIG. 7, the smart phone may include: a memory 701, a central processing unit (Central Processing Unit, CPU) 702 (also called a processor, hereinafter referred to as CPU), a peripheral interface 703, and an RF (Radio Frequency) circuit 705, audio circuit 706, speaker 711, touch screen 712, power management chip 708, input/output (I/O) subsystem 709, other input/control devices 710, and external ports 704, these components through one or more communication buses or The signal line 707 comes to communicate.

It should be understood that the illustrated smartphone 700 is only an example of a terminal, and the smartphone 700 may have more or fewer parts than shown in the figure, and two or more parts may be combined, or There can be different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The smart phone integrated with the image noise reduction device provided in this embodiment will be described in detail below.

Memory 701, which can be accessed by CPU 702, peripheral interface 703, etc. The memory 701 can include high-speed random access memory, and can also include non-volatile memory, such as one or more disk storage devices, flash memory devices , Or other volatile solid-state storage devices.

Peripheral interface 703, which can connect input and output peripherals of the device to CPU 702 and memory 701.

I/O subsystem 709, which can connect input and output peripherals on the device, such as touch screen 712 and other input/control devices 710, to peripheral interface 703. The I/O subsystem 709 may include a display controller 7091 and one or more input controllers 7092 for controlling other input/control devices 710. Among them, one or more input controllers 7092 receive electrical signals from other input/control devices 710 or send electrical signals to other input/control devices 710, which may include physical buttons (press buttons, rocker buttons, etc.) ), dial pad, slide switch, joystick, click wheel. It is worth noting that the input controller 7092 can be connected to any of the following: a keyboard, an infrared port, a USB interface, and a pointing device such as a mouse.

A touch screen 712, which is an input interface and an output interface between the user terminal and the user, and displays visual output to the user, and the visual output may include graphics, text, icons, video, and the like.

The display controller 7091 in the I/O subsystem 709 receives electrical signals from the touch screen 712 or sends electrical signals to the touch screen 712. The touch screen 712 detects the contact on the touch screen, and the display controller 7091 converts the detected contact into interaction with the user interface object displayed on the touch screen 712, that is, realizes human-computer interaction, and the user interface object displayed on the touch screen 712 may be running Icons for games, icons connected to the corresponding network, etc. It is worth noting that the device may also include a light mouse, which is a touch-sensitive surface that does not display visual output, or an extension of the touch-sensitive surface formed by a touch screen.

The RF circuit 705 is mainly used to establish communication between the mobile phone and the wireless network (that is, the network side), and realize data reception and transmission between the mobile phone and the wireless network. For example, sending and receiving short messages, e-mail, etc. Specifically, the RF circuit 705 receives and transmits RF signals, which are also called electromagnetic signals. The RF circuit 705 converts electrical signals into electromagnetic signals or converts electromagnetic signals into electrical signals, and communicates with the communication network and other devices through the electromagnetic signals Communicate. The RF circuit 705 may include known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC ( COder-DECoder (codec) chipset, subscriber identity module (Subscriber Identity Module, SIM), etc.

The audio circuit 706 is mainly used to receive audio data from the peripheral interface 703, convert the audio data into electrical signals, and send the electrical signals to the speaker 711.

The speaker 711 is used to restore the voice signal received by the mobile phone from the wireless network through the RF circuit 705 to a sound and play the sound to the user.

The power management chip 708 is used for power supply and power management for the hardware connected to the CPU 702, the I/O subsystem, and the peripheral interface.

The terminal provided in the embodiment of the present application may determine the target sub-region to be subjected to noise reduction processing based on the brightness, and determine the noise reduction intensity according to the brightness of each pixel in the target sub-region and the maximum and minimum brightness values of the skin-tone area, Based on the intensity of noise reduction, the corresponding pixels are subjected to noise reduction processing to achieve the local noise reduction effect on the skin color area based on brightness, so that the noise distribution of the skin color area is more uniform.

The image noise reduction device, storage medium, and terminal provided in the above embodiments can execute the image noise reduction method provided in any embodiment of the present application, and have corresponding function modules and beneficial effects for performing the method. For technical details that are not described in detail in the above embodiments, please refer to the image noise reduction method provided in any embodiment of the present application.

Note that the above are only the preferred embodiments of the present application and the applied technical principles. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the concept of the present application. The scope is determined by the scope of the appended claims.

Claims

An image noise reduction method, characterized in that it includes:

Determine the brightness information of the skin color area in the target image;

Determining a target sub-area in the skin-color area whose brightness is lower than a preset brightness threshold, and determining a target noise reduction intensity according to the brightness of the target sub-area and the brightness information;

Perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.
The method of claim 1, before determining the brightness information of the skin color region in the target image, further comprising:

Obtain a target image in a color-separated color mode, perform face recognition on the target image, and determine the face information contained in the target image, where the face information includes the number of faces, and faces, eyebrows, and eyes , Contour information of nose and mouth;

Determine the facial skin area of the target image according to the outline information;

Acquiring the brightness and color of each pixel in the facial skin area, and determining an investigation area in the target image according to the brightness and color, wherein the pixels in the investigation area and the facial skin The deviation of pixels in the area in brightness and color is less than the set threshold;

The investigation area and the facial skin area constitute a skin color area.
The method according to claim 2, wherein the acquiring the target image of the color separation mode includes:

Determine whether the acquired target image is the color separation mode of color and light;

When the acquired target image is not the color separation mode, the target image is subjected to mode conversion to generate the target image in the color separation mode.
The method according to claim 2, characterized in that after the skin color area is formed by the investigation area and the facial skin area, the method further comprises:

Perform noise statistics on the target image, and determine the noise level of the skin area and the remaining areas of the target image except the skin area based on the statistical results;

When the noise level of the skin color area is higher than the noise level of the remaining areas in the target image except the skin color area, a local noise reduction event is triggered, where the local noise reduction event is used to indicate the determination of the skin color in the target image The operation of the brightness information of the area is performed.
The method according to claim 4, characterized in that after performing noise statistics on the target image and determining the noise level of the remaining area other than the skin color area in the skin color area and the target image based on the statistical result, further comprising :

Determining an overall noise reduction intensity for performing overall noise reduction processing on the target image according to the noise level, and performing overall noise reduction processing on the target image based on the overall noise reduction intensity.
The method according to claim 1, wherein a target sub-area of the skin-color area whose brightness is lower than a preset brightness threshold is determined, and a target noise reduction is determined according to the brightness of the target sub-area and the brightness information Strength, including:

Comparing the brightness of each pixel in the skin color area with a preset brightness threshold, marking target pixels whose brightness is lower than the preset brightness threshold, and clustering the target pixels into at least one target sub-region;

The target noise reduction intensity based on the brightness is determined according to the brightness, average brightness value, maximum brightness value and minimum brightness value of the target pixel in each target sub-region.
The method according to claim 6, wherein before performing noise reduction processing on the target sub-region based on the target noise reduction intensity, further comprising:

When the target image contains at least two human faces, obtain the color of the skin color area, and determine the first color mean, maximum color value, and minimum color value of the skin color area in the target image according to the color;

Calculate the second color average of the skin color area corresponding to each face separately;

For the target skin color area where the second color average value is less than the first color average value, a color-based target drop is determined according to the color of the target skin color area, the first color average value, the maximum color value, and the minimum color value Noise strength.
The method according to claim 7, characterized in that after determining the color-based target noise reduction intensity according to the color of the target skin color region, the first color mean, the maximum color value and the minimum color value, include:

A weighted operation result of the brightness-based target noise reduction intensity and the color-based target noise reduction intensity is determined, and the weighted calculation result is used as the target noise reduction intensity.
The method according to any one of claims 1 to 8, wherein after performing noise reduction processing on the target sub-region based on the target noise reduction intensity, further comprising:

Perform noise statistics on the target image after noise reduction processing, and determine the noise level of the skin color region based on the statistical results;

Determine whether the noise level belongs to a preset noise interval;

If yes, output the target image after noise reduction processing;

Otherwise, a mixing weight is determined according to the noise level, and the target image and the noise-reduced target image are mixed based on the mixing weight, and the mixed-processed target image is output.
An image noise reduction device, characterized in that it includes:

The information determination module is used to determine the brightness information of the skin color area in the target image;

A noise reduction intensity determination module, configured to determine a target sub-region with a brightness lower than a preset brightness threshold in the skin-color region, and determine a target noise reduction intensity according to the brightness of the target sub-region and the brightness information;

The noise reduction processing module is configured to perform noise reduction processing on the target sub-region based on the target noise reduction intensity to obtain a target image after noise reduction processing.
The device of claim 10, wherein the device further comprises:

The face information determination module is used to obtain a target image with separated color and light color mode, perform face recognition on the target image, and determine the face information contained in the target image, wherein the face information includes the number of faces , And contour information for face, eyebrows, eyes, nose and mouth;

A facial skin area determining module, configured to determine the facial skin area of the target image according to the outline information;

The investigation area determination module is used to obtain the brightness and color of each pixel in the facial skin area, and determine the investigation area in the target image according to the brightness and color, wherein the pixels in the investigation area The deviation of the pixels from the pixels in the facial skin area in brightness and color is less than the set threshold;

The skin color area constituting module is used for the investigation area and the facial skin area to constitute a skin color area.
The device according to claim 11, wherein the face information determination module is specifically configured to:

Determine whether the acquired target image is the color separation mode of color and light;

When the acquired target image is not the color separation mode, the mode conversion is performed on the target image to generate the target image in the color separation mode.

Perform face recognition on the target image to determine the face information contained in the target image, where the face information includes the number of faces, and contour information of the face, eyebrows, eyes, nose, and mouth.
The device according to claim 11, wherein the device further comprises:

A noise level determining module, configured to perform noise statistics on the target image, and determine the noise levels of the skin area and the remaining areas of the target image except the skin area based on the statistical results;

A local noise reduction event triggering module is used to trigger a local noise reduction event when the noise level of the skin color area is higher than the noise level of the remaining areas except the skin color area in the target image, wherein the local noise reduction The event is used to indicate the execution of the operation to determine the brightness information of the skin color region in the target image.
The device according to claim 13, wherein the local noise reduction event triggering module is specifically configured to:

Determining an overall noise reduction intensity for performing overall noise reduction processing on the target image according to the noise level, and performing overall noise reduction processing on the target image based on the overall noise reduction intensity.
The device according to claim 10, wherein the noise reduction intensity determination module is specifically configured to:

Comparing the brightness of each pixel in the skin color area with a preset brightness threshold, marking target pixels whose brightness is lower than the preset brightness threshold, and clustering the target pixels into at least one target sub-region;

The target noise reduction intensity based on the brightness is determined according to the brightness, average brightness value, maximum brightness value and minimum brightness value of the target pixel in each target sub-region.
The device according to claim 15, wherein the device further comprises:

The color data determination module is used to obtain the color of the skin color area when the target image contains at least two human faces, and determine the first color mean, maximum color value and minimum color value of the skin color area in the target image according to the color ; Calculate the second color mean of the skin color area corresponding to each face separately;

The noise reduction intensity determination module is specifically used for:

For the target skin color area where the second color average value is less than the first color average value, a color-based target drop is determined according to the color of the target skin color area, the first color average value, the maximum color value, and the minimum color value Noise strength.
The apparatus according to claim 16, wherein the noise reduction intensity determination module is specifically configured to:

A weighted operation result of the brightness-based target noise reduction intensity and the color-based target noise reduction intensity is determined, and the weighted calculation result is used as the target noise reduction intensity.
The device according to claims 10-17, characterized in that the device further comprises:

The noise level judgment module is used to perform noise statistics on the target image after noise reduction processing, determine the noise level of the skin color area based on the statistical result, and determine whether the noise level belongs to a preset noise interval;

The target image output module is used to output the target image after noise reduction processing if the noise level belongs to the preset noise interval; if the noise level does not belong to the preset noise interval, determine the mixing weight according to the noise level, based on the mixing weight The target image and the noise-reduced target image are mixed, and the mixed target image is output.
A computer-readable storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the image noise reduction method according to any one of claims 1-9 is implemented.
A terminal, characterized in that it includes a memory, a processor, and a computer program stored on the memory and executable by the processor, and when the processor executes the computer program, any one of claims 1-9 is implemented Image noise reduction method.