Disclosure of Invention
The invention aims to solve the technical problem of providing an image spatial domain noise reduction method and device, which can be beneficial to improving the definition of an image after noise reduction and ensuring the overall noise reduction effect of the image.
In order to solve the technical problem, a first aspect of the present invention discloses an image spatial domain noise reduction method, which includes:
Acquiring an image to be denoised, and establishing a multi-scale pyramid of the image to be denoised, wherein the multi-scale pyramid comprises a Gaussian pyramid and a Laplacian pyramid;
constructing a directional filtering pyramid of the image to be noise reduced according to the Gaussian pyramid;
determining point information corresponding to each pixel point in the direction filtering pyramid, wherein the point information comprises category information corresponding to the pixel point, and the category information is used for indicating that the pixel point is a flat point or an edge texture point;
non-local mean filtering is carried out on all the pixel points in each layer of the Laplacian pyramid point by point to obtain a filtering result of each pixel point in each layer of the Laplacian pyramid, and the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid is determined;
performing the non-local mean filtering on each edge texture point in each layer of the direction filtering pyramid point by point to obtain a filtering result of each edge texture point in each layer of the direction filtering pyramid, and determining the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid;
and performing fusion reconstruction operation according to the classification of each pixel point in the direction filtering pyramid, the filtering result of each pixel point in each layer of the Laplacian pyramid, the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, the filtering result of each edge texture point in each layer of the direction filtering pyramid and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, so as to obtain a noise-reduced image corresponding to the image to be noise-reduced.
As an optional implementation manner, in the first aspect of the present invention, the constructing a directional filtering pyramid of the image to be noise reduced according to the gaussian pyramid includes:
obtaining each direction filtering result of the direction filtering pyramid through the middle result of the Gaussian pyramid;
constructing a residual diagram of the direction filtering pyramid according to each direction filtering result of the direction filtering pyramid;
wherein, each direction filtering result of the direction filtering pyramid is expressed as:
OF l,m =I l *f m ,
wherein OF l,m Filtering the filtering result of the pixel point in the first layer of the pyramid in the mth direction, I l Is the image of the first layer, i is the image layer number, f m A direction filter corresponding to the m-th direction is an image convolution operation;
and, the residual map of the direction filtering pyramid is expressed as:
wherein OR is l,m Filtering the residue of the pyramid for the directionThe difference graph, upsampled () is the operator of the upsampling operation.
In an optional implementation manner, in the first aspect of the present invention, for each pixel point in the direction filtering pyramid, if the pixel point is the edge texture point, the point information corresponding to the pixel point further includes main direction information of the pixel point and edge intensity information of the pixel point; if the pixel point is the flat point, the point information corresponding to the pixel point further comprises edge intensity information of the pixel point;
The determining the point information corresponding to each pixel point in the direction filtering pyramid includes:
comparing residual values of the pixel points in residual graphs of different directions of the direction filtering pyramid to obtain a maximum residual value corresponding to the pixel points;
comparing the maximum residual value with a preset edge low threshold value:
when the maximum residual value is smaller than the preset edge low threshold value, judging that the category of the pixel point is used for indicating that the pixel point is a flat point;
when the maximum residual error value is greater than or equal to the preset edge low threshold value, judging that the category of the pixel point is used for indicating that the pixel point is an edge texture point, and determining main direction information of the edge texture point, wherein the main direction information is a direction corresponding to the maximum residual error value;
and determining the edge intensity information of the pixel point according to the maximum residual value and a preset edge high threshold value.
As an optional implementation manner, in the first aspect of the present invention, a calculation formula of the edge intensity information of the pixel point is:
wherein, edge is provided str Edge intensity information for the pixel point Th_hi is the preset edge high threshold, mx_OR l And the maximum residual value corresponding to the pixel point is obtained.
In an optional implementation manner, in a first aspect of the present invention, the performing the non-local mean filtering on each of the edge texture points in each layer of the directional filtering pyramid point by point includes:
and selecting a residual image point-by-point in the direction filtering pyramid according to the main direction information corresponding to the edge texture points for all the edge texture points in each layer of the direction filtering pyramid to carry out the non-local mean filtering.
In an optional implementation manner, in a first aspect of the present invention, the performing a fusion reconstruction operation according to a classification of each pixel point in the direction filtering pyramid, a filtering result of each pixel point in each layer of the laplacian pyramid, a number of similar points corresponding to each pixel point in each layer of the laplacian pyramid, a filtering result of each edge texture point in each layer of the direction filtering pyramid, and a number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, to obtain a denoised image corresponding to the denoised image, where the performing includes:
For each flat point in the direction filtering pyramid, accumulating filtering results of target pixel points matched with the flat point positions in each layer of the Laplacian pyramid layer by layer according to structures of the Gaussian pyramid and the Laplacian pyramid, and reconstructing noise reduction results corresponding to the flat points;
for each pair of edge texture points in the direction filtering pyramid, according to the number of similar points corresponding to each pixel point in each layer of the laplace pyramid, the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid and the determined edge intensity information of each pixel point in the direction filtering pyramid, fusing a filtering result of each pixel point matched with the position of the edge texture point in each layer of the laplace pyramid with a filtering result of each edge texture point in each layer of the direction filtering pyramid, and reconstructing a noise reduction result corresponding to the edge texture point;
and integrating the denoising result corresponding to the flat point and the denoising result corresponding to the edge texture point to obtain a denoising image corresponding to the image to be denoised.
In an optional implementation manner, in the first aspect of the present invention, the formula of the reconstructed noise reduction result corresponding to the edge texture point is:
wherein out is the noise reduction result corresponding to the edge texture point, count or For the number of similar points corresponding to each edge texture point of each layer of the direction filtering pyramid, f or For each filtering result of each edge texture point in each layer of the direction filtering pyramid, count lp For the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, f lp For the filtering result of each pixel point in each layer of the Laplacian pyramid, edge str And l is the number of image layers for determining the edge intensity information of all pixel points in the direction filtering pyramid.
The second aspect of the invention discloses an image spatial domain noise reduction device, which comprises:
the acquisition module is used for acquiring an image to be denoised and establishing a multi-scale pyramid of the image to be denoised, wherein the multi-scale pyramid comprises a Gaussian pyramid and a Laplacian pyramid;
the construction module is used for constructing a directional filtering pyramid of the image to be noise reduced according to the Gaussian pyramid;
The determining module is used for determining point information corresponding to each pixel point in the direction filtering pyramid, wherein the point information comprises category information corresponding to the pixel point, and the category information is used for indicating that the pixel point is a flat point or an edge texture point;
the filtering module is used for carrying out non-local mean filtering on all the pixel points in each layer of the Laplacian pyramid point by point to obtain a filtering result of each pixel point in each layer of the Laplacian pyramid, and determining the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid;
the filtering module is further configured to perform the non-local mean filtering on each edge texture point in each layer of the direction filtering pyramid point by point to obtain a filtering result of each edge texture point in each layer of the direction filtering pyramid, and determine the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid;
and the fusion reconstruction module is used for executing fusion reconstruction operation according to the classification of each pixel point in the direction filtering pyramid, the filtering result of each pixel point in each layer of the laplacian pyramid, the number of similar points corresponding to each pixel point in each layer of the laplacian pyramid, the filtering result of each edge texture point in each layer of the direction filtering pyramid and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, so as to obtain a noise-reduced image corresponding to the image to be noise reduced.
In a second aspect of the present invention, a specific implementation manner of the constructing module according to the gaussian pyramid to construct the direction filtering pyramid of the image to be noise reduced is:
obtaining each direction filtering result of the direction filtering pyramid through the middle result of the Gaussian pyramid;
constructing a residual diagram of the direction filtering pyramid according to each direction filtering result of the direction filtering pyramid;
wherein, each direction filtering result of the direction filtering pyramid is expressed as:
OF l,m =I l *f m ,
wherein OF l,m Filtering the filtering result of the pixel point in the first layer of the pyramid in the mth direction, I l For the picture of the first layer, i is the number of picture layers,f m a direction filter corresponding to the m-th direction is an image convolution operation;
and, the residual map of the direction filtering pyramid is expressed as:
wherein OR is l,m For the residual map of the directional filter pyramid, upsample () is the operator of the upsampling operation.
In a second aspect of the present invention, for each of the pixel points in the direction filtering pyramid, if the pixel point is the edge texture point, the point information corresponding to the pixel point further includes main direction information of the pixel point and edge intensity information of the pixel point; if the pixel point is the flat point, the point information corresponding to the pixel point further comprises edge intensity information of the pixel point;
The specific implementation mode of the determining module for determining the point information corresponding to each pixel point in the direction filtering pyramid is as follows:
comparing residual values of the pixel points in residual graphs of different directions of the direction filtering pyramid to obtain a maximum residual value corresponding to the pixel points;
comparing the maximum residual value with a preset edge low threshold value:
when the maximum residual value is smaller than the preset edge low threshold value, judging that the category of the pixel point is used for indicating that the pixel point is a flat point;
when the maximum residual error value is greater than or equal to the preset edge low threshold value, judging that the category of the pixel point is used for indicating that the pixel point is an edge texture point, and determining main direction information of the edge texture point, wherein the main direction information is a direction corresponding to the maximum residual error value;
and determining the edge intensity information of the pixel point according to the maximum residual value and a preset edge high threshold value.
As an optional implementation manner, in the second aspect of the present invention, a calculation formula of the edge intensity information of the pixel point is:
wherein, edge is provided str For the edge intensity information of the pixel, th_hi is the preset edge high threshold, mx_or l And the maximum residual value corresponding to the pixel point is obtained.
In a second aspect of the present invention, a specific implementation manner of the filtering module for performing the non-local mean filtering on each edge texture point in each layer of the directional filtering pyramid point by point is:
and selecting a residual image point-by-point in the direction filtering pyramid according to the main direction information corresponding to the edge texture points for all the edge texture points in each layer of the direction filtering pyramid to carry out the non-local mean filtering.
In a second aspect of the present invention, the fusion reconstruction module performs a fusion reconstruction operation according to a classification of each pixel point in the direction filtering pyramid, a filtering result of each pixel point in each layer of the laplacian pyramid, a number of similar points corresponding to each pixel point in each layer of the laplacian pyramid, a filtering result of each edge texture point in each layer of the direction filtering pyramid, and a number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, so as to obtain a denoised image corresponding to the image to be denoised, where the specific embodiment of obtaining the denoised image corresponding to the image to be denoised is:
For each flat point in the direction filtering pyramid, accumulating filtering results of target pixel points matched with the flat point positions in each layer of the Laplacian pyramid layer by layer according to structures of the Gaussian pyramid and the Laplacian pyramid, and reconstructing noise reduction results corresponding to the flat points;
for each pair of edge texture points in the direction filtering pyramid, according to the number of similar points corresponding to each pixel point in each layer of the laplace pyramid, the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid and the determined edge intensity information of each pixel point in the direction filtering pyramid, fusing a filtering result of each pixel point matched with the position of the edge texture point in each layer of the laplace pyramid with a filtering result of each edge texture point in each layer of the direction filtering pyramid, and reconstructing a noise reduction result corresponding to the edge texture point;
and integrating the denoising result corresponding to the flat point and the denoising result corresponding to the edge texture point to obtain a denoising image corresponding to the image to be denoised.
As an optional implementation manner, in the second aspect of the present invention, the formula of the reconstructed noise reduction result corresponding to the edge texture point is:
wherein out is the noise reduction result corresponding to the edge texture point, count or For the number of similar points corresponding to each edge texture point of each layer of the direction filtering pyramid, f or For each filtering result of each edge texture point in each layer of the direction filtering pyramid, count lp For the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, f lp For the filtering result of each pixel point in each layer of the Laplacian pyramid, edge str And l is the number of image layers for determining the edge intensity information of all pixel points in the direction filtering pyramid.
In a third aspect, the present invention discloses another image spatial domain noise reduction device, which includes:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory to execute the image spatial domain noise reduction method disclosed in the first aspect of the present invention.
A fourth aspect of the present invention discloses a computer storage medium storing computer instructions for performing the image spatial domain noise reduction method disclosed in the first aspect of the present invention when the computer instructions are called.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, a multi-scale pyramid is established according to the image to be reduced, a directional filtering pyramid is established by utilizing the middle result of the Gaussian pyramid, and fusion reconstruction is carried out on the image to be reduced according to the non-local mean filtering result of each pyramid. Therefore, the invention can effectively avoid the situation of insufficient scene adaptability by utilizing the multi-scale pyramid, and simultaneously the problem that the texture edge of the image is excessively smoothed or the edge color noise residue is generated after noise reduction is effectively avoided by adding the directional filtering pyramid, so that the overall definition is ensured while the noise reduction effect is ensured, and the efficiency and the accuracy of the image noise reduction work are improved.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or article.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The invention discloses an image space domain noise reduction method and device, which can be beneficial to improving the definition of an image after noise reduction and ensure the overall noise reduction effect of the image. The following will describe in detail.
Example 1
Referring to fig. 1, fig. 1 is a flowchart of an image spatial domain noise reduction method according to an embodiment of the present invention. The image spatial domain noise reduction method described in fig. 1 may be applied to an image processing system, where the image processing system may include any one of an image processing device, an intelligent server for controlling the image processing device, or an intelligent platform, where the image processing device may include an image noise reduction device, and the intelligent server includes a local server or a cloud server, and the image processing device may be capable of establishing a communication connection with a user device, where the user device includes, but is not limited to, at least one of a smart phone (Android mobile phone, iOS mobile phone, etc.), a smart phone number watch, a tablet computer, a palm computer, a vehicle-mounted computer, a desktop computer, a netbook, a personal digital assistant (Personal Digital Assistant, PDA), an intelligent navigator, and a mobile internet device (Mobile Internet Devices, MID), and the embodiment of the present invention is not limited. As shown in fig. 1, the spatial domain noise reduction method for an image may include the following operations:
101. Acquiring an image to be denoised, and establishing a multi-scale pyramid of the image to be denoised, wherein the multi-scale pyramid comprises a Gaussian pyramid and a Laplacian pyramid.
In the embodiment of the invention, a multi-scale pyramid of an image to be noise-reduced is obtained through a series of Gaussian smoothing and downsampling operations, wherein the Gaussian pyramid contains low-frequency characteristics of the image to be noise-reduced, and the Laplacian pyramid contains more noise of the image to be noise-reduced and high-frequency information of partial edge textures.
In the embodiment of the invention, optionally, the multi-scale pyramid may further include a DOG pyramid, a wavelet or a curved wave, etc. to replace the existing pyramid, and the invention is not limited.
In the embodiment of the invention, the Gaussian low-pass filters with different scales (the layer number is n, and n is more than or equal to 2) are convolved with the image to obtain a multi-level Gaussian pyramid and a Laplacian pyramid. For example, given image I, an n-layer Gaussian pyramid { G ] of the image is constructed l The following are noted:
wherein, is the image convolution operation, l e 0,1,., n, downsampled (·) is the downsampling operation of the image; laplacian pyramid { L l The calculation process of } is as follows:
where, upsample (·) is an operator that uses gaussian kernels to upsample the image (double the image size) on each layer, it is known from the equation that the top layer of the gaussian pyramid (i=n) coincides with the top layer of the laplacian pyramid.
102. And constructing a directional filtering pyramid of the image to be noise reduced according to the Gaussian pyramid.
In the embodiment of the invention, the intermediate result of the Gaussian pyramid is utilized to obtain the result of each directional filtering, meanwhile, the operation of the Laplacian pyramid is imitated, the filtering result of each layer of directional filtering pyramid is subtracted from the previous layer of input original image, and the high-frequency residual error of each layer of directional filtering is obtained. The Gaussian filtering has isotropic characteristics, but the Gaussian filtering comprises filtering processing for each direction, and the filtering result for each direction can be simply obtained by carrying out certain buffer output processing on the intermediate result while obtaining the Gaussian pyramid filtering result for each layer. Taking a four-way filter as an example, the expression is as follows:
the filtering result OF OF each direction in the first layer direction filtering pyramid l,m Residual map OR l,m The calculation process of (2) is as follows:
103. and determining point information corresponding to each pixel point in the direction filtering pyramid, wherein the point information comprises category information corresponding to the pixel point, and the category information is used for indicating that the pixel point is a flat point or an edge texture point.
In the embodiment of the invention, each pixel point in each layer of the statistical direction filtering pyramid filters the maximum value of the residual image in different directions, compares the maximum value with a preset edge low threshold, marks the pixel point lower than the edge low threshold as a flat point, marks the pixel point higher than the edge low threshold as an edge texture point, and marks the direction of the maximum value of the residual as the main direction of the pixel point, wherein the edge low threshold can be a preset fixed threshold or a self-adaptive threshold, such as a threshold calculated according to information such as a noise curve and/or gradient difference, and can be selected according to practical conditions.
104. Non-local mean filtering is carried out on all the pixel points in each layer of the Laplacian pyramid point by point to obtain a filtering result of each pixel point in each layer of the Laplacian pyramid, and the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid is determined.
In the embodiment of the invention, NLM (non-local mean filtering) filtering is performed on all pixel points of each layer in a Laplacian pyramid, a gray value of a pixel point i is assumed to be f (i), a central window P (i) with a window size of r and a search window with a window size of t (t > s) are respectively extracted by taking the pixel point as the center, a pixel point j is selected through the search area, the pixel point j is taken as the center, a window P (j) to be matched with the same size as the central window is selected, and if the difference value between the sum of gray values of a plurality of pixel points in the P (j) and the sum of gray values of a plurality of pixel points in the P (i) is smaller than a preset similarity threshold, the similarity between the window to be matched and the central window is considered to be higher, and the pixel point j is marked as a similarity point.
In the embodiment of the present invention, first, the gaussian weighted euclidean distance d (i, j) between the center window P (i) and the similar window P (j) is calculated as follows:
d(i,j)=gs 1 *P(i)-P(j) 2 ,
next, the weight coefficients w (i, j) of the center window P (i) and the similar window P (j) are calculated as follows:
Wherein h is used to control the smoothness, gs 1 Is a preset Gaussian function;
then, counting the number of similar points in the search window, and carrying out weighted average operation on the similar points, wherein the specific formula is as follows:
and finally, traversing each layer of the Laplacian pyramid to carry out NLM filtering, and recording the filtering result of each pixel point after each layer of filtering and the number of corresponding similar points.
105. Non-local mean value filtering is carried out on each edge texture point in each layer of the direction filtering pyramid point by point to obtain a filtering result of each edge texture point in each layer of the direction filtering pyramid, and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid is determined.
In the embodiment of the invention, for each edge texture point in each layer of the direction filtering pyramid, a residual image corresponding to the main direction of the edge texture point is selected, and NLM filtering is performed on the edge texture point, wherein, optionally, according to preset requirements or actual conditions, the residual images corresponding to other directions of the edge texture point are selected to perform NLM filtering, and a four-direction filter is taken as an example, the maximum value of the edge texture point in four directions of residual images is counted, and the direction is defined as the main direction with the most obvious characteristic, and simultaneously, the NLM filtering can be performed on the residual image corresponding to the main direction or the residual image corresponding to the other three directions.
106. And performing fusion reconstruction operation according to the classification of each pixel point in the direction filtering pyramid, the filtering result of each pixel point in each layer of the Laplacian pyramid, the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, the filtering result of each edge texture point in each layer of the direction filtering pyramid and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, so as to obtain a noise-reduced image corresponding to the image to be noise-reduced.
In the embodiment of the invention, for the flat points in the direction filtering pyramid, the results after NLM filtering of each layer can be accumulated layer by layer directly according to the structures of the Gaussian pyramid and the Laplacian pyramid, and the noise-reduced results are reconstructed; for the edge texture points in the direction filtering pyramid, the edge texture points of each layer can be reconstructed after being fused on NLM filtering results of the direction filtering pyramid and the Laplacian pyramid according to the number of similar points corresponding to the edge texture points and the edge intensity value.
Therefore, the image spatial domain noise reduction method described by the embodiment of the invention can establish a multi-scale pyramid according to the image to be noise reduced, establish a direction filtering pyramid by using the middle result of the Gaussian pyramid, and perform fusion reconstruction on the image to be noise reduced according to the non-local mean filtering result of each pyramid, so that the situation of insufficient scene adaptability is effectively avoided by utilizing the multi-scale pyramid, meanwhile, the problem that texture edges of the image are excessively smoothed or edge color noise residues are generated after noise reduction is effectively avoided by adding the direction filtering pyramid, the noise reduction effect is ensured, the overall definition is considered, and the efficiency and the accuracy of the image noise reduction work are improved.
In an optional embodiment, for each pixel point in the direction filtering pyramid, if the pixel point is an edge texture point, the point information corresponding to the pixel point further includes main direction information of the pixel point and edge intensity information of the pixel point; if the pixel points are flat points, the point information corresponding to the pixel points also comprises edge intensity information of the pixel points;
the determining the point information corresponding to each pixel point in the direction filtering pyramid may include the following operations:
comparing residual values of the pixel points in residual graphs of different directions of the directional filtering pyramid to obtain a maximum residual value corresponding to the pixel points;
comparing the maximum residual value with a preset edge low threshold value:
when the maximum residual error value is smaller than a preset edge low threshold value, judging that the category of the pixel points is used for indicating that the pixel points are flat points;
when the maximum residual error value is greater than or equal to a preset edge low threshold value, judging that the category of the pixel point is used for indicating that the pixel point is an edge texture point, and determining main direction information of the edge texture point, wherein the main direction information is the direction corresponding to the maximum residual error value;
and determining the edge intensity information of the pixel point according to the maximum residual value and a preset edge high threshold value.
In this optional embodiment, each pixel point in each layer of the statistical direction filtering pyramid filters the maximum value of the residual image in different directions, compares the maximum value with a preset edge low threshold, marks a pixel point lower than the edge low threshold as a flat point, marks a pixel point higher than the edge low threshold as an edge texture point, and marks the direction of the maximum value of the residual as the main direction of the pixel point, where the edge low threshold may be a preset fixed threshold or an adaptive threshold, for example, a threshold calculated according to information such as a noise curve and/or a gradient difference, and may be selected according to practical situations.
In this alternative embodiment, the formula of the maximum value of the filtering residual map of each pixel point in each layer of the statistical direction filtering pyramid in different directions may be:
mx_OR l =MAX(OR l,0 ,OR l,1 ,...,OR l,m ),
the edge texture point and the flat point may be determined by the following formula:
the formula for calculating the edge intensity of the pixel point by the phase of the edge high threshold value can be:
it can be seen that implementing this alternative embodiment can compare, for each pixel point in the direction filtering pyramid, residual values of the pixel point in residual maps of different directions of the direction filtering pyramid, to obtain a maximum residual value corresponding to the pixel point, and compare the maximum residual value with a preset edge low threshold value: when the maximum residual value is smaller than the preset edge low threshold value, the category of the pixel point is judged to be used for representing the pixel point as a flat point, when the maximum residual value is larger than or equal to the preset edge low threshold value, the category of the pixel point is judged to be used for representing the pixel point as an edge texture point, main direction information of the edge texture point is determined, the main direction information is the direction corresponding to the maximum residual value, the edge intensity information of the pixel point is determined according to the maximum residual value and the preset edge high threshold value, the definition of the image after noise reduction is improved, and the problem that the texture edge is excessively smoothed or the edge color noise remains in the image after noise reduction is effectively avoided.
Example two
Referring to fig. 2, fig. 2 is a flowchart illustrating another image spatial domain noise reduction method according to an embodiment of the invention. The image spatial domain noise reduction method described in fig. 2 may be applied to an image processing system, where the image processing system may include any one of an image processing device, an intelligent server for controlling the image processing device, or an intelligent platform, where the image processing device may include an image noise reduction device, and the intelligent server includes a local server or a cloud server, and the image processing device may be capable of establishing a communication connection with a user device, where the user device includes, but is not limited to, at least one of a smart phone (Android mobile phone, iOS mobile phone, etc.), a smart phone number watch, a tablet computer, a palm computer, a vehicle-mounted computer, a desktop computer, a netbook, a personal digital assistant (Personal Digital Assistant, PDA), an intelligent navigator, and a mobile internet device (Mobile Internet Devices, MID), and the embodiment of the present invention is not limited. As shown in fig. 2, the spatial domain noise reduction method for an image may include the following operations:
201. acquiring an image to be denoised, and establishing a multi-scale pyramid of the image to be denoised, wherein the multi-scale pyramid comprises a Gaussian pyramid and a Laplacian pyramid.
202. And obtaining each direction filtering result of the direction filtering pyramid through the intermediate result of the Gaussian pyramid.
In the embodiment of the invention, the intermediate results of the Gaussian pyramid are utilized to obtain the results of directional filtering, the directional filtering pyramid is the same as the Gaussian Laplacian pyramid and comprises a low-frequency filtering result and a residual diagram, wherein the filtering result is obtained by caching and calculating some intermediate results of the Gaussian pyramid of the current layer.
203. And constructing a residual diagram of the directional filter pyramid according to each directional filter result of the directional filter pyramid.
In the embodiment of the invention, the operation of the Laplacian pyramid is imitated, the filtering result of the filtering pyramid in each layer direction is subtracted from the input original image of the previous layer, and the high-frequency residual error of the filtering in each layer direction is obtained.
204. And determining point information corresponding to each pixel point in the direction filtering pyramid, wherein the point information comprises category information corresponding to the pixel point, and the category information is used for indicating that the pixel point is a flat point or an edge texture point.
205. Non-local mean filtering is carried out on all the pixel points in each layer of the Laplacian pyramid point by point to obtain a filtering result of each pixel point in each layer of the Laplacian pyramid, and the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid is determined.
206. Non-local mean value filtering is carried out on each edge texture point in each layer of the direction filtering pyramid point by point to obtain a filtering result of each edge texture point in each layer of the direction filtering pyramid, and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid is determined.
207. And performing fusion reconstruction operation according to the classification of each pixel point in the direction filtering pyramid, the filtering result of each pixel point in each layer of the Laplacian pyramid, the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, the filtering result of each edge texture point in each layer of the direction filtering pyramid and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, so as to obtain a noise-reduced image corresponding to the image to be noise-reduced.
For other descriptions of step 201 and step 204-step 207 in the embodiment of the present invention, please refer to the detailed descriptions of step 101 and step 103-step 106 in the first embodiment, and the detailed descriptions of the embodiment of the present invention are omitted.
Therefore, the image spatial domain noise reduction method described by the embodiment of the invention can establish a multi-scale pyramid according to the image to be noise reduced, establish a direction filtering pyramid by utilizing the middle result of the Gaussian pyramid, and perform fusion reconstruction on the image to be noise reduced according to the non-local mean filtering result of each pyramid, and calculate the direction filtering pyramid by introducing the middle result of the Gaussian pyramid, so that the operation efficiency is improved, the noise reduction efficiency of the image to be noise reduced is further improved, meanwhile, the problem that texture edges of the image after noise reduction are excessively smoothed or edge color noise residues are generated is effectively avoided by adding the direction filtering pyramid, the overall definition is guaranteed while the noise reduction effect is ensured, and the efficiency and the accuracy of the image noise reduction work are improved.
In an alternative embodiment, the formula for calculating the edge intensity information of the pixel point is:
wherein, edge is provided str For the edge intensity information of the pixel point, th_hi is a preset edge high threshold, mx_OR l The maximum residual value corresponding to the pixel point.
In this optional embodiment, the edge intensity information of the pixel point is obtained by calculating the position of the maximum residual error value of the pixel point and a preset edge high threshold, and optionally, the edge high threshold may be a preset fixed threshold or an adaptive threshold, for example, a threshold obtained by calculating information such as a noise curve and/or a gradient difference, which may be selected according to practical situations, and this embodiment is not limited.
Therefore, by implementing the alternative embodiment, the edge intensity information of the pixel point can be calculated according to the maximum residual value corresponding to the pixel point and the preset edge high threshold value, so that the accuracy and reliability of the determined edge intensity information are improved.
In another alternative embodiment, the non-local mean filtering of the edge texture points in each layer of the directional filtering pyramid point by point may include the following operations:
and selecting residual images in the direction filtering pyramid to perform non-local mean value filtering point by point according to the main direction information corresponding to the edge texture points for all the edge texture points in each layer of the direction filtering pyramid.
In this alternative embodiment, for an edge texture point, according to the main direction information of the edge texture point, selecting a residual image in a direction filtering pyramid to perform non-local mean filtering point by point, performing filtering window and filtering process similar to the operation of the laplacian pyramid, that is, performing NLM (non-local mean filtering) filtering on all edge texture points of each layer in the direction filtering pyramid, assuming that the gray value of the edge texture point i is f (i), respectively extracting a central window P (i) with a window size r and a search window with a window size t (t > s) with the edge texture point as the center, traversing and selecting a pixel point j in the search area, and selecting a window P (j) to be matched with the same size as the central window with the pixel point j, if the difference between the sum of gray values of a plurality of pixel points in P (j) and the sum of gray values of a plurality of edge texture points in P (i) is smaller than a preset similarity threshold, considering that the window to be matched is higher in similarity with the central window, and marking the pixel point j as a similarity point.
Therefore, according to the main direction information corresponding to the edge texture points, the non-local mean filtering can be carried out point by selecting the residual image in the direction filtering pyramid by the implementation of the alternative embodiment, the calculated amount of the non-local mean filtering is effectively reduced, the calculation efficiency is improved, and the problem that the edge texture of the image is excessively smoothed or the edge color noise remains after the noise reduction is effectively avoided by adding the direction filtering pyramid.
In yet another optional embodiment, performing a fusion reconstruction operation according to a classification of each pixel point in the direction filtering pyramid, a filtering result of each pixel point in each layer of the laplacian pyramid, a number of similar points corresponding to each pixel point in each layer of the laplacian pyramid, a filtering result of each edge texture point in each layer of the direction filtering pyramid, and a number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, and obtaining a noise-reduced image corresponding to the image to be noise reduced may include the following operations:
for each flat point in the direction filtering pyramid, accumulating filtering results of target pixel points matched with the flat point positions in each layer of the Laplacian pyramid layer by layer according to the structures of the Gaussian pyramid and the Laplacian pyramid, and reconstructing noise reduction results corresponding to the flat points;
for each edge texture point in the direction filtering pyramid, according to the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid and the determined edge strength information of each pixel point in the direction filtering pyramid, fusing the filtering result of each pixel point matched with the position of the edge texture point in each layer of the Laplacian pyramid with the filtering result of each edge texture point in each layer of the direction filtering pyramid, and reconstructing a noise reduction result corresponding to the edge texture point;
And integrating the denoising result corresponding to the flat point and the denoising result corresponding to the edge texture point to obtain a denoising image corresponding to the image to be denoised.
In the alternative embodiment, for the flat points in the direction filtering pyramid, the results after NLM filtering of each layer can be accumulated layer by layer directly according to the structures of the Gaussian pyramid and the Laplacian pyramid, and the noise-reduced results are reconstructed; for the edge texture points in the direction filtering pyramid, the edge texture points of each layer can be reconstructed after being fused on NLM filtering results of the direction filtering pyramid and the Laplacian pyramid according to the number of similar points corresponding to the edge texture points and the edge intensity value.
Therefore, by implementing the alternative embodiment, different noise reduction fusion reconstruction strategies can be executed on the flat points and the edge texture points in the directional filtering pyramid, the noise reduction effect and the definition requirements can be effectively balanced, and the efficiency and the accuracy of the image noise reduction work are improved.
In yet another alternative embodiment, the formula for reconstructing the denoised result corresponding to the edge texture point is:
wherein out is the noise reduction result corresponding to the edge texture point, count or For the number of similar points corresponding to each edge texture point of each layer of the direction filtering pyramid, f or For the filtering result of each edge texture point in each layer of the directional filtering pyramid, count lp For the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, f lp For filtering result of each pixel point in each layer of Laplacian pyramid, edge str And l is the number of image layers for determining the edge intensity information of all pixel points in the direction filtering pyramid.
In this alternative embodiment, the pyramid top layer (l=n) is weighted depending on the number of similarity points to ensure that the top layer is superimposed on the true directional characteristics, and the remaining layers depend on edge intensity control for balancing noise reduction and sharpness requirements.
Therefore, by implementing the alternative embodiment, different fusion reconstruction strategies can be executed on the edge texture points of different layers, the reconstruction process can be quantified according to a formula, the noise reduction effect and the definition requirement of the reconstructed image are guaranteed, and the fusion reconstruction efficiency and the quantification performance can be improved.
Example III
Referring to fig. 3, fig. 3 is a schematic structural diagram of an image spatial domain noise reduction device according to an embodiment of the present invention. The image spatial domain noise reduction device described in fig. 3 may be applied to an image processing system, where the image processing system may include any one of an image processing device, an intelligent server for controlling the image processing device, or an intelligent platform, where the image processing device may include an image noise reduction device, and the intelligent server includes a local server or a cloud server, and the image processing device may be capable of establishing a communication connection with a user device, where the user device includes, but is not limited to, at least one of a smart phone (Android mobile phone, iOS mobile phone, etc.), a smart phone number watch, a tablet computer, a palm computer, a vehicle-mounted computer, a desktop computer, a netbook, a personal digital assistant (Personal Digital Assistant, PDA), an intelligent navigator, and a mobile internet device (Mobile Internet Devices, MID), and the embodiment of the present invention is not limited. As shown in fig. 3, the apparatus for spatial domain noise reduction method of an image may include:
The acquiring module 301 is configured to acquire an image to be denoised, and establish a multi-scale pyramid of the image to be denoised, where the multi-scale pyramid includes a gaussian pyramid and a laplacian pyramid;
the construction module 302 is configured to construct a directional filtering pyramid of the image to be noise reduced according to the gaussian pyramid;
the determining module 303 is configured to determine point information corresponding to each pixel point in the direction filtering pyramid, where the point information includes category information corresponding to the pixel point, and the category information is used to indicate that the pixel point is a flat point or an edge texture point;
the filtering module 304 is configured to perform non-local mean filtering on all the pixels in each layer of the laplace pyramid point by point, obtain a filtering result of each pixel in each layer of the laplace pyramid, and determine the number of similar points corresponding to each pixel in each layer of the laplace pyramid;
the filtering module 304 is further configured to perform non-local mean filtering on each edge texture point in each layer of the direction filtering pyramid, obtain a filtering result of each edge texture point in each layer of the direction filtering pyramid, and determine the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid;
the fusion reconstruction module 305 is configured to perform fusion reconstruction operation according to the classification of each pixel point in the direction filtering pyramid, the filtering result of each pixel point in each layer of the laplacian pyramid, the number of similar points corresponding to each pixel point in each layer of the laplacian pyramid, the filtering result of each edge texture point in each layer of the direction filtering pyramid, and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, so as to obtain a denoised image corresponding to the image to be denoised.
Therefore, the image spatial domain noise reduction device described in fig. 3 can be implemented to establish a multi-scale pyramid according to the image to be reduced, establish a direction filtering pyramid by using the middle result of the Gaussian pyramid, and perform fusion reconstruction on the image to be reduced according to the non-local mean filtering result of each pyramid, so that the situation of insufficient scene adaptability is effectively avoided by utilizing the multi-scale pyramid, meanwhile, the problem that texture edges of the image are excessively smoothed or edge color noise residues are generated after noise reduction is effectively avoided by adding the direction filtering pyramid, the noise reduction effect is ensured, the overall definition is considered, and the efficiency and the accuracy of the image noise reduction work are improved.
In an alternative embodiment, the specific manner in which the constructing module 302 constructs the directional filtering pyramid of the image to be noise reduced according to the gaussian pyramid includes:
obtaining each direction filtering result of the direction filtering pyramid through the intermediate result of the Gaussian pyramid;
constructing a residual diagram of the directional filter pyramid according to each directional filter result of the directional filter pyramid;
wherein, each direction filtering result of the direction filtering pyramid is expressed as:
OF l,m =I l *f m ,
wherein OF l,m For the filtering result of the pixel point in the first layer of the directional filtering pyramid in the mth direction, I l Is the image of the first layer, i is the image layer number, f m A direction filter corresponding to the m-th direction is an image convolution operation;
and, the residual map of the directional filter pyramid is represented as:
wherein OR is l,m For the residual map of the directional filter pyramid, upsample () is the operator of the upsampling operation.
Therefore, the image spatial domain noise reduction device described in fig. 3 can be implemented to establish a multi-scale pyramid according to the image to be reduced, establish a direction filtering pyramid by using the middle result of the Gaussian pyramid, and perform fusion reconstruction on the image to be reduced according to the non-local mean filtering result of each pyramid, and calculate the direction filtering pyramid by introducing the middle result of the Gaussian pyramid, so that the operation efficiency is improved, the noise reduction efficiency of the image to be reduced is further improved, meanwhile, the problem that texture edges are excessively smoothed or edge color noise residues are generated in the image after noise reduction is effectively avoided, the overall definition is guaranteed while the noise reduction effect is ensured, and the efficiency and the accuracy of the image noise reduction work are improved.
In another optional embodiment, for each pixel point in the direction filtering pyramid, if the pixel point is an edge texture point, the point information corresponding to the pixel point further includes main direction information of the pixel point and edge intensity information of the pixel point; if the pixel points are flat points, the point information corresponding to the pixel points also comprises edge intensity information of the pixel points;
The specific manner of determining the point information corresponding to each pixel point in the direction filtering pyramid by the determining module 303 includes:
comparing residual values of the pixel points in residual graphs of different directions of the directional filtering pyramid to obtain a maximum residual value corresponding to the pixel points;
comparing the maximum residual value with a preset edge low threshold value:
when the maximum residual error value is smaller than a preset edge low threshold value, judging that the category of the pixel points is used for indicating that the pixel points are flat points;
when the maximum residual error value is greater than or equal to a preset edge low threshold value, judging that the category of the pixel point is used for indicating that the pixel point is an edge texture point, and determining main direction information of the edge texture point, wherein the main direction information is the direction corresponding to the maximum residual error value;
and determining the edge intensity information of the pixel point according to the maximum residual value and a preset edge high threshold value.
As can be seen, implementing the image spatial domain noise reduction device described in fig. 3 can compare, for each pixel point in the direction filtering pyramid, residual values of the pixel point in residual images of different directions of the direction filtering pyramid to obtain a maximum residual value corresponding to the pixel point, and compare the maximum residual value with a preset edge low threshold value: when the maximum residual value is smaller than the preset edge low threshold value, the category of the pixel point is judged to be used for representing the pixel point as a flat point, when the maximum residual value is larger than or equal to the preset edge low threshold value, the category of the pixel point is judged to be used for representing the pixel point as an edge texture point, main direction information of the edge texture point is determined, the main direction information is the direction corresponding to the maximum residual value, the edge intensity information of the pixel point is determined according to the maximum residual value and the preset edge high threshold value, the definition of the image after noise reduction is improved, and the problem that the texture edge is excessively smoothed or the edge color noise remains in the image after noise reduction is effectively avoided.
In yet another alternative embodiment, the formula for calculating the edge intensity information of the pixel is:
wherein, edge is provided str For the edge intensity information of the pixel point, th_hi is a preset edge high threshold, mx_OR l The maximum residual value corresponding to the pixel point.
Therefore, the implementation of the image spatial domain noise reduction device described in fig. 3 can calculate the edge intensity information of the pixel point according to the maximum residual value corresponding to the pixel point and the preset edge high threshold value, so that the accuracy and reliability of the determined edge intensity information are improved.
In yet another alternative embodiment, the specific manner of performing the non-local mean filtering on the edge texture points in each layer of the directional filtering pyramid by points by the filtering module 304 includes:
and selecting residual images in the direction filtering pyramid to perform non-local mean value filtering point by point according to the main direction information corresponding to the edge texture points for all the edge texture points in each layer of the direction filtering pyramid.
Therefore, the image spatial domain noise reduction device described in fig. 3 can select a residual image in the direction filtering pyramid to perform non-local mean value filtering point by point according to the main direction information corresponding to the edge texture points, so that the calculated amount of the non-local mean value filtering is effectively reduced, the calculation efficiency is improved, and the problem that the edge texture of the image is excessively smoothed or the edge color noise is remained after noise reduction is effectively avoided by adding the direction filtering pyramid.
In yet another alternative embodiment, the fusion reconstruction module 305 performs a fusion reconstruction operation according to the classification of each pixel point in the direction filtering pyramid, the filtering result of each pixel point in each layer of the laplacian pyramid, the number of similar points corresponding to each pixel point in each layer of the laplacian pyramid, the filtering result of each edge texture point in each layer of the direction filtering pyramid, and the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid, and the specific manner of obtaining the image after noise reduction corresponding to the image to be noise reduced includes:
for each flat point in the direction filtering pyramid, accumulating filtering results of target pixel points matched with the flat point positions in each layer of the Laplacian pyramid layer by layer according to the structures of the Gaussian pyramid and the Laplacian pyramid, and reconstructing noise reduction results corresponding to the flat points;
for each edge texture point in the direction filtering pyramid, according to the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, the number of similar points corresponding to each edge texture point in each layer of the direction filtering pyramid and the determined edge strength information of each pixel point in the direction filtering pyramid, fusing the filtering result of each pixel point matched with the position of the edge texture point in each layer of the Laplacian pyramid with the filtering result of each edge texture point in each layer of the direction filtering pyramid, and reconstructing a noise reduction result corresponding to the edge texture point;
And integrating the denoising result corresponding to the flat point and the denoising result corresponding to the edge texture point to obtain a denoising image corresponding to the image to be denoised.
Therefore, the implementation of the image spatial domain noise reduction device described in fig. 3 can execute different noise reduction fusion reconstruction strategies on the flat points and the edge texture points in the directional filtering pyramid, effectively balance the noise reduction effect and the definition requirement, and improve the efficiency and the accuracy of the image noise reduction work.
In yet another alternative embodiment, the formula for reconstructing the denoised result corresponding to the edge texture point is:
wherein out is the noise reduction result corresponding to the edge texture point, count or For the number of similar points corresponding to each edge texture point of each layer of the direction filtering pyramid, f or For the filtering result of each edge texture point in each layer of the directional filtering pyramid, count lp For the number of similar points corresponding to each pixel point in each layer of the Laplacian pyramid, f lp For filtering result of each pixel point in each layer of Laplacian pyramid, edge str And l is the number of image layers for determining the edge intensity information of all pixel points in the direction filtering pyramid.
Therefore, the implementation of the image spatial domain noise reduction device described in fig. 3 can execute different fusion reconstruction strategies on edge texture points of different layers, and can quantify the reconstruction process according to a formula, so that the noise reduction effect and definition requirements of the reconstructed image are ensured, and the fusion reconstruction efficiency and the quantification performance can be improved.
Example IV
Referring to fig. 4, fig. 4 is a schematic structural diagram of another spatial domain noise reduction device for images according to an embodiment of the present invention. As shown in fig. 4, the spatial domain noise reduction apparatus for an image may include:
a memory 401 storing executable program codes;
a processor 402 coupled with the memory 401;
the processor 402 invokes executable program codes stored in the memory 401 to perform the steps in the image spatial domain noise reduction method described in the first or second embodiment of the present invention.
Example five
The embodiment of the invention discloses a computer storage medium which stores computer instructions for executing the steps in the image spatial domain noise reduction method described in the first or second embodiment of the invention when the computer instructions are called.
Example six
An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the image spatial domain noise reduction method described in the first embodiment or the second embodiment.
The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above detailed description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course by means of hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in part in the form of a software product that may be stored in a computer-readable storage medium including Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic disc Memory, tape Memory, or any other medium that can be used for computer-readable carrying or storing data.
Finally, it should be noted that: the embodiment of the invention discloses an image spatial domain noise reduction method and device, which are disclosed by the embodiment of the invention and are only used for illustrating the technical scheme of the invention, but not limiting the technical scheme; although the invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.