CN116934605A - Image denoising method, device, electronic equipment and computer program product - Google Patents
Image denoising method, device, electronic equipment and computer program product Download PDFInfo
- Publication number
- CN116934605A CN116934605A CN202210364759.2A CN202210364759A CN116934605A CN 116934605 A CN116934605 A CN 116934605A CN 202210364759 A CN202210364759 A CN 202210364759A CN 116934605 A CN116934605 A CN 116934605A
- Authority
- CN
- China
- Prior art keywords
- image
- layer
- noise
- denoising
- noisy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000004590 computer program Methods 0.000 title claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 18
- 230000004913 activation Effects 0.000 claims description 29
- 238000011156 evaluation Methods 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 230000003213 activating effect Effects 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 10
- 238000010030 laminating Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 7
- 238000010606 normalization Methods 0.000 description 7
- 239000002131 composite material Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The application relates to the field of image processing, and provides an image denoising method, an image denoising device, electronic equipment and a computer program product. The method comprises the following steps: acquiring an original noise-containing image, and performing downsampling treatment on the original noise-containing image to obtain a first noise-containing image; under the condition that the first noisy image contains noise estimation information, inputting the noise estimation information contained in the first noisy image into a one-way convolution network to obtain characteristic information; inputting the original noise-containing image and the characteristic information into a multiple cavity convolution network for training to obtain a first denoising image; and under the condition that the first noisy image does not contain noise estimation information, inputting the original noisy image into a multiple cavity convolution network for training to obtain a first denoising image. According to the application, the noise characteristic information of the image under different depths is extracted through the multi-cavity convolution network, so that the extracted noise characteristic information is closer to the real noise distribution, and the method is better suitable for processing the noise image under the real environment.
Description
Technical Field
The present application relates to the field of image processing, and in particular, to an image denoising method, an image denoising apparatus, an electronic device, and a computer program product.
Background
In the field of image denoising, existing solutions can be roughly divided into two categories, traditional denoising devices and deep neural network denoising devices. The manual design of conventional denoising methods can blur some edges of an image, resulting in image distortion. In recent years, with the continuous development of deep neural networks, many denoising methods based on deep learning, such as convolution blind denoising methods based on real noise modeling, are derived, and the method is very dependent on pure noiseless images and real noiseless images appearing in a pair mode, but the pure noiseless image acquisition process is very complicated, and the analysis of certain specific noises alone cannot be suitable for processing noise images in a real environment.
Disclosure of Invention
The embodiment of the application provides an image denoising method, an image denoising device, electronic equipment and a computer program product, which are used for solving the technical problem that the existing image denoising method is not suitable for processing noise images in a real environment.
In a first aspect, an embodiment of the present application provides an image denoising method, including:
acquiring an original noise-containing image, and performing downsampling processing on the original noise-containing image to obtain a first noise-containing image;
inputting the noise estimation information contained in the first noise-containing image into a one-way convolution network to obtain characteristic information under the condition that the first noise-containing image contains the noise estimation information;
inputting the original noise-containing image and the characteristic information into a multiple cavity convolution network for training to obtain a first denoising image;
and under the condition that the first noisy image does not contain noise estimation information, inputting the original noisy image into a multiple hole convolution network for training to obtain a first denoising image.
In one embodiment, the step of downsampling the original noisy image to obtain a first noisy image includes:
pooling the original noisy image according to a preset step length to obtain a plurality of noisy images;
and splicing the plurality of noisy images to obtain a first noisy image, wherein the first noisy image has the same size as the original noisy image.
In one embodiment, the image denoising method further includes:
and determining the single-path convolutional network and the multiple-hole convolutional network according to a convolutional layer, an activating layer and a normalizing layer, wherein the multiple-hole convolutional network comprises a first layer, a second layer, a third layer and a fourth layer, the first layer comprises the convolutional layer, the activating layer and the normalizing layer, and the second layer, the third layer and the fourth layer comprise the convolutional layer and the activating layer.
In one embodiment, the step of inputting the original noise-containing image and the feature information into a multiple hole convolution network for training to obtain a first denoising image includes:
inputting the original noisy image and the characteristic information into a multiple cavity convolution network to obtain the output of the first layer, the output of the second layer, the output of the third layer and the output of the fourth layer;
performing preset superposition operation on the output of the second layer, the output of the third layer and the output of the fourth layer, and inputting the result of the preset superposition operation into the convolution layer and the activation layer to obtain target output;
and obtaining a first denoising image according to the output of the first layer and the target output.
In one embodiment, after the step of obtaining the first denoised image, the step of obtaining the first denoised image comprises:
calculating a gradient value of the multiple hole convolution network according to a first preset loss function under the condition that the first noisy image contains noise estimation information;
under the condition that the first noisy image does not contain noise estimation information, calculating a gradient value of the multiple cavity convolution network according to a second preset loss function;
under the condition that the parameters of the multiple-hole convolutional network do not meet preset conditions, updating the parameters of the multiple-hole convolutional network according to the gradient value;
and under the condition that the parameters of the multiple hole convolutional network accord with preset conditions, obtaining a new multiple hole convolutional network.
In one embodiment, after the step of obtaining a new multiple hole convolutional network if the parameters of the multiple hole convolutional network meet a preset rule, the method includes:
superposing the first noisy image and the first denoising image to obtain an image combination, and performing up-sampling processing on the image combination to obtain a second noisy image;
inputting the noise estimation information contained in the second noisy image into a one-way convolution network to obtain characteristic information under the condition that the second noisy image contains the noise estimation information;
inputting the original noise-containing image and the characteristic information into the new multiple hole convolution network for training to obtain a second denoising image;
and under the condition that the second noisy image does not contain noise estimation information, inputting the original noisy image into the new multiple hole convolution network for training to obtain a second denoising image.
In one embodiment, after the step of obtaining the second denoised image, the step of obtaining the second denoised image comprises:
determining a multiple cavity convolution network to be tested according to the second denoising image;
reading a preset test set to obtain a test image and a comparison image, and inputting the test image into the multi-cavity convolution network to be tested to obtain a target denoising image;
comparing the target denoising image with the comparison image to obtain a comparison result, and determining an evaluation result of the multi-cavity convolutional network to be tested according to a preset evaluation index and the comparison result;
and under the condition that the evaluation result does not meet the preset requirement, adjusting the multiple cavity convolution network to be tested.
In a second aspect, an embodiment of the present application provides an image denoising apparatus, including:
the downsampling module is used for acquiring an original noise-containing image, and downsampling the original noise-containing image to obtain a first noise-containing image;
the characteristic information obtaining module is used for inputting the noise estimation information contained in the first noisy image into a one-way convolution network to obtain characteristic information under the condition that the first noisy image contains the noise estimation information;
the first training module is used for inputting the original noisy image and the characteristic information into a multiple cavity convolution network for training to obtain a first denoising image;
and the second training module is used for inputting the original noisy image into a multiple cavity convolution network for training under the condition that the first noisy image does not contain noise estimation information, so as to obtain a first denoising image.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory storing a computer program, where the processor implements the steps of the image denoising method according to the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which when executed by a processor implements the steps of the image denoising method according to the first aspect.
According to the image denoising method, the device, the electronic equipment and the computer program product, the spatial correlation of real noise is reduced by downsampling an original noisy image, a one-way convolution network for noise estimation is added, the multiple-cavity convolution network can adapt to denoising tasks containing noise estimation information, the multiple-cavity convolution network can extract noise characteristic information under different depths of the image, the extracted noise characteristic information is enabled to be closer to real noise distribution, and the method is better suitable for processing noise images under real environments.
Drawings
In order to more clearly illustrate the application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of an image denoising method according to an embodiment of the present application;
FIG. 2 is a second flowchart of an image denoising method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an image denoising apparatus according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, fig. 1 is a schematic flow chart of an image denoising method according to an embodiment of the present application. The image denoising method provided by the embodiment of the application can comprise the following steps:
step S100, an original noise-containing image is obtained, and downsampling processing is carried out on the original noise-containing image to obtain a first noise-containing image;
specifically, an original noisy image acquired under a real environment is acquired, the original noisy image is firstly converted into a pixel matrix, under the condition that the pixel matrix converted from the original noisy image is 4×4, the pixel matrix converted from the original noisy image is converted into 4 matrices of 2×2 through a convolution kernel with the transverse step length and the longitudinal step length, 4 reduced noisy images are obtained through inverse conversion, and the obtained reduced noisy images are spliced into a first noisy image with the same size as the original noisy image in a 2×2 mode. And each pixel point of the original noise-containing image is downsampled according to the requirements of the transverse and longitudinal step sizes to obtain a plurality of thumbnail images with smaller scales, so that the spatial correlation of noise in the original noise-containing image is reduced, and the subsequent denoising operation is facilitated.
The spatial correlation means that compared with artificially synthesized noise points, in real environments with complex illumination, angle and other attributes, real noise points in a shot image are obviously different at different positions of the same image, for example, red noise points are obviously more distributed than green noise points in a region where red flowers are located in a shot image, and artificially synthesized noise points are uniformly distributed in each region on the image.
Step S200, inputting the noise estimation information contained in the first noise-containing image into a one-way convolution network to obtain characteristic information under the condition that the first noise-containing image contains the noise estimation information;
specifically, after the first noisy image is obtained, whether the first noisy image contains noise estimation information is judged, and when the first noisy image contains the noise estimation information, the noise estimation information contained in the first noisy image is input into a one-way convolution network which sequentially comprises a convolution layer, a ReLU (Linear rectification function ) activation layer, a convolution layer, a normalization layer, a ReLU activation layer and a convolution layer, so that characteristic information of the noise estimation information is obtained. The noise estimation information refers to that image noise in the real world is not expressed by a linear function (such as a gaussian function), and the image denoising method in the embodiment of the application provides an estimation function to express noise in the first noisy image.
Step S300, inputting the original noisy image and the characteristic information into a multiple cavity convolution network for training to obtain a first denoising image;
specifically, feature information of an original noisy image and noise estimation information is input into a multiple hole convolution network, and the multiple hole convolution network provided by the image denoising method in the embodiment of the application consists of four layers: the first layer is formed by sequentially laminating a convolution layer, a ReLU activation layer, a convolution layer, a normalization layer and a ReLU activation layer; the second layer is formed by sequentially laminating a convolution layer, a ReLU activation layer, a convolution layer and a ReLU activation layer; the third layer is formed by sequentially laminating a convolution layer, a ReLU activation layer, a convolution layer and a ReLU activation layer; the fourth layer is formed by overlapping a convolution layer and a ReLU activation layer in sequence, the output of the second layer, the output of the third layer and the output of the fourth layer are subjected to overlapping operation of one channel dimension, then the output of the second layer, the output of the third layer and the output of the fourth layer are sent into the convolution layer and the ReLU activation layer, and finally the output of the fourth layer and the output of the first layer are added to obtain a first denoising image. Noise characteristic information of the noisy image at different depths is extracted through the multiple cavity convolution network, so that the extracted noise characteristic information is closer to real noise distribution and is better suitable for processing the noise image in a real environment.
Step S400, inputting the original noise-containing image into a multiple hole convolution network for training under the condition that the first noise-containing image does not contain noise estimation information, and obtaining a first denoising image.
Specifically, after the first noisy image is obtained, whether the first noisy image contains noise estimation information or not is judged, and under the condition that the first noisy image does not contain the noise estimation information, the original noisy image is directly input into the multiple cavity convolution network for training, so that the first denoising image is obtained.
According to the embodiment, the original noise-containing image is downsampled, the spatial correlation of real noise is reduced, the one-way convolution network for noise estimation is increased, the multiple-hole convolution network can adapt to a denoising task containing noise estimation information, and the multiple-hole convolution network can extract noise characteristic information of the image at different depths, so that the extracted noise characteristic information is closer to real noise distribution and is better suitable for processing noise images in real environments.
In one embodiment, the image denoising method provided by the embodiment of the present application may further include:
step S101, pooling the original noisy image according to a preset step length to obtain a plurality of noisy images;
step S102, the plurality of noisy images are spliced to obtain a first noisy image, wherein the first noisy image has the same size as the original noisy image.
Specifically, an original noisy image acquired in a real environment is acquired, the original noisy image is firstly converted into a pixel matrix, and under the condition that the pixel matrix converted from the original noisy image is 4×4, the process of pooling the original noisy image according to a preset step length is as follows: converting the 4 multiplied by 4 pixel matrix into 4 matrixes of 2 multiplied by 2 through a convolution kernel with the transverse step length and the longitudinal step length, obtaining 4 reduced noise-containing images through the inverse conversion of the pixel matrix and the images, and splicing the obtained reduced noise-containing images into a first noise-containing image with the same size as the original noise-containing image in a mode of 2 multiplied by 2. Each pixel point of the original noise-containing image is downsampled according to the requirements of the transverse and longitudinal step sizes to obtain a plurality of thumbnail images with smaller sizes, so that the spatial correlation of noise in the original noise-containing image is reduced, the spatial correlation of real noise of the image is disturbed, and then the noise is removed by using a multi-cavity convolution network trained on artificially synthesized noise points, so that a better denoising effect is achieved.
According to the embodiment, the down-sampling operation is carried out on the original noisy image, so that the spatial correlation of noise in the original noisy image is reduced, the spatial correlation of real noise of the image is disturbed, and then the first noisy image is denoised by using the multiple cavity convolution network trained on the artificially synthesized noisy point, so that a better denoising effect is achieved.
In one embodiment, the image denoising method provided by the embodiment of the present application may further include:
step S10, determining the single-path convolutional network and the multiple-hole convolutional network according to a convolutional layer, an activating layer and a normalizing layer, wherein the multiple-hole convolutional network comprises a first layer, a second layer, a third layer and a fourth layer, the first layer comprises the convolutional layer, the activating layer and the normalizing layer, and the second layer, the third layer and the fourth layer comprise the convolutional layer and the activating layer.
Specifically, the single-path convolution network is formed by sequentially superposing a convolution layer, a ReLU activation layer, a convolution layer, a normalization layer, a ReLU activation layer and a convolution layer, and under the condition that a first noisy image contains noise estimation information, the noise estimation information contained in the first noisy image is input into the single-path convolution network to obtain characteristic information; the multiple hole convolution network is composed of four layers: the first layer is formed by sequentially laminating a convolution layer, a ReLU activation layer, a convolution layer, a normalization layer and a ReLU activation layer; the second layer is formed by sequentially laminating a convolution layer, a ReLU activation layer, a convolution layer and a ReLU activation layer; the third layer is formed by sequentially laminating a convolution layer, a ReLU activation layer, a convolution layer and a ReLU activation layer; the fourth layer is formed by sequentially laminating a convolution layer and a ReLU activation layer.
According to the embodiment, the noise characteristic information of the noisy image at different depths is extracted through the multiple cavity convolution network, so that the extracted noise characteristic information is closer to real noise distribution, and the method is better suitable for processing the noise image in a real environment.
In one embodiment, the image denoising method provided by the embodiment of the present application may further include:
step S301, inputting the original noisy image and the characteristic information into a multiple hole convolution network to obtain the output of the first layer, the output of the second layer, the output of the third layer and the output of the fourth layer;
step S302, performing preset superposition operation on the output of the second layer, the output of the third layer and the output of the fourth layer, and inputting the result of the preset superposition operation into the convolution layer and the activation layer to obtain a target output;
step S303, obtaining a first denoising image according to the output of the first layer and the target output.
Specifically, the multiple hole convolutional network includes four layers, an original noise-containing image and characteristic information are input into the multiple hole convolutional network, first, output of the first layer is obtained through the first layer, then output of the second layer, output of the third layer and output of the fourth layer are sequentially obtained, the output of the second layer, the output of the third layer and the output of the fourth layer are subjected to superposition operation of one channel dimension (namely preset superposition operation in the embodiment), a result of the preset superposition operation is obtained, a result of the preset superposition operation is input into a convolutional layer and a ReLU activation layer, target output is obtained, and finally the target output and the output of the first layer are added, so that the first denoising image is obtained.
According to the embodiment, the noise characteristic information of the noisy image at different depths is extracted through the multiple cavity convolution network, so that the extracted noise characteristic information is closer to real noise distribution, and the method is better suitable for processing the noise image in a real environment.
In one embodiment, the image denoising method provided by the embodiment of the present application may further include:
step S410, calculating the gradient value of the multiple hole convolution network according to a first preset loss function under the condition that the first noisy image contains noise estimation information;
step S420, calculating the gradient value of the multiple hole convolution network according to a second preset loss function under the condition that the first noisy image does not contain noise estimation information;
step S430, updating the parameters of the multiple hole convolutional network according to the gradient value under the condition that the parameters of the multiple hole convolutional network do not meet the preset conditions;
step S440, obtaining a new multi-hole convolutional network under the condition that the parameters of the multi-hole convolutional network meet the preset conditions.
Specifically, if the process of obtaining the first denoising image is a training process, the first denoising image is required to be subjected to a loss function to obtain a composite loss value of the current training stage, and the composite loss value is used for a real sample image and a multiple hole rollThe first denoising image output by the product network performs pixel-by-pixel distance calculation, and the calculation formula comprises: the image denoising method in the embodiment of the application provides a Loss function, namely loss=alpha L b +βL e +γL nb In the case where the first noisy image contains noise estimation information, the parameter α is set to 1, and the parameters β and γ are set to 0; when the training data contains noise estimation information, the parameter α is set to 0, and the super parameters β and γ are set to 1. After the composite loss value is obtained, calculating the gradient value of the multiple hole convolution network according to the composite loss value, and updating the parameters of the multiple hole convolution network according to the gradient value to obtain a new multiple hole convolution network after updating the parameters.
According to the embodiment, a new loss function is designed according to whether noise estimation information is contained or not, and the current multi-cavity convolution network structure can be effectively combined, so that the convergence direction of the model points to a better denoising effect.
Referring to fig. 2, fig. 2 is a second flowchart of an image denoising method according to an embodiment of the present application, where in an embodiment, the image denoising method provided by the embodiment of the present application may further include:
step S450, superposing the first noisy image and the first denoising image to obtain an image combination, and performing up-sampling processing on the image combination to obtain a second noisy image;
specifically, after the first denoising image is obtained, the first denoising image is overlapped with a first denoising image with a certain proportion, and the purpose of the operation is to enable the final output to be blurred as little as possible by a multiple cavity convolution network, so that more edge texture detail information is reserved. The superimposed image combination is restored into a single noise-containing image with the same size as the original noise-containing image through up-sampling operation, and the purpose of the operation is to restore the image combination formed by a plurality of sub-images into a single image with the same size as the original noise-containing image on the one hand, and on the other hand, the spatial correlation of the real noise of the image can be further reduced through up-sampling, so that the denoising effect of the multi-cavity convolution network is better.
Step S460, inputting the noise estimation information contained in the second noisy image into a one-way convolution network to obtain characteristic information when the second noisy image contains the noise estimation information;
specifically, whether the second noisy image contains noise estimation information is judged, and under the condition that the second noisy image contains the noise estimation information, the noise estimation information contained in the second noisy image is input into a one-way convolution network which sequentially comprises a convolution layer, a ReLU activation layer, a convolution layer, a normalization layer, a ReLU activation layer and the convolution layer, so that characteristic information of the noise estimation information is obtained.
Step S470, inputting the original noise-containing image and the characteristic information into the new multi-cavity convolution network for training to obtain a second denoising image;
specifically, the characteristic information of the original noise-containing image and the noise estimation information is input into a multiple cavity convolution network for training to obtain a second denoising image. Noise characteristic information of the noisy image at different depths is extracted through the multiple cavity convolution network, so that the extracted noise characteristic information is closer to real noise distribution and is better suitable for processing the noise image in a real environment.
Step S480, inputting the original noisy image into the new multiple hole convolution network to train to obtain a second denoising image when the second noisy image does not contain noise estimation information.
Specifically, after the second noisy image is obtained, judging whether the second noisy image contains noise estimation information, and under the condition that the second noisy image does not contain the noise estimation information, directly inputting the original noisy image into the multiple cavity convolution network for training to obtain a second denoising image.
According to the embodiment, through up-sampling is carried out on the image combination obtained by superposing the first noisy image and the first denoising image, more edge texture detail information is reserved, a one-way convolution network for noise estimation is added, the multiple cavity convolution network can adapt to a denoising task containing noise estimation information, and the multiple cavity convolution network can extract noise characteristic information under different depths of the image, so that the extracted noise characteristic information is closer to real noise distribution, and the method is better suitable for processing noise images under real environments.
In one embodiment, the image denoising method provided by the embodiment of the present application may further include:
step S481, according to the second denoising image, determining a multi-cavity convolution network to be tested;
step S482, reading a preset test set to obtain a test image and a comparison image, and inputting the test image into the multi-cavity convolution network to be tested to obtain a target denoising image;
step S483, comparing the target denoising image with the comparison image to obtain a comparison result, and determining an evaluation result of the multi-cavity convolutional network to be tested according to a preset evaluation index and the comparison result;
step S484, adjusting the multi-cavity convolution network to be tested under the condition that the evaluation result does not meet the preset requirement.
Specifically, after the second denoising image is obtained, whether the process of obtaining the second denoising image is a training process or not is further determined, if the process of obtaining the second denoising image is a training process, the first denoising image is required to be subjected to a loss function to obtain a composite loss value of the current training stage, gradient values of the multiple cavity convolution network are calculated according to the composite loss value, whether parameters of the multiple cavity convolution network are updated or not is further determined according to the gradient values, and finally the multiple cavity convolution network to be tested is obtained. The process of evaluating the multiple hole convolutional network to be tested may be: selecting a test image and a comparison image in a preset test set in a paired mode, inputting the test image into a multiple hole convolutional network to be tested to obtain a target denoising image, comparing the target denoising image with the comparison image to obtain a comparison result, determining an evaluation result of the multiple hole convolutional network to be tested according to a preset evaluation index and the comparison result, wherein the preset evaluation index comprises a peak signal-to-noise ratio, structural similarity, root mean square error and the like, taking the multiple hole convolutional network to be tested as a final multiple hole convolutional network under the condition that the evaluation result meets the preset requirement, adjusting parameters of the multiple hole convolutional network to be tested under the condition that the evaluation result does not meet the preset requirement, and returning to the process to train again until the obtained multiple hole convolutional network meets the preset requirement.
According to the embodiment, the multiple cavity convolutional network is evaluated, so that the denoising quality of the multiple cavity convolutional network is improved.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an image denoising apparatus according to an embodiment of the present application, and the image denoising apparatus according to the embodiment of the present application is described below, and the image denoising apparatus described below and the image denoising method described above may be referred to correspondingly.
The downsampling module 301 is configured to obtain an original noise-containing image, and downsample the original noise-containing image to obtain a first noise-containing image;
the feature information obtaining module 302 is configured to input, when the first noisy image includes noise estimation information, the noise estimation information included in the first noisy image into a one-way convolutional network to obtain feature information;
the first training module 303 is configured to input the original noise-containing image and the feature information into a multiple hole convolutional network for training, so as to obtain a first denoising image;
and the second training module 304 is configured to input the original noisy image into a multiple hole convolutional network for training to obtain a first denoised image when the first noisy image does not include noise estimation information.
The downsampling module, as known, comprises:
the pooling unit is used for pooling the original noisy images according to a preset step length to obtain a plurality of noisy images;
and the image stitching unit is used for stitching the plurality of noisy images to obtain a first noisy image, wherein the first noisy image and the original noisy image have the same size.
As can be seen, the image denoising apparatus further includes:
the convolution network determining module is used for determining the single-path convolution network and the multiple-hole convolution network according to a convolution layer, an activation layer and a normalization layer, wherein the multiple-hole convolution network comprises a first layer, a second layer, a third layer and a fourth layer, the first layer comprises the convolution layer, the activation layer and the normalization layer, and the second layer, the third layer and the fourth layer comprise the convolution layer and the activation layer.
The first training module, as known, comprises:
the convolution network output unit is used for inputting the original noise-containing image and the characteristic information into a multi-cavity convolution network to obtain the output of the first layer, the output of the second layer, the output of the third layer and the output of the fourth layer;
the superposition operation unit is used for carrying out preset superposition operation on the output of the second layer, the output of the third layer and the output of the fourth layer, and inputting the result of the preset superposition operation into the convolution layer and the activation layer to obtain target output;
and the first denoising image obtaining unit is used for obtaining a first denoising image according to the output of the first layer and the target output.
As can be seen, the image denoising apparatus further includes:
the first gradient value calculation module is used for calculating the gradient value of the multiple cavity convolution network according to a first preset loss function under the condition that the first noisy image contains noise estimation information;
the second gradient value calculation module is used for calculating the gradient value of the multiple cavity convolution network according to a second preset loss function under the condition that the first noisy image does not contain noise estimation information;
the parameter updating module is used for updating the parameters of the multiple hole convolutional network according to the gradient value under the condition that the parameters of the multiple hole convolutional network do not accord with preset conditions;
the new convolutional network obtaining module is used for obtaining a new multiple hole convolutional network under the condition that the parameters of the multiple hole convolutional network meet preset conditions.
As can be seen, the image denoising apparatus further includes:
the downsampling module is used for superposing the first noisy image and the first denoising image to obtain an image combination, and upsampling the image combination to obtain a second noisy image;
the single-path convolution network input module is used for inputting the noise estimation information contained in the second noisy image into a single-path convolution network to obtain characteristic information under the condition that the second noisy image contains the noise estimation information;
the third training module is used for inputting the original noise-containing image and the characteristic information into the new multiple cavity convolution network for training to obtain a second denoising image;
and the fourth training module is used for inputting the original noisy image into the new multiple cavity convolution network for training under the condition that the second noisy image does not contain noise estimation information, so as to obtain a second denoising image.
As can be seen, the image denoising apparatus further includes:
the multiple cavity convolutional network testing module is used for determining a multiple cavity convolutional network to be tested according to the second denoising image;
the target denoising image acquisition module is used for reading a preset test set to obtain a test image and a comparison image, and inputting the test image into the multi-cavity convolution network to be tested to obtain a target denoising image;
the evaluation result determining module is used for comparing the target denoising image with the comparison image to obtain a comparison result, and determining an evaluation result of the multi-cavity convolutional network to be tested according to a preset evaluation index and the comparison result;
and the multiple hole convolutional network adjusting module is used for adjusting the multiple hole convolutional network to be tested under the condition that the evaluation result does not meet the preset requirement.
Fig. 4 illustrates a physical schematic diagram of an electronic device, as shown in fig. 4, which may include: processor 410, communication interface (Communication Interface) 420, memory 430 and communication bus 440, wherein processor 410, communication interface 420 and memory 430 communicate with each other via communication bus 440. Processor 410 may call a computer program in memory 430 to perform the steps of the image denoising method
Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, embodiments of the present application also provide a computer program product, which includes a computer program, the computer program may be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer program may perform the steps of the image denoising method provided in the above embodiments
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present application without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the image denoising method according to the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. An image denoising method, comprising:
acquiring an original noise-containing image, and performing downsampling processing on the original noise-containing image to obtain a first noise-containing image;
inputting the noise estimation information contained in the first noise-containing image into a one-way convolution network to obtain characteristic information under the condition that the first noise-containing image contains the noise estimation information;
inputting the original noise-containing image and the characteristic information into a multiple cavity convolution network for training to obtain a first denoising image;
and under the condition that the first noisy image does not contain noise estimation information, inputting the original noisy image into a multiple hole convolution network for training to obtain a first denoising image.
2. The method of image denoising according to claim 1, wherein the step of downsampling the original noisy image to obtain a first noisy image comprises:
pooling the original noisy image according to a preset step length to obtain a plurality of noisy images;
and splicing the plurality of noisy images to obtain a first noisy image, wherein the first noisy image has the same size as the original noisy image.
3. The image denoising method according to claim 1, further comprising:
and determining the single-path convolutional network and the multiple-hole convolutional network according to a convolutional layer, an activating layer and a normalizing layer, wherein the multiple-hole convolutional network comprises a first layer, a second layer, a third layer and a fourth layer, the first layer comprises the convolutional layer, the activating layer and the normalizing layer, and the second layer, the third layer and the fourth layer comprise the convolutional layer and the activating layer.
4. The image denoising method according to claim 3, wherein the step of inputting the original noisy image and the feature information into a multiple hole convolution network for training to obtain a first denoised image comprises:
inputting the original noisy image and the characteristic information into a multiple cavity convolution network to obtain the output of the first layer, the output of the second layer, the output of the third layer and the output of the fourth layer;
performing preset superposition operation on the output of the second layer, the output of the third layer and the output of the fourth layer, and inputting the result of the preset superposition operation into the convolution layer and the activation layer to obtain target output;
and obtaining a first denoising image according to the output of the first layer and the target output.
5. The image denoising method according to claim 1, wherein after the step of obtaining the first denoised image, comprising:
calculating a gradient value of the multiple hole convolution network according to a first preset loss function under the condition that the first noisy image contains noise estimation information;
under the condition that the first noisy image does not contain noise estimation information, calculating a gradient value of the multiple cavity convolution network according to a second preset loss function;
under the condition that the parameters of the multiple-hole convolutional network do not meet preset conditions, updating the parameters of the multiple-hole convolutional network according to the gradient value;
and under the condition that the parameters of the multiple hole convolutional network accord with preset conditions, obtaining a new multiple hole convolutional network.
6. The image denoising method according to claim 5, wherein after the step of obtaining a new multiple hole convolutional network if the parameters of the multiple hole convolutional network meet a preset rule, the method comprises:
superposing the first noisy image and the first denoising image to obtain an image combination, and performing up-sampling processing on the image combination to obtain a second noisy image;
inputting the noise estimation information contained in the second noisy image into a one-way convolution network to obtain characteristic information under the condition that the second noisy image contains the noise estimation information;
inputting the original noise-containing image and the characteristic information into the new multiple hole convolution network for training to obtain a second denoising image;
and under the condition that the second noisy image does not contain noise estimation information, inputting the original noisy image into the new multiple hole convolution network for training to obtain a second denoising image.
7. The method of denoising an image according to claim 6, wherein after the step of obtaining a second denoised image, comprising:
determining a multiple cavity convolution network to be tested according to the second denoising image;
reading a preset test set to obtain a test image and a comparison image, and inputting the test image into the multi-cavity convolution network to be tested to obtain a target denoising image;
comparing the target denoising image with the comparison image to obtain a comparison result, and determining an evaluation result of the multi-cavity convolutional network to be tested according to a preset evaluation index and the comparison result;
and under the condition that the evaluation result does not meet the preset requirement, adjusting the multiple cavity convolution network to be tested.
8. An image denoising apparatus, comprising:
the downsampling module is used for acquiring an original noise-containing image, and downsampling the original noise-containing image to obtain a first noise-containing image;
the characteristic information obtaining module is used for inputting the noise estimation information contained in the first noisy image into a one-way convolution network to obtain characteristic information under the condition that the first noisy image contains the noise estimation information;
the first training module is used for inputting the original noisy image and the characteristic information into a multiple cavity convolution network for training to obtain a first denoising image;
and the second training module is used for inputting the original noisy image into a multiple cavity convolution network for training under the condition that the first noisy image does not contain noise estimation information, so as to obtain a first denoising image.
9. An electronic device comprising a processor and a memory storing a computer program, characterized in that the processor implements the steps of the image denoising method of any one of claims 1 to 7 when executing the computer program.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the image denoising method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210364759.2A CN116934605A (en) | 2022-04-07 | 2022-04-07 | Image denoising method, device, electronic equipment and computer program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210364759.2A CN116934605A (en) | 2022-04-07 | 2022-04-07 | Image denoising method, device, electronic equipment and computer program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116934605A true CN116934605A (en) | 2023-10-24 |
Family
ID=88388408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210364759.2A Pending CN116934605A (en) | 2022-04-07 | 2022-04-07 | Image denoising method, device, electronic equipment and computer program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116934605A (en) |
-
2022
- 2022-04-07 CN CN202210364759.2A patent/CN116934605A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112233038B (en) | True image denoising method based on multi-scale fusion and edge enhancement | |
| CN114140353B (en) | Swin-Transformer image denoising method and system based on channel attention | |
| CN112507997B (en) | Face super-resolution system based on multi-scale convolution and receptive field feature fusion | |
| CN108205803B (en) | Image processing method, and training method and device of neural network model | |
| CN110163813B (en) | Image rain removing method and device, readable storage medium and terminal equipment | |
| CN110210524B (en) | Training method of image enhancement model, image enhancement method and device | |
| CN110766632A (en) | Image denoising method based on channel attention mechanism and characteristic pyramid | |
| CN111340716B (en) | Image deblurring method for improving double-discrimination countermeasure network model | |
| CN110148088B (en) | Image processing method, image rain removing method, device, terminal and medium | |
| CN105657402A (en) | Depth map recovery method | |
| CN110879982A (en) | Crowd counting system and method | |
| CN112884668A (en) | Lightweight low-light image enhancement method based on multiple scales | |
| CN109064402B (en) | Single image super-resolution reconstruction method based on enhanced non-local total variation model prior | |
| CN113658040A (en) | Face super-resolution method based on prior information and attention fusion mechanism | |
| CN115205147A (en) | Multi-scale optimization low-illumination image enhancement method based on Transformer | |
| CN111951164A (en) | Image super-resolution reconstruction network structure and image reconstruction effect analysis method | |
| CN115170410A (en) | Image enhancement method and device integrating wavelet transformation and attention mechanism | |
| CN112102259A (en) | Image segmentation algorithm based on boundary guide depth learning | |
| CN115700731A (en) | Underwater image enhancement method based on dual-channel convolutional neural network | |
| CN115326809A (en) | Apparent crack detection method and detection device for tunnel lining | |
| CN106027854A (en) | United filtering denoising method which is applied to a camera and is applicable to be realized in FPGA (Field Programmable Gate Array) | |
| CN114926883A (en) | Face image processing method meeting various degradation models | |
| CN113392728A (en) | Target detection method based on SSA sharpening attention mechanism | |
| CN117058079A (en) | Thyroid imaging image automatic diagnosis method based on improved ResNet model | |
| CN111626379A (en) | X-ray image detection method for pneumonia |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |