CN111028171A - Method, device and server for determining noise level of image - Google Patents
Method, device and server for determining noise level of image Download PDFInfo
- Publication number
- CN111028171A CN111028171A CN201911254120.3A CN201911254120A CN111028171A CN 111028171 A CN111028171 A CN 111028171A CN 201911254120 A CN201911254120 A CN 201911254120A CN 111028171 A CN111028171 A CN 111028171A
- Authority
- CN
- China
- Prior art keywords
- denoising
- image
- processed
- model
- noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012549 training Methods 0.000 claims abstract description 39
- 230000000694 effects Effects 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 33
- 230000000007 visual effect Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000007906 compression Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 230000002349 favourable effect Effects 0.000 description 4
- 238000002372 labelling Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method, a device and a server for determining the noise level of an image, which are characterized in that firstly, an acquired image to be processed is input into a plurality of pre-trained denoising models, a denoised image to be processed corresponding to each denoising model is output, and the denoising strength corresponding to each denoising model is preset; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model; and determining the noise level of the image to be processed according to the denoised image to be processed and the denoising intensity corresponding to each denoising model. According to the method, the to-be-processed images are denoised by adopting the denoising models with different denoising strengths, the denoising grade of the to-be-processed images is determined according to the denoising effect of each denoising model, and compared with a mode of judging the noise grade of the images through visual comparison, the mode improves the accuracy of judging the denoising grade, and further improves the denoising effect of the denoising model obtained by training based on the to-be-processed images marked with the denoising grades.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, and a server for determining a noise level of an image.
Background
For an image with noise, a deep learning mode can be generally adopted to train a denoising model, and the noise in the image is removed through the denoising model, so that the image quality is improved. The quality of the image after the noise removal is often dependent on the denoising effect of the denoising model; and the denoising effect of the denoising model is related to the sample set for training the denoising model. A sample image in a sample set needs to be labeled with its noise level.
In the related art, a marking person needs to preset a standard image of each noise level, and then compares the sample image with the standard image of each noise level through naked eyes of the marking person to determine the noise level of the sample image and mark the noise level, but because the image content of the sample image to be marked is different from that of the standard image, the accuracy of judging the noise level of the sample image through the naked eye comparison is low, and the denoising effect of the denoising model obtained based on the sample image training is poor.
Disclosure of Invention
The invention aims to provide a method, a device and a server for determining the noise level of an image, so as to improve the accuracy of determining the noise level of a sample image and further improve the denoising effect of a denoising model obtained based on sample image training.
In a first aspect, an embodiment of the present invention provides a method for determining a noise level of an image, where the method includes: acquiring an image to be processed; inputting the image to be processed into a plurality of pre-trained denoising models, and outputting the denoised image to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model; and determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising intensity corresponding to each denoising model.
In an alternative embodiment, each of the denoising models is obtained by training in the following manner: determining a sample image pair based on a sample set corresponding to the current denoising model; the sample image pair comprises a standard image and a noise image obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model; and training a preset initial network model through the sample image pair to obtain a current denoising model.
In an optional embodiment, the initial network model includes a network add path and a network drop path; the network uplink comprises a first convolution block, a residual error body and a second convolution block; the network drop comprises a third volume block.
In an optional embodiment, the step of training the preset initial network model through the sample image pair to obtain the current denoising model includes: respectively inputting the noise images in the sample image pair into a first volume block and a third volume block; in the network path, processing the noise image by the first convolution block, the residual error body and the second convolution block in sequence to obtain a first intermediate result; in the network downlink, processing the noise image through a third convolution block to obtain a second intermediate result; adding the first intermediate result and the second intermediate result to obtain an addition result; calculating a loss value according to the addition result and a standard image in the sample image pair, adjusting parameters in the initial network model according to the loss value, and continuously executing the step of determining the sample image pair based on a sample set corresponding to the current denoising model until the loss value is converged to obtain the current denoising model.
In an optional embodiment, the step of determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model includes: selecting a denoised image to be processed with the optimal display effect from the denoised images to be processed corresponding to each denoising model; and determining the denoising intensity of the denoising model corresponding to the denoised image to be processed with the optimal display effect as the noise level of the image to be processed.
In an optional embodiment, after the step of determining the noise level of the image to be processed, the method further includes: and taking the noise level of the image to be processed as the marking information of the image to be processed, and carrying the marking information to the image to be processed.
In a second aspect, an embodiment of the present invention provides an apparatus for determining a noise level of an image, where the apparatus includes: the image acquisition module is used for acquiring an image to be processed; the noise processing module is used for inputting the images to be processed into a plurality of pre-trained denoising models and outputting the denoised images to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model; and the noise level determining module is used for determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model.
In an optional embodiment, the apparatus further comprises a model training module, configured to: determining a sample image pair based on a sample set corresponding to the current denoising model; the sample image pair comprises a standard image and a noise image obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model; and training a preset initial network model through the sample image pair to obtain a current denoising model.
In a third aspect, an embodiment of the present invention provides a server, including a processor and a memory, where the memory stores machine executable instructions capable of being executed by the processor, and the processor executes the machine executable instructions to implement the method for determining the image noise level according to any one of the foregoing embodiments.
In a fourth aspect, embodiments of the invention provide a machine-readable storage medium having stored thereon machine-executable instructions that, when invoked and executed by a processor, cause the processor to carry out a method of determining an image noise level as described in any one of the preceding embodiments.
The embodiment of the invention has the following beneficial effects:
the invention provides a method, a device and a server for determining the noise level of an image, which are characterized in that firstly, an image to be processed is obtained; inputting the image to be processed into a plurality of pre-trained denoising models, and outputting a denoised image to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model; and then determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising intensity corresponding to each denoising model. According to the method, the to-be-processed images are denoised by adopting the denoising models with different denoising strengths, then the denoising grade of the to-be-processed images is determined according to the denoising effect of each denoising model, compared with a mode of judging the noise grade of the images through visual comparison, the mode improves the accuracy of judging the denoising grade, and is favorable for improving the denoising effect of the denoising model obtained by training based on the to-be-processed images marked with the denoising grades.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention as set forth above.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a method for determining a noise level of an image according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for training a denoising model according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an initial network model according to an embodiment of the present invention;
FIG. 4 is a flow chart of another method for determining a noise level of an image according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an apparatus for determining a noise level of an image according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Due to the limitation of storage space and transmission bandwidth, images transmitted on a network are usually compressed images obtained through multiple times of compression, compression noise is generated in the compression process of the images, and the image distortion may be caused by the high compression noise to influence user experience. In the related art, for an image with noise, a deep learning mode can be generally adopted to train a denoising model, and the noise in the image is removed through the denoising model, so that the image quality is improved. The sample images in the training sample set are typically required to be labeled with noise levels.
In the related technology, a marking person compares a sample image with a standard image of each noise level by naked eyes to further determine the noise level of the sample image and marks the noise level, but because the image content of the sample image to be marked is different from that of the standard image, the difficulty of judging the noise level of the sample image by the naked eyes is high, the accuracy is low, and the denoising effect of a denoising model obtained based on sample image training is poor.
Based on the above, the embodiments of the present invention provide a method, an apparatus, and a server for determining an image noise level, where the technique may be applied in noise level determination scenarios of various image noises, especially noise level determination scenarios of compression noise and gaussian noise. To facilitate understanding of the present embodiment, a detailed description is first provided for a method for determining a noise level of an image, as disclosed in the embodiment of the present invention, and as shown in fig. 1, the method includes the following specific steps:
step S102, acquiring an image to be processed.
The image to be processed is usually a photo taken by a camera, or a video frame in a video; the image to be processed can be an image with blurring or loss of detail texture caused by the existence of noise; the image to be processed may include a person, an animal, a building, a landscape, and the like.
Step S104, inputting the to-be-processed image into a plurality of pre-trained denoising models, and outputting a denoised to-be-processed image corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; and each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model.
The denoising model can be a neural network model or a deep learning model. Each denoising model presets a corresponding denoising intensity, which can be represented by letters or numbers, for example, 1, 2, 3, etc., and generally, the larger the number is, the higher the denoising intensity is. The denoising strength can be determined by a sample set of a training model, namely the strength of noise existing in a sample image in the sample set, and the denoising strength of the denoising model is determined. During specific implementation, the number of the denoising models can be set according to user requirements, and the more the denoising models are set, the more corresponding denoising strength gears are, and the more subsequent noise levels can be determined.
The image to be processed is input into the denoising model, the denoising model can perform denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model, and due to the fact that the denoising intensity corresponding to each denoising model is different, the image to be processed output by each denoising model after denoising is different for the same image to be processed, namely the denoising result of each denoising model on the same image to be processed is different.
And S106, determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model.
The denoising effect of the denoised image to be processed corresponding to each denoising model is different, and the denoising effect is best when the denoising intensity of the denoising model is the same as or matched with the denoising intensity existing in the image to be processed; the larger the difference between the denoising intensity of the denoising model and the noise intensity existing in the image to be processed is, the worse the denoising effect is; the noise intensity existing in the image to be processed corresponds to the noise level, which can also be understood as that the denoising intensity corresponds to the noise level, that is, the stronger the denoising intensity is, the higher the noise level of the image to be processed is, wherein the noise level can be represented by a number, for example, 1, 2, 3, etc., and the larger the number is, the larger the noise level is.
In specific implementation, the noise level appearing in the to-be-processed image can be obtained through the denoising effect of the denoising model, for example, the noise level of the to-be-processed image can be determined according to the denoising intensity corresponding to the denoising model with the best denoising effect, and generally, the denoising effect is best, namely, the noise in the to-be-processed image can be effectively removed, and the detail information in the image is kept as much as possible. For example, the denoising model with the denoising intensity of 1 has the best denoising effect on the image to be processed, and 1 is determined as the noise level of the image to be processed.
The invention provides a method for determining the noise level of an image, which comprises the steps of firstly obtaining an image to be processed; inputting the image to be processed into a plurality of pre-trained denoising models, and outputting a denoised image to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model; and then determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising intensity corresponding to each denoising model. According to the method, the to-be-processed images are denoised by adopting the denoising models with different denoising strengths, then the denoising grade of the to-be-processed images is determined according to the denoising effect of each denoising model, compared with a mode of judging the noise grade of the images through visual comparison, the mode improves the accuracy of judging the denoising grade, and is favorable for improving the denoising effect of the denoising model obtained by training based on the to-be-processed images marked with the denoising grades.
The method is realized on the basis of the method of the embodiment; the method mainly describes a specific process for training each denoising model, and provides a method for training the denoising models as shown in fig. 2, wherein the method comprises the following specific steps:
step S202, determining a sample image pair based on a sample set corresponding to a current denoising model; the sample image pair comprises a standard image and a noise image obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model.
The sample set usually includes a plurality of standard images, which are usually high-definition images, and noise images can be obtained by adding noise with different denoising intensities to the high-definition images. The sample image pair generally comprises a standard image and a noise image corresponding to the standard image, and in the specific implementation, under the condition that the denoising intensity corresponding to the denoising model is determined, the standard image can be obtained from the sample set, and then the noise corresponding to the denoising intensity is added to the standard image to obtain the noise image, so that a sample image pair is obtained; or directly adding noise according to the intensity of the denoising intensity to all the standard images in the sample set to obtain a noise image, so as to combine the standard images and the corresponding noise images into a sample image pair, and storing the sample image pair in the sample set to extract the required sample image pair from the sample set.
The sample set may adopt a data set DIV2K, where the data set DIV2K generally includes 1000 high-definition images (2K resolution), where 800 images serve as a training set, 100 images serve as a verification set, and another 100 images serve as a test set, and at the same time, the images in the data set DIV2K are compressed or reduced in quality, so as to obtain low-definition images (corresponding to the above noise images), and the images in the data set DIV2K may be compressed or reduced in quality generally according to the number of denoising models and corresponding denoising strengths. The more the quality of the image in the data set DIV2K is generally degraded, the higher the noise intensity of the resulting noisy image.
For example, assuming that the quality of the high-definition image of the data set DIV2K is 100, and the quality of the high-definition image is reduced to 20, 30, 40, 60, and 80, respectively, five sets of noise images with different noise intensities can be obtained, wherein the noise intensity corresponding to the noise image with the quality of 20 is the highest, and the noise intensity corresponding to the noise image with the quality of 80 is the weakest. The five groups of noise images with different noise intensities respectively correspond to five groups of denoising models with different denoising intensities, for example, a noise image corresponding sample set with the quality of 20 can be trained to obtain a strong denoising model, a noise image corresponding sample set with the quality of 30 can be trained to obtain a medium-strong denoising model, a noise image corresponding sample set with the quality of 40 can be trained to obtain a medium denoising model, a noise image corresponding sample set with the quality of 60 can be trained to obtain a medium-weak denoising model, and a noise image corresponding sample set with the quality of 80 can be trained to obtain a weak denoising model.
And S204, training a preset initial network model through the sample image pair to obtain a current denoising model.
Inputting the sample image pair into a preset initial network model to obtain an output result, adjusting model parameters in the initial network model according to the output result, and continuing to execute the step of determining the sample image pair based on the sample set corresponding to the current denoising model until the output result is in accordance with expectation or the preset training iteration number (for example, 500 times) is reached, finishing training and obtaining the current denoising model. In a specific implementation, each denoising model can be determined by using the training method of the current denoising model, and in the training process, a sample image pair corresponding to each denoising model is different, that is, the standard images in the sample image pair are the same, but the noise intensity of the noise image corresponding to each standard image is different.
In some embodiments, the initial network model includes a network add and a network drop; the network uplink comprises a first convolution block, a residual error body and a second convolution block; the network downlink includes a third volume block, and fig. 3 is a schematic structural diagram of the initial network model. The first convolution block in fig. 3 is typically composed of convolution layers, normalization layers and activation function layers, wherein the convolution layers may be 3 × 3 convolution or 5 × 5 convolution; the residual volume typically consists of a number of residual blocks, typically consisting of 2 3 x 3 convolutions, 2 normalization layers and 1 activation function layer, the second and third convolution blocks typically having the same structure as the first convolution block.
Based on the network structure of the initial model, the step S204 can be implemented by the following steps 10-13:
and step 10, respectively inputting the noise images in the sample image pair into the first volume block and the third volume block.
Step 11, in the network path, processing the noise image by the first convolution block, the residual error body and the second convolution block in sequence to obtain a first intermediate result; and in the network downlink, processing the noise image through a third convolution block to obtain a second intermediate result.
In the network path, firstly performing convolution processing on a noise image through a first convolution block, then sending the noise image after the convolution processing into a residual error body, outputting a characteristic diagram corresponding to the noise image, and then sending the characteristic diagram to a second convolution module for convolution processing to obtain a first intermediate result, wherein the first intermediate result is the characteristic diagram after the convolution processing; in the network downlink, the noise image is convolved by three convolution blocks to obtain a second intermediate result, and the second intermediate result is the noise image after convolution.
Step 12, adding the first intermediate result and the second intermediate result to obtain an addition result; the addition result may be an image obtained by superimposing the convolution-processed feature map and the convolution-processed noise image.
And step 13, calculating a loss value according to the addition result and a standard image in the sample image pair, adjusting parameters in the initial network model according to the loss value, and continuously executing the step of determining the sample image pair based on the sample set corresponding to the current denoising model until the loss value is converged to obtain the current denoising model.
In specific implementation, a loss value corresponding to the initial network model can be obtained according to the addition result and a residual error value or a difference value of the standard image; the addition result and the standard image may be input into a preset loss function, so as to obtain a loss value, where the loss function may be a mean square error loss function, a cross entropy loss function, a difference loss function, or the like. In general, the loss value may characterize the difference between the addition result and the standard image, and in general, the larger the loss value, the larger the difference between the addition result and the standard image.
Parameters in the initial network model can be adjusted based on the loss values to achieve the purpose of training. In the training process, a sample image pair is continuously determined from a sample set corresponding to the current denoising model, and a noise image in the sample image pair is input into the initial network model to obtain a loss value, until model parameters in the initial network model converge, namely when the loss value converges, the training is finished to obtain the current denoising model.
Firstly, determining a sample image pair based on a sample set corresponding to a current denoising model; the sample image pair comprises a standard image and a noise image which is obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model, and then the preset initial network model is trained through the sample image pair to obtain the current denoising model. The denoising model obtained by the mode training can perform denoising processing on the image according to the denoising intensity, so that the denoising processing on the image to be processed is facilitated, and the noise level of the image to be processed is obtained according to the denoising level of the denoising image.
The embodiment of the invention also provides another method for determining the noise level of the image, which is realized on the basis of the method of the embodiment; the method mainly describes a specific process of determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model (specifically, the method is realized by the following steps S406-S408); as shown in fig. 4, the training method includes the following steps:
step S402, acquiring an image to be processed.
Step S404, inputting the image to be processed into a plurality of pre-trained denoising models, and outputting a denoised image to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; and each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model.
Step S406, selecting the denoised image to be processed with the optimal display effect from the denoised images to be processed corresponding to each denoising model.
The image to be processed after denoising output by each denoising model can be displayed in a graphical interaction interface of a user terminal, a user can judge and display the image with the best denoising effect in the denoised images to be processed corresponding to the denoising models through naked eyes, then the user triggers a control or a button corresponding to the image with the good denoising effect in the graphical interaction interface, and an image determination designation is sent to determine the denoised image with the best display effect; the denoised image to be processed can also be calculated by adopting a preset algorithm or formula, and the denoised image to be processed with the optimal display effect can be automatically determined based on the calculation result. The preset algorithm or formula can calculate the size, display accuracy and the like of pixel points in the image, and can also calculate detail information in the image.
Step S408, determining the denoising intensity of the denoising model corresponding to the denoised image to be processed with the optimal display effect as the noise level of the image to be processed.
For example, when there are five denoising models, the denoising strength of the denoising model can be represented as 5, 4, 3, 2, and 1 from large to small, if the effect of the denoised image to be processed output by the denoising model with the denoising strength of 3 is optimal, the noise level of the image to be processed is determined as 3, and it can also be understood that the denoising model with the denoising strength of 3 can effectively remove the noise in the image to be processed, and can retain the detail information in the original image as much as possible.
Step S410, taking the noise level of the image to be processed as the labeling information of the image to be processed, and carrying the labeling information to the image to be processed.
The noise level is marked on the image to be processed, the image to be processed can be used as a training image, and the accuracy of the standard information of the training image is favorable for improving the denoising effect of a denoising model obtained by training based on the image to be processed marked with the denoising level.
Firstly, inputting an acquired image to be processed into a plurality of pre-trained denoising models, and outputting a denoised image to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; selecting a denoised image to be processed with the optimal display effect from the denoised image to be processed corresponding to each denoising model; determining the denoising intensity of the denoising model corresponding to the denoised image to be processed with the optimal display effect as the noise level of the image to be processed; and then taking the noise level of the image to be processed as the marking information of the image to be processed, and carrying the marking information to the image to be processed. According to the invention, the to-be-processed image is subjected to denoising treatment through the denoising models corresponding to different denoising grades, so that a plurality of to-be-processed images with the same content after denoising can be obtained, and the image display effect can be more conveniently compared by naked eyes, thereby reducing the difficulty of determining the noise grade and simultaneously improving the efficiency and the accuracy of image noise grade marking.
Corresponding to the embodiment of the method for determining the noise level of the image, the embodiment of the present invention further provides an apparatus for determining the noise level of the image, as shown in fig. 5, the apparatus includes:
and the image acquisition module 50 is used for acquiring the image to be processed.
The noise processing module 51 is configured to input the to-be-processed image into a plurality of pre-trained denoising models, and output a denoised to-be-processed image corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; and each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model.
And the noise level determining module 52 is configured to determine a noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model.
The device for determining the noise level of the image firstly acquires an image to be processed; inputting the image to be processed into a plurality of pre-trained denoising models, and outputting a denoised image to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model; and then determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising intensity corresponding to each denoising model. In the method, the to-be-processed images are denoised by adopting the denoising models with different denoising strengths, then the denoising grade of the to-be-processed images is determined according to the denoising effect of each denoising model, compared with a method for judging the noise grade of the images through visual comparison, the method improves the accuracy of judging the denoising grade, and is favorable for improving the denoising effect of the denoising model obtained by training based on the to-be-processed images marked with the denoising grades.
Further, the apparatus further comprises a model training module configured to: determining a sample image pair based on a sample set corresponding to the current denoising model; the sample image pair comprises a standard image and a noise image obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model; and training a preset initial network model through the sample image pair to obtain a current denoising model.
Specifically, the initial network model includes a network add path and a network drop path; the network uplink comprises a first convolution block, a residual error body and a second convolution block; the network drop comprises a third volume block.
Further, the model training module is further configured to: respectively inputting the noise images in the sample image pair into a first volume block and a third volume block; in the network path, processing the noise image by the first convolution block, the residual error body and the second convolution block in sequence to obtain a first intermediate result; in the network downlink, processing the noise image through a third convolution block to obtain a second intermediate result; adding the first intermediate result and the second intermediate result to obtain an addition result; calculating a loss value according to the addition result and a standard image in the sample image pair, adjusting parameters in the initial network model according to the loss value, and continuously executing the step of determining the sample image pair based on a sample set corresponding to the current denoising model until the loss value is converged to obtain the current denoising model.
Further, the noise processing module 51 is configured to: selecting a denoised image to be processed with the optimal display effect from the denoised images to be processed corresponding to each denoising model; and determining the denoising intensity of the denoising model corresponding to the denoised image to be processed with the optimal display effect as the noise level of the image to be processed.
Further, the device further comprises an information labeling module, configured to: and taking the noise level of the image to be processed as the marking information of the image to be processed, and carrying the marking information to the image to be processed.
The implementation principle and the generated technical effect of the apparatus for determining the noise level of an image provided by the embodiment of the present invention are the same as those of the foregoing method embodiment, and for the sake of brief description, no mention is made in the apparatus embodiment, and reference may be made to the corresponding contents in the foregoing method embodiment.
An embodiment of the present invention further provides a server, which is shown in fig. 6 and includes a processor 101 and a memory 100, where the memory stores machine executable instructions capable of being executed by the processor 101, and the processor 101 executes the machine executable instructions to implement the method for determining the noise level of an image.
Further, the server shown in fig. 6 further includes a bus 102 and a communication interface 103, and the processor 101, the communication interface 103 and the memory 100 are connected through the bus 102.
The memory 100 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used. The bus 102 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 6, but that does not indicate only one bus or one type of bus.
The processor 101 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 101. The processor 101 may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100, and completes the steps of the method of the foregoing embodiment in combination with the hardware thereof.
The embodiment of the present invention further provides a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the method for determining the image noise level, and specific implementation may refer to method embodiments, and is not described herein again.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the apparatus and/or the electronic device described above may refer to corresponding processes in the foregoing method embodiments, and are not described herein again.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method of determining a noise level of an image, the method comprising:
acquiring an image to be processed;
inputting the images to be processed into a plurality of pre-trained denoising models, and outputting the denoised images to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model;
and determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model.
2. The method of claim 1, wherein each denoising model is trained by:
determining a sample image pair based on a sample set corresponding to the current denoising model; the sample image pair comprises a standard image and a noise image obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model;
and training a preset initial network model through the sample image pair to obtain the current denoising model.
3. The method of claim 2, wherein the initial network model comprises a network up-route and a network down-route; the network uplink comprises a first convolution block, a residual error body and a second convolution block; the network drop comprises a third volume block.
4. The method of claim 3, wherein the step of training a preset initial network model through the sample image pair to obtain the current denoising model comprises:
inputting the noise images in the sample image pair to the first and third convolution blocks, respectively;
in the network uplink, processing the noise image sequentially through the first convolution block, the residual error body and the second convolution block to obtain a first intermediate result; in the network downlink, processing the noise image through the third convolution block to obtain a second intermediate result;
adding the first intermediate result and the second intermediate result to obtain an addition result;
calculating a loss value according to the addition result and the standard image in the sample image pair, adjusting parameters in the initial network model according to the loss value, and continuing to execute the step of determining the sample image pair based on the sample set corresponding to the current denoising model until the loss value is converged to obtain the current denoising model.
5. The method according to claim 1, wherein the step of determining the noise level of the image to be processed according to the denoised image corresponding to each of the denoising models and the denoising strength corresponding to each of the denoising models comprises:
selecting a denoised image to be processed with the optimal display effect from the denoised image to be processed corresponding to each denoising model;
and determining the denoising intensity of the denoising model corresponding to the denoised image to be processed with the optimal display effect as the noise level of the image to be processed.
6. The method according to any one of claims 1 to 5, characterized in that after the step of determining the noise level of the image to be processed, the method further comprises:
and taking the noise level of the image to be processed as the marking information of the image to be processed, and carrying the marking information to the image to be processed.
7. An apparatus for determining a noise level of an image, the apparatus comprising:
the image acquisition module is used for acquiring an image to be processed;
the noise processing module is used for inputting the images to be processed into a plurality of pre-trained denoising models and outputting the denoised images to be processed corresponding to each denoising model; each denoising model is preset with denoising intensity corresponding to the denoising model; each denoising model carries out denoising processing on the image to be processed according to the denoising intensity corresponding to the denoising model;
and the noise level determining module is used for determining the noise level of the image to be processed according to the denoised image to be processed corresponding to each denoising model and the denoising strength corresponding to each denoising model.
8. The apparatus of claim 7, further comprising a model training module to:
determining a sample image pair based on a sample set corresponding to the current denoising model; the sample image pair comprises a standard image and a noise image obtained by adding noise to the standard image based on the denoising intensity corresponding to the current denoising model;
and training a preset initial network model through the sample image pair to obtain the current denoising model.
9. A server comprising a processor and a memory, the memory storing machine executable instructions executable by the processor, the processor executing the machine executable instructions to implement the method of determining an image noise level of any of claims 1-6.
10. A machine-readable storage medium having stored thereon machine-executable instructions which, when invoked and executed by a processor, cause the processor to carry out the method of determining the noise level of an image of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911254120.3A CN111028171B (en) | 2019-12-06 | 2019-12-06 | Method, device and server for determining noise level of image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911254120.3A CN111028171B (en) | 2019-12-06 | 2019-12-06 | Method, device and server for determining noise level of image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111028171A true CN111028171A (en) | 2020-04-17 |
| CN111028171B CN111028171B (en) | 2023-04-18 |
Family
ID=70208304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911254120.3A Active CN111028171B (en) | 2019-12-06 | 2019-12-06 | Method, device and server for determining noise level of image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111028171B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696064A (en) * | 2020-06-15 | 2020-09-22 | 北京金山云网络技术有限公司 | Image processing method, image processing device, electronic equipment and computer readable medium |
| CN111738956A (en) * | 2020-06-24 | 2020-10-02 | 哈尔滨工业大学 | Image denoising system based on characteristic modulation |
| CN111768351A (en) * | 2020-06-22 | 2020-10-13 | Oppo广东移动通信有限公司 | Image denoising method, image denoising device, storage medium and electronic device |
| CN113362259A (en) * | 2021-07-13 | 2021-09-07 | 商汤集团有限公司 | Image noise reduction processing method and device, electronic equipment and storage medium |
| CN113610734A (en) * | 2021-08-23 | 2021-11-05 | 眸芯科技（上海）有限公司 | Image edge processing method based on oriented filtering and application |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204462A (en) * | 2015-05-04 | 2016-12-07 | 南京邮电大学 | Non-local mean denoising method based on image multiple features fusion |
| US20170213321A1 (en) * | 2016-01-22 | 2017-07-27 | Siemens Healthcare Gmbh | Deep Unfolding Algorithm For Efficient Image Denoising Under Varying Noise Conditions |
| WO2017153410A1 (en) * | 2016-03-07 | 2017-09-14 | Connaught Electronics Ltd. | Method for generating a noise-reduced image based on a noise model of multiple images, as well as camera system and motor vehicle |
| CN108280811A (en) * | 2018-01-23 | 2018-07-13 | 哈尔滨工业大学深圳研究生院 | A kind of image de-noising method and system based on neural network |
| CN109285129A (en) * | 2018-09-06 | 2019-01-29 | 哈尔滨工业大学 | Image real noise based on convolutional neural networks removes system |
| US20190156524A1 (en) * | 2017-11-20 | 2019-05-23 | ClariPI Inc. | Apparatus and method for ct image denoising based on deep learning |
| CN109978778A (en) * | 2019-03-06 | 2019-07-05 | 浙江工业大学 | Convolutional neural networks medicine CT image denoising method based on residual error study |
| CN110111266A (en) * | 2019-04-08 | 2019-08-09 | 西安交通大学 | A kind of approximate information pass-algorithm improved method based on deep learning denoising |
-
2019
- 2019-12-06 CN CN201911254120.3A patent/CN111028171B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204462A (en) * | 2015-05-04 | 2016-12-07 | 南京邮电大学 | Non-local mean denoising method based on image multiple features fusion |
| US20170213321A1 (en) * | 2016-01-22 | 2017-07-27 | Siemens Healthcare Gmbh | Deep Unfolding Algorithm For Efficient Image Denoising Under Varying Noise Conditions |
| WO2017153410A1 (en) * | 2016-03-07 | 2017-09-14 | Connaught Electronics Ltd. | Method for generating a noise-reduced image based on a noise model of multiple images, as well as camera system and motor vehicle |
| US20190156524A1 (en) * | 2017-11-20 | 2019-05-23 | ClariPI Inc. | Apparatus and method for ct image denoising based on deep learning |
| CN108280811A (en) * | 2018-01-23 | 2018-07-13 | 哈尔滨工业大学深圳研究生院 | A kind of image de-noising method and system based on neural network |
| CN109285129A (en) * | 2018-09-06 | 2019-01-29 | 哈尔滨工业大学 | Image real noise based on convolutional neural networks removes system |
| CN109978778A (en) * | 2019-03-06 | 2019-07-05 | 浙江工业大学 | Convolutional neural networks medicine CT image denoising method based on residual error study |
| CN110111266A (en) * | 2019-04-08 | 2019-08-09 | 西安交通大学 | A kind of approximate information pass-algorithm improved method based on deep learning denoising |
Non-Patent Citations (3)
| Title |
|---|
| FILIPPOS KOKKINOS等: "Iterative Joint Image Demosaicking and Denoising Using a Residual Denoising Network" * |
| ROBERT WALECKI等: "Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity" * |
| 徐少平等: "基于两阶段支持向量回归的快速噪声水平估计算法" * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696064A (en) * | 2020-06-15 | 2020-09-22 | 北京金山云网络技术有限公司 | Image processing method, image processing device, electronic equipment and computer readable medium |
| CN111696064B (en) * | 2020-06-15 | 2023-08-18 | 北京金山云网络技术有限公司 | Image processing method, device, electronic equipment and computer readable medium |
| CN111768351A (en) * | 2020-06-22 | 2020-10-13 | Oppo广东移动通信有限公司 | Image denoising method, image denoising device, storage medium and electronic device |
| CN111738956A (en) * | 2020-06-24 | 2020-10-02 | 哈尔滨工业大学 | Image denoising system based on characteristic modulation |
| CN111738956B (en) * | 2020-06-24 | 2021-06-01 | 哈尔滨工业大学 | Image denoising system based on characteristic modulation |
| CN113362259A (en) * | 2021-07-13 | 2021-09-07 | 商汤集团有限公司 | Image noise reduction processing method and device, electronic equipment and storage medium |
| CN113362259B (en) * | 2021-07-13 | 2024-01-09 | 商汤集团有限公司 | Image noise reduction processing method and device, electronic equipment and storage medium |
| CN113610734A (en) * | 2021-08-23 | 2021-11-05 | 眸芯科技（上海）有限公司 | Image edge processing method based on oriented filtering and application |
| CN113610734B (en) * | 2021-08-23 | 2023-10-03 | 眸芯科技（上海）有限公司 | Image edge processing method based on guided filtering and application |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111028171B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111028171B (en) | Method, device and server for determining noise level of image | |
| Ndajah et al. | SSIM image quality metric for denoised images | |
| CN110232670B (en) | Method for enhancing visual effect of image based on high-low frequency separation | |
| CN109859152B (en) | Model generation method, image enhancement method, device and computer-readable storage medium | |
| CN112053308B (en) | Image deblurring method and device, computer equipment and storage medium | |
| Jaroensri et al. | Generating training data for denoising real rgb images via camera pipeline simulation | |
| CN112150400B (en) | Image enhancement method and device and electronic equipment | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| WO2016127271A1 (en) | An apparatus and a method for reducing compression artifacts of a lossy-compressed image | |
| CN107993200A (en) | Picture noise level estimation method based on deep learning | |
| CN112862753A (en) | Noise intensity estimation method and device and electronic equipment | |
| CN111415304A (en) | Underwater vision enhancement method and device based on cascade deep network | |
| EP3913572A1 (en) | Loss function for image reconstruction | |
| CN110111261B (en) | Adaptive balance processing method for image, electronic device and computer readable storage medium | |
| CN111325671B (en) | Network training method and device, image processing method and electronic equipment | |
| CN111784580A (en) | Super-resolution method and device for image and server | |
| CN113870143A (en) | Distribution line inspection image enhancement method and system | |
| CN112801536A (en) | Image processing method and device and electronic equipment | |
| CN111524060B (en) | System, method, storage medium and device for blurring portrait background in real time | |
| CN116109878B (en) | Image reproduction identification method, system, device and storage medium | |
| CN117173021A (en) | Video processing method and device | |
| CN111179245A (en) | Image quality detection method, device, electronic equipment and storage medium | |
| CN111028182A (en) | Image sharpening method and device, electronic equipment and computer-readable storage medium | |
| CN107644412B (en) | Printing effect judgment method and device | |
| RU2405200C2 (en) | Method and device for fast noise filtration in digital images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |