CN111583142B - Image noise reduction method and device, electronic equipment and storage medium - Google Patents
Image noise reduction method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111583142B CN111583142B CN202010366153.3A CN202010366153A CN111583142B CN 111583142 B CN111583142 B CN 111583142B CN 202010366153 A CN202010366153 A CN 202010366153A CN 111583142 B CN111583142 B CN 111583142B
- Authority
- CN
- China
- Prior art keywords
- image
- noise
- noise reduction
- reduced
- sample image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 324
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003860 storage Methods 0.000 title claims abstract description 30
- 238000010586 diagram Methods 0.000 claims abstract description 130
- 238000012545 processing Methods 0.000 claims abstract description 57
- 238000012549 training Methods 0.000 claims description 44
- 238000004590 computer program Methods 0.000 claims description 15
- 238000009826 distribution Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000011218 segmentation Effects 0.000 claims description 6
- 230000006872 improvement Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 10
- 235000019580 granularity Nutrition 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- UEJJHQNACJXSKW-UHFFFAOYSA-N 2-(2,6-dioxopiperidin-3-yl)-1H-isoindole-1,3(2H)-dione Chemical compound O=C1C2=CC=CC=C2C(=O)N1C1CCC(=O)NC1=O UEJJHQNACJXSKW-UHFFFAOYSA-N 0.000 description 1
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The disclosure relates to an image noise reduction method and device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring an image to be noise-reduced and a noise reduction force diagram corresponding to the image to be noise-reduced, wherein the noise reduction force diagram is used for identifying noise reduction force corresponding to pixel points contained in the image to be noise-reduced; and carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain a noise reduction image, wherein noise contained in the noise reduction image is less than noise contained in the image to be noise reduced. The embodiment of the disclosure can realize the improvement of the noise reduction precision.
Description
Technical Field
The disclosure relates to the field of computer technology, and in particular, to an image noise reduction method and device, an electronic device and a storage medium.
Background
Because of the light source irradiation and the hardware limitation of the terminal equipment, the shot photo has more noise, so that the problems of image blurring, unclear and the like are caused, so that the noise reduction of the image is an important link in the shooting of a digital camera and a mobile phone.
Disclosure of Invention
The disclosure provides an image noise reduction technical scheme for improving noise reduction effect.
According to an aspect of the present disclosure, there is provided an image noise reduction method including:
acquiring an image to be noise-reduced and a noise reduction force diagram corresponding to the image to be noise-reduced, wherein the noise reduction force diagram is used for identifying noise reduction force corresponding to pixel points contained in the image to be noise-reduced;
and carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain a noise reduction image, wherein noise contained in the noise reduction image is less than noise contained in the image to be noise reduced.
In a possible implementation manner, the obtaining the image to be noise reduced and the noise reduction dynamics diagram corresponding to the image to be noise reduced includes any one of the following:
according to the setting operation of the noise reduction force, determining a noise reduction force diagram corresponding to the image to be noise reduced;
determining a noise reduction force diagram corresponding to the image to be reduced according to equipment parameters of terminal equipment for acquiring the image to be reduced;
preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced.
In a possible implementation manner, the preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced includes:
And carrying out semantic segmentation on the image to be denoised to obtain a denoising dynamics diagram corresponding to the image to be denoised.
In one possible implementation manner, the image to be noise reduced is an image in a first format, and the method further includes:
and performing image format conversion on the noise reduction image in the first format to obtain a noise reduction image in a second format, wherein the noise distribution reduction degree of the image in the first format is higher than that of the image in the second format.
In a possible implementation manner, the noise reduction processing is performed on the image to be noise reduced according to the noise reduction dynamics diagram through a noise reduction network, so as to obtain a noise reduction image, and the method further includes:
and training the noise reduction network through a preset training set, wherein the training set comprises a noise-free sample image, and the noise-free sample image is obtained by superposing a plurality of images in an original format.
In a possible implementation manner, the training the noise reduction network through a preset training set includes:
obtaining a noisy sample image corresponding to the noiseless sample image and a noise reduction force diagram corresponding to the noisy sample image according to the noiseless sample image and an exposure gain value corresponding to the noiseless sample image;
Inputting the noisy sample image and a noise reduction dynamics diagram corresponding to the noisy sample image into the noise reduction network for noise reduction treatment to obtain a noisy sample image after noise reduction;
and training the noise reduction network according to the noise-reduced noisy sample image and the noise-free sample image.
In one possible implementation manner, the obtaining, according to the noise-free sample image and the exposure gain value corresponding to the noise-free sample image, a noise-reduced sample image corresponding to the noise-free sample image and a noise-reduced dynamics diagram corresponding to the noise-reduced sample image includes:
determining an exposure gain value corresponding to the noise-free sample image;
determining first noise and second noise corresponding to the noiseless sample image according to the exposure gain value;
and obtaining a noisy sample image corresponding to the noiseless sample image according to the first noise, the second noise and the noiseless sample image.
In one possible implementation manner, the obtaining, according to the noise-free sample image and the exposure gain value corresponding to the noise-free sample image, a noise-reduction force diagram corresponding to the noise-free sample image and a noise-reduction force diagram corresponding to the noise-reduction sample image further includes:
Determining the noise reduction strength corresponding to the noise-free sample image according to the exposure gain value;
and constructing the noise reduction force diagram according to the noise reduction force, wherein the noise reduction force diagram is consistent with the noise sample image in size.
According to an aspect of the present disclosure, there is provided an image noise reduction apparatus including:
the device comprises an acquisition module, a noise reduction module and a display module, wherein the acquisition module is used for acquiring an image to be reduced in noise and a noise reduction force diagram corresponding to the image to be reduced in noise, and the noise reduction force diagram is used for identifying noise reduction force corresponding to pixel points contained in the image to be reduced in noise;
the processing module is used for carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain a noise reduction image, wherein noise contained in the noise reduction image is less than noise contained in the image to be noise reduced.
In one possible implementation manner, the obtaining module is further configured to:
according to the setting operation of the noise reduction force, determining a noise reduction force diagram corresponding to the image to be noise reduced;
determining a noise reduction force diagram corresponding to the image to be reduced according to equipment parameters of terminal equipment for acquiring the image to be reduced;
preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced.
In one possible implementation manner, the obtaining module is further configured to:
and carrying out semantic segmentation on the image to be denoised to obtain a denoising dynamics diagram corresponding to the image to be denoised.
In one possible implementation manner, the image to be noise reduced is an image in a first format, and the apparatus further includes:
the conversion module is used for carrying out image format conversion on the noise reduction image in the first format to obtain a noise reduction image in the second format, wherein the noise distribution reduction degree of the image in the first format is higher than that of the image in the second format.
In one possible implementation manner, the apparatus is implemented through a noise reduction network, and the apparatus further includes:
the training module is used for training the noise reduction network through a preset training set, the training set comprises a noise-free sample image, and the noise-free sample image is obtained by superposing a plurality of images in an original format.
In one possible implementation, the training module is further configured to:
obtaining a noisy sample image corresponding to the noiseless sample image and a noise reduction force diagram corresponding to the noisy sample image according to the noiseless sample image and an exposure gain value corresponding to the noiseless sample image;
Inputting the noisy sample image and a noise reduction dynamics diagram corresponding to the noisy sample image into the noise reduction network for noise reduction treatment to obtain a noisy sample image after noise reduction;
and training the noise reduction network according to the noise-reduced noisy sample image and the noise-free sample image.
In one possible implementation, the training module is further configured to:
determining an exposure gain value corresponding to the noise-free sample image;
determining first noise and second noise corresponding to the noiseless sample image according to the exposure gain value;
and obtaining a noisy sample image corresponding to the noiseless sample image according to the first noise, the second noise and the noiseless sample image.
In one possible implementation, the training module is further configured to:
determining the noise reduction strength corresponding to the noise-free sample image according to the exposure gain value;
and constructing the noise reduction force diagram according to the noise reduction force, wherein the noise reduction force diagram is consistent with the noise sample image in size.
According to an aspect of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.
According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.
In this way, the image to be noise reduced and the noise reduction force diagram corresponding to the image to be noise reduced can be obtained, and the noise reduction force diagram is used for identifying the noise reduction force corresponding to each pixel point contained in the image to be noise reduced. And further, carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram, so as to obtain a noise reduction image, wherein the noise contained in the noise reduction image is less than the noise contained in the image to be noise reduced. According to the image noise reduction method and device, the electronic equipment and the storage medium, the noise reduction force diagram is used for identifying the noise reduction force corresponding to each pixel point contained in the image to be noise reduced, so that noise reduction treatment can be carried out on different noise granularities of different areas in the image to be noise reduced according to the noise reduction force diagram, the problems of inconsistent overall noise granularity and inconsistent local noise granularity in the image to be noise reduced can be solved, and noise reduction precision can be improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the technical aspects of the disclosure.
FIG. 1 illustrates a flow chart of an image denoising method according to an embodiment of the present disclosure;
FIG. 2 illustrates a schematic diagram of a noise reduction network in accordance with an embodiment of the present disclosure;
FIG. 3 shows a schematic diagram of an image denoising method according to an embodiment of the present disclosure;
FIG. 4 shows a block diagram of an image noise reduction device according to an embodiment of the present disclosure;
fig. 5 illustrates a block diagram of an electronic device 800, according to an embodiment of the disclosure;
fig. 6 illustrates a block diagram of an electronic device 1900 according to an embodiment of the disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Furthermore, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
Fig. 1 shows a flowchart of an image denoising method according to an embodiment of the present disclosure, which may be performed by an electronic device such as a terminal device or a server, and the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like, which may be implemented by a processor invoking computer readable instructions stored in a memory. Alternatively, the method may be performed by a server.
As shown in fig. 1, the image denoising method may include:
in step S11, an image to be noise reduced and a noise reduction force diagram corresponding to the image to be noise reduced are obtained, where the noise reduction force diagram is used to identify noise reduction forces corresponding to each pixel point included in the image to be noise reduced.
For example, the image to be noise reduced may be an image acquired by the terminal device or an image obtained by combining a plurality of images acquired by the terminal device, or the image to be noise reduced may also be an uploaded image. The noise reduction degree is used for representing the degree or the noise reduction level of noise reduction.
In a possible implementation manner, the obtaining the image to be noise reduced and the noise reduction dynamics diagram corresponding to the image to be noise reduced includes any one of the following:
according to the setting operation of the noise reduction force, determining a noise reduction force diagram corresponding to the image to be noise reduced;
determining a noise reduction force diagram corresponding to the image to be reduced according to equipment parameters of terminal equipment for acquiring the image to be reduced;
preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced.
For example, the noise reduction dynamics map corresponding to the image to be reduced may be specified by a user, for example: the noise reduction force diagram may be an image obtained according to a specified noise reduction force corresponding to each pixel point included in the image to be noise reduced. Alternatively, the noise reduction force diagram may be set according to parameters of the terminal device, for example: the noise reduction force diagram can be an exposure gain value corresponding to the image to be noise reduced. Alternatively, the noise reduction force diagram may be obtained by preprocessing an image to be noise reduced, for example: performing semantic segmentation on the image to be denoised to obtain a denoising dynamics diagram; or performing motion detection on a plurality of images which are synthesized into the image to be denoised, and determining the denoising degree of each pixel according to the motion condition of the target object in each image so as to obtain a denoising degree map; or determining the corresponding noise reduction force of each image according to the exposure weight fusion graphs corresponding to the images with different exposure degrees, which are synthesized to the image to be noise reduced, and further determining the corresponding noise reduction force graph of the image to be noise reduced according to the corresponding noise reduction force of the images and the exposure weight fusion graph corresponding to each image.
In a possible implementation manner, the preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced may include:
and carrying out semantic segmentation on the image to be denoised to obtain a denoising dynamics diagram corresponding to the image to be denoised.
For example, the image to be denoised may be semantically segmented, and each object included in the image to be denoised may be segmented and marked, so as to determine a corresponding denoising strength for each object, and obtain a denoising strength map of the object to be denoised.
The source of the noise reduction force diagram in the embodiment of the disclosure is wide, so that the noise reduction method provided by the embodiment of the disclosure is wide in application and friendly to users.
In step S12, the noise reduction processing is performed on the image to be noise reduced according to the noise reduction dynamics diagram, so as to obtain a noise reduction image, where noise included in the noise reduction image is less than noise included in the image to be noise reduced.
For example, the noise reduction image may be obtained by performing noise reduction processing on the noise reduction image according to the noise reduction force corresponding to each pixel point included in the noise reduction image indicated in the noise reduction force diagram. For example, the noise reduction image can be obtained by performing noise reduction processing on the image to be noise reduced through a noise reduction network which is trained in advance and used for performing noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram.
In this way, the image to be noise reduced and the noise reduction force diagram corresponding to the image to be noise reduced can be obtained, and the noise reduction force diagram is used for identifying the noise reduction force corresponding to each pixel point contained in the image to be noise reduced. And further, carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram, so as to obtain a noise reduction image, wherein the noise contained in the noise reduction image is less than the noise contained in the image to be noise reduced. According to the image noise reduction method provided by the embodiment of the disclosure, the noise reduction force diagram is used for identifying the noise reduction force corresponding to each pixel point contained in the image to be reduced, so that different noise granularities of different areas in the image to be reduced can be subjected to noise reduction according to the noise reduction force diagram, the problems of inconsistent global noise granularity and inconsistent local noise granularity in the image to be reduced can be solved, and the noise reduction precision can be improved.
In one possible implementation manner, the image to be noise reduced is an image in a first format, and the method may further include:
and performing image format conversion on the noise reduction image in the first format to obtain a noise reduction image in a second format, wherein the noise distribution reduction degree of the image in the first format is higher than that of the image in the second format.
For example, the image to be noise reduced is an unprocessed original image acquired by the image acquisition device, which corresponds to a first format, where the first format is a format that makes the noise distribution reduction degree of the image higher, for example: the first format may be a raw domain (raw) format, so that after the noise reduction is performed on the image to be reduced to obtain a noise reduction image, the noise reduction image obtained is also the first format, and then the noise reduction image may be subjected to image conversion (for example, image conversion may be performed by any of ISP (Image Signal Processor, image signal processor) processing including operations such as demosaicing and tone mapping, PS (Photoshop, image processing), LD, and the like), and converted from the first format into a second format having a noise distribution reduction degree lower than that of the first format, for example, converted into png (Portable Network Graphics ) format, jpg format, and the like, and then displayed.
Because the noise distribution reduction degree of the image to be reduced in the first format is higher, better noise reduction effect can be obtained through the image to be reduced in the first format, the noise reduction image with higher precision is obtained, and the noise reduction image is converted into the second format and then displayed.
In a possible implementation manner, the noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram may be implemented through a noise reduction network, so as to obtain a noise reduction image, and the method further includes: and training the noise reduction network through a preset training set, wherein the training set comprises a noise-free sample image, and the noise-free sample image is obtained by superposing a plurality of images in an original format.
For example, the noise reduction network for performing noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain the noise reduction image may be trained according to a preset training set. The training set comprises a noise-free sample image, wherein the noise-free sample image is obtained by overlapping unprocessed original format images acquired by a plurality of terminal devices, and the original format images can be RAW format images. For example, N RAW format images may be continuously collected, and the N RAW format images may be superimposed on average to obtain a noise-free sample image.
In one possible implementation, the noise-free sample image may also be obtained by performing inverse ISP processing (e.g., demosaicing, tone mapping, etc.) on the noise-free RGB image.
Therefore, the noise-free sample image is obtained by superposing a plurality of images in an original format, and the noise-free image obtained by simulating and simulating the noise-free sample image is more in line with the actual noise, so that the noise reduction network obtained by training the noise-free sample image can migrate to the practical application and has higher migration performance.
In a possible implementation manner, the training the noise reduction network through a preset training set may include:
obtaining a noisy sample image corresponding to the noiseless sample image and a noise reduction force diagram corresponding to the noisy sample image according to the noiseless sample image and an exposure gain value corresponding to the noiseless sample image;
inputting the noisy sample image and a noise reduction dynamics diagram corresponding to the noisy sample image into the noise reduction network for noise reduction treatment to obtain a noisy sample image after noise reduction;
and training the noise reduction network according to the noise-reduced noisy sample image and the noise-free sample image.
For example, according to the exposure gain value corresponding to the noise-free sample image, noise simulation processing may be performed on the noise-free sample image, so as to obtain a noise-containing sample image corresponding to the noise-free sample image.
In one possible implementation manner, the obtaining, according to the noise-free sample image and the exposure gain value corresponding to the noise-free sample image, a noise-reduced sample image corresponding to the noise-free sample image and a noise-reduced dynamics diagram corresponding to the noise-reduced sample image may include:
determining an exposure gain value corresponding to the noise-free sample image;
determining first noise and second noise corresponding to the noiseless sample image according to the exposure gain value;
and obtaining a noisy sample image corresponding to the noiseless sample image according to the first noise, the second noise and the noiseless sample image.
For example, the exposure gain value corresponding to the terminal device when acquiring the image in the RAW format of the synthesized noise-free sample image may be determined as the exposure gain value corresponding to the noise-free sample image; or the exposure gain value corresponding to the noiseless sample image can be randomly taken as the value.
Because the noise of the image in the original format mainly comprises light quantum noise caused by a light source and readout noise caused by hardware of terminal equipment, and the light quantum noise accords with poisson distribution, and the readout noise accords with gaussian distribution, a noisy sample image obtained by simulation according to a noiseless sample image can approximately accord with single heteroscedure gaussian distribution shown in the following formula (one):
y～N(μ＝x，δ 2 ＝λ read +λ shot x) formula (I)
Wherein y represents a noisy sample image, x represents a non-noisy sample image, y-N () represents a monoheteroscedastic Gaussian distribution function, δ 2 Representing variance, lambda read Represents readout noise lambda shot Representing light quantum noise.
The exposure gain value is related to the analog gain, the digital gain and the readout variance corresponding to the sensor of the terminal device, and different exposure gain values correspond to different analog gain, digital gain and readout variance.
After determining the analog gain, the digital gain and the readout variance, determining a first noise according to the digital gain and the readout variance, wherein the first noise can be the readout noise, and the association relationship between the first noise and the digital gain and the readout variance can refer to a formula (II); the second noise may be determined according to the analog gain and the digital gain, and the second noise may be optical quantum noise, where the correlation relationship between the second noise and the analog gain and the digital gain may refer to formula (iii).
λ read ＝g d 2 δ r 2 Formula II
λ shot ＝g d g a Formula (III)
Wherein g d Digital gain, g, representing the sensor of the terminal device a Analog gain, delta, representing the sensor of the terminal device r 2 Indicating the read variance of the sensor of the terminal device.
In this way, there is a correspondence between the exposure gain value and the first noise and the second noise, and after the exposure gain value is determined, the first noise and the second noise corresponding to the exposure gain value can be obtained according to the correspondence, and then a noisy sample image corresponding to the noiseless sample image can be obtained according to the formula (one).
In one possible implementation manner, the obtaining, according to the noise-free sample image and the exposure gain value corresponding to the noise-free sample image, a noise-reduction force diagram corresponding to the noise-free sample image and a noise-reduction force diagram corresponding to the noise-reduction sample image may further include:
determining the noise reduction strength corresponding to the noise-free sample image according to the exposure gain value;
and constructing the noise reduction force diagram according to the noise reduction force, wherein the noise reduction force diagram is consistent with the noise sample image in size.
For example, the exposure gain value is between a maximum exposure gain value and a minimum exposure gain value, wherein the maximum exposure gain value and the minimum exposure gain value are both preset values. According to the exposure gain value, the maximum exposure gain value and the minimum exposure gain value, the noise reduction force corresponding to the noise-free sample image can be obtained, and the processing process can refer to a formula (IV).
p＝(ISO-ISO min )/(ISO max -ISO min )*p max Formula (IV)
Wherein, p is used for representing noise reduction force, ISO is used for representing exposure gain value, ISO max Indicating the maximum exposure gain value, ISO min Represents the minimum exposure gain value, p max Indicating the maximum noise reduction level (preset value).
After the noise reduction force is determined, a noise reduction force diagram corresponding to the noise reduction sample image can be constructed according to the noise reduction force, the length and the width of the noise reduction force diagram are consistent with those of the noise reduction sample image, and any pixel value in the noise reduction force diagram can be the noise reduction force.
After the noisy sample image and the noise reduction dynamics diagram corresponding to the noisy sample image are obtained, the noisy sample image and the noise reduction dynamics diagram corresponding to the noisy sample image can be input into a noise reduction network to be subjected to noise reduction treatment, so that the noise reduction image corresponding to the noisy sample image is obtained.
By way of example, the network structure of the noise reduction network may be as shown with reference to fig. 2. As shown in fig. 2, after the noisy sample image is split into four channels by the rggb mode or the bggr mode, the four channels are input into a noise reduction network together with the noise reduction dynamics graph, where the noise reduction network includes a first matrix dot-multiplying module, a second matrix dot-multiplying module, a third matrix dot-multiplying module and a matrix adding module. The first matrix dot multiplication module can carry out dot multiplication processing on the input four-way format noisy sample image and the noise reduction dynamics graph to obtain a first processing result. And the second matrix dot multiplication module carries out dot multiplication processing on the first processing result and the noise reduction dynamics graph to obtain a second processing result. And the third matrix dot multiplication module carries out dot multiplication processing on the second processing result and the noise reduction dynamics graph to obtain a third processing result. And the matrix adding module performs adding processing on the third processing result and the four-channel noisy sample image to obtain a noise reduction image.
After obtaining the noise reduction image corresponding to the noise reduction sample image according to the noise reduction network, determining the network loss of the noise reduction network according to the noise reduction image corresponding to the noise reduction sample image and the noise-free sample image corresponding to the noise reduction sample image, and further adjusting the network parameters of the noise reduction network according to the network loss until the noise reduction network meets the training requirements, for example: the network loss of the noise reduction network is less than a loss threshold.
It should be noted that, the network loss of the noise reduction network may be determined using a 1-range or 2-range and other loss functions, and the loss functions are not defined herein in the embodiments of the present disclosure. For example: the network loss of the noise reduction network can be calculated in a 1-range form, and the loss function can be referred to the following formula (five):
wherein L represents the loss of the noise reduction network, H represents the height corresponding to the noisy sample image, W represents the width corresponding to the noisy sample image, T1 represents the noise-free sample image, and T2 represents the noise-reduced image.
In order for those skilled in the art to better understand the disclosed embodiments, the disclosed embodiments are described below by way of specific examples.
Referring to fig. 3, after a noise-free sample image is acquired, a noise-free sample image with known noise strength can be synthesized according to the noise-free sample image, and a noise strength map with the same size as the noise-free sample image can be generated according to the noise strength. Inputting the noisy sample image and the noise dynamics diagram corresponding to the noisy sample image into a noise reduction network to obtain a noise reduction image corresponding to the noisy sample image. The noise reduction network is subjected to learning training through a 1-range formula.
In this way, in the noise reduction network obtained by training in the embodiment of the present disclosure, the noise intensity graph may be applied to the feature graphs of different layers in the network multiple times in an embedded manner, so that the influence of the noise intensity graph on the noise reduction network is enhanced, so that the noise reduction network obtained by training performs noise reduction according to the noise intensity graph to obtain a noise reduction image with higher precision, and because the noise reduction network obtained by training has the capability of adaptively adjusting the noise reduction intensity, the complexity of the training process of the noise reduction network is reduced, not only can an optimal noise reduction effect be obtained, but also a storage space can be effectively saved.
After training to obtain the noise reduction network, the image to be noise reduced and the noise reduction dynamics diagram corresponding to the image to be noise reduced can be input into the noise reduction network to obtain the noise reduction image corresponding to the image to be noise reduced, and the obtained noise reduction image is still in an original format, so that ISP processing can be performed on the noise reduction image, and the noise reduction image can be converted into pictures in formats of png (Portable Network Graphics ) and the like.
It will be appreciated that the above-mentioned method embodiments of the present disclosure may be combined with each other to form a combined embodiment without departing from the principle logic, and are limited to the description of the present disclosure. It will be appreciated by those skilled in the art that in the above-described methods of the embodiments, the particular order of execution of the steps should be determined by their function and possible inherent logic.
In addition, the disclosure further provides an image noise reduction device, an electronic device, a computer readable storage medium, and a program, where the foregoing may be used to implement any one of the image noise reduction methods provided in the disclosure, and corresponding technical schemes and descriptions and corresponding descriptions referring to method parts are not repeated.
Fig. 4 illustrates a block diagram of an image noise reduction apparatus according to an embodiment of the present disclosure, as illustrated in fig. 4, the image noise reduction apparatus may include:
the obtaining module 401 may be configured to obtain an image to be denoised and a denoising dynamics map corresponding to the image to be denoised, where the denoising dynamics map is used to identify denoising dynamics corresponding to pixel points included in the image to be denoised;
the processing module 402 may be configured to perform noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain a noise reduced image, where noise included in the noise reduced image is less than noise included in the image to be noise reduced.
In this way, the image to be noise reduced and the noise reduction force diagram corresponding to the image to be noise reduced can be obtained, and the noise reduction force diagram is used for identifying the noise reduction force corresponding to each pixel point contained in the image to be noise reduced. And further, carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram, so as to obtain a noise reduction image, wherein the noise contained in the noise reduction image is less than the noise contained in the image to be noise reduced. According to the image noise reduction device provided by the embodiment of the disclosure, the noise reduction force diagram is used for identifying the noise reduction force corresponding to each pixel point contained in the image to be reduced, so that the noise reduction treatment can be performed on different noise granularities of different areas in the image to be reduced according to the noise reduction force diagram, the problems of inconsistent global noise granularities and inconsistent local noise granularities in the image to be reduced can be solved, and the noise reduction precision can be improved.
In one possible implementation, the acquiring module may be further configured to:
according to the setting operation of the noise reduction force, determining a noise reduction force diagram corresponding to the image to be noise reduced;
determining a noise reduction force diagram corresponding to the image to be reduced according to equipment parameters of terminal equipment for acquiring the image to be reduced;
preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced.
In one possible implementation manner, the acquiring module may be further configured to:
and carrying out semantic segmentation on the image to be denoised to obtain a denoising dynamics diagram corresponding to the image to be denoised.
In one possible implementation manner, the image to be noise reduced is an image in a first format, and the apparatus may further include:
the conversion module is used for carrying out image format conversion on the noise reduction image in the first format to obtain a noise reduction image in the second format, wherein the noise distribution reduction degree of the image in the first format is higher than that of the image in the second format.
In one possible implementation manner, the apparatus is implemented through a noise reduction network, and the apparatus may further include:
The training module can be used for training the noise reduction network through a preset training set, wherein the training set comprises a noise-free sample image, and the noise-free sample image is obtained by superposing a plurality of images in an original format.
In one possible implementation, the training module may be further configured to:
obtaining a noisy sample image corresponding to the noiseless sample image and a noise reduction force diagram corresponding to the noisy sample image according to the noiseless sample image and an exposure gain value corresponding to the noiseless sample image;
inputting the noisy sample image and a noise reduction dynamics diagram corresponding to the noisy sample image into the noise reduction network for noise reduction treatment to obtain a noisy sample image after noise reduction;
and training the noise reduction network according to the noise-reduced noisy sample image and the noise-free sample image.
In one possible implementation, the training module may be further configured to:
determining an exposure gain value corresponding to the noise-free sample image;
determining first noise and second noise corresponding to the noiseless sample image according to the exposure gain value;
and obtaining a noisy sample image corresponding to the noiseless sample image according to the first noise, the second noise and the noiseless sample image.
In one possible implementation, the training module may be further configured to:
determining the noise reduction strength corresponding to the noise-free sample image according to the exposure gain value;
and constructing the noise reduction force diagram according to the noise reduction force, wherein the noise reduction force diagram is consistent with the noise sample image in size.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a non-volatile computer readable storage medium.
The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.
Embodiments of the present disclosure also provide a computer program product comprising computer readable code which, when run on a device, causes a processor in the device to execute instructions for implementing the image denoising method as provided in any of the embodiments above.
The disclosed embodiments also provide another computer program product for storing computer readable instructions that, when executed, cause a computer to perform the operations of the image denoising method provided in any of the above embodiments.
The electronic device may be provided as a terminal, server or other form of device.
Fig. 5 illustrates a block diagram of an electronic device 800, according to an embodiment of the disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.
Referring to fig. 5, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.
The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.
The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the electronic device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.
Fig. 6 illustrates a block diagram of an electronic device 1900 according to an embodiment of the disclosure. For example, electronic device 1900 may be provided as a server. Referring to FIG. 6, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.
The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A method of image denoising, comprising:
acquiring an image to be noise-reduced and a noise reduction force diagram corresponding to the image to be noise-reduced, wherein the noise reduction force diagram is used for identifying noise reduction force corresponding to pixel points contained in the image to be noise-reduced;
Carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain a noise reduction image, wherein noise contained in the noise reduction image is less than noise contained in the image to be noise reduced;
the noise reduction processing of the image to be noise reduced according to the noise reduction dynamics diagram is achieved through a pre-trained noise reduction network, so that a noise reduction image is obtained, and the noise reduction dynamics diagram is repeatedly acted on feature diagrams of different layers in the noise reduction network in an embedded mode;
the obtaining the image to be noise reduced and the noise reduction dynamics diagram corresponding to the image to be noise reduced comprises any one of the following steps:
according to the setting operation of the noise reduction force, determining a noise reduction force diagram corresponding to the image to be noise reduced;
determining a noise reduction force diagram corresponding to the image to be reduced according to equipment parameters of terminal equipment for acquiring the image to be reduced;
preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced.
2. The method of claim 1, wherein the preprocessing the image to be denoised to obtain a denoising dynamics map corresponding to the image to be denoised, includes:
and carrying out semantic segmentation on the image to be denoised to obtain a denoising dynamics diagram corresponding to the image to be denoised.
3. The method according to claim 1 or 2, wherein the image to be noise reduced is an image of a first format, the method further comprising:
and performing image format conversion on the noise reduction image in the first format to obtain a noise reduction image in a second format, wherein the noise distribution reduction degree of the image in the first format is higher than that of the image in the second format.
4. The method according to claim 1, wherein the method further comprises:
and training the noise reduction network through a preset training set, wherein the training set comprises a noise-free sample image, and the noise-free sample image is obtained by superposing a plurality of images in an original format.
5. The method of claim 4, wherein the training the noise reduction network with a preset training set comprises:
obtaining a noisy sample image corresponding to the noiseless sample image and a noise reduction force diagram corresponding to the noisy sample image according to the noiseless sample image and an exposure gain value corresponding to the noiseless sample image;
inputting the noisy sample image and a noise reduction dynamics diagram corresponding to the noisy sample image into the noise reduction network for noise reduction treatment to obtain a noisy sample image after noise reduction;
And training the noise reduction network according to the noise-reduced noisy sample image and the noise-free sample image.
6. The method according to claim 5, wherein the obtaining the noisy sample image corresponding to the noiseless sample image and the noise reduction dynamics map corresponding to the noisy sample image according to the noiseless sample image and the exposure gain value corresponding to the noiseless sample image includes:
determining an exposure gain value corresponding to the noise-free sample image;
determining first noise and second noise corresponding to the noiseless sample image according to the exposure gain value;
and obtaining a noisy sample image corresponding to the noiseless sample image according to the first noise, the second noise and the noiseless sample image.
7. The method of claim 6, wherein the obtaining the noisy sample image corresponding to the noisy sample image and the noise reduction dynamics map corresponding to the noisy sample image according to the noiseless sample image and the exposure gain value corresponding to the noiseless sample image, further comprises:
determining the noise reduction strength corresponding to the noise-free sample image according to the exposure gain value;
and constructing the noise reduction force diagram according to the noise reduction force, wherein the noise reduction force diagram is consistent with the noise sample image in size.
8. An image noise reduction apparatus, comprising:
the device comprises an acquisition module, a noise reduction module and a display module, wherein the acquisition module is used for acquiring an image to be reduced in noise and a noise reduction force diagram corresponding to the image to be reduced in noise, and the noise reduction force diagram is used for identifying noise reduction force corresponding to pixel points contained in the image to be reduced in noise;
the processing module is used for carrying out noise reduction processing on the image to be noise reduced according to the noise reduction dynamics diagram to obtain a noise reduction image, wherein noise contained in the noise reduction image is less than noise contained in the image to be noise reduced;
the device realizes the noise reduction processing of the image to be noise reduced according to the noise reduction dynamics diagram through a pre-trained noise reduction network to obtain a noise reduction image, and the noise reduction dynamics diagram acts on feature diagrams of different layers in the noise reduction network for a plurality of times in an embedded mode;
the acquisition module is further configured to:
according to the setting operation of the noise reduction force, determining a noise reduction force diagram corresponding to the image to be noise reduced;
determining a noise reduction force diagram corresponding to the image to be reduced according to equipment parameters of terminal equipment for acquiring the image to be reduced;
preprocessing the image to be noise reduced to obtain a noise reduction dynamics diagram corresponding to the image to be noise reduced.
9. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010366153.3A CN111583142B (en) | 2020-04-30 | 2020-04-30 | Image noise reduction method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010366153.3A CN111583142B (en) | 2020-04-30 | 2020-04-30 | Image noise reduction method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111583142A CN111583142A (en) | 2020-08-25 |
| CN111583142B true CN111583142B (en) | 2023-11-28 |
Family
ID=72113339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010366153.3A Active CN111583142B (en) | 2020-04-30 | 2020-04-30 | Image noise reduction method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111583142B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347125B (en) * | 2020-11-16 | 2024-06-11 | 李增国 | Equipment data processing method and Internet of things data processing method |
| CN113689373B (en) * | 2021-10-21 | 2022-02-11 | 深圳市慧鲤科技有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN113689361B (en) * | 2021-10-27 | 2022-02-22 | 深圳市慧鲤科技有限公司 | Image processing method and device, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014068253A (en) * | 2012-09-26 | 2014-04-17 | Nikon Corp | Imaging device and program |
| CN110163827A (en) * | 2019-05-28 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Training method, image de-noising method, device and the medium of image denoising model |
| KR20190099914A (en) * | 2018-02-20 | 2019-08-28 | 삼성전자주식회사 | Electronic apparatus, method for processing image thereof and computer-readable recording medium |
| CN110290289A (en) * | 2019-06-13 | 2019-09-27 | Oppo广东移动通信有限公司 | Image denoising method, device, electronic equipment and storage medium |
| CN110428378A (en) * | 2019-07-26 | 2019-11-08 | 北京小米移动软件有限公司 | Processing method, device and the storage medium of image |
| CN110555808A (en) * | 2018-05-31 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | Image processing method, device, equipment and machine-readable storage medium |
| CN110570364A (en) * | 2019-08-05 | 2019-12-13 | 天津大学 | depth neural network-based sub-focal plane polarization image denoising method |
| CN110599410A (en) * | 2019-08-07 | 2019-12-20 | 北京达佳互联信息技术有限公司 | Image processing method, device, terminal and storage medium |
| CN110728627A (en) * | 2018-07-16 | 2020-01-24 | 宁波舜宇光电信息有限公司 | Image noise reduction method, device, system and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3934506B2 (en) * | 2002-08-06 | 2007-06-20 | オリンパス株式会社 | Imaging system and image processing program |
| CN107818554B (en) * | 2016-09-12 | 2023-04-21 | 索尼公司 | Information processing apparatus and information processing method |
| US10949951B2 (en) * | 2018-08-23 | 2021-03-16 | General Electric Company | Patient-specific deep learning image denoising methods and systems |
-
2020
- 2020-04-30 CN CN202010366153.3A patent/CN111583142B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014068253A (en) * | 2012-09-26 | 2014-04-17 | Nikon Corp | Imaging device and program |
| KR20190099914A (en) * | 2018-02-20 | 2019-08-28 | 삼성전자주식회사 | Electronic apparatus, method for processing image thereof and computer-readable recording medium |
| CN110555808A (en) * | 2018-05-31 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | Image processing method, device, equipment and machine-readable storage medium |
| CN110728627A (en) * | 2018-07-16 | 2020-01-24 | 宁波舜宇光电信息有限公司 | Image noise reduction method, device, system and storage medium |
| CN110163827A (en) * | 2019-05-28 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Training method, image de-noising method, device and the medium of image denoising model |
| CN110290289A (en) * | 2019-06-13 | 2019-09-27 | Oppo广东移动通信有限公司 | Image denoising method, device, electronic equipment and storage medium |
| CN110428378A (en) * | 2019-07-26 | 2019-11-08 | 北京小米移动软件有限公司 | Processing method, device and the storage medium of image |
| CN110570364A (en) * | 2019-08-05 | 2019-12-13 | 天津大学 | depth neural network-based sub-focal plane polarization image denoising method |
| CN110599410A (en) * | 2019-08-07 | 2019-12-20 | 北京达佳互联信息技术有限公司 | Image processing method, device, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111583142A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113538519B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN111310616B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN110889469B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN110390394B (en) | Batch normalization data processing method and device, electronic equipment and storage medium | |
| CN110378976B (en) | Image processing method and device, electronic equipment and storage medium | |
| JP2021528742A (en) | Image processing methods and devices, electronic devices, and storage media | |
| CN111583142B (en) | Image noise reduction method and device, electronic equipment and storage medium | |
| CN111553864B (en) | Image restoration method and device, electronic equipment and storage medium | |
| CN111223040B (en) | Network training method and device, and image generation method and device | |
| CN111340733B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN111539410B (en) | Character recognition method and device, electronic equipment and storage medium | |
| TW202032425A (en) | Method, apparatus and electronic device for image processing and storage medium | |
| CN113139947B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN109685041B (en) | Image analysis method and device, electronic equipment and storage medium | |
| CN112001364A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN111369482B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN110675355B (en) | Image reconstruction method and device, electronic equipment and storage medium | |
| CN110415258B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN110633715B (en) | Image processing method, network training method and device and electronic equipment | |
| CN109447258B (en) | Neural network model optimization method and device, electronic device and storage medium | |
| CN113139484B (en) | Crowd positioning method and device, electronic equipment and storage medium | |
| CN109165722B (en) | Model expansion method and device, electronic equipment and storage medium | |
| CN111583144B (en) | Image noise reduction method and device, electronic equipment and storage medium | |
| CN109635926B (en) | Attention feature acquisition method and device for neural network and storage medium | |
| CN111488964A (en) | Image processing method and device and neural network training method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |