CN110782414A - Dark light image denoising method based on dense connection convolution - Google Patents
Dark light image denoising method based on dense connection convolution Download PDFInfo
- Publication number
- CN110782414A CN110782414A CN201911048121.2A CN201911048121A CN110782414A CN 110782414 A CN110782414 A CN 110782414A CN 201911048121 A CN201911048121 A CN 201911048121A CN 110782414 A CN110782414 A CN 110782414A
- Authority
- CN
- China
- Prior art keywords
- image
- dense connection
- convolution
- denoising
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000003062 neural network model Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 6
- 238000002474 experimental method Methods 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 7
- 230000006870 function Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 238000003491 array Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010034960 Photophobia Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 208000013469 light sensitivity Diseases 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a dim light image denoising method based on dense connection convolution, which comprises the following steps: constructing a dense connection denoising convolutional neural network model and training the dense connection denoising convolutional neural network model; acquiring an image to be processed, and preprocessing the image to be processed to obtain a target image to be processed; and processing the target image to be processed through the trained dense connection denoising convolutional neural network model to obtain a denoised image. The invention can keep a large amount of image detail information even when processing the image in a dim light or backlight scene, obtains a high-quality de-noising image and achieves a good de-noising effect.
Description
Technical Field
The invention relates to the technical field of digital image processing, in particular to a dim light image denoising method based on dense connection convolution.
Background
The image noise refers to unnecessary or redundant interference information existing in image data, the quality of an image is seriously affected by the existence of the noise, and the noise reduction is particularly important.
The existing image denoising method generally utilizes local information of an image to smooth processing, or divides the image into blocks with certain sizes, combines two-dimensional image blocks with similar structures together to form three-dimensional arrays according to the similarity between image blocks, processes the three-dimensional arrays by a joint filtering method, and returns the processed result to an original image through inverse transformation, thereby obtaining the denoised image.
However, when the two image denoising methods are used for processing an image in a dark light or backlight scene, the image usually loses much detail information, a high-quality denoised image cannot be obtained, and a good denoising effect cannot be achieved.
Disclosure of Invention
In view of the above, the present invention provides a dim light image denoising method based on dense connection convolution, which can retain a large amount of image detail information even when processing an image in a dim light or backlight scene, obtain a high-quality denoised image, and achieve a good denoising effect.
The invention is realized by adopting the following scheme: a dim light image denoising method based on dense connection convolution comprises the following steps:
constructing a dense connection denoising convolutional neural network model and training the dense connection denoising convolutional neural network model;
acquiring an image to be processed, and preprocessing the image to be processed to obtain a target image to be processed;
and processing the target image to be processed through the trained dense connection denoising convolutional neural network model to obtain a denoised image.
Further, the constructing of the dense connection denoising convolutional neural network model specifically includes: the dense connection denoising convolutional neural network model comprises eight dense connection convolutional blocks, and the four-channel image is used as the input of the network to process the dense connection convolutional blocks; in the first four blocks, each block is subjected to down sampling, and in the last four blocks, the result with the same size is connected with the previous result to be subjected to up sampling, and then the block processing is carried out; and finally outputting the 3-channel RGB image.
Further, each densely connected convolutional block comprises four convolutional layers, each convolutional layer is connected to each convolutional layer after the last convolutional layer by jumping in the block, and the output of the 1 × 1 convolutional layer is added with the input of the block to obtain the output of the current densely connected convolutional block.
Preferably, the combination of blocks of different sizes is 1/2 times downsampled each time from the size of the input block and finally 2 times upsampled each time until the size of the input is returned, in the same size block, the output of the previous block being connected to the following block.
Further, the training of the dense connection denoising convolutional neural network model specifically comprises:
constructing an experiment training data set, wherein the data set comprises a plurality of groups of exposure images, and each group of exposure images comprises a short exposure image and a long exposure image of the same scene; and training an experimental training data set by using the constructed dense connection convolution neural network model, wherein a loss function is an average value of absolute values of differences between a network output result and a long exposure image, and a training target is the minimum loss function.
Further, the preprocessing the image to be processed specifically includes:
processing the image to be processed into a plurality of single color channel images;
and splicing the multiple single-color channel images to obtain a target image to be processed.
Further, the processing of the target image to be processed by the trained dense connection denoising convolutional neural network model to obtain a denoised image specifically comprises: the neural network model processes the image on different scales through some dense connection convolutions and outputs the image after removing the noise.
Compared with the prior art, the invention has the following beneficial effects: the invention adopts the constructed densely connected convolutional neural network to process the input noise image, thereby realizing that a large amount of image detail information can be reserved even when processing the image in a dim light or backlight scene, obtaining a high-quality de-noised image and achieving good de-noising effect.
Drawings
Fig. 1 is a design diagram of a neural network block according to an embodiment of the present invention.
FIG. 2 is an overall design diagram of a dense connection neural network model according to an embodiment of the present invention.
FIG. 3 is an original image according to an embodiment of the present invention.
Fig. 4 is a diagram illustrating an embodiment of the present invention for reducing the image of fig. 3 by a conventional method.
FIG. 5 is the image of FIG. 3 restored by the method of the embodiment of the invention.
FIG. 6 is an original magnified image within the box of FIG. 3 according to an embodiment of the present invention.
FIG. 7 is an enlarged image within the box of FIG. 3 restored using a prior art method according to an embodiment of the present invention.
FIG. 8 is an enlarged image within the box of FIG. 3 restored using the method of the present embodiment according to the present embodiment of the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiment provides a dim light image denoising method based on dense connection convolution, which comprises the following steps:
constructing a dense connection denoising convolutional neural network model and training the dense connection denoising convolutional neural network model;
acquiring an image to be processed, and preprocessing the image to be processed to obtain a target image to be processed;
and processing the target image to be processed through the trained dense connection denoising convolutional neural network model to obtain a denoised image.
In this embodiment, as shown in fig. 2, the constructing a dense connection denoising convolutional neural network model specifically includes: the dense connection denoising convolutional neural network model comprises eight dense connection convolutional blocks, and the four-channel image is used as the input of the network to process the dense connection convolutional blocks; in the first four blocks, each block is subjected to down sampling, and in the last four blocks, the result with the same size is connected with the previous result to be subjected to up sampling, and then the block processing is carried out; and finally outputting the 3-channel RGB image.
In this embodiment, as shown in fig. 1, each densely connected convolutional block includes four convolutional layers, each convolutional layer is connected to each convolutional layer after it by jumping within the block, the last convolutional layer is 1 × 1 convolutional layer, and the output of the 1 × 1 convolutional layer is added to the input of the block to obtain the output of the current densely connected convolutional block.
Preferably, the combination of blocks of different sizes is 1/2 times downsampled each time from the size of the input block and finally 2 times upsampled each time until the size of the input is returned, in the same size block, the output of the previous block being connected to the following block.
In this embodiment, the training of the dense connection denoising convolutional neural network model specifically includes:
constructing an experiment training data set, wherein the data set comprises a plurality of groups of exposure images, and each group of exposure images comprises a short exposure image and a long exposure image of the same scene; i.e. a sample set of images uploaded by the user, each set of exposure images comprises a short exposure image and a long exposure image of the same scene. The short-exposure image of the same scene refers to an image shot under the condition that the shot scene, light conditions (such as backlight, taillight, dark light and the like), exposure time, light sensitivity, exposure amount and the like are the same, and the long-exposure image of the same scene refers to an image shot under the shooting condition that the short-exposure image is the same and other conditions are obviously better than those of the short-exposure image; and training an experimental training data set by using the constructed dense connection convolution neural network model, wherein a loss function is an average value of absolute values of differences between a network output result and a long exposure image, and a training target is the minimum loss function.
In this embodiment, the preprocessing the image to be processed specifically includes:
processing the image to be processed into a plurality of single color channel images;
and splicing the multiple single-color channel images to obtain a target image to be processed.
In particular, the terminal may convert the channel mode of the acquired image to be processed into a plurality of single-color channel images by calling a preset function. The channels that hold color information for an image are called color channels, and each color channel holds information for a color element in the image. A color channel image composed of information of only one color element is a single color channel image. For example, in an RGB mode image, R is a red channel, G is a green channel, and B is a blue channel.
For example, an Image to be processed acquired by the terminal is an original Image (RAW), that is, unprocessed and uncompressed original Image information, and at this time, the terminal invokes a preset function to convert a multi-color single-channel mode of the original Image into a plurality of single-color channel images. Specifically, the terminal extracts each color in the original image through a called preset function and generates a plurality of single-color channel images. The preset function may be a function in the prior art.
In this embodiment, the processing the target image to be processed through the trained dense connection denoising convolutional neural network model to obtain a denoised image specifically includes: the neural network model processes the image on different scales through some dense connection convolutions and outputs the image after removing the noise.
By adopting the method of the embodiment, a large amount of image detail information can be kept even when the image under a dark light scene or a backlight scene is processed, a high-quality de-noised image is obtained, and a good de-noising effect is achieved.
Specifically, in order to verify the effectiveness of the present embodiment, the denoising processing of the existing method and the method of the present embodiment is applied to the original image shown in fig. 3, and the results are shown in fig. 4 to 8. Fig. 4 is an image denoised by the existing method, and fig. 5 is an image denoised by the method of the present embodiment. For clarity, the present embodiment enlarges the image in the block in fig. 3, as shown in fig. 6, fig. 7 is the image in the block after denoising by using the prior art, and fig. 8 is the image in the block after denoising by using the method of the present embodiment. As can be seen from the attached drawings, the denoising effect of the method of the embodiment is obviously superior to that of the prior art, and the obtained image denoising effect is better.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is directed to preferred embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow. However, any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present invention are within the protection scope of the technical solution of the present invention.
Claims (6)
1. A dim light image denoising method based on dense connection convolution is characterized by comprising the following steps:
constructing a dense connection denoising convolutional neural network model and training the dense connection denoising convolutional neural network model;
acquiring an image to be processed, and preprocessing the image to be processed to obtain a target image to be processed;
and processing the target image to be processed through the trained dense connection denoising convolutional neural network model to obtain a denoised image.
2. The dim-light image denoising method based on dense connection convolution of claim 1, wherein the constructing of the dense connection denoising convolution neural network model specifically comprises: the dense connection denoising convolutional neural network model comprises eight dense connection convolutional blocks, and the four-channel image is used as the input of the network to process the dense connection convolutional blocks; in the first four blocks, each block is subjected to down sampling, and in the last four blocks, the result with the same size is connected with the previous result to be subjected to up sampling, and then the block processing is carried out; and finally outputting the 3-channel RGB image.
3. The method of claim 2, wherein each densely-connected convolution block comprises four convolution layers, each convolution layer is connected to each subsequent convolution layer by jumping within the block, the last convolution layer is a 1 x 1 convolution layer, and the output of the 1 x 1 convolution layer is added to the input of the block to obtain the output of the current densely-connected convolution block.
4. The dim-light image denoising method based on dense connection convolution of claim 1, wherein the training of the dense connection denoising convolution neural network model specifically comprises:
constructing an experiment training data set, wherein the data set comprises a plurality of groups of exposure images, and each group of exposure images comprises a short exposure image and a long exposure image of the same scene; and training an experimental training data set by using the constructed dense connection convolution neural network model, wherein a loss function is an average value of absolute values of differences between a network output result and a long exposure image, and a training target is the minimum loss function.
5. The dim-light image denoising method based on dense connection convolution of claim 1, wherein the preprocessing the image to be processed specifically comprises:
processing the image to be processed into a plurality of single color channel images;
and splicing the multiple single-color channel images to obtain a target image to be processed.
6. The dim-light image denoising method based on dense connection convolution as claimed in claim 1, wherein the processing of the target image to be processed by the trained dense connection denoising convolution neural network model to obtain the denoised image is specifically: the neural network model processes the image on different scales through some dense connection convolutions and outputs the image after removing the noise.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911048121.2A CN110782414A (en) | 2019-10-30 | 2019-10-30 | Dark light image denoising method based on dense connection convolution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911048121.2A CN110782414A (en) | 2019-10-30 | 2019-10-30 | Dark light image denoising method based on dense connection convolution |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110782414A true CN110782414A (en) | 2020-02-11 |
Family
ID=69388107
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911048121.2A Pending CN110782414A (en) | 2019-10-30 | 2019-10-30 | Dark light image denoising method based on dense connection convolution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110782414A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111612709A (en) * | 2020-05-11 | 2020-09-01 | 济南浪潮高新科技投资发展有限公司 | Image noise reduction method based on DnCNNs improvement |
| CN111626950A (en) * | 2020-05-19 | 2020-09-04 | 上海集成电路研发中心有限公司 | Online training device and method for image denoising model |
| CN113610725A (en) * | 2021-08-05 | 2021-11-05 | 深圳市慧鲤科技有限公司 | Picture processing method and device, electronic equipment and storage medium |
| CN113643188A (en) * | 2020-04-27 | 2021-11-12 | 浙江宇视科技有限公司 | Deep learning-based noise reduction method and device, storage medium and equipment |
| WO2023125750A1 (en) * | 2021-12-31 | 2023-07-06 | 虹软科技股份有限公司 | Image denoising method and apparatus, and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109410149A (en) * | 2018-11-08 | 2019-03-01 | 安徽理工大学 | A kind of CNN denoising method extracted based on Concurrent Feature |
| CN110390650A (en) * | 2019-07-23 | 2019-10-29 | 中南大学 | OCT image denoising method based on intensive connection and generation confrontation network |
-
2019
- 2019-10-30 CN CN201911048121.2A patent/CN110782414A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109410149A (en) * | 2018-11-08 | 2019-03-01 | 安徽理工大学 | A kind of CNN denoising method extracted based on Concurrent Feature |
| CN110390650A (en) * | 2019-07-23 | 2019-10-29 | 中南大学 | OCT image denoising method based on intensive connection and generation confrontation network |
Non-Patent Citations (1)
| Title |
|---|
| DI ZHAO等: "End-to-End Denoising of Dark Image Using Residual Dense Network", 《2019 8TH INTERNATIONAL SYMPOSIUM ON NEXT GENERATION ELECTRONICS (ISNE)》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113643188A (en) * | 2020-04-27 | 2021-11-12 | 浙江宇视科技有限公司 | Deep learning-based noise reduction method and device, storage medium and equipment |
| CN111612709A (en) * | 2020-05-11 | 2020-09-01 | 济南浪潮高新科技投资发展有限公司 | Image noise reduction method based on DnCNNs improvement |
| CN111612709B (en) * | 2020-05-11 | 2023-03-28 | 山东浪潮科学研究院有限公司 | Image noise reduction method based on DnCNNs improvement |
| CN111626950A (en) * | 2020-05-19 | 2020-09-04 | 上海集成电路研发中心有限公司 | Online training device and method for image denoising model |
| CN113610725A (en) * | 2021-08-05 | 2021-11-05 | 深圳市慧鲤科技有限公司 | Picture processing method and device, electronic equipment and storage medium |
| WO2023125750A1 (en) * | 2021-12-31 | 2023-07-06 | 虹软科技股份有限公司 | Image denoising method and apparatus, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110782414A (en) | Dark light image denoising method based on dense connection convolution | |
| CN110008817B (en) | Model training method, image processing method, device, electronic equipment and computer readable storage medium | |
| CN110517327B (en) | Underwater image enhancement method based on color correction and contrast stretching | |
| CN108604369B (en) | Method, device and equipment for removing image noise and convolutional neural network | |
| JP4803178B2 (en) | Image processing apparatus, computer program product, and image processing method | |
| JP7362297B2 (en) | Image processing device, image processing method, and program | |
| CN111080541A (en) | Color image denoising method based on bit layering and attention fusion mechanism | |
| CN111784603A (en) | RAW domain image denoising method, computer device and computer readable storage medium | |
| CN106412448B (en) | Wide dynamic range processing method and system based on single frame image | |
| CN113632134B (en) | Method, computer readable storage medium, and HDR camera for generating high dynamic range image | |
| CN113362246A (en) | Image banding artifact removing method, device, equipment and medium | |
| CN107767356B (en) | Image processing method and device | |
| CN111260580A (en) | Image denoising method based on image pyramid, computer device and computer readable storage medium | |
| CN111724312A (en) | Method and terminal for processing image | |
| CN115511755B (en) | Video stream image self-adaptive enhancement method and system | |
| CN105763768A (en) | Image processing method, device and system | |
| JP2013008346A (en) | Edge improving method for image | |
| US10650499B1 (en) | Fast and effective image inpainting for reticle removal | |
| WO2020187423A1 (en) | Noise estimation | |
| US9002133B2 (en) | Multi layered image enhancement technique | |
| CN111383188B (en) | Image processing method, system and terminal equipment | |
| CN106709888B (en) | A kind of high dynamic range images production method based on human vision model | |
| CN113706393A (en) | Video enhancement method, device, equipment and storage medium | |
| CN117196999B (en) | Self-adaptive video stream image edge enhancement method and system | |
| CN112489103B (en) | High-resolution depth map acquisition method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200211 |
|
| RJ01 | Rejection of invention patent application after publication |