CN111429364A - Image restoration method and device - Google Patents
Image restoration method and device Download PDFInfo
- Publication number
- CN111429364A CN111429364A CN202010124621.6A CN202010124621A CN111429364A CN 111429364 A CN111429364 A CN 111429364A CN 202010124621 A CN202010124621 A CN 202010124621A CN 111429364 A CN111429364 A CN 111429364A
- Authority
- CN
- China
- Prior art keywords
- rnan
- neural network
- image
- improved
- fourier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013528 artificial neural network Methods 0.000 claims abstract description 92
- 238000011084 recovery Methods 0.000 claims abstract description 9
- 230000007246 mechanism Effects 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 9
- 238000012886 linear function Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000009467 reduction Effects 0.000 claims description 7
- 238000005286 illumination Methods 0.000 abstract description 6
- 238000002073 fluorescence micrograph Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10064—Fluorescence image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image recovery method and device based on Fourier ring correlation and an improved RNAN neural network, wherein the method comprises the following steps: acquiring an original image, and removing thermal noise of the original image by utilizing Fourier ring correlation; taking the original image without the thermal noise as the input of a preset improved RNAN neural network, and inputting the original image into the improved RNAN neural network; and acquiring the output of the improved RNAN neural network to obtain a final recovery image. The method takes an RNAN neural network related to and improved by a Fourier ring as a main body, realizes the recovery of a weak illumination environment and a fluorescence image by combining the advantages of the RNAN neural network and the RNAN neural network, and is simple and easy to realize.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image recovery method and device based on Fourier ring correlation and an improved RNAN neural network.
Background
In the field of image processing, achieving restoration of low-light images and fluorescent images is a current leading problem. The images obtained by today's advanced sensors still have the inevitable problems of noise and less than ideal lighting conditions, and even if the fluorescent markers are well-developed in the fluorescent image, the final image is not clear enough due to the defects of the imaging of the device.
Therefore, how to obtain a clear image under the conditions of weak light, few photons and thermal noise is a problem to be solved.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, an object of the present invention is to provide an image restoration method based on fourier-loop correlation and an improved RNAN neural network, which mainly uses fourier-loop correlation and an improved RNAN neural network, combines the advantages of the two methods to realize restoration of a low-light environment and a fluorescent image, and is simple and easy to implement.
Another object of the present invention is to propose an image restoration apparatus based on fourier-loop correlation and an improved RNAN neural network.
In order to achieve the above object, an embodiment of an aspect of the present invention provides an image restoration method based on fourier-loop correlation and an improved RNAN neural network, including the following steps: acquiring an original image, removing thermal noise of the original image by utilizing Fourier ring correlation, taking the original image without the thermal noise as the input of a preset improved RNAN neural network, and inputting the input into the improved RNAN neural network; and acquiring the output of the improved RNAN neural network to obtain a final recovery image.
According to the image restoration method based on the Fourier ring correlation and the improved RNAN neural network, the Fourier ring correlation is utilized to firstly remove the thermal noise in the image, the image after the thermal noise is removed is taken as the input of the neural network to be processed, and finally a clearer restored image is obtained, so that the Fourier ring correlation and the improved RNAN neural network are taken as main bodies, the restoration of a weak illumination environment and a fluorescent image is realized by combining the advantages of the Fourier ring correlation and the improved RNAN neural network, and the image restoration method is simple and easy to realize.
In addition, the image restoration method based on the fourier-loop correlation and the improved RNAN neural network according to the above embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, the removing the thermal noise of the original image by using fourier loop correlation includes: sampling odd and even different positions of a single image to obtain two pairs of sub-images, performing Fourier transform and calculating Fourier ring correlation; thermal noise in the image is removed by a preset cutoff value.
Further, in an embodiment of the present invention, the taking the original image with thermal noise removed as an input of a preset improved RNAN neural network includes: the image obtained by removing the thermal noise of the original image by utilizing the correlation of the Fourier ring is used as the input of the network, and a linear function is used as a metric function of the relation between two pixels in the attention mechanism.
Further, in an embodiment of the present invention, the method further includes: and adding a total variation regular term.
Further, in an embodiment of the present invention, the preset improved RNAN neural network includes a first convolution layer and a second convolution layer, the first convolution layer is a first layer of the preset improved RNAN neural network and serves as a shallow feature extraction layer, the second convolution layer is a last layer of the preset improved RNAN neural network and serves as a shallow feature reconstruction layer, a first RNAB (residual non local association block) and a second RNAB, the first RNAB and the second RNAB have the same structure and are composed of a net and a convolution layer, an RNAB activation function is Re L u, and first to third RAB (residual association block), the first to third RABs have the same structure and are used for performing dimensionality reduction processing on data and features, wherein RABs and RNABs have the same structure.
In order to achieve the above object, another embodiment of the present invention provides an image restoration apparatus based on fourier-loop correlation and an improved RNAN neural network, including: the preprocessing module is used for acquiring an original image and removing thermal noise of the original image by utilizing the correlation of a Fourier ring; the input module is used for inputting the original image with the thermal noise removed as the input of a preset improved RNAN neural network and inputting the original image into the improved RNAN neural network; and the output module is used for acquiring the output of the improved RNAN neural network to obtain a final recovery image.
According to the image restoration device based on the Fourier ring correlation and the improved RNAN neural network, the thermal noise in the image is firstly removed by utilizing the Fourier ring correlation, the image with the thermal noise removed is taken as the input of the neural network to be processed, and a clearer restored image is finally obtained, so that the restoration of a weak illumination environment and a fluorescent image is realized by taking the Fourier ring correlation and the improved RNAN neural network as main bodies and combining the advantages of the Fourier ring correlation and the improved RNAN neural network, and the image restoration device is simple and easy to realize.
In addition, the image restoration apparatus based on the fourier-loop correlation and the improved RNAN neural network according to the above embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, the preprocessing module is further configured to perform sampling on a single image at odd and even different positions to obtain two pairs of sub-images, perform fourier transform, and calculate fourier loop correlation; thermal noise in the image is removed by a preset cutoff value.
Further, in an embodiment of the present invention, the input module is further configured to use an image obtained by removing thermal noise of the original image by using fourier ring correlation as an input of the network, and use a linear function as a metric function of a relation between two pixels in the attention mechanism.
Further, in an embodiment of the present invention, the method further includes: and the introducing module is used for adding a total variation regular term.
Further, in an embodiment of the present invention, the preset improved RNAN neural network includes a first convolution layer and a second convolution layer, the first convolution layer is a first layer of the preset improved RNAN neural network and is used as a shallow feature extraction layer, the second convolution layer is a last layer of the preset improved RNAN neural network and is used as a shallow feature reconstruction layer, the first RNAB and the second RNAB have the same structure and are composed of a ResNet and a convolution layer, an RNAB activation function is a Re L u function, and the first RAB to the third RAB have the same structure and are used for performing dimensionality reduction processing on data and features, wherein the RAB and the RNAB have the same structure.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a flowchart of an image restoration method based on fourier-loop correlation and an improved RNAN neural network according to an embodiment of the present invention;
figure 2 is a block diagram of an improved RNAN network according to an embodiment of the present invention;
FIG. 3 is a flow diagram of an image restoration method based on Fourier Ring correlation and an improved RNAN neural network according to one embodiment of the present invention;
FIG. 4 is a flow chart of an image restoration method based on Fourier Ring correlation and an improved RNAN neural network according to a specific embodiment of the present invention;
fig. 5 is a schematic structural diagram of an image restoration apparatus based on fourier-loop correlation and an improved RNAN neural network according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
An image restoration method and apparatus based on fourier-loop correlation and an improved RNAN neural network according to an embodiment of the present invention will be described below with reference to the accompanying drawings, and first, an image restoration method based on fourier-loop correlation and an improved RNAN neural network according to an embodiment of the present invention will be described with reference to the accompanying drawings.
Fig. 1 is a flowchart of an image restoration method based on fourier-loop correlation and an improved RNAN neural network according to an embodiment of the present invention.
As shown in fig. 1, the image restoration method based on fourier-loop correlation and improved RNAN neural network includes the following steps:
in step S101, an original image is acquired, and thermal noise of the original image is removed by fourier-loop correlation.
It can be understood that, in the embodiment of the present invention, the first correlation of the fourier ring is calculated to obtain the ring correlation histogram of the image, the cutoff value is set to remove the thermal noise point with weak correlation, and the processed image is obtained
Further, in one embodiment of the present invention, removing thermal noise of the original image using fourier-loop correlation comprises: sampling odd and even different positions of a single image to obtain two pairs of sub-images, performing Fourier transform and calculating Fourier ring correlation; thermal noise in the image is removed by a preset cutoff value.
It can be understood that, sampling of different positions of odd and even is carried out on a single image, fourier transform is carried out on 4 sub-images, namely two pairs of sub-images, fourier ring correlation is calculated, and thermal noise in the image is removed through a set cut-off value.
Specifically, the fourier loop correlation in the embodiment of the present invention is mainly used to achieve the effect of eliminating the thermal noise of the original image. Firstly, sampling odd points, even points, odd even points and even odd points of a single image respectively to obtain four subgraphs which are divided into two groups; then Fourier transform is carried out on the correlation data, and then correlation histograms are obtained by utilizing the definition calculation of Fourier ring correlation. In order to simplify the operation, after an analysis test, for example, a cutoff value is defined as 0.1, that is, a point with correlation smaller than 0.1 is considered as a thermal noise point, and an image with thermal noise removed is obtained after the thermal noise is filtered.
In step S102, the original image from which the thermal noise is removed is input to a modified RNAN neural network as a preset input.
It will be appreciated that embodiments of the invention take the image with the thermal noise removed as an input to the improved RNAN neural network.
In one embodiment of the invention, the preset improved RNAN neural network comprises a first convolution layer and a second convolution layer, wherein the first convolution layer is a first layer of the preset improved RNAN neural network and is used as a shallow feature extraction layer, the second convolution layer is a last layer of the preset improved RNAN neural network and is used as a shallow feature reconstruction layer, the first RNAB and the second RNAB are identical in structure and are composed of ResNet and convolution layers, an activation function of the RNAB is a Re L u function, and the first RAB, the second RNAB, the third RAB and the first RAB are identical in structure and are used for carrying out dimension reduction processing on data and features, and the RAB and the RNAB are identical in structure.
It can be understood that the structure of the modified RNAN neural network is shown in fig. 2, and includes two convolution layers, two RNABs, three RAB blocks, and a network structure with residual addition, as follows:
the first layer and the last layer are convolution layers and are used as a shallow feature extraction layer and a reconstruction layer, the RNAB is formed by improving a ResNet layer and a convolution layer, an activation function is a Re L u function, partial non-local information is input, the structure is simple, the calculation effect is good, the RAB input does not contain the non-local information, the structure is the same as the RNAB, dimension reduction processing can be carried out on data and features, and a network structure with residual error addition is adopted integrally, so that features of different depth levels can be extracted from the input, and the problem of gradient disappearance can be avoided.
Further, in an embodiment of the present invention, the taking the original image with thermal noise removed as an input of the preset improved RNAN neural network includes: the image obtained by removing the thermal noise of the original image by utilizing the correlation of the Fourier ring is used as the input of the network, and a linear function is used as a metric function of the relation between two pixels in the attention mechanism.
It can be understood that the neural network according to the embodiment of the present invention is improved on the basis of maintaining the original RNAN structure, and an image obtained by removing thermal noise from the original image by using fourier-loop correlation is used as an input of the network, a linear function is used as a metric function of a two-pixel relationship in an attention mechanism, and a total variation regularization term is added to prevent overfitting, so as to finally obtain a restored image.
Specifically, the improved RNAN network in the embodiment of the present invention is mainly used for recovering the image obtained in the previous step. The original RNAN network mainly comprises three parts, and the embodiment of the invention reserves the structure of the original RNAN neural network, namely two convolution layers, two RNABs, three RAB blocks and a network structure with residual addition. The embodiment of the invention converts the expression of the relation between two pixels in the RNAB partial attention mechanism from an exponential form to a linear form, and adds a total variation regular term aiming at the overfitting phenomenon to correct so as to obtain a clearer image.
In step S103, the output of the improved RNAN neural network is acquired, resulting in a final restored image.
It can be understood that, in the embodiment of the present invention, the image is processed by the improved RNAN network to finally obtain the restored image of the original image after being denoised.
The image restoration method based on the fourier-loop correlation and improved RNAN neural network will be further described with reference to fig. 3 and 4, specifically as follows:
first, a given original image is preprocessed. If the given original image is large, it may be blocked. For a given image, the image is converted into four sub-images and divided into two pairs, wherein the image obtained by sampling at even points on the horizontal and vertical coordinates and the image obtained by sampling at odd points on the horizontal and vertical coordinates are used as one pair, and the image with the horizontal and vertical coordinates of odd number and even number respectively is used as the other pair. And calculating Fourier transform of the two pairs of images, then calculating Fourier ring correlation of the two pairs of images to obtain a correlation histogram, and removing a part of the correlation histogram, which is smaller than a cut-off value, to obtain an image with thermal noise removed.
The improved RNAN neural network keeps the original structure, as shown in figure 2, wherein an attention mechanism is utilized in an RNAB block, and an exponential function describing the relationship between two pixel points by the attention mechanism in the original network structure is modified into a linear function in the embodiment of the invention.
According to the image restoration method based on the Fourier ring correlation and the improved RNAN neural network, the Fourier ring correlation is utilized to firstly remove the thermal noise in the image, the image after the thermal noise is removed is taken as the input of the neural network to be processed, and finally a clearer restored image is obtained, so that the Fourier ring correlation and the improved RNAN neural network are taken as main bodies, the restoration of a weak illumination environment and a fluorescent image is realized by combining the advantages of the Fourier ring correlation and the improved RNAN neural network, and the image restoration method is simple and easy to realize.
Next, an image restoration apparatus based on fourier-loop correlation and an improved RNAN neural network proposed according to an embodiment of the present invention is described with reference to the drawings.
Fig. 5 is a schematic structural diagram of an image restoration apparatus based on fourier-loop correlation and an improved RNAN neural network according to an embodiment of the present invention.
As shown in fig. 5, the image restoration apparatus 10 based on fourier-loop correlation and the improved RNAN neural network includes: a pre-processing module 100, an input module 200 and an output module 300.
The preprocessing module 100 is configured to obtain an original image, and remove thermal noise of the original image by using fourier ring correlation; the input module 200 is configured to use the original image with the thermal noise removed as an input of a preset improved RNAN neural network, and input the input to the improved RNAN neural network; the output module 300 is used for obtaining the output of the improved RNAN neural network to obtain the final restored image. The device 10 of the embodiment of the invention takes the RNAN neural network related to and improved by the Fourier ring as a main body, realizes the recovery of the weak illumination environment and the fluorescence image by combining the advantages of the RNAN neural network and the Fourier ring, and is simple and easy to realize
Further, in an embodiment of the present invention, the preprocessing module 100 is further configured to perform odd-even sampling at different positions on a single image to obtain two pairs of sub-images, perform fourier transform, and calculate fourier-loop correlation; thermal noise in the image is removed by a preset cutoff value.
Further, in an embodiment of the present invention, the input module 200 is further configured to use an image obtained by removing thermal noise of the original image by using fourier ring correlation as an input of the network, and use a linear function as a metric function of a relation between two pixels in the attention mechanism.
Further, in an embodiment of the present invention, the method further includes: and the introducing module is used for adding a total variation regular term.
Further, in an embodiment of the invention, the preset improved RNAN neural network comprises a first convolution layer and a second convolution layer, wherein the first convolution layer is a first layer of the preset improved RNAN neural network and is used as a shallow feature extraction layer, the second convolution layer is a last layer of the preset improved RNAN neural network and is used as a shallow feature reconstruction layer, the first RNAB and the second RNAB are identical in structure and are composed of ResNet and convolution layers, an activation function of the RNAB is a Re L u function, and the first RAB, the second RAB, the first RAB, the second RAB and the third RAB are identical in structure and are used for carrying out dimension reduction processing on data and features, and the RAB and the RNAB are identical in structure.
It should be noted that the foregoing explanation of the embodiment of the image restoration method based on fourier-loop correlation and improved RNAN neural network is also applicable to the image restoration apparatus based on fourier-loop correlation and improved RNAN neural network of this embodiment, and will not be described herein again.
According to the image restoration device based on the Fourier ring correlation and the improved RNAN neural network, the Fourier ring correlation is utilized to firstly remove the thermal noise in the image, the image after the thermal noise is removed is taken as the input of the neural network to be processed, and finally a clearer restored image is obtained, so that the Fourier ring correlation and the improved RNAN neural network are taken as main bodies, the restoration of a weak illumination environment and a fluorescent image is realized by combining the advantages of the Fourier ring correlation and the improved RNAN neural network, and the image restoration device is simple and easy to realize.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. An image restoration method based on Fourier ring correlation and improved RNAN neural network, characterized by comprising the following steps:
acquiring an original image, and removing thermal noise of the original image by utilizing Fourier ring correlation;
taking the original image without the thermal noise as the input of a preset improved RNAN neural network, and inputting the original image into the improved RNAN neural network; and
and acquiring the output of the improved RNAN neural network to obtain a final recovery image.
2. The method of claim 1, wherein removing thermal noise from the original image using fourier-loop correlation comprises:
sampling odd and even different positions of a single image to obtain two pairs of sub-images, performing Fourier transform and calculating Fourier ring correlation;
thermal noise in the image is removed by a preset cutoff value.
3. The method of claim 2, wherein the inputting the original image with thermal noise removed as the preset modified RNAN neural network comprises:
the image obtained by removing the thermal noise of the original image by utilizing the correlation of the Fourier ring is used as the input of the network, and a linear function is used as a metric function of the relation between two pixels in the attention mechanism.
4. The method of claim 3, further comprising:
and adding a total variation regular term.
5. The method according to any of claims 1-4, wherein the pre-defined modified RNAN neural network comprises:
a first convolutional layer and a second convolutional layer, wherein the first convolutional layer is a first layer of the preset improved RNAN neural network and is used as a shallow feature extraction layer; the second convolutional layer is the last layer of the preset improved RNAN neural network and is used as a shallow feature reconstruction layer;
a first RNAB and a second RNAB, wherein the first RNAB and the second RNAB have the same structure and are composed of ResNet and a convolution layer, and the activation function of the RNAB is a Re L u function;
the structure of the first RAB to the third RAB is the same, and the first RAB to the third RAB are used for performing dimensionality reduction processing on data and characteristics, wherein the RAB and the RNAB are the same in structure.
6. An image restoration apparatus based on fourier-loop correlation and an improved RNAN neural network, comprising:
the preprocessing module is used for acquiring an original image and removing thermal noise of the original image by utilizing the correlation of a Fourier ring;
the input module is used for inputting the original image with the thermal noise removed as the input of a preset improved RNAN neural network and inputting the original image into the improved RNAN neural network; and
and the output module is used for acquiring the output of the improved RNAN neural network to obtain a final recovery image.
7. The apparatus of claim 6, wherein the pre-processing module is further configured to sample a single image at odd and even different positions to obtain two pairs of sub-images, perform Fourier transform, and calculate Fourier-loop correlation; thermal noise in the image is removed by a preset cutoff value.
8. The apparatus of claim 7, wherein the input module is further configured to use an image obtained by removing thermal noise from the original image by using fourier-loop correlation as an input of the network, and use a linear function as a metric function of a two-pixel relationship in the attention mechanism.
9. The method of claim 8, further comprising:
and the introducing module is used for adding a total variation regular term.
10. The method according to any of claims 6-9, wherein the pre-defined modified RNAN neural network comprises:
a first convolutional layer and a second convolutional layer, wherein the first convolutional layer is a first layer of the preset improved RNAN neural network and is used as a shallow feature extraction layer; the second convolutional layer is the last layer of the preset improved RNAN neural network and is used as a shallow feature reconstruction layer;
a first RNAB and a second RNAB, wherein the first RNAB and the second RNAB have the same structure and are composed of ResNet and a convolution layer, and the activation function of the RNAB is a Re L u function;
the structure of the first RAB to the third RAB is the same, and the first RAB to the third RAB are used for performing dimensionality reduction processing on data and characteristics, wherein the RAB and the RNAB are the same in structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010124621.6A CN111429364A (en) | 2020-02-27 | 2020-02-27 | Image restoration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010124621.6A CN111429364A (en) | 2020-02-27 | 2020-02-27 | Image restoration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111429364A true CN111429364A (en) | 2020-07-17 |
Family
ID=71551723
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010124621.6A Pending CN111429364A (en) | 2020-02-27 | 2020-02-27 | Image restoration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111429364A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112819910A (en) * | 2021-01-08 | 2021-05-18 | 上海理工大学 | Hyperspectral image reconstruction method based on double-ghost attention machine mechanism network |
-
2020
- 2020-02-27 CN CN202010124621.6A patent/CN111429364A/en active Pending
Non-Patent Citations (2)
| Title |
|---|
| SAMI KOHO等: ""Fourier ring correlation simplifies image restoration in fluorescence microscopy"", 《NATURE COMMUNICATIONS》 * |
| YULUN ZHANG等: ""Residual Non-Local Attention Networks for Image Restoration"", 《ICLR 2019》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112819910A (en) * | 2021-01-08 | 2021-05-18 | 上海理工大学 | Hyperspectral image reconstruction method based on double-ghost attention machine mechanism network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yan et al. | Attention-guided network for ghost-free high dynamic range imaging | |
| CN107507173B (en) | No-reference definition evaluation method and system for full-slice image | |
| EP2209090B1 (en) | Image restoring apparatus and method thereof | |
| CN112419184B (en) | Spatial attention map image denoising method integrating local information and global information | |
| JP5941674B2 (en) | Cell contour forming apparatus and method, and cell contour forming program | |
| CN112733950A (en) | Power equipment fault diagnosis method based on combination of image fusion and target detection | |
| CN101976436B (en) | Pixel-level multi-focus image fusion method based on correction of differential image | |
| CN104075965B (en) | A kind of micro-image grain graininess measuring method based on watershed segmentation | |
| KR101426298B1 (en) | apparatus and method for compensating image for enhancing fog removing efficiency | |
| CN113962976B (en) | Quality evaluation method for pathological slide digital image | |
| CN108664839B (en) | Image processing method and device | |
| CN111178499B (en) | Medical image super-resolution method based on generation countermeasure network improvement | |
| CN110866870B (en) | Super-resolution processing method for amplifying medical image by any multiple | |
| CN112330613B (en) | Evaluation method and system for cytopathology digital image quality | |
| CN116757988B (en) | Infrared and visible light image fusion method based on semantic enrichment and segmentation tasks | |
| CN113327206A (en) | Image fuzzy processing method of intelligent power transmission line inspection system based on artificial intelligence | |
| CN111429364A (en) | Image restoration method and device | |
| Chang et al. | Beyond camera motion blur removing: How to handle outliers in deblurring | |
| CN112801899B (en) | Internal and external circulation driving image blind deblurring method and device based on complementary structure perception | |
| CN117392496A (en) | Target detection method and system based on infrared and visible light image fusion | |
| US10194880B2 (en) | Body motion display device and body motion display method | |
| CN109615584B (en) | SAR image sequence MAP super-resolution reconstruction method based on homography constraint | |
| CN113191949B (en) | Multi-scale super-resolution pathology image digitizing method, system and storage medium | |
| CN102708549A (en) | Method for enhancing vehicle-mounted night vision image | |
| CN115760875A (en) | Full-field medical picture region segmentation method based on self-supervision learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200717 |