CN111583107A - Image super-resolution reconstruction method and system based on attention mechanism - Google Patents
Image super-resolution reconstruction method and system based on attention mechanism Download PDFInfo
- Publication number
- CN111583107A CN111583107A CN202010257518.9A CN202010257518A CN111583107A CN 111583107 A CN111583107 A CN 111583107A CN 202010257518 A CN202010257518 A CN 202010257518A CN 111583107 A CN111583107 A CN 111583107A
- Authority
- CN
- China
- Prior art keywords
- information
- module
- image
- super
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000000605 extraction Methods 0.000 claims abstract description 48
- 230000006835 compression Effects 0.000 claims description 13
- 238000007906 compression Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 8
- 238000012935 Averaging Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 abstract description 12
- 230000000007 visual effect Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 12
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 101100365548 Caenorhabditis elegans set-14 gene Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image super-resolution reconstruction method and system based on an attention mechanism, which comprises the following steps: extracting preliminary image characteristic information in the low-resolution image; gradually extracting the preliminary image feature information through a plurality of connected information extraction modules to obtain depth image feature information; and reconstructing a super-resolution image according to the depth image characteristic information. The image information is extracted through the information extraction module based on the attention mechanism, the correlation among channels is improved, the characteristic diagram information of each channel is fully fused, the definition and the visual effect of the reconstructed super-resolution image are improved, more details such as contour textures are effectively recovered, and finally the reconstruction module reconstructs high-resolution images with different scales according to the information.
Description
Technical Field
The invention relates to the technical field of image super-resolution reconstruction, in particular to an attention mechanism-based image super-resolution reconstruction method and system.
Background
Super-Resolution (SR) reconstruction was proposed by Harris et al in the first 60 s of the 20 th century, and can be divided into single reconstruction and multiple reconstructions according to the number of low-Resolution images required for reconstruction, where single-image Super-Resolution reconstruction is to reconstruct a low-Resolution image into a high-Resolution image visually pleasing through a certain algorithm.
In recent years, a super-resolution reconstruction method based on a Convolutional Neural Network (CNN) achieves good results, for example, in 2014, Dong et al first introduces a three-layer convolutional neural network into an image, and proposes a super-resolution reconstruction model (SRCNN) based on the convolutional neural network; in 2016, Kim et al proposed a very deep convolutional network-accurate super-resolution reconstruction model (VDSR) that increased the depth of the network to 20 layers; in order to control the number of model parameters, Kim et al also propose a deep recursive convolutional network model (DRCN) by recursive supervision and skip connection, which makes a significant improvement compared with the SRCNN; tai et al propose a Deep Recursive Residual Network (DRRN) that employs a weight sharing strategy to alleviate the huge parameter requirements in very deep networks; in 2018, an Information Distillation Network (IDN) was proposed as a deep but compact convolutional network, enabling the reconstruction of high resolution images directly from the original low resolution images. In order to improve the super-resolution effect of an image, deepening and expanding a network become a design trend, but the problems of large calculation amount, more memory consumption, long training time and the like are caused only by deepening the number of network layers, so that the method is not suitable for mobile and embedded visual application scenes. In addition, the above convolutional network-based approach, which handles channel characteristics equally, lacks flexibility in handling different types of information, such as high frequency information and low frequency information. Therefore, the problems of blurry reconstructed output images, poor visual effect, long operation time and the like commonly exist in the existing image super-resolution reconstruction method.
Disclosure of Invention
In order to solve the above problems, an object of the present invention is to provide a method and a system for reconstructing super-resolution image based on attention mechanism, so as to improve the definition and visual effect of the reconstructed output image.
The technical scheme adopted by the invention for solving the problems is as follows:
in the first aspect of the invention, an attention mechanism-based image super-resolution reconstruction method extracts preliminary image characteristic information in a low-resolution image; extracting the preliminary image feature information in a multi-dimensional way through a plurality of connected information extraction modules to obtain depth image feature information; reconstructing a super-resolution image according to the depth image characteristic information; the information extraction module comprises a plurality of groups of convolution modules and attention modules which are arranged at intervals, and the primary image feature information passes through the convolution modules to obtain first feature information containing a plurality of feature channels; and the first characteristic information executes an attention mechanism through the attention module, learns the characteristic weight of each characteristic channel, and adjusts the first characteristic information by using the characteristic weight to obtain the input of the next convolution module.
The image super-resolution reconstruction method based on the attention mechanism at least has the following beneficial effects: by arranging the information extraction module based on the attention mechanism, the correlation among channels is improved, the characteristic diagram information of each channel is fully fused, and the definition and the visual effect of the reconstructed super-resolution image are improved.
Further, the attention mechanism includes: averaging input information by using a global average pooling method to obtain global information; using a gating unit consisting of two fully-connected layers to learn the feature weight of each feature channel from the global information; and adjusting the input information by using the characteristic weight to obtain output information. The global average pooling averages the information of all points in the space into a value, can shield the distribution information in the space, and can better focus on the correlation among channels; a gating mechanism formed by two layers of full connection fuses characteristic diagram information of each channel, obtains a characteristic diagram weighted value by utilizing the correlation between the channels, and realizes self-adaptive adjustment of channel characteristics.
Further, the information extraction module comprises a local shallow layer network for extracting second feature information and a local deep layer network for extracting third feature information; in the information extraction module, the second feature information and the third feature information are processed according to the following operations:
Rk＝C(S(P1 k,1/s),Fk-1)
wherein, Fk-1Is the input of the current information extraction module, C, S represents the join operation and the slice operation, respectively, P1 kFor the output of the local shallow network in the kth information extraction module, RkThe second characteristic information;
Pk＝P2 k+Rk＝Cb(S(P1 k,1-1/s))+C(S(P1 k,1/s),Fk-1)
wherein, P2 k、CbRespectively for the output of the local deep network and the stacking convolution operation of the local deep network in the kth information extraction module, PkIs the third characteristic information.
And after each feature extraction, the association and fusion of the feature map information of each channel are carried out, so that the value of the weight value of the feature map is ensured. The third characteristic information is obtained by combining the first characteristic information, the second characteristic information and the local deep network characteristic, so that the third characteristic information is richer and more effective in content.
Further, the local shallow layer network comprises a first convolution module, a first attention module, a second convolution module, a second attention module, a third convolution module and a third attention module which are connected in sequence, wherein the characteristic dimension of the first convolution module is 48, the characteristic dimension of the second convolution module is 32, and the characteristic dimension of the third convolution module is 64.
Further, the local deep network includes a fourth convolution module, a fourth attention module, a fifth convolution module, a fifth attention module, a sixth convolution module, and a sixth attention module, which are connected in sequence, where a characteristic dimension of the fourth convolution module is 64, a characteristic dimension of the fifth convolution module is 48, and a characteristic dimension of the sixth convolution module is 80.
Further, the information extraction module is further provided with an information compression module, the information compression module is provided with a compression convolution module for reducing dimensions, and the characteristic dimension of the compression convolution module is 64. The information compression module can compress redundant information of the features in the third feature information, and increase feature map information of the third feature information, so that the outline details are clearer.
Further, the reconstructed super-resolution image adopts a sub-pixel convolution layer reconstruction method, and the depth image characteristic information is processed according to the following operations:
ISR＝HREC(Hn(Fn-1))+U(ILR)
wherein HRECU denotes the reconstruction operation and the bicubic interpolation operation, I, respectivelySRRepresents the final output result, ILRRepresenting the original input, HnRepresenting the nth information extraction function.
Therefore, the low-resolution images are reconstructed into high-resolution images with different scales by utilizing the information learned by the information extraction module.
Further, the reconstructing the super-resolution image further includes a loss function operation, where the loss function adopts a mean absolute error mode, and the final output result is processed according to the following operations:
in a second aspect of the present invention, an attention-based image super-resolution reconstruction system includes: the characteristic extraction module is used for extracting the preliminary image characteristic information in the low-resolution image; the information extraction modules are used for extracting the preliminary image feature information in a multi-dimensional mode to obtain depth image feature information; the reconstruction module is used for reconstructing a super-resolution image according to the depth image characteristic information; the information extraction module comprises a plurality of groups of convolution modules arranged at intervals and an attention module, and the attention module executes an attention mechanism to adjust output information passing through the convolution modules.
The image super-resolution reconstruction system based on the attention mechanism at least has the following beneficial effects: by arranging the information extraction module based on the attention mechanism, the correlation among channels is improved, the characteristic diagram information of each channel is fully fused, and the definition and the visual effect of the reconstructed super-resolution image are improved.
In a third aspect of the present invention, a storage medium is characterized in that the storage medium stores computer-executable instructions for causing a computer to execute the attention-based image super-resolution reconstruction method as described above.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The invention is further illustrated with reference to the following figures and examples.
FIG. 1 is a flowchart of an image super-resolution reconstruction method based on an attention mechanism according to an embodiment of the present invention;
FIG. 2 is a flow chart of the attention machine mechanism of FIG. 1;
FIG. 3 is a block diagram of the information extraction module of FIG. 1;
FIG. 4 is a block diagram of an image super-resolution reconstruction system based on an attention mechanism according to an embodiment of the present invention;
FIG. 5 is a block diagram of the local deep network and information compression module of FIG. 4;
FIG. 6 is a block diagram of the attention module of FIG. 4;
FIG. 7 is a comparison graph of the image reconstruction results of the Set5 data Set according to the present invention and the comparison method;
FIG. 8 is a comparison of the image reconstruction results on the Urban100 data set using the present invention and comparison method.
Detailed Description
Referring to fig. 1 and 3, an embodiment of the present invention provides an image super-resolution reconstruction method based on an attention mechanism, including step S110 of extracting preliminary image feature information in a low-resolution image; step S120, gradually extracting preliminary image characteristic information through a plurality of connected information extraction modules 200 to obtain depth image characteristic information; and S130, reconstructing a super-resolution image according to the depth image characteristic information. The information extraction module 200 comprises a plurality of groups of convolution modules 210 and attention modules 220 which are arranged at intervals, and the preliminary image feature information is subjected to the convolution modules 210 to obtain first feature information containing a plurality of feature channels; the first feature information is processed by the attention module 220 to execute an attention mechanism, learn a feature weight of each feature channel, and adjust the first feature information by using the feature weight to obtain an input of a next convolution module.
The image information is extracted by the plurality of connected information extraction modules 200, and an attention mechanism is executed, so that the correlation among channels is improved, the characteristic diagram information of each channel is fully fused, and the definition and the visual effect of the reconstructed super-resolution image are improved.
In this embodiment, features are extracted from the original image, and the original image data is extracted by convolution of two 3 × 3, and the feature dimension is 64, as shown in the following formula:
F0＝HFE(ILR)
wherein ILRRepresenting the original input, HFEA function representing the extracted features is then selected,F0representing the extracted features and the input for the next step.
In addition, in the embodiment, the number of the information extraction modules 200 is 4, and the process of extracting the image information is shown in the following formula:
Fk＝Hk(Fk-1),k＝1,2,…,n
wherein HkDenotes the kth information extraction function, Fk-1And FkRespectively representing the input and output of the kth information extraction block.
Referring to fig. 2 and 6, another embodiment, performing an attention mechanism includes: step S121, averaging input information by using a global average pooling method to obtain global information; step S122, learning the feature weight of each feature channel from global information by using a gate control unit formed by two fully-connected layers 222; and S123, adjusting the input information by using the characteristic weight to obtain output information.
In the present embodiment, an attention mechanism is executed, and the specific flow is as follows:
and step S121, averaging the input information by using a global average pooling method to obtain global information. By using Global Average Pooling (GAP) 221 to average the information of all points in space to a value, the distribution information in space can be masked, and the correlation between channels can be better concerned. In this embodiment, X ═ X1,…,xc,…,xC]As input, the size of the C feature maps is H × W, the result after global average pooling presents global information z among the C feature maps, and the global information z of the C-th feature mapcAs shown in the following equation:
wherein xc(i, j) is the C-th feature map xcValue of position (i, j), HGAPRepresenting a global average pooling function, such global information being helpful in expressing the entire image;
step S122, learning the feature weight of each feature channel from the global information by using the gate unit composed of two fully-connected layers 222. The two layers of full connection form a gating mechanism, and the calculation method of the gating unit s is shown as the following formula:
s＝g(W2(W1z))
where the sum g represents the Sigmoid function and the ReLU function, respectively. W1、W2Are all operated in full connection with layer 222, where W1The dimension of (1) is C/r, r is a scaling factor, in the embodiment, the value of r is 16, and the number of channels is reduced by setting the parameter, so that the calculated amount is reduced. Then passes through a ReLU layer 223 with unchanged output dimension; w2Is C, and the output dimension is 1 x C; and finally obtaining s through a Sigmoid function. s is a weight that can characterize C feature maps, and this weight is learned through the fully-connected layers 222 and the non-linear layers, so that end-to-end training can be performed, and the two fully-connected layers 222 serve to fuse feature map information of each channel.
And S123, adjusting the input information by using the characteristic weight to obtain output information. Finally, the obtained characteristic diagram weight value s is used for readjusting the input xcThe following formula is shown.
Wherein s iscAnd xcIs the scaling factor and feature map in the c-th channel. Therefore, the channel characteristics can be adjusted in a self-adaptive mode through the characteristic diagram attention mechanism, and the representation capability of the network is enhanced.
Referring to fig. 3 and 5, another embodiment, the information extraction module 200 includes a local shallow network 400 for extracting second feature information and a local deep network 500 for extracting third feature information; in the information extraction module 200, the second feature information and the third feature information are processed as follows:
Rk＝C(S(P1 k,1/s),Fk-1)
wherein, Fk-1Is a current information extraction module 200, C is a join operation 510, S is a slice operation 520, P1 kFor the output of the local shallow network in the kth information extraction module 200, RkIs the second characteristic information.
Pk＝P2 k+Rk＝Cb(S(P1 k,1-1/s))+C(S(P1 k,1/s),Fk-1)
Wherein, P2 k、CbThe output of the local deep network and the stacking convolution operation of the local deep network, P, in the kth information extraction module 200, respectivelykIs the third characteristic information.
In another embodiment, the local shallow network 400 includes a first convolution module, a first attention module, a second convolution module, a second attention module, a third convolution module, and a third attention module connected in sequence, wherein the characteristic dimension of the first convolution module is 48, the characteristic dimension of the second convolution module is 32, and the characteristic dimension of the third convolution module is 64.
In another embodiment, the local deep network 500 includes a fourth convolution module, a fourth attention module, a fifth convolution module, a fifth attention module, a sixth convolution module, and a sixth attention module, which are connected in sequence, wherein the feature dimension of the fourth convolution module is 64, the feature dimension of the fifth convolution module is 48, and the feature dimension of the sixth convolution module is 80.
In another embodiment, the information extraction module 200 is further provided with an information compression module 230, and the information compression module 230 is provided with a compression convolution module for dimension reduction, and the characteristic dimension of the compression convolution module is 64. The information compression module 230 can compress redundant information of the features in the third feature information, and increase feature map information of the third feature information, so that the detail of the outline is clearer.
In another embodiment, the reconstruction of the super-resolution image adopts a sub-pixel convolution layer ESPCN reconstruction method, and the depth image characteristic information is processed according to the following operations:
ISR＝HREC(Hn(Fn-1))+U(ILR)
wherein HRECU denotes the reconstruction operation and the bicubic interpolation operation, I, respectivelySRRepresents the final output result, ILRRepresenting the original input, HnRepresenting the nth information extraction function.
In another embodiment, reconstructing the super-resolution image further includes a loss function operation, where the loss function is implemented by using an average absolute error, and the final output result is processed according to the following operations:
referring to fig. 4, another embodiment, an attention-based image super-resolution reconstruction system includes: a feature extraction module 100, configured to extract preliminary image feature information in the low-resolution image; the information extraction modules 200 are used for extracting the preliminary image feature information in a multi-dimensional manner to obtain depth image feature information; and a reconstruction module 300, configured to reconstruct the super-resolution image according to the depth image feature information.
The present embodiment extracts features from an original low-resolution image through a feature extraction block, and then adaptively adjusts feature channel information by an information extraction module 200 including a plurality of convolution modules 210 and an attention module 220, thereby enhancing the expressive power of the features. Through comparison of experimental data, the method can effectively recover more details such as contour texture and the like, and finally the reconstruction module reconstructs high-resolution images with different scales according to the information.
To better illustrate the effectiveness of the present invention, test data sets were experimented with and data compared to a plurality of advanced super-resolution models. The contents of the comparative experiment are as follows:
the experiment used four widely used benchmark test data sets of Set5, Set14, Urban100 and Manga109, and data were enhanced in three ways: rotate the picture 90 °, 180 ° and 270 °; horizontally turning over the picture; the image is reduced by factors of 0.9,0.8,0.7 and 0.6. And rebuilding data in the reference test data set in different scales after double and triple down sampling, and qualitatively and quantitatively comparing a rebuilding result with other most advanced super-resolution models, wherein the rebuilding result comprises SRCNN, VDSR, DRCN, DRRN and IDN. The first table shows the quantitative comparison results of different image super-resolution models under three different scales of x2, x3 and x4 by using the peak signal-to-noise ratio (PSNR) and the Structural Similarity (SSIM), wherein the higher the values of the two parameters are, the better the reconstruction effect is.
As can be seen from the data in Table I, the PSNR and SSIM of the method provided by the invention on the Set5, B100, Urban100 and Manga109 data sets are mostly superior to other super-resolution models, which fully shows the effectiveness of the attention-based mechanism in image super-resolution reconstruction.
The second table shows the running time quantitative comparison results of the x2, x3 and x4 image super-resolution models with different scales, wherein the time unit is second, and the compared super-resolution models are also SRCNN, VDSR, DRCN, DRRN and IDN.
Watch two
As can be seen from the table two data, the runtime of the method proposed by the present invention on the Set5, B100, Urban100 and Manga109 data sets is mostly lower than that of other super-resolution models, which fully indicates the high efficiency in image super-resolution reconstruction based on the attention mechanism.
Fig. 7 and 8 show examples of reconstruction results on Set5 and Urban100 data sets at the x4 scale, comparing super resolution models again SRCNN, VDSR, DRCN, DRRN, IDN. Compared with a contrast method, the method can generate clearer edges and richer details, and has better visual effect.
The above is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments, and the present invention shall fall within the protection scope of the present invention as long as the technical effects of the present invention are achieved by the same means.
Claims (10)
1. An image super-resolution reconstruction method based on an attention mechanism is characterized by comprising the following steps:
extracting preliminary image characteristic information in the low-resolution image;
extracting the preliminary image feature information in a multi-dimensional way through a plurality of connected information extraction modules to obtain depth image feature information;
reconstructing a super-resolution image according to the depth image characteristic information;
the information extraction module comprises a plurality of groups of convolution modules and attention modules which are arranged at intervals, and the preliminary image feature information passes through the convolution modules to obtain first feature information containing a plurality of feature channels; and the first characteristic information executes an attention mechanism through the attention module, learns the characteristic weight of each characteristic channel, and adjusts the first characteristic information by using the characteristic weight to obtain the input of the next convolution module.
2. The method for image super-resolution reconstruction based on attention mechanism according to claim 1, wherein the attention mechanism comprises:
averaging input information by using a global average pooling method to obtain global information;
using a gating unit consisting of two fully-connected layers to learn the feature weight of each feature channel from the global information;
and adjusting the input information by using the characteristic weight to obtain output information.
3. The method for image super-resolution reconstruction based on attention mechanism as claimed in claim 2, wherein the information extraction module comprises a local shallow layer network for extracting second feature information and a local deep layer network for extracting third feature information; in the information extraction module, the second feature information and the third feature information are processed according to the following operations:
Rk＝C(S(P1 k,1/s),Fk-1)
wherein, Fk-1Is the input of the current information extraction module, C, S represents the join operation and the slice operation, respectively, P1 kFor the output of the local shallow network in the kth information extraction module, RkThe second characteristic information;
Pk＝P2 k+Rk＝Cb(S(P1 k,1-1/s))+C(S(P1 k,1/s),Fk-1)
wherein, P2 k、CbRespectively for the output of the local deep network and the stacking convolution operation of the local deep network in the kth information extraction module, PkIs the third characteristic information.
4. The method for image super-resolution reconstruction based on attention mechanism of claim 3, wherein the local shallow layer network comprises a first convolution module, a first attention module, a second convolution module, a second attention module, a third convolution module and a third attention module which are connected in sequence, wherein the characteristic dimension of the first convolution module is 48, the characteristic dimension of the second convolution module is 32, and the characteristic dimension of the third convolution module is 64.
5. The method for image super-resolution reconstruction based on attention mechanism according to claim 4, wherein the local deep network comprises a fourth convolution module, a fourth attention module, a fifth convolution module, a fifth attention module, a sixth convolution module and a sixth attention module which are connected in sequence, wherein the characteristic dimension of the fourth convolution module is 64, the characteristic dimension of the fifth convolution module is 48, and the characteristic dimension of the sixth convolution module is 80.
6. The method for reconstructing the super-resolution image based on the attention mechanism is characterized in that the information extraction module is further provided with an information compression module, the information compression module is provided with a compressed convolution module for dimension reduction, and the characteristic dimension of the compressed convolution module is 64.
7. The method for reconstructing the super-resolution image based on the attention mechanism according to claim 6, wherein the reconstructed super-resolution image is reconstructed by a sub-pixel convolution layer reconstruction method, and the depth image feature information is processed according to the following operations:
ISR＝HREC(Hn(Fn-1))+U(ILR)
wherein HRECU denotes the reconstruction operation and the bicubic interpolation operation, I, respectivelySRRepresents the final output result, ILRRepresenting the original input, HnRepresenting the nth information extraction function.
8. The method for reconstructing image super resolution based on attention mechanism according to claim 7, wherein said reconstructing super resolution image further comprises a loss function operation, said loss function is in a mean absolute error manner, and said final output result is processed according to the following operations:
9. a system for applying the attention-based image super-resolution reconstruction method according to any one of claims 1 to 8, comprising:
the characteristic extraction module is used for extracting the preliminary image characteristic information in the low-resolution image;
the information extraction modules are used for extracting the preliminary image feature information in a multi-dimensional mode to obtain depth image feature information;
the reconstruction module is used for reconstructing a super-resolution image according to the depth image characteristic information;
the information extraction module comprises a plurality of groups of convolution modules arranged at intervals and an attention module, and the attention module executes an attention mechanism to adjust output information passing through the convolution modules.
10. Storage medium, characterized in that it stores computer-executable instructions for causing a computer to perform the method for attention-based image super-resolution reconstruction as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010257518.9A CN111583107A (en) | 2020-04-03 | 2020-04-03 | Image super-resolution reconstruction method and system based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010257518.9A CN111583107A (en) | 2020-04-03 | 2020-04-03 | Image super-resolution reconstruction method and system based on attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111583107A true CN111583107A (en) | 2020-08-25 |
Family
ID=72122501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010257518.9A Pending CN111583107A (en) | 2020-04-03 | 2020-04-03 | Image super-resolution reconstruction method and system based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111583107A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112164011A (en) * | 2020-10-12 | 2021-01-01 | 桂林电子科技大学 | Motion image deblurring method based on self-adaptive residual error and recursive cross attention |
| CN112200724A (en) * | 2020-10-22 | 2021-01-08 | 长沙理工大学 | Single-image super-resolution reconstruction system and method based on feedback mechanism |
| CN113393382A (en) * | 2021-08-16 | 2021-09-14 | 四川省人工智能研究院(宜宾) | Binocular picture super-resolution reconstruction method based on multi-dimensional parallax prior |
| CN113409191A (en) * | 2021-06-02 | 2021-09-17 | 广东工业大学 | Lightweight image super-resolution method and system based on attention feedback mechanism |
| CN113822805A (en) * | 2021-10-13 | 2021-12-21 | 柚皮(重庆)科技有限公司 | Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment |
| CN114004784A (en) * | 2021-08-27 | 2022-02-01 | 西安市第三医院 | Method for detecting bone condition based on CT image and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123707A (en) * | 2014-08-07 | 2014-10-29 | 重庆大学 | Local rank priori based single-image super-resolution reconstruction method |
| US20180211157A1 (en) * | 2017-01-25 | 2018-07-26 | Boe Technology Group Co., Ltd. | Image processing method and device |
| WO2018216629A1 (en) * | 2017-05-22 | 2018-11-29 | キヤノン株式会社 | Information processing device, information processing method, and program |
| CN108921789A (en) * | 2018-06-20 | 2018-11-30 | 华北电力大学 | Super-resolution image reconstruction method based on recurrence residual error network |
| CN109859106A (en) * | 2019-01-28 | 2019-06-07 | 桂林电子科技大学 | A kind of image super-resolution rebuilding method based on the high-order converged network from attention |
| CN109886871A (en) * | 2019-01-07 | 2019-06-14 | 国家新闻出版广电总局广播科学研究院 | The image super-resolution method merged based on channel attention mechanism and multilayer feature |
| CN109903228A (en) * | 2019-02-28 | 2019-06-18 | 合肥工业大学 | A kind of image super-resolution rebuilding method based on convolutional neural networks |
| WO2019192588A1 (en) * | 2018-04-04 | 2019-10-10 | 华为技术有限公司 | Image super resolution method and device |
-
2020
- 2020-04-03 CN CN202010257518.9A patent/CN111583107A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123707A (en) * | 2014-08-07 | 2014-10-29 | 重庆大学 | Local rank priori based single-image super-resolution reconstruction method |
| US20180211157A1 (en) * | 2017-01-25 | 2018-07-26 | Boe Technology Group Co., Ltd. | Image processing method and device |
| WO2018216629A1 (en) * | 2017-05-22 | 2018-11-29 | キヤノン株式会社 | Information processing device, information processing method, and program |
| WO2019192588A1 (en) * | 2018-04-04 | 2019-10-10 | 华为技术有限公司 | Image super resolution method and device |
| CN108921789A (en) * | 2018-06-20 | 2018-11-30 | 华北电力大学 | Super-resolution image reconstruction method based on recurrence residual error network |
| CN109886871A (en) * | 2019-01-07 | 2019-06-14 | 国家新闻出版广电总局广播科学研究院 | The image super-resolution method merged based on channel attention mechanism and multilayer feature |
| CN109859106A (en) * | 2019-01-28 | 2019-06-07 | 桂林电子科技大学 | A kind of image super-resolution rebuilding method based on the high-order converged network from attention |
| CN109903228A (en) * | 2019-02-28 | 2019-06-18 | 合肥工业大学 | A kind of image super-resolution rebuilding method based on convolutional neural networks |
Non-Patent Citations (4)
| Title |
|---|
| WANG S: "Semi-coupled Dictionary Learning with Applications to Image Super-Resolution and Photo-sketch Synthesis", 《2012 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION.CVPR》 * |
| 钟诚等: "一种基于注意力机制的三维点云物体识别方法", 《计算机技术与发展》 * |
| 陈沅涛: "混合空间新型贝叶斯网络模型的图像分割应用研究", 《计算机工程与科学》 * |
| 鲁甜: "基于特征图注意力机制的图像超分辨率重建", 《计算机工程》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112164011A (en) * | 2020-10-12 | 2021-01-01 | 桂林电子科技大学 | Motion image deblurring method based on self-adaptive residual error and recursive cross attention |
| CN112164011B (en) * | 2020-10-12 | 2023-02-28 | 桂林电子科技大学 | Motion image deblurring method based on self-adaptive residual error and recursive cross attention |
| CN112200724A (en) * | 2020-10-22 | 2021-01-08 | 长沙理工大学 | Single-image super-resolution reconstruction system and method based on feedback mechanism |
| CN113409191A (en) * | 2021-06-02 | 2021-09-17 | 广东工业大学 | Lightweight image super-resolution method and system based on attention feedback mechanism |
| CN113393382A (en) * | 2021-08-16 | 2021-09-14 | 四川省人工智能研究院(宜宾) | Binocular picture super-resolution reconstruction method based on multi-dimensional parallax prior |
| CN114004784A (en) * | 2021-08-27 | 2022-02-01 | 西安市第三医院 | Method for detecting bone condition based on CT image and electronic equipment |
| CN114004784B (en) * | 2021-08-27 | 2022-06-03 | 西安市第三医院 | Method for detecting bone condition based on CT image and electronic equipment |
| CN113822805A (en) * | 2021-10-13 | 2021-12-21 | 柚皮(重庆)科技有限公司 | Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111583107A (en) | Image super-resolution reconstruction method and system based on attention mechanism | |
| CN109118431B (en) | Video super-resolution reconstruction method based on multiple memories and mixed losses | |
| CN108830790B (en) | Rapid video super-resolution reconstruction method based on simplified convolutional neural network | |
| CN111242846B (en) | Fine-grained scale image super-resolution method based on non-local enhancement network | |
| CN116664400A (en) | Video high space-time resolution signal processing method | |
| CN112884650B (en) | Image mixing super-resolution method based on self-adaptive texture distillation | |
| CN112580473A (en) | Motion feature fused video super-resolution reconstruction method | |
| Hui et al. | Two-stage convolutional network for image super-resolution | |
| CN113837946A (en) | Lightweight image super-resolution reconstruction method based on progressive distillation network | |
| CN115829876A (en) | Real degraded image blind restoration method based on cross attention mechanism | |
| CN115526779A (en) | Infrared image super-resolution reconstruction method based on dynamic attention mechanism | |
| Qin et al. | Difficulty-aware image super resolution via deep adaptive dual-network | |
| CN115953294A (en) | Single-image super-resolution reconstruction method based on shallow channel separation and aggregation | |
| CN112188217B (en) | JPEG compressed image decompression effect removing method combining DCT domain and pixel domain learning | |
| CN112150356A (en) | Single compressed image super-resolution reconstruction method based on cascade framework | |
| Liu et al. | From coarse to fine: Hierarchical pixel integration for lightweight image super-resolution | |
| CN113362239A (en) | Deep learning image restoration method based on feature interaction | |
| CN110956669A (en) | Image compression coding method and system | |
| CN113077403B (en) | Color image reconstruction method based on local data block tensor enhancement technology | |
| CN114820354A (en) | Traditional image compression and enhancement method based on reversible tone mapping network | |
| CN113674151A (en) | Image super-resolution reconstruction method based on deep neural network | |
| CN108629737B (en) | Method for improving JPEG format image space resolution | |
| Niu et al. | Lightweight and accurate single image super-resolution with channel segregation network | |
| Jiang et al. | Image interpolation model based on packet losing network | |
| CN112261415B (en) | Image compression coding method based on overfitting convolution self-coding network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200825 |