CN113822805A - Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment - Google Patents
Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment Download PDFInfo
- Publication number
- CN113822805A CN113822805A CN202111191406.9A CN202111191406A CN113822805A CN 113822805 A CN113822805 A CN 113822805A CN 202111191406 A CN202111191406 A CN 202111191406A CN 113822805 A CN113822805 A CN 113822805A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- super
- reconstruction
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 36
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000003745 diagnosis Methods 0.000 title claims abstract description 22
- 230000004927 fusion Effects 0.000 claims abstract description 22
- 239000003814 drug Substances 0.000 claims abstract description 21
- 238000000605 extraction Methods 0.000 claims abstract description 14
- 241000196324 Embryophyta Species 0.000 claims description 34
- 238000010586 diagram Methods 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 8
- 238000013178 mathematical model Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 4
- 230000010339 dilation Effects 0.000 claims description 4
- 239000012634 fragment Substances 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000012549 training Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000003321 amplification Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- OCKGFTQIICXDQW-ZEQRLZLVSA-N 5-[(1r)-1-hydroxy-2-[4-[(2r)-2-hydroxy-2-(4-methyl-1-oxo-3h-2-benzofuran-5-yl)ethyl]piperazin-1-yl]ethyl]-4-methyl-3h-2-benzofuran-1-one Chemical compound C1=C2C(=O)OCC2=C(C)C([C@@H](O)CN2CCN(CC2)C[C@H](O)C2=CC=C3C(=O)OCC3=C2C)=C1 OCKGFTQIICXDQW-ZEQRLZLVSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 229940126680 traditional chinese medicines Drugs 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image super-resolution reconstruction method, which comprises the steps of acquiring an initial image and a reconstruction network, and extracting the characteristics of the initial image by using an initial characteristic extraction module; and the initial feature map sequentially passes through a plurality of MTFRB modules, the feature maps output by each MTFRB module are spliced, the feature fusion module is used for reducing the dimension of the spliced feature map, and the reconstruction module is used for performing super-resolution reconstruction on the fused feature map. The method fully utilizes the extracted initial image feature information through feature fusion, reduces detail loss and has a good super-resolution reconstruction effect. The invention also provides a method and equipment for diagnosing the leaf disease of the traditional Chinese medicine plant, wherein the method for diagnosing the leaf disease of the traditional Chinese medicine plant comprises the steps of super-resolution reconstruction, screenshot, classified diagnosis and the like; by means of the super-resolution reconstruction technology, the accuracy of plant leaf disease diagnosis in a low-resolution scene is greatly improved, and the leaf disease diagnosis workload is reduced.
Description
Technical Field
The invention belongs to the technical field of traditional Chinese medicines and artificial intelligence, and particularly relates to an image super-resolution reconstruction method, a traditional Chinese medicine plant leaf disease diagnosis method and equipment.
Background
Timely discovery and symptomatic medicine administration are scientific and effective methods for controlling the leaf diseases of Chinese medicinal plants, but the types and the expression forms of the leaf diseases are various, common workers are difficult to diagnose quickly and accurately, and finally the diseases are diffused in a large area, so that serious economic and environmental losses are caused. With the development of the technology, people gradually apply the artificial intelligence technology to the field of leaf disease control, and the conventional method is to collect leaves manually, then shoot and obtain electronic images of the traditional Chinese medicine plant leaves, input the images into a computer, and extract and classify the features of the leaves by using a trained artificial neural network, so that the aim of auxiliary diagnosis is fulfilled.
Although the method improves the accuracy of disease diagnosis, the workload of acquiring the leaves and shooting images in the early stage is large. The method is a potential feasible method for solving the problem that the traditional Chinese medicine plant leaves are shot by erecting image acquisition equipment from a high place, and the resolution ratio of the obtained leaves is lower due to the fact that the distance from a camera to the plant leaves is relatively long, so that the disease diagnosis accuracy is greatly reduced.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an image super-resolution reconstruction method for improving the resolution of the shot plant leaf image, and also provides a method and equipment for diagnosing the plant leaf disease of the traditional Chinese medicine, wherein the correct rate of the plant leaf disease diagnosis is improved by means of an image super-resolution technology.
In order to achieve the above purpose, the solution adopted by the invention is as follows: an image super-resolution reconstruction method comprises the following steps:
a1, acquiring an initial image needing super-resolution reconstruction, and acquiring a trained image super-resolution reconstruction network, wherein the image super-resolution reconstruction network comprises an initial feature extraction module, an MTFRB module, a feature fusion module and a reconstruction module;
a2, performing feature extraction on the initial image by using the initial feature extraction module to obtain an initial feature map;
a3, sequentially passing the initial feature map through a plurality of MTFRB modules (the feature map output by the previous MTFRB module is used as the input of the next MTFRB module), and performing further feature extraction on the initial feature map by using the MTFRB modules;
a4, extracting feature maps output by all MTFRB modules, splicing the feature maps output by each MTFRB module (in the channel direction), and then reducing the dimension of the spliced feature maps (in the channel direction) by using the feature fusion module to generate a fusion feature map;
and A5, performing super-resolution reconstruction on the fusion feature map by using the reconstruction module to obtain a target image, wherein the resolution of the target image is greater than that of the initial image.
Further, the mathematical model of the MTFRB module is:
F1＝f11[σ1(f3(X)),σ2(f3dc(X)),σ3(f5(X))]
F2＝σ4(fdi1(σ1(f3(X))))
F3＝σ5(fdi2(σ3(f5(X))))
Y＝X+f12[F1,F2,F3]
wherein X is a characteristic diagram input into the MTFRB module, f3Representing a convolution operation with a convolution kernel size of 3 x 3, f5Representing a convolution operation with a convolution kernel size of 5 x 5, f3dcA deformable convolution operation, σ, representing a convolution kernel size of 3 x 31、σ2、σ3、σ4And σ5Both represent a non-linear activation function ReLU, f11And f12All represent convolution operations with a convolution kernel size of 1 x 1, fdi1And fdi2Each represents a dilation convolution operation with a convolution kernel size of 3 x 3 and a dilation rate of 2 [ ·]And the splicing operation of the characteristic graphs in the MTFRB module is shown, and Y is the characteristic graph output by the MTFRB module.
Further, the mathematical model of the reconstruction module is:
Fr＝fsp1(f3r1(W))
IHR＝f3r2(Fr)
wherein W represents a feature map input to the reconstruction module, f3r1And f3r2Respectively representing the convolution operations of two convolution kernels of size 3 x 3 in said reconstruction module, fsp1Representing a sub-pixel convolution operation, FrRepresentation of a reconstruction modelFeature map, I, output after sub-pixel convolution operation in blockHRRepresenting the target image output by the reconstruction module.
Further, the super-resolution reconstruction network is further provided with a global spatial attention module, and the global spatial attention module can be represented as the following mathematical model:
Fsm＝[σb0(f1b0(Y0)),σb1(f1b1(Y1)),…,σbm(f1bm(Ym))]
Fss＝δ(f1bs(Fsm))
W＝Fsa(Z,Fss)
wherein, Y0Represents said initial characteristic map, Y1A characteristic diagram, Y, representing the output of the first said MTFRB modulemA characteristic diagram, Y, representing the output of the mth said MTFRB module0、Y1...YmAs input to the global spatial attention module, f1b0、f1b1...f1bmAnd f1bsConvolution operations, σ, all with a convolution kernel size of 1 x 1b0、σb1...σbmAre all ReLU activation functions [ ·]Showing the splicing operation of the characteristic diagrams therein, FsmRepresenting the generated intermediate attention map, δ being the sigmoid activation function, FssRepresenting the generated global spatial attention map, Z representing the fused feature map, Fsa(Z,Fss) And the global space attention diagram is multiplied by the fused feature diagram, W is the feature diagram generated after the global space attention diagram is fused with the fused feature diagram, and W is used as the feature diagram input into the reconstruction module.
Further, the super-resolution reconstruction network is provided with a branch module, and the mathematical model of the branch module is as follows:
Fk＝fsp2(f3k(Fsm))
FU＝Fr+Fk
wherein, FsmRepresenting said intermediate attention map, f3kRepresenting a convolution operation with a convolution kernel size of 3 x 3, fsp2Representing a sub-pixel convolution operation in said branching module, FrA feature map representing the output of said reconstruction module after a sub-pixel convolution operation, FURepresents FrAnd FkAnd generating a feature map after element summation.
The invention also provides a method for diagnosing the leaf diseases of the traditional Chinese medicine plants, which comprises the following steps:
s1, using the scene image with the monitored plant leaves as an initial image, and performing super-resolution reconstruction on the scene image according to the image super-resolution reconstruction method to obtain a reconstructed image with resolution higher than that of the scene image;
s2, intercepting leaf image fragments suspected of being diseased from the reconstructed image;
s3, acquiring a trained image classification network, inputting the image segments obtained in the step S2 into the image classification network, and classifying the image segments by using the image classification network, thereby realizing plant leaf disease diagnosis.
The invention also provides a Chinese medicine plant leaf disease diagnosis device which comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used for executing the Chinese medicine plant leaf disease diagnosis method by loading the computer program.
The invention has the beneficial effects that:
(1) in the super-resolution reconstruction network, because the position depths of different MTFRB modules in the network are different, the output feature maps contain feature information of different levels, and the feature maps output by all the MTFRB modules are fused by using a feature fusion module, so that the extracted initial image feature information can be fully utilized, the super-resolution reconstruction effect is improved, and meanwhile, gradient explosion or gradient disappearance in the model training process is avoided;
(2) for the MTFRB module, common convolution with different convolution kernel sizes is matched with deformable convolution, extracting the features of the feature map input into the MTFRB module to extract the features as much as possible under different visual fields, after the extracted features are preliminarily fused, two expansion convolutions are utilized to further extract the features of the feature graph output by the previous ordinary convolution on a larger visual field, finally the feature graph output by the expansion convolutions and the feature graph output by the first fusion are fused for the second time, and through two times of feature fusion, the occupation ratio of useful information in the output characteristic diagram of the MTFRB module can be greatly improved, the redundancy of the useless information is reduced, meanwhile, the characteristic reuse is increased, the detail loss caused in the characteristic transmission process is reduced, the reconstructed image texture is clearer, and the disease diagnosis is more favorably carried out according to the appearance of the surfaces of the traditional Chinese medicine plant leaves;
(3) the global space attention module is matched with the fusion feature map, so that more accurate weight parameters can be given to different positions on the fusion feature map space, the importance of a useful area on the feature map space can be more accurately improved, and the important promotion effect on the high-frequency information of the blade surface is realized;
(4) the feature graph input into the global space attention module is output from the initial feature extraction module and the MTFRB module, and a special feature extraction module does not need to be arranged aiming at the global space attention mechanism, so that the calculated amount is reduced, and the features of the network backbone middle layer are more fully utilized;
(5) after the traditional attention mechanism extracts the features, only multiplication operation is carried out on the feature graph in a weight parameter mode, and the utilization rate of the extracted feature information in the attention mechanism is low, the invention fuses an intermediate attention diagram generated in a global space attention module with a reconstruction module through a branch module, and because the information in the intermediate attention diagram comes from different depths of a network, the information is directly transmitted to the tail part of the network in a jump connection mode, so that the loss of details in the network transmission process is reduced, and the utilization rate of the feature information is improved;
(6) according to the method, the resolution of the scene image is improved by using the super-resolution reconstruction algorithm, then the image fragment is intercepted for diagnosis, even if the resolution of the acquired traditional Chinese medicine plant leaf image is low, the type of the leaf disease can be accurately diagnosed, the accuracy of diagnosing the plant leaf disease in a low-resolution scene is greatly improved, a user can shoot the leaf image through a lens arranged at a high position, the workload is reduced, the early stage of the disease can be timely found, the disease spreading can be effectively controlled by adopting a treatment means in a targeted manner, and the damage of the leaf disease to economy and environment is reduced.
Drawings
FIG. 1 is a schematic diagram of a network structure for super-resolution image reconstruction according to an embodiment;
FIG. 2 is a schematic structural diagram of an MTFRB module in the image super-resolution reconstruction network shown in FIG. 1;
FIG. 3 is a schematic diagram of a super-resolution image reconstruction network according to another embodiment;
in the drawings:
1-initial image, 2-target image, 3-initial feature extraction module, 4-MTFRB module, 5-feature fusion module, 6-reconstruction module, 7-global space attention module and 8-branch module.
Detailed Description
The invention is further described below with reference to the accompanying drawings:
example 1:
the image super-resolution reconstruction network shown in FIG. 1 is constructed by using a python code by means of a pytorch framework. In this embodiment, the initial feature extraction module 3 is a convolution operation with a convolution kernel size of 3 × 3, the structure of the MTFRB module 4 is shown in fig. 2, and the number of the MTFRB modules 4 is 6. For the feature map input to the MTFRB module 4, firstly, the feature is extracted for the first time by using the deformable convolution with convolution kernels respectively having sizes of 3 × 3 and 5 × 5 and convolution kernel size of 3 × 3, and after the output feature maps are spliced, the number of channels is reduced by one convolution operation of 1 × 1, thereby realizing the first feature fusion. And then, further extracting features from the feature graph output by the previous ordinary convolution on a larger view by utilizing the expansion convolution with the two convolution kernels with the size of 3 × 3 and the expansion rate of 2, finally splicing the feature graph output by the expansion convolution with the feature graph after the first fusion, and reducing the number of channels through the convolution operation of 1 × 1, so that the number of the channels input into the feature graph of the MTFRB module 4 is equal to the number of the channels input into the feature graph of the MTFRB module 4, and realizing the second fusion. In order to make the model easier to train, residual connection is also added into the MTFRB module 4, as shown in fig. 2, the feature map input into the MTFRB module 4 is directly transmitted to the output end of the MTFRB module 4 through the residual connection, and is subjected to element summation operation with the feature map output after the second fusion in the MTFRB module 4. In this embodiment, the feature fusion module 5 is implemented by using a convolution operation with a convolution kernel size of 1 × 1, and the reconstruction module 6 includes three parts, namely a first 3 × 3 convolution layer, a sub-pixel convolution layer and a second 3 × 3 convolution layer, which are connected in sequence.
In the super-resolution reconstruction network, the step size of all convolution operations is 1. Specifically, when the super-resolution reconstruction network runs, after the initial image 1 is input into the initial feature extraction module 3, the number of output channels is 128, for the interior of each MTFRB module 4, the number of channels input and output by 3 × 3 convolution, 5 × 5 convolution, deformable convolution and expansion convolution is 128, the number of channels input by two 1 × 1 convolutions in the interior of the MTFRB module 4 is 384, and the number of input channels is 128. For feature fusion module 5, the number of input channels is 768 and the number of output channels is 128. For reconstruction module 6, the number of input channels for the first convolutional layer is 128, and the number of output channels is determined by the amplification factor. Assuming that the amplification factor is n, the number of channels output by the first convolution layer of the reconstruction module 6 is 128 × n. After the feature map is subjected to sub-pixel convolution, the number of channels becomes 128, and the feature map size becomes n times of the original size. And finally, the number of the characteristic diagram channels output by the second convolution layer of the reconstruction module 6 is 3, and the amplified target image 2 is obtained.
When the super-resolution reconstruction network model is trained, a DIV2K data set is used as a training set, and a low-resolution image corresponding to a high-resolution image is obtained through down-sampling. The loss function is L1, the optimizer is SGD, the learning rate is fixed at 0.0002, and the epoch number is set at 1500. After training is completed, testing is performed on three data sets, Set5, BSDS100, and Urban 100. Two parameters of peak signal-to-noise ratio (PSNR) and Structural Similarity (SSIM) are used as the measuring standard of the image reconstruction effect. During testing, the image is converted into YCbCr format, then the relevant parameters are calculated on the Y channel, and the comparison result of the test data and other models is shown as follows.
From the comparison results, the image reconstruction effects of the image super-resolution reconstruction method provided by the invention under different magnification factors are higher than those of the current advanced-performance RCAN and SAN models in example 1, which shows that the super-resolution reconstruction method provided by the invention has a good magnification effect on low-resolution images, and can well improve the resolution of low-resolution traditional Chinese medicine plant leaf images, thereby improving the diagnosis accuracy.
Example 2:
on the basis of the model of the embodiment 1, a global space attention module 7 is added to obtain the model of the embodiment 2A. On the basis of the embodiment 1, a global spatial attention module 7 and a branch module 8 are added to obtain an embodiment 2B model (shown in FIG. 3). For the global spatial attention module 7, the initial feature map and the feature map output by the MTFRB module 4 are reduced in dimension by convolution with 1 × 1, so that the number of channels is changed from 128 to 16. After the ReLU activation function, the feature maps are spliced together to generate an intermediate attention map with the channel number of 112. And then, dimension reduction is carried out on the intermediate attention diagram by using 1-by-1 convolution again, the channel is changed into 1, and the full local space attention diagram is generated through activation of the sigmoid function and is multiplied by the fused feature diagram. For the branching module 8, the intermediate attention map is first input into one convolution layer of 3 × 3, and then the feature maps of 128 × n (n is the image magnification) channels are output. After the feature map is convolved by sub-pixels in the branch module 8, the number of channels is changed into 128, the length and width of the image are changed into n times of the original length and width, and finally the feature map is fused with the feature map output by the sub-pixel convolution in the reconstruction module 6 through element summation.
The models of example 2A and example 2B were trained and tested using the same data set, training environment, optimizer, learning rate, etc. as in example 1, and the comparison results are shown below:
as can be seen from the comparison experiment results, after the global spatial attention module 7 and the branch module 8 are added, on the basis of the embodiment 1, the image super-resolution reconstruction effect is improved, which indicates that the global spatial attention module 7 and the branch module 8 also have a promoting effect on improving the leaf disease diagnosis accuracy of the traditional Chinese medicine plant.
Example 3:
the traditional Chinese medicine plant leaf is placed on the ground, a camera arranged at a high position is used for shooting the leaf, 150 scene image test data sets with traditional Chinese medicine plant leaf images are obtained, and the 150 images contain 20 common traditional Chinese medicine plant leaf diseases and normal plant leaves. The experimental group firstly uses the trained model of embodiment 2B in embodiment 2 to carry out super-resolution reconstruction with 4 times of magnification on a scene image, then intercepts the image segment of the traditional Chinese medicine plant leaf, and finally classifies the image segment of the plant leaf through the trained classification network so as to diagnose whether the leaf in the image is normal or is specifically suffered from certain leaf diseases. The classification network adopts mobileNet v3, and the classification network adopts more plant leaf disease image data sets for training. The contrast group is the same as the experimental group in other experimental steps and experimental conditions except that the super-resolution network is not adopted for reconstructing the scene image. The experimental result shows that the Chinese medicinal plant leaf disease diagnosis correct rate in the experimental group is 94.8%, and the Chinese medicinal plant leaf disease diagnosis correct rate in the control group is 72.6%. After super-resolution reconstruction, the accuracy of the classification network in diagnosing the leaf diseases of the traditional Chinese medicine plants is greatly improved.
The above-mentioned embodiments only express the specific embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (7)
1. An image super-resolution reconstruction method is characterized in that: the method comprises the following steps:
a1, acquiring an initial image needing super-resolution reconstruction, and acquiring a trained image super-resolution reconstruction network, wherein the image super-resolution reconstruction network comprises an initial feature extraction module, an MTFRB module, a feature fusion module and a reconstruction module;
a2, performing feature extraction on the initial image by using the initial feature extraction module to obtain an initial feature map;
a3, sequentially passing the initial feature map through a plurality of MTFRB modules, and performing further feature extraction on the initial feature map by using the MTFRB modules;
a4, splicing the feature maps output by each MTFRB module, and then using the feature fusion module to perform dimension reduction on the spliced feature maps to generate a fusion feature map;
and A5, performing super-resolution reconstruction on the fusion feature map by using the reconstruction module to obtain a target image, wherein the resolution of the target image is greater than that of the initial image.
2. The image super-resolution reconstruction method according to claim 1, wherein: the mathematical model of the MTFRB module is as follows:
F1＝f11[σ1(f3(X)),σ2(f3dc(X)),σ3(f5(X))]
F2＝σ4(fdi1(σ1(f3(X))))
F3＝σ5(fdi2(σ3(f5(X))))
Y＝X+f12[F1,F2,F3]
wherein X is the input to the MTFRBCharacteristic diagram of the module, f3Representing a convolution operation with a convolution kernel size of 3 x 3, f5Representing a convolution operation with a convolution kernel size of 5 x 5, f3dcA deformable convolution operation, σ, representing a convolution kernel size of 3 x 31、σ2、σ3、σ4And σ5Both represent a non-linear activation function ReLU, f11And f12All represent convolution operations with a convolution kernel size of 1 x 1, fdi1And fdi2Each represents a dilation convolution operation with a convolution kernel size of 3 x 3 and a dilation rate of 2 [ ·]And the splicing operation of the characteristic graphs in the MTFRB module is shown, and Y is the characteristic graph output by the MTFRB module.
3. The image super-resolution reconstruction method according to claim 1, wherein: the mathematical model of the reconstruction module is as follows:
Fr＝fsp1(f3r1(W))
IHR＝f3r2(Fr)
wherein W represents a feature map input to the reconstruction module, f3r1And f3r2Respectively representing the convolution operations of two convolution kernels of size 3 x 3 in said reconstruction module, fsp1Representing a sub-pixel convolution operation, FrFeature map, I, representing the output of a reconstruction module after a sub-pixel convolution operationHRRepresenting the target image output by the reconstruction module.
4. The image super-resolution reconstruction method according to claim 3, wherein: the super-resolution reconstruction network is also provided with a global spatial attention module, and the global spatial attention module can be expressed as the following mathematical model:
Fsm＝[σb0(f1b0(Y0)),σb1(f1b1(Y1)),…,σbm(f1bm(Ym))]
Fss＝δ(f1bs(Fsm))
W＝Fsa(Z,Fss)
wherein, Y0Represents said initial characteristic map, Y1A characteristic diagram, Y, representing the output of the first said MTFRB modulemA characteristic diagram, Y, representing the output of the mth said MTFRB module0、Y1...YmAs input to the global spatial attention module, f1b0、f1b1...f1bmAnd f1bsConvolution operations, σ, all with a convolution kernel size of 1 x 1b0、σb1...σbmAre all ReLU activation functions [ ·]Showing the splicing operation of the characteristic diagrams therein, FsmRepresenting the generated intermediate attention map, δ being the sigmoid activation function, FssRepresenting the generated global spatial attention map, Z representing the fused feature map, Fsa(Z,Fss) Representing the multiplication of the global spatial attention map with the fused feature map, W represents the feature map input to the reconstruction module.
5. The image super-resolution reconstruction method according to claim 4, wherein: the super-resolution reconstruction network is provided with a branch module, and the mathematical model of the branch module is as follows:
Fk＝fsp2(f3k(Fsm))
FU＝Fr+Fk
wherein, FsmRepresenting said intermediate attention map, f3kRepresenting a convolution operation with a convolution kernel size of 3 x 3, fsp2Representing a sub-pixel convolution operation in said branching module, FrA feature map representing the output of said reconstruction module after a sub-pixel convolution operation, FURepresents FrAnd FkAnd generating a feature map after element summation.
6. A method for diagnosing the leaf diseases of Chinese herbal plants is characterized by comprising the following steps: the method comprises the following steps:
s1, performing super-resolution reconstruction on the scene image with the monitored plant leaves as an initial image according to the image super-resolution reconstruction method of any one of claims 1-5, and obtaining a reconstructed image with resolution higher than that of the scene image;
s2, intercepting leaf image fragments suspected of being diseased from the reconstructed image;
s3, acquiring a trained image classification network, inputting the image segments obtained in the step S2 into the image classification network, and classifying the image segments by using the image classification network, thereby realizing plant leaf disease diagnosis.
7. A diagnosis device for leaf diseases of traditional Chinese medicine plants is characterized in that: comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used for executing the traditional Chinese medicine plant leaf disease diagnosis method according to claim 6 by loading the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111191406.9A CN113822805B (en) | 2021-10-13 | 2021-10-13 | Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111191406.9A CN113822805B (en) | 2021-10-13 | 2021-10-13 | Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113822805A true CN113822805A (en) | 2021-12-21 |
| CN113822805B CN113822805B (en) | 2022-10-18 |
Family
ID=78920251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111191406.9A Active CN113822805B (en) | 2021-10-13 | 2021-10-13 | Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113822805B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114283069A (en) * | 2022-01-17 | 2022-04-05 | 柚皮(重庆)科技有限公司 | Brain magnetic resonance image super-resolution reconstruction method |
| CN114549962A (en) * | 2022-03-07 | 2022-05-27 | 重庆锐云科技有限公司 | Garden plant leaf disease classification method |
| CN115187814A (en) * | 2022-07-25 | 2022-10-14 | 重庆芸山实业有限公司 | Chrysanthemum mosaic disease diagnosis method and device based on artificial intelligence |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583107A (en) * | 2020-04-03 | 2020-08-25 | 长沙理工大学 | Image super-resolution reconstruction method and system based on attention mechanism |
| CN111652804A (en) * | 2020-05-28 | 2020-09-11 | 西安电子科技大学 | Super-resolution reconstruction method based on expansion convolution pyramid and bottleneck network |
| US20200302249A1 (en) * | 2019-03-19 | 2020-09-24 | Mitsubishi Electric Research Laboratories, Inc. | Systems and Methods for Multi-Spectral Image Fusion Using Unrolled Projected Gradient Descent and Convolutinoal Neural Network |
| CN112767253A (en) * | 2021-02-05 | 2021-05-07 | 西南科技大学 | Multi-scale feature fusion binocular image super-resolution reconstruction method |
| CN113436076A (en) * | 2021-07-26 | 2021-09-24 | 柚皮(重庆)科技有限公司 | Image super-resolution reconstruction method with characteristics gradually fused and electronic equipment |
-
2021
- 2021-10-13 CN CN202111191406.9A patent/CN113822805B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200302249A1 (en) * | 2019-03-19 | 2020-09-24 | Mitsubishi Electric Research Laboratories, Inc. | Systems and Methods for Multi-Spectral Image Fusion Using Unrolled Projected Gradient Descent and Convolutinoal Neural Network |
| CN111583107A (en) * | 2020-04-03 | 2020-08-25 | 长沙理工大学 | Image super-resolution reconstruction method and system based on attention mechanism |
| CN111652804A (en) * | 2020-05-28 | 2020-09-11 | 西安电子科技大学 | Super-resolution reconstruction method based on expansion convolution pyramid and bottleneck network |
| CN112767253A (en) * | 2021-02-05 | 2021-05-07 | 西南科技大学 | Multi-scale feature fusion binocular image super-resolution reconstruction method |
| CN113436076A (en) * | 2021-07-26 | 2021-09-24 | 柚皮(重庆)科技有限公司 | Image super-resolution reconstruction method with characteristics gradually fused and electronic equipment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114283069A (en) * | 2022-01-17 | 2022-04-05 | 柚皮(重庆)科技有限公司 | Brain magnetic resonance image super-resolution reconstruction method |
| CN114549962A (en) * | 2022-03-07 | 2022-05-27 | 重庆锐云科技有限公司 | Garden plant leaf disease classification method |
| CN115187814A (en) * | 2022-07-25 | 2022-10-14 | 重庆芸山实业有限公司 | Chrysanthemum mosaic disease diagnosis method and device based on artificial intelligence |
| CN115187814B (en) * | 2022-07-25 | 2024-05-10 | 重庆芸山实业有限公司 | Artificial intelligence-based chrysanthemum mosaic disease diagnosis method and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113822805B (en) | 2022-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110033410B (en) | Image reconstruction model training method, image super-resolution reconstruction method and device | |
| WO2021184891A1 (en) | Remotely-sensed image-based terrain classification method, and system | |
| CN111127374B (en) | Pan-sharing method based on multi-scale dense network | |
| CN113822805B (en) | Image super-resolution reconstruction method and Chinese medicinal plant leaf disease diagnosis method and equipment | |
| CN110223234A (en) | Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion | |
| CN109325915B (en) | Super-resolution reconstruction method for low-resolution monitoring video | |
| CN110148103A (en) | EO-1 hyperion and Multispectral Image Fusion Methods, computer readable storage medium, electronic equipment based on combined optimization | |
| Chen et al. | Single image super-resolution using deep CNN with dense skip connections and inception-resnet | |
| CN110599502B (en) | Skin lesion segmentation method based on deep learning | |
| CN106157249A (en) | Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood | |
| Wang et al. | Improved face super-resolution generative adversarial networks | |
| CN111915545A (en) | Self-supervision learning fusion method of multiband images | |
| CN114419612A (en) | Image super-resolution reconstruction method and device for scenic spot license plate recognition | |
| Xue et al. | TC-net: transformer combined with cnn for image denoising | |
| Huan et al. | MAENet: multiple attention encoder–decoder network for farmland segmentation of remote sensing images | |
| Zheng et al. | Double-branch dehazing network based on self-calibrated attentional convolution | |
| CN117671509B (en) | Remote sensing target detection method and device, electronic equipment and storage medium | |
| Manzari et al. | A robust network for embedded traffic sign recognition | |
| Zhou et al. | MSAR‐DefogNet: Lightweight cloud removal network for high resolution remote sensing images based on multi scale convolution | |
| CN113160078B (en) | Method, device and equipment for removing rain from traffic vehicle image in rainy day and readable storage medium | |
| CN105844589A (en) | Method for realizing light field image super-resolution based on mixed imaging system | |
| Guo et al. | A novel lightweight multi-dimension feature fusion network for single-image super-resolution reconstruction | |
| CN114882203B (en) | Image super-resolution reconstruction method for power grid inspection robot | |
| Zhao et al. | Multi-scale residual fusion network for super-resolution reconstruction of single image | |
| Li et al. | Semantic prior-driven fused contextual transformation network for image inpainting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220922 Address after: 518000 Room 910-912, Tianliao Building, Tangchang Road, Tanglang Community, Taoyuan Street, Nanshan District, Shenzhen, Guangdong, China Applicant after: Shenzhen Youxiang Technology Co.,Ltd. Address before: 400000 2-3-20, building 4, No. 188, Lianggang Avenue, Yubei District, Chongqing Applicant before: Pomelo peel (Chongqing) Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |