CN114722928B - Blue algae image recognition method based on deep learning - Google Patents

Blue algae image recognition method based on deep learning Download PDF

Info

Publication number
CN114722928B
CN114722928B CN202210318599.8A CN202210318599A CN114722928B CN 114722928 B CN114722928 B CN 114722928B CN 202210318599 A CN202210318599 A CN 202210318599A CN 114722928 B CN114722928 B CN 114722928B
Authority
CN
China
Prior art keywords
blue algae
feature
matrix
granularity
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210318599.8A
Other languages
Chinese (zh)
Other versions
CN114722928A (en
Inventor
戚荣志
陈春雨
李水艳
叶凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hohai University HHU
Original Assignee
Hohai University HHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hohai University HHU filed Critical Hohai University HHU
Priority to CN202210318599.8A priority Critical patent/CN114722928B/en
Publication of CN114722928A publication Critical patent/CN114722928A/en
Application granted granted Critical
Publication of CN114722928B publication Critical patent/CN114722928B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A20/00—Water conservation; Efficient water supply; Efficient water use
    • Y02A20/152—Water filtration

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a blue algae image recognition method based on deep learning, which utilizes ResNet based on bilinear network improvement to perform coarse-granularity feature extraction to obtain a coarse-granularity feature map of a blue algae image; adding an attention mechanism module of a convolution module after ResNet, and obtaining important blue algae characteristic information in the image through learning weights to obtain enhanced fine granularity characteristics; carrying out bilinear fusion on blue algae fine granularity image features extracted by A, B feature extraction functions in the bilinear model to obtain an M multiplied by N matrix b; and carrying out summation pooling on the matrix b to obtain a matrix xi, carrying out vector expansion recombination on the matrix xi to obtain a feature vector x, and predicting the bilinear vector x by using a classification function. The invention fully plays the characteristic extraction advantages of the bilinear model and the attention mechanism on the fine-grained image, achieves higher blue algae identification precision, enhances the robustness of the network model, and effectively completes the identification of the blue algae image.

Description

Blue algae image recognition method based on deep learning
Technical Field
The invention relates to a blue algae image recognition method based on deep learning, and belongs to the technical field of computer vision.
Background
Water is a source of everything, is an important natural resource for human beings, is a basic condition for survival and development of various lives of human beings, animals, plants and the like, and is an extremely important guarantee for sustainable development of society and economy. However, with the development of society, industry and the like and population growth, water resource pollution is increasingly serious, wherein eutrophication of water is a major problem faced by water resources in China, particularly inland lakes. An important feature of eutrophication is the mass propagation of algae material, particularly blue algae. Blue algae grow abnormally, are easy to accumulate, decay and subside to form water bloom, and accumulate at river mouth and near shore to destroy the balance of water landscape and ecological system, and the blue algae release toxin and consume dissolved oxygen in the growth process to cause massive death of water organisms, so that the water quality of lakes is deteriorated and the drinking water safety of areas around the lakes is seriously threatened. Therefore, the blue algae distribution information is rapidly and comprehensively mastered, and the blue algae distribution information is very important for controlling blue algae bloom, evaluating the ecological environment risk of blue algae, researching the cause of abnormal growth of blue algae and establishing a water quality early warning system.
The domestic traditional monitoring method for blue algae mainly comprises manual monitoring and remote sensing monitoring. The manual monitoring is to collect water samples of representative stations, carry out laboratory detection on chlorophyll, algae density, eutrophication conditions and the like, and carry out qualitative and quantitative evaluation on the occurrence of blue algae according to detection results. Meanwhile, by combining manual on-site inspection, the aggregation state and distribution situation of on-site blue algae are observed, and a blue algae distribution schematic diagram of the easily-developed water area is drawn. The blue algae bloom remote sensing monitoring method is divided into algal bloom identification and quantitative inversion. The algal bloom recognition synthesizes a blue algae water bloom image through 3 red, green and blue channels or vegetation indexes; the quantitative inversion method is to study the relation between the water reflection spectrum characteristic and the water quality index concentration by measuring the water radiation value in a certain wavelength range, and establish the water quality index inversion algorithm of chlorophyll a, algae density and the like. In practical application, manual monitoring and remote sensing monitoring have certain limitations. For example, the cost of manual monitoring resources is huge, the monitoring area of the water body is small, the remote sensing monitoring is easily influenced by external condition factors such as weather, illumination and the like, and the space-time resolution is not high.
Along with the continuous perfection of water conservancy infrastructure, important rivers and lakes, shorelines and hydraulic engineering have built a large number of video monitoring stations, a large number of image video resources are provided for river and lake management and water ecological protection, and the monitoring of pollution conditions of water resources such as blue algae is facilitated. However, the current use of image resources is mainly classified by manual query browsing and manual labeling, and the efficiency is low. With the continuous maturity of machine learning technology in recent years, computers are becoming more and more widely used in solving the problem of automatic classification of images. As one of machine learning algorithms, deep learning, especially convolutional neural networks, has the fastest development and autonomous learning characteristics, and is a reasonable solution for realizing automatic blue algae identification and classification (without blue algae, particles, bands and sheets). However, blue algae image recognition has the characteristic of fine-grained image classification with small inter-class difference and large intra-class difference, and how to effectively locate locally differentiated areas of images and comprehensively express image characteristic information is a problem to be solved in blue algae image classification by using deep learning.
Disclosure of Invention
The invention aims to: aiming at the problems and the defects existing in the prior art, the invention provides a blue algae image identification method based on deep learning.
The technical scheme is as follows: a blue algae image recognition method based on deep learning comprises the following steps:
(1) Carrying out coarse-granularity feature extraction by utilizing ResNet based on bilinear network improvement to obtain a coarse-granularity feature map of the blue algae image;
(2) Adding an attention mechanism module (Convolutional Block Attention Module, CBAM) of the convolution module after ResNet, and obtaining important blue algae characteristic information in the image through learning weights to obtain enhanced fine granularity characteristics;
(3) Carrying out bilinear fusion on blue algae fine granularity image features extracted by A, B feature extraction functions in the bilinear model to obtain an M multiplied by N matrix b;
(4) And (3) carrying out summation pooling on the matrix b obtained in the step (3) to obtain a matrix xi, carrying out vector expansion recombination on the matrix xi to obtain a feature vector x, and predicting the bilinear vector x by using a classification function.
Further, in step (1), feature extraction is performed by using the bilinear network-based improvement ResNet to obtain a coarse-granularity feature map of the blue algae image, and compared with the prototype network ResNet18, the improvement ResNet increases the convolution kernel number of the convolution layer and reduces part of the convolution layer.
The improved ResNet network is based on the prototype network ResNet, the convolution kernel number of the first layer of convolution layers is changed from 64 to 128, and in addition, the number of convolution layers after max pooling (maximum pooling) operation in ResNet is reduced, and the number of convolution layers is reduced from 4 layers to 2 layers. In modified ResNet, the calculation is simplified using a ReLU activation function, downsampling is performed using a convolution kernel of step size 2, the convolution kernel size of the convolution layer being 3×3. Compared with the traditional convolutional neural network, the improved ResNet network uses residual units to solve the performance degradation problem when the network is deepened, and the residual units are similar to short-circuit connection in a circuit in structure. The residual unit can be expressed as: y l＝h(xl)+F(xl,Wl),xl+1＝f(yl). Where x l and x l+1 represent the input and output, respectively, of the first residual unit, each residual unit typically comprising a multi-layer structure. F is a residual function representing the learned residual, and h (x l)＝xl represents identity mapping, F is a ReLU activation function).
Further, in step (2), the acquired coarse granularity feature map is input to a channel attention module (Channel Attention Module, CAM) and a spatial attention module (Spatial Attention Module, SAM) of the attention mechanism module CBAM to acquire channel attention and spatial attention, respectively, and the acquired channel attention and spatial attention are weighted to the coarse granularity feature map output in step (1) respectively, so as to obtain a new fine granularity feature map, which specifically includes the following steps:
2.1 The coarse granularity characteristic diagram F (H multiplied by W multiplied by C) of the image output in the step (1) is subjected to global maximum pooling and global average pooling based on width and height respectively to obtain two channel attention characteristic diagrams of 1 multiplied by C, and then the channel attention characteristic diagrams are respectively sent to a two-layer multi-layer sensor. And then, adding the characteristics output by the multi-layer perceptron on the basis of element layers, and generating a final channel attention characteristic M_c through sigmoid activation operation. And finally, performing element level multiplication operation on the channel attention feature M_c and the coarse granularity feature map F of the input blue algae image to generate input features required by the spatial attention module.
2.2 The coarse granularity characteristic diagram F of the blue algae image output by the channel attention module is used as an input characteristic diagram of the space attention module. Firstly, carrying out global maximum pooling and global average pooling based on channels to obtain two H multiplied by W multiplied by 1 feature graphs, and then carrying out channel splicing operation on the 2 feature graphs based on the channels. Then, a 7×7 convolution operation is performed to reduce the dimension to 1 channel, i.e., h×w×1. The spatial attention features, i.e., M_s, are then generated via sigmoid. And finally multiplying the spatial attention feature M_s with the input feature of the module to obtain a generated feature.
Further, in step (3), the feature functions f A(l,I)∈Rc×M and f B(l,I)∈Rc×N of A, B in the bilinear model are subjected to bilinear fusion multiplication at the same position to obtain an m×n-dimensional matrix b:
Further, in step (4), summation pooling is adopted, matrix b at all positions is accumulated according to the following formula to obtain matrix ζ, and multidimensional vector expansion is performed on matrix ζ to obtain feature vector x:
ξ(I)＝∑lb(l,I,fA,fB)；x＝vec(ξ(I))
The beneficial effects are that: compared with the prior art, the blue algae identification method based on deep learning improves a basic feature extraction network aiming at the characteristic of identifying more dependent colors in blue algae image identification, increases the convolution kernel number of a convolution layer, and simultaneously improves the operation efficiency and reduces part of the convolution layer in order to accelerate the training speed. In addition, the method uses the bilinear model to effectively extract the slight difference among blue algae types, combines the attention module in the network, effectively extracts valuable characteristic information, fully plays the characteristic extraction advantages of the bilinear model and the attention mechanism on the fine-grained image, achieves higher blue algae identification precision, enhances the robustness of the network model, and effectively completes the identification of the blue algae image.
Drawings
FIG. 1 is a schematic diagram of a model structure used in the method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method according to an embodiment of the invention.
Detailed Description
The present application is further illustrated below in conjunction with specific embodiments, it being understood that these embodiments are meant to be illustrative of the application and not limiting the scope of the application, and that modifications of the application, which are equivalent to those skilled in the art to which the application pertains, fall within the scope of the application defined in the appended claims after reading the application.
As shown in FIG. 1, the blue algae image recognition method based on deep learning comprises the following steps of using a blue algae image recognition model BA-ResNet-16:
(1) Carrying out coarse-granularity feature extraction by utilizing an improvement ResNet based on a bilinear network to obtain a coarse-granularity feature map of the blue algae image;
(2) Adding an attention mechanism module CBAM of a convolution module after ResNet, and obtaining important characteristic information in the blue algae image through learning weight to obtain enhanced fine granularity characteristics;
(3) Carrying out bilinear fusion on fine-grained features extracted by A, B feature extraction functions in the bilinear model to obtain an M multiplied by N matrix b;
(4) And (3) carrying out summation pooling on the matrix b obtained in the step (3) to obtain a matrix xi, carrying out vector expansion recombination on the matrix xi to obtain a feature vector x, and predicting the bilinear vector x by using a classification function.
In the step (1), the feature extractor is composed of a modified ResNet, and can perform feature extraction of blue algae image coarse granularity. The modified ResNet consists of 15 convolutional layers and a maximum pooling layer, and all the activation units of the hidden layers use Relu function. The improved ResNet model refers to a VGG19 network, structural modification is performed on the basis of the model, and a residual error unit is added into the network through a short circuit mechanism. The variation is mainly reflected in ResNet directly downsampling using a convolution of step size 2. The residual unit can be expressed as: y l＝h(xl)+F(xl,Wl),xl+1＝f(yl). Where x l and x l+1 represent the input and output, respectively, of the first residual unit, each residual unit typically comprising a multi-layer structure. F is a residual function representing the learned residual, and h (x l)＝xl represents identity mapping, and F is a ReLU activation function. In this embodiment, resNet is improved, firstly, because blue algae are identified more depending on color information, we increase the convolution kernel number of the convolution Layer, the convolution kernel number of the first Layer is increased from 64 to 128 in the original network, secondly, in order to increase training speed, adjust the network structure, reduce part of the convolution Layer of Layer1 to two layers, and reduce the overall Layer number of the network from 18 layers to 16.
The network is composed of five parts:
1) The first part consists of one convolution layer, the convolution kernel size 7*7, the number of channels 128.
2) The second part is a max pooling operation.
3) The third part is a basic res block with two convolutions, a convolution kernel size 3*3, a number of channels 128.
4) The fourth part consists of three layers of similar structure, each layer consists of a basic res block with two-layer convolution and a downsampled res block with two-layer convolution, the number of channels is 256, 512, 1024, the convolution kernel size is 3*3, and the pixel fill is 1 pixel point.
5) The fifth part is the average pooling operation. The improved network architecture is shown in the following table:
TABLE 1
All Res blocks are connected through ReLU activation calculation, gradient calculation and back propagation can be effectively performed by adopting a ReLU activation function while calculation is simplified, the expression of the gradient elimination and gradient explosion ReLU activation function is a function of activation functions running on neurons of an artificial neural network, the function is responsible for mapping inputs of the neurons to output ends, the activation function is used for adding nonlinear factors, the expression capacity of the neural network to a model is improved, and the problem that a linear model cannot solve can be solved. In a neural network, linear rectification is used as an activation function of a neuron, which defines a nonlinear output result of the neuron after linear transformation w T x+b, that is, for an input vector x of the neuron from a neural network of a previous layer, the neuron using the linear rectification activation function outputs max (0,w T x+b) to an output of the neuron of a next layer, the definition of the ReLU activation function is f (x) =max (0, x), the whole feature extraction function is defined as f (·), and f A and f B correspond to be f (·).
In the step (2), the coarse-granularity feature map F (hxw×c) of the extracted blue algae image is subjected to global maximum pooling and global average pooling based on width and height, respectively, to obtain two channel attention feature maps of 1×1×c. The attention weight beta= [ beta 1,…,βc ] of the channel is obtained by calculating the average value of the corresponding feature map of each channel in the coarse granularity feature map F of the blue algae image through a global average pooling method, wherein the attention weight beta c of the channel c is obtained by calculating through the average pooling method as follows:
Wherein F c∈RH×W represents a feature map corresponding to a channel c of the coarse-granularity feature matrix F of the blue-green algae image, and F c (i, j) represents a point corresponding to a position (i, j) on the coarse-granularity feature map of the blue-green algae image.
Calculating the maximum value in the blue algae matrix graph corresponding to each channel through a global maximum pooling method, and obtaining attention weights alpha= [ alpha 1,…,αc ] of all channels, wherein the weights alpha c corresponding to the channel c are as follows:
αc＝max({Fc(i,j),i∈[1,H],j∈[1,W]})
Wherein F c∈RH×W represents a feature map corresponding to channel c of the coarse-granularity feature matrix F of the blue algae image, and F c (i, j) represents a point corresponding to the (i, j) position on the feature map.
Then, the two attention weights are respectively sent to a multi-layer sensor of a two-layer neural network. And then, adding the characteristics output by the multi-layer perceptron on the basis of element layers, and generating a final channel attention characteristic M_c through sigmoid activation operation. And finally, performing element level multiplication operation on the M_c and the input feature map F to generate input features required by the spatial attention module.
And then taking the coarse granularity characteristic diagram F of the blue algae image output by the channel attention module as an input characteristic diagram of the space attention module. Firstly, carrying out global maximum pooling and global average pooling based on channels to obtain two H multiplied by W multiplied by 1 feature graphs, and then carrying out channel splicing operation on the 2 feature graphs based on the channels. Then, a 7×7 convolution operation is performed to reduce the dimension to 1 channel, i.e., h×w×1. The spatial attention features, i.e., M_s, are then generated via sigmoid. Finally, multiplying the spatial attention characteristic and an input characteristic diagram of the module to obtain the generated blue algae characteristic;
in the step (3), the feature functions F A(l,I)∈Rc×W and F B(l,I)∈Rc×N are subjected to bilinear fusion multiplication at the same position to obtain an m×n-dimensional matrix b:
f A、fB is a feature function for mapping Cheng Wei features of the image I with a position l, and feature dimensions extracted by the two feature functions f A、fB are c×m and c×n, respectively, and then the output b of the pooling function P will be a matrix of m×n, where the position I is a position in the image, and x ij represents a feature value of an ith row and a jth column in the image.
In the step (4), summation pooling is adopted, matrix b at all positions is accumulated and summed to obtain matrix xi according to the following formula, and multidimensional vector expansion is carried out on the matrix xi to obtain matrix vector x:
ξ(I)＝∑lb(l,I,fA,fB)；x＝vec(ξ(I))
and finally, the vector x is sent to a full-connection layer to classify blue algae by using a softmax function. Blue algae identification is used as a multi-classification problem, the output of the blue algae identification is a plurality of categories, and the embodiment uses a softmax function at an output layer, so that each output is output in a probability form, and the highest-scoring category in all the outputs is the most probable category. The Softmax function is calculated as follows:
Where V i represents the ith element in the array, here the input of the predictive tag, and y i represents the probability output after Softmax regression. If blue algae, granular, band-shaped and sheet-shaped labels are finally obtained, the corresponding input is [1,2,3,4], and after the regression treatment of softmax, the blue algae, the granular, the band-shaped and the sheet-shaped labels are obtained:
The simplification can be obtained:
And if the probability output corresponding to the sheet-shaped label is 0.64 maximum, the blue algae image to be identified is processed by the softmax classification function to obtain a prediction label which is sheet-shaped after model feature extraction.
As shown in fig. 2, a flow chart of a blue algae image recognition method based on deep learning is provided.
The parameters of blue algae image recognition models BA-ResNet-16 obtained through the steps (1), (2), (3) and (4) are optimized by using a grid search method, the model network comprises two basic feature extraction networks ResNet-16 and a full connection layer, wherein each feature extraction network comprises four basic residual error units (Basic Residual Block), three downsampling residual error units (Down Residual Block) and two attention modules (CBAM), and the specific structure is shown in figure 1.
The experimental environment of this embodiment is: the operating system is Windows 10; the CPU is AMD Ryren 5 4600H with Radeon Graphics, and the memory is 16G; the deep learning frame is PyTorch; code is written using the Python 3.8.6 development language. The specific software and hardware information is shown in table 2. On this basis, parameters related to the process of initializing the model include batch_size, learning rate, activation function, optimization function and iteration number, and the training parameter settings of the BA-ResNet-16 model are shown in table 3.
TABLE 2
TABLE 3 Table 3
Parameters (parameters) Batch_size Learning rate Activation function Optimizing functions Number of iterations
Parameter value 32 0.01 Relu SGD 30
(2) After the input data is transmitted to the blue algae image recognition model, the output characteristics of the characteristic extraction network are obtained;
In this embodiment, 20000 blue algae image data is used as the original data, but for the task of image recognition, the original image is insufficient, the image is classified and labeled, and the image needs to be screened to obtain the image with training value in consideration of the problems of light, angle and the like. In addition, in order to increase the size of the training sample and improve the generalization capability of the model, enhancement processing is carried out on the classified and screened data, and data enhancement means comprise methods of scaling, rotation, brightness conversion and the like, so that 30000 images are finally obtained. The specific data set structure is shown in table 4 with 70% of the image data in the data set as the training set, 20% of the image as the validation set, and 10% of the image as the test set.
TABLE 4 Table 4
(3) Training a convolutional neural network model according to the acquired characteristics;
Inputting a blue algae image into the blue algae image recognition model BA-ResNet-16, and extracting coarse-granularity characteristics by utilizing ResNet based on bilinear network improvement to obtain a coarse-granularity characteristic diagram of the blue algae image; adding a attention mechanism module CBAM of a convolution module after ResNet, and obtaining important blue algae characteristic information in the image through learning weights to obtain enhanced fine granularity characteristics; carrying out bilinear fusion on fine-grained features extracted by A, B feature extraction functions in the bilinear model to obtain an M multiplied by N matrix b; and carrying out summation pooling on the obtained matrix b to obtain a matrix xi, carrying out vector expansion recombination on the matrix xi to obtain a feature vector x, and finally predicting the bilinear vector x by using a classification function.
And then judging the difference between the prediction result and the true category of the input blue algae image data, and updating the model parameters of the convolutional neural network. In our example, blue algae identification is a multi-classification problem, whose output is 4 categories, expressed in terms of probability. And using a softmax function at an output layer, wherein the highest score in all outputs is the most possible blue algae category. After obtaining the prediction classification result of blue algae, using Cross entropy (Cross-entropy) as the loss function of the embodiment, and updating parameters of the network through an inverse error algorithm. In the inverse error algorithm, for each training sample, the relevant input samples are provided to the input layer neurons first, and the signals are passed forward layer by layer until the output layer produces the output value. And then according to the output errors, the errors are reversely propagated to neurons of the hidden layer, and finally, the connected weight and the threshold value of the neurons are adjusted according to the errors calculated by the neurons of the hidden layer. The algorithm iterates the loop to perform the above steps until the training stops.
(4) And carrying out prediction and identification of input data according to the trained convolutional neural network model, and outputting a prediction result.
The input data in the step is blue algae image data which needs to be identified, and the output result is blue algae type.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (3)

1. The blue algae image recognition method based on deep learning is characterized by comprising the following steps of:
(1) Carrying out coarse-granularity feature extraction by utilizing ResNet based on bilinear network improvement to obtain a coarse-granularity feature map of the blue algae image;
(2) Adding an attention mechanism module of a convolution module after ResNet, and obtaining important blue algae characteristic information in the image through learning weights to obtain enhanced fine granularity characteristics;
(3) Carrying out bilinear fusion on blue algae fine granularity image features extracted by A, B feature extraction functions in the bilinear model to obtain an M multiplied by N matrix b;
(4) Carrying out summation pooling on the matrix b obtained in the step (3) to obtain a matrix xi, carrying out vector expansion recombination on the matrix xi to obtain a feature vector x, and predicting the bilinear vector x by using a classification function;
In the step (1), feature extraction is performed by using a bilinear network-based improvement ResNet, so that a coarse-granularity feature map of the blue algae image is obtained, and compared with the prototype network ResNet, the improved ResNet increases the convolution kernel number of the convolution layer and reduces part of the convolution layer;
The improved ResNet network is based on the prototype network ResNet, the convolution kernel number of the first layer of convolution layer is changed from 64 to 128, the number of convolution layers after the maximum pooling operation in ResNet is reduced, and the original 4 layers are reduced to 2 layers; in the modified ResNet, simplified computation is performed using a ReLU activation function, downsampling is performed using a convolution kernel of step size 2, the convolution kernel size of the convolution layer being 3×3; the residual unit is expressed as: y l＝h(xl)+F(xl,Wl),xl+1＝f(yl); wherein x l and x l+1 represent the input and output, respectively, of the first residual unit, each residual unit generally comprising a multi-layer structure; f is a residual function representing the learned residual, and h (x l)＝xl represents identity mapping, F is a ReLU activation function;
In step (2), inputting the obtained coarse-granularity feature map into a channel attention module and a space attention module of an attention mechanism module to obtain channel attention and space attention respectively, and weighting the obtained channel attention and space attention to the coarse-granularity feature map output in step (1) respectively to obtain a new fine-granularity feature map, wherein the specific steps are as follows:
2.1 The coarse granularity characteristic diagram F with the size of H multiplied by W multiplied by C output in the step (1) is subjected to global maximum pooling and global average pooling based on width and height respectively to obtain two channel attention characteristic diagrams with the size of 1 multiplied by C, and then the channel attention characteristic diagrams are respectively sent to a two-layer multi-layer perceptron; then, adding the characteristics output by the multi-layer perceptron on the basis of element layers, and generating a final channel attention characteristic M_c through sigmoid activation operation; finally, carrying out element level multiplication operation on the channel attention feature M_c and a coarse granularity feature map F of the input blue algae image to generate input features required by the spatial attention module;
2.2 Taking the coarse granularity characteristic diagram F of the blue algae image output by the channel attention module as an input characteristic diagram of the space attention module; firstly, carrying out global maximum pooling and global average pooling based on channels to obtain two H multiplied by W multiplied by 1 feature graphs, and then carrying out channel splicing operation on the 2 feature graphs based on the channels; then through a 7×7 convolution operation, the dimension is reduced to 1 channel, namely H×W×1; generating a spatial attention feature, namely M_s, through sigmoid; and finally multiplying the spatial attention feature M_s with the input feature of the module to obtain a generated feature.
2. The blue algae image recognition method based on deep learning according to claim 1, wherein in the step (3), the feature functions f A(l,I)∈Rc×M and f B(l,I)∈Rc×N of A, B in the bilinear model are subjected to bilinear fusion multiplication at the same position to obtain an m×n-dimensional matrix b:
3. The blue algae image recognition method based on deep learning according to claim 1, wherein in the step (4), summation pooling is adopted, matrix b of all positions is accumulated according to the following formula to obtain matrix ζ, and multidimensional vector expansion is performed on matrix ζ to obtain feature vector x:
ξ(I)＝∑lb(l,I,fA,fB)；x＝vec(ξ(I))。
CN202210318599.8A 2022-03-29 2022-03-29 Blue algae image recognition method based on deep learning Active CN114722928B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210318599.8A CN114722928B (en) 2022-03-29 2022-03-29 Blue algae image recognition method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210318599.8A CN114722928B (en) 2022-03-29 2022-03-29 Blue algae image recognition method based on deep learning

Publications (2)

Publication Number Publication Date
CN114722928A CN114722928A (en) 2022-07-08
CN114722928B true CN114722928B (en) 2024-04-16

Family

ID=82239080

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210318599.8A Active CN114722928B (en) 2022-03-29 2022-03-29 Blue algae image recognition method based on deep learning

Country Status (1)

Country Link
CN (1) CN114722928B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115115940A (en) * 2022-08-30 2022-09-27 中水三立数据技术股份有限公司 Blue algae bloom monitoring method and monitoring and early warning system thereof

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263863A (en) * 2019-06-24 2019-09-20 南京农业大学 Fine granularity mushroom phenotype recognition methods based on transfer learning Yu bilinearity InceptionResNetV2
CN113128593A (en) * 2021-04-20 2021-07-16 南京林业大学 Plant fine-grained identification method based on bilinear convolutional neural network
CN114067153A (en) * 2021-11-02 2022-02-18 暨南大学 Image classification method and system based on parallel double-attention light-weight residual error network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263863A (en) * 2019-06-24 2019-09-20 南京农业大学 Fine granularity mushroom phenotype recognition methods based on transfer learning Yu bilinearity InceptionResNetV2
CN113128593A (en) * 2021-04-20 2021-07-16 南京林业大学 Plant fine-grained identification method based on bilinear convolutional neural network
CN114067153A (en) * 2021-11-02 2022-02-18 暨南大学 Image classification method and system based on parallel double-attention light-weight residual error network

Also Published As

Publication number Publication date
CN114722928A (en) 2022-07-08

Similar Documents

Publication Publication Date Title
CN111368886B (en) Sample screening-based label-free vehicle picture classification method
Zhang et al. Asymmetric cross-attention hierarchical network based on CNN and transformer for bitemporal remote sensing images change detection
CN112132149B (en) Semantic segmentation method and device for remote sensing image
Pan et al. Intelligent diagnosis of northern corn leaf blight with deep learning model
CN111259827A (en) Automatic detection method and device for water surface floating objects for urban river supervision
Liang et al. Environmental microorganism classification using optimized deep learning model
CN114722928B (en) Blue algae image recognition method based on deep learning
CN112149612A (en) Marine organism recognition system and recognition method based on deep neural network
CN115546196A (en) Knowledge distillation-based lightweight remote sensing image change detection method
Yang et al. Automatic identification of harmful algae based on multiple convolutional neural networks and transfer learning
Albelwi Deep architecture based on DenseNet-121 model for weather image recognition
CN111242028A (en) Remote sensing image ground object segmentation method based on U-Net
CN114399642A (en) Convolutional neural network fluorescence spectrum feature extraction method
Zhang et al. Multi-input multi-output temporal convolutional network for predicting the long-term water quality of ocean ranches
CN114170154A (en) Remote sensing VHR image change detection method based on Transformer
Leontyev et al. Application of assimilation and filtration methods for satellite water sensing data for plankton population evolution processes predictive modeling
CN117372853A (en) Underwater target detection algorithm based on image enhancement and attention mechanism
Liu et al. Plant intelligence-based PILLO underwater target detection algorithm
CN117011640A (en) Model distillation real-time target detection method and device based on pseudo tag filtering
CN116824352A (en) Water surface floater identification method based on semantic segmentation and image anomaly detection
CN113343861B (en) Remote sensing image water body region extraction method based on neural network model
CN115761256A (en) Hyperspectral image waveband selection method based on depth multi-level representation learning
Zhao et al. Recognition and Classification of Concrete Cracks under Strong Interference Based on Convolutional Neural Network.
Babu et al. Water Quality Prediction using Neural Networks
CN115272865A (en) Target detection method based on adaptive activation function and attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant