CN116091763A - Apple leaf disease image semantic segmentation system, segmentation method, device and medium - Google Patents
Apple leaf disease image semantic segmentation system, segmentation method, device and medium Download PDFInfo
- Publication number
- CN116091763A CN116091763A CN202211171326.1A CN202211171326A CN116091763A CN 116091763 A CN116091763 A CN 116091763A CN 202211171326 A CN202211171326 A CN 202211171326A CN 116091763 A CN116091763 A CN 116091763A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- apple leaf
- semantic segmentation
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 108
- 201000010099 disease Diseases 0.000 title claims abstract description 99
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 99
- 241000219998 Philenoptera violacea Species 0.000 title claims abstract description 97
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 43
- 238000012795 verification Methods 0.000 claims abstract description 24
- 230000004927 fusion Effects 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 238000012360 testing method Methods 0.000 claims abstract description 14
- 238000005070 sampling Methods 0.000 claims abstract description 12
- 238000002372 labelling Methods 0.000 claims abstract description 11
- 238000010586 diagram Methods 0.000 claims description 38
- 238000010606 normalization Methods 0.000 claims description 13
- 238000003860 storage Methods 0.000 claims description 10
- 230000003416 augmentation Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 5
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000010200 validation analysis Methods 0.000 claims description 3
- 235000013399 edible fruits Nutrition 0.000 claims description 2
- 238000003709 image segmentation Methods 0.000 claims description 2
- 230000004044 response Effects 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 6
- 238000010276 construction Methods 0.000 description 5
- 241000700605 Viruses Species 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 241000894006 Bacteria Species 0.000 description 1
- 241000607479 Yersinia pestis Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an apple leaf disease image semantic segmentation system, a segmentation method, equipment and a medium, wherein the method comprises the following steps: step 1, collecting an apple leaf disease image; step 2, manually labeling the acquired image, dividing the acquired image into a training verification set and a test set, and processing the training verification set by using a preprocessing module; step 3, adopting a lightweight network RegNetY-600MF as an encoder to encode the training set image; step 4, decoding the training set image by using the multi-level feature fusion network as a decoder; step 5, comparing the semantic segmentation characteristic value with the labeling characteristic value, and training by using a back propagation algorithm to obtain an apple leaf disease semantic segmentation model; and 6, inputting the image to be tested into a semantic segmentation model after preprocessing to obtain segmentation characteristic values, and performing up-sampling to obtain an apple leaf disease semantic segmentation image. The invention not only ensures the semantic segmentation precision, but also quickens the segmentation speed and enhances the real-time response capability in application.
Description
Technical Field
The invention relates to the technical field of image semantic segmentation, in particular to an apple leaf disease image semantic segmentation system and method, computer equipment and storage medium.
Background
After the apple plants are affected by bacteria and viruses, most of the apple plants have certain characteristics on leaves, such as various disease spots, mould breeding, local dryness and the like. The influence of the complex background and the similarity of the lesions causes great difficulty in identifying and dividing the apple leaf lesions. The disease type and degree of crops can be rapidly judged, and non-professional personnel can be helped to find the disease condition in time and take medicine according to symptoms, so that economic loss is reduced. Therefore, how to quickly and accurately divide the apple leaf disease area is of great importance.
The existing apple leaf disease identification method depends on the expertise to evaluate through visible characteristics, so that the expertise is not only required to have abundant experience, but also the evaluation process is complicated and time-consuming, has a certain subjectivity, and is difficult to meet the application requirements of large-area and rapid pest and disease monitoring in actual production. In deep learning which has received extensive attention in recent years, semantic segmentation can realize the classification of semantic information labels of pixels in an image to be predicted, so that a refined segmentation result of the predicted image is obtained, and the defect of manual diagnosis can be reduced by carrying out apple leaf disease identification by using a semantic segmentation method.
The current semantic segmentation network Fcn, unet, segNet, PSPNet, deep Lab v3+ and the like have large calculated amount and operation occupied memory, restrict the network segmentation speed, only consider the segmentation precision and can not meet the real-time response of the network, but apple virus segmentation not only needs to meet the precision of the semantic segmentation network, but also ensures the real-time performance of the algorithm, requires the semantic segmentation algorithm to have high real-time processing speed and has outstanding response capability.
Disclosure of Invention
The first object of the invention is to provide an apple leaf disease image semantic segmentation system, which uses a lightweight network RegNetY-600MF as an encoder and a multistage feature fusion network as a decoder, and fuses spatial information and semantic information in a feature map to obtain segmented objects with rich features. The apple leaf disease image semantic segmentation method has high recognition precision, and simultaneously uses the low convolution layer number and the low channel number to reach the low network parameter number so as to accelerate the response speed of the segmentation algorithm and improve the real-time processing capability in application.
The second aim of the invention is to provide a semantic segmentation method for apple leaf disease images. The invention can be used for carrying out semantic segmentation on the apple leaf disease image, not only can accurately segment the background and disease area of the apple leaf disease image in a complex environment, but also can ensure the real-time response in application, and improves the reliability of the semantic segmentation of the apple leaf disease image.
A third object of the present invention is to provide a computer device.
A fourth object of the present invention is to provide a computer-readable storage medium.
The first object of the present invention can be achieved by adopting the following technical scheme:
an apple leaf disease image semantic segmentation system comprising:
and a pretreatment module: the method is used for carrying out random noise adding, random scaling, random mirroring, random clipping, random splicing and normalization treatment on the manually marked apple leaf disease image;
an encoder: the method is used for encoding the preprocessed apple leaf images into output characteristic diagrams with the sizes of 1/4, 1/8 and 1/16 of the input sizes.
A decoder: the method is used for carrying out multi-scale feature fusion on three output feature images of the encoder, classifying the obtained feature image elements, and outputting apple leaf disease segmentation images by utilizing up-sampling to restore the feature image size so as to realize semantic segmentation of the apple leaf disease images.
The second object of the invention can be achieved by adopting the following technical scheme:
the semantic segmentation method for the apple leaf disease image comprises the following steps:
step 1: collecting an apple leaf disease image by using a collecting device;
step 2: manually labeling the acquired images, dividing the acquired images into a training verification set and a test set, inputting the images of the training verification set into a pre-training module for preprocessing and data enhancement processing, dividing the processed images into the training set and the verification set, and performing scaling and normalization processing on the images in the test set;
step 3: and inputting the training set apple leaf images processed by the preprocessing module into an encoder, and adopting a lightweight network RegNetY-600MF to encode the apple leaf disease training set images by the encoder to obtain three output characteristic images, and transmitting the three output characteristic images to a decoder.
Step 4: and respectively acquiring semantic information and spatial information in the three output feature images by using a multistage feature fusion network as a decoder, carrying out multi-scale feature fusion on the three output feature images to obtain the decoder output feature images, and processing the decoder output feature images by using a convolution classifier to obtain semantic segmentation feature values.
Step 5: comparing the semantic segmentation characteristic value with the labeling characteristic value pixel by pixel, and obtaining an apple leaf disease semantic segmentation model after performing network training by using a back propagation algorithm;
step 6: and (3) inputting the apple leaf disease image to be tested into a semantic segmentation model after scaling and normalization treatment to obtain a semantic segmentation characteristic value of the image to be tested, and performing up-sampling operation on the characteristic value to obtain the apple leaf disease semantic segmentation image.
Further, the step 2 of preprocessing samples in the training verification set by using a preprocessing module to form a training set and a verification set specifically includes:
and dividing the sample image of the apple leaf diseases and the corresponding label sample image into a training verification set and a test set.
The resolution ratio of the sample images in the training verification set and the corresponding label sample images is adjusted to a preset size;
the image augmentation mode comprises scaling, cutting, overturning, noise adding, splicing and the like, and the image augmentation processing and normalization processing are carried out on the sample images in the training verification set and the corresponding label sample images according to the probability of 0.5 probability of each image augmentation mode.
According to 8: the scale of 2 divides the color sample image and the corresponding label sample image into a training set and a validation set.
Further, the encoder in the step 3 uses a lightweight regnet-600 MF network. The regnet-600 MF network has the advantages of being rich in semantic information, accurate and small in model calculation amount, and can be used as an encoder to acquire the semantic information of the apple leaf disease image in a short time.
The regnet-600 MF network starts with a 3 x 3 convolution with 32 output channels and a step size of 2. The sequence then ends with a 48-channel D module of 1/4 resolution, 3 128-channel D modules of 1/8 resolution, 2 256-channel D modules of 1/16 resolution, 11 256-channel D modules of 1/16 resolution, and finally with a 320-channel D module of 1/16 resolution.
The structure of the D module is as follows: each D block is added to the output after one convolution, one group convolution, and one convolution in sequence, each convolution followed by a batch norm and ReLU operation.
The three output characteristic diagrams are respectively as follows: the first layer of output characteristic diagram is 1/4 of the resolution of the original image, and the number of channels is 128; the second layer of output characteristic diagram is 1/8 of the resolution of the original image, and the number of channels is 256; the third layer output characteristic diagram is 1/16 of the original image resolution, and the channel number is 320.
Further, the step 4 specifically includes the following steps:
and step 41, respectively acquiring semantic information and spatial information in the three output feature images by using a multi-level feature fusion network as a decoder, and carrying out multi-scale feature fusion on the semantic information and the spatial information to obtain the decoder output feature images.
The structure of the multistage feature fusion network is as follows: the 1/16, 1/8, 1/4 resolution feature maps are convolved with a 128-channel respectively. The convolved 1/16 resolution characteristic diagram and the convolved 1/8 resolution characteristic diagram pass through an SEB module, and the convolved 1/8 resolution characteristic diagram and the convolved 1/4 resolution characteristic diagram pass through the SEB module. Up-sampling the 1/16 resolution characteristic diagram through the SEB module through the RCU module and adding the 1/8 resolution characteristic diagram through the SEB module; the added 1/8 resolution feature map is added with the 1/4 resolution feature map through the SEB module through the RCU module, and the added 1/8 resolution feature map is output from the decoder through the RCU module.
The construction process of the SEB module comprises the following steps: the input of the SEB module comprises two adjacent encoder output feature maps, wherein the low-resolution feature map changes the channel number into the channel number of the high-resolution feature map through convolution operation with the convolution kernel size, the resolution of the low-resolution feature map is restored to be 2 times of that of the original feature map through upsampling operation, and pixel-by-pixel addition operation is carried out on the restored feature map and the high-resolution feature map to obtain the SEB module output feature map.
The construction process of the RCU module is as follows: each input path of the RCU module sequentially passes through two residual convolution units, the first of which is convolved. The second convolution is followed by the BatchNorm and Prelu activation functions, except for all convolutions after the last convolution.
And 42, classifying pixels of the feature image output by the decoder by using a convolution classifier, and outputting an apple leaf disease segmentation image by utilizing up-sampling to restore the feature image size so as to realize semantic segmentation of the apple leaf disease image.
Further, in the step 5, the back propagation algorithm uses a cross entropy loss function, the optimizer selects SGD, and trains for 50 total iterations, the first 25 iterations use an initial learning rate of 0.001, the last 25 iterations use an initial learning rate of 0.0001, and the learning rate attenuation policy is an exponential attenuation policy with a value of 0.98.
The third object of the present invention can be achieved by adopting the following technical scheme:
a computer device comprising a processor and a memory, the processor and the memory being communicable;
a memory for storing a control program;
and the processor is used for realizing the steps of the method when executing the program stored in the memory.
The fourth object of the present invention can be achieved by adopting the following technical scheme:
a computer readable storage medium, wherein a control program is stored in the computer readable storage medium, and the control program realizes the steps of the apple leaf disease image semantic segmentation method when being executed by a processor.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a semantic segmentation method with higher efficiency and better real-time performance, which takes a lightweight network RegNetY-600MF as an encoder, takes a multistage feature fusion network as a decoder, fuses semantic information and space information in a feature map to obtain segmented objects with rich features, and uses a low convolution layer number and a low channel number to achieve a low network parameter quantity so as to accelerate the response speed of a segmentation algorithm. And (3) finely labeling the sample image of the apple leaf disease to generate a label sample image corresponding to the sample image, and preprocessing to obtain an apple leaf disease image data set. Training a data set by using the proposed semantic segmentation method to obtain an apple leaf disease image semantic segmentation model. Through testing, the apple leaf disease image semantic segmentation model ensures the accuracy of leaf disease region segmentation in a semantic segmentation task, accelerates the segmentation speed and enhances the response speed in practical application.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an apple leaf disease image semantic segmentation method according to an embodiment of the invention.
Fig. 2 is a sample image of apple leaf disease in an embodiment of the present invention.
Fig. 3 is a label sample image corresponding to an apple leaf disease sample image according to an embodiment of the present invention.
Fig. 4 is a structural block diagram of an apple leaf disease image semantic segmentation system according to an embodiment of the invention.
Fig. 5 is a block diagram of the structure of a D module in the apple leaf disease image semantic segmentation encoder according to an embodiment of the present invention.
Fig. 6 is a structural diagram of an apple leaf disease image semantic segmentation decoder according to an embodiment of the present invention.
Fig. 7 is a diagram illustrating a process of constructing an SEB module in a decoder architecture according to an embodiment of the present invention.
Fig. 8 is a construction process of an RCU module in a decoder architecture according to an embodiment of the present invention.
Fig. 9 shows the semantic segmentation effect of different algorithms according to an embodiment of the present invention on an apple leaf disease image dataset.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.
The apple leaf disease image semantic segmentation system comprises a preprocessing module, an encoder, a decoder and a convolution classifier which are sequentially connected, wherein the preprocessing module is used for carrying out random noise addition, random scaling, random mirroring, random cutting, random splicing and normalization processing on apple leaf disease samples; the encoder is used for encoding the preprocessed apple leaf images into output characteristic diagrams with the sizes of 1/4, 1/8 and 1/16 of the input size; the decoder is used for generating an apple leaf disease segmentation image, and performing multi-scale feature fusion on three output feature images of the encoder to obtain an output feature image of the decoder; the convolution classifier classifies pixels of the feature image output by the decoder, and the feature image size is restored by up-sampling to output apple leaf disease segmentation images so as to realize semantic segmentation of the apple leaf disease images.
Examples
As shown in fig. 1, the embodiment provides a semantic segmentation method for apple leaf disease images, which comprises the following steps:
step 1, acquiring an apple leaf disease image by using an acquisition device.
In this embodiment, a sample image (RGB sample image) of the apple leaf disease is obtained first, and may be obtained by using an acquisition device, for example, by using a camera to acquire a sample image of the apple leaf disease, or may refer to a sample image of a related database, for example, a color sample image of the apple leaf disease is obtained by searching from a hundred-degree database.
And 2, manually labeling the acquired images, dividing the acquired images into a training verification set and a test set, inputting the images of the training verification set into a pre-training module for preprocessing and data enhancement processing, dividing the processed images into the training set and the verification set, and performing scaling and normalization processing on the images in the test set.
After the sample image of the apple leaf disease is obtained, fine labeling can be performed manually, specifically, a user selects the sample image, labels a target area, then a labeling command is stored, and the sample image of the apple leaf disease is labeled with pixel types corresponding to the disease area and the background, so that a label sample image corresponding to the sample image is generated, namely, the label sample image is provided with a pixel type label corresponding to the disease area and the background.
Fig. 2 shows a sample image of apple fruit virus and fig. 3 shows a corresponding sample image of label.
And dividing the sample image of the apple leaf diseases and the corresponding label sample image into a training verification set and a test set.
The resolutions of sample images in the training verification set and the test set and corresponding label sample images are adjusted to a preset size;
the image augmentation mode comprises scaling, cutting, overturning, noise adding, splicing and the like, the image augmentation processing and the normalization processing are carried out on the sample images in the training verification set and the corresponding label sample images according to the probability of 0.5 probability of each image augmentation mode, and the normalization processing is carried out on the test set images.
According to 8: the scale of 2 divides the color sample image and the corresponding label sample image into a training set and a validation set.
And 3, as shown in fig. 4, inputting the training set apple leaf images processed by the preprocessing module into an encoder, and adopting a lightweight network RegNetY-600MF to encode the apple leaf disease training set images by the encoder to obtain three output characteristic diagrams, and transmitting the three output characteristic diagrams into a decoder.
The regnet-600 MF network starts with a 3 x 3 convolution with 32 output channels and a step size of 2. The sequence then ends with a 48-channel D module of 1/4 resolution, 3 128-channel D modules of 1/8 resolution, 2 256-channel D modules of 1/16 resolution, 11 256-channel D modules of 1/16 resolution, and finally with a 320-channel D module of 1/16 resolution. The specific parameters are shown in table 1 below:
TABLE 1 encoder RegNetY-600MF internal structure parameter map
The D module is a residual structure introducing group convolution and cavity convolution, and reduces the calculated amount of the model while increasing the receptive field. As shown in fig. 5, each D module sequentially performs one convolution, one 3×3 group convolution and one convolution, and then adds the convolutions to the output, where each convolution is followed by a batch norm and a ReLU operation, where fig. 5a is a D module with a step size of 1, and fig. 5b is a D module with a step size of 2; the apple leaf disease semantic segmentation model is balanced in terms of semantic segmentation accuracy and response speed by means of the addition of the D module.
The sizes of the input apple leaf disease images are that three output characteristic images are respectively: the first layer output characteristic diagram is 1/4 of the resolution of the original image (namely, the number of channels is 128; the second layer of output characteristic diagram is 1/8 of the resolution of the original image (namely, the number of channels is 256); the third layer output feature map is 1/16 of the original image resolution (i.e., the channel number is 320). And the three output characteristic diagrams obtained by the coding operation are transmitted to a decoder to finish the decoding operation.
And 4, as shown in fig. 6, using a multi-level feature fusion network as a decoder to respectively acquire semantic information and spatial information in the three output feature images, performing multi-scale feature fusion on the three output feature images to obtain an output feature image of the decoder, and processing the output feature image by a convolution classifier to obtain semantic segmentation feature values.
Further, the step 4 specifically includes the following steps:
and step 41, respectively acquiring semantic information and spatial information in the three output feature images by using a multi-level feature fusion network as a decoder, and carrying out multi-scale feature fusion on the semantic information and the spatial information to obtain the decoder output feature images.
The structure of the multistage feature fusion network is as follows: the 1/16, 1/8, 1/4 resolution feature maps are convolved with a 128-channel respectively. The convolved 1/16 resolution characteristic diagram and the convolved 1/8 resolution characteristic diagram pass through an SEB module, and the convolved 1/8 resolution characteristic diagram and the convolved 1/4 resolution characteristic diagram pass through the SEB module. Up-sampling the 1/16 resolution characteristic diagram through the SEB module through the RCU module and adding the 1/8 resolution characteristic diagram through the SEB module; the added 1/8 resolution feature map is added with the 1/4 resolution feature map through the SEB module through the RCU module, and the added 1/8 resolution feature map is output from the decoder through the RCU module.
As shown in fig. 7, the construction process of the SEB module is as follows: the input of the SEB module comprises two adjacent encoder output feature maps, wherein the low-resolution feature map changes the channel number into the channel number of the high-resolution feature map through convolution operation with the convolution kernel size, the resolution of the low-resolution feature map is restored to be 2 times of that of the original feature map through upsampling operation, and pixel-by-pixel addition operation is carried out on the restored feature map and the high-resolution feature map to obtain the SEB module output feature map.
As shown in fig. 8, the construction process of the RCU module is as follows: each input path of the RCU module sequentially passes through two residual convolution units, the first of which is convolved. The second convolution is followed by the BatchNorm and Prelu activation functions, except for all convolutions after the last convolution.
And 42, classifying pixels of the feature image output by the decoder by using a convolution classifier, and outputting an apple leaf disease segmentation image by utilizing up-sampling to restore the feature image size so as to realize semantic segmentation of the apple leaf disease image.
The convolution classifier uses two convolution operations, and the parameters of the convolution classifier are respectively as follows: the input channels are the number of apple leaf disease sample image segmentation objects, the convolution kernel size is 1, and the activation function selects softmax.
Step 5, comparing the semantic segmentation characteristic value with the labeling characteristic value pixel by pixel, and obtaining an apple leaf disease semantic segmentation model after performing network training by using a back propagation algorithm;
the back propagation algorithm uses a cross entropy loss function, the optimizer selects SGD, training is performed for 50 times in total, the first 25 times use an initial learning rate of 0.001, the last 25 times use an initial learning rate of 0.0001, and the learning rate attenuation strategy is an exponential attenuation strategy with a value of 0.98.
And 6, inputting the apple leaf disease image to be tested into a semantic segmentation model after scaling and normalization treatment to obtain a semantic segmentation characteristic value of the image to be tested, and performing up-sampling operation on the obtained semantic segmentation characteristic value to obtain a semantic segmentation image of the apple leaf disease image to be tested.
The algorithm performance was compared using various semantic segmentation algorithms available and the present example trained and tested on apple leaf disease segmentation datasets, respectively. The evaluation indexes are shown in table 1, wherein the index val mIOU and the index test mIOU are used for evaluating the semantic segmentation precision of the algorithm on the verification set and the test set, and the index parameter is used for evaluating the parameter of the model; the index computing power consumption is used for evaluating computing power required by model operation, and the parameter and computing power consumption indirectly evaluate the semantic segmentation speed of the algorithm, wherein the smaller the parameter and computing power consumption is, the faster the semantic segmentation speed is. As can be seen from the data in table l, the semantic segmentation method in this embodiment not only can ensure the semantic segmentation accuracy, but also greatly reduces the number of model parameters, and can greatly improve the accuracy and the real-time segmentation efficiency of the segmentation of apple leaf diseases on the task of segmentation of apple leaf diseases.
Table 1 shows the comparison of different evaluation indexes of the algorithm on the apple leaf disease segmentation data set
The semantic segmentation results of the existing various semantic segmentation algorithms and the semantic segmentation results of the embodiment in the apple leaf disease dataset are shown in fig. 9, and it can be seen that the semantic segmentation effect obtained by the method is best, the semantic segmentation algorithm is closest to the real label corresponding to the original image, the segmentation label is accurate and has obvious boundary information of the classified objects.
The present embodiment provides a computer device comprising a processor and a memory. The memory is used for storing an apple leaf disease image sample data set and a control program for performing operations such as preprocessing, encoding, decoding and the like on an apple leaf disease image; the processor is used for executing the control program to complete a part of or the whole steps and complete semantic segmentation of the apple leaf disease sample image to be processed; the computer device may communicate with external devices or with a local area network, wide area network, or public network via a network adapter.
The computer readable storage medium of the embodiment is used for storing a control program, and the control program can realize the semantic segmentation of apple leaf disease images through the execution of a processor, and the computer readable storage medium can be a magnetic storage device, a random access storage disk, an optical disk, a U disk, a mobile hard disk, a computer memory and other media. The control program is executed by the processor to realize the steps of the apple leaf disease image semantic segmentation method.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular for system embodiments, the description is relatively simple as it is substantially similar to method embodiments, as relevant to see a section of the description of method embodiments.
The above-mentioned embodiments are only preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can make equivalent substitutions or modifications according to the technical solution and the inventive concept of the present invention within the scope of the present invention disclosed in the present invention patent, and all those skilled in the art belong to the protection scope of the present invention.
Claims (10)
1. An apple leaf disease image semantic segmentation system, the system comprising:
and a pretreatment module: the method is used for carrying out random noise adding, random scaling, random mirroring, random clipping, random splicing and normalization treatment on the manually marked apple leaf disease image;
an encoder: the method comprises the steps of encoding a preprocessed apple leaf image into an output characteristic diagram with the sizes of 1/4, 1/8 and 1/16 of the input size;
a decoder: the method is used for carrying out multi-scale feature fusion on three output feature images of the encoder, classifying the obtained feature image elements, and outputting apple leaf disease segmentation images by utilizing up-sampling to restore the feature image size so as to realize semantic segmentation of the apple leaf disease images.
2. A method for performing apple leaf disease image semantic segmentation using the apple leaf disease image semantic segmentation system according to claim 1, wherein the method comprises the steps of:
step 1: collecting an apple leaf disease image by using a collecting device;
step 2: manually labeling the acquired images, dividing the acquired images into a training verification set and a test set, inputting the images of the training verification set into a pre-training module for preprocessing and data enhancement processing, dividing the processed images into the training set and the verification set, and performing scaling and normalization processing on the images in the test set;
step 3: inputting the training set apple leaf images processed by the preprocessing module into an encoder, and adopting a lightweight network RegNetY-600MF to encode the apple leaf disease training set images by the encoder to obtain three output characteristic images, and transmitting the three output characteristic images to a decoder;
step 4: using a multistage feature fusion network as a decoder to respectively acquire semantic information and spatial information in three output feature images, carrying out multi-scale feature fusion on the three output feature images to obtain an output feature image of the decoder, and processing the output feature image by a convolution classifier to obtain semantic segmentation feature values;
step 5: comparing the semantic segmentation characteristic value with the labeling characteristic value pixel by pixel, and obtaining an apple leaf disease semantic segmentation model after performing network training by using a back propagation algorithm;
step 6: and (3) inputting the apple leaf disease image to be tested into a semantic segmentation model after scaling and normalization treatment to obtain a semantic segmentation characteristic value of the image to be tested, and performing up-sampling operation on the characteristic value to obtain the apple leaf disease semantic segmentation image.
3. The apple leaf disease image semantic segmentation method according to claim 2, wherein the step 2 uses a preprocessing module to preprocess samples in a training verification set to form a training set and a verification set, and specifically comprises:
the resolution ratio of the sample images in the training verification set and the corresponding label sample images is adjusted to a preset size;
the image augmentation mode comprises scaling, cutting, overturning, noise adding, splicing and the like, and the image augmentation processing and normalization processing are carried out on the sample images in the training verification set and the corresponding label sample images according to the probability of 0.5 probability of each image augmentation mode;
according to 8: the scale of 2 divides the color sample image and the corresponding label sample image into a training set and a validation set.
4. The apple leaf disease image semantic segmentation method according to claim 2, wherein the encoder in the step 3 obtains three output feature graphs by using a lightweight regnet-600 MF network; the regnet-600 MF network starts with a 3 x 3 convolution with 32 output channels and 2 steps; then the three modules are passed through a 48-channel D module with 1/4 resolution, 3 128-channel D modules with 1/8 resolution, 2 256-channel D modules with 1/16 resolution, 11 256-channel D modules with 1/16 resolution, and finally the three modules are ended by a 320-channel D module with 1/16 resolution;
the structure of the D module is as follows: each D module sequentially carries out convolution, 3 multiplied by 3 group convolution and one convolution and then adds the convolution with the output, and each convolution carries the operations of BatchNorm and ReLU;
the three output characteristic diagrams are respectively as follows: the first layer of output characteristic diagram is 1/4 of the resolution of the original image, and the number of channels is 128; the second layer of output characteristic diagram is 1/8 of the resolution of the original image, and the number of channels is 256; the third layer output characteristic diagram is 1/16 of the original image resolution, and the channel number is 320.
5. The apple leaf disease image semantic segmentation method according to claim 2, wherein the decoder in step 4 uses a multi-level feature fusion network of multi-scale information fusion;
the structure of the multistage feature fusion network is as follows: the feature maps of 1/16, 1/8 and 1/4 resolution are respectively convolved by a 128-channel; the convolved 1/16 resolution characteristic diagram and the convolved 1/8 resolution characteristic diagram pass through an SEB module, and the convolved 1/8 resolution characteristic diagram and the convolved 1/4 resolution characteristic diagram pass through the SEB module; up-sampling the 1/16 resolution characteristic diagram through the SEB module through the RCU module and adding the 1/8 resolution characteristic diagram through the SEB module; the added 1/8 resolution feature map is added with the 1/4 resolution feature map through the SEB module through the RCU module, and the added 1/8 resolution feature map is output from the decoder through the RCU module.
6. The multi-level feature fusion network of claim 5, wherein the SEB module is constructed by:
the input of the SEB module comprises two adjacent encoder output feature maps, wherein the low-resolution feature map changes the channel number into the channel number of the high-resolution feature map through convolution operation with the convolution kernel size, the resolution of the low-resolution feature map is restored to be 2 times of that of the original feature map through upsampling operation, and pixel-by-pixel addition operation is carried out on the restored feature map and the high-resolution feature map to obtain the SEB module output feature map.
7. The multi-level feature fusion network of claim 5, wherein the RCU module is constructed by:
each input path of the RCU module sequentially passes through two residual convolution units, wherein the first convolution is; the second convolution is followed by the BatchNorm and Prelu activation functions, except for all convolutions after the last convolution.
8. The convolutional classifier of claim 5, wherein the convolutional classifier uses a two-convolutional operation with parameters of: the input channels are the number of apple leaf disease sample image segmentation objects, the convolution kernel size is 1, and the activation function selects softmax.
9. The method according to claim 2, wherein the back propagation algorithm in step 5 uses a cross entropy loss function, the optimizer selects SGD, and trains 50 iterations in total, the initial learning rate of the first 25 iterations is 0.001, the initial learning rate of the last 25 iterations is 0.0001, and the learning rate attenuation policy is an exponential attenuation policy with a value of 0.98.
10. An electronic device comprising a processor, a memory, and a computer readable storage medium, the processor and the memory being communicable;
a memory for storing a control program;
a processor for implementing the method steps of any of claims 2-9 when executing a program stored on a memory;
a computer readable storage medium storing a control program for a fruit leaf disease image semantic segmentation task, which when executed by a processor, implements the apple leaf disease image semantic segmentation method steps of any one of claims 2-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211171326.1A CN116091763A (en) | 2022-09-24 | 2022-09-24 | Apple leaf disease image semantic segmentation system, segmentation method, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211171326.1A CN116091763A (en) | 2022-09-24 | 2022-09-24 | Apple leaf disease image semantic segmentation system, segmentation method, device and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116091763A true CN116091763A (en) | 2023-05-09 |
Family
ID=86199744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211171326.1A Pending CN116091763A (en) | 2022-09-24 | 2022-09-24 | Apple leaf disease image semantic segmentation system, segmentation method, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116091763A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117372881A (en) * | 2023-12-08 | 2024-01-09 | 中国农业科学院烟草研究所(中国烟草总公司青州烟草研究所) | Intelligent identification method, medium and system for tobacco plant diseases and insect pests |
-
2022
- 2022-09-24 CN CN202211171326.1A patent/CN116091763A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117372881A (en) * | 2023-12-08 | 2024-01-09 | 中国农业科学院烟草研究所(中国烟草总公司青州烟草研究所) | Intelligent identification method, medium and system for tobacco plant diseases and insect pests |
| CN117372881B (en) * | 2023-12-08 | 2024-04-05 | 中国农业科学院烟草研究所(中国烟草总公司青州烟草研究所) | Intelligent identification method, medium and system for tobacco plant diseases and insect pests |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021042828A1 (en) | Neural network model compression method and apparatus, and storage medium and chip | |
| CN113159051B (en) | Remote sensing image lightweight semantic segmentation method based on edge decoupling | |
| CN111126386B (en) | Sequence domain adaptation method based on countermeasure learning in scene text recognition | |
| CN110210513B (en) | Data classification method and device and terminal equipment | |
| CN113642390B (en) | Street view image semantic segmentation method based on local attention network | |
| CN112561027A (en) | Neural network architecture searching method, image processing method, device and storage medium | |
| CN110930378B (en) | Emphysema image processing method and system based on low data demand | |
| CN112149526B (en) | Lane line detection method and system based on long-distance information fusion | |
| CN110188827A (en) | A kind of scene recognition method based on convolutional neural networks and recurrence autocoder model | |
| CN110599502A (en) | Skin lesion segmentation method based on deep learning | |
| CN110991247B (en) | Electronic component identification method based on deep learning and NCA fusion | |
| CN111008570B (en) | Video understanding method based on compression-excitation pseudo-three-dimensional network | |
| CN117237733A (en) | Breast cancer full-slice image classification method combining self-supervision and weak supervision learning | |
| CN111242028A (en) | Remote sensing image ground object segmentation method based on U-Net | |
| CN116091763A (en) | Apple leaf disease image semantic segmentation system, segmentation method, device and medium | |
| CN114445665A (en) | Hyperspectral image classification method based on Transformer enhanced non-local U-shaped network | |
| CN111898614A (en) | Neural network system, image signal and data processing method | |
| CN114037893A (en) | High-resolution remote sensing image building extraction method based on convolutional neural network | |
| US11397868B2 (en) | Fungal identification by pattern recognition | |
| CN113378938A (en) | Edge transform graph neural network-based small sample image classification method and system | |
| CN116612283A (en) | Image semantic segmentation method based on large convolution kernel backbone network | |
| CN116486156A (en) | Full-view digital slice image classification method integrating multi-scale feature context | |
| CN117011219A (en) | Method, apparatus, device, storage medium and program product for detecting quality of article | |
| CN114581789A (en) | Hyperspectral image classification method and system | |
| CN112651978B (en) | Sublingual microcirculation image segmentation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |