CN112288022B - A Grain Insect Recognition Method and Recognition System Based on Feature Fusion of SSD Algorithm - Google Patents

Abstract

The invention relates to a grain worm identification method and identification system based on SSD algorithm feature fusion. The identification method comprises the following steps: establishing a data set; establishing a neural network model and using the data set to train it, and obtaining a trained neural network model ; Collect the images of grain worms to be identified, input them into the trained neural network model, and detect the types and locations of grain worms in them. The technical scheme provided by the present invention fuses the output feature maps of the convolutional layers conv4_3 and conv5_3 in the neural network model, deletes the block 11 that is unfavorable for small target detection, and adopts the K-means clustering algorithm to obtain the first suitable for grain insects. It can improve the defect of the default a priori frame in the original SSD and make it more conducive to the identification and positioning of grain worms, which can solve the problem of poor recognition accuracy of grain worms in the prior art.

Description

A Grain Insect Recognition Method and Recognition System Based on Feature Fusion of SSD Algorithm

technical field

The invention relates to the technical field of grain worm identification, in particular to a grain worm identification method and identification system based on SSD algorithm feature fusion.

Background technique

Grain, oil and food will be attacked by stored grain pests in the whole process of production, processing and storage. Grain worms will not only eat grain, causing the loss of grain quantity, but also the products of their life metabolism will heat up the grain, aggravate the activity of grain microorganisms, make the grain rot and deteriorate, and may induce the production of microbial toxins. In addition, due to the existence of the corpses and excrement of the pests in the grains, the grains are also polluted, the hygienic quality of the grains is degraded, and the health of the users is endangered.

At present, with the development of computer technology, the informatization requirements of the food industry are also getting higher and higher. Because of the introduction of intelligent warehousing technology, grain quality monitoring in warehousing is becoming more and more efficient. As an important part of grain quality monitoring, grain and insect detection based on image processing has gradually become a research hotspot in recent years. Grain and insect image detection includes traditional digital processing and deep learning image processing. According to the online monitoring requirements of intelligent warehousing, the accuracy and real-time performance of grain and insect detection need to be further improved. The traditional image processing technology has been gradually eliminated due to the problems of accuracy and real-time performance. The target detection algorithm in the deep learning image processing technology can not only accurately locate the detection target, but also the single-stage target detection algorithm has achieved real-time requirements after continuous optimization.

The single-stage target detection algorithm is represented by YOLO (You Only Live Once) and SSD (Single Shot Multi-Box Detector), which are famous for their fast detection speed. At present, traffic sign detection, UAV target detection, remote sensing target detection, In various industries such as pedestrian video surveillance, however, due to the small size of grain worms, the detection method in the prior art has the problem of inaccurate identification results when identifying them.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a grain worm identification method and identification system based on SSD algorithm feature fusion, so as to solve the problem of inaccurate identification of grain worms in the prior art.

To achieve the above object, the present invention adopts the following technical solutions:

A grain worm identification method based on feature fusion of SSD algorithm, comprising the following steps:

Step 1: Create a dataset;

Step 2: establishing a neural network model, and using the training data of the dataset to train it to obtain a trained neural network model;

Step 3: Collect the images of grain worms to be identified, input them into the trained neural network model, and detect the types and locations of grain worms in them;

The training data in the data set includes multiple pictures of grain worms, each picture has only one kind of grain worm image, and the resolution of each picture is 640×480;

The feature maps of the neural network model include block12, block7, block8, block9 and block10, wherein block12 is formed by the fusion of the feature maps output by the convolutional layers conv4_3 and conv5_3 through the TOP-DOWN module, and block7, block8, block9 and block10 are respectively The feature maps output by the convolutional layers conv7, conv8_2, conv9_2 and conv10_2, and the size of the prior frame of each feature map is obtained by clustering the K-means algorithm.

Further, the feature map method of using the TOP-DOWN module to fuse the output of the convolutional layers conv4_3 and conv5_3 is:

Perform one convolution and two deconvolutions on the feature map output by the convolutional layer conv5_3, and then access the BN module;

Perform two convolution operations on the output feature map of the shallow convolution layer conv4_3, and then access the BN module;

Feature fusion is performed on the feature map output by the convolutional layer conv5_3 and the output feature map of the convolutional layer conv4_3, and finally the final fused feature map is output through the Relu activation function.

Further, feature fusion is performed on the feature map output by the convolution layer conv5_3 and the feature map output by the convolution layer conv4_3 by means of point multiplication.

Further, the method for calculating the size of the prior frame of each feature map by the K-means algorithm includes the following steps:

Label and initialize the grain worms in the dataset images to obtain m cluster centers;

Calculate the distance between all GTs in the dataset and each cluster center, take the cluster center with the closest distance as its index, and take the feature points with the same index as the same cluster;

When two consecutive clustering results are the same, it is judged that the clustering is over;

The minimum bounding box of each cluster is taken as the prior box.

Further, the grain insects include, but are not limited to, corn weevil, red millet, beetle, rust millet and Indian millet.

A grain insect identification system based on SDD algorithm feature fusion, comprising a processor and a memory, and the memory stores a computer program for execution on the processor; when the processor executes the computer program, the SDD algorithm-based algorithm is realized. A feature fusion method for identifying grains and insects, the method comprises the following steps:

Step 1: Create a dataset;

Step 2: establishing a neural network model, and using the training data of the dataset to train it to obtain a trained neural network model;

Step 3: Collect the images of grain worms to be identified, input them into the trained neural network model, and detect the types and locations of grain worms in them;

The training data in the data set includes multiple pictures of grain worms, each picture has only one kind of grain worm image, and the resolution of each picture is 640×480;

The feature maps of the neural network model include block12, block7, block8, block9 and block10, wherein block12 is formed by the fusion of the feature maps output by the convolutional layers conv4_3 and conv5_3 through the TOP-DOWN module, and block7, block8, block9 and block10 are respectively The feature maps output by the convolutional layers conv7, conv8_2, conv9_2 and conv10_2, and the size of the prior frame of each feature map is obtained by clustering the K-means algorithm.

Further, the feature map method of using the TOP-DOWN module to fuse the output of the convolutional layers conv4_3 and conv5_3 is:

Perform one convolution and two deconvolutions on the feature map output by the convolutional layer conv5_3, and then access the BN module;

Perform two convolution operations on the output feature map of the shallow convolution layer conv4_3, and then access the BN module;

Feature fusion is performed on the feature map output by the convolutional layer conv5_3 and the output feature map of the convolutional layer conv4_3, and finally the final fused feature map is output through the Relu activation function.

Further, feature fusion is performed on the feature map output by the convolution layer conv5_3 and the feature map output by the convolution layer conv4_3 by means of point multiplication.

Further, the method for calculating the size of the prior frame of each feature map by the K-means algorithm includes the following steps:

Label and initialize the grain worms in the dataset images to obtain m cluster centers;

Calculate the distance between all GTs in the dataset and each cluster center, take the cluster center with the closest distance as its index, and take the feature points with the same index as the same cluster;

When two consecutive clustering results are the same, it is judged that the clustering is over;

The minimum bounding box of each cluster is taken as the prior box.

Further, the grain insects include, but are not limited to, corn weevil, red millet, beetle, rust millet and Indian millet.

Beneficial effects of the present invention: The technical solution provided by the present invention fuses the feature maps output by the convolutional layers conv4_3 and conv5_3 in the neural network model, deletes block 11 that is unfavorable for small target detection, and adopts K-means clustering algorithm Obtain a priori frame suitable for grain worms, improve the defects of the default a priori frame in the original SSD algorithm, and make it more conducive to the identification and positioning of grain worms. Therefore, the technical solution provided by the present invention can solve the problem of poor identification accuracy of grain insects in the prior art.

Description of drawings

Fig. 1 is the flow chart of the grain insect identification method based on the feature fusion of SSD algorithm in the method embodiment of the present invention;

2 is a schematic structural diagram of a neural network model in a method embodiment of the present invention;

FIG. 3 is a schematic diagram of a fusion process of a shallow layer feature map and a deep layer feature map in an embodiment of the method of the present invention.

Detailed ways

The purpose of the present invention is to provide a grain worm identification method and identification system based on SSD algorithm feature fusion. The prior frame of the feature layer is used to solve the problem of inaccurate identification of grain insects in the prior art.

Method example:

The present embodiment provides a method for identifying grains and insects based on SSD algorithm feature fusion, the process of which is shown in Figure 1 and includes the following steps:

Step 1: Create a dataset.

The training data in the dataset includes images of a variety of grain insects, including but not limited to corn weevil, red grain thief, grain beetle, rust red flat grain thief and Indian grain borer. When collecting images, live adult worms are used for shooting. Since live worms are more active, the diversity of collected samples can be guaranteed. When collecting pictures, first take a video of the grain worm, and then take a screenshot of it to complete the production of the data set picture.

The training data in the data set of this embodiment has a total of 1998 images, the size of each image is 640×480, each image has 3-10 grain worms, and each picture has only one kind of grain worm. There are 1438 images in the dataset for training the neural network model, 360 images for validating the neural network model, and 200 for testing the neural network model.

Step 2: Build a neural network model.

The neural network model established in this embodiment uses the VGG16 network as the feature extraction network, and its structure is shown in Figure 2. The circular mark in the figure is the TOP-DOWN module shown in Figure 2. Using this module, the conv4_3 Fusion with the output feature map of conv5_3.

The neural network model in this embodiment includes block12, block7, block8, block9 and block10, wherein block12 is formed by fusion of the feature maps output by the convolutional layers conv4_3 and conv5_3 through the TOP-DOWN module, and block7, block8, block9 and block10 are respectively Feature maps output by convolutional layers conv7, conv8_2, conv9_2 and conv10_2. The size of the prior frame of each feature map is obtained by clustering the K-means algorithm.

Step 3: using the training data in the dataset to train the established neural network model to obtain a trained neural network model.

When using the training data in the data set to train the established neural network model, the image of the training data in the data set is used as the input, and the type and location of the grain worms in the image are used as the output to obtain the trained neural network model.

Step 4: Collect the image of the grain worm to be identified, input it into the trained neural network model, and identify the type and quantity of the grain worm.

In this embodiment, the TOP-DOWN module is used to fuse the shallow feature map output by conv4_3 and the deep feature map output by conv5_3. The fusion process is shown in Figure 3, including steps:

Step 1.1: Perform one convolution and two deconvolutions on the deep feature map to convert its size to twice the original size, and then access the BN module, where convolution and deconvolution are operations in deep learning;

Step 1.2: Perform two convolution operations on the shallow feature map, and then access the BN (Batch Normalization, batch normalization) module;

Step 1.3: Perform feature fusion on the deep feature map and the shallow feature map. In this embodiment, the deep feature map and the shallow feature map are combined with the shallow feature map and the deep feature map by performing a dot product operation in each channel. perform feature fusion;

Step 1.4: Use the Relu activation function to activate the feature map after point multiplication to obtain the final fused feature map, where the Relu activation function is an operation that uses nonlinearity in the convolutional neural network.

In this embodiment, in the process of fusing the deep feature map and the shallow feature map, when performing the convolution operation, the convolution kernel is a 3×3×C1 convolution kernel.

In this embodiment, the K-means algorithm is used for clustering, traversing the data set, and determining the aspect ratio of the prior frame of each feature map, so that the established neural network model can more easily and accurately locate the grain worms.

The process of using K-means to obtain the prior frame includes the following steps:

Step 2.1: Label and initialize the grain worms in the dataset image to obtain m cluster centers, that is, randomly select m bounding boxes from all GT (ground truth, manually marked bounding boxes), where m is greater than a positive integer of 1;

Step 2.2: Calculate the distance between all GTs in the data set and each cluster center, select the cluster center with the smallest distance and save its index; when two consecutive clustering results are consistent, it is judged that the clustering is over.

Step 2.3: Take the labeled points of the same cluster center as the same class, and obtain the minimum bounding box of each class, and the bounding box is the a priori box.

According to the clustering results, the length-width ratio of the prior frame corresponding to the feature map output by each layer is shown in Table 1.

Table 1

Feature map Aspect ratio of a priori box Block12 [1, 1', 2, 1./2, 1./4, 1./3] Block7 [1, 1', 2, 1./2, 1./4, 1./3] Block8 [1, 1', 2, 1./2, 1./4, 1./3] Block9 [1, 1', 2, 1./2, 1./4, 1./3] Block10 [1, 1', 2, 1./2]

In this embodiment, Precision (Precision), Recall (Recall), AP (Average Precision), mAP (Average Accuracy) and FPS (Frame Rate per Second) are used to measure the pros and cons of the established neural network model . All indicators are that the larger the value, the better the detection performance. The FPS represents the detection speed, and the larger the value, the faster the detection speed.

The formulas for calculating precision and recall are:

all det ectioons=TP+FP

all ground truths=FN+TP

where TP is the number of positive samples that are correctly classified as positive samples (True positive), FP is the number of negative samples that are wrongly classified as positive samples (False positive), and FN is the number of positive samples that are wrongly classified as negative samples ( False negative).

Compared with the original SSD algorithm, the optimized SSD model has a great improvement in mAP from 88.56% to 96.74%. Although the FPS is reduced from 25 to 21, it can still meet the requirements of real-time detection. The mAP comparison of the optimized neural network detection results is shown in Table 1. The optimized neural network model has greatly improved the target detection accuracy of grain worms.

Table 1

Model mAP/% red grain thief Beetle corn elephant Rusty Red Flat Grain Thief Indian grain borer FPS SSD before optimization 88.56 91.27 90.37 93.43 76.06 91.66 25 Optimized SSD 96.74 95.40 98.63 96.95 95.67 97.06 twenty one

System example:

This embodiment provides a system for identifying grains and insects based on SDD algorithm feature fusion, including a processor and a memory, and the memory stores a computer program for execution on the processor; when the processor executes the computer program, the As provided in the above method embodiments, the method for identifying grain insects based on feature fusion of SDD algorithm is provided.

The embodiments of the present invention disclosed above are only used to help clarify the technical solutions of the present invention, and do not describe all the details, nor limit the present invention to only the described specific embodiments. Obviously, many modifications and variations are possible in light of the content of this specification. These embodiments are selected and described in this specification in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is to be limited only by the claims and their full scope and equivalents.

Those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements will not make the essence of the corresponding technical solutions. It departs from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A characteristic fusion grain insect identification method based on an SSD algorithm is characterized by comprising the following steps:

the method comprises the following steps: establishing a data set;

step two: establishing a neural network model, and training the neural network model by adopting training data of a data set to obtain a trained neural network model;

step three: collecting grain insect images to be identified, inputting the grain insect images into the trained neural network model, and detecting the types and positions of the grain insects;

the training data in the data set comprises a plurality of images of the grain insects, each image only has an image of one grain insect, and the resolution of each image is 640 multiplied by 480;

the characteristic diagrams of the neural network model comprise block12, block7, block8, block9 and block10, wherein block12 is formed by fusing characteristic diagrams output by convolution layers conv4_3 and conv5_3 through TOP-DOWN modules, block7, block8, block9 and block10 are characteristic diagrams output by convolution layers conv7, conv8_2, conv9_2 and conv10_2 respectively, and the prior frames of the characteristic diagrams are obtained by clustering through a K-means algorithm.

2. The SSD algorithm based feature fusion grainworm identification method as claimed in claim 1, wherein the feature map method of fusing convolutional layer conv4_3 and conv5_3 outputs using TOP-DOWN module is:

performing convolution and deconvolution for one time on the feature graph output by the convolution layer conv5_3, and then accessing a BN module;

carrying out convolution operation twice on the shallow convolution layer conv4_3 output characteristic diagram, and then accessing a BN module;

and performing feature fusion on the feature map output by the convolutional layer conv5_3 and the feature map output by the convolutional layer conv4_3, and finally outputting the finally fused feature map through a Relu activation function.

3. The SSD algorithm-based characteristic fusion grainworm identification method as claimed in claim 2, wherein the characteristic fusion is performed on the characteristic graph output by the convolution layer conv5_3 and the characteristic graph output by the convolution layer conv4_3 in a dot multiplication mode.

4. The SSD algorithm-based feature fusion grainworm identification method as claimed in claim 1, wherein the method for calculating the prior frame size of each feature map by K-means algorithm comprises the steps of:

marking the grain insects in the data set image, and initializing to obtain m clustering centers;

calculating the distance between all GT in the data set and each clustering center, taking the clustering center closest to the GT as the index of the GT, and taking the characteristic points with the same index as the same cluster;

when the clustering results of two consecutive times are the same, judging that clustering is finished;

and taking the minimum bounding box of each cluster as a prior box.

5. The SSD algorithm-based feature fusion-based grain insect recognition method of claim 1, wherein the grain insects comprise zealand corn weevil, tribolium castaneum, cornice beetle, tribolium castaneum and indian meal moth.

6. A characteristic fusion type grain insect recognition system based on an SSD algorithm comprises a processor and a memory, wherein the memory is stored with a computer program executed on the processor; the method is characterized in that when the processor executes the computer program, the method for identifying the grain insects based on the feature fusion of the SSD algorithm is realized, and the method comprises the following steps:

the method comprises the following steps: establishing a data set;

step two: establishing a neural network model, and training the neural network model by adopting training data of a data set to obtain a trained neural network model;

step three: collecting the images of the grain insects to be identified, inputting the images into the trained neural network model, and detecting the types and positions of the grain insects;

the training data in the data set comprises a plurality of images of the grain insects, each image only has an image of one grain insect, and the resolution of each image is 640 multiplied by 480;

the characteristic diagrams of the neural network model comprise block12, block7, block8, block9 and block10, wherein block12 is formed by fusing characteristic diagrams output by convolution layers conv4_3 and conv5_3 through TOP-DOWN modules, block7, block8, block9 and block10 are characteristic diagrams output by convolution layers conv7, conv8_2, conv9_2 and conv10_2 respectively, and the prior frames of the characteristic diagrams are obtained by clustering through a K-means algorithm.

7. The SSD algorithm-based feature-fused grainworm identification system of claim 6, wherein the feature map method of fusing convolutional layer conv4_3 and conv5_3 outputs using TOP-DOWN modules is:

performing convolution and deconvolution for one time on the feature graph output by the convolution layer conv5_3, and then accessing a BN module;

carrying out convolution operation twice on the shallow convolution layer conv4_3 output characteristic diagram, and then accessing a BN module;

and performing feature fusion on the feature map output by the convolutional layer conv5_3 and the feature map output by the convolutional layer conv4_3, and finally outputting the finally fused feature map through a Relu activation function.

8. The SSD-algorithm-based feature-fused grainworm identification system according to claim 7, wherein the feature map output by the convolutional layer conv5_3 and the convolutional layer conv4_3 are feature-fused in a dot-by-dot manner.

9. The SSD algorithm based feature-fused grainworm identification system of claim 6, wherein the method for calculating the prior frame size of each feature map by K-means algorithm comprises the steps of:

marking the grain insects in the data set image, and initializing to obtain m clustering centers;

calculating the distance between all GT in the data set and each clustering center, taking the clustering center closest to the GT as the index of the GT, and taking the characteristic points with the same index as the same cluster;

when the clustering results of two consecutive times are the same, judging that clustering is finished;

and taking the minimum bounding box of each cluster as a prior box.

10. The SSD algorithm-based feature-fused grain insect recognition system of claim 6, wherein the grain insects comprise zealand corn weevil, tribolium castaneum, cornus beetle, tribolium rusticanum and indian meal moth.

Images