CN111832642A - An image recognition method based on VGG16 in insect taxonomy - Google Patents

Abstract

The invention relates to an image recognition method based on VGG16 in insect taxonomy, comprising: S1, establishing an image data set; S2, processing insect images in the image data set to obtain a training data set; S3, using a VGG16 model to analyze the training data S4, extract some images from the image data set as the reference image and the image to be recognized, and perform corner detection to correct the reference image; S5, input the image to be recognized and the corrected reference image after processing and input after training The VGG16 model is used to extract image features; S6, visualize the extracted image features to obtain a feature map; S7, calculate the image similarity SSIM of the image to be recognized and all reference images under the order of each class of insects The average value is obtained, and the image to be recognized is classified into the category with the largest average value as the target level element to which it belongs. The invention improves the accuracy and efficiency of insect classification.

Description

An image recognition method based on VGG16 in insect taxonomy

technical field

The invention belongs to the technical field of insect image recognition and classification, in particular to an image recognition method based on VGG16 in insect taxonomy.

Background technique

In the traditional insect classification method, whether it is traditional taxonomy or numerical taxonomy, the main basis for classifying insects is the body characteristics of insects, these body characteristics include insect color, markings, body appendages (such as nodules, engraved point, cilia, etc.) and size (such as body length, body width) and so on. However, these body characteristics cannot fully reflect the differences between the body morphological characteristics of different insect groups, and classification of insects based on these body characteristics alone cannot achieve a better accuracy.

With the rapid development and wide application of computer science, it is possible to use computer vision as a means to extract and analyze biological images including insects, and to classify biological species using the features extracted by computers. Different species of insects have different body sizes and shapes. For the mathematical morphological characteristics of insects, in addition to body length and body width, it also includes some mathematical characteristics such as insect body area, perimeter, eccentricity, and roundness. These quantitative features that can represent the body shape of different insect groups may more accurately and comprehensively reflect the differences between different insect groups. Before the emergence of deep learning convolutional neural networks, the extraction of these features in pattern recognition mainly relied on manual extraction, and there was a certain degree of subjectivity. In addition, most of the existing methods for image recognition based on deep learning do not pay attention to what results are produced during the operation of the neural network. The fully connected layer is destructive to the image space structure, which will bring loss to the image recognition accuracy to a certain extent. And there are few researches on image order-level recognition in insect taxonomy based on convolutional neural network.

SUMMARY OF THE INVENTION

Based on the above-mentioned shortcomings and deficiencies in the prior art, one of the objectives of the present invention is to at least solve one or more of the above-mentioned problems existing in the prior art. In other words, one of the objectives of the present invention is to provide one of the aforementioned requirements An image recognition method based on VGG16 in insect taxonomy.

In order to achieve the above-mentioned purpose of the invention, the present invention adopts the following technical solutions:

An image recognition method based on VGG16 in insect taxonomy, comprising the following steps:

S1. Collect insect images of various insect orders, classify the images according to the insect orders, and establish an image data set;

S2. Perform target and background segmentation and size normalization on the insect images of the image dataset to obtain a training dataset;

S3. Using the method of migration learning, use the VGG16 model to train the training data set, and obtain the VGG16 model after training;

S4, extract some images from the image data set as the reference image and the image to be recognized, perform corner detection on the image to be recognized and the reference image, obtain the offset value by calculating the mean value of the corner points, and then correct the reference image according to the offset value;

S5, the to-be-recognized image and the corrected reference image are input into the trained VGG16 model after target and background segmentation and size normalization to extract image features;

S6. Use TensorBoard to visualize the extracted image features to obtain a feature map and save it;

S7, calculate the image similarity SSIM of the feature map between the image to be recognized and all reference images under the order level of each type of insect, and obtain the average value, classify the image to be recognized into the class with the largest average value, and obtain the insect order to which the image to be recognized belongs. tier element.

As a preferred solution, the insect orders include Lepidoptera, Orthoptera, Hemiptera, Coleoptera and Homoptera.

As a preferred solution, the image data set needs to meet the following conditions:

Each class of insects includes no less than 10 species of insect pests, and at least 30 original images of each insect.

As a preferred solution, a fully convolutional neural network (FCN) is used for the segmentation of the target and the background to turn the background into black.

As a preferred solution, the size normalization normalizes the image size to 224*224.

As a preferred solution, in the step S2, data enhancement processing is also performed after size normalization.

As a preferred solution, the data enhancement processing includes one or more of flipping, rotation, brightness adjustment, and color adjustment.

As a preferred solution, the VGG16 model includes 13 convolutional layers, 3 fully connected layers and 5 pooling layers.

As a preferred solution, in the step S7, the feature maps obtained by convolution of the first to fifth convolutional layers of the VGG16 model are used as the input for calculating the image similarity.

As a preferred solution, in the step S7, the image similarity SSIM is measured from three aspects of brightness, contrast and structure:

Among them, x and y represent the image to be recognized and the reference image, respectively, μ _x is the mean value of x, μ _y is the mean value of y, σ _x is the variance of x, σ _y is the variance of y, and σ _xy is the covariance of x and y Variance; c1=(k ₁ L) ² , c2=(k ₂ L) ² are used to maintain stable constants, L is the dynamic range of image pixel values 0-255, k ₁ =0.01, k ₂ =0.03.

Compared with the prior art, the present invention has the following beneficial effects:

The image recognition method for insect classification based on VGG16 of the present invention is convenient for classifying insects and improves the accuracy and efficiency of insect classification.

The present invention extracts features from insect images through a deep learning convolutional neural network, thereby avoiding the subjectivity of traditional artificial feature extraction, and the extracted features are more comprehensive.

At present, the research on insect taxonomy is mainly based on traditional mathematical morphological features, and there is almost no application of deep learning convolutional neural network in insect taxonomy, so the present invention is a new method of deep learning in insect taxonomy. try.

Description of drawings

1 is a flowchart of an image recognition method based on VGG16 in insect taxonomy according to an embodiment of the present invention;

Fig. 2 is the network structure diagram of VGG16 of the embodiment of the present invention;

FIG. 3 is a feature map of an insect image according to an embodiment of the present invention after convolution of each layer of VGG16.

Detailed ways

In order to describe the embodiments of the present invention more clearly, the following will describe specific embodiments of the present invention with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative efforts, and obtain other implementations.

As shown in FIG. 1 , the image recognition method based on VGG16 in insect taxonomy according to the embodiment of the present invention includes the following steps:

S1. Collect insect images of various insect orders, classify the images according to the insect orders, and establish an image data set;

Specifically, insect images of various insect orders can be collected from public data sets, data sets obtained by web crawlers, and data sets collected in the field; according to the collected insect images, they are classified according to insect orders. The order-level element includes five categories: Lepidoptera, Orthoptera, Hemiptera, Coleoptera, and Homoptera. Each order-level element includes no less than 10 types of pests, and at least 30 original images of each insect.

S2. Perform target and background segmentation and size normalization on the insect images of the image dataset to obtain a training dataset;

Specifically, a fully convolutional neural network (FCN) is used to segment the insect images of the image dataset from the target and the background, and the background is changed to black in order to better extract insect features.

Since the forward propagation of the VGG16 fully connected layer is the product of the weight of the current layer and the output of the previous layer, as follows:

a ^k =σ(z ^k )=σ(W ^k a ^k-1 +b ^k )

Among them, k represents the k-th layer, W is the weight of the k-th layer, b ^k is the bias of the k-th layer, and a ^k-1 represents the output of the k-1 layer (ie, the previous layer), which is also the k-th layer. Input, a ^k represents the output of the kth layer (ie the current layer), and σ is the activation function.

The shape of the VGG16 weight matrix is fixed. If the size of the input is not fixed, forward propagation cannot be performed, resulting in inability to calculate and model training will be interrupted. In order for the model training to work properly, the image size needs to be normalized to 224*224.

In addition, in order to obtain better feature extraction and classification effects, the deep learning convolutional neural network VGG16 model requires a high number of training data sets. Therefore, when the number of images in the image data set is small, data enhancement can be used. The methods of sample augmentation of the dataset include: flipping, rotating, brightness adjustment, color adjustment, etc.

Therefore, in order to enable the images to be input into the convolutional neural network in this embodiment of the present invention, the size of all images is first adjusted, and a python program is used to normalize the size of all images to 224*224.

In order to enable the trained model to have better generalization ability, a large number of training data sets are required to train the model. When the number of original image data sets is insufficient, the data set can be expanded by means of data enhancement (that is, the sample Expansion), the commonly used methods include: flip, brightness adjustment, color adjustment, etc.

S3. Using the method of migration learning, use the VGG16 model to train the training data set, and obtain the VGG16 model after training;

Specifically, as shown in Figure 2, the deep learning convolutional neural network VGG16 includes:

13 convolutional layers, represented by convx_x respectively, including conv1-1, conv1-2, conv2-1, conv2-2, conv3-1, conv3-2, conv3-3, conv4-1, conv4-2, conv4 -3, conv5-1, conv5-2, conv5-3;

5 pooling layers, represented by poolx respectively, including pool1 located between conv1-2 and conv2-1, pool2 located between conv2-2 and conv3-1, and located between conv3-3 and conv4-1. pool3, pool4 located between conv4-3 and conv5-1, pool5 located after conv5-3;

The 3 fully connected layers are represented by fcxxx respectively, including fc4096, fc4096 and fc1000 located after pool5 in turn. Among them, the convolution layer and the fully connected layer have weight coefficients, so they are also called weight layers, and the total number is 13+3=16, which is the source of VGG16.

Since the ImageNet image dataset includes insects, the VGG16 trained by ImageNet should have a better extraction effect on insect features. Therefore, the transfer learning method is used, and the deep learning convolutional neural network model VGG16 is used to perform the insect images on the ImageNet dataset. For training, the training data set after image segmentation, size normalization and data enhancement is input into the VGG16 model, and the insect image training model is obtained through the fine-tuning network, that is, the trained VGG16 model.

S4, extract some images from the image data set as the reference image and the image to be recognized, perform image corner detection on the image to be recognized and the reference image, obtain the offset value by calculating the average value of the corner points, and then correct the reference image according to the offset value, Specifically correct the relative position of the insects in the image;

Specifically, a picture is randomly selected from the image data set as the image to be recognized, and the selection of the reference image includes different insect species under different order levels as much as possible. The recognition image and each reference image are detected by the corner points, the average value of the corner points of the two images is obtained, the offset value is obtained by comparing the average value of the corner points of the two images, and the reference image is corrected according to the offset value, so that the The relative positions of the insects are roughly the same.

S5, subject the image to be recognized and the corrected reference image to image preprocessing, including target and background segmentation, size normalization, and input the trained VGG16 model after preprocessing to extract image features;

Specifically, the image to be recognized and the corrected reference image are input into the trained VGG16 model to extract image features.

S6. Use TensorBoard to visualize the extracted image features to obtain a feature map and save it;

Specifically, use the visualization tool tensorboard to view the feature map obtained by convolution of the image through the convolution layer. As shown in Figure 3, 1-13 correspond to the feature map obtained by convolution of the 1-13th convolution layer in turn; save the feature map It is used as the basis for the analysis and selection of the convolution feature map in the next step.

The VGG16 shallow network extracts texture and detail features. The shallow network contains more features and also has the ability to extract key features. The deep network extracts contours, shapes, and the strongest features, but with the number of convolutional layers. Deepening, the extracted features are high-dimensional abstract features, and it is likely that some useless features will be extracted due to excessive extraction. Therefore, the embodiment of the present invention uses the feature maps obtained by convolution of the first to fifth layers of convolution layers to form volumes. The product feature set is used as the basis for the next image similarity analysis.

Among them, the first two convolutional layers convolve the input image with 64 3×3 filters with a stride of 1 to obtain a feature map of size 224*224*64, where the padding parameter is the same volume The parameters in the product; then relu, use a 2 × 2 filter with a stride of 2 to build a maximum pooling layer, and the size becomes 112*112*64 after pooling; the third and fourth convolutional layers use 128 3×3 filters with stride 1 convolve the input image to obtain a feature map of size 112*112*128, where the padding parameter is the parameter in the same convolution; then relu, use a 2× 2. The filter with stride 2 builds the maximum pooling layer, and the size becomes 56*56*128 after pooling; the fifth layer convolutional layer uses 256 filters of 3×3 and stride 1 to input image Perform convolution to obtain a feature map with a size of 56*56*256, where the padding parameter is the parameter in the same convolution, then relu is performed, and a 2×2 filter with a stride of 2 is used to construct a maximum pooling layer, After pooling, the size becomes 28*28*256, and the parameters of other convolutional layers can refer to the prior art, which will not be repeated here.

S7, calculate the image similarity SSIM of the feature map between the image to be recognized and all reference images under the order level of each type of insect, and obtain the average value, classify the image to be recognized into the class with the largest average value, and obtain the insect order to which the image to be recognized belongs. tier element.

Specifically, the image structure similarity analysis is performed using the convolutional feature set constructed in the previous step. SSIM is a full-reference image quality evaluation index, which measures image similarity from three aspects: brightness, contrast, and structure. The range of structural similarity is (0, 1), and its calculation formula is as follows:

Among them, x and y represent the image to be recognized and the reference image, respectively, μ _x is the mean value of x, μ _y is the mean value of y, σ _x is the variance of x, σ _y is the variance of y, and σ _xy is the covariance of x and y Variance; c1=(k ₁ L) ² , c2=(k ₂ L) ² are used to maintain stable constants, L is the dynamic range of image pixel values 0-255, k ₁ =0.01, k ₂ =0.03.

Calculate the SSIM value of the to-be-recognized image and all reference images under each object level, and calculate the average SSIM value of the to-be-identified image and all images under each object level. The category corresponding to the maximum SSMI mean value is the Identify the order element to which the image belongs.

The above is only a detailed description of the preferred embodiments and principles of the present invention. For those of ordinary skill in the art, according to the ideas provided by the present invention, there will be changes in the specific implementation, and these changes should also be It is regarded as the protection scope of the present invention.

Claims (10)

1. An image identification method based on VGG16 in insect taxonomy is characterized by comprising the following steps:

s1, collecting insect images of various insect eye level elements, classifying the images according to the insect eye level elements, and establishing an image data set;

s2, carrying out target and background segmentation and size normalization on the insect image of the image data set to obtain a training data set;

s3, training a training data set by using a VGG16 model by adopting a transfer learning method to obtain a trained VGG16 model;

s4, extracting partial images from the image data set as reference images and images to be identified, carrying out corner detection on the images to be identified and the reference images, calculating a corner mean value to obtain an offset value, and correcting the reference images according to the offset value;

s5, performing target and background segmentation and size normalization on the image to be recognized and the corrected reference image, inputting the image to the trained VGG16 model, and extracting image features;

s6, visualizing the extracted image features by using a TensorBoard to obtain a feature map and storing the feature map;

and S7, calculating feature map image similarity SSIM of the image to be recognized and all reference images under each type of insect eye level, calculating a mean value, and classifying the image to be recognized to the type with the largest mean value to obtain the insect eye level to which the image to be recognized belongs.

2. The image identification method based on VGG16 in insect taxonomy according to claim 1, wherein the insect order-level elements include Lepidoptera, Orthoptera, Hemiptera, Coleoptera and Homoptera.

3. The image identification method based on VGG16 in insect taxonomy according to claim 2, wherein the image data set satisfies the following condition:

each insect eye level element comprises not less than 10 insect pest species, and each insect original image comprises at least 30.

4. The image identification method based on VGG16 in insect taxonomy according to claim 1, wherein the target and background segmentation adopts a full convolution neural network (FCN) to change the background to black.

5. The method of claim 1, wherein the size normalization normalizes the size of the image to 224 x 224 based on VGG16 image recognition in insect taxonomy.

6. The method of claim 1, wherein in step S2, the size normalization is followed by data enhancement.

7. The image identification method based on VGG16 in insect taxonomy according to claim 6, wherein the data enhancement processing comprises one or more of flipping, rotating, brightness adjusting and color adjusting.

8. The image identification method based on VGG16 in insect taxonomy according to claim 1, wherein the VGG16 model comprises 13 convolutional layers, 3 fully-connected layers and 5 pooling layers.

9. The image identification method based on VGG16 in insect taxonomy according to claim 8, wherein in step S7, feature maps obtained by convolution of the first to fifth convolutional layers of the VGG16 model are used as input for calculating image similarity.

10. The image recognition method based on VGG16 in insect taxonomy according to claim 1, wherein in step S7, the similarity SSIM of the image is measured from three aspects of brightness, contrast and structure:

wherein x and y respectively represent the image to be identified and the reference image, mu_xIs the mean value of x, μ_yIs the mean value of y, σ_xIs the variance of x, σ_yIs the variance of y, σ_xyIs the covariance of x and y; c1 ═ k₁L)²、c2＝(k₂L)²A constant for maintaining stability, L being the dynamic range of image pixel values 0-255, k₁＝0.01，k₂＝0.03。

Images