CN110210510B - Defective shrimp rapid identification method based on deep convolutional neural network - Google Patents

Abstract

The invention discloses a defective shrimp rapid identification method based on a deep convolutional neural network, which comprises the following steps of: 1) the first convolution layer and the second convolution layer of the traditional LeNet-5 are improved and are amplified into a three-layer parallel network; 2) adding a combined classifier idea, improving the traditional LeNet-5 classification mode, and forming a classification combination layer; 3) and optimizing and adjusting the network structure by taking the minimum parameters and the optimal recognition rate as indexes. The invention can be directly used in the sample pretreatment link of the prawn breeding and processing factory to remove inferior prawns and realize the online evaluation and measurement of the prawn quality so as to meet the requirements of fine selection and classification of prawn products.

Description

Defective shrimp rapid identification method based on deep convolutional neural network

Technical Field

The invention relates to the technical field of nondestructive testing of agricultural products, in particular to a defective shrimp rapid identification method based on a deep convolutional neural network.

Background

In the industrial chain of prawn culture, impurities or defective shrimps in shrimp products are usually removed by a manual selection method, so that the mode is low in efficiency and easy to fatigue workers; at present, labor force is generally expensive, defective shrimps are not suitable for picking up by using a manual mode, an efficient and intelligent method is urgently needed to replace manual work to finish automatic sorting of the shrimps, the most core part in automatic sorting of the shrimps is algorithm design and innovation, and the quality of the algorithm is directly related to the quality evaluation of the shrimp products. The conditions of defective shrimps and fish and shrimp attachments in the shrimp groups generally caught from the ponds of the aquaculture plants mainly include the following categories: the defective shrimps are mainly formed due to different growth environment conditions, nutrition conditions and fishing modes, and if the defective shrimps are not cleared in time, the defective shrimps are mixed with normal shrimp groups and flow into the market after being packaged, so that the life health of consumers is seriously influenced.

Based on the difference between defective shrimps and normal shrimps on the appearance surface, a plurality of international water-production research teams disclose and authorize a plurality of prawns grading algorithms which can be implemented and are based on the machine vision technology. Although these methods are effective in particular prawn classification problems, direct application of these methods to classification problems of prawns containing multiple defect types leads to two problems:

(1) most of the prawn classification algorithms proposed by researchers before are designed based on the cognition of human eyes to the appearance of prawn bodies, for example, Luoyang proposes a prawn quality detection and classification device based on a machine vision technology, the patent firstly discloses a prawn classification method based on the vision technology, and the influence of the change of the prawn appearance form on the recognition of defective prawns is researched (patent publication No. CN 203018326U); liu Zi Hao discloses a method for judging the integrity and consistency of shrimp bodies (patent publication No. CN 105389586A); zhang Wei discloses a prawn appearance consistency detection method based on image characteristic spectrum (patent publication No. CN 106991667A); liu Xiang discloses an automatic prawn form parameter measurement method based on image recognition and a cascade classifier (patent publication No. CN 108388874A). In the patent disclosed above, since different researchers have different cognitions of the same thing, they may more or less incorporate personal cognitive factors when designing the prawn classification algorithm, which may cause great deviation in algorithm construction; moreover, highly integrated optimization algorithms require a great deal of time and effort to optimize, taking time and effort.

(2) Although some scholars in the early period propose various prawn defect recognition algorithms, the false positive rate is always high, namely the probability of misjudging defective shrimps into normal shrimps is high, and particularly, diseased shrimps or dead shrimps can flow into normal shrimp groups, so that the normal shrimp groups are seriously polluted, and great economic loss and catastrophic consequences are brought.

Disclosure of Invention

The traditional prawn identification technology has the defects of poor identification effect, long identification time and incapability of being applied to large-scale prawn processing plants. In order to overcome the defects in the prior art, the invention discloses a high-efficiency intelligent recognition algorithm for penaeus vannamei boone, and the specific technical scheme is as follows:

a defective shrimp rapid identification method based on a deep convolutional neural network is characterized by comprising the following steps:

1) the first and second convolution layers of the traditional LeNet-5 are improved and are amplified into a three-layer parallel network structure;

2) adding a combined classifier idea, and improving a traditional LeNet-5 classification layer to form a classification combined layer structure;

3) and optimizing and adjusting the improved network structure by taking the minimum parameters and the optimal recognition rate as indexes.

The method for improving the traditional LeNet-5 convolutional layer comprises the following steps:

1) respectively expanding the original LeNet-5 first convolution layer and the original LeNet-5 second convolution layer into a parallel three-layer structure, and establishing a relationship between the first convolution layer and the second convolution layer to generate communication;

2) distributing convolution kernels in the three layers of amplified parallel networks according to a form from top to bottom according to 1 x 1, 3 x 3 and 5 x 5, sequentially traversing images and capturing effective features;

3) for the three layers of parallel networks after amplification, convolution steps in the three layers of parallel networks are distributed according to 2, 5 and 9 from top to bottom respectively, and the capability of the network for adapting to different types of samples is increased.

Further, the step of improving the traditional LeNet-5 classification layer comprises the following steps:

and constructing a combined classifier layer between the second full connection layer (FC4) and the last classification layer of LeNet-5, and respectively integrating the features into SVM, Softmax and Random forest sub-classifiers to obtain the sub-labels of the corresponding classes.

Further, the step of optimally adjusting the improved network structure comprises:

1) fixing other parameters, respectively adjusting partial convolution layers, pooling layers and full-connection layers of the improved network, and exploring the change of model parameters and the influence on the final recognition rate;

2) fixing other parameters and the number of network layers to be unchanged, adjusting and improving the number of partial convolution kernels in the convolution layers in the network, and exploring and obtaining the number of corresponding convolution kernels under the minimum model parameters and the optimal recognition rate.

Advantageous effects

The method for quickly identifying defective shrimps based on the convolutional neural network has the beneficial effects that:

(1) the original LeNet-5 network is improved, so that the algorithm execution efficiency can be improved, the calculation load is reduced, and the overall recognition rate of defective shrimps is increased;

(2) provides a rapid and efficient defective shrimp identification means for prawn breeding and processing factories, and provides a certain technical guarantee for the economic benefits of manufacturers.

Drawings

FIG. 1 is a schematic diagram of the structure of the intelligent prawn identification network of the present invention;

FIG. 2 learning rate parameter optimization diagram

FIG. 3 is a diagram of network clipping rate parameter optimization

FIG. 4 sample lot size parameter optimization map

FIG. 5 is a table of the relationship between the parameter design and the image output size of the network architecture

FIG. 6 is a table of experimental results for the combination classifier and the individual classifiers

FIG. 7 is a table of experimental results relating network structure and convolution kernel number variation

Detailed Description

The method of the invention is further described in detail by taking penaeus vannamei as a research object and combining the attached drawings of the specification.

As shown in fig. 1, this embodiment shows a novel deep convolutional neural network structure for shrimp recognition, which includes an image input layer, a convolutional layer (Conv1-2), a parallel convolutional layer (Conv11, Conv12, Conv13, Conv21, Conv22, Conv23), a Pooling layer (Pooling1-2), a fully-connected layer (FC3-4), a classifier combining layer, and a classification layer (FC 5). The image input layer is mainly used for reading in images and carrying out some preprocessing operations; the parallel convolution layer adopts the initialized convolution kernel to carry out convolution processing on the whole image, and the effective characteristic expression in the parallel convolution layer is deeply mined, which is the first innovation point of the invention; the pooling layer mainly carries out value average or maximum processing on the convolved image; the full-connection layer is used for performing linear dimensionality reduction integration on the characteristics learned by the previous convolutional layer; the classification combination layer is used for fusing a conventional classifier with a general effect through a certain algorithm to improve the classification recognition rate to the maximum extent, and the classification combination layer is a second innovation point of the invention; the classification layer is used for synthesizing the results of the classification combination layer and outputting the final classification result. Adjusting the network structure optimization parameters to be the third innovation point of the invention; the three important innovative aspects of the present invention are explained in detail below.

1. Building parallel convolutional layers

(1) Superposition of convolution kernels

The original LeNet-5 neural network structure is a single-layer through structure, in order to improve algorithm execution efficiency, a first convolution layer and a second convolution layer of LeNet-5 are longitudinally expanded, in order to improve identification efficiency of defective shrimps, 1 x 1, 3 x 3 and 5 x 5 convolution kernel traversal images are sequentially adopted from top to bottom in an expanded three-layer network, the purpose is to effectively classify the prawns with different colors, shapes and textures, and the reason is that the 1 x 1 convolution kernel almost can learn the feature expression of the defective shrimps with slight texture difference with normal shrimps due to small coverage area when traversing the images; when the 3 x 3 convolution kernel traverses the image, the coverage area is medium and can cover corresponding color information, and the feature expression of defective shrimps with color difference with normal shrimps can be learned; when the 5 x 5 convolution kernel traverses the image, the coverage area is large, the corresponding shape characteristics can be covered, and the defective shrimp feature expression which is different from the normal shrimp in appearance shape and form can be learned. Moreover, the introduction of the small convolution kernel can simplify a large number of network model parameters to a certain extent, for example, the number of the network parameters constructed by adopting 1 × 1 and 3 × 3 convolution kernels is respectively 25 times and 3 times less than that of the network parameters constructed by completely using 5 × 5 convolution kernels, so that the execution efficiency of prawn identification can be improved, the prawn identification work can be completed quickly and efficiently, and a theoretical basis is provided for the online identification of prawns.

(2) Execution efficiency of three-layer parallel network

The traditional LeNet-5 network adopts a single-layer network structure, and the training mode is executed in a single GPU, so that a plurality of pieces of parallel information cannot be processed in parallel. The parallel three-layer convolutional network structure is constructed, each layer of network is placed in the independent GPU acceleration module for independent training, and the parallel three-layer convolutional network structure has the advantage that the CUDA module can synchronously accelerate the training process of the deep convolutional neural network. Moreover, when training the network, the powerful GPU computing power can handle a large number of matrix operations. The three types of GPUs employed in the present invention are all NVIDIA GTX 1050Ti (4 GB). If only one GPU is used for training the prawn intelligent recognition network, and the GPU hardware design structure is not expanded, the maximum scale of the parallel network is limited. Therefore, each part of the network structure is distributed into the GPU, and only one third of deep convolution neural network parameters are stored in each GPU memory, so that convenient communication is provided for the GPU. In addition, the GPUs can access memory from each other, while the process does not access host memory. The design of the shrimp recognition network allows the GPU to communicate only at specific layers of the network, thereby controlling the performance loss of the communication. Therefore, it is effective to train the shrimp recognition network simultaneously using multiple GPUs.

(3) Convolution step stacking

In order to enhance the generalization performance of the network, different convolution steps are respectively embedded into the three layers of parallel convolution layers which are constructed, wherein the convolution steps are 2, 5 and 9 from top to bottom. The method aims to greatly influence the features learned by the deep convolutional neural network by different convolution step sizes. For example, a large convolution step (9) may obtain information about the prawn image, including size, shape and position; in contrast, for small convolution steps (2), it is easy to capture information of the image such as color, texture, and image granularity. According to the nine types of prawn samples contained in the data set, the factors can be used as references for identifying different types of prawns. Some types of prawns have great difference in consistency of color and shape, and some types of prawns have great difference in consistency of texture and fine granularity, so that accurate identification on different types of prawns can be obtained by adopting different convolution step lengths, and the overall classification accuracy can be improved.

In summary, the calculation of the size after image output in the deep neural network structure constructed by the present invention can be expressed by the following formulas (1.1) and (1.2):

wherein (H)_in,W_in) Indicates the input size of the image, (H)_out,W_out) Indicating the output size of the image, padding [0 ]]And padding [1 ]]Respectively representing the number of pixels filled from the x-axis direction and the y-axis direction of the image; dilation [0 ]]And dilation [1 ]]Respectively showing the areas of the pixels expanded from the x-axis direction and the y-axis direction of the image; kernel _ size [0 ]]And kernel _ size [1 ]]The number of convolution kernels given from the x-axis direction and the y-axis direction of the image are respectively represented. A table relating to network parameter settings and image output sizes is shown in fig. 5.

2. Building a classifier Assembly layer

The traditional LeNet-5 classification layer adopts a structure based on a single classifier, the structure is often connected behind a full connection layer (FC4), but the features learned by a deep convolutional network can only be combined with a specially designed classifier to obtain a better recognition effect, so the invention proposes to effectively superpose three SVM, Softmax and Random forest sub-classifiers in a multi-classifier combination mode to form a combined classifier layer as shown in FIG. 4, which is located between the full connection layer-FC 4 and a final classification layer, each element in the combined classifier layer is fully connected with each neuron in FC4, so that each feature generated in FC4 can be respectively embedded into three sub-classifiers, each single feature learned in FC4 and each single classifier combination can generate a class label, so that nine class labels can be generated, then, nine kinds of category labels are classified in sequence by adopting a category judgment rule of 'minority obeying majority', so that the final sample category decision can be obtained, and an experiment comparison result relation table of the combined classifier and the single classifier is shown in fig. 6.

3. Deep convolution network structure for optimizing prawn identification

(1) Network layer optimization

The deep convolutional neural network optimization mainly aims at convolutional layers, pooling layers and full-link layers, and the adjustment of the layers explores the influence of the change of a network structure on the feature expression of the prawns in the model. Firstly, fixing other hyper-parameters (learning rate, network clipping rate and sample batch size) unchanged, deleting a first convolution layer (Conv1) and a first Pooling layer (Pooling1), deleting a second convolution layer (Conv2) and a second Pooling layer (Pooling2), deleting other layers only of the first convolution layer (Conv1), deleting only the first Pooling layer (Pooling1), deleting only the second Pooling layer (Pooling2), deleting only the first fully-connected layer (FC3), deleting only the second fully-connected layer (FC4), deleting the first and second fully-connected layers (FC3-4) simultaneously, exploring the influence of changes of model parameters on final recognition rate, and the experimental result is shown in FIG. 7. The network convergence rate and the network convergence duration in fig. 7 can be expressed by the following equations (1.3) and (1.4).

Wherein initial_accRepresenting the accuracy of the first sample training, steady_accRepresenting the training accuracy of the last batch of samples when the network converges; wherein initial_lossRepresents the training of the first sampleLoss value of scouring, steady_lossRepresenting the last batch of sample training loss values when the network converges;

(2) hyper-parametric optimization

The hyper-parameters discussed in the invention mainly refer to Learning rate (Learning rate), network cutting rate (Dropout rate), sample Batch size (Batch-size) and the number of convolution kernels, and the optimization of the four hyper-parameters can improve the recognition rate and the execution speed of the network to a certain extent. The following optimization is performed with other parameters fixed and the number of network layers unchanged.

Learning rate optimization

In the adjustment and optimization of the network hyper-parameters, the optimal learning rate parameters need to be found out through comparison experiments in different learning rate selections, wherein the comparison interval is from 10^-8To 10^-1And comparing 8 groups of numerical values together, and calculating the change rule of the identification accuracy of the prawns when the network converges to the global optimum. The experimental pair is shown in fig. 2.

Network cropping rate optimization

In the adjustment and optimization of network hyper-parameters, the optimal network cutting rate parameter needs to be found out through comparison experiments in different network cutting rate selections, wherein the comparison value range is from 0.01 to 0.99, some isolated points are selected discontinuously, 11 groups of network cutting rate values are compared, and when the network converges to the global optimum, the learning rate is 10^-3，10^-4And 10^-5The experimental pair is shown in fig. 3 by comparing the change rule of the prawn identification accuracy under the condition.

Sample batch size optimization

In the adjustment of the network hyper-parameters, the optimal parameters need to be found out through comparison experiments in different sample batch size selections, wherein the comparison numerical values comprise 7 groups of values including 8, 16, 32, 64, 150, 200 and 256, and the change rule of the network convergence rate and the network convergence duration of the prawn identification is calculated when the network converges to the global optimum. The experimental pair is shown in fig. 4.

Convolution kernel number optimization

In the optimization of the number of convolution kernels, the number of convolution kernels is adjusted for two convolution layers of the network respectively to observe the influence on the modeling data stream and the recognition rate under different parameters, for example, the number of convolution kernels in the first and second layers is adjusted to 4 and 12 respectively, the number of convolution kernels in the first and second layers is adjusted to 32 and 64, the number of convolution kernels in the first and second layers is adjusted to 64 and 128, and the number of convolution kernels in the first and second layers is adjusted to 128 and 256 respectively, the last three groups of parameters are optimized and added with the optimal network clipping numerical operation, so that the network parameters are further optimized, and the aim of improving the test accuracy is fulfilled.

Claims (2)

1. A defective shrimp rapid identification method based on a deep convolutional neural network is characterized by comprising the following steps:

1) improving the first and second convolution layers of the traditional LeNet-5 and amplifying the first and second convolution layers into a three-layer parallel network structure, wherein the step of improving the convolution layers of the traditional LeNet-5 comprises the following steps:

respectively expanding the original LeNet-5 first convolution layer and the original LeNet-5 second convolution layer into a parallel three-layer structure, and establishing a relationship between the first convolution layer and the second convolution layer to generate communication;

distributing convolution kernels in the three layers of amplified parallel networks according to a form from top to bottom according to 1 x 1, 3 x 3 and 5 x 5, sequentially traversing images and capturing effective features;

the convolution steps of the three layers of the amplified parallel networks are distributed according to the form from top to bottom according to 2, 5 and 9, so that the capability of the network for adapting to different types of samples is improved;

2) improving a traditional LeNet-5 classification layer to form a classification combination layer structure, wherein the step of improving the traditional LeNet-5 classification layer comprises the following steps: constructing a combined classifier layer between a second full connection layer (FC4) and a last classification layer of LeNet-5, and respectively integrating the characteristics into SVM, Softmax and Random forest sub-classifiers to obtain sub-labels of corresponding classes;

3) and optimizing and adjusting the improved network structure by taking the minimum parameters and the optimal recognition rate as indexes.

2. The method for rapidly identifying defective shrimps based on the deep convolutional neural network as claimed in claim 1, wherein the step of optimizing and adjusting the improved network structure comprises the following steps:

1) fixing other parameters, respectively adjusting partial convolution layers, pooling layers and full-connection layers of the improved network, and exploring the change of model parameters and the influence on the final recognition rate;

2) fixing other parameters and the number of network layers to be unchanged, adjusting and improving the number of partial convolution kernels in the convolution layers in the network, and exploring and obtaining the number of corresponding convolution kernels under the minimum model parameters and the optimal recognition rate.

Images