CN116071560A - A method of fruit recognition based on convolutional neural network - Google Patents

Abstract

The invention discloses a fruit identification method based on a convolutional neural network, which comprises the following steps: s1: fruit image data acquisition is completed through a camera and a lens; s2: image processing, namely, aiming at the characteristics of the image, completing the work of image preprocessing, image segmentation, morphological processing and connected region feature analysis; s3: constructing a convolutional neural network model, and extracting multi-scale features from the image processing result through convolutional neural network training; s4: after the convolutional neural network training is completed, the data result is sent to the chip through the serial port, and the chip controls the steering engine to enable the mechanical arm to sort fruits; the method has the advantages that the K-means fruit segmentation algorithm is improved, morphological processing and connected region feature analysis are utilized to complete the selection and positioning of a target region, the characteristic of the specific edge implicit extraction of the convolutional neural network is utilized, meanwhile, the characteristic learning is carried out on a sample, the fruit variety identification and classification are realized, and the accurate identification rate of the fruit variety is improved.

Description

A fruit recognition method based on convolutional neural network

Technical Field

The invention belongs to the technical field related to image recognition, and in particular relates to a fruit recognition method based on a convolutional neural network.

Background Art

At present, the mechanization level in the current fruit industry structure in my country is low, and most of the production links, especially fruit picking, mainly rely on time-consuming and labor-intensive manual work. The entire fruit production operation includes picking, storage, transportation, processing and sales. Therefore, the research and development of fruit agricultural production robots is an inevitable trend to improve fruit production efficiency and save labor costs. Whether it is a picking and sorting robot, or a fruit quality and variety detection system in the fruit production link, their normal operation depends on the correct recognition of the fruit by the image processing module. For example, the picking robot can only provide motion parameters for the mechanical arm and complete the fruit picking operation after it recognizes the fruit from the fruit tree and obtains the accurate position of the fruit.

In recent years, with the maturity of machine vision and computer technology, deep learning technology has developed rapidly. It can excellently complete various computer vision tasks. After a large amount of data training, it can automatically learn the characteristic information of different things, obtain the differences between categories, and convert the original data into more abstract and advanced expressions, thereby completing image classification, detection and other tasks, and thus is increasingly widely used in crop growth monitoring; however, the current fruit variety recognition method based on image deep learning has a low accuracy rate and cannot fully meet the requirements of practical applications.

Summary of the invention

The object of the present invention is to provide a fruit recognition method based on a convolutional neural network to solve the problem of low accuracy of the fruit variety recognition method proposed in the above background technology.

To achieve the above object, the present invention provides the following technical solution: a fruit recognition method based on convolutional neural network, comprising the following steps:

S1: Fruit image data collection, which is completed through cameras and lenses;

S2: Image processing, based on the image characteristics, complete image preprocessing, image segmentation, morphological processing and connected area feature analysis;

S3: Construct a convolutional neural network model and perform multi-scale feature extraction on the image processing results through convolutional neural network training;

S4: After completing the convolutional neural network training, the data results are sent to the chip through the serial port, and the chip controls the servo to make the robotic arm sort the fruits.

Preferably, in the step S1, the camera is responsible for collecting the fruit image information, which is mainly selected based on the detection accuracy and the field of view size. The lens parameter is mainly the focal length, and its selection is mainly determined based on the working distance.

Preferably, the image preprocessing in S2 is to extract the R space color feature component of the image according to the characteristics of the fruit image, and perform Gaussian filtering. The Gaussian template is represented by discretizing the two-dimensional Gaussian function. For a matrix M of size (2u+1)×(2u+1), the element at the (a, b) position is represented as:

Where α is the standard deviation of the Gaussian function, and the size of the Gaussian filter template is 5×5.

Preferably, the image segmentation in S2 is to segment the image using an improved K-means clustering segmentation algorithm to form segmented regions. The specific clustering segmentation algorithm is as follows:

Step 1: First, convert the grayscale value of the sample image into a one-dimensional sample data set C of size N, where N is the number of sample image pixels. Assume that the number of iterations is I and the cluster center when the cluster type is j is Z _j (I). Randomly select a sample object from the data set C as the initial cluster center Z ₁ (1);

Calculate the shortest distance d(X _m ) between each sample X m and the existing cluster center, m=1, 2...N, where X _m is the mth sample in the data set; then calculate the probability P(X _m ) of each sample object being selected as the next cluster center:

Select the next cluster center according to the roulette wheel method and repeat the steps until k objects are selected to form the initial cluster center Z _j (1), j = 1, 2, 3, ... k, specifically, the number of cluster centers k = 4;

Step 2: Iterative calculation: Calculate the distance D(X _m ; Z _j (I)) between each sample xm in the sample data set C and the initial cluster according to the similarity criterion, m = 1, 2, 3....N, j = 1, 2, 3...k, as shown in the following formula (5), and divide each data object into the cluster set S _j with the smallest distance, then X _m∈ S _j :

为聚类j的元素，聚类j的元素个数为n_j，聚类中心更新公式如式(6)所示：Step 3: Cluster center update: According to the cluster center update formula, calculate the mean of each cluster set as the new cluster center of the set, and update the new cluster set center.

is the element of cluster j, the number of elements of cluster j is n _j , and the cluster center update formula is shown in formula (6):

Step 4: Termination condition: Update the cluster set centers cyclically until the cluster centers no longer change or the error square sum is locally minimized. The clustering criterion function J is calculated as follows:

The precision error is ξ. If |J(I+1)-J(I)|<ξ, the algorithm ends and the iteration is terminated. Otherwise, the iterative calculation and clustering center are repeatedly performed until the termination condition is met.

Preferably, the morphological processing in S2 is that there is still a lot of noise interference information in the area obtained by image segmentation. In order to remove the noise area, the segmented area is processed using a morphological processing algorithm. Let A represent the image matrix, B represent the structural element, and the morphological processing method is shown in the following formula:

A rectangular structure element with a size of 10×10 is used to open the cluster segmentation area.

Preferably, the connected region feature analysis in S2 is after morphological processing, and the small noise interference area has been eliminated; considering the actual size of the fruit, it is necessary to further eliminate the non-target area; the pixels with the same grayscale and satisfying 8 adjacency are determined as the same area, and the noise interference area is filtered through the connected region area feature, and the area X with the pixel area in (SMin, SMax) is extracted according to the following formula:

Among them, SMn and SMax are the lower and upper limit values of the area pixel parameters respectively.

Preferably, the convolutional neural network model constructed in S3 is as follows:

1. Convolutional layer

将矩阵中输入的每个元素与中间矩阵的每个元素对应相乘并相加.这便是卷积操作；In a convolutional neural network, the first convolution layer directly accepts pixel-level input of the image. Each convolution operation only processes a small piece of the image, and then passes it to the subsequent network after convolution. Each layer of convolution extracts the most effective features of the data. The convolution layer is the core layer of the convolutional neural network, and convolution is the core of the convolution layer. The operation of simple convolution is a simple two-dimensional spatial convolution operation.

Multiply and add each element of the input matrix with each element of the intermediate matrix. This is the convolution operation;

The middle matrix is called the convolution kernel, also called the weight filter or filter. It is the core of the convolution process. It can detect the horizontal and vertical edges of the image, enhance the weight of the central area of the image, etc. When the convolution is extended to a higher dimension, the convolution kernel is used as a moving window on the input matrix to perform the corresponding convolution operation. CNN mines the spatial local correlation information of natural images by strengthening the local connection mode of nodes between adjacent layers in the neural network. The input of the m-th layer node is part of the m-1-th layer node, and these nodes have spatially adjacent visual receptive fields. The number of weights of each neuron in the convolutional neural network is the size of the convolution kernel, that is, each neuron is only connected to part of the pixels of the image. Two convolution layers (Covn1, Covn2) are used, and the convolution kernel is 3×3. The original image can be convolved by the learned convolution kernel. The result after convolution plus the threshold is activated through the activation function to form the feature map output by this layer. The calculation formula is:

Where l is the number of layers, k is the convolution kernel number, _Mj is the jth of M feature maps; f is the activation function; bias is the threshold; the first layer of the convolution layer (convl) uses 64 randomly initialized 3×3×3 convolution kernels to convolve the image with a step size of 1, and the size of the image after convolution is consistent with the original size; Conv2 uses 16 3×3×64 convolution kernels with a step size of 1;

2. Pooling layer

The purpose of the pooling layer is to further reduce the overfitting degree of network training parameters and models. There are usually three ways of pooling: ① Maximum pooling, selecting the maximum value in the pooling window as the sampling value; ② Mean pooling, adding all values in the pooling window and averaging them, and using the average value as the sampling value; ③ Random pooling, using the probability method to determine which one to choose; The pooling layer can reduce the size of the image, improve the operation speed of the entire algorithm and reduce the impact of noise; The pooling layer is mainly for the feature map after convolution, and calculates the mean or maximum value of part of the image area according to the convolution method to represent the sampling area; This is to describe large images and aggregate statistics of features at different positions. This mean or maximum value is the method of aggregate statistics, that is, pooling; The two downsampling layers in the middle use the maximum pooling with a convolution kernel of 3×3 and a step size of 2; Its operation process can be expressed as:

Where g(·) is the downsampling function; f is the activation function;

3. Activation Function

The activation function is used to ensure the nonlinearity of the result. That is, after the convolution operation, the output value is added with an offset and input to the activation function as the input of the next layer. Common activation functions are Sigmoid, Tanh and ReLu. The activation function uses Relu. When x<0, it is hard saturated. When x>0, the derivative is 1, keeping the gradient from decaying, thereby alleviating the gradient vanishing problem. It can converge faster. The expression is:

4. Fully connected layer

The fully connected layer is equivalent to the role of a "classifier", that is, the "distributed feature representation" learned by the convolutional layer, activation function and pooling layer is mapped to the sample label space; the CNN model uses two fully connected layers, each with 128 neurons, and the final output uses a softmax classifier. The Softmax function is a polynomial regression function, which is suitable for solving the recognition problem of multiple categories of images. Dropout is used in the fully connected layer to randomly ignore some neurons to avoid overfitting of the model and enhance the robustness of the model.

Compared with the prior art, the present invention provides a fruit recognition method based on convolutional neural network, which has the following beneficial effects:

The present invention improves the K-means fruit segmentation algorithm, and uses morphological processing and connected region feature analysis to complete the selection and positioning of the target area, thereby realizing the detection and recognition of fruits, determining the area of the fruit, scaling the image with the area as the center point, and adjusting it to a suitable size. At the same time, the unique feature of the convolutional neural network is used to implicitly extract features while learning features of samples. The present invention also reduces the pre-processing work of the program, trains and extracts fruit features, and performs verification tests through fruit images of different test sets, thereby realizing the recognition and classification of fruit varieties and improving the accurate recognition rate of fruit varieties.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

FIG1 is a flow chart of a fruit recognition method based on a convolutional neural network proposed by the present invention;

FIG2 is a flow chart of a detection algorithm of the present invention;

FIG3 is a convolutional neural network model diagram of the present invention;

FIG4 is a schematic diagram of color components in R space;

FIG5 is a schematic diagram of improved K-means segmentation results;

Fig. 6 is a schematic diagram of morphological processing results;

FIG. 7 is a schematic diagram of connected region feature analysis.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Please refer to Figures 1-3. The present invention provides a technical solution: a fruit recognition method based on a convolutional neural network, comprising the following steps:

S1: Fruit image data collection, which is completed through cameras and lenses;

S2: Image processing, based on the image characteristics, complete image preprocessing, image segmentation, morphological processing and connected area feature analysis;

Image acquisition is the premise of image processing. After the fruit image is acquired, in order to improve the image processing speed and the adaptability of the algorithm, the R space color feature components of the image are extracted according to the characteristics of the fruit image, and Gaussian filtering is performed. The Gaussian template is represented by the discretization of the two-dimensional Gaussian function. For a matrix M of size (2u+1)×(2u+1), the elements at the (a, b) position are expressed as:

Where α is the standard deviation of the Gaussian function, the size of the Gaussian filter template is 5×5, and the detection results are shown in Figure 4;

According to the characteristics of the image, the improved K-means clustering segmentation algorithm is used to segment the image to form segmentation areas. The specific clustering segmentation algorithm is as follows:

Step 1: First, convert the grayscale value of the sample image into a one-dimensional sample data set C of size N, where N is the number of sample image pixels. Assume that the number of iterations is I and the cluster center when the cluster type is j is Z _j (I). Randomly select a sample object from the data set C as the initial cluster center Z ₁ (1);

Calculate the shortest distance d(X _m ) between each sample X m and the existing cluster center, m=1, 2...N, where X _m is the mth sample in the data set; then calculate the probability P(X _m ) of each sample object being selected as the next cluster center:

Select the next cluster center according to the roulette wheel method and repeat the steps until k objects are selected to form the initial cluster center Z _j (1), j = 1, 2, 3, ... k, specifically, the number of cluster centers k = 4;

Step 2: Iterative calculation: Calculate the distance D(X _m ; Z _j (I)) between each sample xm in the sample data set C and the initial cluster according to the similarity criterion, m = 1, 2, 3....N, j = 1, 2, 3...k, as shown in the following formula (5), and divide each data object into the cluster set S _j with the smallest distance, then X _m∈ S _j :

为聚类j的元素，聚类j的元素个数为n_j，聚类中心更新公式如式(6)所示：Step 3: Cluster center update: According to the cluster center update formula, calculate the mean of each cluster set as the new cluster center of the set, and update the new cluster set center.

is the element of cluster j, the number of elements of cluster j is n _j , and the cluster center update formula is shown in formula (6):

Step 4: Termination condition: Update the cluster set centers cyclically until the cluster centers no longer change or the error square sum is locally minimized. The clustering criterion function J is calculated as follows:

The precision error is ξ. If |J(I+1)-J(I)|<ξ, the algorithm ends and the iteration is terminated. Otherwise, the iterative calculation and clustering center are repeatedly performed until the termination condition is met. The image segmentation result is shown in Figure 5.

Morphological processing is a process in which there is still a lot of noise interference information in the area obtained by image segmentation. In order to remove the noise area, the segmented area is processed using the morphological processing algorithm. Let A represent the image matrix and B represent the structural element. The morphological processing method is shown in the following formula:

A rectangular structural element with a size of 10×10 is used to open the cluster segmentation area. The image processing result is shown in Figure 6;

The connected region feature analysis is that after morphological processing, the small noise interference area has been removed; considering the actual size of the fruit, it is necessary to further remove the non-target area; the pixels with the same grayscale and 8 adjacent pixels are judged as the same area, and the noise interference area is filtered out through the connected region area feature. The area X with the pixel area in (SMin, SMax) is extracted according to the following formula:

Where SMn and SMax are the lower and upper limits of the area pixel parameters, respectively. The image processing result is shown in Figure 7;

S3: Construct a convolutional neural network model and perform multi-scale feature extraction on the image processing results through convolutional neural network training;

S4: After completing the convolutional neural network training, the data results are sent to the chip through the serial port, and the chip controls the servo to make the robotic arm sort the fruits.

Preferably, in step S1, the camera is responsible for collecting fruit image information, which is mainly selected based on detection accuracy and field of view size. The lens parameter is mainly focal length, and its selection is mainly determined based on working distance.

As shown in Figure 3, the convolutional neural network model in S3 is as follows:

1. Convolutional layer

将矩阵中输入的每个元素与中间矩阵的每个元素对应相乘并相加.这便是卷积操作；In a convolutional neural network, the first convolution layer directly accepts pixel-level input of the image. Each convolution operation only processes a small piece of the image, and then passes it to the subsequent network after convolution. Each layer of convolution extracts the most effective features of the data. The convolution layer is the core layer of the convolutional neural network, and convolution is the core of the convolution layer. The operation of simple convolution is a simple two-dimensional spatial convolution operation.

Multiply and add each element of the input matrix with each element of the intermediate matrix. This is the convolution operation;

The middle matrix is called the convolution kernel, also called the weight filter or filter. It is the core of the convolution process. It can detect the horizontal and vertical edges of the image, enhance the weight of the central area of the image, etc. When the convolution is extended to a higher dimension, the convolution kernel is used as a moving window on the input matrix to perform the corresponding convolution operation. CNN mines the spatial local correlation information of natural images by strengthening the local connection mode of nodes between adjacent layers in the neural network. The input of the m-th layer node is part of the m-1-th layer node, and these nodes have spatially adjacent visual receptive fields. The number of weights of each neuron in the convolutional neural network is the size of the convolution kernel, that is, each neuron is only connected to part of the pixels of the image. Two convolution layers (Covn1, Covn2) are used, and the convolution kernel is 3×3. The original image can be convolved by the learned convolution kernel. The result after convolution plus the threshold is activated through the activation function to form the feature map output by this layer. The calculation formula is:

Where l is the number of layers, k is the convolution kernel number, _Mj is the jth of M feature maps; f is the activation function; bias is the threshold; the first layer of the convolution layer (convl) uses 64 randomly initialized 3×3×3 convolution kernels to convolve the image with a step size of 1, and the size of the image after convolution is consistent with the original size; Conv2 uses 16 3×3×64 convolution kernels with a step size of 1;

2. Pooling layer

The purpose of the pooling layer is to further reduce the overfitting degree of network training parameters and models. There are usually three ways of pooling: ① Maximum pooling, selecting the maximum value in the pooling window as the sampling value; ② Mean pooling, adding all values in the pooling window and averaging them, and using the average value as the sampling value; ③ Random pooling, using the probability method to determine which one to choose; The pooling layer can reduce the size of the image, improve the operation speed of the entire algorithm and reduce the impact of noise; The pooling layer is mainly for the feature map after convolution, and calculates the mean or maximum value of part of the image area according to the convolution method to represent the sampling area; This is to describe large images and aggregate statistics of features at different positions. This mean or maximum value is the method of aggregate statistics, that is, pooling; The two downsampling layers in the middle use the maximum pooling with a convolution kernel of 3×3 and a step size of 2; Its operation process can be expressed as:

Where g(·) is the downsampling function; f is the activation function;

3. Activation Function

The activation function is used to ensure the nonlinearity of the result. That is, after the convolution operation, the output value is added with an offset and input to the activation function as the input of the next layer. Common activation functions are Sigmoid, Tanh and ReLu. The activation function uses Relu. When x<0, it is hard saturated. When x>0, the derivative is 1, keeping the gradient from decaying, thereby alleviating the gradient vanishing problem. It can converge faster. The expression is:

4. Fully connected layer

The fully connected layer is equivalent to the role of a "classifier", that is, the "distributed feature representation" learned by the convolutional layer, activation function and pooling layer is mapped to the sample label space; the CNN model uses two fully connected layers, each with 128 neurons, and the final output uses a softmax classifier. The Softmax function is a polynomial regression function, which is suitable for solving the recognition problem of multiple categories of images. Dropout is used in the fully connected layer to randomly ignore some neurons to avoid overfitting of the model and enhance the robustness of the model.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims (7)

1. A fruit recognition method based on convolutional neural network, characterized in that it comprises the following steps: S1: Fruit image data collection, which is completed through cameras and lenses; S2: Image processing, based on the image characteristics, complete image preprocessing, image segmentation, morphological processing and connected area feature analysis; S3: Construct a convolutional neural network model and perform multi-scale feature extraction on the image processing results through convolutional neural network training; S4: After completing the convolutional neural network training, the data results are sent to the chip through the serial port, and the chip controls the servo to make the robotic arm sort the fruits. 2. A fruit recognition method based on convolutional neural network according to claim 1, characterized in that: in the step of S1, the camera is responsible for the collection of fruit image information, which is mainly selected based on detection accuracy and visual field size, and the lens parameter is mainly focal length, which is mainly determined based on working distance. 3. A fruit recognition method based on convolutional neural network according to claim 1, characterized in that: the image preprocessing in S2 is to extract the R space color feature component of the image according to the characteristics of the fruit image, and perform Gaussian filtering. The Gaussian template is represented by discretization of the two-dimensional Gaussian function. For a matrix M of size (2u+1)×(2u+1), the element at the (a, b) position is represented as:

Where α is the standard deviation of the Gaussian function, and the size of the Gaussian filter template is 5×5. 4. a kind of fruit identification method based on convolutional neural network according to claim 3, it is characterized in that: the image segmentation among the described S2 is to utilize improved K mean value cluster segmentation algorithm that image is segmented, forms segmentation area, and concrete cluster segmentation algorithm is as follows: Step 1: First, convert the grayscale value of the sample image into a one-dimensional sample data set C of size N, where N is the number of sample image pixels. Assume that the number of iterations is I and the cluster center when the cluster type is j is Zj(I). Randomly select a sample object from the data set C as the initial cluster center Z1(1); Calculate the shortest distance d(Xm) between each sample Xm and the existing cluster center, m=1, 2...N, where Xm is the mth sample in the data set; then calculate the probability P(Xm) of each sample object being selected as the next cluster center:

Select the next cluster center according to the roulette wheel method and repeat the steps until k objects are selected to form the initial cluster center Zj(1), j=1, 2, 3, ...k, specifically, the number of cluster centers k=4; Step 2: Iterative calculation: According to the similarity criterion, the distance D(Xm; Zj(I)) between each sample xm in the sample data set C and the initial cluster is calculated, m = 1, 2, 3....N, j = 1, 2, 3...k, as shown in the following formula (5), and each data object is divided into the cluster set Sj with the smallest distance, then Xm∈Sj:

Step 3: Cluster center update: According to the cluster center update formula, calculate the mean of each cluster set as the new cluster center of the set, and update the new cluster set center.

is the element of cluster j, the number of elements in cluster j is nj, and the cluster center update formula is shown in formula (6):

Step 4: Termination condition: Update the cluster set centers cyclically until the cluster centers no longer change or the error square sum is locally minimized. The clustering criterion function J is calculated as follows:

The precision error is ξ. If |J(I+1)-J(I)|<ξ, the algorithm ends and the iteration is terminated. Otherwise, the iterative calculation and clustering center are repeatedly performed until the termination condition is met. 5. a kind of fruit identification method based on convolutional neural network according to claim 4, it is characterized in that: the morphological treatment among the described S2 is that the region that obtains through image segmentation also has a lot of noise interference information, in order to eliminate the noise area, utilize the morphological treatment algorithm that the segmented region is processed, establish A to represent image matrix, B represents structural element, the morphological treatment method is shown as follows:

A rectangular structure element with a size of 10×10 is used to open the cluster segmentation area. 6. a kind of fruit identification method based on convolutional neural network according to claim 5, it is characterized in that: the connected region feature analysis among described S2 is after morphological treatment, and tiny noise interference region has been eliminated; Consider the actual size of fruit, need to further eliminate non-target area; Gray scale is consistent, and satisfy 8 adjacent pixels to be judged as same area, by connected region area feature, filter noise interference region, extract pixel area in (SMin, SMax) region X according to following formula,

Among them, SMn and SMax are the lower and upper limit values of the area pixel parameters respectively. 7. A fruit identification method based on convolutional neural network according to claim 1, characterized in that: the convolutional neural network model constructed in S3 is as follows: 1. Convolutional layer In a convolutional neural network, the first convolution layer directly accepts pixel-level input of the image. Each convolution operation only processes a small piece of the image, which is then passed to the subsequent network after convolution. Each convolution layer extracts the most effective features of the data. The convolution layer is the core layer of the convolutional neural network, and convolution is the core of the convolution layer. The operation of simple convolution is a simple two-dimensional spatial convolution operation.

Multiply and add each element of the input matrix with each element of the intermediate matrix. This is the convolution operation; The middle matrix is called the convolution kernel, also called the weight filter or filter. It is the core of the convolution process. It can detect the horizontal and vertical edges of the image, enhance the weight of the central area of the image, etc. When the convolution is extended to a higher dimension, the convolution kernel is used as a moving window on the input matrix to perform the corresponding convolution operation. CNN mines the spatial local correlation information of natural images by strengthening the local connection mode of nodes between adjacent layers in the neural network. The input of the m-th layer node is part of the m-1-th layer node, and these nodes have spatially adjacent visual receptive fields. The number of weights of each neuron in the convolutional neural network is the size of the convolution kernel, that is, each neuron is only connected to part of the pixels of the image. Two convolution layers (Covn1, Covn2) are used, and the convolution kernel is 3×3. The original image can be convolved by the learned convolution kernel. The result after convolution plus the threshold is activated through the activation function to form the feature map output by this layer. The calculation formula is:

Where l is the number of layers, k is the convolution kernel number, Mj is the jth of M feature maps; f is the activation function; bias is the threshold; the first layer of the convolution layer (convl) uses 64 randomly initialized 3×3×3 convolution kernels to convolve the image with a step size of 1, and the size of the image after convolution is consistent with the original size; Conv2 uses 16 3×3×64 convolution kernels with a step size of 1; 2. Pooling layer The purpose of the pooling layer is to further reduce the overfitting degree of network training parameters and models. There are usually three ways of pooling: ① Maximum pooling, selecting the maximum value in the pooling window as the sampling value; ② Mean pooling, adding all values in the pooling window and averaging them, and using the average value as the sampling value; ③ Random pooling, using the probability method to determine which one to choose; The pooling layer can reduce the size of the image, improve the operation speed of the entire algorithm and reduce the impact of noise; The pooling layer is mainly for the feature map after convolution, and calculates the mean or maximum value of part of the image area according to the convolution method to represent the sampling area; This is to describe large images and aggregate statistics of features at different positions. This mean or maximum value is the method of aggregate statistics, that is, pooling; The two downsampling layers in the middle use the maximum pooling with a convolution kernel of 3×3 and a step size of 2; Its operation process can be expressed as:

Where g(·) is the downsampling function; f is the activation function; 3. Activation Function The activation function is used to ensure the nonlinearity of the result. That is, after the convolution operation, the output value is added with an offset and input to the activation function as the input of the next layer. Common activation functions are Sigmoid, Tanh and ReLu. The activation function uses Relu. When x<0, it is hard saturated. When x>0, the derivative is 1, keeping the gradient from decaying, thereby alleviating the gradient vanishing problem. It can converge faster. The expression is:

4. Fully connected layer The fully connected layer is equivalent to the role of a "classifier", that is, the "distributed feature representation" learned by the convolutional layer, activation function and pooling layer is mapped to the sample label space; the CNN model uses two fully connected layers, each with 128 neurons, and the final output uses a softmax classifier. The Softmax function is a polynomial regression function, which is suitable for solving the recognition problem of multiple categories of images. Dropout is used in the fully connected layer to randomly ignore some neurons to avoid overfitting of the model and enhance the robustness of the model.

Images