CN116071560A - Fruit identification method based on convolutional neural network - Google Patents
Fruit identification method based on convolutional neural network Download PDFInfo
- Publication number
- CN116071560A CN116071560A CN202211452782.3A CN202211452782A CN116071560A CN 116071560 A CN116071560 A CN 116071560A CN 202211452782 A CN202211452782 A CN 202211452782A CN 116071560 A CN116071560 A CN 116071560A
- Authority
- CN
- China
- Prior art keywords
- image
- convolution
- neural network
- layer
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 59
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000012545 processing Methods 0.000 claims abstract description 31
- 230000000877 morphologic effect Effects 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 13
- 230000011218 segmentation Effects 0.000 claims abstract description 12
- 238000004458 analytical method Methods 0.000 claims abstract description 10
- 238000003709 image segmentation Methods 0.000 claims abstract description 9
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 239000010410 layer Substances 0.000 claims description 81
- 230000006870 function Effects 0.000 claims description 48
- 238000011176 pooling Methods 0.000 claims description 36
- 230000004913 activation Effects 0.000 claims description 27
- 238000004364 calculation method Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 12
- 210000002569 neuron Anatomy 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 claims description 3
- 230000002238 attenuated effect Effects 0.000 claims description 3
- 239000012792 core layer Substances 0.000 claims description 3
- 230000008034 disappearance Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 238000004891 communication Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 2
- 238000012271 agricultural production Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a fruit identification method based on a convolutional neural network, which comprises the following steps: s1: fruit image data acquisition is completed through a camera and a lens; s2: image processing, namely, aiming at the characteristics of the image, completing the work of image preprocessing, image segmentation, morphological processing and connected region feature analysis; s3: constructing a convolutional neural network model, and extracting multi-scale features from the image processing result through convolutional neural network training; s4: after the convolutional neural network training is completed, the data result is sent to the chip through the serial port, and the chip controls the steering engine to enable the mechanical arm to sort fruits; the method has the advantages that the K-means fruit segmentation algorithm is improved, morphological processing and connected region feature analysis are utilized to complete the selection and positioning of a target region, the characteristic of the specific edge implicit extraction of the convolutional neural network is utilized, meanwhile, the characteristic learning is carried out on a sample, the fruit variety identification and classification are realized, and the accurate identification rate of the fruit variety is improved.
Description
Technical Field
The invention belongs to the technical field of image recognition, and particularly relates to a fruit recognition method based on a convolutional neural network.
Background
At present, the degree of mechanization in the existing fruit industry structure in China is low, and most production links, especially fruit picking, are mainly carried out by labor and time consuming. The whole fruit production operation comprises picking, storing, transporting, processing, selling and other links, so that the research and development of the fruit agricultural production robot is a necessary trend of improving the fruit production efficiency and saving the labor cost. The normal operation of the picking robot and the fruit quality and variety detection system in the fruit production link depends on the correct identification of the fruit by the image processing module, for example, the picking robot can provide motion parameters for the mechanical arm only by identifying the fruit from the fruit tree and acquiring the accurate position of the fruit, so that the picking operation of the fruit is completed.
In recent years, with the maturity of machine vision and computer technology, the deep learning technology is rapidly developed, can excellently complete various computer vision tasks, automatically learn characteristic information of different things through a large amount of data training, acquire differences of various categories, convert original data into more abstract and high-level expression, further complete tasks such as image classification, detection and the like, and therefore, the deep learning technology is more widely applied to crop growth monitoring; however, the accuracy of the fruit variety identification method based on image deep learning is low at present, and the requirements of practical application cannot be completely met.
Disclosure of Invention
The invention aims to provide a fruit identification method based on a convolutional neural network, which aims to solve the problem of low accuracy of the fruit variety identification method proposed in the background art.
In order to achieve the above purpose, the present invention provides the following technical solutions: a fruit identification method based on convolutional neural network comprises the following steps:
s1: fruit image data acquisition is completed through a camera and a lens;
s2: image processing, namely, aiming at the characteristics of the image, completing the work of image preprocessing, image segmentation, morphological processing and connected region feature analysis;
s3: constructing a convolutional neural network model, and extracting multi-scale features from the image processing result through convolutional neural network training;
s4: after the convolutional neural network training is completed, the data result is sent to the chip through the serial port, and the chip controls the steering engine to enable the mechanical arm to sort fruits.
Preferably, in the step S1, the camera is responsible for collecting fruit image information, and the camera is mainly selected according to the detection precision and the view field size, and the lens parameter is mainly the focal length, and the selection is mainly determined according to the working distance.
Preferably, the image preprocessing in S2 is to extract R space color feature components of the image according to the characteristics of the fruit image, perform gaussian filtering, and the gaussian template is represented by discretizing a two-dimensional gaussian function, and for a matrix M with a size of (2u+1) × (2u+1), the elements at the (a, b) positions are represented as:
where α is the standard deviation of the gaussian function and the gaussian filter template size is 5 x 5.
Preferably, the image segmentation in S2 is to segment the image by using an improved K-means clustering segmentation algorithm to form segmented regions, and the specific clustering segmentation algorithm is as follows:
stepl: firstly, converting the gray value of a sample image into a one-dimensional sample data set C with the size of N, wherein the number of pixels of the N sample images is overlappedThe clustering center is Z when the generation operation times are I and the clustering type is j j (I) Randomly selecting a sample object from the data set C as an initial clustering center Z 1 (1)；
Calculating the shortest distance d (X) between each sample Xm and the existing cluster center m ) M=1, 2..n, wherein X m Is the mth sample in the dataset; then calculate the probability P (X) that each sample object is selected as the next cluster center m )：
Selecting the next clustering center according to the wheel disc method, and repeating until k objects are selected to form an initial clustering center Z j (1) J=1, 2,3,..k, specifically, the number of cluster centers k=4;
step2: iterative calculation: calculating the distance D (X) of each sample xm in the sample data set C from the initial cluster according to the similarity criterion m ；Z j (I) M=1, 2, 3..n, j=1, 2,3 … k, as shown in the following formula (5), each data object is divided into cluster clusters S having the smallest distance j In (B), then X m∈ S j ：
Step3: cluster center update: calculating the mean value in each cluster set as a new cluster center of the set according to a cluster center updating formula, updating to obtain a new cluster set center, and setting
The number of elements of the cluster j is n j The cluster center update formula is shown in formula (6):
step4: termination condition: and circularly updating the cluster set centers until each cluster center is not changed or the square sum of errors is locally minimum, wherein the calculation method of the cluster criterion function J is as follows:
and the precision error is xi, if the absolute value of J (I+1) -J (I) is < xi, the algorithm is ended, the iteration is terminated, and otherwise, the iteration calculation and the clustering center are repeatedly executed until the termination condition is met.
Preferably, in the morphological processing in S2, the region obtained by dividing the image has much noise interference information, and in order to remove the noise region, the divided region is processed by using a morphological processing algorithm, where a represents an image matrix, B represents a structural element, and the morphological processing method is as follows:
the cluster segmentation region is opened using a rectangular structural element of size 10 x 10.
Preferably, the feature analysis of the connected region in S2 is that after morphological processing, a tiny noise interference region is removed; considering the actual size of the fruit, the non-target area needs to be further removed; pixels satisfying 8 adjacency with uniform gray scale are judged as the same area, noise interference area is filtered through the area characteristic of the connected area, the area X of the pixel area (SMin, SMax) is extracted according to the following formula,
wherein SMn and SMax are the lower and upper values of the parameters of the area pixel, respectively.
Preferably, the construction convolutional neural network model in S3 is as follows:
1. convolutional layer
In convolutional neural networks, the first convolutional layer directly accepts the graphEach convolution operation only processes a small image, changes convolution and then transmits the image to a later network, and each layer of convolution can extract the most effective characteristic in the data; the convolution layer is the core layer of the convolution neural network, and the convolution is the core of the convolution layer, and the operation of simple convolution is a simple two-dimensional space convolution operation
This is a convolution operation;
the middle matrix is called a convolution kernel, also called a weight filter or a filter (filter), which is the core of the convolution process and can detect the horizontal edge, the vertical edge, the weight of the central region of the enhanced picture, and the like of the image; when the convolution is generalized to a higher dimension, the convolution kernel is used as a moving window on the input matrix, so that corresponding convolution operation can be performed; the CNN in the layer digs the space local association information of the natural image by enhancing the local connection mode of the nodes between adjacent layers in the neural network, the man-conveying of the nodes in the layer m is a part of the nodes in the layer m-1, and the nodes have the space adjacent visual receptive fields; the weight number of each neuron in the convolutional neural network is the size of a convolutional kernel, namely, each neuron is only connected with a part of pixels of the picture; by adopting two convolution layers (Covnl, covn 2), the convolution kernel 3 multiplied by 3 can be used for carrying out convolution operation on an original image by a learned convolution check, and a characteristic diagram output by the layer is formed by the action of an activation function of a convolved result and a threshold value, wherein the calculation formula is as follows:
where l is the number of layers, k is the number of convolution kernels, M j The j-th feature map is M feature maps; f is an activation function; bias is a threshold; a first layer (convl) of the convolution layer adopts 64 convolution check pictures of 3 multiplied by 3 to carry out convolution by random initialization, the step length is 1, and the size of the graph after the convolution is consistent with the original size; the Conv2 uses 16 convolution kernels of 3×3×64 with a step size of1；
2. Pooling layer
The purpose of the pooling layer is to further reduce the overfitting degree of the network training parameters and the model, and the pooling mode is generally 3 modes: (1) maximum pooling, and selecting the maximum value in a pooling window as a sampling value; (2) averaging, namely adding all values in a pooling window to average, and taking the average value as a sampling value; (3) randomly pooling, and determining which item to select by a probability method; the pooling layer can reduce the size of the image, improve the operation speed of the whole algorithm and reduce the influence of noise; the pooling layer is mainly used for solving an average value or a maximum value of a partial region of the image according to a convolution method aiming at the convolved feature map and representing a sampled region of the image; this is to describe a large image, aggregate statistics is performed on features at different locations, and this mean or maximum is the method of aggregate statistics, i.e. pooling; the two lower sampling layers adopt the maximum pooling with convolution kernel of 3 multiplied by 3 and step length of 2; the operation process can be expressed as follows:
wherein g (·) is a downsampling function; f is an activation function;
3. activation function
The activation function is used for guaranteeing the nonlinearity of the result, namely, after convolution operation, the output value is additionally added with offset, and the offset is input to the activation function to serve as the input of the next layer; common activation functions are Sigmoid, tanh and ReLu; the activation function is Relu, when x <0, hard saturation, when x >0, the derivative is 1, and the gradient is kept not to be attenuated, so that the problem of gradient disappearance is relieved.
4. Full connection layer
The full connection layer is equivalent to the function of a classifier; that is, the "distributed feature representation" of the convolution layer, activation function, and pooling layer chemistry is mapped to the sample label space; the CNN model of (2) adopts two fully-connected layers, the number of neurons in each layer is 128, and finally, the output adopts a Softmax classifier, and the Softmax function is a polynomial regression function, so that the recognition problem of multiple types of pictures is solved, and a part of neurons are randomly ignored by using Dropout in the fully-connected layers, so that the overfitting of the model is avoided, and the robustness of the model is enhanced.
Compared with the prior art, the fruit identification method based on the convolutional neural network has the following beneficial effects:
according to the invention, the K-means fruit segmentation algorithm is improved, the morphological processing and the connected region feature analysis are utilized to complete the selection and positioning of the target region, the detection and recognition of fruits are realized, the region of the fruits is determined, the image is scaled and regulated to a proper size by taking the region as a center point, the characteristic learning is carried out on the sample while the characteristic is extracted by utilizing the characteristic of the unique edge implicit expression of the convolutional neural network, the training and extraction of the fruit characteristics are also carried out by reducing the processing work in the early stage of the program, the fruit variety recognition and classification are realized by carrying out verification test on the fruit images of different test sets, and the accurate recognition rate of the fruit variety is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate and together with the embodiments of the invention and do not constitute a limitation to the invention, and in which:
fig. 1 is a flowchart of a fruit identification method based on convolutional neural network according to the present invention;
FIG. 2 is a flow chart of a detection algorithm of the present invention;
FIG. 3 is a model diagram of a convolutional neural network of the present invention;
FIG. 4 is a schematic diagram of R-space color components;
FIG. 5 is a schematic diagram of an improved K-means segmentation result;
FIG. 6 is a morphology a schematic diagram of a processing result;
fig. 7 is a schematic diagram of connected region feature analysis.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1-3, the present invention provides a technical solution: the fruit identification method based on the convolutional neural network comprises the following steps:
s1: fruit image data acquisition is completed through a camera and a lens;
s2: image processing, namely, aiming at the characteristics of the image, completing the work of image preprocessing, image segmentation, morphological processing and connected region feature analysis;
the image acquisition is a precondition of image processing, after fruit image acquisition, in order to improve the image processing speed and the self-adaptability of an algorithm, aiming at the characteristics of the fruit image, R space color characteristic components of the image are extracted, gaussian filtering processing is carried out, a Gaussian template is represented by discretizing a two-dimensional Gaussian function, and elements of the (a, b) positions of a matrix M with the size of (2u+1) x (2u+1) are represented as follows:
wherein alpha is the standard deviation of a Gaussian function, the size of a Gaussian filtering template is 5 multiplied by 5, and the detection result is shown in figure 4;
aiming at the characteristics of the image, the image is segmented by utilizing an improved K-means clustering segmentation algorithm to form segmented areas, wherein the specific clustering segmentation algorithm is as follows:
stepl: firstly, converting a gray value of a sample image into a one-dimensional sample data set C with the size of N, wherein the number of pixels of the N sample images is set to be Z when the iterative operation frequency is I and the cluster type is j j (I) Randomly selecting a sample object from the data set C as the initialInitial cluster center Z 1 (1)；
Calculating the shortest distance d (X) between each sample Xm and the existing cluster center m ) M=1, 2..n, wherein X m Is the mth sample in the dataset; then calculate the probability P (X) that each sample object is selected as the next cluster center m )：
Selecting the next clustering center according to the wheel disc method, and repeating until k objects are selected to form an initial clustering center Z j (1) J=1, 2,3,..k, specifically, the number of cluster centers k=4;
step2: iterative calculation: calculating the distance D (X) of each sample xm in the sample data set C from the initial cluster according to the similarity criterion m ；Z j (I) M=1, 2, 3..n, j=1, 2,3 … k, as shown in the following formula (5), each data object is divided into cluster clusters S having the smallest distance j In (B), then X m∈ S j ：
Step3: cluster center update: calculating the mean value in each cluster set as a new cluster center of the set according to a cluster center updating formula, updating to obtain a new cluster set center, and setting
The number of elements of the cluster j is n j The cluster center update formula is shown in formula (6):
step4: termination condition: and circularly updating the cluster set centers until each cluster center is not changed or the square sum of errors is locally minimum, wherein the calculation method of the cluster criterion function J is as follows:
and if the precision error is xi, if the absolute value of J (I+1) -J (I) | < xi, ending the algorithm, and ending the iteration, otherwise, repeatedly executing the iterative calculation and the clustering center until the ending condition is met, wherein the image segmentation result is shown in figure 5.
In the morphological processing, a region obtained by image segmentation has much noise interference information, in order to remove the noise region, the segmented region is processed by a morphological processing algorithm, and a represents an image matrix and B represents a structural element, and the morphological processing method is shown as follows:
performing an opening operation on the clustering segmentation areas by adopting rectangular structural elements with the size of 10 multiplied by 10, wherein the image processing result is shown in fig. 6;
the feature analysis of the communication area is that after morphological treatment, the tiny noise interference area is removed; considering the actual size of the fruit, the non-target area needs to be further removed; pixels satisfying 8 adjacency with uniform gray scale are judged as the same area, noise interference area is filtered through the area characteristic of the connected area, the area X of the pixel area (SMin, SMax) is extracted according to the following formula,
wherein SMn and SMax are the lower and upper values of the parameters of the area pixel, respectively, and the image processing result is shown in fig. 7;
s3: constructing a convolutional neural network model, and extracting multi-scale features from the image processing result through convolutional neural network training;
s4: after the convolutional neural network training is completed, the data result is sent to the chip through the serial port, and the chip controls the steering engine to enable the mechanical arm to sort fruits.
Preferably, in the step S1, the camera is responsible for collecting fruit image information, and is mainly selected according to detection accuracy and visual field size, and the lens parameters are mainly focal lengths, and the selection is mainly determined according to working distances.
As shown in fig. 3, the convolutional neural network model is constructed in S3 as follows:
1. convolutional layer
In the convolutional neural network, a first convolutional layer directly receives the input of an image pixel level, each convolutional operation only processes a small image, the small image is transmitted to a later network after convolutional change, and each layer of convolutional can extract the most effective characteristic in data; the convolution layer is the core layer of the convolution neural network, and the convolution is the core of the convolution layer, and the operation of simple convolution is a simple two-dimensional space convolution operation
This is a convolution operation;
the middle matrix is called a convolution kernel, also called a weight filter or a filter (filter), which is the core of the convolution process and can detect the horizontal edge, the vertical edge, the weight of the central region of the enhanced picture, and the like of the image; when the convolution is generalized to a higher dimension, the convolution kernel is used as a moving window on the input matrix, so that corresponding convolution operation can be performed; the CNN in the layer digs the space local association information of the natural image by enhancing the local connection mode of the nodes between adjacent layers in the neural network, the man-conveying of the nodes in the layer m is a part of the nodes in the layer m-1, and the nodes have the space adjacent visual receptive fields; the weight number of each neuron in the convolutional neural network is the size of a convolutional kernel, namely, each neuron is only connected with a part of pixels of the picture; by adopting two convolution layers (Covnl, covn 2), the convolution kernel 3 multiplied by 3 can be used for carrying out convolution operation on an original image by a learned convolution check, and a characteristic diagram output by the layer is formed by the action of an activation function of a convolved result and a threshold value, wherein the calculation formula is as follows:
where l is the number of layers, k is the number of convolution kernels, M j The j-th feature map is M feature maps; f is an activation function; bias is a threshold; a first layer (convl) of the convolution layer adopts 64 convolution check pictures of 3 multiplied by 3 to carry out convolution by random initialization, the step length is 1, and the size of the graph after the convolution is consistent with the original size; 16 convolution kernels of 3×3×64 are adopted in Conv2, and the step size is 1;
2. pooling layer
The purpose of the pooling layer is to further reduce the overfitting degree of the network training parameters and the model, and the pooling mode is generally 3 modes: (1) maximum pooling, and selecting the maximum value in a pooling window as a sampling value; (2) averaging, namely adding all values in a pooling window to average, and taking the average value as a sampling value; (3) randomly pooling, and determining which item to select by a probability method; the pooling layer can reduce the size of the image, improve the operation speed of the whole algorithm and reduce the influence of noise; the pooling layer is mainly used for solving an average value or a maximum value of a partial region of the image according to a convolution method aiming at the convolved feature map and representing a sampled region of the image; this is to describe a large image, aggregate statistics is performed on features at different locations, and this mean or maximum is the method of aggregate statistics, i.e. pooling; the two lower sampling layers adopt the maximum pooling with convolution kernel of 3 multiplied by 3 and step length of 2; the operation process can be expressed as follows:
wherein g (·) is a downsampling function; f is an activation function;
3. activation function
The activation function is used for guaranteeing the nonlinearity of the result, namely, after convolution operation, the output value is additionally added with offset, and the offset is input to the activation function to serve as the input of the next layer; common activation functions are Sigmoid, tanh and ReLu; the activation function is Relu, when x <0, hard saturation, when x >0, the derivative is 1, and the gradient is kept not to be attenuated, so that the problem of gradient disappearance is relieved.
4. Full connection layer
The full connection layer is equivalent to the function of a classifier; that is, the "distributed feature representation" of the convolution layer, activation function, and pooling layer chemistry is mapped to the sample label space; the CNN model of (2) adopts two fully-connected layers, the number of neurons in each layer is 128, and finally, the output adopts a Softmax classifier, and the Softmax function is a polynomial regression function, so that the recognition problem of multiple types of pictures is solved, and a part of neurons are randomly ignored by using Dropout in the fully-connected layers, so that the overfitting of the model is avoided, and the robustness of the model is enhanced.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. The fruit identification method based on the convolutional neural network is characterized by comprising the following steps of:
s1: fruit image data acquisition is completed through a camera and a lens;
s2: image processing, namely, aiming at the characteristics of the image, completing the work of image preprocessing, image segmentation, morphological processing and connected region feature analysis;
s3: constructing a convolutional neural network model, and extracting multi-scale features from the image processing result through convolutional neural network training;
s4: after the convolutional neural network training is completed, the data result is sent to the chip through the serial port, and the chip controls the steering engine to enable the mechanical arm to sort fruits.
2. The fruit identification method based on convolutional neural network according to claim 1, wherein: in the step S1, the camera is responsible for collecting fruit image information, and is mainly selected according to detection precision and visual field size, and the lens parameters are mainly focal lengths, and the selection is mainly determined according to working distances.
3. The fruit identification method based on convolutional neural network according to claim 1, wherein: the image preprocessing in S2 is to extract R space color feature components of the image according to the characteristics of the fruit image, perform gaussian filtering processing, and express the gaussian template by discretizing a two-dimensional gaussian function, and express the element of the (a, b) position of a matrix M with the size of (2u+1) × (2u+1) as follows:
where α is the standard deviation of the gaussian function and the gaussian filter template size is 5 x 5.
4. A fruit identification method based on convolutional neural network as claimed in claim 3, wherein: the image segmentation in the S2 is to segment the image by using an improved K-means clustering segmentation algorithm to form segmented areas, wherein the specific clustering segmentation algorithm is as follows:
stepl: firstly, converting a gray value of a sample image into a one-dimensional sample data set C with the size of N, wherein the number of pixels of the N sample images is set as Zj (I) when the iterative operation times are I and the clustering type is j, and randomly selecting a sample object from the data set C as an initial clustering center Z1 (1);
calculating the shortest distance d (Xm) of each sample Xm to the existing cluster center, m=1, 2..n, wherein Xm is the m-th sample in the data set; the probability P (Xm) that each sample object is selected as the next cluster center is then calculated:
selecting the next clustering center according to a wheel disc method, repeating the steps until k objects are selected to form an initial clustering center Zj (1), j=1, 2,3,..k, and specifically, the number of the clustering centers k=4;
step2: iterative calculation: calculating the distance D (Xm; zj (I)) between each sample Xm in the sample data set C and the initial cluster according to the similarity criterion, wherein m=1, 2, 3..N, j=1, 2,3 … k, and dividing each data object into cluster clusters Sj with the smallest distance as shown in the following formula (5), wherein Xm epsilon Sj:
step3: cluster center update: calculating the mean value in each cluster set as a new cluster center of the set according to a cluster center updating formula, updating to obtain a new cluster set center, and setting
For the elements of the cluster j, the number of the elements of the cluster j is nj, and the update formula of the cluster center is shown in the formula (6): />
Step4: termination condition: and circularly updating the cluster set centers until each cluster center is not changed or the square sum of errors is locally minimum, wherein the calculation method of the cluster criterion function J is as follows:
and the precision error is xi, if the absolute value of J (I+1) -J (I) is < xi, the algorithm is ended, the iteration is terminated, and otherwise, the iteration calculation and the clustering center are repeatedly executed until the termination condition is met.
5. The fruit identification method based on convolutional neural network of claim 4, wherein: in the morphological processing in S2, the region obtained by image segmentation has much noise interference information, in order to remove the noise region, the segmented region is processed by using a morphological processing algorithm, let a denote an image matrix, B denote structural elements, and the morphological processing method is as follows:
the cluster segmentation region is opened using a rectangular structural element of size 10 x 10.
6. The fruit identification method based on convolutional neural network of claim 5, wherein the fruit identification method comprises the following steps: the feature analysis of the communication area in the S2 is that after morphological treatment, a tiny noise interference area is removed; considering the actual size of the fruit, the non-target area needs to be further removed; pixels satisfying 8 adjacency with uniform gray scale are judged as the same area, noise interference area is filtered through the area characteristic of the connected area, the area X of the pixel area (SMin, SMax) is extracted according to the following formula,
wherein SMn and SMax are the lower and upper values of the parameters of the area pixel, respectively.
7. The fruit identification method based on convolutional neural network according to claim 1, wherein: the convolutional neural network model constructed in the step S3 is as follows:
1. convolutional layer
In convolutional neural networks, the first convolutional layer directly accepts input at the image pixel level, and each convolutional operation only processesA small image is transmitted to a later network after convolution change, and each layer of convolution can extract the most effective characteristic in the data; the convolution layer is the core layer of the convolution neural network, and the convolution is the core of the convolution layer, and the operation of simple convolution is a simple two-dimensional space convolution operation
This is a convolution operation;
the middle matrix is called a convolution kernel, also called a weight filter or a filter (filter), which is the core of the convolution process and can detect the horizontal edge, the vertical edge, the weight of the central region of the enhanced picture, and the like of the image; when the convolution is generalized to a higher dimension, the convolution kernel is used as a moving window on the input matrix, so that corresponding convolution operation can be performed; the CNN in the layer digs the space local association information of the natural image by enhancing the local connection mode of the nodes between adjacent layers in the neural network, the man-conveying of the nodes in the layer m is a part of the nodes in the layer m-1, and the nodes have the space adjacent visual receptive fields; the weight number of each neuron in the convolutional neural network is the size of a convolutional kernel, namely, each neuron is only connected with a part of pixels of the picture; by adopting two convolution layers (Covnl, covn 2), the convolution kernel 3 multiplied by 3 can be used for carrying out convolution operation on an original image by a learned convolution check, and a characteristic diagram output by the layer is formed by the action of an activation function of a convolved result and a threshold value, wherein the calculation formula is as follows:
wherein l is the number of layers, k is the number of convolution kernels, and Mj is the number of M feature graphs in the j; f is an activation function; bias is a threshold; a first layer (convl) of the convolution layer adopts 64 convolution check pictures of 3 multiplied by 3 to carry out convolution by random initialization, the step length is 1, and the size of the graph after the convolution is consistent with the original size; 16 convolution kernels of 3×3×64 are adopted in Conv2, and the step size is 1;
2. pooling layer
The purpose of the pooling layer is to further reduce the overfitting degree of the network training parameters and the model, and the pooling mode is generally 3 modes: (1) maximum pooling, and selecting the maximum value in a pooling window as a sampling value; (2) averaging, namely adding all values in a pooling window to average, and taking the average value as a sampling value; (3) randomly pooling, and determining which item to select by a probability method; the pooling layer can reduce the size of the image, improve the operation speed of the whole algorithm and reduce the influence of noise; the pooling layer is mainly used for solving an average value or a maximum value of a partial region of the image according to a convolution method aiming at the convolved feature map and representing a sampled region of the image; this is to describe a large image, aggregate statistics is performed on features at different locations, and this mean or maximum is the method of aggregate statistics, i.e. pooling; the two lower sampling layers adopt the maximum pooling with convolution kernel of 3 multiplied by 3 and step length of 2; the operation process can be expressed as follows:
wherein g (·) is a downsampling function; f is an activation function;
3. activation function
The activation function is used for guaranteeing the nonlinearity of the result, namely, after convolution operation, the output value is additionally added with offset, and the offset is input to the activation function to serve as the input of the next layer; common activation functions are Sigmoid, tanh and ReLu; the activation function is Relu, when x <0, hard saturation, when x >0, the derivative is 1, and the gradient is kept not to be attenuated, so that the problem of gradient disappearance is relieved.
4. Full connection layer
The full connection layer is equivalent to the function of a classifier; that is, the "distributed feature representation" of the convolution layer, activation function, and pooling layer chemistry is mapped to the sample label space; the CNN model of (2) adopts two fully-connected layers, the number of neurons in each layer is 128, and finally, the output adopts a Softmax classifier, and the Softmax function is a polynomial regression function, so that the recognition problem of multiple types of pictures is solved, and a part of neurons are randomly ignored by using Dropout in the fully-connected layers, so that the overfitting of the model is avoided, and the robustness of the model is enhanced.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211452782.3A CN116071560A (en) | 2022-11-21 | 2022-11-21 | Fruit identification method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211452782.3A CN116071560A (en) | 2022-11-21 | 2022-11-21 | Fruit identification method based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116071560A true CN116071560A (en) | 2023-05-05 |
Family
ID=86170659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211452782.3A Pending CN116071560A (en) | 2022-11-21 | 2022-11-21 | Fruit identification method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116071560A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117315596A (en) * | 2023-09-07 | 2023-12-29 | 重庆云网科技股份有限公司 | Deep learning-based motor vehicle black smoke detection and identification method |
| CN117422717A (en) * | 2023-12-19 | 2024-01-19 | 长沙韶光芯材科技有限公司 | Intelligent mask stain positioning method and system |
| CN117636055A (en) * | 2023-12-12 | 2024-03-01 | 北京易恒盈通科技有限公司 | Cloud storage method and system for digital information |
| CN117746272A (en) * | 2024-02-21 | 2024-03-22 | 西安迈远科技有限公司 | Unmanned aerial vehicle-based water resource data acquisition and processing method and system |
| CN118334633A (en) * | 2024-06-13 | 2024-07-12 | 内蒙古向新而行农业科技有限公司 | Insect condition forecasting method and system |
-
2022
- 2022-11-21 CN CN202211452782.3A patent/CN116071560A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117315596A (en) * | 2023-09-07 | 2023-12-29 | 重庆云网科技股份有限公司 | Deep learning-based motor vehicle black smoke detection and identification method |
| CN117636055A (en) * | 2023-12-12 | 2024-03-01 | 北京易恒盈通科技有限公司 | Cloud storage method and system for digital information |
| CN117422717A (en) * | 2023-12-19 | 2024-01-19 | 长沙韶光芯材科技有限公司 | Intelligent mask stain positioning method and system |
| CN117422717B (en) * | 2023-12-19 | 2024-02-23 | 长沙韶光芯材科技有限公司 | Intelligent mask stain positioning method and system |
| CN117746272A (en) * | 2024-02-21 | 2024-03-22 | 西安迈远科技有限公司 | Unmanned aerial vehicle-based water resource data acquisition and processing method and system |
| CN118334633A (en) * | 2024-06-13 | 2024-07-12 | 内蒙古向新而行农业科技有限公司 | Insect condition forecasting method and system |
| CN118334633B (en) * | 2024-06-13 | 2024-09-03 | 内蒙古向新而行农业科技有限公司 | Insect condition forecasting method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116071560A (en) | Fruit identification method based on convolutional neural network | |
| CN112418117B (en) | Small target detection method based on unmanned aerial vehicle image | |
| CN112949704B (en) | Tobacco leaf maturity state identification method and device based on image analysis | |
| WO2018052587A1 (en) | Method and system for cell image segmentation using multi-stage convolutional neural networks | |
| CN109993100B (en) | Method for realizing facial expression recognition based on deep feature clustering | |
| CN114283162B (en) | Real scene image segmentation method based on contrast self-supervision learning | |
| CN111191583A (en) | Space target identification system and method based on convolutional neural network | |
| CN110728694B (en) | Long-time visual target tracking method based on continuous learning | |
| CN111598001A (en) | Apple tree pest and disease identification method based on image processing | |
| CN110288033B (en) | Sugarcane top feature identification and positioning method based on convolutional neural network | |
| CN112749675A (en) | Potato disease identification method based on convolutional neural network | |
| AU2020102476A4 (en) | A method of Clothing Attribute Prediction with Auto-Encoding Transformations | |
| CN113610035A (en) | Rice tillering stage weed segmentation and identification method based on improved coding and decoding network | |
| CN113221913A (en) | Agriculture and forestry disease and pest fine-grained identification method and device based on Gaussian probability decision-level fusion | |
| CN111563542A (en) | Automatic plant classification method based on convolutional neural network | |
| CN111310820A (en) | Foundation meteorological cloud chart classification method based on cross validation depth CNN feature integration | |
| Kundur et al. | Insect pest image detection and classification using deep learning | |
| Nigam et al. | 12 Wheat rust disease identification using deep learning | |
| CN114612399A (en) | Picture identification system and method for mobile phone appearance mark | |
| Jain et al. | Hierarchy in Disease Severity Classification in Banana Leaves through the Integration of Convolutional Neural Network and Decision Tree Models | |
| CN109558883B (en) | Blade feature extraction method and device | |
| CN110853052A (en) | Tujia brocade pattern primitive segmentation method based on deep learning | |
| CN115565168A (en) | Sugarcane disease identification method based on attention system residual error capsule network | |
| CN111144464B (en) | Fruit automatic identification method based on CNN-Kmeans algorithm | |
| CN109784356B (en) | Matrix variable limited Boltzmann machine image classification method based on Fisher discriminant analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |