CN111652308A - Flower identification method based on ultra-lightweight full-convolution neural network - Google Patents

Flower identification method based on ultra-lightweight full-convolution neural network Download PDF

Info

Publication number
CN111652308A
CN111652308A CN202010476319.7A CN202010476319A CN111652308A CN 111652308 A CN111652308 A CN 111652308A CN 202010476319 A CN202010476319 A CN 202010476319A CN 111652308 A CN111652308 A CN 111652308A
Authority
CN
China
Prior art keywords
flower
neural network
ultra
data set
lightweight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010476319.7A
Other languages
Chinese (zh)
Other versions
CN111652308B (en
Inventor
徐光柱
朱泽群
尹思璐
雷帮军
石勇涛
陈鹏
夏平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Three Gorges University CTGU
Original Assignee
China Three Gorges University CTGU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Three Gorges University CTGU filed Critical China Three Gorges University CTGU
Publication of CN111652308A publication Critical patent/CN111652308A/en
Application granted granted Critical
Publication of CN111652308B publication Critical patent/CN111652308B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/457—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Image Analysis (AREA)

Abstract

The flower identification method based on the ultra-light-weight full-convolution neural network comprises the steps of cleaning and segmenting a collected data set, binarizing a color image in the data set by adopting an OTSU algorithm, marking a maximum connected region by adopting a maximum connected region method, extracting position information of the connected region, generating a matrix mask, and overlapping an original picture to obtain a segmentation graph of flowers; training a weight-class flower classification neural network by adopting a transfer learning method, screening the flower segmentation graph, and constructing the retained flower segmentation graph into a new data set; and constructing an ultra-lightweight convolutional neural network suitable for a weak computational platform, inputting a newly constructed data set into the convolutional neural network for training, and realizing flower recognition and classification. Compared with other existing methods, the ultra-lightweight neural network and the flower data set augmentation method provided by the invention have the advantage that the speed and the precision are greatly improved.

Description

Flower identification method based on ultra-lightweight full-convolution neural network
Technical Field
The invention relates to the technical field of intelligent flower identification, in particular to a flower identification method based on an ultra-lightweight full-convolution neural network.
Background
With the great improvement of the living standard of people in China, the flower consumption market in China becomes more and more prosperous, particularly after 2000 years, the domestic flower industry is subject to rapid development, and the production scale is at the top of the world. Due to the excellent properties of flowers, the diversified demands stimulated thereby also make them of great commercial value. In order to obtain the maximum utilization efficiency, the rapid and accurate flower variety classification has great fundamental significance for the development and utilization of the resources.
In the intelligent recognition field, people have designed different flower intelligent recognition systems according to the characteristics of the shape, color and the like of flowers, but the characteristics of low-level or middle-level are extracted by the traditional method, and the traditional method has no good generalization capability and semantic meaning, so people begin to adopt advanced pattern recognition, image processing and deep learning technology to automatically recognize flowers, and design various different deep convolutional neural network models, thereby greatly improving the accuracy of flower classification, such as:
the literature: prasad M, Jwa Lakshmamma B, Chandana A H, et al, an implementation of flow images with a controlled neural network [ J ]. International Journal of Engineering & Technology,2018,7(11):384-91, a novel deep convolutional neural network is proposed, feature extraction is performed by using convolution kernels of four different sizes, Softmax regression function is applied after two layers of full connectivity layers for final classification, and a random pooling technique combining maximum pooling and average pooling advantages is provided.
The literature: wudi, Houling swallow, Liu Xiu Lei, et al, an improved deep neural network flower image classification [ J ]. Henan university school newspaper (Nature science edition), 2019, an Inception V3 network is adaptively modified according to flower classification tasks, and a neural network with a superior classification effect is trained by using a migration learning technology by combining a Tanh-ReLu function with a soft saturation characteristic and the Inception V3 network.
The literature:
Figure BDA0002515995350000011
M,Budak U,Guo Y,et al.Efficient deep features selectionsand classification for flower species recognition[J]measurement 2019,137(7-13), applying a mixed model of AlexNet and VGG16 to flower species classification, using a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel as a classifier, and applying the features extracted by the mixed model to the classifier for classification.
In recent years, there are many documents that continuously improve these methods by modifying activation functions, feature fusion, etc., such as: the literature: identification of chrysanthemum patterns and varieties based on convolutional neural networks [ J ]. agricultural engineering, 2018,34 (5);
the literature: (vii) gao changeable, heart soothing, liu chang yan, et al. floral recognition algorithm based on convolutional neural network of ReLU function [ J ] computer technology and development, 2018,28(5): 154-7;
the literature: yi hong, Yi Xiang, Jie, et al, flower image classification fused with selective convolution characteristics [ J ]. Chinese graphic report, 2019,24(5): 762-72.
Chinese patent "application number: 201910548293.X also provides a flower recognition method based on a CNN feature fusion frame, combines various effective features of pictures with CNN with better picture recognition performance, trains recognition models based on each feature, combines the fusion frame formed by combining simple flower recognition models, and improves the accuracy of flower recognition by using different features of flowers.
In order to pursue a better classification effect, the network model structure becomes more complicated, the number of network layers is deeper, and although the accuracy of related tasks is improved, a plurality of problems are also brought. The first one is: storage of model weights. The deep neural network contains a large number of weight parameters, the accuracy rate is improved by deepening the network, the remarkable parameter quantity is increased, and considerable memory space is needed for storing model files. The second is that: applying a predicted time problem. An increase in the number of parameters also leads to an increase in the computational load of the network. In prediction, the input data needs to be propagated forward to spend huge computation resources, which will result in a large increase of the application latency.
There are also related patents that propose flower identification methods based on mobile terminals, such as flower identification methods and devices thereof "application number: 201810538860.9' provides a method for flower recognition under android system, which comprises collecting flower images, constructing a flower database, extracting features through a deep convolution neural network, and distinguishing flowers according to Euclidean distance of the features, wherein the method is used for distinguishing flowers from Euclidean distance in order to adapt to the computing power of mobile terminal equipment, and is slightly insufficient in accuracy;
chinese patent: ' a flower recognition method on an intelligent terminal ' and ' application number: 201410582707.8' extracting local shape feature of the picture, feature coding, feature multi-layer clustering, global shape feature extraction and global color feature extraction, extracting information related to flower category from the picture by a feature fusion method, training by using a linear support vector machine to obtain a support vector machine model, extracting features and quickly extracting features by using a KD tree structure, and then classifying by using the support vector machine model.
Because the difference between the flowers in nature is large, large-scale labeled data sets are deficient, and under a weak computational platform, the heavy-weight neural network model cannot be supported sufficiently, and the limited data sets are not suitable for larger models, so that the flower classification has a plurality of difficulties at present: 1) the weak computational force platform cannot support a heavy-magnitude neural network model, and the ultra-light-magnitude neural network has the problem of low accuracy; 2) flowers at different angles lose part of information, and particularly in an ultra-lightweight model, the characteristic extraction capability is poor, so that the classification error is serious; 3) the flower data set has complex source and uneven quality, and has difficulty in extracting features.
Disclosure of Invention
In order to solve the technical problem, the invention provides a flower identification method based on an ultra-lightweight full-convolution neural network, which provides an automatic flower picture data set augmentation method aiming at the defects of a data set; on the other hand, aiming at the weak computational platform, particularly the increasing demand of the development of mobile terminal application, the ultra-lightweight full convolution neural network suitable for the edge computing equipment is designed to classify the flower images. The ultra-lightweight neural network and the flower data set augmentation method provided by the invention can effectively operate and are well verified through experiments. In addition, compared with other existing methods, the method disclosed by the invention has the advantage that the speed and the precision are greatly improved.
The technical scheme adopted by the invention is as follows:
a flower identification method based on an ultra-light-weight full-convolution neural network comprises the steps of firstly, designing and implementing a crawler program of a flower image to obtain a rough data set, then cleaning and segmenting the collected data set, binarizing color images in the data set by adopting an OTSU algorithm, marking the largest 3 connected regions by a maximum connected region method, extracting position information of the connected regions, generating a matrix mask, and overlapping the matrix mask with an original image to obtain a segmentation image of flowers;
then, training a weight-class flower classification neural network by adopting a transfer learning method, screening the flower segmentation graph, and constructing a new data set by using the retained flower segmentation graph;
and finally, constructing an ultra-light convolutional neural network suitable for the weak computational force platform, inputting the newly constructed data set into the convolutional neural network for training, realizing flower recognition and classification, and finally applying the convolutional neural network to edge computing equipment.
The flower identification method based on the ultra-lightweight full-convolution neural network comprises the following steps:
step 1: providing an automatic flower data effective augmentation method, obtaining a screened flower segmentation graph, and forming a new data set, namely, a target _ dataset;
step 2: constructing an ultra-lightweight convolutional neural network, and training an ultra-lightweight convolutional neural network model by using the new data set obtained in the step 1;
and step 3: and identifying the flowers by using the trained ultra-lightweight convolutional neural network model.
In the step 1, the automatic flower data effective augmentation method comprises the following steps:
step 1.1: and finally converting the color picture into a binary picture:
firstly graying a color picture, and then reducing noise by using Gaussian filtering processing; then, determining a threshold value of a foreground background of the gray image by using an OTSU algorithm, setting a pixel point value larger than the threshold value to be 255, and setting a pixel point value smaller than the threshold value to be 0, thereby completing binarization of the color image;
step 1.2: the maximum 3 connected regions are marked using the maximum connected region method. One connected region is a pixel set formed by adjacent pixels with the same value, all the connected regions are calibrated by scanning all pixel points, and the maximum 3 connected regions are obtained according to the area attribute of the region. The minimum threshold value of the area is set according to actual requirements.
Step 1.3: extracting the position information of 3 connected domains to be selected, setting the pixel point value in the region to be 1, and setting the pixel point value outside the region to be 0, and generating 3 mask matrixes with the same size as the original image.
Step 1.4: superposing the mask matrix and the original picture to obtain final 3 candidate segmentation maps;
step 1.5: training a weight-class flower image classification neural network, screening candidate segmentation maps, automatically discarding the wrongly segmented flower maps by comparing set accuracy, processing all pictures collected by the network, and combining the pictures with a common data set to obtain a new data map set, thereby completing the manufacture of a target _ dataset data set.
In the step 2, a total of 23 layers of ultra-lightweight convolutional neural networks are constructed, the input size is 224 × 224, and a characteristic diagram with the size of 14 × 14 is output after 4 times of maximum pooling;
the depth convolution of 3 × 3 is responsible for the filtering effect, the point-by-point convolution of 1 × 1 is responsible for the conversion of channels, the number of two convolution kernels is doubled and the size is reduced continuously in the process of increasing the number of network layers, and high-level features are abstracted from an input image.
In the step 2, the ultra-lightweight convolutional neural network model outputs 2 image position constraint terms, namely, central point coordinates (x, y) values of flowers, which are normalized with respect to the length and width of the whole image, in addition to outputting N kinds of information at an output layer;
the loss function of the network is:
Figure BDA0002515995350000041
wherein N is the number of samples; m is a groupOther numbers; y isicIndicating a variable 0 or1, if the category is the same as that of the sample i, the variable is 1, otherwise the variable is 0; p is a radical oficA predicted probability of belonging to class c for the observation sample i; is the loss value of the center point coordinate.
The expression term of the center point coordinate in the loss function is formula (1), wherein lobjDefined as 0 if a flower is present, the value is 1, absent; (x, y),
Figure BDA0002515995350000042
respectively representing the coordinate of the actual central point and the coordinate of the predicted central point;
calculating the loss value of the coordinates of the center point of the flower:
Figure BDA0002515995350000043
the invention relates to a flower identification method based on an ultra-lightweight full-convolution neural network, which has the following technical effects:
1: according to the method, firstly, an effective flower data set augmentation method is designed aiming at the conditions that the current flower data set has complicated sources, uneven quality and insufficient types and affects the performance of a model, the flower image segmentation is carried out through a maximum connected region method, a weight-level network screens a segmentation graph to enhance the flower data set, and the detection precision of a neural network model can be greatly improved through experimental verification and training by adopting the enhanced data set.
2: then, the invention constructs an ultra-light-weight network which is used for carrying out classification detection on flowers in real time on weak computing platforms such as a mobile terminal and the like, the model adopts an alternate and repeated structure of 1 x 1 convolution and 3 x 3 convolution, and in addition to outputting N kinds of information, 2 image position constraint items, namely coordinates of the center point of the flower, are additionally output on an output layer, so that a trained filter focuses more on the area of the flower, and the extraction capability of the flower characteristics is enhanced.
3: the ultra-lightweight network model can effectively reduce the parameter and the calculated amount on the premise of ensuring the accuracy of the model, and can be widely applied to various mobile devices with limited computing capability due to the advantages of high speed and high efficiency.
4: the ultra-lightweight neural network and the flower data set augmentation method provided by the invention can effectively operate and are well verified through experiments. In addition, compared with other existing methods, the method disclosed by the invention has the advantage that the speed and the precision are greatly improved.
Drawings
Fig. 1 is a specific flow chart for capturing flower images.
FIG. 2(a) is a non-floral picture taken by a crawler;
FIG. 2(b) is a difficult-to-distinguish picture of a target obtained by a crawler;
FIG. 2(c) is a target picture of an error (not a rose) obtained by the crawler.
Fig. 3 is an illustration in which a plurality of flower bodies are present.
Fig. 4 is a flow chart of flower segmentation and identification.
FIG. 5(a) is a diagram before the picture is binarized;
fig. 5(b) is a graph of the result of the binarization of the picture.
Fig. 6 is a mask matrix diagram.
FIG. 7(a) is a first diagram of the preliminary segmentation result;
FIG. 7(b) is a second diagram of the preliminary segmentation result;
fig. 7(c) is a third diagram of the preliminary segmentation result.
Fig. 8 is a flower picture quantity distribution graph of two data sets.
Fig. 9 is a diagram of an ultra lightweight network model architecture.
FIG. 10 is a graph of training loss trends for two networks on an enhanced data set.
FIG. 11 is a comparison graph of classification accuracy for Oxford102 data set in various ways.
Detailed Description
The mobile digital life becomes a mainstream life style more and more, and the defect of the heavyweight neural network makes the application of the heavy neural network to weak computing platforms such as AI edge computing equipment difficult. The simplified network and method can be applied to a weak computing platform such as a mobile terminal in real time, but the accuracy rate of the simplified network and method often cannot meet the requirement. In order to solve this contradiction, researchers often use a model compression method, i.e., pruning parameters or transforming data types of trained models, so that storage of network weight parameters becomes more compact, thereby solving the memory problem and the prediction speed problem. Compared with the method of directly processing the original model weight, the lightweight model has the advantages that a more efficient convolution network calculation mode is constructed, so that network parameters are reduced, and good network performance is achieved. Meanwhile, for the lightweight model, the quality of the data set is also a key factor for determining the performance of the lightweight model, and the data set with various types, strong representativeness and enough number can greatly improve the performance of the model.
Firstly, cleaning and segmenting a collected data set, binarizing color images in the data set by adopting an OTSU algorithm, marking the largest 3 connected regions by adopting a maximum connected region method, extracting position information of the connected regions, generating a matrix mask, and overlapping the matrix mask with an original picture to obtain a segmentation graph of flowers; then, training a weight-class flower classification neural network by adopting a transfer learning method, screening the flower segmentation graph, and constructing a new data set by using the retained flower segmentation graph; and finally, constructing an ultra-light convolutional neural network suitable for the weak computational force platform, inputting the newly constructed data set into the convolutional neural network for training, realizing flower recognition and classification, and finally applying the convolutional neural network to edge computing equipment.
The flower identification method based on the ultra-lightweight full-convolution neural network comprises the following steps:
step 1: an automatic flower data effective augmentation method is provided, and a screened flower segmentation chart is obtained through the process shown in fig. 4 to form a new data set, namely, target _ dataset;
step 2: constructing an ultra-lightweight convolutional neural network, and training an ultra-lightweight convolutional neural network model by using the new data set obtained in the step 1;
and step 3: and identifying the flowers by using the trained ultra-lightweight convolutional neural network model.
The details of each step are as follows:
step 1: the development and development of deep learning technology are based on data, a high-quality target task data set is beneficial to accurately extracting effective features through an algorithm, a solid foundation is laid for obtaining a network model with excellent performance, and the Oxfor102 data set commonly used nowadays only comprises 8189 pictures of 102 flowers, and the quality and the number of the Oxfor102 data set are not enough to support an ultra-light neural network model under a weak computing platform.
In order to obtain a better data set, 52753 102 kinds of flower pictures corresponding to the kinds of the Oxford102 data set are downloaded from a Google search engine, and then cleaning is carried out, firstly, a user makes rules for picture capturing, the rules are input into a crawler program, the program starts to request a query page from the Google search engine, all picture links appearing on the webpage are analyzed for the returned webpage, and meanwhile, the next search page is obtained to prepare for the next round of capturing. The analyzed download link is further screened according to the given specification of the user, the link addresses meeting the requirements are stored, and finally the file is downloaded to the local disk according to the addresses. The specific grabbing process is shown in fig. 1.
Since the crawler downloads pictures according to given keywords only, there will be many dirty data, such as rose picture dirty data obtained by the crawler shown in fig. 2(a), fig. 2(b), and fig. 2 (c). And the network pictures have complicated sources and uneven quality, so further image segmentation processing is required. In addition to the above dirty data, not only the pictures but also a reality that a plurality of flower bodies exist simultaneously. As shown in fig. 3.
Aiming at the problems, the invention provides an automatic flower data effective augmentation method based on a weight-level neural network model. Firstly, extracting a maximum connected region candidate domain in a picture, then using a pre-trained neural network model to carry out predictive screening on a flower map to obtain a final segmentation map, and integrating the final segmentation map into a new data set.
The data augmentation of flowers is divided into five steps, and the flow is shown in figure 4:
the first step is as follows: the method is to finally convert a color picture into a binary picture, and the main method is as follows: firstly graying a color picture, and then reducing noise by using Gaussian filtering processing; and then determining a threshold value of the foreground and the background of the grayed image by using an OTSU algorithm, setting the pixel point value larger than the threshold value to be 255, and setting the pixel point value smaller than the threshold value to be 0, thereby completing the binarization of the color picture. An example of selecting a floral map with a complex background is shown in fig. 5(a) and 5(b), after binarization processing.
Among them, the OTSU algorithm is described in Otsu N.A. threshold selection method from among the programs-leveldiagrams [ J ]. IEEE transactions on systems, man, and cybernetics,1979,9(1):62-6. The OTSU algorithm is also called "Otsu algorithm" or "maximum inter-class variance method". The method is considered as the optimal algorithm for selecting the threshold value in image segmentation, is simple in calculation and is not influenced by the brightness and the contrast of an image, so that the method is widely applied to digital image processing. The invention also chooses this more reliable binary segmentation algorithm.
The second step is that: the maximum 3 connected regions are marked using the maximum connected region method. A connected region is a pixel set formed by adjacent pixels with the same value, and all the connected regions are calibrated by scanning all the pixel points. And acquiring the maximum 3 connected domains according to the area attribute of the region.
The third step: extracting the position information of 3 connected domains to be selected, setting the pixel point value in the region to be 1, and setting the pixel point value outside the region to be 0, and generating 3 mask matrixes with the same size as the original image. The mask matrix diagram in this embodiment is shown in fig. 6.
The fourth step: and overlapping the mask matrix and the original picture to obtain the final 3 candidate segmentation maps. As shown in fig. 7(a), 7(b), and 7 (c). As can be seen from fig. 7(a) to 7(c), the segmentation result fig. 7(c) is not a flower picture and should be discarded.
The fifth step: training a weight-class flower image classification neural network and screening candidate segmentation maps. Although the heavyweight network has higher demand on the computing power of the equipment, the performance of the heavyweight network is also better correspondingly, the heavyweight neural network used for screening the segmentation graph can adopt models with better performance at present, such as VGG (variable gradient generator), ResNet (152), and Darknet (53), and the Darknet (53) network model is selected in the invention. And inputting the candidate segmentation graph into a trained weighted neural network for prediction screening. And automatically discarding the wrongly-segmented flower map by comparing the set accuracy. After all 52753 pictures collected over the network were processed, 20554 total data images were obtained by merging with Oxford102, and the creation of the Larger _ dataset was completed.
The method for automatically screening out qualified flower pictures by combining the pre-trained neural network avoids the defect that a large amount of picture data is manually screened, and the workload is greatly reduced.
The final gathering of the sorted Larger _ dataset is greatly enhanced compared with the Oxford102 dataset. The data volume of each flower is expanded by 2 to 4 times for102 flowers contained. The minimum pinkprimrose pictures are increased from 40 to 100, and the maximum petunia are increased from 258 to 644. The picture distribution of the two data sets is shown in fig. 8.
Step 2: aiming at a weak computing power platform with poor computing power, the invention researches and constructs an ultra-light network model, inputs the Larger _ dataset data set obtained in the step 1 into the model for training, and finally applies the Larger _ dataset data set to the classification and identification of flowers on edge computing equipment, so that the performance of the Larger _ dataset data set has good performance. The designed ultra-lightweight network model has 23 layers in total, the network structure is shown in fig. 9, the input size is 224 × 224, the feature graph with the output size of 14 × 14 is output after 4 times of maximum pooling, and the specific network structure is shown in fig. 9.
In view of the structure of the depth-separable deconvolution structure in the MobileNet network, the paper of the MobileNet network structure mentions that the proposed depth-separable deconvolution structure can effectively reduce the parameter number and the calculated amount on the premise of ensuring the model accuracy compared with the standard convolution. The depth convolution of 3 × 3 is responsible for the filtering action, the point-by-point convolution of 1 × 1 is responsible for the conversion of channels, the number of two convolution kernels is doubled and the size is reduced continuously in the process of increasing the number of network layers, and high-level features are abstracted from an input image. Due to the advantages of high speed and high efficiency, the ultra-lightweight network model can be widely applied to various mobile devices with limited computing power.
As shown in fig. 9, in addition to the class _ number category information, the model outputs 2 additional image position constraint items, namely, the coordinates (x, y) of the center point of the flower, which are normalized with respect to the length and width of the whole image, at the output layer. The use of the constraint item leads the trained filter to pay more attention to the area of the flower, and enhances the extraction capability of the flower characteristics.
The loss function of the network is:
Figure BDA0002515995350000081
wherein N is the number of samples; m is the number of categories; y isicAn indication variable (0 or 1), which is 1 if the class is the same as the class of sample i, and 0 otherwise; p is a radical oficA predicted probability of belonging to class c for the observation sample i; is the loss value of the center point coordinate.
The expression term of the center point coordinate in the loss function is formula (1), wherein lobjDefined as 0 if a flower is present, the value is 1, absent; (x, y),
Figure BDA0002515995350000091
respectively representing the actual central point coordinate and the predicted central point coordinate.
Calculating the loss value of the coordinates of the center point of the flower:
Figure BDA0002515995350000092
and (3) training an ultra-lightweight network model on a Larger _ dataset data set, wherein the training parameters are shown in a table 1.
TABLE 1 training parameter settings table
Figure BDA0002515995350000093
The training loss graph obtained after training is shown in fig. 10, and it can be seen from fig. 10 that the average loss rate is lowest when the iteration is 25000 times.
At the same time, the present invention also performs the same training on the Oxford102 dataset before processing to compare the detection results after the two datasets have been trained. The details of Top1 accuracy obtained after training through the two data sets are shown in table 2.
TABLE 2 Classification accuracy Table for networks trained on two data sets
Figure BDA0002515995350000094
And the comparison shows that better classification accuracy can be obtained by using the enhanced data set, and the super-lightweight network model is obviously improved.
The trained network is compared on the Oxford102 flower data set, and the result shows that compared with other existing models, the ultra-lightweight network model trained after data enhancement by the method has good accuracy and detection speed in flower classification, as shown in FIG. 11. Based on the characteristics of limited operational capability and high storage cost of mobile terminal equipment, the flower classification application using the ultra-lightweight network model as the basic backbone network has very high practical prospect.
References to other models existing in fig. 11:
[8] yangling, Wangxiyuan, Zhang Yu, a refined image classification of an improved deep convolutional neural network [ J ]. Jiangxi university journal (Nature science edition), 2017,41(05): 476-83.
[9]
Figure BDA0002515995350000101
M,Budak U,Guo Y,et al.Efficient deep features selections andclassification for flower species recognition[J].Measurement,2019,137(7-13)。
[10]Ge W,Yu Y.Borrowing treasures from the wealthy:Deep transferlearning through selective joint fine-tuning；proceedings of the Proceedingsof the IEEE conference on computer vision and pattern recognition,F,2017[C].
[11]Hiary H,Saadeh H,Saadeh M,et al.Flower classification using deepconvolutional neural networks[J].IET Computer Vision,2018,12(6):855-62.
[12]Prasad M,Jwala Lakshmamma B,Chandana A H,et al.An efficientclassification of flower images with convolutional neural networks[J].International Journal of Engineering&Technology,2018,7(11):384-91。

Claims (6)

1. The flower identification method based on the ultra-lightweight full-convolution neural network is characterized by comprising the following steps of:
firstly, cleaning and segmenting a collected data set, binarizing a color image in the data set by adopting an OTSU algorithm, marking a maximum connected region by adopting a maximum connected region method, extracting position information of the connected region, generating a matrix mask, and overlapping an original picture to obtain a segmentation graph of flowers;
then, training a weight-class flower classification neural network by adopting a transfer learning method, screening the flower segmentation graph, and constructing a new data set by using the retained flower segmentation graph;
and finally, constructing an ultra-light convolutional neural network suitable for the weak computational force platform, inputting the newly constructed data set into the convolutional neural network for training, and realizing flower recognition and classification.
2. The flower identification method based on the ultra-light-weight full-convolution neural network is characterized by comprising the following steps of:
step 1: providing an automatic flower data effective augmentation method, obtaining a screened flower segmentation graph, and forming a new data set;
step 2: constructing an ultra-lightweight convolutional neural network, and training an ultra-lightweight convolutional neural network model by using the new data set obtained in the step 1;
and step 3: and identifying the flowers by using the trained ultra-lightweight convolutional neural network model.
3. A flower recognition method based on ultra-lightweight complete convolutional neural network as claimed in claim 2, characterized in that:
in the step 1, the automatic flower data effective augmentation method comprises the following steps:
step 1.1: and finally converting the color picture into a binary picture:
firstly graying a color picture, and then reducing noise by using Gaussian filtering processing; then, determining a threshold value of a foreground background of the gray image by using an OTSU algorithm, setting a pixel point value larger than the threshold value to be 255, and setting a pixel point value smaller than the threshold value to be 0, thereby completing binarization of the color image;
step 1.2: marking the largest 3 connected regions by adopting a maximum connected region method; one connected region is a pixel set formed by adjacent pixels with the same value, all the connected regions are calibrated by scanning all pixel points, and the maximum 3 connected regions are obtained according to the area attribute of the region;
step 1.3: extracting the position information of 3 connected domains to be selected, setting the pixel point value in the region to be 1, and setting the pixel point value outside the region to be 0, and generating 3 mask matrixes with the same size as the original image;
step 1.4: superposing the mask matrix and the original picture to obtain final 3 candidate segmentation maps;
step 1.5: training a weight-class flower image classification neural network, screening candidate segmentation maps, automatically discarding the wrongly segmented flower maps by comparing set accuracy, processing all pictures collected by the network, and combining the pictures with a common data set to obtain a new data map set, thereby completing the manufacture of a target _ dataset data set.
4. A flower recognition method based on ultra-lightweight complete convolutional neural network as claimed in claim 2, characterized in that:
in the step 2, a total of 23 layers of ultra-lightweight convolutional neural networks are constructed, the input size is 224 × 224, and a characteristic diagram with the size of 14 × 14 is output after 4 times of maximum pooling;
the depth convolution of 3 × 3 is responsible for the filtering effect, the point-by-point convolution of 1 × 1 is responsible for the conversion of channels, the number of two convolution kernels is doubled and the size is reduced continuously in the process of increasing the number of network layers, and high-level features are abstracted from an input image.
5. A flower recognition method based on ultra-lightweight complete convolutional neural network as claimed in claim 2, characterized in that:
in the step 2, the ultra-lightweight convolutional neural network model outputs 2 image position constraint terms, namely, central point coordinates (x, y) values of flowers, which are normalized with respect to the length and width of the whole image, in addition to outputting N kinds of information at an output layer;
the loss function of the network is:
Figure FDA0002515995340000021
wherein N is the number of samples; m is the number of categories; y isicIndicating a variable 0 or1, if the category is the same as that of the sample i, the variable is 1, otherwise the variable is 0; p is a radical oficA predicted probability of belonging to class c for the observation sample i; the loss value of the central point coordinate is taken as the loss value of the central point coordinate;
the expression term of the center point coordinate in the loss function is formula (1), wherein lobjDefined as 0 if a flower is present, the value is 1, absent; (x, y),
Figure FDA0002515995340000022
respectively representing the coordinate of the actual central point and the coordinate of the predicted central point;
calculating the loss value of the coordinates of the center point of the flower:
Figure FDA0002515995340000023
6. an automatic flower data effective augmentation method is characterized by comprising the following steps:
step 1.1: and finally converting the color picture into a binary picture:
firstly graying a color picture, and then reducing noise by using Gaussian filtering processing; then, determining a threshold value of a foreground background of the gray image by using an OTSU algorithm, setting a pixel point value larger than the threshold value to be 255, and setting a pixel point value smaller than the threshold value to be 0, thereby completing binarization of the color image;
step 1.2: marking the largest 3 connected regions by adopting a maximum connected region method; one connected region is a pixel set formed by adjacent pixels with the same value, all the connected regions are calibrated by scanning all pixel points, and the maximum 3 connected regions are obtained according to the area attribute of the region;
step 1.3: extracting the position information of 3 connected domains to be selected, setting the pixel point value in the region to be 1, and setting the pixel point value outside the region to be 0, and generating 3 mask matrixes with the same size as the original image;
step 1.4: superposing the mask matrix and the original picture to obtain final 3 candidate segmentation maps;
step 1.5: training a weight-class flower image classification neural network, screening candidate segmentation maps, automatically discarding the wrongly segmented flower maps by comparing set accuracy, processing all pictures collected by the network, and combining the pictures with a common data set to obtain a new data map set, thereby completing the manufacture of a target _ dataset data set.
CN202010476319.7A 2020-05-13 2020-05-29 Flower identification method based on ultra-lightweight full convolutional neural network Active CN111652308B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010403907 2020-05-13
CN2020104039078 2020-05-13

Publications (2)

Publication Number Publication Date
CN111652308A true CN111652308A (en) 2020-09-11
CN111652308B CN111652308B (en) 2024-02-23

Family

ID=72346888

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010476319.7A Active CN111652308B (en) 2020-05-13 2020-05-29 Flower identification method based on ultra-lightweight full convolutional neural network

Country Status (1)

Country Link
CN (1) CN111652308B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030223627A1 (en) * 2001-10-16 2003-12-04 University Of Chicago Method for computer-aided detection of three-dimensional lesions
US20160148078A1 (en) * 2014-11-20 2016-05-26 Adobe Systems Incorporated Convolutional Neural Network Using a Binarized Convolution Layer
EP3171297A1 (en) * 2015-11-18 2017-05-24 CentraleSupélec Joint boundary detection image segmentation and object recognition using deep learning
CN107480649A (en) * 2017-08-24 2017-12-15 浙江工业大学 A kind of fingerprint pore extracting method based on full convolutional neural networks
US20180032866A1 (en) * 2016-07-28 2018-02-01 Samsung Electronics Co., Ltd. Neural network method and apparatus
CN108416440A (en) * 2018-03-20 2018-08-17 上海未来伙伴机器人有限公司 A kind of training method of neural network, object identification method and device
CN109325484A (en) * 2018-07-30 2019-02-12 北京信息科技大学 Flowers image classification method based on background priori conspicuousness
KR101979654B1 (en) * 2018-01-15 2019-05-17 주식회사 비엔인더스트리 License plate recognition apparatus and the method thereof
US20190251694A1 (en) * 2018-02-14 2019-08-15 Elekta, Inc. Atlas-based segmentation using deep-learning
KR20190098671A (en) * 2018-02-14 2019-08-22 삼성전자주식회사 High speed processing method of neural network and apparatus using thereof
CN110188705A (en) * 2019-06-02 2019-08-30 东北石油大学 A kind of remote road traffic sign detection recognition methods suitable for onboard system
CN110853009A (en) * 2019-11-11 2020-02-28 北京端点医药研究开发有限公司 Retina pathology image analysis system based on machine learning
CN110929603A (en) * 2019-11-09 2020-03-27 北京工业大学 Weather image identification method based on lightweight convolutional neural network

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030223627A1 (en) * 2001-10-16 2003-12-04 University Of Chicago Method for computer-aided detection of three-dimensional lesions
US20160148078A1 (en) * 2014-11-20 2016-05-26 Adobe Systems Incorporated Convolutional Neural Network Using a Binarized Convolution Layer
EP3171297A1 (en) * 2015-11-18 2017-05-24 CentraleSupélec Joint boundary detection image segmentation and object recognition using deep learning
US20180032866A1 (en) * 2016-07-28 2018-02-01 Samsung Electronics Co., Ltd. Neural network method and apparatus
CN107480649A (en) * 2017-08-24 2017-12-15 浙江工业大学 A kind of fingerprint pore extracting method based on full convolutional neural networks
KR101979654B1 (en) * 2018-01-15 2019-05-17 주식회사 비엔인더스트리 License plate recognition apparatus and the method thereof
KR20190098671A (en) * 2018-02-14 2019-08-22 삼성전자주식회사 High speed processing method of neural network and apparatus using thereof
US20190251694A1 (en) * 2018-02-14 2019-08-15 Elekta, Inc. Atlas-based segmentation using deep-learning
CN108416440A (en) * 2018-03-20 2018-08-17 上海未来伙伴机器人有限公司 A kind of training method of neural network, object identification method and device
CN109325484A (en) * 2018-07-30 2019-02-12 北京信息科技大学 Flowers image classification method based on background priori conspicuousness
CN110188705A (en) * 2019-06-02 2019-08-30 东北石油大学 A kind of remote road traffic sign detection recognition methods suitable for onboard system
CN110929603A (en) * 2019-11-09 2020-03-27 北京工业大学 Weather image identification method based on lightweight convolutional neural network
CN110853009A (en) * 2019-11-11 2020-02-28 北京端点医药研究开发有限公司 Retina pathology image analysis system based on machine learning

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
吕石磊，卢思华等: "基于改进YOLOv3-LITE轻量级神经网络的柑橘识别方法", vol. 35, no. 17, pages 205 - 214 *
张娟;韩殿元;黄心渊;: "自然背景下的梅花图像分割算法研究", 北京林业大学学报, no. 03 *
李兴维; 朱泽群; 万秋波; 屈金山: "基于深度学习的实时车牌检测", 《信息通信》 *
林君宇;李奕萱;郑聪尉;罗雯波;许蕾;: "应用卷积神经网络识别花卉及其病症", 小型微型计算机系统, no. 06 *

Also Published As

Publication number Publication date
CN111652308B (en) 2024-02-23

Similar Documents

Publication Publication Date Title
Wei et al. Superpixel hierarchy
Shi et al. Road detection from remote sensing images by generative adversarial networks
CN110825899B (en) Clothing image retrieval method integrating color features and residual network depth features
Kadam et al. Detection and localization of multiple image splicing using MobileNet V1
CN112446388A (en) Multi-category vegetable seedling identification method and system based on lightweight two-stage detection model
CN108509978A (en) The multi-class targets detection method and model of multi-stage characteristics fusion based on CNN
CN107679250A (en) A kind of multitask layered image search method based on depth own coding convolutional neural networks
CN106372648A (en) Multi-feature-fusion-convolutional-neural-network-based plankton image classification method
CN110222767B (en) Three-dimensional point cloud classification method based on nested neural network and grid map
CN106126581A (en) Cartographical sketching image search method based on degree of depth study
CN106354735A (en) Image target searching method and device
CN110210431B (en) Point cloud semantic labeling and optimization-based point cloud classification method
CN114694038A (en) High-resolution remote sensing image classification method and system based on deep learning
CN110633708A (en) Deep network significance detection method based on global model and local optimization
CN110188816B (en) Image fine granularity identification method based on multi-stream multi-scale cross bilinear features
CN110321785A (en) A method of introducing ResNet deep learning network struction dermatoglyph classification prediction model
CN113920516B (en) Calligraphy character skeleton matching method and system based on twin neural network
CN112381030B (en) Satellite optical remote sensing image target detection method based on feature fusion
CN111339935A (en) Optical remote sensing picture classification method based on interpretable CNN image classification model
CN106874421A (en) Image search method based on self adaptation rectangular window
CN113177555A (en) Target processing method and device based on cross-level, cross-scale and cross-attention mechanism
CN111046917A (en) Object-based enhanced target detection method based on deep neural network
CN111652273A (en) Deep learning-based RGB-D image classification method
CN113269224A (en) Scene image classification method, system and storage medium
Lin et al. Scene recognition using multiple representation network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant