CN115099297A - Soybean plant phenotype data statistical method based on improved YOLO v5 model - Google Patents
Soybean plant phenotype data statistical method based on improved YOLO v5 model Download PDFInfo
- Publication number
- CN115099297A CN115099297A CN202210441866.0A CN202210441866A CN115099297A CN 115099297 A CN115099297 A CN 115099297A CN 202210441866 A CN202210441866 A CN 202210441866A CN 115099297 A CN115099297 A CN 115099297A
- Authority
- CN
- China
- Prior art keywords
- model
- training
- soybean plant
- yolo
- improved
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 244000068988 Glycine max Species 0.000 title claims abstract description 92
- 238000007619 statistical method Methods 0.000 title claims abstract description 15
- 238000012549 training Methods 0.000 claims abstract description 60
- 238000000034 method Methods 0.000 claims abstract description 50
- 238000005070 sampling Methods 0.000 claims abstract description 19
- 230000006872 improvement Effects 0.000 claims abstract description 10
- 230000007246 mechanism Effects 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 11
- 238000012795 verification Methods 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 7
- 230000002708 enhancing effect Effects 0.000 claims description 7
- 230000002829 reductive effect Effects 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 235000010469 Glycine max Nutrition 0.000 abstract description 25
- 238000009395 breeding Methods 0.000 abstract description 10
- 230000001488 breeding effect Effects 0.000 abstract description 10
- 238000001514 detection method Methods 0.000 description 34
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 5
- 238000013523 data management Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- IJGRMHOSHXDMSA-UHFFFAOYSA-N Atomic nitrogen Chemical compound N#N IJGRMHOSHXDMSA-UHFFFAOYSA-N 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 241000196324 Embryophyta Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000009418 agronomic effect Effects 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229910052757 nitrogen Inorganic materials 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Computational Biology (AREA)
- Algebra (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of artificial intelligent breeding of soybeans, and discloses a soybean plant phenotype data statistical method based on an improved YOLO v5 model, which comprises the following steps: obtaining a sampling picture of a target soybean plant; identifying the sampling picture through an identification model trained based on a specified training method; counting and storing the phenotypic characteristics about the soybean plants output by the recognition model; wherein the identification model is a YOLO v5 model improved based on a specified improvement method; the sampling picture obtained by shooting and sampling the target soybean plant can be input into a pre-trained recognition model for recognition, the recognition model can quickly and accurately give the phenotype characteristic data of the target soybean plant, then the phenotype characteristic data can be counted, and the follow-up workers can conveniently look up and call the phenotype characteristic data.
Description
Technical Field
The invention relates to the technical field of artificial intelligent breeding of soybeans, in particular to a soybean plant phenotype data statistical method based on an improved YOLO v5 model.
Background
Soybeans are important food crops and important economic crops in China. However, the soybean industry in China highly depends on import for many years, and the import sources are highly concentrated, thereby seriously threatening the national grain and oil safety.
The soybean seed test work is a key link for analyzing the genetic rule of the soybean and breeding the soybean by acquiring repeatable soybean phenotype data and quantitatively analyzing the relationship between the repeatable soybean phenotype data and the yield, quality and the like of the soybean. The phenotype of the soybean refers to all physical, physiological and traits of the growth and development process and the result of the soybean under the combined action of genome and planting environment. The influence of phenotype on yield, quality, stress tolerance, etc. can be quantitatively analyzed through reproducible high quality phenotypic data. For example, the soybean leaf shape is an important appearance characteristic and an agronomic trait of the soybean, the growth state of the plant can be intuitively reflected, the leaf shape not only influences the sunlight capture efficiency, but also influences the absorption efficiency of a nitrogen reservoir, the yield can be adjusted, and the pod number trait of the soybean is also closely linked with the leaf shape.
The traditional soybean seed testing mode mainly depends on manual work, and has the problems of limited data acquisition amount, high cost of manpower and material resources, poor data sharing capability, low measurement precision, poor repeatability, poor objectivity and the like, so that the function of phenotypic data in the soybean breeding decision making process is greatly limited, and the mode is also one of the reasons for limiting the development of the soybean industry. Therefore, an intelligent and high-throughput method capable of accurately, quickly and nondestructively detecting soybean phenotypic parameters is urgently needed for modern soybean breeding, and breeding efficiency is improved.
Disclosure of Invention
The invention aims to provide a soybean plant phenotype data statistical method based on an improved YOLO v5 model, and the method solves the following technical problems:
how to improve the detection efficiency and accuracy of soybean phenotypic parameters during soybean seed test.
The purpose of the invention can be realized by the following technical scheme:
a soybean plant phenotype data statistical method based on an improved YOLO v5 model comprises the following steps:
obtaining a sampling picture of a target soybean plant;
identifying the sampling picture through an identification model trained based on a specified training method;
counting and saving phenotypic characteristics output by the recognition model about the soybean plants;
wherein the identification model is a improved YOLO v5 model based on a specified improvement method.
Through the technical scheme, the sampling picture obtained by shooting and sampling the target soybean plant can be input into the pre-trained recognition model for recognition, the recognition model can quickly and accurately give the phenotype characteristic data of the target soybean plant, then the phenotype characteristic data can be counted, and the follow-up workers can conveniently look up and call the phenotype characteristic data.
As a further scheme of the invention: the specified training method comprises the following steps:
acquiring a training sample set aiming at soybean plants;
enhancing and expanding the training sample set to obtain a training data set;
and randomly dividing the training data set into a training set, a verification set and a test set.
As a further scheme of the invention: the method for acquiring the training sample set aiming at the soybean plants comprises the following steps:
shooting the soybean plants at different time periods, different weather conditions and different angles to obtain shot images;
and labeling all the shot images to obtain the training sample set, wherein the labels comprise various category labels.
As a further scheme of the invention: the method for obtaining the training data set after enhancing and expanding the training sample set comprises the following steps:
performing data enhancement on each shot image in the training sample set;
and performing combination of calibration frames and combination and splicing of pictures on the read 4 images through operations of color gamut change, cutting, scaling, random arrangement and the like to obtain a training data set.
As a further scheme of the invention: the specified training method further comprises:
setting initial hyper-parameters, wherein the training size of the improved YOLO v5 model is 640 multiplied by 640, the batch size is 16, the number of categories is 5, the initial learning rate is 1e-2, and the iteration number is 100;
inputting the training set and the verification set into the improved YOLO v5 model for training;
and according to the training verification loss rate change curve, determining the final learning rate and iteration times when the loss tends to be stable along with the increase of the iteration times.
As a further scheme of the invention: the specified improvement method comprises the following steps:
replacing Backbone feature extraction network Backbone of YOLO v5 with MobileNetv 2;
the MobileNetv2 structure comprises:
for a trunk feature extraction part of the improved YOLO v5 model, performing dimensionality increase by using 1 × 1 convolution;
after feature extraction is carried out by using 3 × 3 depth separable convolution, dimension reduction is carried out by using 1 × 1 convolution;
the input and the output of the residual edge part are directly connected;
features are projected onto a representation of the low-dimensional features using linear bottleneck operations.
As a further scheme of the invention: the specified improvement method comprises the following steps:
introducing an ECANet attention mechanism;
the ECANet attention mechanism comprises the following structures:
obtaining the global feature of each channel through global average pooling, and outputting the global feature with the dimension of C multiplied by 1;
obtaining the relationship between local channels by using one-dimensional convolution;
putting the weight file into an algorithm of a YOLO v5 model to detect the verification set, and observing the detection condition;
using a Sigmoid to activate a function to output a new weight with the dimensionality of C multiplied by 1;
and the new weight and the input feature graph are multiplied to calculate the redistribution of the channel feature weight, inhibit invalid features and enhance valid features.
As a further scheme of the invention: the ECANet attention introducing mechanism mainly comprises the following steps:
an ECANet attention mechanism is placed on the enhanced feature extraction network, and the attention mechanism is added on three effective feature layers extracted from the main network;
and an ECANet attention mechanism is added to the up-sampling result in the FPN, so that information loss is reduced, and the integrated feature on each layer is optimized.
As a further scheme of the invention: the specified improvement method comprises the following steps:
modified loss function:
replacing GIOU _ Loss with CIOU _ Loss;
classical NMS was replaced by DIOU _ NMS.
The invention has the beneficial effects that:
according to the invention, a sampling picture obtained by shooting and sampling the target soybean plant can be input into a pre-trained recognition model for recognition, the recognition model can rapidly give the phenotype characteristic data of the target soybean plant, and then the phenotype characteristic data can be counted, so that the follow-up workers can conveniently look up and call the phenotype characteristic data.
Drawings
The invention will be further described with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of the basic process of the statistical method of the phenotypic data of soybean plants according to the present invention;
FIG. 2 is a schematic structural diagram of a general YOLO v 5;
FIG. 3 is a schematic diagram of a general feature extraction network in YOLO v 5;
FIG. 4 is a diagram of the structure of the Focus module in the general YOLO v 5;
FIG. 5 is a block diagram of a CSP module in the general YOLO v 5;
FIG. 6 is a diagram of a depth separable convolution process;
FIG. 7 is a structural diagram of a Bottleneck Residual block according to the present invention;
FIG. 8 is a diagram of the structure of ECANet in the present invention;
FIG. 9 is an exemplary diagram of GIOU degenerating to IOU;
FIG. 10 shows the structure of the improved YOLO v5 model of the present invention;
FIG. 11 is a graph comparing the performance of the improved YOLO v5 model of the present invention with other models;
FIG. 12 is a schematic diagram of the detection process of the phenotypic data statistics method application system of the present invention;
FIG. 13 is a client-side initial interface;
FIG. 14 is a leaf shape identification and count display interface;
FIG. 15 is a flower identification and counting display interface;
FIG. 16 is a data management interface.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in FIG. 1, the invention relates to a soybean plant phenotype data statistical method based on an improved YOLO v5 model, which comprises the following steps:
obtaining a sampling picture of a target soybean plant;
identifying the sampling picture through an identification model trained based on a specified training method;
counting and storing the phenotypic characteristics about the soybean plants output by the recognition model;
wherein the identification model is an improved YOLO v5 model improved based on a specified improvement method.
According to the invention, a sampling picture obtained by shooting and sampling the target soybean plant can be input into a pre-trained recognition model for recognition, the recognition model can rapidly give the phenotype characteristic data of the target soybean plant, and then the phenotype characteristic data can be counted, so that the follow-up workers can conveniently look up and call the phenotype characteristic data.
As a further scheme of the invention: the specified training method comprises the following steps:
acquiring a training sample set aiming at soybean plants;
enhancing the extended training sample set to obtain a training data set;
the training data set is randomly divided into three parts, namely a training set, a verification set and a test set.
As a further scheme of the invention: the method for obtaining the training sample set aiming at the soybean plants comprises the following steps:
shooting soybean plants at different time periods, different weather conditions and different angles to obtain shot images;
and marking all shot images to obtain a training sample set, wherein the marks comprise various types of labels.
Specifically, a camera is adopted to shoot soybean plants, and in order to ensure the applicability and generalization capability of the improved YOLO v5 model, different conditions are fully considered, and soybean plants in different time periods, different weather conditions and different angles are selected to be shot.
LabelImg was used to manually label 5 categories of information, 5 categories of labels being: oval (Oval), Ovoid (Oval), Lanceolate (needle-shaped), White (White flower), Violet (purple flower). LabelImg is a data labeling tool commonly used in deep learning, Qt (PyQt) is used as a graphical interface of the LabelImg, a required environment of lxml + python needs to be configured before use, and a labeling result is saved as an XML file in a PASCALVOC format.
As a further scheme of the invention: the method for obtaining the training data set after enhancing the extended training sample set comprises the following steps:
performing data enhancement on each shot image in the training sample set;
and performing combination of calibration frames and combination and splicing of pictures on the read 4 images through operations of color gamut change, cutting, scaling, random arrangement and the like to obtain a training data set.
The data set is the input and basis of deep learning, and the quality of the data set determines the quality of deep learning model training and the accuracy of prediction to a certain extent. Because the soybean plant image sample acquisition time is limited, and marking a large number of soybean plant sample pictures is a very time-consuming matter, in order to improve the generalization capability of the deep learning model, data preprocessing work including data enhancement needs to be carried out on the soybean plant original data set.
The data enhancement mode adopted by the YOLO v5 model is Mosaic, that is, the combination of the calibration frames and the combination and splicing of the pictures are performed on the read 4 images each time through operations such as color gamut change, clipping, scaling, random arrangement and the like.
As a further scheme of the invention: the specified training method further comprises:
setting initial hyper-parameters, wherein the training size of the improved YOLO v5 model is 640 multiplied by 640, the batch size is 16, the number of categories is 5, the initial learning rate is 1e-2, and the iteration number is 100;
inputting the training set and the verification set into an improved YOLO v5 model for training;
and according to the training verification loss rate change curve, determining the final learning rate and iteration times when the loss tends to be stable along with the increase of the iteration times.
As a further scheme of the invention: the specified improved method comprises the following steps:
replacing Backbone feature extraction network Backbone of YOLO v5 with MobileNetv 2;
the MobileNetv2 structure comprises:
performing dimensionality increase by using 1 × 1 convolution aiming at a trunk feature extraction part of an improved YOLO v5 model;
after feature extraction is carried out by using 3 × 3 depth separable convolution, dimension reduction is carried out by using 1 × 1 convolution;
the input and the output of the residual edge part are directly connected;
the features are projected to a representation of the low-dimensional features using a linear bottleneck operation.
Specifically, as shown in fig. 2 and 3, the structure of the general YOLO v5 is mainly divided into four parts, namely an input end (input), a Backbone network (Backbone), a transition layer (Neck), and an output layer (Head).
The feature extraction network structure mainly comprises Focus, CSP and SPP, and the middle of the feature extraction network structure is connected through a convolutional layer.
The structure of the Focus module is shown in fig. 4, and the feature map is changed into a feature map of 320x320x12 through a slicing operation; then 32 convolution kernels are used for convolution to obtain a feature map of 320x320x 32. The volume block (CBL) consists of a volume layer (Conv), a normalization layer (BN) and a leakage _ relu activation function.
The CSP module structure is shown in fig. 5, and the CSPDarknet53 is a Backbone feature extraction network Backbone structure modified based on Darknet53, which modifies the resblock _ body structure and adds the CSPnet structure. Therefore, the network learning capability can be effectively enhanced while the network operation accuracy is ensured, the CNN is smaller, the calculation bottleneck can be effectively reduced, and the memory cost can be reduced. After the SPP module is positioned in the feature extraction backbone network, the extraction range of the backbone feature network can be greatly expanded by adopting the SPP module.
Aiming at the real-time requirement of soybean seed test, the method adopts MobileNetv2 to replace a Backbone feature extraction network Backbone of YOLO v5, so as to reduce the parameter quantity of an original model, shorten the model reasoning time and improve the deduction capability of an overall model.
As a representative of the lightweight network, MobileNetv2 is a network based on a deep separable convolution structure, and its core idea is to split the standard convolution into two parts, deep convolution and point-by-point convolution, as shown in fig. 6. Carrying out deep convolution, and carrying out convolution operation on each input channel to obtain an output characteristic diagram with the number consistent with that of input characteristic diagram channels; and performing point-by-point convolution, performing dimension ascending and dimension descending on the feature map by using 1 × 1 convolution operation, and combining all depth convolutions to obtain output. Wherein N, M represents the number of input and output channels, D K ×D K The size of the convolution kernel. The model parameters and the calculated amount are reduced while the model performance is ensured.
MobileNetv2 uses a linear bottleneck inverse residual module to improve feature extraction capability based on deep separable convolution. The core of the system is composed of a BottleneckResidualblock module, and the structure is shown in FIG. 7. Can be divided into two parts: the main part firstly utilizes 1 multiplied by 1 convolution to carry out dimension increasing, and the problem that when the main part passes through an activation function ReLU, too much feature information is lost due to too low dimension is solved; then, respectively utilizing a 3 × 3 depth separable convolution and a 1 × 1 convolution to carry out feature extraction and dimension reduction; the residual edge part input and output are directly connected and finally the features are projected to the representation of the low-dimensional features using a linear bottleneck operation.
As a further scheme of the invention: the specified improved method comprises the following steps:
introducing an ECANet attention mechanism;
the ECANet attention mechanism comprises:
obtaining the global feature of each channel through global average pooling, and outputting the global feature with the dimensionality of C multiplied by 1;
obtaining the relationship between local channels by using one-dimensional convolution;
putting the weight file into an algorithm of a YOLO v5 model to detect the verification set, and observing the detection condition;
using a Sigmoid to activate a function to output a new weight with the dimensionality of C multiplied by 1;
and the new weight and the input feature graph are multiplied to calculate the redistribution of the channel feature weight, inhibit invalid features and enhance valid features.
As a further scheme of the invention: the main steps of introducing an ECANet attention mechanism comprise:
an ECANet attention mechanism is placed on the enhanced feature extraction network, and the attention mechanism is added on three effective feature layers extracted from the main network;
wherein C is the number of channels, and an ECANet attention mechanism is added to the upsampling result in the FPN to reduce information loss and optimize the integration characteristics on each layer.
Aiming at the conditions that partial image leaves are large in number, the leaves are mutually shielded and the small leaves exist in a soybean plant data set, an ECANet attention mechanism is introduced for improving the algorithm identification accuracy and robustness.
The attention mechanism simulates the processing behavior of human brain on visual information, focuses on the interested area, extracts more detailed features of the target, improves the detection effect, namely enhances effective features and inhibits ineffective features, and has a powerful and effective expression form.
ECANet is an improved channel attention mechanism based on SENet. SENET realizes channel information interaction through full connection, and partial information characteristic expression is lost in the process of the channel information interaction and has large calculation overhead. Based on the method, the ECANet realizes information interaction between channels through one-dimensional convolution based on a method for adaptively selecting the size of a convolution kernel, so that the complexity of the model is greatly reduced on the basis of keeping the performance of the model.
FIG. 8 is a diagram showing the structure of ECANet. First, global features of each channel are obtained through global average pooling, and the global features with the dimension of C (the number of channels) multiplied by 1 are output. And secondly, the relationship between local channels is directly obtained by using one-dimensional convolution, so that the loss of key information and larger parameter calculation amount caused by the dimension reduction process are avoided. The Sigmoid activation function is then used to output a new weight with dimensions C × 1 × 1. And finally, completing the redistribution of the channel feature weight by the product operation of the new weight and the input feature graph, inhibiting the invalid features and enhancing the valid features.
In order to extract more detailed characteristics of soybean plant leaf image, an ECANet attention mechanism is firstly placed on an enhanced characteristic extraction network, and the attention mechanism is added on three effective characteristic layers extracted from a main network. Aiming at the problems of information attenuation, cross-scale fusion aliasing effect and channel reduction inherent defects in the process of fusing information in the FPN in the YOLO v5, the invention adds an ECANet attention mechanism to the up-sampling result in the FPN to reduce information loss and optimize the integration characteristics on each layer. Through the introduction of the ECANet attention mechanism, the model can better fit the relevant characteristic information among small target channels, useless information is ignored and suppressed, the model can be finally concentrated on the specific category of training soybean plant leaves, the training of the characteristics is enhanced, and the detection performance of the model on the soybean plant data set is improved.
As a further scheme of the invention: the specified improved method comprises the following steps:
modified loss function:
replacing GIOU _ Loss with CIOU _ Loss;
the classical NMS is replaced with DIOU _ NMS.
Wherein, replacing the GIOU _ losseyolo v5 with the CIOU _ Loss uses the GIOU _ Loss as a Loss function of the bounding box (target location), and uses the binary cross entropy and logs Loss function to calculate the class probability and the Loss of the target score, which are defined as:
in the above formula, A, B is the intersection ratio of the prediction frame and the real frame, IOU is the intersection ratio of the prediction frame and the real frame, and C is the minimum bounding rectangle of the prediction frame and the target frame. However, only the factor of the overlap ratio between the prediction frame and the target frame is considered, the regression problem of the target frame cannot be described well, when the prediction frame is inside the target frame and the sizes of the prediction frames are the same, the GIOU is degenerated to the IOU at this time, the relative positions of the prediction frames in each target frame cannot be distinguished, and thus the detection is inaccurate, as shown in fig. 9.
Therefore, CIOU _ Loss is used to replace GIOU _ Loss, and the calculation formula is as follows:
where a is a balance parameter not involved in the gradient calculation, and v is a parameter for measuring the uniformity of the aspect ratio. The CIOU _ Loss comprehensively considers the factors such as the overlapping area, the center point distance, the length-width ratio and the like of the target frame and the prediction frame, and overcomes the defects of a GIOU _ Loss Loss function, so that the regression process of the target frame is more stable, the convergence speed is higher, and the convergence precision is higher.
The method is characterized in that a Non-Maximum-value-suppression (NMS) technology is commonly used in a target detection model post-processing process, DIOU _ NMS is used for replacing the classical NMS Non-Maximum-value suppression (NMS), the Non-Maximum-value-suppression (NMS) technology is used for reserving the most accurate detection frames and removing redundant detection frames, IOUs are the only screening indexes in the detection frame processing process, each detection frame corresponds to an IOU value, and all detection frames exceeding the NMSThreshold set value are removed. However, in the practical situation of the soybean plant leaves, the leaves are often shielded, that is, two different objects are very close to each other, and at this time, the IOU value is relatively large, so that error suppression is easy to occur, and the condition of missing detection is caused. To avoid missed detection, the classic NMS is replaced by DIOU _ NMS herein. The DIOU loss function is defined as
In the above formula: b is a prediction box; b gt Is a real frame; ρ is a unit of a gradient 2 (b,b gt ) The distance between the central points of the prediction frame and the real frame is taken as the distance; i is the diagonal distance of the minimum bounding rectangle of the 2 frames. Assuming that the algorithm model detects a candidate box set as Hi, for the prediction box M with the highest class confidence, pi of the DIOU _ NMS method update formula can be defined as:
in the formula: DIOU (M, H) i ) For DIOU with respect to M and H i A value of (d); epsilon is a threshold value manually set by NMS operation; p is a radical of i Classifying scores of target categories of different soybean plants; IOU is cross-over ratio; i is the number of the anchor frames corresponding to each grid.
The DIOU _ NMS not only focuses on the IOU, but also focuses on the distance, the overlapping area and the aspect ratio between the prediction frame and the real frame, and considers that 2 prediction frames with a longer central point distance may be located on different target detection objects, and combines the intersection ratio (IOU) of 2 rectangular frames (the prediction frame and the real frame) with the central point distance, so as to optimize the IOU loss on the one hand, and guide the learning of the central point on the other hand, so that the prediction frame can be regressed more accurately, which is beneficial to the identification of the target with overlapped occlusion.
The specific structure of the improved YOLO v5 model of the invention is shown in FIG. 10. Extracting effective characteristics of the soybean plant images in three scales of 52 x 52, 26 x 26 and 13 x 13 by adopting a lightweight network MobileNetv2 as a main network, and then effectively excavating depth information of soybean plants in a characteristic diagram and weakening irrelevant characteristics by ECANet attention mechanism enhancement; applying an FPN + PANet + ECANet framework to the feature map in the Neck part to perform feature fusion from bottom to top and from top to bottom, fully utilizing high-level semantic information and bottom-level spatial information in the feature map, and aggregating features to the maximum extent; on the basis, the introduction of CIOULoss + DIOU _ NMS enables the network to pay extra attention to the position information of the central point of the boundary box so as to more accurately return the small target and the shielding target prediction box. And finally outputting the result with the label, the confidence coefficient and the frame coordinate information.
In order to evaluate the performance of the improved YOLO v5 model soybean plant target detection model, improved YOLO v5, YOLO v5, SSD and fast-RCNN target detection network models are selected, as shown in FIG. 11, the optimal training parameter settings of the models are selected, and the training results of the models are compared according to indexes such as precision (P), recall (R), harmonic mean (F1), frame rate (FPS) and the like, so that the improved YOLO v5 is obtained with the best real-time performance and the lowest error detection and omission rates.
In order to more intuitively verify the feasibility and robustness of the improved YOLO v5 model in practical application, part of soybean plant leaf images are selected in a test set for testing, the visual results of the improved YOLO v5, YOLO v5, SSD and fast-RCNN on the detection results of soybean plant leaves are compared, the improved YOLO v5 has higher detection confidence, only the improved YOLO v5 in 4 algorithms has no abnormal condition of missed detection, and in the case of dense target images, the improved YOLO v5 in 4 algorithms has lowest false detection and missed detection rate, so that the practical requirements of soybean species test can be well met.
When the phenotypic characteristics of the soybean plants output by the identification model are actually counted, the improved YOLO v5 model is adopted to identify and count the leaf shapes, the leaf blade numbers, the flower colors and the flower numbers of the soybean plants; and displaying and storing the result of the model detection, and providing a data basis for further breeding analysis.
Therefore, the soybean plant phenotype data statistical system has three main functions: target detection, result display and data management;
the target detection module has the functions of identifying and counting the leaf shape, the leaf number, the flower color and the flower number of the soybean plant based on an improved YOLO v5 target detection algorithm, and stores a prediction result;
the data management module mainly provides functions of increasing, deleting, modifying and searching soybean data, faces agricultural technicians and scientific researchers, and provides a data basis for breeding analysis work;
the system adopts a design mode with a front end and a rear end separated, the model is deployed at the Web end, the front end adopts a lightweight frame VUE, the rear end adopts a Web frame flash frame developed based on Python, and the B/S framework is adopted for design.
The specific implementation process is as shown in fig. 12, after a user uploads a soybean plant image on a browser, the user selects a corresponding module to perform the next operation, an HTTP request sends a message to an HTTP Server, the HTTP Server sends POST information to a flash Server, the flash Server calls an API of a model dynamic link library to perform detection, the detection result includes the leaf shape, the number of blades, the flower color, the number of flowers and the leaf area of the soybean plant, the detection result returns to the browser through the flash Server and the HTTP Server in the form of JSON data, the browser receives the JSON data, and finally the detection result is displayed at the browser end.
The initial client interface of the soybean plant phenotype data statistical system is shown in fig. 13, the leaf shape recognition and counting is shown in fig. 14, the flower recognition and counting is shown in fig. 15, and the data management interface is shown in fig. 16. Wherein, the header information is a system title; the middle information comprises a module component, a detection component and a detection result display component, and the soybean plant phenotype statistical system platform has the functions of soybean seed test and data storage and can well assist breeding experts in developing breeding work.
While one embodiment of the present invention has been described in detail, the description is merely a preferred embodiment of the present invention and should not be taken as limiting the scope of the invention. All equivalent changes and modifications made within the scope of the present invention shall fall within the scope of the present invention.
Claims (9)
1. A soybean plant phenotype data statistical method based on an improved YOLO v5 model is characterized by comprising the following steps:
obtaining a sampling picture of a target soybean plant;
identifying the sampling picture through an identification model trained based on a specified training method;
counting and saving phenotypic characteristics output by the recognition model about the soybean plants;
wherein the identification model is an improved YOLO v5 model improved based on a specified improvement method.
2. The statistical method of soybean plant phenotype data based on the improved YOLO v5 model of claim 1, wherein the designated training method comprises:
acquiring a training sample set aiming at soybean plants;
enhancing and expanding the training sample set to obtain a training data set;
and randomly dividing the training data set into a training set, a verification set and a test set.
3. The soybean plant phenotype data statistics method based on the improved YOLO v5 model of claim 2, wherein the method for obtaining the training sample set for soybean plants comprises:
shooting the soybean plants at different time periods, different weather conditions and different angles to obtain shot images;
and labeling all the shot images to obtain the training sample set, wherein the labels comprise various category labels.
4. The statistical method for soybean plant phenotype data based on improved YOLO v5 model of claim 3, wherein the method for enhancing the training data set obtained after expanding the training sample set comprises:
performing data enhancement on each shot image in the training sample set;
and performing combination of calibration frames and combination and splicing of pictures on the read 4 images through operations of color gamut change, cutting, scaling, random arrangement and the like to obtain a training data set.
5. The statistical method for soybean plant phenotype data based on the improved YOLO v5 model according to claim 2, wherein the designated training method further comprises:
setting initial hyper-parameters, wherein the training size of the improved YOLO v5 model is 640 multiplied by 640, the batch size is 16, the number of categories is 5, the initial learning rate is 1e-2, and the iteration number is 100;
inputting the training set and the verification set into the improved YOLO v5 model for training;
and according to the training verification loss rate change curve, determining the final learning rate and iteration times when the loss tends to be stable along with the increase of the iteration times.
6. The statistical method of soybean plant phenotype data based on the improved YOLO v5 model of claim 1, wherein the specified improvement method comprises:
replacing Backbone feature extraction network Backbone of YOLO v5 with MobileNetv 2;
the MobileNetv2 structure comprises:
performing dimensionality increase by using 1 × 1 convolution aiming at a trunk feature extraction part of an improved YOLO v5 model;
after feature extraction is carried out by using 3 × 3 depth separable convolution, dimension reduction is carried out by using 1 × 1 convolution;
the input and the output of the residual edge part are directly connected;
projecting the features onto a representation of the low-dimensional features using a linear bottleneck operation;
the Backbone feature extraction network Backbone of YOLO v5 is replaced by MobileNetv 2.
7. The statistical method of soybean plant phenotype data based on the improved YOLO v5 model of claim 6, wherein the specified improvement method comprises:
introducing an ECANet attention mechanism;
the ECANet attention mechanism comprises the following structures:
obtaining the global feature of each channel through global average pooling, and outputting the global feature with the dimension of C multiplied by 1;
obtaining the relationship between local channels by using one-dimensional convolution;
using a Sigmoid to activate a function to output a new weight with the dimensionality of C multiplied by 1;
and the new weight and the input feature graph are multiplied to calculate the redistribution of the channel feature weight, inhibit invalid features and enhance valid features.
8. The soybean plant phenotype data statistics method based on improved YOLO v5 model of claim 7, characterized in that the main step of introducing ECANet attention mechanism comprises:
an ECANet attention mechanism is placed on the enhanced feature extraction network, and the attention mechanism is added on three effective feature layers extracted from the main network;
and an ECANet attention mechanism is added to the up-sampling result in the FPN, so that information loss is reduced, and the integrated feature on each layer is optimized.
9. The statistical method of soybean plant phenotype data based on the improved YOLO v5 model of claim 1, wherein the specified improvement method comprises:
modified loss function:
replacing GIOU _ Loss with CIOU _ Loss;
the classical NMS is replaced with DIOU _ NMS.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210441866.0A CN115099297B (en) | 2022-04-25 | 2022-04-25 | Soybean plant phenotype data statistical method based on improved YOLO v5 model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210441866.0A CN115099297B (en) | 2022-04-25 | 2022-04-25 | Soybean plant phenotype data statistical method based on improved YOLO v5 model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115099297A true CN115099297A (en) | 2022-09-23 |
| CN115099297B CN115099297B (en) | 2024-06-18 |
Family
ID=83287154
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210441866.0A Active CN115099297B (en) | 2022-04-25 | 2022-04-25 | Soybean plant phenotype data statistical method based on improved YOLO v5 model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115099297B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116434066A (en) * | 2023-04-17 | 2023-07-14 | 安徽农业大学 | Deep learning-based soybean pod seed test method, system and device |
| CN117218615A (en) * | 2023-08-29 | 2023-12-12 | 武汉理工大学 | Soybean pod-falling phenotype investigation method |
| CN117809105A (en) * | 2023-12-28 | 2024-04-02 | 西南交通大学 | Rubbish identification and classification method for color similar gray scale re-verification |
| CN117853801A (en) * | 2024-01-09 | 2024-04-09 | 华中农业大学 | Deep learning classification model and application thereof in corn multiple growth stage phenotype detection |
| CN117975172A (en) * | 2024-03-29 | 2024-05-03 | 安徽农业大学 | Method and system for constructing and training whole pod recognition model |
| WO2024119304A1 (en) * | 2022-12-05 | 2024-06-13 | 深圳华大生命科学研究院 | Yolov6-based directed target detection network, training method therefor, and directed target detection method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2020103494A4 (en) * | 2020-11-17 | 2021-01-28 | China University Of Mining And Technology | Handheld call detection method based on lightweight target detection network |
| CN113392775A (en) * | 2021-06-17 | 2021-09-14 | 广西大学 | Sugarcane seedling automatic identification and counting method based on deep neural network |
| CN113822198A (en) * | 2021-09-23 | 2021-12-21 | 华南农业大学 | Peanut growth monitoring method, system and medium based on UAV-RGB image and deep learning |
-
2022
- 2022-04-25 CN CN202210441866.0A patent/CN115099297B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2020103494A4 (en) * | 2020-11-17 | 2021-01-28 | China University Of Mining And Technology | Handheld call detection method based on lightweight target detection network |
| CN113392775A (en) * | 2021-06-17 | 2021-09-14 | 广西大学 | Sugarcane seedling automatic identification and counting method based on deep neural network |
| CN113822198A (en) * | 2021-09-23 | 2021-12-21 | 华南农业大学 | Peanut growth monitoring method, system and medium based on UAV-RGB image and deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 熊俊涛;刘振;林睿;陈淑绵;陈伟杰;杨振刚;: "自然环境下树上绿色芒果的无人机视觉检测技术", 农业机械学报, no. 11, 12 September 2018 (2018-09-12) * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024119304A1 (en) * | 2022-12-05 | 2024-06-13 | 深圳华大生命科学研究院 | Yolov6-based directed target detection network, training method therefor, and directed target detection method |
| CN116434066A (en) * | 2023-04-17 | 2023-07-14 | 安徽农业大学 | Deep learning-based soybean pod seed test method, system and device |
| CN116434066B (en) * | 2023-04-17 | 2023-10-13 | 安徽农业大学 | Deep learning-based soybean pod seed test method, system and device |
| CN117218615A (en) * | 2023-08-29 | 2023-12-12 | 武汉理工大学 | Soybean pod-falling phenotype investigation method |
| CN117218615B (en) * | 2023-08-29 | 2024-04-12 | 武汉理工大学 | Soybean pod-falling phenotype investigation method |
| CN117809105A (en) * | 2023-12-28 | 2024-04-02 | 西南交通大学 | Rubbish identification and classification method for color similar gray scale re-verification |
| CN117853801A (en) * | 2024-01-09 | 2024-04-09 | 华中农业大学 | Deep learning classification model and application thereof in corn multiple growth stage phenotype detection |
| CN117975172A (en) * | 2024-03-29 | 2024-05-03 | 安徽农业大学 | Method and system for constructing and training whole pod recognition model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115099297B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115099297B (en) | Soybean plant phenotype data statistical method based on improved YOLO v5 model | |
| CN113392775B (en) | Sugarcane seedling automatic identification and counting method based on deep neural network | |
| CN113705478B (en) | Mangrove single wood target detection method based on improved YOLOv5 | |
| Liu et al. | A vision-based robust grape berry counting algorithm for fast calibration-free bunch weight estimation in the field | |
| Rudolph et al. | Efficient identification, localization and quantification of grapevine inflorescences and flowers in unprepared field images using Fully Convolutional Networks | |
| CN110569747A (en) | method for rapidly counting rice ears of paddy field rice by using image pyramid and fast-RCNN | |
| CN114005029B (en) | Method and system for identifying disease and insect pests of bergamot based on improved yolov network | |
| CN110967339A (en) | Method and device for analyzing corn ear characters and corn character analysis equipment | |
| CN110287882A (en) | A kind of big chrysanthemum kind image-recognizing method based on deep learning | |
| Liu et al. | Deep learning based research on quality classification of shiitake mushrooms | |
| CN110533100A (en) | A method of CME detection and tracking is carried out based on machine learning | |
| CN112464762A (en) | Agricultural product screening system and method based on image processing | |
| CN115546187A (en) | Agricultural pest and disease detection method and device based on YOLO v5 | |
| Valiente et al. | Non-destructive image processing analysis for defect identification and maturity detection on avocado fruit | |
| CN112966698A (en) | Freshwater fish image real-time identification method based on lightweight convolutional network | |
| CN112465821A (en) | Multi-scale pest image detection method based on boundary key point perception | |
| CN116385717A (en) | Foliar disease identification method, foliar disease identification device, electronic equipment, storage medium and product | |
| Nagpal et al. | An application of deep learning for sweet cherry phenotyping using yolo object detection | |
| CN116258844A (en) | Rapid and accurate identification method for phenotype character of cotton leaf | |
| CN114972264A (en) | Method and device for identifying mung bean leaf spot based on MS-PLNet model | |
| Rony et al. | BottleNet18: Deep Learning-Based Bottle Gourd Leaf Disease Classification | |
| Jing et al. | Optimizing the YOLOv7-Tiny Model with Multiple Strategies for Citrus Fruit Yield Estimation in Complex Scenarios | |
| CN112488230A (en) | Crop water stress degree judging method and device based on machine learning | |
| CN110533030A (en) | Sun film image timestamp information extracting method based on deep learning | |
| Huber et al. | A Concept Study for Feature Extraction and Modeling for Grapevine Yield Prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |