CN115527192A

CN115527192A - Rape seedling stage hybrid positioning method and hybrid removing method

Info

Publication number: CN115527192A
Application number: CN202211292590.0A
Authority: CN
Inventors: 张兵; 王新宇; 姜子扬
Original assignee: Nanjing Jimu Robot Technology Co ltd
Current assignee: Nanjing Jimu Robot Technology Co ltd
Priority date: 2022-10-21
Filing date: 2022-10-21
Publication date: 2022-12-27

Abstract

The invention provides a rape seedling stage hybrid positioning method and a hybrid removing method, wherein the rape seedling stage hybrid positioning method comprises the following steps: obtaining image maps of a plurality of seedling-stage plants in a plot; identifying all seedling-stage plants in the image map to obtain plant pictures, and configuring index information for each plant picture; extracting the characteristics of each plant picture based on the high-dimensional characteristics of the preset dimensions to obtain characteristic vectors; calculating Euclidean distance between any two characteristic vectors, and determining the outlier index of each characteristic vector; presetting an outlier index threshold value, and screening plant pictures corresponding to the feature vectors of which the outlier indexes exceed the preset outlier index threshold value; and determining the position information of the screened plant pictures according to the index information of the screened plant pictures to obtain the position information of the hybrid plants. The method is used for solving the problem that the database in the prior art needs a large amount of variety and characteristic data and still cannot achieve the high-precision hybrid identification effect.

Description

Rape seedling stage hybrid positioning method and hybrid removing method

Technical Field

The invention relates to the technical application of image recognition in the field of seed production agriculture, in particular to a rape seedling stage hybrid plant positioning method and a hybrid plant removing method.

Background

Crossing refers to the first generation of hybrids prepared using two different crop varieties or lines, where the hybrid crop has significant heterosis and is an important way to develop crop production. Taking rape as an example, the seedling stage is a key stage of crop seed production, if main hybrid plants in the field can be removed in the seedling stage, a better growth environment can be provided for breeding, and the field-patrol roguing workload in the bolting stage and the flowering stage can be effectively reduced, so that the roguing in the seedling stage is an important work in the hybrid crop seed production process. In the field for hybrid seed production of crops, only one variety or two specific varieties can be usually planted, and the crops with other varieties in the field for seed production are hybrid plants to be removed. In the prior art, the traditional impurity removal method mainly judges that the impurities in the field crop seedlings are removed through manual field patrol and human eye comparison.

The mode of distinguishing the hybrid plants completely by human eyes in the prior art is very dependent on the agricultural skill experience of workers, the workers can only patrol the field for work by accumulating long-term agricultural work experience, the labor cultivation cost is high, the mode of distinguishing the hybrid plants completely by human eyes has the problem of low efficiency, complete hybrid plant removal is difficult to achieve for large-area seed production plots, and then purity evaluation of seed production fields is influenced, so that the image recognition technology is applied to the field of crop hybrid plant recognition.

The traditional Chinese patent with publication number CN114140702A discloses a method for removing impurity plants and improving seed production purity of a hybrid rice seed production field, wherein an unmanned aerial vehicle is used for carrying a CCD camera to shoot field rice plants in a rice plant type period, real-time video stream data is generated, rice images are collected from the real-time video stream data, and the rice images are identified through an impurity plant identification model to detect whether impurity plants exist in the images; and if the mixed plants are detected, outputting position coordinates of the mixed plants in the image, wherein the position coordinates of the mixed plants comprise the position information of the rows and the columns of the distributed plants, transmitting the position coordinates of the mixed plants in the image to a mixed plant removing device, and identifying and removing the mixed plants from the position of the mixed plant removing device to the position of the mixed plants.

In the above-described conventional techniques, identification is performed directly by the hybrid identification model, and the variety of the hybrid is required to be determined in advance every time identification is performed, and the identification model can identify the variety of the hybrid. Because different varieties of hybrid plants may exist in the same plot, the variety of each hybrid plant needs to be confirmed in advance for identification, the applicability is low, the number of all varieties of rice exceeds 14 thousands, the establishment of a database containing all varieties of rice is not practical, and the hybrid plants cannot be identified when not existing in the database. And the characteristic difference between the hybrid plant and the target object is small, the accuracy rate of directly identifying the hybrid plant is low, and the hybrid plant removing effect is poor, so that the requirement of the seed production field on the purity of more than 99.7 percent of seeds is difficult to meet, a high-precision hybrid plant model needs a large amount of data support, a database needs a large amount of characteristic data of each variety on the basis of a large amount of varieties, and great technical difficulty exists.

In view of the above, there is a need to improve the detection method of hybrid plants in seedling stage in the prior art to solve the above problems.

Disclosure of Invention

The invention aims to disclose a rape seedling stage hybrid positioning method and a hybrid removing method, which are used for solving the problems that in the prior art, hybrid varieties need to be known in advance and the hybrid needs to be uploaded in a database in advance, so that the database needs to upload almost all variety pictures of the same crop, and each variety needs to upload a large amount of characteristic data correspondingly, and the high-precision hybrid identification effect is still difficult to achieve.

In order to realize the aim, the invention provides a rape seedling stage hybrid positioning method, which comprises the following steps:

obtaining image maps of a plurality of seedling-stage plants in a plot;

identifying all seedling-stage plants in the image map to obtain plant pictures, and configuring index information for each plant picture;

extracting the characteristics of each plant picture based on the high-dimensional characteristics of the preset dimensions to obtain characteristic vectors;

calculating Euclidean distance between any two feature vectors, and determining an outlier index of each feature vector;

presetting an outlier index threshold, and screening plant pictures corresponding to the feature vectors of which the outlier indexes exceed the preset outlier index threshold;

and determining the position information of the screened plant pictures according to the index information of the screened plant pictures to obtain the position information of the hybrid plants.

As a further improvement of the invention, the step of identifying all the seedling plants in the image map to obtain the plant picture comprises the following steps: and identifying the seedling-stage plants in the image map through a preset seedling-stage plant model, and determining plant pictures adapting to the sizes of the identified seedling-stage plants according to the identified seedling-stage plants.

As a further improvement of the present invention, before the step of extracting the feature of each plant picture based on the high-dimensional feature of the preset dimension to obtain the feature vector, the method further comprises: the size of each plant picture was standardized so that the size of each plant picture was consistent.

As a further improvement of the present invention, before the step of extracting the features of each plant picture based on the high-dimensional features of the preset dimensions to obtain the feature vector, the method further comprises: and distinguishing the foreground plant part and the background plant part of each plant picture so as to keep the foreground plant part.

As a further improvement of the invention, the step of extracting the features of each plant picture based on the high-dimensional features of the preset dimensions to obtain the feature vector comprises the following substeps:

pre-deploying a plurality of models with different orders of magnitude;

and dynamically calling one or more order models according to the actual calculation power to simultaneously extract the characteristics of a plurality of plant pictures so as to obtain the characteristic vector.

As a further improvement of the present invention, the step of calculating the euclidean distance between any two feature vectors and determining the outlier index of each feature vector comprises the following sub-steps:

calculating Euclidean distance between each eigenvector and other eigenvectors to form an eigenvector matrix, wherein each row of the eigenvector matrix corresponds to one eigenvector;

and summing each row of the feature matrix to obtain the outlier index of the feature vector.

As a further improvement of the invention, the step of screening the plant pictures corresponding to the feature vectors of which the outlier index exceeds the preset outlier index threshold value comprises the following substeps:

obtaining the planting proportion corresponding to each variety of the plant in the seedling stage in the plot;

determining an outlier index threshold of the variety to be identified according to the planting proportion of the variety to be identified;

and screening the plant picture corresponding to the feature vector of which the outlier index exceeds the outlier index threshold value of the variety to be identified as the variety picture to be identified.

As a further improvement of the invention, the step of configuring the index information for each plant picture comprises the following substeps: and configuring index information for each plant picture of each image map, wherein the index information comprises an image map ID and a plant picture number corresponding to the plant picture.

As a further improvement of the invention, the high-dimensional features need to be trained by an optimized loss function to extract a model, so as to minimize Euclidean distances between homogeneous features and maximize Euclidean distances between heterogeneous features.

The invention also discloses a rape seedling stage hybrid removing method, which is realized based on the rape seedling stage hybrid positioning method and comprises the following steps:

setting a plot area threshold value, and comparing the current plot area with the plot area threshold value;

if the current plot area is smaller than the plot area threshold value, acquiring an image map in real time by adopting an unmanned vehicle to obtain the position information of the weeds so as to realize real-time removal;

and if the current plot area is larger than the plot area threshold value, acquiring a plurality of image maps covering the whole current plot by adopting an unmanned aerial vehicle to obtain the position information of the mixed plants, and planning a path according to the position information of the mixed plants to realize remote control removal.

Compared with the prior art, the invention has the beneficial effects that:

firstly, obtaining an image map containing a plurality of plants in a plot, then identifying the plants in the seedling stage in the image map to obtain a plant picture only containing one plant, extracting high-dimensional features of the plant picture, obtaining feature vectors, calculating Euclidean distance between any two feature vectors, obtaining an outlier index of each plant through the Euclidean distance, comparing each calculated outlier index with a preset outlier index threshold value, screening out the plant picture corresponding to the outlier index with the value larger than the outlier index threshold value, judging that the plant corresponding to the plant picture is a mixed plant, and determining position information of the plant picture according to the index information of the screened plant picture, wherein the position information is the position of the mixed plant. Compared with the prior art that the hybrid variety needs to be known firstly before the hybrid identification, the hybrid variety needs to be in a database, and in order to achieve the purpose, massive variety pictures in the database need to be uploaded, each variety needs a large amount of feature data to support, and the purity requirement of over 99.7% required by cross breeding is still difficult to achieve, in the invention, the high-dimensional features of each plant in the image are extracted, the Euclidean distance between feature vectors is calculated, the outlier index is obtained, the outlier index is compared with the preset outlier index threshold, when the outlier index exceeds the preset outlier index threshold, the plant corresponding to the outlier index can be judged to be the hybrid, the variety of the hybrid is not needed to be known in advance, meanwhile, the hybrid is not needed to exist in the database when the hybrid is identified, the efficiency is improved in a mode of uploading a large amount of features in the database through the outlier index judgment of the hybrid, the cross identification precision is effectively improved, and the purity requirement of the breeding on 99.7% is met. Due to the configuration of the index information, when the plant picture of the hybrid plant is identified, the position of the hybrid plant is directly positioned according to the index information so as to carry out subsequent removal on the hybrid plant.

Drawings

FIG. 1 is a flow chart of the hybrid identification and positioning process of the hybrid positioning method in the seedling stage of the present invention;

FIG. 2 is a flowchart illustrating the step S2 of the present invention;

FIG. 3 is a flowchart illustrating the steps S3 of the present invention;

FIG. 4 is a flowchart illustrating the operation of step S4 according to the present invention;

FIG. 5 is a flowchart illustrating the steps S5 of the present invention;

FIG. 6 is a flow chart of the method for removing the hybrid plants in the seedling stage of the rape in the invention;

FIG. 7 is a detailed flowchart for describing steps S0 and S7 in the present invention;

FIG. 8 is a schematic diagram illustrating a rogue operation according to the present invention.

Detailed Description

The present invention is described in detail with reference to the embodiments shown in the drawings, but it should be understood that these embodiments are not intended to limit the present invention, and those skilled in the art should understand that functional, methodological, or structural equivalents or substitutions made by these embodiments are within the scope of the present invention.

Briefly, the positioning method for hybrid plants in seedling stage of rape disclosed in the embodiments of the present application can be used for performing hybrid identification and positioning on hybrid plants in seedling stage of hybrid crops in a plot. In the case of rape, the varieties used for hybridization in the same hybridization plot are usually one variety or two designated varieties as male parent and female parent, and the other varieties except the one variety or two varieties used for hybridization are all mixed plants, and the mixed plants in the plot need to be removed to ensure the purity of the hybrid seeds. The rape impurity removal comprises seedling-stage impurity removal and bud-stage impurity removal, wherein the seedling-stage impurity removal is used for distinguishing the impurity plants for removal according to the characteristics of seedling phase, leaf morphology, leaf color, leaf margin and the like of the plants; the difference between bud period roguing and seedling period roguing lies in that the bud phase characteristics are changed into bud morphology characteristics and leaf morphology characteristics are changed into plant type characteristics, and compared with the bud phase characteristics and leaf morphology characteristics, the bud morphology characteristics and plant type characteristics are more difficult to distinguish, so that seedling period roguing is a more common roguing means for hybrid seed production. However, even if the difficulty of rogue in the seedling stage is smaller than that of rogue identification in the bud stage, the method for identifying the rogue in the prior art still needs to know the varieties of the rogue in the plot in advance and ensure that all the varieties of the rogue in the plot exist in the database, so that a large number of photos of the varieties of crop plants need to be uploaded in the database, and the database needs to be maintained in real time as the varieties of the crop continuously increase, and new varieties of the photos are uploaded to enter the database. In the process of identifying the plants in the seedling stage, the characteristics of the seedling phase, the leaf shape, the leaf color, the leaf margin and the like need to be identified, and in order to improve the identification accuracy, the mass variety pictures in the database need to upload mass photos containing the characteristics corresponding to each variety so as to support the requirement of characteristic identification. The difficulty of realizing the database is extremely high, and even if the database is realized, all plant characteristics are still difficult to cover, when the hybrid characteristics do not exist in the database, the hybrid identification is difficult to realize, so that the hybrid identification and removal are not thorough, and the 99.7% of seed purity required by hybrid seed production is difficult to achieve.

According to the positioning method for the hybrid plants in the seedling stage of the rape, the seedlings existing in the plot are identified in advance, each plant picture is intercepted, index information is configured for each plant picture, the characteristic vector of high-dimensional features in each plant picture is extracted, the Euclidean distance between any two characteristic vectors is calculated, the corresponding outlier index of each plant picture is calculated according to the obtained Euclidean distance, and whether the outlier index is a hybrid plant along with the corresponding special plant piece is judged according to the comparison between the outlier index and the preset outlier index threshold value. Compared with the seedling-stage plant impurity removal method in the prior art, the method has the advantages that the variety of the impurity plant is not required to be known in advance, the variety of the impurity plant is not required to exist in a database, and the types of the variety pictures required to be uploaded in the database are effectively reduced; the method for extracting the feature vectors and calculating the outlier index effectively improves the accuracy of the hybrid plant identification, further reduces the number of feature pictures to be uploaded for each variety in the database, and reduces the model training amount for practical application. And after the hybrid plants are identified, the index information configured by the plant pictures is convenient for positioning the identified hybrid plants at the highest speed so as to remove the hybrid plants in the subsequent process, thereby improving the efficiency of removing the hybrid plants and further improving the purity of seed production.

Referring to fig. 1 to 5, a hybrid identification and positioning process for positioning a hybrid at a seedling stage of a rape (hereinafter referred to as positioning method) includes steps S1 to S6. The positioning method aims to identify each seedling-stage plant in an image map containing a plurality of seedling-stage plants, obtain a plant picture corresponding to each identified plant, and configure index information for each plant picture for subsequent positioning. Extracting high-dimensional features of each plant picture, calculating Euclidean distance through the obtained feature vectors to obtain an outlier index corresponding to each plant picture, comparing the outlier index with a preset outlier index threshold value one by one, and when the outlier index exceeds the outlier index threshold value, knowing that the plant picture corresponding to the outlier index has a large difference with other plant pictures, so that the plant shot by the plant picture corresponding to the outlier index is obtained as a mixed plant, and determining the position of the screened impurities through preset index information for subsequent removal. Compared with the traditional hybrid plant identification method, the method has the advantages that the outlier index is obtained by calculating the Euclidean distance, whether the plant picture is the hybrid plant can be judged by comparing the outlier index with the preset outlier index threshold value, the hybrid plant identification and judgment with high accuracy can be carried out on the plant in the seedling stage without establishing a full-variety crop database, and the precision requirement of the cross breeding is met by more than 99%.

The positioning method of the hybrid plants in the seedling stage of the rape is used for the hybrid breeding of the seedlings of the hybrid breeding plots of the plants such as but not limited to the rape planting. The crops planted in a plot subjected to cross breeding are usually a specific variety or two varieties, all varieties except the specific variety appear in the plot, and the appearance of the hybrid is usually seeds or weeds of other varieties mixed in the seeds of the specific variety when the crops are planted (the weeds do not influence the purity of breeding), so that the number of the hybrid in the plot is obviously smaller than that of the plants of the specific variety used for breeding. Based on the situation, the method comprises the following steps:

s1, obtaining image maps of a plurality of seedling-stage plants in a plot. In the step S1, the seedling-stage plants in the plot are shot by adopting mobile operation equipment (including an unmanned aerial vehicle, a ground walking vehicle and the like) with a shooting function to obtain a plurality of image maps of the rows of the seedling-stage plants containing the main planting types of the plot, and the step provides preparation conditions for subsequent identification and mixed plant positioning.

And S2, identifying all seedling-stage plants in the image map to obtain plant pictures, and configuring index information for each plant picture. In this embodiment, step S2 is implemented based on a deep learning object detection model trained by yolov5 structure, and all recognition objects (i.e., plants) in the plot are subjected to first recognition screening and interception without distinguishing varieties by a method of recognizing and marking all seedling stage plants in the plot through low-dimensional features, and the images obtained by screening are a set including specific varieties or mixed plants. The seedling stage plant identification screening without distinguishing varieties can achieve the identification precision with the accuracy rate of more than 99.7 percent on the basis of less training sets, and effectively reduces the learning pressure and the identification quantity for the accurate identification of the subsequent hybrid plants so as to improve the identification accuracy of the hybrid plants. Establishing a first model to identify all plants in the seedling stage in the image map obtained in the step S1, then intercepting each identified plant as a plant picture, and establishing index information between the plant picture and the image map obtained in the step S1 so as to be related to the image map after the position of the impurity plant is identified subsequently, and further directly finding the position of the impurity plant in the plot for removal.

And S3, extracting the characteristics of each plant picture based on the high-dimensional characteristics of the preset dimensionality to obtain a characteristic vector. And (3) extracting the high-dimensional features of all the seedling-stage plant pictures obtained in the step S2 through a convolutional neural network, and extracting one high-dimensional feature from each picture. As an example, in this embodiment, the high-dimensional feature may be a feature vector of 1 × 2048, the dimensional value may be freely set according to the requirements of the precision and the speed of the computing platform in actual use, the computing speed may be increased by using a smaller dimensional value, the corresponding precision may be lost, the identifying precision may be increased by losing the computing speed when using a larger dimensional value, and the dimension 2048 is a value obtained after multiple experiments and having higher computing precision and proper computing speed.

And S4, calculating the Euclidean distance between any two characteristic vectors, and determining the outlier index of each characteristic vector. And (4) extracting high-dimensional features from each plant picture in the step (S3) to obtain feature vectors, calculating Euclidean distances between any two feature vectors and forming a similar matrix, summing the Euclidean distances of each row in the similar matrix to obtain an outlier index of the corresponding plant picture, and judging whether the corresponding plant picture is a hybrid plant or not according to the obtained numeric value of the outlier index.

S5, presetting an outlier index threshold value, and screening plant pictures corresponding to the feature vectors of which the outlier indexes exceed the preset outlier index threshold value. Setting the outlier threshold value into the conditions that the plot only contains one variety and the two varieties are used for cross breeding, and dynamically adjusting the numerical value of the outlier threshold value according to actual needs when the plot only contains seedling-stage plants of one variety; when the plot contains two varieties of hybrid plants, the two varieties are assumed to be the A variety and the B variety respectively, the planting proportion of the A variety and the B variety in the plot needs to be known in advance in the step of setting the outlier index threshold, when the planting proportion of the A variety is 80% and the planting proportion of the B variety is 20%, the outlier index threshold of the B variety is identified to be 0.2 +/-0.05 (error value), and since the planting proportion of the B variety 20% is lower than the planting proportion of the A variety 80%, the B variety has a larger outlier index value relative to the A variety, the outlier index threshold is set according to the outlier value of the B variety, and when the outlier index calculated in the step S6 is larger than the preset 0.2 +/-0.05 outlier index threshold, the plant picture corresponding to the outlier index can be judged to be a hybrid plant.

And S6, determining the position information of the screened plant pictures according to the index information of the screened plant pictures to obtain the position information of the hybrid plants. And (4) matching the plant pictures obtained in the step (S5) with the index information established between each plant picture and the image map in the step (S2), and further accurately positioning the positions of the hybrid plants in the plot so as to accurately remove the hybrid plants, thereby removing impurities from the cross breeding plot, improving the breeding purity of the cross breeding plot and meeting the requirement on the purity of over 99 percent of cross breeding.

As shown in fig. 2, in the present embodiment, the step S2 includes a substep S21 to a substep S25:

and S21, identifying the seedling-stage plant in the image map through a preset seedling-stage plant model. Firstly, early-stage data set preparation is needed, in the embodiment, the field picture data of nearly 1000 varieties of rapes are adopted, and the rape plants in the field picture data are labeled, so that a data set for detecting the seedling-stage rape plants is established. The preset seedling-stage plant detection model adopts a deep learning algorithm, uses the data set for detecting the seedling-stage rape plants, is matched with yolov5 target detection pipeline for training, and can detect the position coordinates of all the seedling-stage plants in the image map obtained in the step S1.

And S22, determining a plant picture adapting to the size of the seedling stage plant according to the identified seedling stage plant. The image map obtained in the step S1 comprises images of a plurality of seedling-stage rape plants, and after the positions of all seedling-stage rape plants are identified and detected through a preset seedling-stage plant model in the step S21, all identified seedling-stage plants are cut into a plant picture in a step S22.

And S23, standardizing the size of each plant picture so as to enable the size of each plant picture to be consistent. And (4) resetting all the identified plant pictures to the same size so as to facilitate subsequent high-dimensional feature extraction.

And S24, distinguishing the foreground plant part and the background plant part of each plant picture to reserve the foreground plant part. Taking the rape as an example, the background of the plant in the seedling stage is certain land/ground, so the background is certain fixed, and the rape plant in the seedling stage is small and is not enough to cover the ground, so the image processing mode in the step S25 distinguishes the part of the rape plant in the foreground from the part of the ground in the background, removes the background, only keeps the foreground part, and effectively improves the subsequent feature extraction precision.

And S25, configuring index information for each plant picture of each image, wherein the index information comprises an image ID and a plant picture number corresponding to the plant picture. In step S22, all the plant pictures are captured and obtained, the serial numbers are numbered according to the detected IDs of the plant pictures, and index information is established between the serial numbers and the IDs of the image maps of the plant pictures, so that when the plant picture is detected to be a hybrid plant subsequently, the position of the hybrid plant in the image map can be found out through the index information between the serial numbers and the IDs of the image maps, so as to remove the subsequent hybrid plant.

As shown in fig. 3, in the present embodiment, step S3 includes substeps S31 to substep S32:

and S31, deploying a plurality of models with different orders of magnitude in advance. Estimating the computational power of the machine, and deploying a plurality of magnitude models simultaneously, wherein the method comprises the following steps: 32 model, 16 model, 1 model, one or more models can be dynamically invoked according to actual computational power. Since all the plant pictures are resize in the same size in step S23, the multiple-order models deployed in this step can extract/process the features of plant pictures of one batch at a time, and if step S23 is omitted, the plant pictures are different in size, and only one plant picture can be processed at a time, which results in wasted computational power.

And S32, dynamically calling one or more order models according to actual calculation force to simultaneously extract high-dimensional features of multiple plant pictures so as to obtain feature vectors. And training a high-dimensional feature extraction model through the optimized loss function, minimizing Euclidean distances among similar feature vectors and maximizing Euclidean distances among heterogeneous feature vectors. By taking 32 plant pictures in total as an example, the size of the plant pictures is fixed in step S23, 32 models can be directly called to process 32 plant images in batch, the feature extraction efficiency is effectively improved, and feature vectors of the 32 plant pictures are extracted at one time. If 33 plant pictures need to be processed, the following two methods can be adopted: and calling 32 models to process 32 plant pictures at one time, and filling 31 residual blank pictures to continue calling the 32 models, or calling the 32 models and the 1 model simultaneously, wherein the 32 models process 32 plant pictures in batches, and the 1 model processes the residual pictures, and in the step S32, the processing number of the plant pictures is dynamically adjusted according to the calculation power of the deployment environment, usually occupies 80-90% of the calculation power of the equipment, and the calculation power of other programs is reserved, so that the purpose of fully utilizing the calculation power of the equipment is achieved. And obtaining the characteristic vector of each plant picture through the characteristic extraction so as to be used for subsequently calculating the outlier index to judge the hybrid plants.

The Loss function is:

wherein, L is a loss value,

is the Euclidean distance between A1 and A0,

is the euclidean distance between B and A0, and α is the minimum euclidean distance correction coefficient between B and A0, and usually this value α is set to about 0.3 and dynamically adjusted according to actual conditions. When a high-dimensional characteristic extraction model is trained, three seedling-stage plant pictures are read in each time, wherein two seedling-stage plant pictures of the same variety comprise A0 and A1, and the other seedling-stage plant picture of the same variety is B; to judge the difference between different varieties, the model needs to be trained through the loss function and the data set of the rape plant at seedling stage in step S21, and two different varieties are distinguished each time. In this embodiment, taking the seedling stage plants of the a variety and the seedling stage plants of the B variety as an example, in one training process, pictures of A0 and A1 are read from a data set of the a variety (hereinafter referred to as IDA), and one picture of the B variety (hereinafter referred to as B picture) is read from a data set of the B variety (hereinafter referred to as IDB), where about 100 pictures of the seedling stage rape corresponding to the ID are included under the IDA and the IDB, respectively, for feature extraction. In the embodiment, a reid-type feature extraction network with resnet50+ tripleloss is used, pictures of A0, A1 and B are firstly read, the resnet50 features of the three pictures are extracted through a model, then the Euclidean distance between A0 and A1 and the Euclidean distance between A0 and B are respectively calculated in the subsequent steps, the tripleloss is calculated and then reversely transmitted to a training model to optimize parameters in the resnet50 feature extraction network, and the aim of calculating the Euclidean distance of approximately 0 for the feature vectors of the same variety of seedling stage plants and calculating the Euclidean distance of approximately 1 for the feature vectors of different variety of seedling stage plants is fulfilled through a multi-round training iterative model. The rape plant data set in the seedling stage in the step S21 is further packagedThe method comprises 10 data sets such as IDC, IDE, IDF and the like, and field picture data of 1000 varieties of rapes, and each data set is trained sequentially through the steps.

As shown in fig. 4, in the present embodiment, step S4 includes substeps S41 to S42:

s41, calculating Euclidean distance between each feature vector and other feature vectors to form a feature matrix, wherein each row of the feature matrix corresponds to one feature vector. Setting the image map to contain m seedling-stage plants, so that m plant images are obtained by identification and interception, and after Euclidean distances are calculated between any two feature vectors, a feature matrix consisting of m-m Euclidean distances can be obtained, wherein each high-dimensional feature needs to be subjected to distance calculation with the image map, and the obtained value is 0; the remaining euclidean distance values are floating point numbers approaching 0 or 1, respectively.

And S42, summing each row of the feature matrix to obtain an outlier index of the feature vector. Summing up each row of the feature matrix in step S41, an outlier index table consisting of m values is obtained.

As shown in fig. 5, in the present embodiment, step S5 includes substeps S51 to substep S53:

s51, obtaining the planting proportion corresponding to each variety of the plant in the seedling stage in the plot. In the case where two varieties of rape a and B exist in a plot, when the planting ratios of the varieties a and B are not the same, the rape variety with the smaller planting ratio has a larger outlier index than the rape variety with the larger planting ratio, and thus the planting ratio of the two varieties in the plot needs to be known in advance.

And S52, determining the outlier index threshold of the variety to be identified according to the planting proportion of the variety to be identified. When all the rapes in the land are of the same variety, the outlier index threshold is usually set to be about 0.3 and is dynamically adjusted according to the actual situation. When two rapes including an A variety and a B variety are planted in a plot, the planting proportion of the A variety and the B variety in the plot needs to be known in advance in the step of setting the outlier threshold, when the planting proportion of the A variety is 80% and the planting proportion of the B variety is 20%, the outlier threshold of the B variety is identified to be 0.2 +/-0.05 (an error value), and since the planting proportion of the B variety of 20% is lower than the planting proportion of the A variety of 80%, the B variety has a larger outlier numerical value relative to the A variety through calculation, the outlier threshold is set according to the outlier of the B variety.

S53, screening the plant picture corresponding to the feature vector of which the outlier index exceeds the outlier index threshold of the variety to be identified as the variety picture to be identified. Comparing the m numerical values in the outlier index table with the outlier index threshold set in the step S52, when the outlier index is greater than the set outlier index threshold, determining that the plant picture corresponding to the outlier index threshold is the picture of the variety to be identified, that is, determining that the plant picture is a hybrid plant, and positioning the position of the plant picture on the image according to the index relationship between the plant picture number corresponding to the hybrid plant and the image ID in the subsequent step S6 so as to position the hybrid plant for removal.

In order to judge the accuracy of the model in identifying and positioning the hybrid plants of the plants in the seedling stage, the precision detection is carried out: the numbers in the test set ID indicate how many plant pictures there are, and the variety of each plant picture is known. By the model training method, when the test set comprises 10 types (namely 10 types), each type comprises 10 IDs, 10 photos are placed in each ID, wherein the same ID is placed in the same plant picture in the same plot, and the plant pictures of the same type but not in the same plot are not placed in the same ID but in the same type (the purpose of introducing other plots and plants of the same type is to improve the test difficulty and the test precision). Detecting the similarity of any two pictures in the detection test set to obtain a 1000 × 1000 similarity matrix, wherein the detection results are similar or dissimilar, the detection results are compared with known results, the detection results are 1 and 0 respectively, and the accuracy rate is counted, wherein the accuracy rate = the same number of results/1000 × 1000.

Table one: accuracy testing form

As can be seen from the table I, the accuracy of the detection method for a single data set only containing ten different varieties in the same plot can reach 99.9%, the cross breeding plot needing to identify the hybrid plants and carry out impurity removal only contains two different varieties of plants for cross breeding and is usually not more than 5 hybrid plants, the detection rate of 99.7% -99.9% of the hybrid plants is remarkably improved compared with that of 60% -70% in the prior art, the training amount is effectively reduced, and the practical application can be realized.

The invention also discloses a rape seedling stage hybrid removing method, which is realized based on the hybrid identification and positioning process for realizing the positioning method in the specific embodiment, and comprises a front step S0 and a rear step S7 as shown in a combined figure 6.

And S0, determining the mobile operation equipment for acquiring the image in the step S1.

And S7, performing impurity removal operation according to the positions of the impurities in the image in the step S6.

As shown in fig. 6, in the present embodiment, the step S0 includes the following substeps:

and S01, setting a plot area threshold value, and comparing the current plot area with the plot area threshold value. When the area of the current land is small, the mobile operation equipment is selected according to actual needs to carry out image acquisition and real-time impurity removal operation, so that the aim of improving the impurity removal efficiency is fulfilled.

And S021, if the current plot area is smaller than the plot area threshold value, acquiring an image and picture in real time by adopting an unmanned vehicle. And when the current plot area is judged to be smaller than the set plot area threshold value, the unmanned vehicle is used as mobile operation equipment for image acquisition, the unmanned vehicle stops after moving for a certain distance in the current plot, and the image containing partial plants in the plot is shot. When the unmanned vehicle is used for image acquisition, the hybrid identification and positioning process system is directly carried on the unmanned vehicle, and the position of the hybrid can be identified in real time after the image is acquired. Because the area of the current plot is smaller than the threshold value of the area of the plot, the number of plants to be identified in the corresponding current plot is small, and the calculation force of the unmanned vehicle can support the hybrid plant identification and positioning process so as to achieve the purpose of identifying and positioning the hybrid plants.

S022, if the area of the current land is larger than the threshold value of the area of the land, collecting a plurality of image maps covering the whole current land by adopting an unmanned aerial vehicle. When judging that the plot area is greater than the set plot area threshold value, use unmanned aerial vehicle to carry out the image collection as mobile operation equipment. The unmanned aerial vehicle is in the plot in the low-altitude flight collection can cover the whole many image maps of present plot, with the image map upload to carry out the equipment of above-mentioned miscellaneous plant discernment and location flow to once only discern and fix a position all miscellaneous plant positions in the present plot. Because the current plot area is greater than the plot area threshold value, the number of plants to be identified in the current plot is large, unmanned vehicle power is difficult to support, and time consumption is too much to remove by adopting real-time image collection, so that the unmanned aerial vehicle is adopted to collect images once and perform hybrid plant identification and positioning through equipment with larger power.

As shown in fig. 7, in the present embodiment, the step S7 includes the following substeps:

and S711, moving the unmanned vehicle to the hybrid position positioned in the step S6 in real time to remove the hybrid. Because the positions of the hybrid plants are correlated and positioned on the image map, and the image map carries RGBD information, the positions of the hybrid plants in the current plot can be directly positioned based on the positions of the hybrid plants in the image map, and a path is constructed to enable the unmanned vehicle to move to the positions of the hybrid plants for removing the hybrid plants.

And S712, marking all the weed plant positions in the image to obtain an impurity removal operation indication diagram, and removing the weed plant positions according to the impurity removal operation indication diagram. And (4) reconstructing and splicing all the images obtained in the step (S022) into a whole map, wherein all the positions of the mixed plants are marked in the whole map, and the map is the indication map for the impurity removing operation. Because the hybrid plant image index information is established in the step S25 of the hybrid plant identification and positioning process, taking 10 image pictures as an example, each image has 20 targets, when the 10 th plant of the third image is a hybrid plant, the ID in the image picture is 3-10, the number of the plant picture where the hybrid plant is located is 50, when the image with the number of 50 is determined to be a hybrid plant, the ID3-10 of the image picture is traced back, the position of the hybrid plant in the image picture can be directly positioned, and the hybrid plant is marked in the image picture by means of red dot printing and the like. Since the image map carries the RGBD information, the position of the alien plant in the current plot can be directly located based on the position of the alien plant in the image map. And (4) removing impurities manually or by a machine according to the obtained impurity removing operation instruction diagram, and if the impurities are removed by the machine, planning a moving path of the machine in the current land parcel.

The above-listed detailed description is merely a detailed description of possible embodiments of the present invention, and it is not intended to limit the scope of the invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention are intended to be included within the scope of the present invention.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.

Furthermore, it should be understood that although the present specification describes embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and it is to be understood that all embodiments may be combined as appropriate by one of ordinary skill in the art to form other embodiments as will be apparent to those of skill in the art from the description herein.

Claims

1. A rape seedling stage hybrid positioning method is characterized by comprising the following steps:

obtaining image maps of a plurality of seedling-stage plants in a plot;

presetting an outlier index threshold value, and screening plant pictures corresponding to the feature vectors of which the outlier indexes exceed the preset outlier index threshold value;

2. The method for locating the hybrid plants in the seedling stage of rape as claimed in claim 1, wherein the step of identifying all the plants in the image map to obtain the plant picture comprises: and identifying the seedling-stage plants in the image map through a preset seedling-stage plant model, and determining plant pictures adapting to the sizes of the identified seedling-stage plants.

3. The method for locating the hybrid plants in the seedling stage of rape as claimed in claim 2, wherein before the step of extracting the features of each plant picture based on the high-dimensional features of the preset dimensions to obtain the feature vectors, the method further comprises: the size of each plant picture is standardized so that the size of each plant picture is consistent.

4. The method for locating the hybrid plants in the seedling stage of rape as claimed in claim 2, wherein before the step of extracting the features of each plant picture based on the high-dimensional features of the preset dimensions to obtain the feature vector, the method further comprises: and distinguishing the foreground plant part and the background plant part of each plant picture so as to keep the foreground plant part.

5. The method for locating the hybrid plants in the seedling stage of rape as claimed in claim 3, wherein the step of extracting the features of each plant picture based on the high-dimensional features of the preset dimensions to obtain the feature vector comprises the following substeps:

pre-deploying a plurality of models with different orders of magnitude;

6. The method for locating the hybrid plant in the seedling stage of rape as claimed in claim 1, wherein the step of calculating the Euclidean distance between any two feature vectors and determining the outlier index of each feature vector comprises the following sub-steps:

7. The method for locating the hybrid plants in the seedling stage of rape according to claim 1, wherein the step of screening the plant pictures corresponding to the feature vectors of which the outlier index exceeds the preset outlier index threshold value comprises the following substeps:

and screening the plant picture corresponding to the feature vector of which the outlier index exceeds the outlier index threshold of the variety to be identified as the variety picture to be identified.

8. The method for locating the hybrid plants in the seedling stage of rape as claimed in claim 1, wherein the step of configuring the index information for each plant picture comprises the following substeps: and configuring index information for each plant picture of each image map, wherein the index information comprises an image map ID and a plant picture number corresponding to the plant picture.

9. The method of claim 1, wherein the high-dimensional features require training of an extraction model through an optimized loss function to minimize Euclidean distance between homogeneous features and maximize Euclidean distance between heterogeneous features.

10. A method for removing hybrid plants at the seedling stage of rape, which is characterized by being realized based on the positioning method of the hybrid plants at the seedling stage of rape of claims 1-9, and comprising the following steps: