CN115880429A

CN115880429A - Method and system for determining vegetable strong seedling judgment model

Info

Publication number: CN115880429A
Application number: CN202211561857.1A
Authority: CN
Inventors: 曹玲玲; 马韫韬; 田雅楠; 曹彩红; 张松阳; 张敬锁; 赵立群; 王忠义; 徐娜
Original assignee: BEIJING AGRICULTURE TECHNOLOGY PROMOTION STATION
Current assignee: BEIJING AGRICULTURE TECHNOLOGY PROMOTION STATION
Priority date: 2022-12-07
Filing date: 2022-12-07
Publication date: 2023-03-31

Abstract

The embodiment of the application discloses a method and a system for determining a vegetable strong seedling judgment model, wherein the method comprises the following steps: acquiring a multi-view image sequence of the vegetable seedlings; performing three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model; extracting phenotype data corresponding to the seedling type based on the seedling three-dimensional data model; and carrying out quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judgment model. The method is characterized in that a machine learning technology is combined, greenhouse plants are taken as research objects, RGB images of multiple visual angles of each plant are collected, a three-dimensional model of the plant is obtained based on a three-dimensional reconstruction technology, the three-dimensional model is combined with a strong seedling index for regression and discriminant analysis, and finally a vegetable strong seedling judgment model is constructed, so that the seedling quality is evaluated efficiently and accurately, and errors of artificial naked eye judgment are reduced.

Description

Method and system for determining vegetable strong seedling judgment model

Technical Field

The embodiment of the application relates to the technical field of vegetable seedling raising, in particular to a method and a system for determining a strong vegetable seedling judgment model.

Background

Seedling raising becomes the primary and key technical link of the modern vegetable industry, and the robustness of vegetable seedlings is the guarantee of later-stage vegetable growth and quality. In the conventional seedling raising process, the strong seedlings are usually judged by observing or measuring single indexes of the seedlings, such as the plant height, the stem thickness, the leaf color and the like, or judged by adopting an experimental formula (stem thickness/plant height x total dry weight) established by the predecessor. As the varieties and the quantity of the seedlings for vegetable seedling culture are increased, and the vegetable seedlings are different from self-rooted seedlings and grafted seedlings, the accurate judgment is difficult to carry out through a single index and an empirical strong seedling index formula.

Therefore, a method for judging strong seedlings of vegetables is needed to help producers and researchers reasonably judge seedlings of different varieties and different types.

Disclosure of Invention

Therefore, the embodiment of the application provides a method and a system for determining a vegetable strong seedling judgment model, which are implemented by combining a machine learning technology, taking greenhouse plants as research objects, acquiring RGB images of multiple visual angles of each plant, acquiring a three-dimensional model of the plant based on a three-dimensional reconstruction technology, combining the RGB images with a strong seedling index to perform regression and discriminant analysis, and finally constructing the vegetable strong seedling judgment model, so that the seedling quality is efficiently and accurately evaluated, and errors caused by artificial naked eye judgment are reduced.

In order to achieve the above object, the embodiments of the present application provide the following technical solutions:

according to a first aspect of the embodiments of the present application, a method for determining a vegetable strong seedling judgment model is provided, where the method includes:

acquiring a multi-view image sequence of the vegetable seedlings;

performing three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model;

extracting phenotype data corresponding to the seedling type based on the seedling three-dimensional data model;

and carrying out quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judgment model.

Optionally, the three-dimensional point cloud reconstruction and correction are performed on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model, including:

point cloud reconstruction is carried out on the multi-view image sequence of the vegetable seedling based on a multi-view three-dimensional motion recovery structure algorithm to generate three-dimensional point cloud;

performing point cloud rotation correction on the three-dimensional point cloud to obtain a corrected point cloud coordinate;

and carrying out actual size conversion on the point cloud of the strong vegetable seedlings according to the corrected point cloud coordinates to obtain a seedling three-dimensional data model.

Optionally, the point cloud reconstruction is performed on the multi-view image sequence of the vegetable seedling based on a multi-view stereo motion recovery structure algorithm to generate a three-dimensional point cloud, including:

removing black backgrounds from the multi-view image sequence of the vegetable seedlings by using a mask, and leaving image information only belonging to plants and white discs;

extracting feature points which are not influenced by brightness, color and rotation in the multi-view image sequence by using a scale invariant feature transform matching algorithm;

filtering out wrong matching points by adopting a random sampling consistency algorithm, and reconstructing sparse point cloud;

clustering and matching images, and diffusing and filtering point clouds to generate three-dimensional point clouds by using a multi-view stereo clustering view algorithm and a patch-based multi-view stereo algorithm.

Optionally, the performing point cloud rotation correction on the three-dimensional point cloud to obtain a corrected point cloud coordinate includes:

fitting the three-dimensional point cloud to a plane by using a random sampling consistency algorithm to obtain an equation of the ground and a corresponding normal vector;

and rotating the ground to be vertical to the standard Z axis to obtain a point cloud coordinate after rotation correction.

Optionally, if the seedling type is a tomato seedling, the phenotype data of the tomato seedling are plant height, stem width, minimum bounding box volume, convex hull volume, canopy projection area, canopy projection convex hull area and stem leaf included angle;

if the seedling type is lettuce or celery seedling, the phenotype data of the lettuce or celery seedling is plant height, top width, bottom width, minimum bounding box volume, convex hull volume, canopy projection area and canopy projection convex hull area.

Optionally, the strong seedling data of each type of seedling is calculated according to the following formula:

according to a second aspect of embodiments of the present application, there is provided a vegetable strong seedling determination model determination system, including:

the multi-view image module is used for acquiring a multi-view image sequence of the vegetable seedling;

the seedling three-dimensional data model determining module is used for performing three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedlings to obtain a seedling three-dimensional data model;

the phenotype data module is used for extracting phenotype data corresponding to the seedling type based on the seedling three-dimensional data model;

and the vegetable strong seedling judging model determining module is used for carrying out quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judging model.

Optionally, the seedling three-dimensional data model determining module is configured to:

and carrying out actual size conversion on the vegetable strong seedling point cloud according to the corrected point cloud coordinate to obtain a seedling three-dimensional data model.

According to a third aspect of embodiments herein, there is provided an electronic device comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the method of the first aspect.

According to a fourth aspect of embodiments herein, there is provided a computer readable storage medium having computer readable instructions stored thereon, the computer readable instructions being executable by a processor to implement the method of the first aspect.

In summary, the embodiment of the present application provides a method and a system for determining a vegetable strong seedling judgment model, by acquiring a multi-view image sequence of vegetable seedlings; performing three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model; extracting phenotype data corresponding to the seedling type based on the seedling three-dimensional data model; and carrying out quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judgment model. The method is characterized in that a machine learning technology is combined, greenhouse plants are taken as research objects, RGB images of multiple visual angles of each plant are collected, a three-dimensional model of the plant is obtained based on a three-dimensional reconstruction technology, the three-dimensional model is combined with a strong seedling index for regression and discriminant analysis, and finally a vegetable strong seedling judgment model is constructed, so that the seedling quality is evaluated efficiently and accurately, and errors of artificial naked eye judgment are reduced.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.

The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so that those skilled in the art can understand and read the present invention, and do not limit the conditions for implementing the present invention, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the functions and purposes of the present invention, should still fall within the scope of the present invention.

Fig. 1 is a schematic flow chart of a method for determining a vegetable strong seedling judgment model according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a quantitative relationship model between three-dimensional phenotype and strong seedling data provided in the embodiments of the present application;

FIG. 3 is a schematic diagram of the tomato seedling reconstruction process and results provided in the examples of the present application;

FIG. 4 is a schematic diagram of a three-dimensional model of lettuce seedlings and extracted phenotypic indicators provided in the embodiments of the present application;

FIG. 5 is a schematic diagram of a celery seedling three-dimensional model and extracted phenotype indexes provided in the embodiment of the application;

FIG. 6 is a schematic diagram of comparative analysis of the indexes of strong seedlings provided by the embodiments of the present application under the observation and classification results of experts;

FIG. 7 is a schematic diagram of a regression relationship between multiple three-dimensional phenotype indicators and strong seedling index provided in the examples of the present application;

FIG. 8 is a schematic diagram of a regression analysis of phenotype index of lettuce and strong seedling index provided in the embodiments of the present application;

FIG. 9 is a schematic diagram of a regression analysis of lettuce phenotype index and strong seedling index provided in the embodiments of the present application;

fig. 10 is a block diagram of a system for determining a vegetable strong seedling determination model according to an embodiment of the present application;

fig. 11 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application;

fig. 12 is a schematic diagram illustrating a computer-readable storage medium provided in an embodiment of the present application.

Detailed Description

The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

At present, in agricultural informatization research, research on a digital plant technology system is more and more concerned by broad scholars. The three-dimensional information of the object in the objective environment can be acquired by adopting a three-dimensional information acquisition technology, so that the geometric shapes of different objects are determined and a three-dimensional space model is established. Machine Learning (Machine Learning) is a multidisciplinary data analysis technology, which can automatically analyze and model by using theories such as probability theory, statistics, decision theory, visualization, optimization and the like for reference, and learn from existing data to realize analysis and prediction. In recent years, machine learning methods are increasingly applied to the agricultural field, for example, in the aspect of crop pest control, researchers use machine learning methods to diagnose and identify crop pests, and the problems of low speed and inaccuracy of manual identification are solved; in crop breeding, machine learning is applied in genome-wide association studies for identification of candidate genes, phenotypic genome prediction, and the like.

Three-dimensional information is acquired, and the method can be divided into an active type and a passive type according to the information acquisition principle. The active mode is to reconstruct a three-dimensional structure of a target object by emitting a specific signal, such as laser, ultrasonic wave, electromagnetic wave, etc., to the target object. Passive is to detect the ambient light reflected by the target object and analyze the two-dimensional image of the target object acquired by the camera to obtain the three-dimensional structure of the object. In addition, monocular vision, binocular vision, and trinocular vision can be classified according to the number of cameras; the method can be classified into a motion recovery structure method, a machine learning method and the like according to different application methods. At present, the three-dimensional reconstruction technology which is most widely applied in the field of plant three-dimensional reconstruction is a method combining a motion recovery Shape (SFM) algorithm and a multi-view stereoscopic vision (MVS) technology. The method has high calculation efficiency, can reconstruct accurate and dense point cloud, has low equipment cost and is easy to realize. By utilizing the digital technology, the construction of a digital eggplant technical system is necessary for analyzing, designing and simulating the life process and the production process of the tomato in a three-dimensional visual mode. The realization of the three-dimensional visual modeling of the tomato morphological structure is the fundamental work for constructing a digital plant technology system.

Fig. 1 illustrates a method for determining a vegetable strong seedling judgment model according to an embodiment of the present application, where the method includes:

step 101: acquiring a multi-view image sequence of the vegetable seedlings;

step 102: performing three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model;

step 103: extracting phenotype data corresponding to the seedling type based on the seedling three-dimensional data model;

step 104: and carrying out quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judgment model.

In a possible embodiment, in step 102, the three-dimensional point cloud reconstruction and correction are performed on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model, which includes:

point cloud reconstruction is carried out on the multi-view image sequence of the vegetable seedling based on a multi-view three-dimensional motion recovery structure algorithm to generate three-dimensional point cloud; performing point cloud rotation correction on the three-dimensional point cloud to obtain a corrected point cloud coordinate; and carrying out actual size conversion on the point cloud of the strong vegetable seedlings according to the corrected point cloud coordinates to obtain a seedling three-dimensional data model.

In a possible embodiment, the point cloud reconstruction is performed on the sequence of multi-view images of the vegetable seedling based on a multi-view stereo motion recovery structure algorithm to generate a three-dimensional point cloud, and includes:

removing a black background from the multi-view image sequence of the vegetable seedling by using a mask, and leaving image information only belonging to plants and white discs; extracting feature points which are not influenced by brightness, color and rotation in the multi-view image sequence by using a scale invariant feature transform matching algorithm; filtering out wrong matching points by adopting a random sampling consistency algorithm, and reconstructing sparse point cloud; clustering and matching images, and diffusing and filtering point clouds are carried out by using a multi-view stereo clustering view algorithm and a patch-based multi-view stereo algorithm to generate three-dimensional point clouds.

In a possible embodiment, the performing point cloud rotation correction on the three-dimensional point cloud to obtain corrected point cloud coordinates includes:

fitting the three-dimensional point cloud to a plane by using a random sampling consistency algorithm to obtain an equation of the ground and a corresponding normal vector; and rotating the ground to be vertical to the standard Z axis to obtain a point cloud coordinate after rotation correction.

In one possible embodiment, if the seedling type is a tomato seedling, the phenotype data of the tomato seedling are plant height, stem width, minimum bounding box volume, convex hull volume, canopy projected area, canopy projected convex hull area, and stem leaf angle.

In one possible embodiment, if the seedling type is a lettuce or celery seedling, the phenotype data of the lettuce or celery seedling is plant height, top width, bottom width, minimum bounding box volume, convex hull volume, canopy projected area and canopy projected convex hull area.

In a possible embodiment, the strong seedling data of each type of seedling is calculated according to the following formula:

the method includes the steps that a machine learning technology is combined, greenhouse tomato plants, lettuce plants and celery plants are used as research objects, RGB images with multiple visual angles of each plant are synchronously collected, and a three-dimensional model of the plant is obtained based on a three-dimensional reconstruction technology; after the three-dimensional phenotypic characteristics of the tomato in the seedling stage are obtained, regression and discriminant analysis are carried out by combining the tomato with the strong seedling index, and finally, a strong seedling index digital model is constructed, and the vegetable strong seedling evaluation grading standard is developed, so that the seedling quality is evaluated, and the error of artificial naked eye judgment is reduced.

In the strong seedling evaluation method provided by the embodiment of the application, the strong seedling data of the tomatoes are manually measured at first, and corresponding data analysis is completed. And further acquiring a multi-view image sequence of the tomato seedlings based on the RGB camera, and further reconstructing a three-dimensional digital model of the tomato seedlings. And further extracting various phenotype parameters based on the constructed three-dimensional digital model. And then combining strong tomato seedling data, and adopting a machine learning method to carry out quantitative regression relation between various phenotypic parameters and strong seedling indexes. And evaluating whether the seedlings are strong seedlings or not through a strong seedling model.

The methods provided by the embodiments of the present application are described in detail below with reference to the figures and experimental data.

The tomato seedlings in the test material grow seedlings thirty days after sowing, and three-dimensional reconstruction and manual measurement of various phenotypic indexes are carried out after the tomato seedlings grow seedlings; the lettuce and celery become seedlings thirty days after sowing, and three-dimensional reconstruction and manual measurement of various phenotypic indexes are carried out after the seedlings become seedlings.

Fig. 2 shows a schematic diagram of a quantitative relationship model between three-dimensional phenotype and strong seedling data provided in the embodiment of the present application.

In the first aspect, tomato seedling three-dimensional reconstruction and phenotype information extraction are performed.

Selecting 60 tomatoes, celery and lettuce in the seedling stage. And acquiring a multi-angle image sequence by adopting an EOS 500D single lens reflex under an indoor bright environment. The seedlings with the seedling tray are placed in the center of a turntable rotating at a constant speed, and a black curtain is arranged behind the turntable to eliminate background interference. A camera is erected at a shooting point with the radius of about 2m of the turntable, and the tomato seedlings, the celery seedlings, the lettuce seedlings and the white turntable are arranged in the center of a visual angle. The camera can take one picture in two seconds and 60 pictures per seedling by the software digiCamControl automatically.

And performing point cloud reconstruction on the obtained multi-view image sequences of tomatoes, lettuce and celery by adopting open source software 3DF based on a multi-view three-dimensional motion recovery structure algorithm (SFM-MVS). The method mainly comprises the following steps:

(1) Importing a seedling multi-view image sequence; (2) Removing the black background by using a mask, and leaving image information only belonging to plants and white discs; (3) Extracting Feature points which are not influenced by brightness, color and rotation in an image sequence by using a Scale Invariant Feature Transform (SIFT) matching algorithm; (3) Filtering out wrong matching points by adopting a Random Sample Consensus (RANSAC) algorithm, and reconstructing a sparse point cloud of the tobacco plants; (4) Clustering and matching images, diffusing and filtering point clouds by using a Multi-view Stereo Clustering view algorithm (Clustering Views for Multi-view Stereo CMVS) and a Multi-view Stereo algorithm based on a patch, and finally generating dense point clouds.

After obtaining the three-dimensional point cloud, generating a ply file and importing the ply file into matlab for point cloud pretreatment, wherein the specific pretreatment process comprises the following steps: because the standard coordinate system is not fixed during photographing, the obtained tomato plants are usually not upright, and the coordinates of the tomato plants need to be rotationally corrected. And fitting the white turntable point cloud into a plane by using a random sampling consistency algorithm to obtain an equation of the ground and a corresponding normal vector, rotating the ground to be vertical to a standard Z axis, and obtaining a point cloud coordinate after rotation correction. And finally, carrying out point cloud actual size conversion on the tomato plants according to the actual area of the white disc above the turntable.

And (3) extracting various phenotype indexes by using an MATLAB program according to the plant characteristics of the tomatoes and the phenotype indexes of common leaves, wherein the specific phenotype parameters are shown in a table 1.

TABLE 1 tomato seedling phenotype parameters List

TABLE 2 lettuce and celery seedling phenotype parameter List

In the second aspect, the strong seedling index is manually determined and calculated.

After multi-view sequence shooting is carried out on tomato, celery and lettuce seedling plants, destructive measurement is carried out on the plants for calculating strong seedling indexes, and the measured indexes are stem thickness, plant height, fresh weight of overground parts and fresh weight of underground parts. And drying the seedlings after the determination is finished, and measuring the dry weights of the overground part and the underground part. The calculation method of the strong seedling index comprises the following steps:

before the measurement, experts are invited to evaluate each tomato seedling by eyes, and 60 tomato seedlings are divided into three types of strong seedlings, medium seedlings and weak seedlings according to experience.

After the measurement data, the phenotype data and the expert evaluation data are obtained, the indexes can be compared and analyzed with each other. And (4) judging the phenotype data result by taking the classification result of the expert evaluation data as a true value so as to verify whether the phenotype information obtained after three-dimensional modeling can replace manpower to judge the strength of the seedling. The adopted machine learning algorithm is four types of support vector machine, random forest, decision tree and Bayes algorithm. And predicting the strong seedling index obtained by calculation by adopting phenotype data so as to verify whether the phenotype information obtained by three-dimensional modeling can replace manual destructive measurement to calculate the strong seedling index.

And in the third aspect, analyzing the seedling three-dimensional reconstruction result.

The three-dimensional reconstruction based on the SFM algorithm can obtain accurate and dense plant point cloud, and specific positions and sizes of plant parts such as leaves, stems and the like can be clearly seen in the point cloud. After size correction, phenotypic data of each part of the leaf, such as leaf length, leaf width, leaf area, plant height, maximum stem width, and the like, can be automatically output.

Figure 3 shows the tomato seedling re-establishment procedure and results. And carrying out data summary analysis on the obtained tomato phenotype indexes. In various phenotypic indexes of the tomato, the plant height ranges from 5.80 cm to 21.40cm. The width of the stem ranges from 0.78 cm to 1.81cm. The convex hull volume ranges from 61.98 to 130.00cm3. The value range of the minimum bounding box volume is 115.02-151.42cm3. The value range of the canopy projection area is 3.85-29.90cm 2. The area of the projected convex hull of the canopy ranges from 4.25 to 31.75cm2. The value range of the strong seedling index is between 0.02 and 0.26.

TABLE 1 summary of tomato phenotypic index data

Fig. 4 shows three-dimensional models of lettuce seedlings and extracted phenotype indexes, and data summarization analysis is performed on each obtained lettuce phenotype index. In various phenotypic indexes of the lettuce, the value range of the plant height is 6.31-14.07 cm. The stem width is 0.79-1.31 cm. The value range of the convex hull volume is 32-123 cm3. The minimum bounding box volume ranges from 24 cm3 to 80cm3. The value range of the canopy projection area is 8-30 cm2. The top width is 2.58-6.59 cm. The value range of the bottom width is 1.45-3.89 cm. The value range of the strong seedling index is between 0.49 and 1.50.

TABLE 2 summary of tomato phenotypic index data

Fig. 5 shows a celery seedling three-dimensional model and extracted phenotypic indicators. And carrying out data summarization analysis on each obtained celery phenotype index, wherein the value range of the plant height in each celery phenotype index is 5.93-15.62 cm. The stem width is 0.34-0.84 cm. The convex hull volume ranges from 22.50 to 73.50cm3. The minimum bounding box volume ranges from 33.20 to 110.40cm3. The value range of the canopy projection area is 8.10-30.01cm 2. The top width is 2.23-5.93 cm. The base width is 0.94-3.46 cm. The value range of the strong seedling index is between 0.49 and 1.50.

TABLE 3 celery phenotypic index data summarization

Fig. 6 shows comparative analysis results of the strong seedling index under the classification results observed by experts. Comparing the strong seedling index result obtained by manual measurement with the judgment result of the expert naked eyes, the obvious difference of the strong seedling index result of the tomato seedlings can be seen under the classification standard of the expert naked eyes, and therefore the consistent relation of the index calculation result of the tomato seedlings and the manual classification result is proved.

And (4) inputting the discrimination result of the artificial strong seedlings as a true value to perform machine learning discrimination, and inputting the acquired three-dimensional phenotype indexes. Four machine learning methods are adopted, namely support vector product, random forest, decision tree and Bayes algorithm. The support vector machine method has the best discrimination result, and the precision is 85.7%. The method is simple in algorithm and good in robustness, because the support vector machine algorithm is insensitive to abnormal values, key samples can be grasped, and a large number of redundant samples can be eliminated.

TABLE 4 discrimination results of various machine learning methods

FIG. 7 shows the regression relationship between various three-dimensional phenotypic indicators and strong seedling index of strong tomato seedlings. The regression analysis is carried out by adopting various phenotype indexes and strong seedling indexes, and the obtained results show that the prediction effects of the four important phenotype characteristics are good, the R2 range is between 79% and 90%, and the effects are superior to those of lettuce and celery models. The reason for this is that tomato plants are relatively upright, the photographing and reconstruction effects are good, and the results of the phenotype extraction algorithm are more accurate than those of other two crops. In addition, the regression performance of the convex hull volume and the minimum bounding box volume and the strong seedling index is not much different from the regression effect of other two-dimensional indexes and the strong seedling index, because the size of the canopy area of the tomato and whether the tomato seedling is strong or not are directly related, and therefore, the monitoring of the canopy index occupies a very important position in the automatic phenotypic monitoring of the tomato. The results show that the tomato strong seedling index can be predicted through various three-dimensional indexes of the tomato seedling stage.

FIG. 8 shows a schematic diagram of regression analysis of lettuce phenotype index and strong seedling index. And carrying out regression analysis on each obtained phenotype index and the strong seedling index. The convex hull of the three-dimensional model can embody the shape and the volume ratio of the lettuce seedlings, and the shape and the volume ratio have great correlation with the strength of the seedlings. Therefore, in the regression analysis results, the regression results of the volume of the convex hull and the strong seedling index have the best effect and the strongest correlation, and R2 reaches 0.84. The size of the minimum bounding box volume is influenced by the three-dimensional shape of the plant, and the numerical value is influenced by the lodging of the plant, the dispersion of leaves and the like, so that the correlation of the minimum bounding box volume with the strong seedling index is slightly reduced compared with the convex hull volume. The correlation between the two-dimensional information of the plant canopy and the strong seedling index is insufficient, and R2 is between 0.5 and 0.7.

Fig. 9 shows a schematic diagram of regression analysis of lettuce phenotype index and strong seedling index. And performing regression analysis on the obtained phenotype indexes and the strong seedling indexes. Celery is thin and weak, has more leaves and more complex forms, so the difficulty of three-dimensional reconstruction is higher. The reconstruction result is slightly different between the calculation of various phenotypic indexes and the true value because dense point clouds at certain positions are unclear, and the modeling regression effect is inferior to that of a lettuce model. In the regression results of the celery phenotype indexes and the strong seedling indexes, the three-dimensional indexes such as the convex hull volume and the minimum bounding box volume still have the highest regression correlation, but the correlation of the two-dimensional indexes such as the projection area is poor, and the R2 is only about 0.4.

In conclusion, the tomato strong seedling index is predicted by the three-dimensional convex hull volume in the tomato seedling stage; the results of strong seedling index analysis and three-dimensional modeling phenotype index evaluation of celery and lettuce can be known, the three-dimensional indexes such as the volume of a convex hull and the volume of a minimum bounding box have better correlation with the strong seedling index, and the prediction analysis of the strong seedling index can be carried out according to the modeling result. Therefore, the phenotype information obtained by three-dimensional modeling can replace manual measurement to calculate the strong seedling index, reduce the error of artificial naked eye judgment and realize the datamation of the strong seedling index. Meanwhile, when the seedlings are strong seedlings, destructive measurement is not needed.

In summary, the embodiment of the present application provides a method for determining a vegetable strong seedling judgment model, by acquiring a multi-view image sequence of a vegetable seedling; performing three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model; extracting phenotype data corresponding to the seedling type based on the seedling three-dimensional data model; and carrying out quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judgment model. The method is characterized in that a machine learning technology is combined, greenhouse plants are taken as research objects, RGB images of multiple visual angles of each plant are collected, a three-dimensional model of the plant is obtained based on a three-dimensional reconstruction technology, the three-dimensional model is combined with a strong seedling index for regression and discriminant analysis, and finally a vegetable strong seedling judgment model is constructed, so that the seedling quality is evaluated efficiently and accurately, and errors of artificial naked eye judgment are reduced.

Based on the same technical concept, an embodiment of the present application further provides a system for determining a vegetable strong seedling judgment model, as shown in fig. 10, the system includes:

a multi-view image module 1001 configured to obtain a multi-view image sequence of a vegetable seedling;

a seedling three-dimensional data model determining module 1002, configured to perform three-dimensional point cloud reconstruction and correction on the multi-view image sequence of the vegetable seedling to obtain a seedling three-dimensional data model;

a phenotype data module 1003, configured to extract, based on the seedling three-dimensional data model, phenotype data corresponding to a seedling type;

and the vegetable strong seedling judgment model determining module 1004 is used for performing quantitative regression analysis according to the strong seedling data of various seedlings and the phenotype data corresponding to the seedling types to obtain a vegetable strong seedling judgment model.

In a possible embodiment, the seedling three-dimensional data model determining module 1002 is configured to:

point cloud reconstruction is carried out on the multi-view image sequence of the vegetable seedlings on the basis of a multi-view three-dimensional motion recovery structure algorithm, and three-dimensional point cloud is generated; performing point cloud rotation correction on the three-dimensional point cloud to obtain a corrected point cloud coordinate; and carrying out actual size conversion on the point cloud of the strong vegetable seedlings according to the corrected point cloud coordinates to obtain a seedling three-dimensional data model.

The embodiment of the application also provides electronic equipment corresponding to the method provided by the embodiment. Referring to fig. 11, a schematic diagram of an electronic device provided in some embodiments of the present application is shown. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program that can be executed on the processor 200, and the processor 200 executes the computer program to perform the method provided by any of the foregoing embodiments of the present application.

The Memory 201 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one physical port 203 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.

Bus 202 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used for storing a program, and the processor 200 executes the program after receiving an execution instruction, and the method disclosed by any of the foregoing embodiments of the present application may be applied to the processor 200, or implemented by the processor 200.

The processor 200 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 200. The Processor 200 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the method in combination with the hardware thereof.

The electronic device provided by the embodiment of the application and the method provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as the method adopted, operated or realized by the electronic device.

Referring to fig. 12, the computer-readable storage medium is an optical disc 30, on which a computer program (i.e., a program product) is stored, and when the computer program is executed by a processor, the computer program performs the method of any of the foregoing embodiments.

It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, a phase change memory (PRAM), a Static Random Access Memory (SRAM), a Dynamic Random Access Memory (DRAM), other types of Random Access Memories (RAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a flash memory, or other optical and magnetic storage media, which are not described in detail herein.

The computer-readable storage medium provided by the above-mentioned embodiments of the present application and the method provided by the embodiments of the present application have the same advantages as the method adopted, executed or implemented by the application program stored in the computer-readable storage medium.

It should be noted that:

the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Moreover, those of skill in the art will understand that although some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the creation apparatus of a virtual machine according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for determining a vegetable strong seedling judgment model is characterized by comprising the following steps:

acquiring a multi-view image sequence of the vegetable seedlings;

2. The method of claim 1, wherein the performing three-dimensional point cloud reconstruction and correction on the sequence of multi-view images of the vegetable seedling to obtain a seedling three-dimensional data model comprises:

point cloud reconstruction is carried out on the multi-view image sequence of the vegetable seedlings on the basis of a multi-view three-dimensional motion recovery structure algorithm, and three-dimensional point cloud is generated;

3. The method of claim 2, wherein the point cloud reconstruction of the sequence of multi-view images of the young vegetable plants based on a multi-view stereo motion restoration structure algorithm to generate a three-dimensional point cloud comprises:

4. The method of claim 2, wherein said performing a point cloud rotation correction on said three-dimensional point cloud to obtain corrected point cloud coordinates comprises:

and rotating the ground to be vertical to the standard Z axis to obtain the point cloud coordinate after rotation correction.

5. The method of claim 1, wherein if the seedling type is a tomato seedling, the phenotypic data of the tomato seedling are plant height, stem width, minimum bounding box volume, convex hull volume, canopy projected area, canopy projected convex hull area, and stem leaf angle;

6. The method of claim 1, wherein the strong seedling data of each type of seedling is calculated according to the following formula:

7. a vegetable strong seedling judgment model determination system is characterized by comprising:

the multi-view image module is used for acquiring a multi-view image sequence of the vegetable seedlings;

8. The system of claim 7, wherein the seedling three-dimensional data model determination module is to:

9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor executes when executing the computer program to implement the method according to any of claims 1-6.

10. A computer-readable storage medium having computer-readable instructions stored thereon, the computer-readable instructions being executable by a processor to implement the method of any one of claims 1-6.