CN113409265B - Method and system for dynamically acquiring and analyzing 3D phenotype of tomato in whole growth period - Google Patents
Method and system for dynamically acquiring and analyzing 3D phenotype of tomato in whole growth period Download PDFInfo
- Publication number
- CN113409265B CN113409265B CN202110669883.5A CN202110669883A CN113409265B CN 113409265 B CN113409265 B CN 113409265B CN 202110669883 A CN202110669883 A CN 202110669883A CN 113409265 B CN113409265 B CN 113409265B
- Authority
- CN
- China
- Prior art keywords
- point
- phenotype
- node
- leaf
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 235000007688 Lycopersicon esculentum Nutrition 0.000 title claims abstract description 47
- 230000012010 growth Effects 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 28
- 240000003768 Solanum lycopersicum Species 0.000 title claims description 52
- 241000196324 Embryophyta Species 0.000 claims abstract description 75
- 230000007613 environmental effect Effects 0.000 claims abstract description 46
- 238000000605 extraction Methods 0.000 claims abstract description 36
- 230000011218 segmentation Effects 0.000 claims abstract description 34
- 210000000056 organ Anatomy 0.000 claims abstract description 26
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 238000007405 data analysis Methods 0.000 claims abstract description 10
- 238000004364 calculation method Methods 0.000 claims abstract description 4
- 235000013399 edible fruits Nutrition 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 11
- 206010063385 Intellectualisation Diseases 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 239000012634 fragment Substances 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 230000003203 everyday effect Effects 0.000 claims description 2
- 238000009313 farming Methods 0.000 claims description 2
- 239000003337 fertilizer Substances 0.000 claims description 2
- 238000005286 illumination Methods 0.000 claims description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 2
- 230000003698 anagen phase Effects 0.000 claims 1
- 238000004321 preservation Methods 0.000 claims 1
- 241000227653 Lycopersicon Species 0.000 abstract 3
- 238000010586 diagram Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 235000013311 vegetables Nutrition 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and a system for dynamically acquiring and analyzing a 3D phenotype of a tomato in a whole growth period. The method comprises the following steps: obtaining an environment difference distribution map based on environment monitoring and determining a target individual plant; acquiring a multi-view image sequence of the whole growth period based on image acquisition; obtaining a local scene point cloud based on three-dimensional reconstruction; obtaining complete point cloud and lobe point cloud through point cloud pretreatment and manual segmentation; organ segmentation is carried out through point search based on the adjacency relation, and extraction of internodes and leaves is achieved; performing automatic extraction calculation of phenotype parameters on the basis of organ segmentation; aiming at point clouds of the same single plant in different periods, phenotype parameter registration is realized by utilizing a real topological structure; data analysis is performed based on environmental differences and phenotypic registration results. The method combines the environmental difference with the 3D phenotype of the tomatoes in the whole growth period, does not need to set a threshold value or manually mark in the aspect of algorithm, and can adapt to the extraction and registration of the phenotypic parameters of the tomatoes in different periods.
Description
Technical Field
The invention belongs to the field of agricultural automation, particularly relates to automatic measurement of tomato phenotype parameters, and particularly relates to a method and a system for dynamically acquiring and analyzing a 3D phenotype of a tomato in a whole growth period.
Background
The tomato is one of the most valuable vegetable crops in the world, and China plays a very important role in the international market as one of three main tomato planting areas in the world. High yield and high quality are important targets for tomato cultivation. However, the phenotype data acquisition of the tomatoes still mainly depends on the traditional phenotype analysis method, and the method is time-consuming, labor-consuming, complex in process, strong in subjectivity, and easy to damage the tomatoes, so that the development of the tomato cultivation technology is seriously hindered.
In the prior art, a laser scanner is used for acquiring three-dimensional information of plants and canopy surfaces to estimate leaf areas, leaf area indexes and plant heights, but hardware equipment is expensive and data redundancy is realized; the method comprises the steps of acquiring point cloud data of plants by adopting three-dimensional reconstruction, completing single plant or stem extraction by combining a region growing algorithm, deep learning and the like, setting different thresholds and manual labels aiming at different single plants, and realizing accurate extraction aiming at the characteristics of the bent stems of tomatoes.
Disclosure of Invention
Technical problem to be solved
Aiming at the technical problems of the prior method for acquiring the phenotypic parameters, the method comprises the following points. Firstly, in a specific cultivation scene, an artificial phenotype measurement means is adopted, the efficiency is low, the subjectivity is strong, the cultivation management mainly depends on experience judgment, and phenotype parameter support and corresponding management schemes in different periods are lacked; secondly, the traditional point cloud organ segmentation algorithm needs to adjust threshold values for different periods, if segmentation or labeling is carried out manually, the traditional point cloud organ segmentation algorithm not only requires the number of samples, but also is time-consuming and labor-consuming, and has certain subjectivity; finally, in the aspect of phenotypic registration, features are mainly found and defined on the point cloud, but the operation is complex, the period of the target point cloud is required to be close enough, and the quality requirement of the point cloud is high. The invention provides a method and a system for dynamically acquiring and analyzing a 3D phenotype of a tomato in a whole growth period, which realize dynamic quantification of phenotypic parameters of a single tomato plant in the whole growth period with obvious difference in greenhouse environment.
(II) technical scheme
In order to solve the technical problem, the invention provides a method and a system for dynamically acquiring and analyzing a 3D phenotype of a tomato in a whole growth period.
The method comprises the steps of dynamically acquiring and analyzing a 3D phenotype of the tomato in the whole growth period, wherein the dynamically acquiring and analyzing comprise environmental monitoring, image acquisition, three-dimensional reconstruction, point cloud processing, organ segmentation, phenotype parameter extraction, phenotype registration and data analysis; wherein the content of the first and second substances,
the environment monitoring is used for measuring environmental parameters in a cultivation scene to obtain a plurality of environmental parameter measuring results, and a plurality of target individual plants with larger environmental differences are screened out from a large number of plants according to an environmental difference distribution map and are used for subsequent data acquisition, processing and analysis;
acquiring images, namely performing multi-angle multi-layer shooting around a single plant and fruits from bottom to top by adopting an RGB (red, green and blue) camera to acquire multi-view image sequences of each target plant and ripe tomato fruits thereof, wherein the acquisition time period covers the whole growth period of the plant;
three-dimensional reconstruction, namely importing the acquired multi-view image sequence into Visual SFM software to reconstruct three-dimensional point cloud data of each part and fruit of a target individual plant;
point cloud processing, including point cloud denoising and scale correction preprocessing, wherein an Alignment registration algorithm is utilized to obtain complete individual plant point cloud, and all point cloud data belonging to lobe fragments in the blade are manually segmented for calculating the subsequent blade width and the blade area;
organ segmentation, skeletonizing point clouds of the rest parts of a single plant except for splinters, adopting a point search mode based on an adjacency relation for a bent stem of a tomato plant, inputting two end point numbers of the obtained skeleton, an adjacency matrix and a main stem, obtaining a joint of a petiole and the stem, namely a node, and end points of each leaf, namely a leaf tip point according to the adjacency relation of each point in the adjacency matrix, and performing extension search from each node to the surrounding adjacency points to realize the segmentation of each internode and each leaf;
phenotypic parameter extraction, namely performing automatic extraction calculation of phenotypic parameters on the basis of organ segmentation;
phenotype registration, namely realizing phenotype parameter registration of different periods by utilizing a real topological structure aiming at point cloud data of the same target individual plant in different growth periods;
and data analysis, namely, considering the difference of different plants on environmental conditions and the dynamic change of the phenotype parameters at different stages of the whole growth period, quantitatively analyzing the relationship among all elements, and determining the influence rule of each environmental parameter on the phenotype change, so as to obtain the management measure optimization schemes at different stages under a specific cultivation scene, and realize the universality of phenotype extraction and the intellectualization of cultivation management.
More specifically, the environmental monitoring is specifically realized by the following means: planting the tomatoes of the same variety in a solar greenhouse in a soilless culture mode, wherein i rows are planted from east to west, and j plants are planted in each row from north to south; in the aspect of cultivation management, a water and fertilizer integrated system and the same farming operation are adopted; uniformly selecting m sample points in a greenhouse, and regularly measuring temperature, humidity and illumination every day; according to the distribution map of the environmental difference, n individuals with larger environmental difference are selected from the sample points.
More specifically, when organ segmentation and phenotype parameter extraction are carried out, skeletons of all internodes are extracted and stored respectively, internode length is obtained by generating a skeleton curve of the internodes, internode number is obtained by the stored internode number, and plant height is obtained by calculating Euclidean distance between two end points on a main stem;
wherein, the internode extraction is realized by the following steps:
step 3-1, inputting the numbers of two end points, namely a bottom point and a growth point on the main stem skeleton, and setting the end points as current traversal points from the bottom point;
step 3-2, finding the adjacent points of the current traversal point;
step 3-3, selecting an adjacent point which is not traversed, marking the adjacent point as traversed, and carrying out the following judgment and operation:
3-3-1 judges whether the adjacent point is a node, if so, entering a settlement step 3-4-1;
3-3-2, judging whether the current traversal node is an end point, if so, entering a settlement step 3-4-1;
3-3-3, judging whether the current traversal node is a leaf endpoint or not, and if so, directly entering a settlement step 3-4-2;
3-3-4, if the above are not the same, marking the point as a middle point, setting the middle point as a current traversal point, and returning to the step 3-2;
step 3-4, settlement step:
3-4-1, if the node is a node or an end point, marking the middle point starting from the last end point or the node as a stem, and returning to the step 3-3;
3-4-2, if the leaf end point is the leaf end point, directly clearing the mark of the middle point;
3-4-3 if it is the end of the stem, the algorithm ends.
More specifically, when organ segmentation and phenotype parameter extraction are carried out, the serial numbers of all points on the main stem are obtained according to the internode extraction result, extraction and storage of all leaves are carried out, and the leaf length and the true leaf number are calculated;
wherein, each leaf is extracted by the following steps:
step 4-1, starting from the bottommost node, setting the node as the current node;
step 4-2, all adjacent points of the current node are searched;
4-3, selecting an adjacent point which is not traversed, and judging whether the adjacent point belongs to the main stem:
4-3-1, if yes, returning to the step 4-3;
4-3-2, if not, continuously traversing upwards according to the adjacency relation, and marking the traversed point as the blade point corresponding to the node until the blade end point is traversed;
and 4-4, judging whether the current node is the last node or not, if so, finishing the algorithm, if not, setting the next node as the current node, and returning to the step 4-2.
More specifically, when phenotypic parameter extraction is performed, the volume, true leaf number, internode number, length of each node, stem length, leaf width, leaf area, fruit transverse diameter and volume, etc. of the individual plant are obtained.
More specifically, phenotypic registration is specifically achieved by: defining a sequencing rule by combining the real topological structure of the tomatoes, defining a sequencing rule from a bottom point, namely the ground-near end, to a top point, namely the growing point, sequencing the segmented frameworks again, giving corresponding numbers, outputting new frameworks and visualizing, and directly realizing the registration of the phenotypic parameters of the crops at different periods.
The invention also provides a system for dynamically acquiring and analyzing the 3D phenotype of the tomato in the whole growth period, which adopts the method for dynamically acquiring and analyzing the 3D phenotype of the tomato in the whole growth period, and is characterized in that:
the system comprises the following modules which are provided with a plurality of modules,
the environment monitoring module is used for measuring environmental parameters in a cultivation scene to obtain a plurality of environmental parameter measuring results, screening a plurality of target individual plants with larger environmental differences from a large number of plants according to an environmental difference distribution map, and acquiring, processing and analyzing subsequent data;
the image acquisition module is used for carrying out multi-angle multi-layer shooting around a single plant and fruits from bottom to top by adopting an RGB camera to acquire multi-view image sequences of each target plant and ripe tomato fruits thereof, and the acquisition time period covers the whole growth period of the plant;
the three-dimensional reconstruction module is used for importing the acquired multi-view image sequence into Visual SFM software and reconstructing three-dimensional point cloud data of each part and fruit of the target individual plant;
the point cloud processing module is used for carrying out point cloud denoising and scale correction preprocessing, acquiring complete single plant point cloud by using an Alignment registration algorithm, manually segmenting all point cloud data belonging to the lobe in the blade, and calculating the subsequent blade width and the subsequent blade area;
the organ segmentation module is used for skeletonizing point clouds of the rest parts of a single plant except for splinters, adopting a point search mode based on an adjacency relation aiming at the bent stems of the tomato plants, obtaining the connection positions of the petioles and the stems, namely nodes, and the end points of each leaf, namely leaf tips, according to the adjacency relation of each point in the adjacency matrix by inputting the obtained numbers of the two end points of the skeleton, the adjacency matrix and the main stem, and realizing the segmentation of each internode and each leaf by extending and searching from each node to the adjacent points around;
the phenotype parameter extraction module is used for automatically extracting and calculating phenotype parameters on the basis of organ segmentation;
the phenotype registration module is used for realizing the registration of phenotype parameters in different periods by utilizing a real topological structure aiming at point cloud data of the same target individual plant in different growth periods;
the data analysis module considers the difference of different plants on environmental conditions and the dynamic change of the phenotype parameters at different stages of the whole growth period, quantitatively analyzes the relationship between all elements, and determines the influence rule of each environmental parameter on the phenotype change, so as to obtain the management measure optimization scheme at different stages under a specific cultivation scene, and realize the generalization of phenotype extraction and the intellectualization of cultivation management.
(III) advantageous effects
According to the method and the system for dynamically acquiring and analyzing the 3D phenotype of the tomatoes in the whole growth period, the environment is monitored aiming at a sunlight greenhouse, the target plants are determined, the influence rule of the environmental difference on different stages of the growth and development of the tomatoes is determined, and reliable phenotype data are provided for management measures in different periods; the tomato point cloud data of the whole growth period is obtained through image acquisition, three-dimensional reconstruction and point cloud processing, and data loss of two-dimensional data is made up; aiming at the bent stem of a tomato plant, skeletonizing point clouds, performing organ segmentation by adopting a point search mode based on adjacency relation without setting any threshold value or manually marking, requiring no sample quantity, directly realizing extraction of internodes and leaves, and having simple and convenient operation and high efficiency, and being suitable for organ segmentation of tomatoes at different periods; the phenotype parameter automatic extraction is realized on the basis of organ segmentation, and the problems of time and labor waste, strong subjectivity, destructiveness and the like of the traditional method are solved; the real topological structure is utilized to realize the registration of the phenotype parameters in different periods, the problem of searching and defining characteristics related to a phenotype registration algorithm is simplified, a threshold value is not required to be set, point cloud characteristics in adjacent periods are not required, for any period of the same single plant, the framework is reordered only by defining an ordering rule and corresponding numbers are given to carry out result output and visualization, the phenotype registration is directly realized, the operation steps are simplified, and the applicability is strong.
Drawings
FIG. 1 is a schematic flow chart of a method for dynamically acquiring and analyzing the 3D phenotype of a tomato in the whole growth period.
FIG. 2 is a schematic diagram of the structure of a whole-growth tomato 3D phenotype dynamic acquisition and analysis system.
Detailed Description
The invention provides a method and a system for dynamically acquiring and analyzing a 3D phenotype of a tomato in a whole growth period in order to solve the technical problem. The flow diagram of the method is shown in fig. 1, and the structural diagram of the system is shown in fig. 2.
The method for dynamically acquiring and analyzing the 3D phenotype of the tomatoes in the whole growth period comprises the steps of environment monitoring, image acquisition, three-dimensional reconstruction, point cloud processing, organ segmentation, phenotype parameter extraction, phenotype registration and data analysis; wherein the content of the first and second substances,
the environment monitoring is used for measuring environmental parameters in a cultivation scene to obtain a plurality of environmental parameter measuring results, and a plurality of target individual plants with larger environmental differences are screened out from a large number of plants according to an environmental difference distribution map and are used for subsequent data acquisition, processing and analysis;
acquiring images, namely performing multi-angle multi-layer shooting around a single plant and fruits from bottom to top by adopting an RGB (red, green and blue) camera to acquire multi-view image sequences of each target plant and ripe tomato fruits thereof, wherein the acquisition time period covers the whole growth period of the plant;
three-dimensional reconstruction, namely importing the acquired multi-view image sequence into Visual SFM software to reconstruct three-dimensional point cloud data of each part and fruit of a target individual plant;
point cloud processing, including point cloud denoising and scale correction preprocessing, wherein an Alignment registration algorithm is utilized to obtain complete individual plant point cloud, and all point cloud data belonging to lobe fragments in the blade are manually segmented for calculating the subsequent blade width and the blade area;
organ segmentation, skeletonizing point clouds of the rest parts of a single plant except for splinters, adopting a point search mode based on an adjacency relation for a bent stem of a tomato plant, inputting two end point numbers of the obtained skeleton, an adjacency matrix and a main stem, obtaining a joint of a petiole and the stem, namely a node, and end points of each leaf, namely a leaf tip point according to the adjacency relation of each point in the adjacency matrix, and performing extension search from each node to the surrounding adjacency points to realize the segmentation of each internode and each leaf;
phenotypic parameter extraction, namely performing automatic extraction calculation of phenotypic parameters on the basis of organ segmentation;
phenotype registration, namely realizing phenotype parameter registration of different periods by utilizing a real topological structure aiming at point cloud data of the same target individual plant in different growth periods;
and data analysis, namely, considering the difference of different plants on environmental conditions and the dynamic change of the phenotype parameters at different stages of the whole growth period, quantitatively analyzing the relationship among all elements, and determining the influence rule of each environmental parameter on the phenotype change, so as to obtain the management measure optimization schemes at different stages under a specific cultivation scene, and realize the universality of phenotype extraction and the intellectualization of cultivation management.
The invention relates to a full-growth-period tomato 3D phenotype dynamic acquisition and analysis system, which adopts the full-growth-period tomato 3D phenotype dynamic acquisition and analysis method, and is characterized in that:
the system comprises the following modules which are provided with a plurality of modules,
the environment monitoring module is used for measuring environmental parameters in a cultivation scene to obtain a plurality of environmental parameter measuring results, screening a plurality of target individual plants with larger environmental differences from a large number of plants according to an environmental difference distribution map, and acquiring, processing and analyzing subsequent data;
the image acquisition module is used for carrying out multi-angle multi-layer shooting around a single plant and fruits from bottom to top by adopting an RGB camera to acquire multi-view image sequences of each target plant and ripe tomato fruits thereof, and the acquisition time period covers the whole growth period of the plant;
the three-dimensional reconstruction module is used for importing the acquired multi-view image sequence into Visual SFM software and reconstructing three-dimensional point cloud data of each part and fruit of the target individual plant;
the point cloud processing module is used for carrying out point cloud denoising and scale correction preprocessing, acquiring complete single plant point cloud by using an Alignment registration algorithm, manually segmenting all point cloud data belonging to the lobe in the blade, and calculating the subsequent blade width and the subsequent blade area;
the organ segmentation module is used for skeletonizing point clouds of the rest parts of a single plant except for splinters, adopting a point search mode based on an adjacency relation aiming at the bent stems of the tomato plants, obtaining the connection positions of the petioles and the stems, namely nodes, and the end points of each leaf, namely leaf tips, according to the adjacency relation of each point in the adjacency matrix by inputting the obtained numbers of the two end points of the skeleton, the adjacency matrix and the main stem, and realizing the segmentation of each internode and each leaf by extending and searching from each node to the adjacent points around;
the phenotype parameter extraction module is used for automatically extracting and calculating phenotype parameters on the basis of organ segmentation;
the phenotype registration module is used for realizing the registration of phenotype parameters in different periods by utilizing a real topological structure aiming at point cloud data of the same target individual plant in different growth periods;
the data analysis module considers the difference of different plants on environmental conditions and the dynamic change of the phenotype parameters at different stages of the whole growth period, quantitatively analyzes the relationship between all elements, and determines the influence rule of each environmental parameter on the phenotype change, so as to obtain the management measure optimization scheme at different stages under a specific cultivation scene, and realize the generalization of phenotype extraction and the intellectualization of cultivation management.
The specific embodiments described in this application are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (7)
1. A method for dynamically acquiring and analyzing a 3D phenotype of a tomato in a whole growth period is characterized by comprising the following steps: the method comprises the steps of environment monitoring, image acquisition, three-dimensional reconstruction, point cloud processing, organ segmentation, phenotype parameter extraction, phenotype registration and data analysis; wherein the content of the first and second substances,
the environment monitoring is used for measuring environmental parameters in a cultivation scene to obtain a plurality of environmental parameter measuring results, and a plurality of target individual plants with larger environmental differences are screened out from a large number of plants according to an environmental difference distribution map and are used for subsequent data acquisition, processing and analysis;
acquiring images, namely performing multi-angle multi-layer shooting around a single plant and fruits from bottom to top by adopting an RGB (red, green and blue) camera to acquire multi-view image sequences of each target plant and ripe tomato fruits thereof, wherein the acquisition time period covers the whole growth period of the plant;
three-dimensional reconstruction, namely importing the acquired multi-view image sequence into Visual SFM software to reconstruct three-dimensional point cloud data of each part and fruit of a target individual plant;
point cloud processing, including point cloud denoising and scale correction preprocessing, wherein an Alignment registration algorithm is utilized to obtain complete individual plant point cloud, and all point cloud data belonging to lobe fragments in the blade are manually segmented for calculating the subsequent blade width and the blade area;
organ segmentation, skeletonizing point clouds of the rest parts of a single plant except for splinters, adopting a point search mode based on an adjacency relation for a bent stem of a tomato plant, inputting two end point numbers of the obtained skeleton, an adjacency matrix and a main stem, obtaining a joint of a petiole and the stem, namely a node, and end points of each leaf, namely a leaf tip point according to the adjacency relation of each point in the adjacency matrix, and performing extension search from each node to the surrounding adjacency points to realize the segmentation of each internode and each leaf;
phenotypic parameter extraction, namely performing automatic extraction calculation of phenotypic parameters on the basis of organ segmentation;
phenotype registration, namely realizing phenotype parameter registration of different periods by utilizing a real topological structure aiming at point cloud data of the same target individual plant in different growth periods;
and data analysis, namely, considering the difference of different plants on environmental conditions and the dynamic change of the phenotype parameters at different stages of the whole growth period, quantitatively analyzing the relationship among all elements, and determining the influence rule of each environmental parameter on the phenotype change, so as to obtain management measure optimization schemes at different periods under a specific cultivation scene, and realize the generalization of phenotype extraction and the intellectualization of cultivation management.
2. The method for the dynamic acquisition and analysis of the 3D phenotype of tomato during the whole growth period according to claim 1, characterized in that the environmental monitoring is achieved by: planting the tomatoes of the same variety in a solar greenhouse in a soilless culture mode, wherein i rows are planted from east to west, and j plants are planted in each row from north to south; in the aspect of cultivation management, a water and fertilizer integrated system and the same farming operation are adopted; uniformly selecting m sample points in a greenhouse, and regularly measuring temperature, humidity and illumination every day; according to the distribution map of the environmental difference, n individuals with larger environmental difference are selected from the sample points.
3. The method for dynamically acquiring and analyzing the 3D phenotype of the tomatoes during the whole growth period according to claim 1, wherein during organ segmentation and phenotype parameter extraction, skeletons of each internode are extracted and stored respectively, internode length is obtained by generating a skeleton curve of the internode, internode number is obtained by the stored internode number, and plant height is obtained by calculating the Euclidean distance between two endpoints on a stem;
wherein, the internode extraction is realized by the following steps:
step 3-1, inputting the numbers of two end points, namely a bottom point and a growth point on the main stem skeleton, and setting the end points as current traversal points from the bottom point;
step 3-2, finding the adjacent points of the current traversal point;
step 3-3, selecting an adjacent point which is not traversed, marking the adjacent point as traversed, and carrying out the following judgment and operation:
3-3-1 judges whether the adjacent point is a node, if so, entering a settlement step 3-4-1;
3-3-2, judging whether the current traversal node is an end point, if so, entering a settlement step 3-4-1;
3-3-3, judging whether the current traversal node is a leaf endpoint or not, and if so, directly entering a settlement step 3-4-2;
3-3-4, if the above are not the same, marking the point as a middle point, setting the middle point as a current traversal point, and returning to the step 3-2;
step 3-4, settlement step:
3-4-1, if the node is a node or an end point, marking the middle point starting from the last end point or the node as a stem, and returning to the step 3-3;
3-4-2, if the leaf end point is the leaf end point, directly clearing the mark of the middle point;
3-4-3 if it is the end of the stem, the algorithm ends.
4. The method for dynamically acquiring and analyzing the 3D phenotype of the tomato during the whole growth period according to claim 3, wherein during organ segmentation and phenotype parameter extraction, the numbers of all points on the main stem are obtained according to the internode extraction result, the extraction and preservation of all leaves are carried out, and the leaf length and the true leaf number are calculated;
wherein, each leaf is extracted by the following steps:
step 4-1, starting from the bottommost node, setting the node as the current node;
step 4-2, all adjacent points of the current node are searched;
4-3, selecting an adjacent point which is not traversed, and judging whether the adjacent point belongs to the main stem:
4-3-1, if yes, returning to the step 4-3;
4-3-2, if not, continuously traversing upwards according to the adjacency relation, and marking the traversed point as the blade point corresponding to the node until the blade end point is traversed;
and 4-4, judging whether the current node is the last node or not, if so, finishing the algorithm, if not, setting the next node as the current node, and returning to the step 4-2.
5. The method for dynamically acquiring and analyzing the 3D phenotype of the tomato during the whole growth period according to claim 1, wherein the phenotypic parameters are extracted, and the volume, true leaf number, internode number, length of each node, length of main stem, leaf length, leaf width, leaf area, and fruit transverse diameter and volume of each plant are obtained.
6. Method for the dynamic acquisition and analysis of the 3D phenotype of tomato during the whole growth phase according to claim 1, characterized in that the phenotype registration is achieved in particular by: defining a sequencing rule by combining the real topological structure of the tomatoes, defining a sequencing rule from a bottom point, namely the ground-near end, to a top point, namely the growing point, sequencing the segmented frameworks again, giving corresponding numbers, outputting new frameworks and visualizing, and directly realizing the registration of the phenotypic parameters of the crops at different periods.
7. A dynamic acquisition and analysis system of 3D phenotype of tomato during whole growth period, which employs the dynamic acquisition and analysis method of 3D phenotype of tomato during whole growth period as claimed in any one of claims 1-6, characterized in that:
the system comprises the following modules which are provided with a plurality of modules,
the environment monitoring module is used for measuring environmental parameters in a cultivation scene to obtain a plurality of environmental parameter measuring results, screening a plurality of target individual plants with larger environmental differences from a large number of plants according to an environmental difference distribution map, and acquiring, processing and analyzing subsequent data;
the image acquisition module is used for carrying out multi-angle multi-layer shooting around a single plant and fruits from bottom to top by adopting an RGB camera to acquire multi-view image sequences of each target plant and ripe tomato fruits thereof, and the acquisition time period covers the whole growth period of the plant;
the three-dimensional reconstruction module is used for importing the acquired multi-view image sequence into Visual SFM software and reconstructing three-dimensional point cloud data of each part and fruit of the target individual plant;
the point cloud processing module is used for carrying out point cloud denoising and scale correction preprocessing, acquiring complete single plant point cloud by using an Alignment registration algorithm, manually segmenting all point cloud data belonging to the lobe in the blade, and calculating the subsequent blade width and the subsequent blade area;
the organ segmentation module is used for skeletonizing point clouds of the rest parts of a single plant except for splinters, adopting a point search mode based on an adjacency relation aiming at the bent stems of the tomato plants, obtaining the connection positions of the petioles and the stems, namely nodes, and the end points of each leaf, namely leaf tips, according to the adjacency relation of each point in the adjacency matrix by inputting the obtained numbers of the two end points of the skeleton, the adjacency matrix and the main stem, and realizing the segmentation of each internode and each leaf by extending and searching from each node to the adjacent points around;
the phenotype parameter extraction module is used for automatically extracting and calculating phenotype parameters on the basis of organ segmentation;
the phenotype registration module is used for realizing the registration of phenotype parameters in different periods by utilizing a real topological structure aiming at point cloud data of the same target individual plant in different growth periods;
the data analysis module considers the difference of different plants on environmental conditions and the dynamic change of the phenotype parameters at different stages of the whole growth period, quantitatively analyzes the relationship among all elements, and determines the influence rule of each environmental parameter on the phenotype change, so as to obtain management measure optimization schemes at different periods under a specific cultivation scene and realize the generalization of phenotype extraction and the intellectualization of cultivation management.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110669883.5A CN113409265B (en) | 2021-06-11 | 2021-06-11 | Method and system for dynamically acquiring and analyzing 3D phenotype of tomato in whole growth period |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110669883.5A CN113409265B (en) | 2021-06-11 | 2021-06-11 | Method and system for dynamically acquiring and analyzing 3D phenotype of tomato in whole growth period |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113409265A CN113409265A (en) | 2021-09-17 |
| CN113409265B true CN113409265B (en) | 2022-04-12 |
Family
ID=77684693
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110669883.5A Active CN113409265B (en) | 2021-06-11 | 2021-06-11 | Method and system for dynamically acquiring and analyzing 3D phenotype of tomato in whole growth period |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113409265B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163854A (en) * | 2019-05-16 | 2019-08-23 | 北京农业信息技术研究中心 | Corn internode phenotypic parameter acquisition methods and device |
| WO2020128162A1 (en) * | 2018-12-21 | 2020-06-25 | Aalto University Foundation Sr. | A method of finding a target environment suitable for growth of a plant variety |
| CN111583328A (en) * | 2020-05-06 | 2020-08-25 | 南京农业大学 | Three-dimensional estimation method for epipremnum aureum leaf external phenotype parameters based on geometric model |
| CN111724433A (en) * | 2020-06-24 | 2020-09-29 | 广西师范大学 | Crop phenotype parameter extraction method and system based on multi-view vision |
| CN112465832A (en) * | 2020-11-25 | 2021-03-09 | 重庆大学 | Single-sided tree point cloud skeleton line extraction method and system based on binocular vision |
-
2021
- 2021-06-11 CN CN202110669883.5A patent/CN113409265B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020128162A1 (en) * | 2018-12-21 | 2020-06-25 | Aalto University Foundation Sr. | A method of finding a target environment suitable for growth of a plant variety |
| CN110163854A (en) * | 2019-05-16 | 2019-08-23 | 北京农业信息技术研究中心 | Corn internode phenotypic parameter acquisition methods and device |
| CN111583328A (en) * | 2020-05-06 | 2020-08-25 | 南京农业大学 | Three-dimensional estimation method for epipremnum aureum leaf external phenotype parameters based on geometric model |
| CN111724433A (en) * | 2020-06-24 | 2020-09-29 | 广西师范大学 | Crop phenotype parameter extraction method and system based on multi-view vision |
| CN112465832A (en) * | 2020-11-25 | 2021-03-09 | 重庆大学 | Single-sided tree point cloud skeleton line extraction method and system based on binocular vision |
Non-Patent Citations (3)
| Title |
|---|
| Accuracy Analysis of a Multi-View Stereo Approach for Phenotyping of Tomato Plants at the Organ Level;Johann Christian Rose,et al.;《Sensors 2015》;20150424;9651-9665 * |
| 利用多时序激光点云数据提取棉花表型参数方法;杨万能 等;《智慧农业》;20210412;51-62 * |
| 基于多时相点云数据的作物表型参数获取及动态量化方法研究;阳旭;《中国优秀硕士学位论文全文数据库（基础科学辑）》;20200215;A005-262 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113409265A (en) | 2021-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921885B (en) | Method for jointly inverting forest aboveground biomass by integrating three types of data sources | |
| AU2020103026A4 (en) | A Single Tree Crown Segmentation Algorithm Based on Super-pixels and Topological Features in Aerial Images | |
| CN111340826A (en) | Single tree crown segmentation algorithm for aerial image based on superpixels and topological features | |
| CN113221765B (en) | Vegetation phenological period extraction method based on digital camera image effective pixels | |
| CN107273820A (en) | A kind of Land Cover Classification method and system | |
| CN113657469B (en) | Automatic observation method and system for woody plant waiting period based on image recognition | |
| CN107507207B (en) | Pruning evaluation method based on fruit tree canopy illumination distribution calculation | |
| CN113221806B (en) | Cloud platform fusion multi-source satellite image and tea tree phenological period based automatic tea garden identification method | |
| CN108710766B (en) | Greenhouse plant liquid manure machine fertilizer regulation parameter calculation method based on growth model | |
| CN102542560B (en) | Method for automatically detecting density of rice after transplantation | |
| CN102663396B (en) | Method for automatically detecting rice milky ripe stage | |
| CN111199192A (en) | Method for detecting integral maturity of field red globe grapes by adopting parallel line sampling | |
| CN107680098A (en) | A kind of recognition methods of sugarcane sugarcane section feature | |
| Patil et al. | A review on leaf area index of horticulture crops and its importance | |
| CN110610438B (en) | Crop canopy petiole included angle calculation method and system | |
| Ouyang et al. | Assessment of canopy size using UAV-based point cloud analysis to detect the severity and spatial distribution of canopy decline | |
| CN108718890A (en) | A kind of grape breeding method based on big data analysis | |
| CN113920106B (en) | Corn growth vigor three-dimensional reconstruction and stem thickness measurement method based on RGB-D camera | |
| KR101810901B1 (en) | Apparatus for simulation for growth state crop organ | |
| Yang et al. | Modeling canopy photosynthesis and light interception partitioning among shoots in bi-axis and single-axis apple trees (Malus domestica Borkh.) | |
| CN114511777A (en) | Peach tree flowering phase monitoring and evaluating method based on image recognition method | |
| CN113409265B (en) | Method and system for dynamically acquiring and analyzing 3D phenotype of tomato in whole growth period | |
| CN109242875A (en) | A kind of winter wheat planting area extracting method and system | |
| CN113052433B (en) | Crop unit yield estimation method based on key time phase and farmland landscape characteristic parameters | |
| CN112215184B (en) | Method for detecting yield of camellia oleifera fruit trees based on three-dimensional laser scanner |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |