CN115995010B - Plant height extraction method and system based on plant population point cloud - Google Patents
Plant height extraction method and system based on plant population point cloud Download PDFInfo
- Publication number
- CN115995010B CN115995010B CN202310282215.6A CN202310282215A CN115995010B CN 115995010 B CN115995010 B CN 115995010B CN 202310282215 A CN202310282215 A CN 202310282215A CN 115995010 B CN115995010 B CN 115995010B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- plant
- point
- crop
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 42
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 36
- 230000011218 segmentation Effects 0.000 claims abstract description 31
- 238000005259 measurement Methods 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 11
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 239000011159 matrix material Substances 0.000 claims description 47
- 239000013598 vector Substances 0.000 claims description 24
- 238000013519 translation Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000007619 statistical method Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 238000012952 Resampling Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000010365 information processing Effects 0.000 abstract description 2
- 241000196324 Embryophyta Species 0.000 description 103
- 238000012360 testing method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 241001057636 Dracaena deremensis Species 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000001488 breeding effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 240000008042 Zea mays Species 0.000 description 1
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 description 1
- 235000002017 Zea mays subsp mays Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 235000005822 corn Nutrition 0.000 description 1
- 244000038559 crop plants Species 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 230000004720 fertilization Effects 0.000 description 1
- 238000003205 genotyping method Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000003973 irrigation Methods 0.000 description 1
- 230000002262 irrigation Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000009333 weeding Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Image Processing (AREA)
Abstract
The invention relates to the technical field of three-dimensional point cloud information processing, and provides a plant height extraction method and system based on plant group point clouds, wherein the method comprises the following steps: collecting image sequences of plants at different angles; carrying out three-dimensional point cloud reconstruction on the image sequence through a motion restoration structure algorithm and a multi-stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model; extracting a horizontal region of interest in the plant three-dimensional dense point cloud model; performing ground segmentation on the horizontal region of interest to obtain crop three-dimensional point clouds and ground three-dimensional point clouds; after preprocessing the three-dimensional point cloud of the crop, carrying out point cloud segmentation on the three-dimensional point cloud of the crop, and determining point cloud information of a single plant of the crop; and extracting the plant height of the single plant of the crop according to the point cloud information of the single plant of the crop to obtain a plant height measurement result. The invention combines the image remote sensing technology with the geometric three-dimensional reconstruction based on multiple views, and realizes the extraction and measurement of plant population plant height under the field scale based on a column space approximation algorithm segmentation algorithm.
Description
Technical Field
The invention relates to the technical field of three-dimensional point cloud information processing, in particular to a plant height extraction method and system based on plant group point clouds.
Background
Intelligent agriculture has become an important trend in agricultural development in China. The intelligent agriculture not only can digitize agricultural crops, but also can provide convenient interactive operation and observation for scientific researchers, and the like, thereby having great effect on promoting the development of agriculture. In the process of crop breeding and field production management, crop phenotype is essential information. The phenotypic characteristics of crops can directly reflect the yield of the crops, play an important role in screening dominant varieties, and can guide field management such as fertilization and irrigation, disinsection and weeding and the like. However, the traditional measurement is usually based on a large number of manual measurement methods, the scale is used for estimating the land block sampling, a large amount of labor is consumed, the labor is intensive, the throughput is low, and errors are easy to occur in the aspects of sampling, scale adjustment, reading and data recording. The progress of crop genotyping studies can be hampered, creating bottlenecks in the breeding program. The plant height parameter is used as one of the most important indexes for representing the phenotype information of crops, not only shows the growth vigor and vitality of the crops, but also has very important significance for the variety selection and the yield prediction of the crops.
At present, a technology for measuring plant height by combining three-dimensional point clouds is proposed, and an interested region is found and plant height is calculated by extracting point cloud data in a specific range. However, the method only carries out color filtering when confirming the region of interest, takes the reserved point set as the region of interest, and only the confirmed region of interest has certain error, thereby affecting the accuracy of the plant height calculation result.
Disclosure of Invention
The invention provides a plant height extraction method and system based on plant population point cloud, which are used for overcoming the defects of low accuracy and certain limitation of a measurement result in the plant height measurement method in the prior art.
In order to solve the technical problems, the technical scheme of the invention is as follows:
a plant height extraction method based on plant population point clouds comprises the following steps:
s1, collecting image sequences of plants at different angles;
s2, carrying out three-dimensional point cloud reconstruction on the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model;
s3, extracting a horizontal region of interest in the plant three-dimensional dense point cloud model;
s4, performing ground segmentation on the horizontal region of interest to obtain a crop three-dimensional point cloud and a ground three-dimensional point cloud;
s5, preprocessing the three-dimensional point cloud of the crop, and then carrying out point cloud segmentation on the three-dimensional point cloud of the crop to determine point cloud information of a single plant of the crop;
and S6, extracting the plant height of the single plant according to the point cloud information of the single plant of the crop, and obtaining a plant height measurement result.
Furthermore, the invention also provides a plant height extraction system based on the plant population point cloud, and the plant height extraction method provided by the invention is applied. The method comprises the following steps:
the image acquisition module is used for acquiring image sequences of plants at different angles;
the three-dimensional point cloud reconstruction module is used for reconstructing the three-dimensional point cloud of the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model;
the interesting region extraction module is used for extracting a horizontal interesting region in the plant three-dimensional dense point cloud model;
the ground segmentation module is used for carrying out ground segmentation on the horizontal region of interest to obtain crop three-dimensional point clouds and ground three-dimensional point clouds;
the crop single plant extraction module is used for carrying out point cloud segmentation on the three-dimensional point cloud of the crop after preprocessing the three-dimensional point cloud of the crop, and determining the point cloud information of the single plant of the crop;
the plant height extraction module is used for extracting plant heights of the single plants according to the point cloud information of the single plants of the crops to obtain a plant height measurement result.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that: according to the invention, an image remote sensing technology is combined with geometric three-dimensional reconstruction based on multiple visual angles, the segmentation algorithm based on the column space approximation algorithm solves the segmentation of plant population point clouds in the field scale by setting the AABB bounding box, so that the extraction and measurement of plant heights in the field scale are realized, the plant height measurement accuracy is ensured, and meanwhile, the measurement cost can be effectively reduced.
Drawings
Fig. 1 is a flowchart of a plant height extraction method based on a plant population point cloud of example 1.
Fig. 2 is a schematic representation of a three-dimensional dense point cloud model of a plant of example 2.
Fig. 3 is an estimated height histogram for trial a.
Fig. 4 is an estimated height histogram for trial b.
Fig. 5 is an estimated height histogram for trial c.
Fig. 6 is an estimated height histogram for trial d.
Fig. 7 is an estimated height histogram for test e.
Fig. 8 is a diagram of the architecture of a plant height extraction system based on a plant population point cloud of example 3.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
it will be appreciated by those skilled in the art that some of the illustrations in the figures may be omitted.
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
Example 1
The embodiment provides a plant height extraction method based on plant group point clouds, as shown in fig. 1, which is a flowchart of the plant height extraction method based on plant group point clouds in the embodiment.
The plant height extraction method based on the plant population point cloud provided by the embodiment comprises the following steps:
s1, collecting image sequences of plants at different angles;
s2, carrying out three-dimensional point cloud reconstruction on the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model;
s3, extracting a horizontal region of interest in the plant three-dimensional dense point cloud model;
s4, performing ground segmentation on the horizontal region of interest to obtain a crop three-dimensional point cloud and a ground three-dimensional point cloud;
s5, preprocessing the three-dimensional point cloud of the crop, and then carrying out point cloud segmentation on the three-dimensional point cloud of the crop to determine point cloud information of a single plant of the crop;
and S6, extracting the plant height of the single plant according to the point cloud information of the single plant of the crop, and obtaining a plant height measurement result.
In the embodiment, the image remote sensing technology is combined with the geometric three-dimensional reconstruction based on multiple visual angles, so that the extraction and measurement of plant heights of plant groups under the field scale are realized, and the accuracy of measurement results can be effectively improved.
In an alternative embodiment, unmanned aerial vehicles are used to acquire image sequences of plants at different angles. Compared with expensive acquisition equipment and a destructive acquisition mode, the embodiment realizes low-cost and nondestructive acquisition of plant point clouds.
Further, in an optional embodiment, in step S2, the step of reconstructing the three-dimensional point cloud of the image sequence by using a motion restoration structure algorithm and a multi-view stereo vision algorithm includes:
s201, extracting and matching a plurality of pairs of characteristic points from pixel points of the image sequence by adopting a SIFT operator.
S202, calculating a rotation matrix R and a translation matrix T which are transformed into a world coordinate system according to the matched pairs of feature point coordinates.
And S203, calculating the coordinates of each pixel point in the image sequence under a world coordinate system through the rotation matrix R and the translation matrix T.
S204, obtaining a plant three-dimensional dense point cloud model based on a multi-view stereoscopic vision algorithm according to the coordinates of each pixel point in a world coordinate system.
In this embodiment, the number of elements in the feature point coordinate point set a and the feature point coordinate point set B that are matched is the same and corresponds one to one, and in order to find the rotation matrix R and the transfer matrix T between the two point sets, the problem is converted into a solution formula b=r×a+t.
Further, in an alternative embodiment, the step of calculating the rotation matrix R and the translation matrix T transformed into the world coordinate system according to the matched pairs of feature point coordinates in the step S202 includes:
for the feature point coordinate point set A and the feature point coordinate point set B which are matched, calculating center points of the point set A and the point set B:
wherein, the element numbers of the point set A and the point set B which are matched are the same and correspond to each other one by one;
、/>
respectively represent the first point set A and the second point set BiCoordinate points->
、/>
The center points of the point sets a and B are indicated, respectively.NThe total number of coordinate points in the corresponding point set.
Re-centering the point set A and the point set B to obtain a new point set A 'and a new point set B':
wherein ,
、/>
representing the first of the new point set A 'and the new point set B', respectivelyiAnd coordinate points.
Calculating covariance matrix between new point set A' and new point set BH：
Covariance matrix obtained by SVD methodHFurther calculating the rotation matrix between the point set A and the point set B by the singular value vectors U, S and V of (2)RTransfer matrixT：
Further, in an alternative embodiment, in step S203, the coordinates of each pixel point in the image sequence in the world coordinate system are calculated through the rotation matrix R and the translation matrix T, and the expression is as follows:
wherein ,Mas a matrix of parameters within the camera,Xis an X-axis coordinate in the world coordinate system,Yis the Y-axis coordinate in the world coordinate system,Z const for the height in the world coordinate system,s[·]representing the world coordinate conversion scaling factor.
The above expression is obtained based on a pixel coordinate and world coordinate conversion formula, wherein:
wherein the first matrix on the right side of the equation is the in-camera parameter matrix, and the second matrix is the out-camera parameter matrix. Assuming that the image coordinates are known and the parameter matrix in the camera is acquired through calibration, the conversion formula can be simplified as follows:
wherein MAs a matrix of parameters within the camera,Rin order to rotate the matrix is rotated,Tin order to translate the matrix,Z const is the world coordinate system height, which can be set to 0.
By matrix transformation, it is possible to obtain:
from this, the rotation matrix and translation matrix are solved to obtain the scaling factors。
In an alternative embodiment, in step S2 of this embodiment, a voxel-based multi-view stereo vision algorithm is used to convert the sparse point cloud model into a dense point cloud model.
In an alternative embodiment, in step S4 of this embodiment, a CSF algorithm is used to perform ground segmentation on the horizontal region of interest, so as to obtain a crop three-dimensional point cloud and a ground three-dimensional point cloud.
In an optional embodiment, the step of preprocessing the three-dimensional point cloud of the crop in step S5 of this embodiment includes:
s511, removing noise based on point cloud statistical filtering: and carrying out statistical analysis on adjacent points in the crop three-dimensional point cloud to obtain average distances among the adjacent points and Gaussian distribution of the average distances, and filtering out points with the average distances outside a section defined by the mean value and the standard deviation as outliers.
Wherein, by carrying out statistical analysis on adjacent points, the average distance between the adjacent points is obtained, the average value and standard deviation can be obtained assuming that the obtained distribution is Gaussian distribution, and the points of which the average distance is outside the interval defined by the average value and standard deviation are regarded as outliers.
S512, resampling the three-dimensional point cloud of the crop subjected to noise removal based on point cloud downsampling.
The point cloud downsampling is to resample the point cloud according to a specific sampling principle, and aims to reduce the point cloud density on the premise of keeping the overall geometric characteristics of the point cloud unchanged, so that the calculation amount and the algorithm complexity of the related processing are reduced.
S513, carrying out point cloud data standardization processing on the resampled crop three-dimensional point cloud to enable the point clouds to be in the same order of magnitude.
The method is characterized in that based on the unified standardization processing of the point clouds, the dimension unit influence among the point clouds can be eliminated, so that the point clouds are in the same order of magnitude, and standard plant group point cloud data are obtained.
Further, in an alternative embodiment, the step of performing point cloud segmentation on the preprocessed three-dimensional point cloud of the crop comprises:
s521, calculating an included angle between a normal vector of the plant population point cloud data and the horizontal direction;
s522, setting an angle thresholdγExtracting three-dimensional point cloud of cropsObtaining a crop population stalk point cloud data set;
s523, setting the length and the width of a bounding box, splitting the crop group stalk point cloud into a plurality of blocks, drawing AABB bounding boxes of the blocks, and approximating the bounding boxes to a crop single plant.
The point cloud data in the plant three-dimensional dense point cloud model in this embodiment includesNData matrix of individual point clouds [N,(x,y,z)], wherein (x,y,z) The coordinate values of the point cloud point are the point cloud coordinate values of the point cloud point in three space dimensions.
Further optionally, the step of calculating an angle between the normal vector of the plant population point cloud data and the horizontal direction includes: and calculating a vector for each point cloud point in the data matrix through point cloud segmentation.
Where the normal estimate for each point of the point cloud dataset can be considered as an approximate extrapolation of the surface normal. The problem of determining a point normal to the surface approximates the problem of estimating a tangent surface normal to the surface, and thus, the conversion becomes an estimation problem of a least squares plane fit. The plane equation is assumed to be:
(1) Plane point set to be fittedx i ,y i ,z i ),i= 1,2,…,n，nIs the total number of points on the plane to be fitted.
The plane equation to be fitted is as follows:ax+by+cz=d(d≥0),a 2 +b 2 +c 2 = 1；
distance of arbitrary point to plane:d i =|ax i +by i +cz i -d|。
(2) To obtain a best fit plane, then the following needs to be satisfied:
wherein 
For all point-to-plane distances.
Therefore, the conversion is to solve the extremum problem:
。
wherein 
As an auxiliary function for solving->
An extremum.
(3) Respectively carrying out deviation guide on d, a, b and c:
wherein 
Is the sign of the partial derivative; will bedBringing into the formula of the distance from any point to a plane:
i.e.
；/>
And the average value of the coordinates of all points of the plane to be fitted in the x-axis, the y-axis and the z-axis respectively is represented.
For a pair ofaDeviation guide is calculated:
order the
，/>
，/>
, wherein Δx i Representing any point in a plane to be fittediIn x-axis coordinates and mean->
Delta of deltay i Representing any point in a plane to be fittediCoordinate and mean in y-axis->
Delta of deltaz i Representing any point in a plane to be fittediCoordinate and mean in z-axis->
Is a difference in (2); then:
in the same way, the processing method comprises the steps of,
，
combining the above formulas:
I.e. to solve the matrixAProblem of eigenvalues and eigenvectors of (a) matrixANamely, isnCovariance matrix of individual points. (a,b,c) T I.e. a feature vector of the matrix,λis a weight parameter, and is generally constant.
(4) Minimum feature vector
From the following componentsa 2 +b 2 +c 2 =1 to the inner product formx,x)=1；
From the following components
It can be seen that->
Therefore, the feature vector which is the least significant-increase to the corresponding feature vector is the normal vector.
Further alternatively, the normal vector is calculated with the horizontal direction to obtain an included angleθThe expression is:
wherein ,v 1 the normal vector is represented by a vector of the normal,v 2 representing a horizontal normal; iv 1 The I is the normal vectorv 1 Length, ||v 2 The I is the horizontal normalv 2 Is a length of (2); included angleθThe value range of (2) is [0 ],π]。
further optionally, the set angle thresholdγThe value range of (2) is [1,2 ]]The method is used for extracting leaf point clouds in the crop group point clouds to obtain crop group stalk point clouds.
The embodiment is realized by setting the bounding box lengthxSum width ofyApproximately dividing the crop group stalk point cloud into crop single plant point clouds; by counting the height of the bounding boxhThe target is the plant height extraction of crops.
In the embodiment, by setting the AABB bounding box and dividing the plant population point cloud under the field scale based on the column space approximation algorithm dividing algorithm, the measurement cost can be effectively reduced while the plant height measurement accuracy is ensured.
Example 2
The embodiment applies the plant height extraction method based on the plant population point cloud, which is provided in the embodiment 1, to carry out a plant height extraction test.
In the embodiment, corn colony plants under the field scale are tested to obtain a three-dimensional dense point cloud model schematic diagram of the plants shown in fig. 2, wherein (a) of fig. 2 is normal vector point cloud, (b) of fig. 2 is stalk point cloud, and (c) of fig. 2 is a cylinder space approximate segmentation schematic diagram. From the figures, each figure is an example figure of a column space approximate segmentation algorithm step, and single plant extraction of plant groups under the field scale is realized.
In this example, corn plants at the field scale were tested, the distribution of the test components is shown in table 1 below, and an estimated height histogram of test a, b, c, d, e shown in fig. 3 to 7 was obtained.
TABLE 1 distribution of test components
The estimated height of each corn plant can be obtained according to the estimated height histogram of the test a, b, c, d, e, and then the actual height of the corresponding corn plant is obtained through a linear model of the actual height and the estimated height constructed based on a large amount of experimental data. From the graph, the experiment can well realize the extraction of the plant height of the plant population under the field scale.
Example 3
The embodiment provides a plant height extraction system based on plant population point clouds, and the plant height extraction method based on the plant population point clouds provided in the embodiment 1 is applied. As shown in fig. 8, an architecture diagram of the plant height extraction system of the present embodiment is shown.
The plant height extraction system provided in this embodiment includes:
the image acquisition module is used for acquiring image sequences of plants at different angles;
the three-dimensional point cloud reconstruction module is used for reconstructing the three-dimensional point cloud of the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model;
the interesting region extraction module is used for extracting a horizontal interesting region in the plant three-dimensional dense point cloud model;
the ground segmentation module is used for carrying out ground segmentation on the horizontal region of interest to obtain crop three-dimensional point clouds and ground three-dimensional point clouds;
the crop single plant extraction module is used for carrying out point cloud segmentation on the three-dimensional point cloud of the crop after preprocessing the three-dimensional point cloud of the crop, and determining the point cloud information of the single plant of the crop;
the plant height extraction module is used for extracting plant heights of the single plants according to the point cloud information of the single plants of the crops to obtain a plant height measurement result.
In an alternative embodiment, the image acquisition module is used in cooperation with an unmanned aerial vehicle, and is used for acquiring image sequences of plants at different angles under a field scale.
In an alternative embodiment, the three-dimensional point cloud reconstruction module extracts and matches a plurality of pairs of feature points from the pixel points of the image sequence by using a SIFT operator, calculates a rotation matrix R and a translation matrix T transformed to a world coordinate system according to the matched plurality of pairs of feature point coordinates, further calculates the coordinate of each pixel point in the image sequence under the world coordinate system by using the rotation matrix R and the translation matrix T, and obtains a plant three-dimensional dense point cloud model based on a multi-view stereoscopic vision algorithm according to the coordinate of each pixel point under the world coordinate system.
In an alternative embodiment, the three-dimensional point cloud reconstruction module converts the sparse point cloud model into a dense point cloud model using a voxel-based multi-view stereoscopic algorithm.
In an alternative embodiment, the ground segmentation module performs ground segmentation on the horizontal region of interest by adopting a CSF algorithm to obtain a crop three-dimensional point cloud and a ground three-dimensional point cloud.
In an optional embodiment, the crop single plant extraction module removes noise based on point cloud statistical filtering, resamples the three-dimensional crop point cloud after noise removal based on point cloud downsampling, and performs point cloud data standardization processing on the resampled three-dimensional crop point cloud to enable the point clouds to be in the same order of magnitude, so that standard plant group point cloud data are obtained.
Further, when the crop single plant extraction module performs point cloud segmentation on the preprocessed crop three-dimensional point cloud, an included angle between a normal vector of plant group point cloud data and the horizontal direction is calculated, and then an angle threshold value is setγExtracting leaf point clouds in the three-dimensional point clouds of crops to obtain a crop group stalk point cloud data set; the length and width of the bounding box are further set, the crop group stalk point cloud is disassembled into a plurality of blocks, AABB bounding boxes of the blocks are drawn, and the bounding boxes are similar to a single crop plant.
In the embodiment, by setting the AABB bounding box and dividing the plant population point cloud under the field scale based on the column space approximation algorithm dividing algorithm, the measurement cost can be effectively reduced while the plant height measurement accuracy is ensured.
It is to be understood that the above examples of the present invention are provided by way of illustration only and not by way of limitation of the embodiments of the present invention. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are desired to be protected by the following claims.
Claims (5)
1. The plant height extraction method based on the plant population point cloud is characterized by comprising the following steps of:
s1, collecting image sequences of plants at different angles;
s2, carrying out three-dimensional point cloud reconstruction on the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model; the step of reconstructing the three-dimensional point cloud of the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm comprises the following steps:
s201, extracting and matching a plurality of pairs of characteristic points from pixel points of the image sequence by adopting a SIFT operator;
s202, calculating a rotation matrix R and a translation matrix T which are transformed into a world coordinate system according to the matched pairs of feature point coordinates; the method comprises the following specific steps:
for the feature point coordinate point set A and the feature point coordinate point set B which are matched, calculating center points of the point set A and the point set B:
wherein, the element numbers of the point set A and the point set B which are matched are the same and correspond to each other one by one；
、/>
Respectively represent the first point set A and the second point set BiCoordinate points->
、/>
Respectively representing the center points of the point set A and the point set B;Nthe total number of coordinate points in the corresponding point set;
re-centering the point set A and the point set B to obtain a new point set A 'and a new point set B':
wherein ,
、/>
representing the first of the new point set A 'and the new point set B', respectivelyiA coordinate point;
calculating covariance matrix between new point set A' and new point set BH：
Covariance matrix obtained by SVD methodHFurther calculating the rotation matrix between the point set A and the point set B by the singular value vectors U, S and V of (2)RTransfer matrixT：
S203, calculating the coordinates of each pixel point in the image sequence under a world coordinate system through the rotation matrix R and the translation matrix T; the expression is as follows:
wherein ,Mas a matrix of parameters within the camera,Xis an X-axis coordinate in the world coordinate system,Yis the Y-axis coordinate in the world coordinate system,Z const for the height in the world coordinate system,s[·]representing the world coordinate conversion scaling factor;
s204, obtaining a plant three-dimensional dense point cloud model based on a multi-view stereoscopic vision algorithm according to the coordinates of each pixel point in a world coordinate system;
s3, extracting a horizontal region of interest in the plant three-dimensional dense point cloud model;
s4, performing ground segmentation on the horizontal region of interest to obtain a crop three-dimensional point cloud and a ground three-dimensional point cloud;
s5, preprocessing the three-dimensional point cloud of the crop, and then carrying out point cloud segmentation on the three-dimensional point cloud of the crop to determine point cloud information of a single plant of the crop; the method for carrying out point cloud segmentation on the preprocessed crop three-dimensional point cloud comprises the following steps of:
s521, calculating an included angle between a normal vector of the plant population point cloud data and the horizontal direction;
s522, setting an angle thresholdγExtracting crop IIIBlade point clouds in the dimension point clouds obtain a crop group stalk point cloud data set;
s523, setting the length and the width of a bounding box, splitting the crop group stalk point cloud into a plurality of blocks, drawing AABB bounding boxes of the blocks, and approximating the bounding boxes to a crop single plant;
wherein the point cloud data in the plant three-dimensional dense point cloud model comprisesNData matrix of individual point clouds [N,(x,y,z)], wherein (x,y,z) The point cloud coordinate values of the point cloud points in three space dimensions are obtained; the step of calculating the included angle between the normal vector of the plant population point cloud data and the horizontal direction comprises the following steps: calculating a vector for each point cloud point in the data matrix through point cloud segmentation;
calculating the normal vector and the horizontal direction to obtain an included angleθThe expression is:
wherein ,v 1 the normal vector is represented by a vector of the normal,v 2 representing a horizontal normal; iv 1 The I is the normal vectorv 1 Length, ||v 2 The I is the horizontal normalv 2 Is a length of (2); included angleθThe value range of (2) is [0 ],π]；
and S6, extracting the plant height of the single plant according to the point cloud information of the single plant of the crop, and obtaining a plant height measurement result.
2. The plant height extraction method based on plant population point clouds according to claim 1, wherein in the step S4, a CSF algorithm is adopted to perform ground segmentation on a horizontal region of interest, so as to obtain a crop three-dimensional point cloud and a ground three-dimensional point cloud.
3. The plant height extraction method based on plant population point clouds according to claim 1, wherein in the step S5, the step of preprocessing the crop three-dimensional point cloud comprises:
s511, carrying out statistical analysis on adjacent points in the three-dimensional point cloud of the crop to obtain average distances among the adjacent points and Gaussian distribution of the average distances, and filtering out points with the average distances outside a section defined by the average value and the standard deviation as outliers;
s512, resampling the three-dimensional point cloud of the crop after noise removal based on point cloud downsampling;
s513, carrying out point cloud data standardization processing on the resampled three-dimensional point cloud of the crop, so that the point clouds are in the same order of magnitude, and obtaining standard plant group point cloud data.
4. The plant height extraction method based on plant population point clouds as claimed in claim 1, wherein the angle threshold valueγThe value range of (2) is [1,2 ]]。
5. The plant height extraction system based on the plant population point cloud, which is applied to the plant height extraction method based on the plant population point cloud according to any one of claims 1 to 4, is characterized by comprising the following steps:
the image acquisition module is used for acquiring image sequences of plants at different angles;
the three-dimensional point cloud reconstruction module is used for reconstructing the three-dimensional point cloud of the image sequence through a motion restoration structure algorithm and a multi-view stereoscopic vision algorithm to obtain a plant three-dimensional dense point cloud model;
the interesting region extraction module is used for extracting a horizontal interesting region in the plant three-dimensional dense point cloud model;
the ground segmentation module is used for carrying out ground segmentation on the horizontal region of interest to obtain crop three-dimensional point clouds and ground three-dimensional point clouds;
the crop single plant extraction module is used for carrying out point cloud segmentation on the three-dimensional point cloud of the crop after preprocessing the three-dimensional point cloud of the crop, and determining the point cloud information of the single plant of the crop;
the plant height extraction module is used for extracting plant heights of the single plants according to the point cloud information of the single plants of the crops to obtain a plant height measurement result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310282215.6A CN115995010B (en) | 2023-03-22 | 2023-03-22 | Plant height extraction method and system based on plant population point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310282215.6A CN115995010B (en) | 2023-03-22 | 2023-03-22 | Plant height extraction method and system based on plant population point cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115995010A CN115995010A (en) | 2023-04-21 |
| CN115995010B true CN115995010B (en) | 2023-06-16 |
Family
ID=85992379
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310282215.6A Active CN115995010B (en) | 2023-03-22 | 2023-03-22 | Plant height extraction method and system based on plant population point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115995010B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065352A (en) * | 2012-12-20 | 2013-04-24 | 北京农业信息技术研究中心 | Plant three-dimensional reconstruction method based on image and scanning data |
| WO2018049843A1 (en) * | 2016-09-14 | 2018-03-22 | 杭州思看科技有限公司 | Three-dimensional sensor system and three-dimensional data acquisition method |
| KR101910751B1 (en) * | 2017-11-22 | 2018-10-22 | 중앙대학교 산학협력단 | Method and system for acquiring 3-D data considering the shape feature of element of construction structure |
| JP2019192059A (en) * | 2018-04-27 | 2019-10-31 | 田中 成典 | Object recognition device |
| CN112348802A (en) * | 2020-11-13 | 2021-02-09 | 黑龙江八一农垦大学 | Plant leaf inclination angle measuring method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110335295B (en) * | 2019-06-06 | 2021-05-11 | 浙江大学 | Plant point cloud acquisition registration and optimization method based on TOF camera |
| CN111667529A (en) * | 2020-05-25 | 2020-09-15 | 东华大学 | Plant point cloud blade segmentation and phenotype characteristic measurement method |
| WO2022032329A1 (en) * | 2020-08-14 | 2022-02-17 | Agriculture Victoria Services Pty Ltd | System and method for image-based remote sensing of crop plants |
| CN113780144A (en) * | 2021-09-06 | 2021-12-10 | 广西大学 | Crop plant number and stem width automatic extraction method based on 3D point cloud |
| CN115239882A (en) * | 2022-07-20 | 2022-10-25 | 安徽理工大学环境友好材料与职业健康研究院(芜湖) | Crop three-dimensional reconstruction method based on low-light image enhancement |
-
2023
- 2023-03-22 CN CN202310282215.6A patent/CN115995010B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065352A (en) * | 2012-12-20 | 2013-04-24 | 北京农业信息技术研究中心 | Plant three-dimensional reconstruction method based on image and scanning data |
| WO2018049843A1 (en) * | 2016-09-14 | 2018-03-22 | 杭州思看科技有限公司 | Three-dimensional sensor system and three-dimensional data acquisition method |
| KR101910751B1 (en) * | 2017-11-22 | 2018-10-22 | 중앙대학교 산학협력단 | Method and system for acquiring 3-D data considering the shape feature of element of construction structure |
| JP2019192059A (en) * | 2018-04-27 | 2019-10-31 | 田中 成典 | Object recognition device |
| CN112348802A (en) * | 2020-11-13 | 2021-02-09 | 黑龙江八一农垦大学 | Plant leaf inclination angle measuring method and device |
Non-Patent Citations (3)
| Title |
|---|
| Error Detection of Packing Box Based on Registration between Incomplete Point Cloud and CAD Model;Jiexin Zhou等;2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC);359-364 * |
| 基于运动恢复结构的玉米植株三维重建与性状提取;梁秀英;周风燃;陈欢;梁博;许锡晨;杨万能;;农业机械学报;51(第06期);209-219 * |
| 植物三维建模ICP点云配准优化;陆健强等;农业工程学报;第38卷(第02期);183-191 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115995010A (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111724433B (en) | Crop phenotype parameter extraction method and system based on multi-view vision | |
| CN102829739B (en) | Object-oriented remote sensing inversion method of leaf area index of crop | |
| Feng et al. | Cotton yield estimation from UAV-based plant height | |
| CN106651900B (en) | A kind of overhead strawberry three-dimensional modeling method in situ based on contours segmentation | |
| CN108594262A (en) | High method is set in a kind of combination unmanned plane and the extraction of ground laser radar | |
| CN101144705A (en) | Method for monitoring pig growth using binocular vision technology | |
| CN110782467B (en) | Horse body ruler measuring method based on deep learning and image processing | |
| CN111429490A (en) | Agricultural and forestry crop three-dimensional point cloud registration method based on calibration ball | |
| Fang et al. | High-throughput volumetric reconstruction for 3D wheat plant architecture studies | |
| CN112669363B (en) | Method for measuring three-dimensional green space of urban green space | |
| CN115661641A (en) | Method and system for evaluating important phenotype parameters of tea garden in different growth stages based on multi-source unmanned aerial vehicle remote sensing | |
| Zhu et al. | A calculation method of phenotypic traits based on three-dimensional reconstruction of tomato canopy | |
| CN110610438B (en) | Crop canopy petiole included angle calculation method and system | |
| Li et al. | Three-dimensional reconstruction and phenotype measurement of maize seedlings based on multi-view image sequences | |
| Miao et al. | Measurement method of maize morphological parameters based on point cloud image conversion | |
| CN112241964A (en) | Light strip center extraction method for line structured light non-contact measurement | |
| CN115995010B (en) | Plant height extraction method and system based on plant population point cloud | |
| CN113554691B (en) | Plant height measuring method | |
| CN117218525A (en) | Plant phenotype parameter extraction method, device, equipment and storage medium | |
| CN110554406B (en) | Method for inverting secondary forest structure parameters based on unmanned aerial vehicle stereo photogrammetry point cloud | |
| Hall et al. | A method for extracting detailed information from high resolution multispectral images of vineyards | |
| CN113487636A (en) | Automatic extraction method for plant height and line spacing of wide-ridge crops based on laser radar | |
| CN112577954A (en) | Urban green land biomass estimation method | |
| CN118115882B (en) | Agricultural robot inspection identification method based on multi-source perception fusion | |
| Zuo et al. | A Review of Plant Phenotype Research Based on 3D Point Cloud Technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |