CN115631136A - 3D point cloud image-based method for rapidly measuring phenotypic parameters of schima superba seedlings - Google Patents
3D point cloud image-based method for rapidly measuring phenotypic parameters of schima superba seedlings Download PDFInfo
- Publication number
- CN115631136A CN115631136A CN202211184370.6A CN202211184370A CN115631136A CN 115631136 A CN115631136 A CN 115631136A CN 202211184370 A CN202211184370 A CN 202211184370A CN 115631136 A CN115631136 A CN 115631136A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- lotus
- wood
- images
- schima superba
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 241000610442 Schima superba Species 0.000 title claims abstract description 74
- 238000003384 imaging method Methods 0.000 title description 3
- 241000196324 Embryophyta Species 0.000 claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 44
- 240000002853 Nelumbo nucifera Species 0.000 claims abstract description 43
- 235000006508 Nelumbo nucifera Nutrition 0.000 claims abstract description 43
- 235000006510 Nelumbo pentapetala Nutrition 0.000 claims abstract description 43
- 239000002023 wood Substances 0.000 claims abstract description 40
- 238000010586 diagram Methods 0.000 claims abstract description 11
- 230000011218 segmentation Effects 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims abstract description 7
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical group [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 claims abstract description 5
- 238000005457 optimization Methods 0.000 claims abstract description 4
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 8
- 238000000513 principal component analysis Methods 0.000 claims description 7
- 238000013138 pruning Methods 0.000 claims description 6
- 238000009616 inductively coupled plasma Methods 0.000 claims description 5
- 238000010845 search algorithm Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 4
- 239000002689 soil Substances 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- MXYUKLILVYORSK-UHFFFAOYSA-N (+/-)-allo-lobeline Natural products C1CCC(CC(=O)C=2C=CC=CC=2)N(C)C1CC(O)C1=CC=CC=C1 MXYUKLILVYORSK-UHFFFAOYSA-N 0.000 description 1
- MXYUKLILVYORSK-HBMCJLEFSA-N (-)-lobeline Chemical compound C1([C@@H](O)C[C@H]2N([C@H](CCC2)CC(=O)C=2C=CC=CC=2)C)=CC=CC=C1 MXYUKLILVYORSK-HBMCJLEFSA-N 0.000 description 1
- 101100498818 Arabidopsis thaliana DDR4 gene Proteins 0.000 description 1
- 239000002028 Biomass Substances 0.000 description 1
- 235000014698 Brassica juncea var multisecta Nutrition 0.000 description 1
- 235000006008 Brassica napus var napus Nutrition 0.000 description 1
- 240000000385 Brassica napus var. napus Species 0.000 description 1
- 235000006618 Brassica rapa subsp oleifera Nutrition 0.000 description 1
- 235000004977 Brassica sinapistrum Nutrition 0.000 description 1
- 241000219000 Populus Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 235000021384 green leafy vegetables Nutrition 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 229960002339 lobeline Drugs 0.000 description 1
- 229930013610 lobeline Natural products 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Probability & Statistics with Applications (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method for quickly measuring phenotypic parameters of a schima superba seedling based on a 3D point cloud image. Firstly, respectively collecting color lotus images and corresponding depth lotus images of the to-be-detected lotus seedlings at various angles; then, after preprocessing, obtaining a precisely registered point cloud wood-lotus image; then, carrying out hierarchical clustering on the point cloud wood load images subjected to fine registration to obtain all cluster clusters of the point clouds of all layers, and then carrying out plant skeleton extraction and optimization according to all cluster clusters of the point clouds of all layers to obtain an undirected skeleton map; extracting a point cloud image of the stem of the schima superba according to the undirected schima superba skeleton diagram and all cluster clusters of the point clouds of all layers; then, carrying out blade segmentation on the precisely registered point cloud arbor figure image to obtain an initial blade point cloud image, and then clustering the initial blade point cloud image to obtain an arbor figure blade point cloud image; and finally calculating the phenotype parameters of the schima superba seedlings. The method can automatically identify the phenotypic parameters of the schima superba seedlings, and improves the identification effect and efficiency.
Description
Technical Field
The invention relates to a method for quickly measuring phenotypic parameters of a schima superba seedling, in particular to a method for quickly measuring phenotypic parameters of the schima superba seedling based on a 3D point cloud image.
Background
The phenotypic parameters are the most visual response of the diversity of each seedling of the schima superba and are the comprehensive expression of the self gene expression and the environmental adaptability of the plant. The phenotypic characters are used as the external characteristics of the genotypes, and the variation size of the population can be intuitively revealed by researching the phenotypic diversity of the population.
Current methods for non-contact measurement of plant phenotypic parameters are divided into 2D image techniques and 3D image techniques. Although developed over the years, many researchers have measured phenotypic parameters of some plants, such as phenotypic parameters for canola, tree diameter, and plant height, using 2D image processing techniques. However, the two-dimensional image lacks depth information, and it is difficult to obtain accurate structural information of the plant, so that the plant phenotype analysis by using the three-dimensional image technology becomes a new trend. RGB-D cameras based on the time-of-flight (ToF) principle have created a convenience for 3D phenotyping techniques. Compared with the common RGB camera, the RGB-D camera is added with a depth measurement, and can sense the surrounding environment and change more conveniently and accurately by combining a depth image processing algorithm. Typical applications include: estimating plant phenotype parameters of the poplar seedlings by using the Kinect V1; establishing an automatic system based on Kinect v2 to obtain a fine 3D point cloud and a grid model of a plant and measuring key growth parameters such as height, leaf area, volume and biomass of the single-potted leaf vegetables; the method is based on the independent leaf segmentation of dense plant point clouds generated by facet excessive segmentation and facet area growth, and is effective to greenhouse ornamental plants through experiments. At present, aiming at the schima superba seedlings, the phenotypic parameters of the schima superba seedlings are automatically extracted through three-dimensional point cloud analysis, and no systematic research exists.
Disclosure of Invention
Aiming at the problems in the background art, the invention aims to provide a method for quickly measuring the phenotype parameters of the seedlings of the schima superba based on a 3D point cloud image. The invention utilizes the camera to obtain high-precision point cloud of the schima superba seedlings, completes the segmentation of stems and leaves based on a skeletonization method, and solves the problem of automatically calculating phenotypic parameters of the schima superba seedlings, such as plant height, stem length, stem direction, leaf length, leaf angle, leaf area and the like.
The technical scheme adopted by the invention comprises the following steps:
1) Respectively acquiring color wood lotus images and corresponding depth wood lotus images of the wood lotus seedlings to be detected at all angles by utilizing an RGB (red, green and blue) camera and a depth camera;
2) Preprocessing the color wood lotus images and the corresponding depth wood lotus images at all angles to obtain precisely registered point cloud wood lotus images;
3) Carrying out hierarchical clustering on the point cloud deadweight image subjected to fine registration by using a slice clustering method to obtain all clustering clusters of each layer of point cloud, and then carrying out plant skeleton extraction and optimization according to all clustering clusters of each layer of point cloud to obtain an undirected skeleton map;
4) Extracting a point cloud image of the stem of the schima superba according to the undirected schima superba skeleton map and all cluster clusters of point clouds of all layers;
5) Carrying out leaf segmentation on the point cloud image of the fine registration by using the point cloud image of the stem of the schima superba to obtain an initial leaf point cloud image, and then clustering the initial leaf point cloud image to obtain a schima superba leaf point cloud image;
6) And calculating the phenotype parameters of the schima superba seedlings to be detected according to the schima superba stem point cloud image and the schima superba leaf point cloud image.
The 2) is specifically as follows:
2.1 Respectively aligning the color wood lotus images and the corresponding depth wood lotus images at all angles to respectively obtain initial point cloud wood lotus images at all angles;
2.2 Respectively removing background and floating point noise in the initial point cloud wood-lotus image under each angle by using a filtering method to respectively obtain a denoising point cloud wood-lotus image under each angle;
2.3 In every two adjacent angles of the denoised point cloud wood-lotus images, performing principal component feature extraction on the current two denoised point cloud wood-lotus images by using a principal component analysis algorithm to obtain principal feature components, solving according to the principal axis direction and the principal feature components of the current two denoised point cloud wood-lotus images to obtain a rotation transformation matrix, performing point cloud rough registration on the denoised point cloud wood-lotus images under the current two adjacent angles according to the current rotation transformation matrix to obtain corresponding rough registration point cloud images, traversing and performing point cloud rough registration on the denoised point cloud wood-lotus images under each group of two adjacent angles to obtain all roughly registered point cloud wood-lotus images;
2.4 Performing point cloud fine registration on all the roughly registered point cloud wood-lotus images by utilizing a KD-Tree accelerated ICP (inductively coupled plasma) algorithm to obtain the finely registered point cloud wood-lotus images.
The 3) is specifically as follows:
3.1 Layering the precisely registered point cloud image in the main axis direction to obtain point clouds of all layers, and clustering the point clouds of all layers respectively to obtain all cluster clusters corresponding to the point clouds of all layers;
3.2 Using the centroid of each cluster as the skeleton node of the corresponding cluster, and then connecting all skeleton nodes of each layer to form an initial wood lotus skeleton map;
3.3 Branch pruning is performed on the initial wood-lotus skeleton map by using a quantity threshold-based branch pruning algorithm to obtain an undirected wood-lotus skeleton map.
The 4) is specifically as follows:
4.1 Identifying a longest branch of the undirected wood-lotus skeleton map using a longest path search algorithm;
4.2 Sequentially calculating and identifying coordinate dot product values between each skeleton node and two adjacent skeleton nodes in the longest branch, wherein if the dot product values are negative numbers, the current skeleton node is the tail end of the stem line, and the path from the starting point of the longest branch to the tail end of the stem line is the stem of the schima superba seedling;
4.3 Extracting all cluster clusters corresponding to the point clouds of all layers based on the stem of the schima superba seedling, and integrating the point clouds corresponding to the extracted cluster clusters to obtain a schima superba stem point cloud image.
The phenotypic parameters include plant height, stem length, stem diameter, growth direction, leaf length.
The stem length was calculated as follows:
respectively calculating Euclidean distances between two adjacent skeleton nodes according to the point cloud image of the stem of the schima superba, and taking the total Euclidean distance obtained by accumulating the Euclidean distances between all two adjacent skeleton nodes as the stem length.
The growth direction is calculated as follows:
and calculating a covariance matrix of the point cloud image of the stem of the schima superba by using a principal component analysis algorithm, obtaining a first characteristic vector as a direction vector of the stem, and calculating the cosine of a vector included angle between the direction vector of the stem and a ground normal vector and using the cosine as the growth direction of the stem of the schima superba.
The leaf length is calculated as follows:
calculating the Euclidean distance between each two adjacent skeleton nodes in the blade skeleton corresponding to each blade in the point cloud image of the wood lotus blade, and taking the total Euclidean distance obtained by accumulating the Euclidean distances between each two adjacent skeleton nodes as the blade length of each single blade.
The invention has the beneficial effects that:
the method for calculating the plant phenotype parameters based on the point cloud image can observe the growth condition of seedlings, is beneficial to helping the cultivation work of the seedlings of the schima superba seedlings, has higher calculation precision compared with other methods, can calculate more phenotype parameters simultaneously, and enables an algorithm to adapt to more growth environments.
Drawings
FIG. 1 is a block flow diagram of the present invention.
FIG. 2 is the plant point cloud after the experimental platform, the flowerpot and the soil are removed in the embodiment;
FIG. 3 is a point cloud with registration completed in an embodiment;
FIG. 4 shows the stem lines extracted in the examples;
FIG. 5 shows a blade according to an embodiment in isolation.
Fig. 6 is a schematic view of a camera coordinate system.
The specific implementation mode is as follows:
the present invention will be described in further detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
2. As shown in fig. 1, the present invention comprises the steps of:
1) Respectively acquiring color wood lotus images and corresponding depth wood lotus images of the wood lotus seedlings to be detected at all angles by utilizing an RGB (red, green and blue) camera and a depth camera;
the method is characterized in that the schima superba seedlings are cultivated in a plantation, and are transplanted into an indoor flowerpot only when images are collected, and the process of collecting the scene point cloud is completed under the normal lighting condition. In order to better acquire blade information, an image acquisition platform with an Azure Kinect V3 camera as a center is built in the embodiment and is used for acquiring a point cloud image of the schima superba seedlings.
The acquisition platform mainly comprises an Azure Kinect camera, a notebook computer (AMD Ryzen R7-4800H/16G/1TB/GTX1650 4G DDR4), a camera support with adjustable height, a laboratory bench with fixed height and a precise rotary turntable. The AzureKinect camera is provided with a depth camera with the resolution of 1024x1024 pixels and a 4096x3072 pixel RGB camera, so that a high-precision point cloud image can be obtained. The notebook stores software and a supporting library required for developing the acquisition of the image of the schima superba seedlings and the processing of the point cloud image: microsoft Visual Studio 2017, PCL1.8.1. The camera support is used for supporting the camera, and the adjustable height of the camera support facilitates the camera to find good shooting height and angle. The experiment platform is used for placing potted schima superba seedlings, and the rotary turntable is used for obtaining plant images at different angles. The plant is placed on the center position of the rotary turntable, and the turntable is placed at the center of the experiment platform. The Azure Kinect camera is placed on the support and is 45 contained angles with the level, and highly setting is about 1.5m, and the distance between camera and the plant is about 0.6 meters or so (be in the best imaging range of camera), and whole plant need be located camera field of vision center and camera field of vision within range except that the plant, the workstation of placing the plant, black curtain should not have other debris. The distance between the plant and the curtain is 1 meter, and the whole back shadow of the plant is positioned in the range of the curtain.
2) Preprocessing the color wood lotus images and the corresponding depth wood lotus images under all angles to obtain a precisely registered point cloud wood lotus image;
2) The method specifically comprises the following steps:
2.1 Respectively aligning the color wood-lotus images and the corresponding depth wood-lotus images at all angles to obtain initial point cloud wood-lotus images at all angles; in this embodiment, a built-in program provided by Microsoft for AzureKinect is directly used to convert a color wood-lotus image and a depth wood-lotus image acquired by a camera into an initial point cloud wood-lotus image after alignment, and the initial point cloud wood-lotus image is output.
2.2 Respectively removing a background (except for the seedlings of the wood lotus) and floating point noise in the initial point cloud wood lotus image under each angle by using a filtering method to respectively obtain a denoised point cloud wood lotus image under each angle; in this embodiment, the following concrete steps are performed:
firstly, in the initial point cloud wood load image, the depth distances from the camera to the spatial areas where the plant and the background curtain are respectively located are different, namely the z-axis coordinate values of the point of the area where the plant is located and the point of the area where the curtain is located in a coordinate system taking the camera as the original point are different. Therefore, a filtering value domain of the PassThrough function in the z-axis direction is only required to be set according to the farthest distance from the camera to the plant and the distance from the camera to the black curtain, and the PassThrough function is used for removing the background curtain in the initial point cloud wooden lotus image.
As shown in fig. 6, when viewed from behind the Azure Kinect camera, the coordinate system uses the position of the camera as the origin, the right hand side is the positive direction of the x axis, the lower side is the positive direction of the y axis (i.e. the direction of the main axis), and the z axis points to the plant, which is consistent with the definition of the world coordinate system.
The next step is removing the flowerpot, and the soil plane in the flowerpot is just the separating surface of the plant and other objects, so that the plant and other parts such as the flowerpot can be distinguished by firstly searching the soil plane and then extracting the point cloud of the plant. The method specifically comprises the following steps: before soil level detection, the area where the soil level is located needs to be located first. Since the height of the pots used in the experiments was fixed, the pot height was 0.10 m, and the soil was always inside the pot, the detection of the soil level would select the area between 0.08m and 0.18m from the experimental table. In the interval range, a soil plane is searched by using a RANSAC plane fitting method, the area above the soil plane is the plant point cloud which needs to be operated next, and the area below the soil plane is removed by a straight-through filtering algorithm.
And finally, removing floating point noise. Due to some limitations of the AzureKinect camera itself, the raw data of the wood-lotus point cloud will always contain some floating point noise. These floating points will accumulate gradually during the processing of the point cloud, which has a great influence on the calculation accuracy of the form parameters, so that the floating points are removed by the radius filtering method. This filtering method (radiausoutlierremoval method in PCL library) calculates the number (K) of neighboring points in each point radius r in the point cloud. And if the number of adjacent points of a certain point is less than the K value, removing the point as an abnormal point (the K value is determined according to different plant conditions). Experiments in the examples show that r =0.05m, k =20, and the filtering effect is the best. FIG. 2 is the plant point cloud after the experimental platform, the flowerpot and the soil are removed.
2.3 ) point clouds is registered by finding the best transformation matrix between each successive cloud and converging these transformation matrices into the coordinate system of the first point cloud. In order to obtain an initial position with higher precision, in denoised point cloud wood-charge images under every two adjacent angles, performing principal component feature extraction on the two current denoised point cloud wood-charge images by using a Principal Component Analysis (PCA) algorithm to obtain principal feature components, solving according to the principal axis direction and the principal feature components of the two current denoised point cloud wood-charge images to obtain a rotation transformation matrix, performing point cloud rough registration on the denoised point cloud wood-charge images under the two current adjacent angles according to the current rotation transformation matrix to obtain corresponding rough registration point cloud images, traversing and performing point cloud rough registration on the denoised point cloud wood-charge images under each group of two adjacent angles to obtain all rough registration point cloud wood-charge images; the initial position error (root mean square error) of the coarsely registered point cloud image is less than 0.005m.
2.4 Performing point cloud fine registration on all the roughly registered point cloud wood-lotus images by utilizing a KD-Tree accelerated ICP algorithm to obtain the precisely registered point cloud wood-lotus images. Fig. 3 shows the point cloud after registration in the embodiment.
3) Carrying out hierarchical clustering on the point cloud image subjected to fine registration by using a slice clustering method to obtain all clustering clusters of each layer of point cloud, and then carrying out plant skeleton extraction and optimization according to all clustering clusters of each layer of point cloud to obtain an undirected skeleton map;
3) The method specifically comprises the following steps:
3.1 Carrying out layering on the accurately registered point cloud wood-lotus image in the main shaft direction by utilizing a straight-through filtering function to obtain point clouds of all layers, and carrying out Euclidean clustering on the point clouds of all layers respectively to obtain all Euclidean clusters corresponding to the point clouds of all layers respectively;
3.2 The centroids of all Euclidean clusters are used as skeleton nodes of corresponding Euclidean clusters, each skeleton node comprises a leaf apex point, a middle node, a stem leaf connecting point and a root node, the leaf apex point has only one adjacent skeleton node, the middle node has only 2 adjacent skeleton nodes, the stem leaf connecting point has at least 2 adjacent skeleton nodes, and the coordinate value of the root node in the main axis direction is smaller than that of other skeleton nodes. Then connecting all framework nodes of each layer to form an initial wood-lotus framework diagram;
3.3 Some skeleton branches which do not accord with the growth trend of the leaves exist in the initial wood lotus skeleton diagram, therefore, branch pruning is carried out on the initial wood lotus skeleton diagram by using a branch pruning algorithm based on a quantity threshold value, branches with the number less than a preset node number are deleted, the maximum growth branch is 3 according to the structure of the wood lotus seedling, namely the number of adjacent skeleton nodes of the stem and leaf connection point should not exceed 3, and finally a continuous and smooth undirected wood lotus skeleton diagram is obtained.
4) Extracting a point cloud image of the stem of the schima superba according to the undirected schima superba skeleton diagram and all cluster clusters of point clouds of all layers;
4) The method specifically comprises the following steps:
4.1 Stem of the schima superba seedling is approximately vertical to the ground plane because not all the seedlings grow straight, in the undirected skeleton diagram of the plant, the branch from the bottom of the stem to the end of the canopy leaf is the longest branch in the whole undirected skeleton diagram, and the longest branch of the undirected schima superba skeleton diagram is identified by using the longest path search algorithm; the longest path searching algorithm starts from a root node of a plant skeleton, the searching path is along the branch direction of the minimum spanning tree of the stored skeleton map, all branches of the minimum spanning tree of the tree and skeleton nodes where the branches are included are traversed, and finally the longest branch of the minimum spanning tree is output, namely the longest branch of the undirected wood-lotus skeleton map.
4.2 Identified longest branch is comprised of the coronal lobe portion, and therefore the coronal lobe line needs to be trimmed off. Leaves of a canopy layer of the schima superba seedlings grow outwards from the tops of stems in an inclined mode, namely, the included angle between a stem line and the leaf line of the canopy layer is an obtuse angle, coordinate dot product values between each skeleton node and two adjacent skeleton nodes in the longest branch obtained through recognition are sequentially calculated, if the dot product values are negative numbers, the current skeleton node is the intersection point of the stem line and the canopy leaf line, namely the tail end of the stem line, the path from the starting point (namely, a root node) of the longest branch to the tail end of the stem line is the stem of the schima superba seedlings, and fig. 4 is the stem line extracted in the embodiment;
4.3 Extracting all Euclidean cluster clusters corresponding to point clouds of all layers based on each skeleton point cloud (namely the mass center of the cluster clusters) of the stem of the schima superba seedling, and integrating the point clouds corresponding to the extracted cluster clusters to obtain a schima superba stem point cloud image.
5) Carrying out blade segmentation on the point cloud arbor image subjected to precise registration by using the arbor stem point cloud image to obtain an initial blade point cloud image, and clustering the initial blade point cloud image by using a DBSCAN clustering method to obtain an arbor blade point cloud image;
in the embodiment, the DBSCAN clustering method has two parameters to be set, one of which is a neighborhood radius used for describing a neighborhood distance threshold of a current point; the second is the number of clustering points, which is used to describe the minimum number of data points in the neighborhood. The neighborhood radius is set to 0.01m, the number of clustering points is set to 50, and the effect is best. The effect of single leaf segmentation in the whole plant. FIG. 5 is a vane of an embodiment shown separated;
6) And calculating the phenotype parameters of the to-be-detected schima superba seedlings according to the schima superba stem point cloud image and the schima superba leaf point cloud image, wherein the phenotype parameters comprise plant height, stem length, stem diameter, growth direction, leaf length, leaf angle and the like, and are used for evaluating the growth and development conditions of the schima superba seedlings.
The height of a schima superba plant is defined as the vertical distance between the lowest point and the highest point of the plant. And determining the point with the minimum y value and the maximum y value according to the point cloud image of the stem of the schima superba and the point cloud image of the leaf of the schima superba, wherein the point with the minimum y value is the lowest point of the plant, the point with the maximum y value is the highest point of the plant, and the absolute difference value of the y values between the point cloud image of the stem of the schima superba and the point cloud image of the leaf of the schima superba is the plant height.
The length of the stem is defined as the length of the stem skeleton line. And respectively calculating Euclidean distances between two adjacent skeleton nodes of the stem according to the point cloud image of the stem of the schima superba, and accumulating all Euclidean distance values to obtain the stem length.
The phenotypic parameter of the growth direction of the stem of the woody lotus seedlings is defined as the angle between the stem of the plant and the ground. The parameter represents the vertical degree of the stem and the ground, the value range of the parameter should be (0 degrees and 90 degrees). For the calculation accuracy of the direction of the stem, a covariance matrix of a point cloud image of the stem of the wood lotus is calculated by utilizing a principal component analysis algorithm, the obtained first characteristic vector is the direction vector of the stem, and then the cosine of a vector included angle between the direction vector of the stem and the normal vector of the ground is calculated and is used as the growth direction of the stem of the wood lotus seedling.
Because the single leaf of the wood lotus is slender, the length of the leaf framework of the wood lotus can be approximate to the leaf length. And each single blade obtained in the blade segmentation step has a corresponding framework, the Euclidean distance between every two adjacent framework nodes in each blade corresponding to the blade framework in the point cloud image of the wood-lotus blade is calculated, and all Euclidean distance values are accumulated together to obtain the blade length of each single blade.
The leaf area of the single sheet of the schima superba leaf is defined as the sum of the areas of all curved surface meshes after the curved surface of the leaf is reconstructed.
And (3) carrying out blade point cloud curved surface reconstruction on the point cloud image of the leaf of the schima superba by utilizing a Delaunay triangulation algorithm. In the process of reconstructing the curved surface mesh, the number of neighborhoods which can be searched by sample points and the maximum distance between connection points need to be constrained, if the number of neighborhoods is more and the maximum distance is smaller, the reconstructed curved surface is more precise, and the calculated area value is closer to the true value. The number of neighborhoods set in this embodiment is 100, and the distance value is 0.07, so that a sufficient number of grids, that is, a high-precision area calculation value can be obtained.
After the curved surface reconstruction of the blade point cloud is completed, the vertex information of each triangle in the grid is obtained through an indexing method, the Euclidean convergence between adjacent vertexes is sequentially calculated to serve as the side length of the triangle, the area of each triangle is calculated through the following formula, and finally the areas of all the triangles are accumulated together to serve as the area of the whole blade.
Where C is the perimeter of a single triangle in the triangular mesh, S a 、S b 、S c Respectively, the lengths of three sides of a single triangle, and a is the area of the single triangle.
The method has the advantages that 25 samples are counted in the experimental process, the determining coefficient (R2) and the Root Mean Square Error (RMSE) are used as indexes, the plant height, the direction of a stem, the stem length, the leaf length and the area of a single leaf are calculated, the determining coefficient of the plant height is 0.939, and the RMSE is 1.944cm. The growth direction of the stem was determined by a coefficient of 0.908 and an RMSE of 5.3. Stem of a treeLength and leaf length, with coefficients of determination of 0.853 and 0.923, respectively, RMSE of 2.381cm and 1.242cm. The leaf area, coefficient of determination was 0.968, RMSE was 0.00048m 2 It is shown that the calculation of the phenotypic parameters based on the 3D point cloud image is very suitable for the measurement of the subject.
Claims (8)
1. A method for rapidly measuring phenotypic parameters of a schima superba seedling based on a 3D point cloud image is characterized by comprising the following steps:
1) Respectively acquiring color wood lotus images and corresponding depth wood lotus images of the wood lotus seedlings to be detected at all angles by using an RGB (red, green and blue) camera and a depth camera;
2) Preprocessing the color wood lotus images and the corresponding depth wood lotus images at all angles to obtain precisely registered point cloud wood lotus images;
3) Carrying out hierarchical clustering on the point cloud image subjected to fine registration by using a slice clustering method to obtain all clustering clusters of each layer of point cloud, and then carrying out plant skeleton extraction and optimization according to all clustering clusters of each layer of point cloud to obtain an undirected skeleton map;
4) Extracting a point cloud image of the stem of the schima superba according to the undirected schima superba skeleton diagram and all cluster clusters of point clouds of all layers;
5) Carrying out leaf segmentation on the point cloud image of the fine registration by using the point cloud image of the stem of the schima superba to obtain an initial leaf point cloud image, and then clustering the initial leaf point cloud image to obtain a schima superba leaf point cloud image;
6) And calculating the phenotype parameters of the to-be-detected schima superba seedlings according to the schima superba stem point cloud image and the schima superba leaf point cloud image.
2. The method for rapidly measuring the phenotypic parameters of the schima superba seedlings based on the 3D point cloud image according to claim 1, wherein the 2) is specifically as follows:
2.1 Respectively aligning the color wood lotus images and the corresponding depth wood lotus images at all angles to respectively obtain initial point cloud wood lotus images at all angles;
2.2 Respectively removing background and floating point noise in the initial point cloud wood-lotus image under each angle by using a filtering method to respectively obtain a denoising point cloud wood-lotus image under each angle;
2.3 In every two adjacent angles of the denoised point cloud wood-lotus images, performing principal component feature extraction on the current two denoised point cloud wood-lotus images by using a principal component analysis algorithm to obtain principal feature components, solving according to the principal axis direction and the principal feature components of the current two denoised point cloud wood-lotus images to obtain a rotation transformation matrix, performing point cloud rough registration on the denoised point cloud wood-lotus images under the current two adjacent angles according to the current rotation transformation matrix to obtain corresponding rough registration point cloud images, traversing and performing point cloud rough registration on the denoised point cloud wood-lotus images under each group of two adjacent angles to obtain all roughly registered point cloud wood-lotus images;
2.4 Performing point cloud fine registration on all the roughly registered point cloud wood-lotus images by utilizing a KD-Tree accelerated ICP (inductively coupled plasma) algorithm to obtain the finely registered point cloud wood-lotus images.
3. The method for rapidly measuring the phenotypic parameters of the schima superba seedlings based on the 3D point cloud image according to claim 1, wherein the 3) is specifically as follows:
3.1 Layering the precisely registered point cloud image in the main axis direction to obtain point clouds of all layers, and clustering the point clouds of all layers respectively to obtain all cluster clusters corresponding to the point clouds of all layers;
3.2 Using the centroid of each cluster as the skeleton node of the corresponding cluster, and then connecting all skeleton nodes of each layer to form an initial wood-lotus skeleton map;
3.3 Branch pruning is performed on the initial wood-lotus skeleton map by using a quantity threshold-based branch pruning algorithm to obtain an undirected wood-lotus skeleton map.
4. The method for rapidly measuring the phenotypic parameters of the schima superba seedlings based on the 3D point cloud image according to claim 1, wherein the 4) is specifically as follows:
4.1 Identifying a longest branch of the undirected wood-lotus skeleton map using a longest path search algorithm;
4.2 Sequentially calculating and identifying coordinate dot product values between each skeleton node and two adjacent skeleton nodes in the longest branch, wherein if the dot product values are negative numbers, the current skeleton node is the tail end of the stem line, and the path from the starting point of the longest branch to the tail end of the stem line is the stem of the schima superba seedling;
4.3 Extracting all cluster clusters corresponding to the point clouds of all layers based on the stem of the schima superba seedling, and integrating the point clouds corresponding to the extracted cluster clusters to obtain a schima superba stem point cloud image.
5. The method for rapidly measuring phenotypic parameters of schima superba seedlings based on 3D point cloud images according to claim 1, wherein the phenotypic parameters include plant height, stem length, stem diameter, growth direction and leaf length.
6. The method for rapidly measuring phenotypic parameters of schima superba seedlings based on 3D point cloud images according to claim 5, wherein the stem length is calculated as follows:
and respectively calculating Euclidean distances between two adjacent skeleton nodes according to the point cloud image of the stem of the schima superba, and taking the total Euclidean distance obtained by accumulating the Euclidean distances between all two adjacent skeleton nodes as the stem length.
7. The method for rapidly measuring the phenotypic parameters of the schima superba seedlings based on the 3D point cloud image according to claim 5, wherein the growth direction is calculated as follows:
and calculating a covariance matrix of the point cloud image of the stem of the schima superba by using a principal component analysis algorithm, obtaining a first characteristic vector as a direction vector of the stem, and calculating the cosine of a vector included angle between the direction vector of the stem and a ground normal vector and using the cosine as the growth direction of the stem of the schima superba.
8. The method for rapidly measuring phenotypic parameters of schima superba seedlings based on 3D point cloud images according to claim 5, wherein the leaf length is calculated as follows:
calculating the Euclidean distance between each two adjacent skeleton nodes in the blade skeleton corresponding to each blade in the point cloud image of the wood lotus blade, and taking the total Euclidean distance obtained by accumulating the Euclidean distances between each two adjacent skeleton nodes as the blade length of each single blade.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211184370.6A CN115631136A (en) | 2022-09-27 | 2022-09-27 | 3D point cloud image-based method for rapidly measuring phenotypic parameters of schima superba seedlings |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211184370.6A CN115631136A (en) | 2022-09-27 | 2022-09-27 | 3D point cloud image-based method for rapidly measuring phenotypic parameters of schima superba seedlings |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115631136A true CN115631136A (en) | 2023-01-20 |
Family
ID=84904223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211184370.6A Pending CN115631136A (en) | 2022-09-27 | 2022-09-27 | 3D point cloud image-based method for rapidly measuring phenotypic parameters of schima superba seedlings |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115631136A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710601A (en) * | 2023-12-27 | 2024-03-15 | 南京林业大学 | Single wood skeleton extraction method and system based on laser point cloud and image information |
-
2022
- 2022-09-27 CN CN202211184370.6A patent/CN115631136A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710601A (en) * | 2023-12-27 | 2024-03-15 | 南京林业大学 | Single wood skeleton extraction method and system based on laser point cloud and image information |
| CN117710601B (en) * | 2023-12-27 | 2024-05-24 | 南京林业大学 | Single wood skeleton extraction method and system based on laser point cloud and image information |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107832681B (en) | Forest single tree high estimation method combining LiDAR point cloud and synchronous remote sensing image | |
| CN111724433B (en) | Crop phenotype parameter extraction method and system based on multi-view vision | |
| CN106529469B (en) | Unmanned aerial vehicle-mounted LiDAR point cloud filtering method based on self-adaptive gradient | |
| CN109146948B (en) | Crop growth phenotype parameter quantification and yield correlation analysis method based on vision | |
| Liang et al. | Automatic plot-wise tree location mapping using single-scan terrestrial laser scanning | |
| Király et al. | Tree height estimation methods for terrestrial laser scanning in a forest reserve | |
| CN110988909B (en) | TLS-based vegetation coverage measuring method for sand vegetation in severe cold fragile area | |
| CN109002418B (en) | Tree breast-height diameter automatic calculation method based on voxel growth and ground laser point cloud | |
| CN109816664A (en) | A kind of three-dimensional point cloud dividing method and device | |
| CN114119863A (en) | Method for automatically extracting street tree target and forest attribute thereof based on vehicle-mounted laser radar data | |
| CN112200854A (en) | Leaf vegetable three-dimensional phenotype measurement method based on video image | |
| CN115393418A (en) | Method for automatically measuring phenotypic parameters of masson pine parts based on three-dimensional point cloud image | |
| CN115937151B (en) | Method for judging curling degree of crop leaves | |
| CN113780144A (en) | Crop plant number and stem width automatic extraction method based on 3D point cloud | |
| CN110853044A (en) | Potted corn point cloud rapid segmentation method based on conditional Euclidean clustering | |
| CN115631136A (en) | 3D point cloud image-based method for rapidly measuring phenotypic parameters of schima superba seedlings | |
| CN112906719A (en) | Standing tree factor measuring method based on consumption-level depth camera | |
| CN113570621A (en) | Tree information extraction method and device based on high-precision point cloud and image | |
| CN116523898A (en) | Tobacco phenotype character extraction method based on three-dimensional point cloud | |
| CN114136208A (en) | Low-cost tree structure automatic reconstruction method for lossless estimation of stumpage volume | |
| Song et al. | Dynamic detection of three-dimensional crop phenotypes based on a consumer-grade RGB-D camera | |
| CN112509142B (en) | Bean strain rapid three-dimensional reconstruction method based on phenotype-oriented precise identification | |
| CN114862872B (en) | Mangrove single wood segmentation method based on Faster R-CNN | |
| CN112381029B (en) | Method for extracting airborne LiDAR data building based on Euclidean distance | |
| CN115294482A (en) | Edible fungus yield estimation method based on unmanned aerial vehicle remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |