CN109410238B - Wolfberry identification and counting method based on PointNet + + network - Google Patents

Wolfberry identification and counting method based on PointNet + + network Download PDF

Info

Publication number
CN109410238B
CN109410238B CN201811098583.0A CN201811098583A CN109410238B CN 109410238 B CN109410238 B CN 109410238B CN 201811098583 A CN201811098583 A CN 201811098583A CN 109410238 B CN109410238 B CN 109410238B
Authority
CN
China
Prior art keywords
pointnet
point
layer
point cloud
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811098583.0A
Other languages
Chinese (zh)
Other versions
CN109410238A (en
Inventor
贾秀芳
王儒敬
李伟
谢成军
孙丙宇
黄河
王雪
李娇娥
徐玲玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Institutes of Physical Science of CAS
Original Assignee
Hefei Institutes of Physical Science of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei Institutes of Physical Science of CAS filed Critical Hefei Institutes of Physical Science of CAS
Priority to CN201811098583.0A priority Critical patent/CN109410238B/en
Publication of CN109410238A publication Critical patent/CN109410238A/en
Application granted granted Critical
Publication of CN109410238B publication Critical patent/CN109410238B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20152—Watershed segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30188—Vegetation; Agriculture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a wolfberry identification and counting method based on a PointNet + + network, which solves the problem that a two-dimensional image identification method is difficult to accurately identify due to shielding overlapping before picking of wolfberry fruits in the prior art. The invention comprises the following steps: collecting and preprocessing a PointNet + + network training sample; acquiring a PointNet + + model based on the PointNet + + network fusion context information; acquiring and preprocessing point cloud data to be identified; and identifying and counting the number of the Chinese wolfberry. The invention integrates a PointNet + + network based on context information and an optimal threshold watershed segmentation algorithm based on distance transformation, and realizes accurate segmentation and counting of the medlar.

Description

Wolfberry identification and counting method based on PointNet + + network
Technical Field
The invention relates to the technical field of image recognition, in particular to a medlar recognition counting method based on a PointNet + + network.
Background
In the planting process of the medlar, the region with low yield can be modified pertinently by accurately predicting the medlar yield, factors such as soil, variety or water content are adjusted, and resources such as manpower, material resources, storage and the like required in harvesting are reasonably arranged.
Currently, in fruit target identification and yield estimation, two-dimensional images are mostly used as data input for yield measurement. However, in practical application, because the medlar particles are small, and most of medlar fruits have the problems of shielding and overlapping before picking, the depth learning method based on the two-dimensional image is low in production efficiency and poor in robustness, and the accuracy of the method is difficult to guarantee.
Therefore, how to design a new identification and counting method to improve the accuracy of identification and counting of the medlar becomes a technical problem to be solved urgently.
Disclosure of Invention
The invention aims to solve the defect that the two-dimensional image recognition method is difficult to accurately recognize due to shielding overlapping before picking of medlar fruits in the prior art, and provides a medlar recognition and counting method based on a PointNet + + network to solve the problem.
In order to achieve the purpose, the technical scheme of the invention is as follows:
a wolfberry identification and counting method based on a PointNet + + network comprises the following steps:
11) collecting and preprocessing PointNet + + network training samples: acquiring 18 pictures and corresponding context information of each Chinese wolfberry tree, wherein the context information comprises time, space, temperature and phenological period information, constructing a three-dimensional model through the 18 pictures to obtain three-dimensional point cloud, and using the three-dimensional point cloud together with the context information as training data;
12) acquiring a PointNet + + model based on the PointNet + + network fusion context information:
setting a PointNet + + network, and setting a sampling layer, a grouping layer, a PointNet layer and a segmentation layer of the PointNet + + network; training a PointNet + + network, introducing time, space, temperature and phenological period context information when point cloud data are obtained as characteristic data, and training a PointNet + + model for segmenting the three-dimensional point cloud data of the Chinese wolfberry;
13) acquiring and preprocessing point cloud data to be identified: acquiring 18 pictures of a Chinese wolfberry tree to be identified and corresponding time, space, temperature and phenological period context information of the pictures, constructing a three-dimensional model and acquiring a three-dimensional point cloud to be identified;
14) identification and counting of the number of the medlar:
inputting the three-dimensional point cloud to be identified and the contextual information as the characteristics into a PointNet + + model to obtain the score of the medlar target, and counting the number of the medlar by a marking method.
The method for setting the PointNet + + network comprises the following steps:
21) setting a PointNet + + network to sequentially comprise a sampling layer, a grouping layer and a hierarchy combination of PointNet layers;
22) setting a sampling layer as a selection center;
obtaining central points by using a k-means + + clustering method, wherein the number of the central points in the first layer is m;
X＝{x1,x2,......,xnis the input set of labeled points,
wherein x isi∈Rd+CD is a point cloud dimension, and C is a characteristic dimension;
if x is taken randomlyk∈Rd+CThe first cluster center is then followed by centering
Figure GDA0001894222710000021
The probability reaches a maximum value, where D (x)j) Is xjDistance from the nearest one of the cluster centers;
23) setting a grouping layer, wherein the grouping layer is used for creating a plurality of sub-point clouds by adjacent points in a given radius;
231) obtaining a given radius;
232) searching for fixed adjacent points by using a kNN algorithm and taking the clustering center as a reference, wherein the radiuses are respectively 0.1, 0.2 and 0.4, and the corresponding maximum points in the circle are 16, 32 and 128;
the input point set of the first layer is n x (d + C), the output point set is n' × kx (d + C), wherein k is the number of points contained in the neighborhood;
24) setting a PointNet layer, wherein the PointNet layer is used for obtaining higher dimensional representation of the sub-point cloud;
setting a point set n ' × kx (d + C) as the input of a PointNet + + network, performing feature extraction on the point set, and finally outputting n ' × (d + C ');
repeating the steps 22), 23) and 24) three times, i.e. combining 3 network layers1、l2、l3A module for continuously extracting the features;
25) setting a segmentation layer, and obtaining the score of each original point by adopting an interpolation and backtracking mode based on context information;
251) input l3Point set of layers n '× (d + C');
252) for l2Point of layer consisting of3After interpolation of the layer points, carrying out convolution of 1 x 1 with the corresponding characteristics obtained in 24) to finally obtain l2The values of the layer points are traced all the time to finally obtain l1A score of the origin;
253) while interpolation is carried out, context information of input corresponding points is taken as characteristic data and introduced into the Unit PointNet + + network of the last layer;
Figure GDA0001894222710000031
for all the context information fused, the first layer input of the multi-layer perceptron is
H1＝σ(W1xi+UD+b1)，
The rest layers are Hm＝σ(WmHm-1+bm) Wherein M ∈ {2,..., M };
254) acquisition Point (x)p,yp,zp)∈RCProbability P of belonging to target medlar class or background classc'(xp,yp,zp) It is expressed as follows:
Figure GDA0001894222710000032
wherein c is { target matrimony vine class, background class },
Figure GDA0001894222710000033
for the weights of the last layer belonging to class c, all parameters (W, U, b) are learned by minimizing the cross entropy loss function.
The PointNet + + network training method comprises the following steps:
31) inputting the three-dimensional point cloud into a sampling layer to obtain a central point of the three-dimensional point cloud;
32) inputting the central point of the three-dimensional point cloud into a grouping layer to obtain a plurality of sub-point clouds in the radius of the central point of the three-dimensional point cloud;
33) inputting a plurality of sub-point clouds into a PointNet + + network, and inputting corresponding characteristic data into a multilayer perceptron of the PointNet + + network for characteristic extraction;
34) repeating step 31), step 32) and step 33) three times in this order, i.e. 3 modules are assembled with such a network layer;
35) and obtaining the probability of the original point serving as the medlar or the background by adopting an interpolation and backtracking mode.
The identification and counting of the number of the medlar also comprises the step of accurately segmenting point cloud: on the basis of distance conversion, a point cloud is segmented by using an optimal threshold watershed segmentation algorithm, and the number of the Chinese wolfberries is counted by using a marking method;
the accurate segmentation of the point cloud comprises the following steps:
41) reconstructing the distance point cloud to distinguish the boundary points and the internal points of the medlar;
setting N (P) as an eight neighborhood point set, P (x)p,yp,zp) Representing the gray value of the point P, P' (x)p,yp,zp) Representing the probability, P ″ (x), that point P belongs to the target Lycium classp,yp,zp) A value representing the distance of the point p from its nearest background point;
reconstructing a point cloud of distances, an order
Figure GDA0001894222710000041
Wherein the content of the first and second substances,
Figure GDA0001894222710000042
| represents the euclidean distance between points p and q;
if distmin＜P”(xp,yp,zp) Let P "(x)p,yp,zp)＝distminThe distance point cloud constructed at this time has a value of P ″ (x)p,yp,zp) A set of (a);
42) establishing a watershed segmentation algorithm based on an optimal threshold;
421) set of distances P "(x) to reconstructp,yp,zp) The distance value of the internal points of the Chinese wolfberry is large, the distance value corresponding to the boundary point is small, and the distance value of the background point is zero; finding out local maximum value points on all distances and endowing the local maximum value points with unique identification;
422) adopting a descending method to descend on plain, descending all points until meeting the marked area, and adding the points into the marked area;
423) combining the areas, and combining the areas with the depth lower than a threshold value;
424) determining a watershed segmentation result;
43) and (4) a target mark counting method is adopted to obtain a segmentation result in the final point cloud, and the number is counted to obtain the number of the actual medlar in the point cloud.
The area merging comprises the following steps:
51) for each region, finding out an inner curvature minimum point, a region adjacent to the region and a minimum point on the boundary of each adjacent region;
52) calculating the watershed depth of each region;
53) calculating an optimal threshold value;
let the point cloud have L reconstructed distance values, the point with distance value i has niNumber of individuals and total pointsIs N, the probability of each distance value occurring is
Figure GDA0001894222710000051
And is
Figure GDA0001894222710000052
The threshold value is set as t, the point cloud is divided into 2 parts, namely a background class a {0, 1, 2,. and t } and a medlar target class B { t +1, t +2,. and L-1} based on the distance value, and the probability of occurrence of the two classes is respectively:
Figure GDA0001894222710000053
the distance mean values of the two types A and B are respectively
Figure GDA0001894222710000054
The total distance mean in the point cloud is:
Figure GDA0001894222710000055
from this, the between-class variance of the two parts A and B is obtained
Figure GDA0001894222710000056
The larger the inter-class variance is, the larger the difference of the distance values between the two classes is, that is, the optimal threshold is determined as follows:
Figure GDA0001894222710000061
54) when the depth is lower than the threshold value, combining the region and the region adjacent to the boundary curvature minimum value point and updating the related information;
55) repeating 54) until the depth of all the areas is greater than a given threshold.
Advantageous effects
Compared with the prior art, the wolfberry identification and counting method based on the PointNet + + network integrates the PointNet + + network based on context information and the optimal threshold watershed segmentation algorithm based on distance transformation, and realizes accurate segmentation and counting of the wolfberry.
The method is based on three-dimensional point cloud, utilizes a PointNet + + network based on context information and an improved watershed segmentation algorithm to carry out accurate segmentation on the medlar, overcomes the defect that two-dimensional image processing and recognition technology is simply utilized to carry out prediction, and can accurately predict the yield of the medlar in the planting base.
The invention not only considers relevant context information (time, space, temperature and phenological period), but also provides an optimal watershed segmentation algorithm improved based on a reconstruction distance to solve the problem of accurate segmentation of the medlar aiming at the problem of inaccuracy of segmentation of overlapped and shielded medlar by using a PointNet + + network, thereby improving the medlar detection capability of a medlar planting base under a complex natural condition and improving the accuracy of medlar yield prediction.
Drawings
FIG. 1 is a sequence diagram of the method of the present invention.
Detailed Description
In order to further understand and appreciate the structural features and advantages achieved by the present invention, a preferred embodiment is described in detail with reference to the accompanying drawings, in which:
as shown in fig. 1, the method for identifying and counting medlar based on PointNet + + network according to the present invention includes the following steps:
the first step, collecting and preprocessing a PointNet + + network training sample.
The method comprises the steps of obtaining 18 pictures of each Chinese wolfberry tree and corresponding context information, wherein the context information comprises time, space, temperature and phenological period information, constructing a three-dimensional model through the 18 pictures according to the existing method to obtain three-dimensional point cloud, and using the three-dimensional point cloud together with the context information as training data.
And secondly, acquiring a PointNet + + model based on the PointNet + + network fusion context information.
First, a PointNet + + network is set, and a sampling layer, a grouping layer, a PointNet layer, and a splitting layer of the PointNet + + network are set. Secondly, training a PointNet + + network, introducing time, space, temperature and phenological period context information when the point cloud data are obtained as characteristic data, and training a PointNet + + model for segmenting the three-dimensional point cloud data of the Chinese wolfberry.
The specific steps for setting the PointNet + + network are as follows:
(1) the method is characterized in that a sampling layer, a grouping layer, a PointNet layer and a segmentation layer which are sequentially included in a PointNet + + network are set, and the hierarchical combination relationship of the sampling layer, the grouping layer and the PointNet layer of the PointNet + + network is set firstly.
(2) Setting a sampling layer as a selection center;
obtaining central points by using a k-means + + clustering method, wherein the number of the central points in the first layer is m;
X＝{x1,x2,......,xnis the input set of labeled points,
wherein x isi∈Rd+CD is a point cloud dimension, and C is a characteristic dimension;
if x is taken randomlyk∈Rd+CThe first cluster center is then followed by centering
Figure GDA0001894222710000071
The probability reaches a maximum value, where D (x)j) Is xjDistance from the nearest one of the cluster centers.
(3) Setting a grouping layer, wherein the grouping layer is used for creating a plurality of sub-point clouds by adjacent points in a given radius;
A1) obtaining a given radius;
A2) searching for fixed adjacent points by using a kNN algorithm and taking the clustering center as a reference, wherein the radiuses are respectively 0.1, 0.2 and 0.4, and the corresponding maximum points in the circle are 16, 32 and 128;
the input point set of the first layer is n x (d + C), the output point set is n' × kx (d + C), wherein k is the number of points contained in the neighborhood;
(4) setting a PointNet layer, wherein the PointNet layer is used for obtaining higher dimensional representation of the sub-point cloud;
setting a point set n ' × kx (d + C) as the input of a PointNet + + network, performing feature extraction on the point set, and finally outputting n ' × (d + C ');
repeating the step (2) (setting sampling layer), the step (3) (setting grouping layer) and the step (4) (setting PointNet layer) three times, namely combining the network layers into 3 l1、l2、l3And the module is used for continuously extracting the features.
(5) Setting a segmentation layer, and obtaining the score of each original point by adopting an interpolation and backtracking mode based on context information;
B1) input l3Point set of layers n '× (d + C');
B2) for l2Point of layer consisting of3After interpolation of the layer points, carrying out convolution of 1 x 1 with the corresponding characteristics obtained in 24) to finally obtain l2The values of the layer points are traced all the time to finally obtain l1A score of the origin;
B3) while interpolation is carried out, context information of input corresponding points is taken as characteristic data and introduced into the Unit PointNet + + network of the last layer;
Figure GDA0001894222710000081
for all the context information fused, the first layer input of the multi-layer perceptron is
H1＝σ(W1xi+UD+b1)，
The rest layers are Hm＝σ(WmHm-1+bm) Wherein M ∈ {2,..., M };
B4) acquisition Point (x)p,yp,zp)∈RCProbability P of belonging to target medlar class or background classc'(xp,yp,zp) It is expressed as follows:
Figure GDA0001894222710000082
wherein, c ═ target lycium barbarumThe background class },
Figure GDA0001894222710000083
for the weights of the last layer belonging to class c, all parameters (W, U, b) are learned by minimizing the cross entropy loss function.
Secondly, training the PointNet + + network comprises the following steps:
(1) and inputting the three-dimensional point cloud into a sampling layer to obtain a central point of the three-dimensional point cloud.
(2) And inputting the central point of the three-dimensional point cloud into a grouping layer to obtain a plurality of sub-point clouds in the radius of the central point of the three-dimensional point cloud.
(3) And inputting a plurality of sub-point clouds into a PointNet + + network, and inputting corresponding characteristic data into a multilayer perceptron of the PointNet + + network for characteristic extraction.
(4) And (3) repeating the steps (1), (2) and (3) in the step of training the PointNet + + network three times, namely combining 3 modules by using the network layer.
(5) And obtaining the probability of the original point serving as the medlar or the background by adopting an interpolation and backtracking mode.
And thirdly, acquiring and preprocessing point cloud data to be identified.
Acquiring 18 pictures of the Chinese wolfberry tree to be identified and corresponding time, space, temperature and phenological period context information of the pictures, constructing a three-dimensional model and acquiring a three-dimensional point cloud to be identified.
And fourthly, identifying and counting the number of the Chinese wolfberry.
Inputting the three-dimensional point cloud to be identified and the contextual information as the characteristics into a PointNet + + model to obtain the score of the medlar target, and counting the number of the medlar by a marking method.
The number of the medlar can be obtained, and counting identification of the medlar is realized. However, in practical applications, although the analysis is performed on the three-dimensional point cloud data according to the characteristic that the distribution of the medlar is easy to overlap, a certain error still occurs. In order to further effectively distinguish the sticky and blocked wolfberries, the scores of the wolfberries target obtained in the fourth step of identifying and counting the number of the wolfberries are utilized, namely, the points in the 3D point cloud with the scores are precisely segmented by utilizing an improved watershed segmentation algorithm. The improved watershed segmentation algorithm is utilized, weak edges generated due to small pixel difference can be processed, the edges are accurately positioned, and even a single Chinese wolfberry in a cluster can be effectively distinguished.
And fifthly, performing accurate segmentation on the point cloud.
On the basis of distance conversion, the point cloud is segmented by using an optimal threshold watershed segmentation algorithm, and the number of the Chinese wolfberries is counted by using a marking method. The method specifically comprises the following steps:
(1) reconstructing the distance point cloud to distinguish the boundary points and the internal points of the medlar;
setting N (P) as an eight neighborhood point set, P (x)p,yp,zp) Representing the gray value of the point P, P' (x)p,yp,zp) Representing the probability, P ″ (x), that point P belongs to the target Lycium classp,yp,zp) A value representing the distance of the point p from its nearest background point;
reconstructing a point cloud of distances, an order
Figure GDA0001894222710000101
Wherein the content of the first and second substances,
Figure GDA0001894222710000102
| represents the euclidean distance between points p and q;
if distmin＜P”(xp,yp,zp) Let P "(x)p,yp,zp)＝distminThe distance point cloud constructed at this time has a value of P ″ (x)p,yp,zp) A collection of (a).
(2) And establishing a watershed segmentation algorithm based on an optimal threshold value. In the method, the selection of the extreme points in the watershed segmentation algorithm is updated, so that the result of PointNet + + network analysis can be utilized, the calculated amount is reduced, the efficiency is improved, and both large data processing and accurate segmentation can be realized. The method comprises the following specific steps:
C1) set of distances P "(x) to reconstructp,yp,zp) And as input, the distance value of the internal points of the Chinese wolfberry is larger, the distance value corresponding to the boundary point is smaller, and the distance value of the background point is zero. Finding out local maximum value points on all distances and endowing the local maximum value points with unique identification;
C2) adopting a descending method to descend on plain, descending all points until meeting the marked area, and adding the points into the marked area;
C3) combining the areas, and combining the areas with the depth lower than a threshold value;
C4) and determining a watershed segmentation result.
Here, the regions are subjected to the merging step individually as follows:
C31) for each region, finding out an inner curvature minimum point, a region adjacent to the region and a minimum point on the boundary of each adjacent region;
C32) calculating the watershed depth of each region;
C33) calculating an optimal threshold value;
let the point cloud have L reconstructed distance values, the point with distance value i has niN, the total number of points, and the probability of each distance value appearing is
Figure GDA0001894222710000111
And is
Figure GDA0001894222710000112
The threshold value is set as t, the point cloud is divided into 2 parts, namely a background class a {0, 1, 2,. and t } and a medlar target class B { t +1, t +2,. and L-1} based on the distance value, and the probability of occurrence of the two classes is respectively:
Figure GDA0001894222710000113
the distance mean values of the two types A and B are respectively
Figure GDA0001894222710000114
The total distance mean in the point cloud is:
Figure GDA0001894222710000115
from this, the between-class variance of the two parts A and B is obtained
Figure GDA0001894222710000116
The larger the inter-class variance is, the larger the difference of the distance values between the two classes is, that is, the optimal threshold is determined as follows:
Figure GDA0001894222710000117
C34) when the depth is lower than the threshold value, combining the region and the region adjacent to the boundary curvature minimum value point and updating the related information;
C35) repeating the step C34) until the depths of all the areas are greater than the given threshold.
(3) And (4) a target mark counting method is adopted to obtain a segmentation result in the final point cloud, and the number is counted to obtain the number of the actual medlar in the point cloud.
The segmentation result in the final point cloud can be obtained by using the segmentation method, and in practical application, the segmentation result is each basin (each medlar target object), and the actual number of the medlar in the point cloud can be accurately obtained only by counting the number of the basins.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are merely illustrative of the principles of the invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (3)

1. A wolfberry identification and counting method based on a PointNet + + network is characterized by comprising the following steps:
11) collecting and preprocessing PointNet + + network training samples: acquiring 18 pictures and corresponding context information of each Chinese wolfberry tree, wherein the context information comprises time, space, temperature and phenological period information, constructing a three-dimensional model through the 18 pictures to obtain three-dimensional point cloud, and using the three-dimensional point cloud together with the context information as training data;
12) acquiring a PointNet + + model based on the PointNet + + network fusion context information:
setting a PointNet + + network, and setting a sampling layer, a grouping layer, a PointNet layer and a segmentation layer of the PointNet + + network; training a PointNet + + network, introducing time, space, temperature and phenological period context information when point cloud data are obtained as characteristic data, and training a PointNet + + model for segmenting the three-dimensional point cloud data of the Chinese wolfberry;
the method for setting the PointNet + + network comprises the following steps:
121) setting a PointNet + + network to sequentially comprise a sampling layer, a grouping layer and a hierarchy combination of PointNet layers;
122) setting a sampling layer as a selection center;
obtaining central points by using a k-means + + clustering method, wherein the number of the central points in the first layer is m;
X＝{x1,x2,......,xnis the input set of labeled points,
wherein x isi∈Rd+CD is a point cloud dimension, and C is a characteristic dimension;
if x is taken randomlyk∈Rd+CThe first cluster center is then followed by centering
Figure FDA0003219794520000011
The probability reaches a maximum value, where D (x)j) Is xjDistance from the nearest one of the cluster centers;
123) setting a grouping layer, wherein the grouping layer is used for creating a plurality of sub-point clouds by adjacent points in a given radius;
1231) obtaining a given radius;
1232) searching for fixed adjacent points by using a kNN algorithm and taking the clustering center as a reference, wherein the radiuses are respectively 0.1, 0.2 and 0.4, and the corresponding maximum points in the circle are 16, 32 and 128;
the input point set of the first layer is n x (d + C), the output point set is n' × kx (d + C), wherein k is the number of points contained in the neighborhood;
124) setting a PointNet layer, wherein the PointNet layer is used for obtaining higher dimensional representation of the sub-point cloud;
setting a point set n ' × kx (d + C) as the input of a PointNet + + network, performing feature extraction on the point set, and finally outputting n ' × (d + C ');
repeating the steps 122), 123), 124) three times in this way, i.e. 3 l of such network layer combinations1、l2、l3A module for continuously extracting the features;
125) setting a segmentation layer, and obtaining the score of each original point by adopting an interpolation and backtracking mode based on context information;
1251) input l3Point set of layers n '× (d + C');
1252) for l2Point of layer consisting of3After interpolation of the layer points, carrying out convolution of 1 x 1 with the corresponding characteristics obtained in 124) to finally obtain l2The values of the layer points are traced all the time to finally obtain l1A score of the origin;
1253) while interpolation is carried out, context information of input corresponding points is taken as characteristic data and introduced into the Unit PointNet + + network of the last layer;
Figure FDA0003219794520000021
for all the context information fused, the first layer input of the multi-layer perceptron is
H1＝σ(W1xi+UD+b1)，
The rest layers are Hm＝σ(WmHm-1+bm) Wherein M ∈ {2,..., M };
1254) acquisition Point (x)p,yp,zp)∈RCProbability P 'of belonging to target matrimony vine or background'c(xp,yp,zp) It is expressed as follows:
Figure FDA0003219794520000031
wherein c is { target matrimony vine class, background class },
Figure FDA0003219794520000032
for the weights of the last layer belonging to class c, all parameters (W, U, b) are learned by minimizing the cross entropy loss function;
the PointNet + + network training method comprises the following steps:
131) inputting the three-dimensional point cloud into a sampling layer to obtain a central point of the three-dimensional point cloud;
132) inputting the central point of the three-dimensional point cloud into a grouping layer to obtain a plurality of sub-point clouds in the radius of the central point of the three-dimensional point cloud;
133) inputting a plurality of sub-point clouds into a PointNet + + network, and inputting corresponding characteristic data into a multilayer perceptron of the PointNet + + network for characteristic extraction;
134) repeating step 131), step 132), step 133) three times in this order, i.e. 3 modules are assembled with such a network layer;
135) obtaining the probability of the original point as the medlar or the background by adopting an interpolation and backtracking mode;
13) acquiring and preprocessing point cloud data to be identified: acquiring 18 pictures of a Chinese wolfberry tree to be identified and corresponding time, space, temperature and phenological period context information of the pictures, constructing a three-dimensional model and acquiring a three-dimensional point cloud to be identified;
14) identification and counting of the number of the medlar: inputting the three-dimensional point cloud to be identified and the contextual information as the characteristics into a PointNet + + model to obtain the score of the medlar target, and counting the number of the medlar by a marking method.
2. The method according to claim 1, wherein the identification and counting of the number of the lycium barbarum comprises the following steps of performing accurate segmentation on the point cloud: on the basis of distance conversion, a point cloud is segmented by using an optimal threshold watershed segmentation algorithm, and the number of the Chinese wolfberries is counted by using a marking method;
the accurate segmentation of the point cloud comprises the following steps:
21) reconstructing the distance point cloud to distinguish the boundary points and the internal points of the medlar;
setting N (P) as an eight neighborhood point set, P (x)p,yp,zp) Representing the gray value of the point P, P' (x)p,yp,zp) Representing the probability, P ″ (x), that point P belongs to the target Lycium classp,yp,zp) A value representing the distance of the point p from its nearest background point;
reconstructing a point cloud of distances, an order
Figure FDA0003219794520000041
Wherein the content of the first and second substances,
Figure FDA0003219794520000042
| represents the euclidean distance between points p and q;
if distmin<P”(xp,yp,zp) Let P "(x)p,yp,zp)＝distminThe distance point cloud constructed at this time has a value of P ″ (x)p,yp,zp) A set of (a);
22) establishing a watershed segmentation algorithm based on an optimal threshold;
221) set of distances P "(x) to reconstructp,yp,zp) The distance value of the internal points of the Chinese wolfberry is large, the distance value corresponding to the boundary point is small, and the distance value of the background point is zero; finding out all distancesThe local maximum value point on the distance is given with a unique identifier;
222) adopting a descending method to descend on plain, descending all points until meeting the marked area, and adding the points into the marked area;
223) combining the areas, and combining the areas with the depth lower than a threshold value;
224) determining a watershed segmentation result;
23) and (4) a target mark counting method is adopted to obtain a segmentation result in the final point cloud, and the number is counted to obtain the number of the actual medlar in the point cloud.
3. The method according to claim 2, wherein the region merging comprises the following steps:
31) for each region, finding out an inner curvature minimum point, a region adjacent to the region and a minimum point on the boundary of each adjacent region;
32) calculating the watershed depth of each region;
33) calculating an optimal threshold value;
let the point cloud have L reconstructed distance values, the point with distance value i has niN, the total number of points, and the probability of each distance value appearing is
Figure FDA0003219794520000051
And is
Figure FDA0003219794520000052
The threshold value is set as t, the point cloud is divided into 2 parts, namely a background class a {0, 1, 2,. and t } and a medlar target class B { t +1, t +2,. and L-1} based on the distance value, and the probability of occurrence of the two classes is respectively:
Figure FDA0003219794520000053
the distance mean values of the two types A and B are respectively
Figure FDA0003219794520000054
The total distance mean in the point cloud is:
Figure FDA0003219794520000055
from this, the between-class variance of the two parts A and B is obtained
Figure FDA0003219794520000056
The larger the inter-class variance is, the larger the difference of the distance values between the two classes is, that is, the optimal threshold is determined as follows:
Figure FDA0003219794520000057
34) when the depth is lower than the threshold value, combining the region and the region adjacent to the boundary curvature minimum value point and updating the related information;
35) repeating 34) until the depth of all the areas is greater than a given threshold.
CN201811098583.0A 2018-09-20 2018-09-20 Wolfberry identification and counting method based on PointNet + + network Active CN109410238B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811098583.0A CN109410238B (en) 2018-09-20 2018-09-20 Wolfberry identification and counting method based on PointNet + + network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811098583.0A CN109410238B (en) 2018-09-20 2018-09-20 Wolfberry identification and counting method based on PointNet + + network

Publications (2)

Publication Number Publication Date
CN109410238A CN109410238A (en) 2019-03-01
CN109410238B true CN109410238B (en) 2021-10-26

Family

ID=65465779

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811098583.0A Active CN109410238B (en) 2018-09-20 2018-09-20 Wolfberry identification and counting method based on PointNet + + network

Country Status (1)

Country Link
CN (1) CN109410238B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109993748B (en) * 2019-03-30 2023-06-20 华南理工大学 Three-dimensional grid object segmentation method based on point cloud processing network
CN110032962B (en) * 2019-04-03 2022-07-08 腾讯科技（深圳）有限公司 Object detection method, device, network equipment and storage medium
CN111784699A (en) * 2019-04-03 2020-10-16 Tcl集团股份有限公司 Method and device for carrying out target segmentation on three-dimensional point cloud data and terminal equipment
CN110276266B (en) * 2019-05-28 2021-09-10 暗物智能科技(广州)有限公司 Rotation-based point cloud data processing method and device and terminal equipment
CN110210431B (en) * 2019-06-06 2021-05-11 上海黑塞智能科技有限公司 Point cloud semantic labeling and optimization-based point cloud classification method
CN111476843B (en) * 2020-05-08 2023-03-24 中国科学院合肥物质科学研究院 Chinese wolfberry branch recognition and positioning method based on attention mechanism and improved PV-RCNN network
CN111860668B (en) * 2020-07-27 2024-04-02 辽宁工程技术大学 Point cloud identification method for depth convolution network of original 3D point cloud processing
CN112287992A (en) * 2020-10-26 2021-01-29 广东博智林机器人有限公司 Reinforcing steel bar cluster classification method and device, electronic equipment and storage medium
CN114973006B (en) * 2022-08-02 2022-10-18 四川省机械研究设计院(集团)有限公司 Method, device and system for picking Chinese prickly ash and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740798A (en) * 2016-01-27 2016-07-06 西安理工大学 Structure analysis based identification method for object in point cloud scene
CN106919901A (en) * 2017-01-20 2017-07-04 江苏大学 A kind of depth ball transversal method of spherefruit identification in strain
CN107679562A (en) * 2017-09-20 2018-02-09 北京航空航天大学 The analyzing and processing method and device of threedimensional model
CN108182704A (en) * 2016-12-08 2018-06-19 广州映博智能科技有限公司 Localization method based on Shape context feature
CN108196535A (en) * 2017-12-12 2018-06-22 清华大学苏州汽车研究院（吴江） Automated driving system based on enhancing study and Multi-sensor Fusion
CN108319959A (en) * 2017-12-29 2018-07-24 中国科学院合肥物质科学研究院 A kind of corps diseases image-recognizing method compressed based on characteristics of image with retrieval
CN108389251A (en) * 2018-03-21 2018-08-10 南京大学 The full convolutional network threedimensional model dividing method of projection based on fusion various visual angles feature

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040013749A1 (en) * 2002-07-22 2004-01-22 Young D. Gary Antioxidant and immune boosting composition and methods of using

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740798A (en) * 2016-01-27 2016-07-06 西安理工大学 Structure analysis based identification method for object in point cloud scene
CN108182704A (en) * 2016-12-08 2018-06-19 广州映博智能科技有限公司 Localization method based on Shape context feature
CN106919901A (en) * 2017-01-20 2017-07-04 江苏大学 A kind of depth ball transversal method of spherefruit identification in strain
CN107679562A (en) * 2017-09-20 2018-02-09 北京航空航天大学 The analyzing and processing method and device of threedimensional model
CN108196535A (en) * 2017-12-12 2018-06-22 清华大学苏州汽车研究院（吴江） Automated driving system based on enhancing study and Multi-sensor Fusion
CN108319959A (en) * 2017-12-29 2018-07-24 中国科学院合肥物质科学研究院 A kind of corps diseases image-recognizing method compressed based on characteristics of image with retrieval
CN108389251A (en) * 2018-03-21 2018-08-10 南京大学 The full convolutional network threedimensional model dividing method of projection based on fusion various visual angles feature

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
《PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space》;Charles R. Qi等;《31st Conference on Neural Information Processing Systems (NIPS 2017)》;20171231;第1-10页 *
《基于WebGIS的农场生产管理信息系统的设计与实现》;魏圆圆 等;《农业工程学报》;20180822;第34卷(第16期);第139-147页 *
《枸杞分级自动计数系统的设计》;李好博;《工业控制计算机》;20080925;第09卷;第87-88页 *

Also Published As

Publication number Publication date
CN109410238A (en) 2019-03-01

Similar Documents

Publication Publication Date Title
CN109410238B (en) Wolfberry identification and counting method based on PointNet + + network
CN110956185B (en) Method for detecting image salient object
CN112418117B (en) Small target detection method based on unmanned aerial vehicle image
CN111259786B (en) Pedestrian re-identification method based on synchronous enhancement of appearance and motion information of video
Peng et al. Automatic image segmentation by dynamic region merging
CN109671102B (en) Comprehensive target tracking method based on depth feature fusion convolutional neural network
CN107145862B (en) Multi-feature matching multi-target tracking method based on Hough forest
CN109635726B (en) Landslide identification method based on combination of symmetric deep network and multi-scale pooling
CN112633382A (en) Mutual-neighbor-based few-sample image classification method and system
CN113157678B (en) Multi-source heterogeneous data association method
CN108171119B (en) SAR image change detection method based on residual error network
Lovell et al. Performance evaluation of indirect immunofluorescence image analysis systems
CN111968124B (en) Shoulder musculoskeletal ultrasonic structure segmentation method based on semi-supervised semantic segmentation
CN114283162A (en) Real scene image segmentation method based on contrast self-supervision learning
Huynh et al. Plant identification using new architecture convolutional neural networks combine with replacing the red of color channel image by vein morphology leaf
Ge et al. Coarse-to-fine foraminifera image segmentation through 3D and deep features
CN115731257A (en) Leaf form information extraction method based on image
CN114782948A (en) Global interpretation method and system for cervical liquid-based cytology smear
CN115019163A (en) City factor identification method based on multi-source big data
Liu et al. Contour-oriented cropland extraction from high resolution remote sensing imagery using richer convolution features network
CN112560925A (en) Complex scene target detection data set construction method and system
CN113723833B (en) Method, system, terminal equipment and storage medium for evaluating quality of forestation actual results
CN114862883A (en) Target edge extraction method, image segmentation method and system
Zanwar et al. A comprehensive survey on soft computing based optical character recognition techniques
CN115205853B (en) Image-based citrus fruit detection and identification method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant