CN114677674A - Apple rapid identification and positioning method based on binocular point cloud - Google Patents
Apple rapid identification and positioning method based on binocular point cloud Download PDFInfo
- Publication number
- CN114677674A CN114677674A CN202210398218.1A CN202210398218A CN114677674A CN 114677674 A CN114677674 A CN 114677674A CN 202210398218 A CN202210398218 A CN 202210398218A CN 114677674 A CN114677674 A CN 114677674A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- clusters
- apple
- color
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a binocular point cloud-based apple rapid identification and positioning method, which comprises the following steps: s1: and using a binocular camera to acquire the spatial three-dimensional information and the color information of the fruit tree to generate an original color point cloud. S2: and preprocessing the original color point cloud to obtain a simplified point cloud. S3: and carrying out region growing segmentation on the simplified point cloud based on colors, obtaining point cloud clusters of different colors, and storing the clusters in an index. S4: and traversing the clusters in the index by using an iterative method, screening the clusters by using colors, and classifying labels to obtain the apple point cloud cluster. S5: and (4) carrying out pose extraction on the apple clusters, and packaging and outputting the obtained information.
Description
Technical Field
The invention relates to the field of agricultural robots and machine vision, in particular to a point cloud-based apple rapid identification and positioning method.
Background
China is the biggest apple producing country in the world, apples need to use a large amount of manual work in the picking period, so that picking operation is one of the most time-consuming and labor-consuming links, in recent years, a picking robot based on machine vision becomes a research focus of agricultural engineering at home and abroad, the aim is to realize automatic fruit picking through an intelligent robot technology, and in the picking process of agricultural robots, identification and positioning of apples on trees are always key technical problems, so that how to improve the accuracy and efficiency of apple image detection becomes the key point of current research.
At present, the convolutional neural network is widely applied to target detection, but the method has high requirements on hardware and long calculation period, so that the picking efficiency of the agricultural robot is greatly reduced.
Disclosure of Invention
In order to reduce the requirements on hardware and improve the picking efficiency of an agricultural robot, the application discloses a rapid apple identification and positioning method based on binocular point cloud.
The invention is realized by the following technical scheme:
s1: using a binocular camera to acquire space three-dimensional information and color information of the fruit tree, and generating an original color point cloud;
s2: preprocessing the original color point cloud to obtain a simplified point cloud;
s3: performing color-based region growing segmentation on the simplified point cloud to obtain point cloud clusters of different colors, and storing the clusters in an index;
s4: traversing the clusters in the index by an iterative method, screening the clusters by colors, and classifying labels to obtain apple point cloud clusters;
s5: and (4) carrying out pose extraction on the apple clusters, and packaging and outputting the obtained information.
Further, S1 should include:
the binocular camera needs to be subjected to internal reference calibration, a camera imaging geometric model is established, and lens distortion is corrected, so that accurate three-dimensional information is obtained.
Further, the preprocessing of the point cloud by S2 includes:
s201, simply and basically filtering the point clouds by using straight-through filtering, and eliminating point clouds outside the dimension of a main picking area;
s202, reducing the number of point clouds by using voxel filtering, simultaneously keeping the shape characteristics of the point clouds, and further accelerating the algorithm speed;
s203, removing obvious outliers by using statistical filtering to finally obtain the simplified point cloud.
Further, S3 color-based cluster-growing segmentation is characterized by a principle similar to the region-growing segmentation algorithm, except that it uses hair colors instead of normals, uses a merging algorithm for over-and under-segmentation control, and after segmentation, attempts are made to merge clusters with similar colors. The average color phase difference of two adjacent clusters is small and they are merged together. Followed by a second merging step. In this step, each cluster is verified according to the number of points it contains. If this number is less than the user defined value, the current cluster will be merged with the nearest neighbor cluster.
Further, the extracting of the cluster in S4 includes:
traversing the clusters in the index by an iteration method, extracting R-G color information of points in each cluster, carrying out classification labeling on the clusters by setting a color threshold, defining the clusters as required apple clusters if the clusters meet the requirements, and extracting.
Further, the further processing of the cluster by S4 includes:
and removing redundant leaf point clouds by using a color threshold value to obtain a final complete apple point cloud.
Optionally, the information extracted at S5 includes centroid coordinates, edge point coordinates, pose, surface normal, color, intensity, and the like of the cluster.
The method has the following effective benefits:
according to the method, the apples, the leaves and the environment are segmented by using the color difference, and then the clusters are screened and extracted through the colors, so that the pose information of the needed apples is obtained. Due to the imaging mode of the binocular camera, the binocular camera can work well in the outdoor environment.
Compared with a convolutional neural network, the method only needs to set a color threshold value, and does not need to train a model in advance; in the identification process, identification and extraction can be carried out through the color information of the point cloud, and a complex network structure is not needed; the requirement on hardware is not high, the speed is high, and the picking efficiency is greatly improved.
Drawings
FIG. 1 is a block diagram of the overall process provided by an embodiment of the present invention.
Fig. 2 is an original color point cloud obtained by a binocular camera according to an embodiment of the present invention.
FIG. 3 is a block diagram of a process for pre-processing a point cloud.
FIG. 4 is a through filtered point cloud provided by an embodiment of the present invention.
Fig. 5 is a voxel-filtered point cloud provided by an embodiment of the present invention.
FIG. 6 is a statistically filtered point cloud provided by an example of the present invention.
FIG. 7 is a point cloud of extracted apple clusters provided by an embodiment of the present invention.
Fig. 8 illustrates apple pose information provided by an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the method for rapidly identifying and positioning an apple based on binocular point cloud provided by the embodiment of the invention comprises the following steps:
s1, firstly, using a binocular camera such as ZED, firstly, calibrating internal reference of the camera, and then acquiring the whole space three-dimensional information and color information of the fruit tree, namely the original color point cloud, as shown in figure 2.
As shown in fig. 3, S2, preprocessing the original color point cloud to obtain a simplified point cloud, and the first step: because the acquired point cloud images have a plurality of non-picking areas, the acquired point cloud images need to be filtered in a three-dimensional space, the non-picking areas are removed, and the number of point clouds is reduced, as shown in fig. 4; secondly, voxel filtering processing is carried out on the point clouds, so that the number of the point clouds is further reduced, the requirement on hardware is lowered, and the processing speed of the point clouds is accelerated, as shown in fig. 5; and thirdly, carrying out statistical filtering on the point cloud, eliminating outliers, reserving the final point cloud, and facilitating clustering segmentation, as shown in FIG. 6.
And S3, performing color-based region growing segmentation on the simplified point cloud, performing cluster segmentation on the point cloud by setting a color threshold and a distance between points, acquiring point cloud clusters of different colors, and storing the clusters in an index.
And S4, traversing the indexes stored in the clusters by using an iteration method, performing threshold value of point cloud color on each index to judge whether the cluster is an apple cluster, if so, extracting the cluster from the indexes, and if not, discarding the cluster. And traversing the extracted point cloud, and eliminating redundant points through setting a color threshold value to obtain complete apple clusters. FIG. 7 shows the clusters of the extracted and processed apples.
S5 processes the apple clusters to obtain the required centroid coordinates, edge point information, and pose information, and packages and outputs them, as shown in fig. 8.
Finally, the method of the present invention is only a preferred embodiment, and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (7)
1. A binocular point cloud-based apple rapid identification and positioning method is characterized by comprising the following steps:
s1: using a binocular camera to acquire space three-dimensional information and color information of the fruit tree, and generating an original color point cloud;
s2: preprocessing the original color point cloud to obtain a simplified point cloud;
s3: performing color-based region growing segmentation on the simplified point cloud to obtain point cloud clusters of different colors, and storing the clusters in an index;
s4: traversing the clusters in the index by an iterative method, screening the clusters through colors, classifying labels to obtain apple point cloud clusters, and further processing the clusters;
s5: and (4) carrying out pose extraction on the apple clusters, and packaging and outputting the obtained information.
2. The method according to claim 1, wherein the step S1 further comprises: the binocular camera needs to be subjected to internal reference calibration, a camera imaging geometric model is established, and lens distortion is corrected, so that accurate three-dimensional information is obtained.
3. The method according to claim 1, wherein the step S2 further comprises:
s201, simply and basically filtering the point clouds by using straight-through filtering, and eliminating point clouds outside the dimension of a main picking area;
s202, reducing the number of point clouds by using voxel filtering, simultaneously keeping the shape characteristics of the point clouds, and further accelerating the algorithm speed;
s203, removing obvious outliers by using statistical filtering to finally obtain the simplified point cloud.
4. A method according to claim 1, characterized in that a color-based region growing segmentation algorithm is used, which is similar in principle to the region growing segmentation algorithm except that it uses hair colors instead of normals: firstly, performing over-segmentation and under-segmentation control by using a merging algorithm and trying to merge clusters with similar colors; in a second step each cluster is verified based on the number of points it contains, and if this number is less than the user defined value, the current cluster will be merged with the nearest neighbor cluster.
5. The method according to claim 1, wherein the step S4 further comprises:
traversing the clusters by an iterative method, extracting R-G color information of points in each cluster, carrying out classification labeling on the clusters by setting a color threshold, defining the clusters as required apple clusters if the clusters meet the requirements, and extracting.
6. The method according to claim 1, wherein the step S4 further comprises:
and (3) further processing the point cloud due to the incomplete segmentation condition of the region growing segmentation based on the color when the light is weak, and removing redundant leaf point cloud by using a color threshold value to obtain the final complete apple point cloud.
7. The method according to claim 1, wherein the information extracted in step S5 further comprises: centroid coordinates, edge point coordinates, and pose of the cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210398218.1A CN114677674A (en) | 2022-04-15 | 2022-04-15 | Apple rapid identification and positioning method based on binocular point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210398218.1A CN114677674A (en) | 2022-04-15 | 2022-04-15 | Apple rapid identification and positioning method based on binocular point cloud |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114677674A true CN114677674A (en) | 2022-06-28 |
Family
ID=82077715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210398218.1A Pending CN114677674A (en) | 2022-04-15 | 2022-04-15 | Apple rapid identification and positioning method based on binocular point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114677674A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114973006A (en) * | 2022-08-02 | 2022-08-30 | 四川省机械研究设计院(集团)有限公司 | Method, device and system for picking Chinese prickly ash and storage medium |
| CN115321090A (en) * | 2022-10-17 | 2022-11-11 | 中国民航大学 | Method, device, equipment, system and medium for automatically receiving and taking luggage in airport |
| CN116168386A (en) * | 2023-03-06 | 2023-05-26 | 东南大学 | Bridge construction progress identification method based on laser radar scanning |
| CN116596996A (en) * | 2023-05-26 | 2023-08-15 | 河北农业大学 | Method and system for acquiring spatial pose information of apple fruits |
-
2022
- 2022-04-15 CN CN202210398218.1A patent/CN114677674A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114973006A (en) * | 2022-08-02 | 2022-08-30 | 四川省机械研究设计院(集团)有限公司 | Method, device and system for picking Chinese prickly ash and storage medium |
| CN114973006B (en) * | 2022-08-02 | 2022-10-18 | 四川省机械研究设计院(集团)有限公司 | Method, device and system for picking Chinese prickly ash and storage medium |
| CN115321090A (en) * | 2022-10-17 | 2022-11-11 | 中国民航大学 | Method, device, equipment, system and medium for automatically receiving and taking luggage in airport |
| CN115321090B (en) * | 2022-10-17 | 2023-01-13 | 中国民航大学 | Method, device, equipment, system and medium for automatically receiving and taking luggage in airport |
| CN116168386A (en) * | 2023-03-06 | 2023-05-26 | 东南大学 | Bridge construction progress identification method based on laser radar scanning |
| CN116596996A (en) * | 2023-05-26 | 2023-08-15 | 河北农业大学 | Method and system for acquiring spatial pose information of apple fruits |
| CN116596996B (en) * | 2023-05-26 | 2024-01-30 | 河北农业大学 | Method and system for acquiring spatial pose information of apple fruits |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114677674A (en) | Apple rapid identification and positioning method based on binocular point cloud | |
| Dias et al. | Multispecies fruit flower detection using a refined semantic segmentation network | |
| CN109961049B (en) | Cigarette brand identification method under complex scene | |
| CN108562589B (en) | Method for detecting surface defects of magnetic circuit material | |
| Li et al. | Detection of fruit-bearing branches and localization of litchi clusters for vision-based harvesting robots | |
| CN108648233B (en) | Target identification and capture positioning method based on deep learning | |
| CN105718945B (en) | Apple picking robot night image recognition method based on watershed and neural network | |
| CN108304873B (en) | Target detection method and system based on high-resolution optical satellite remote sensing image | |
| US10282589B2 (en) | Method and system for detection and classification of cells using convolutional neural networks | |
| CN109166145B (en) | Fruit tree leaf growth parameter extraction method and system based on cluster segmentation | |
| Xie et al. | Detecting trees in street images via deep learning with attention module | |
| CN106407883A (en) | Complex table and method for identifying handwritten numbers in complex table | |
| CN108107444A (en) | Substation's method for recognizing impurities based on laser data | |
| CN113128610A (en) | Industrial part pose estimation method and system | |
| CN109559324A (en) | A kind of objective contour detection method in linear array images | |
| CN109145964B (en) | Method and system for realizing image color clustering | |
| CN110070571B (en) | Phyllostachys pubescens morphological parameter detection method based on depth camera | |
| Zhang et al. | A method for organs classification and fruit counting on pomegranate trees based on multi-features fusion and support vector machine by 3D point cloud | |
| CN112507896B (en) | Method for detecting cherry fruits by adopting improved YOLO-V4 model | |
| CN111783693A (en) | Intelligent identification method of fruit and vegetable picking robot | |
| CN114492619A (en) | Point cloud data set construction method and device based on statistics and concave-convex property | |
| CN116721121A (en) | Plant phenotype color image feature extraction method | |
| CN113744191A (en) | Automatic cloud detection method for satellite remote sensing image | |
| Septiarini et al. | Image processing techniques for tomato segmentation applying k-means clustering and edge detection approach | |
| CN106683105A (en) | Image segmentation method and image segmentation device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |