CN114972172B - Apple stem and calyx detection method based on three-dimensional point cloud - Google Patents
Apple stem and calyx detection method based on three-dimensional point cloud Download PDFInfo
- Publication number
- CN114972172B CN114972172B CN202210351063.6A CN202210351063A CN114972172B CN 114972172 B CN114972172 B CN 114972172B CN 202210351063 A CN202210351063 A CN 202210351063A CN 114972172 B CN114972172 B CN 114972172B
- Authority
- CN
- China
- Prior art keywords
- calyx
- apple
- dimensional point
- dimensional
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241001164374 Calyx Species 0.000 title claims abstract description 42
- 238000001514 detection method Methods 0.000 title claims abstract description 20
- 244000081841 Malus domestica Species 0.000 title 1
- 241000220225 Malus Species 0.000 claims abstract description 53
- 235000013399 edible fruits Nutrition 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims abstract description 17
- 235000021016 apples Nutrition 0.000 claims abstract description 6
- 238000004458 analytical method Methods 0.000 claims abstract description 4
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 241000196324 Embryophyta Species 0.000 description 1
- 241000238631 Hexapoda Species 0.000 description 1
- 241000607479 Yersinia pestis Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/254—Projection of a pattern, viewing through a pattern, e.g. moiré
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/64—Analysis of geometric attributes of convexity or concavity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20224—Image subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention discloses a three-dimensional point cloud-based apple stem and calyx detection method, which specifically comprises the following steps: step S1: acquiring dense three-dimensional point clouds of apples to be detected by using three-dimensional sensing equipment, converting the dense three-dimensional point clouds into apple depth maps D (x, y), and then transmitting the apple depth maps D (x, y) to a computer for analysis and processing; step S2: sequentially binarizing the apple depth map D (x, y) to obtain M binary images B m (x, y); step S3: obtaining a convex hull image A m (x, y) of a binary image B m (x, y) by utilizing a two-dimensional convex hull algorithm, and obtaining a convex residual region Q m (x, y) by differencing the convex hull image A m (x, y) and the binary image B m (x, y); step S4: and fusing all the convex residual regions Q m (x, y) to obtain a mask image Q (x, y) of the complete fruit stem calyx region. According to the concave characteristics of the fruit stem calyx, the three-dimensional surface shape of the apple is not required to be reconstructed, the dense three-dimensional point cloud of the apple is collected, the depth map is subjected to threshold segmentation, and the two-dimensional convex hull algorithm is combined, so that the fruit stem calyx area is effectively detected, and the method has the advantages of non-contact, low cost, high precision, high speed and strong robustness.
Description
Technical Field
The invention belongs to the technical field of detection, and particularly relates to a three-dimensional point cloud-based apple stem and calyx detection method.
Background
China is a large country for apple production and consumption, has huge market scale and is in the forefront of the world. With the continuous rise of social consumption level, people have an increasing demand for safe and high-quality apples. The evaluation indexes of the apple quality mainly comprise fruit shape, quality, color, plant diseases and insect pests, mechanical damage and the like.
Traditional apple quality detection adopts manual or mechanical mode more, and is time consuming and labor consuming, and the reliability is lower, and easily causes mechanical damage. The machine vision has the advantages of non-contact, high speed, high precision and the like, is widely applied to apple quality detection, and gradually replaces the traditional manual or mechanical detection mode. However, the external defects of apples are similar to the color characteristics of the fruit stem calyx, the traditional two-dimensional image detection technology based on the color characteristics is difficult to effectively distinguish, and the fruit stem calyx is easy to be identified as the external defects by mistake.
The invention patent with the application number 202011307487X utilizes a surface structured light technology to reconstruct the three-dimensional surface shape of apples, and detects the fruit stem calyx area according to the concave characteristics of the fruit stem calyx and by combining a gray morphology filling algorithm, but the method needs to construct a plurality of groups of three-dimensional images, so that the detection efficiency of the fruit stem calyx is lower. Therefore, how to accurately and rapidly detect the apple stem calyx has important meaning and application value.
Disclosure of Invention
The invention provides a three-dimensional point cloud-based apple stem and calyx detection method, which aims to solve the problems in the background technology.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: the apple stalk and calyx detection method based on the three-dimensional point cloud comprises the following steps:
Step S1: acquiring dense three-dimensional point clouds of apples to be detected by using three-dimensional sensing equipment, and converting the dense three-dimensional point clouds into apple depth maps D (X, Y), wherein (X, Y) represents pixel coordinates of the three-dimensional point clouds along X-axis and Y-axis directions, D (X, Y) represents depth values of the three-dimensional point clouds along Z-axis directions, and then transmitting the apple depth maps D (X, Y) to a computer for analysis processing;
Step S2: calculating a maximum depth value D max and a minimum depth value D min of the apple depth map D (x, y), selecting M depth threshold values T m in a depth interval [ D min,Dmax ] in equal step length, and sequentially binarizing the apple depth map D (x, y) to obtain M binary images B m (x, y);
step S3: obtaining a convex hull image A m (x, y) of a binary image B m (x, y) by utilizing a two-dimensional convex hull algorithm, and obtaining a convex residual region Q m (x, y) by differencing the convex hull image A m (x, y) and the binary image B m (x, y);
Step S4: according to the concave characteristics of the fruit stalk calyx, the convex residual region Q m (x, y) can be considered to belong to the fruit stalk calyx region; and fusing all the convex residual regions Q m (x, y), and removing burrs and slits through morphological filtering to obtain a mask image Q (x, y) of the complete fruit stem calyx region.
Preferably, in the step S1, the three-dimensional sensing device may use a speckle projection method, a fringe projection method, or a line laser scanning method.
Preferably, in the step S2, the selected M depth thresholds T m may be expressed as:
wherein: m=1, 2, …, M; m represents the number of depth thresholds T m, the value interval of which is [20,40].
Preferably, in the step S2, the calculation formula of the binary image B m (x, y) is as follows:
Preferably, in the step S3, the calculation formula of the convex residual region Q m (x, y) is as follows:
Qm(x,y)＝Am(x,y)-Bm(x,y)。
preferably, in the step S4, the mask image Q (x, y) of the stem calyx area has the following calculation formula:
Q(x,y)＝Q1(x,y)∪Q2(x,y)∪…∪QM(x,y)。
the beneficial effects of adopting above technical scheme are:
1. According to the three-dimensional point cloud-based apple stem and calyx detection method, according to the concave characteristics of the stem calyx, the three-dimensional surface shape of an apple is not required to be reconstructed, the dense three-dimensional point cloud of the apple is collected, the depth map is subjected to threshold segmentation, and the two-dimensional convex hull algorithm is combined, so that the stem calyx area is effectively detected, and the method has the advantages of being non-contact, low in cost, high in precision, high in speed and strong in robustness.
Drawings
An apple image in fig. 1 (a); (b) an apple three-dimensional point cloud; (c) apple depth map;
FIG. 2 shows binary images B 5(x,y)、B10 (x, y) and B 20 (x, y);
Fig. 3 shows convex hull images a 5(x,y)、A10 (x, y) and a 20 (x, y);
Fig. 4 shows convex residual regions Q 5(x,y)、Q10 (x, y) and Q 20 (x, y);
figure 5 shows the stem calyx region and its edges before and after morphological filtering;
Detailed Description
The following detailed description of the embodiments of the invention, given by way of example only, is presented in the accompanying drawings to aid in a more complete, accurate and thorough understanding of the concepts and aspects of the invention, and to aid in its practice, by those skilled in the art.
As shown in fig. 1 to 5, the method for detecting the apple stalks and the calyx based on the three-dimensional point cloud is characterized in that according to the concave characteristics of the apple stalks and the calyx, the three-dimensional surface shape of the apple is not required to be reconstructed, the dense three-dimensional point cloud of the apple is collected, the depth map is subjected to threshold segmentation, and the two-dimensional convex hull algorithm is combined, so that the apple stalk calyx area is effectively detected, and the method has the advantages of non-contact, low cost, high precision, high speed and strong robustness.
The following describes specific modes of operation with specific examples:
example 1:
The invention provides a three-dimensional point cloud-based apple stem and calyx detection method, which specifically comprises the following steps:
Step S1: fig. 1 (a) shows an image of an apple to be tested, a three-dimensional sensing device is used for collecting a dense three-dimensional point cloud of the apple to be tested, as shown in fig. 1 (b), and the dense three-dimensional point cloud is converted into an apple depth map D (X, Y), as shown in fig. 1 (c), wherein (X, Y) represents pixel coordinates of the three-dimensional point cloud along an X-axis and a Y-axis, D (X, Y) represents depth values of the three-dimensional point cloud along a Z-axis, and then the apple depth map D (X, Y) is transmitted to a computer for analysis and processing;
Step S2: calculating a maximum depth value D max and a minimum depth value D min of the apple depth map D (x, y), selecting M=30 depth thresholds T m in a depth interval [ D min,Dmax ] in equal step length, and sequentially binarizing the apple depth map D (x, y) to obtain M=30 binary images B m (x, y), as shown in fig. 2;
Step S3: obtaining a convex hull image A m (x, y) of the binary image B m (x, y) by using a two-dimensional convex hull algorithm, as shown in FIG. 3; the convex hull image a m (x, y) is differenced with the binary image B m (x, y) to obtain a convex residual region Q m (x, y), as shown in fig. 4;
Step S4: according to the concave characteristics of the fruit stalk calyx, the convex residual region Q m (x, y) can be considered to belong to the fruit stalk calyx region; and fusing all the convex residual regions Q m (x, y), removing burrs and slits through morphological filtering to obtain a mask image Q (x, y) of the complete fruit stem calyx region, wherein the fruit stem calyx region and the edge of the fruit stem calyx region before and after the morphological filtering are respectively shown in FIG. 5.
While the invention has been described above by way of example with reference to the accompanying drawings, it is to be understood that the invention is not limited to the particular embodiments described, but is capable of numerous insubstantial modifications of the inventive concept and solution; or the invention is not improved, and the conception and the technical scheme are directly applied to other occasions and are all within the protection scope of the invention.
Claims (6)
1. A three-dimensional point cloud-based apple stalk and calyx detection method is characterized by comprising the following steps of: the method specifically comprises the following steps:
Step S1: acquiring dense three-dimensional point clouds of apples to be detected by using three-dimensional sensing equipment, and converting the dense three-dimensional point clouds into apple depth maps D (X, Y), wherein (X, Y) represents pixel coordinates of the three-dimensional point clouds along X-axis and Y-axis directions, D (X, Y) represents depth values of the three-dimensional point clouds along Z-axis directions, and then transmitting the apple depth maps D (X, Y) to a computer for analysis processing;
Step S2: calculating a maximum depth value D max and a minimum depth value D min of the apple depth map D (x, y), selecting M depth threshold values T m in a depth interval [ D min,Dmax ] in equal step length, and sequentially binarizing the apple depth map D (x, y) to obtain M binary images B m (x, y);
step S3: obtaining a convex hull image A m (x, y) of a binary image B m (x, y) by utilizing a two-dimensional convex hull algorithm, and obtaining a convex residual region Q m (x, y) by differencing the convex hull image A m (x, y) and the binary image B m (x, y);
Step S4: according to the concave characteristics of the fruit stalk calyx, the convex residual region Q m (x, y) can be considered to belong to the fruit stalk calyx region; and fusing all the convex residual regions Q m (x, y), and removing burrs and slits through morphological filtering to obtain a mask image Q (x, y) of the complete fruit stem calyx region.
2. The three-dimensional point cloud-based apple stem and calyx detection method according to claim 1, wherein the method comprises the following steps of: in the step S1, the three-dimensional sensing device may use a speckle projection method, a fringe projection method, or a line laser scanning method.
3. The three-dimensional point cloud-based apple stem and calyx detection method according to claim 1, wherein the method comprises the following steps of: in the step S2, the selected M depth thresholds T m may be expressed as:
wherein: m=1, 2, …, M; m represents the number of depth thresholds T m, the value interval of which is [20,40].
4. The three-dimensional point cloud-based apple stem and calyx detection method according to claim 1, wherein the method comprises the following steps of: in the step S2, the calculation formula of the binary image B m (x, y) is as follows:
5. The three-dimensional point cloud-based apple stem and calyx detection method according to claim 1, wherein the method comprises the following steps of: in the step S3, the calculation formula of the convex residual region Q m (x, y) is as follows:
Qm(x,y)＝Am(x,y)-Bm(x,y)。
6. The three-dimensional point cloud-based apple stem and calyx detection method according to claim 1, wherein the method comprises the following steps of: in the step S4, the calculation formula of the mask image Q (x, y) of the fruit stem calyx area is as follows:
Q(x,y)＝Q1(x,y)∪Q2(x,y)∪…∪QM(x,y)。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210351063.6A CN114972172B (en) | 2022-04-02 | 2022-04-02 | Apple stem and calyx detection method based on three-dimensional point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210351063.6A CN114972172B (en) | 2022-04-02 | 2022-04-02 | Apple stem and calyx detection method based on three-dimensional point cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114972172A CN114972172A (en) | 2022-08-30 |
| CN114972172B true CN114972172B (en) | 2024-08-13 |
Family
ID=82977301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210351063.6A Active CN114972172B (en) | 2022-04-02 | 2022-04-02 | Apple stem and calyx detection method based on three-dimensional point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972172B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101613550B1 (en) * | 2014-11-28 | 2016-04-19 | 세종대학교산학협력단 | Device and method for detecting apple |
| CN112686885A (en) * | 2021-01-13 | 2021-04-20 | 北京农业信息技术研究中心 | Fruit skin defect detection method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112488998B (en) * | 2020-11-19 | 2022-10-14 | 安徽农业大学 | Apple stem and calyx detection method based on stripe projection |
-
2022
- 2022-04-02 CN CN202210351063.6A patent/CN114972172B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101613550B1 (en) * | 2014-11-28 | 2016-04-19 | 세종대학교산학협력단 | Device and method for detecting apple |
| CN112686885A (en) * | 2021-01-13 | 2021-04-20 | 北京农业信息技术研究中心 | Fruit skin defect detection method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114972172A (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109035224B (en) | Submarine pipeline detection and three-dimensional reconstruction method based on multi-beam point cloud | |
| CN109816664B (en) | Three-dimensional point cloud segmentation method and device | |
| Qiao et al. | An automatic active contour method for sea cucumber segmentation in natural underwater environments | |
| CN108960011B (en) | Partially-shielded citrus fruit image identification method | |
| CN105424709A (en) | Fruit surface defect detection method based on image marking | |
| CN111597930A (en) | Coastline extraction method based on remote sensing cloud platform | |
| Feng et al. | A separating method of adjacent apples based on machine vision and chain code information | |
| CN112884880B (en) | Line laser-based honey pomelo three-dimensional modeling device and method | |
| CN111723821A (en) | Detection and identification method and device for power plant instrument image | |
| CN115731257A (en) | Leaf form information extraction method based on image | |
| CN117115196B (en) | Visual detection method and system for cutter abrasion of cutting machine | |
| CN108960100A (en) | A kind of recognition methods of the sugarcane sugarcane section based on image procossing | |
| CN110176020A (en) | A kind of bird's nest impurity method for sorting merging 2D and 3D rendering | |
| CN104992448A (en) | Automatic positioning method for robot picking grapes in a damage-free way | |
| Ma et al. | Rapid estimation of apple phenotypic parameters based on 3D reconstruction | |
| CN114972172B (en) | Apple stem and calyx detection method based on three-dimensional point cloud | |
| CN107993243B (en) | Wheat tillering number automatic detection method based on RGB image | |
| CN117036944B (en) | Tree carbon sink amount calculating method and system based on point cloud data and image recognition | |
| CN102750547B (en) | Fruit size grading method based on compressed sensing | |
| WO2023134251A1 (en) | Clustering-based light bar extraction method and apparatus | |
| CN104881652B (en) | A kind of line number automatic testing method based on corncob male features | |
| CN103994718B (en) | Tiny electronic element location recognition method based on microscope and computer vision | |
| CN105354823A (en) | Tree-ring image edge extraction and segmentation system | |
| CN113421301B (en) | Method and system for positioning central area of field crop | |
| CN114463258A (en) | Locomotive brake pad abrasion detection method based on deep learning technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |