CN105825171A - RGB-D-based rapid identification and positioning method for fruit on tree - Google Patents

RGB-D-based rapid identification and positioning method for fruit on tree Download PDF

Info

Publication number
CN105825171A
CN105825171A CN201610137746.6A CN201610137746A CN105825171A CN 105825171 A CN105825171 A CN 105825171A CN 201610137746 A CN201610137746 A CN 201610137746A CN 105825171 A CN105825171 A CN 105825171A
Authority
CN
China
Prior art keywords
fruit
depth
rgb
branch
closed curve
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610137746.6A
Other languages
Chinese (zh)
Other versions
CN105825171B (en
Inventor
刘继展
周尧
朱新新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu University
Original Assignee
Jiangsu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu University filed Critical Jiangsu University
Priority to CN201610137746.6A priority Critical patent/CN105825171B/en
Publication of CN105825171A publication Critical patent/CN105825171A/en
Application granted granted Critical
Publication of CN105825171B publication Critical patent/CN105825171B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30188—Vegetation; Agriculture

Abstract

The invention discloses an RGB-D-based rapid identification and positioning method for a fruit on a tree. An RGB-D sensor is used for obtaining a characteristic of depth and color information corresponding to a coordinate synchronously; layer-by-layer cutting of a depth spherical surface is carried out by starting from a nearest point, wherein cutting by the depth spherical surface on a leaf with a sheet characteristic is carried out to only obtain an unclosed curve that is not considered to be processed; for all closed curves obtained by depth spherical surface cutting, a branch and a bamboo pole that have the diameters less than the diameter of the fruit obviously are rejected according to short axis lengths; andthe rest of closed curves of the fruit are processed to generate a three-dimensional contour of a depth and color point cloud, so that the identification and positioning of the mature fruit are completed. According to the invention, on the basis of advantages of the geometric difference between the spatial entity fruit and the sheet-shaped leaf and synchronous obtaining of the coordinate, reflectivity and depth information of the RGB-D sensor, identification and positioning of the fruit on the tree can be realized rapidly without the need to consider a shielding problem as the most troubling concern for identification and positioning of the fruit on the tree. The method that is simple and reliable and has excellent real-time and practical properties can be applied to rapid identification and positioning of the fruit on the tree.

Description

The quick recognition positioning method of fruit on a kind of tree based on RGB-D
Art
The present invention relates to field of agricultural robots, quickly know particularly to fruit on a kind of tree based on RGB-D Other localization method.
Background technology
In picking robot, on tree, the identification of fruit and location are key technology difficult problems.Due in natural conditions Under, Conventional visual technology exist excessively sensitive to light, easily by ambient interferences, cannot be distinguished by advancing coloud nearside fruit leaf and fixed The deficiencies such as the coupling difficulty of position, utilize depth information to realize fruit identification location in recent years and have been subjected to get more and more Attention.But single depth information is only capable of carrying out shape recognition and cannot obtaining color distortion, thus cannot district Being divided into ripe and immature fruit, the branch and leaf being simultaneously still vulnerable in complicated canopy space interfere；And by the degree of depth with Visible ray or the fusion of the information such as infrared, then become non-due to the coordinate transformation between different components and images match The most complicated, have a strong impact on its recognition effect and real-time.RGB-D is the synchronization gain degree of depth to believe with color The novel consumer level senser element of breath, but existing recognition positioning method based on RGB-D mainly continues tradition The division processing method of coloured image, necessarily changes although utilizing depth information to make identification and locating effect obtain Kind, but the overlap that positions of the upper fruit identification of long-standing problem tree the problem such as is blocked and is the most effectively solved, and identification can Also the needs of actual application can not be met by property and real-time.
Summary of the invention
For the deficiency of fruit recognition positioning method on existing tree, the present invention provides fruit on a kind of tree based on RGB-D Real quickly recognition positioning method, realizes quick and precisely identification and the location of ripe objective fruit upper to tree.
In order to solve above technical problem, the concrete technical scheme that the present invention uses comprises the following steps:
Step one, RGB-D sensor (1) passes through real time scan, the basis of synchronization gain front influences of plant crown The coordinate degree of depth one to one and color data (D, E) │(D,θ)；
Wherein in polar coordinate space, with RGB-D sensor as zero, each coordinate points (D, θ) deep Degrees of data is D, and the color data of each coordinate points (D, θ) is E.
Step 2, in depth data, the first closest approach A of inspected object in discovery influences of plant crown；
Step 3, from the beginning of closest approach A, utilizes the numerical value D of the degree of depth1=DA+ i Δ D (i=1,2 ..., n) Successively cut, wherein DAFor the depth value of the nearest depth data point A being detected object, D For the degree of depth spacing successively cut, D1=DA+ i Δ D is the sphere in three dimensions.
Fruit (4) and branch (7), bamboo pole (8) etc. possess the object of spatial entities feature by degree of depth sphere D1After cutting, there is closed curve (2) in depth point cloud；And the leaf of strip-like features (6) is by degree of depth sphere D1After cutting, only there is non-close curve (3) in depth point cloud.Therefore, to degree of depth sphere D1Successively cut, Until there is closed curve (2), and all non-close curves (3) are considered leaf (6) and disregard；
Step 4, to all closed curves (2), calculates its minor axis length L, and all minor axis length L is surpassed Closed curve (2) the place object of the maximum gauge value [L] crossing branch (7) and bamboo pole (8) is considered as fruit (4), And all minor axis length L is less than the closed curve (2) of branch (7) with the maximum gauge value [L] of bamboo pole (8) Place object is considered branch (7) or bamboo pole (8) and processes the most further；
All minor axis length L are exceeded the branch (7) the maximum gauge value [L] with bamboo pole (8) by step 5 Closed curve (2), utilizes the 3-dimensional profile of degree of depth point cloud constructions closed curve (2) place fruit (4)；
Step 6, according to color data and the one-to-one relationship of depth data of closed curve (2), it is thus achieved that The color data values of the 3-dimensional profile point cloud of fruit (4), thus the Maturity of fruit (4) is carried out according to color Judge；
Step 7, reaches Maturity to pluck the mature fruit (5) required, utilizes coordinate to mature fruit (5) 3-dimensional profile carry out space orientation, and then complete to pluck.
Described RGB-D sensor is can synchronization gain depth data, Realsense, Kinect of color data Deng any one.
Described fruit be diameter be noticeably greater than branch (7) and the Fructus Mali pumilae of bamboo pole (8) diameter, Fructus Lycopersici esculenti, Fructus Persicae, pears, Any one of Citrus etc..
The present invention has beneficial effect.The present invention utilize spatial entities fruit and lamellar leaf disparity and RGB-D sensor synchronization gain coordinate, reflectance, the advantage of depth information, quickly realize the upper mature fruit of tree Identification and determine method, and form the occlusion issue of maximum puzzlement regardless of fruit identification location upper to tree, method Simple and reliable, real-time is with practical.
Accompanying drawing explanation
Fig. 1 is the method flow diagram of the present invention.
Fig. 2 is that influences of plant crown based on RGB-D detects scene schematic diagram.
Fig. 3 is that the sphere cutting of fruit obtains closed curve schematic diagram.
Fig. 4 is that the sphere cutting of leaf obtains non-close curve synoptic diagram.
Fig. 5 is the minor axis length schematic diagram of closed curve.
In figure: 1.RGB-D sensor, 2. closed curve, 3. non-close curve, 4. fruit, 5. mature fruit, 6. leaf, 7. branch, 8. bamboo pole.
Detailed description of the invention
Below in conjunction with the accompanying drawings and specific embodiment, technical scheme is described in further details.
Fig. 1 illustrates the method flow diagram of the present invention.
Such as Fig. 1 and Fig. 2, the concrete of the present invention identifies that location implementation process is as follows:
(1) RGB-D sensor 1 is by real time scan, synchronization gain front influences of plant crown according to coordinate one The degree of depth of one correspondence and color data (D, E) │(D,θ)；
Wherein in polar coordinate space, with RGB-D sensor as zero, each coordinate points (D, θ) deep Degrees of data is D, and the color data of each coordinate points (D, θ) is E.
(2) in depth data, the first closest approach A of inspected object in discovery influences of plant crown；
(3) from the beginning of closest approach A, the numerical value D of the degree of depth is utilized1=DA+ i Δ D (i=1,2 ..., n) Successively cut, wherein DAFor the depth value of the nearest depth data point A being detected object, D For the degree of depth spacing successively cut, D1=DA+ i Δ D is the sphere in three dimensions.
Fruit 4 and branch 7, bamboo pole 8 etc. possess the object of spatial entities feature by degree of depth sphere D1After cutting, There is closed curve 2 (Fig. 3) in depth point cloud；And the leaf 6 of strip-like features is by degree of depth sphere D1After cutting, Only there is non-close curve 3 (Fig. 4) in depth point cloud.Therefore, to degree of depth sphere D1Successively cut, until going out Existing closed curve 2, and all non-close curves 3 are considered leaf 6 and disregard；
(4) such as Fig. 5, to all closed curves 2, its minor axis length L is calculated, and by all minor axis length L The closed curve 2 place object of the maximum gauge value [L] exceeding branch 7 and bamboo pole 8 is considered as fruit 4, and owns Minor axis length L is regarded less than the closed curve 2 place object of branch 7 with the maximum gauge value [L] of bamboo pole 8 Process further is refused for branch 7 or bamboo pole 8；
(5) all minor axis length L are exceeded the closed curve 2 of branch 7 and the maximum gauge value [L] of bamboo pole 8, Utilize the 3-dimensional profile of degree of depth point cloud constructions closed curve 2 place fruit 4；
(6) according to color data and the one-to-one relationship of depth data of closed curve 2, it is thus achieved that fruit 4 The color data values of 3-dimensional profile point cloud, thus carry out the Maturity of fruit 4 according to color and judge；
(7) reach Maturity to pluck the mature fruit 5 required, utilize the coordinate 3-dimensional to mature fruit 5 Profile carries out space orientation, and then completes to pluck.

Claims (3)

1. the quick recognition positioning method of fruit on a tree based on RGB-D, it is characterised in that comprise the following steps:
Step one, RGB-D sensor (1) pass through real time scan, synchronization gain front influences of plant crown according to coordinate depth data one to one and color data (D, E) │(D, θ )；
Wherein in polar coordinate space, with RGB-D sensor as zero, the depth data of each coordinate points (D, θ) is D, and the color data of each coordinate points (D, θ) is E；
Step 2, in depth data, the first closest approach A of inspected object in discovery influences of plant crown；
Step 3, from the beginning of closest approach A, utilizes the numerical value D of the degree of depth1=DA+ i Δ D successively cuts, wherein DAFor the depth value of the nearest depth data point A being detected object, D is the degree of depth spacing successively cut, D1=DA+ i Δ D is the sphere in three dimensions；I=1,2 ..., n；
Fruit (4) and branch (7), bamboo pole (8) etc. possess the object of spatial entities feature by degree of depth sphere D1After cutting, there is closed curve (2) in depth point cloud；And the leaf of strip-like features (6) is by degree of depth sphere D1After cutting, only there is non-close curve (3) in depth point cloud；Therefore, to degree of depth sphere D1Successively cutting, until there is closed curve (2), and all non-close curves (3) are considered leaf (6) and disregard；
Step 4, to all closed curves (2), calculate its minor axis length L, and closed curve (2) the place object that all minor axis length L exceed the maximum gauge value [L] of branch (7) and bamboo pole (8) is considered as fruit (4), and all minor axis length L are considered branch (7) or bamboo pole (8) less than closed curve (2) the place object of branch (7) with the maximum gauge value [L] of bamboo pole (8) and refuse process further；
Step 5, all minor axis length L are exceeded the closed curve (2) of branch (7) and the maximum gauge value [L] of bamboo pole (8), utilizes the three-D profile of degree of depth point cloud constructions closed curve (2) place fruit (4)；
Step 6, color data according to closed curve (2) and the one-to-one relationship of depth data, obtain color data values ZZ of the three-D profile point cloud of fruit (4), thus carry out the Maturity judgement of fruit (4) according to color data values ZZ；
Step 7, reaches Maturity to pluck the mature fruit (5) required, utilizes coordinate that the three-D profile of mature fruit (5) carries out space orientation, and then completes to pluck.
The quick recognition positioning method of fruit on a kind of tree based on RGB-D the most according to claim 1, it is characterised in that: described RGB-D sensor be can synchronization gain depth data, color data Realsense, Kinect in any one.
The quick recognition positioning method of fruit on a kind of tree based on RGB-D the most according to claim 1, it is characterised in that: described fruit is any one that diameter is noticeably greater than in branch (7) and the Fructus Mali pumilae of bamboo pole (8) diameter, Fructus Lycopersici esculenti, Fructus Persicae, pears, Citrus.
CN201610137746.6A 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D Active CN105825171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610137746.6A CN105825171B (en) 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610137746.6A CN105825171B (en) 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D

Publications (2)

Publication Number Publication Date
CN105825171A true CN105825171A (en) 2016-08-03
CN105825171B CN105825171B (en) 2019-02-01

Family

ID=56987778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610137746.6A Active CN105825171B (en) 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D

Country Status (1)

Country Link
CN (1) CN105825171B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875412A (en) * 2017-02-28 2017-06-20 重庆理工大学 A kind of two segmentation localization methods of overlap fruit based on aberration
CN106919901A (en) * 2017-01-20 2017-07-04 江苏大学 A kind of depth ball transversal method of spherefruit identification in strain
CN108064561A (en) * 2018-01-29 2018-05-25 骆通运 A kind of apple automation picker and method
CN108470339A (en) * 2018-03-21 2018-08-31 华南理工大学 A kind of visual identity of overlapping apple and localization method based on information fusion
CN109584292A (en) * 2018-11-14 2019-04-05 南京农业大学 A kind of fruit tree three-dimensional shape measurement system based on Kinect Auto-calibration
CN116267226A (en) * 2023-05-16 2023-06-23 四川省农业机械研究设计院 Mulberry picking method and device based on intelligent machine vision recognition of maturity

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103065149A (en) * 2012-12-21 2013-04-24 上海交通大学 Netted melon fruit phenotype extraction and quantization method
CN103727877A (en) * 2013-12-20 2014-04-16 北京农业信息技术研究中心 Fruit identifying and locating method, device and system
CN103824324A (en) * 2014-02-28 2014-05-28 北京农业信息技术研究中心 Fruit tree crown layer leaf and fruit three-dimensional reconstruction method and system
CN104700404A (en) * 2015-03-02 2015-06-10 中国农业大学 Fruit location identification method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103065149A (en) * 2012-12-21 2013-04-24 上海交通大学 Netted melon fruit phenotype extraction and quantization method
CN103727877A (en) * 2013-12-20 2014-04-16 北京农业信息技术研究中心 Fruit identifying and locating method, device and system
CN103824324A (en) * 2014-02-28 2014-05-28 北京农业信息技术研究中心 Fruit tree crown layer leaf and fruit three-dimensional reconstruction method and system
CN104700404A (en) * 2015-03-02 2015-06-10 中国农业大学 Fruit location identification method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
司永胜 等: "《苹果采摘机器人果实识别与定位方法》", 《农业机械学报》 *
董建民 等: "《基于Kinect视觉系统的西红柿自动识别与定位》", 《中国农机学报》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106919901A (en) * 2017-01-20 2017-07-04 江苏大学 A kind of depth ball transversal method of spherefruit identification in strain
CN106875412A (en) * 2017-02-28 2017-06-20 重庆理工大学 A kind of two segmentation localization methods of overlap fruit based on aberration
CN106875412B (en) * 2017-02-28 2020-06-23 重庆理工大学 Segmentation positioning method for two overlapped fruits
CN108064561A (en) * 2018-01-29 2018-05-25 骆通运 A kind of apple automation picker and method
CN108064561B (en) * 2018-01-29 2020-07-14 骆通运 Automatic apple picking device and method
CN108470339A (en) * 2018-03-21 2018-08-31 华南理工大学 A kind of visual identity of overlapping apple and localization method based on information fusion
CN109584292A (en) * 2018-11-14 2019-04-05 南京农业大学 A kind of fruit tree three-dimensional shape measurement system based on Kinect Auto-calibration
CN109584292B (en) * 2018-11-14 2022-04-19 南京农业大学 Fruit tree three-dimensional form measuring system based on Kinect is demarcation independently
CN116267226A (en) * 2023-05-16 2023-06-23 四川省农业机械研究设计院 Mulberry picking method and device based on intelligent machine vision recognition of maturity

Also Published As

Publication number Publication date
CN105825171B (en) 2019-02-01

Similar Documents

Publication Publication Date Title
CN105825171A (en) RGB-D-based rapid identification and positioning method for fruit on tree
CN103049912B (en) Random trihedron-based radar-camera system external parameter calibration method
CN103411553B (en) The quick calibrating method of multi-linear structured light vision sensors
CN106356757B (en) A kind of power circuit unmanned plane method for inspecting based on human-eye visual characteristic
CN108535321B (en) Building thermal performance detection method based on three-dimensional infrared thermal imaging technology
CN110264416A (en) Sparse point cloud segmentation method and device
CN104463108B (en) A kind of monocular real time target recognitio and pose measuring method
CN103258345B (en) A kind of tree limb parameter extracting method based on ground laser radar 3-D scanning
CN110415342A (en) A kind of three-dimensional point cloud reconstructing device and method based on more merge sensors
CN109270544A (en) Mobile robot self-localization system based on shaft identification
Qingchun et al. Design of structured-light vision system for tomato harvesting robot
CN112686938B (en) Power transmission line clear distance calculation and safety alarm method based on binocular image ranging
CN114241298A (en) Tower crane environment target detection method and system based on laser radar and image fusion
CN104469155A (en) On-board figure and image virtual-real superposition method
CN103544493A (en) Fruit distinguishing and locating method with laser scanning and machine vision combined
CN104400265A (en) Feature extraction method applicable to corner weld of laser vision guided welding robot
CN110132226A (en) The distance and azimuth angle measurement system and method for a kind of unmanned plane line walking
CN105096285A (en) Image fusion and target tracking system based on multi-core DSP
CN109523528A (en) A kind of transmission line of electricity extracting method based on unmanned plane binocular vision SGC algorithm
CN105631852A (en) Depth image contour line-based indoor human body detection method
CN111856436A (en) Combined calibration device and calibration method for multi-line laser radar and infrared camera
CN104182968A (en) Method for segmenting fuzzy moving targets by wide-baseline multi-array optical detection system
CN114758215A (en) Distribution network live working robot grounding ring and bolt identification and positioning method thereof
Hu et al. Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN
CN116385432B (en) Light-weight decoupling wheat scab spore detection method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant