CN105825171B - The quick recognition positioning method of fruit on a kind of tree based on RGB-D - Google Patents

The quick recognition positioning method of fruit on a kind of tree based on RGB-D Download PDF

Info

Publication number
CN105825171B
CN105825171B CN201610137746.6A CN201610137746A CN105825171B CN 105825171 B CN105825171 B CN 105825171B CN 201610137746 A CN201610137746 A CN 201610137746A CN 105825171 B CN105825171 B CN 105825171B
Authority
CN
China
Prior art keywords
depth
fruit
rgb
closed curve
spherical surface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610137746.6A
Other languages
Chinese (zh)
Other versions
CN105825171A (en
Inventor
刘继展
周尧
朱新新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu University
Original Assignee
Jiangsu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu University filed Critical Jiangsu University
Priority to CN201610137746.6A priority Critical patent/CN105825171B/en
Publication of CN105825171A publication Critical patent/CN105825171A/en
Application granted granted Critical
Publication of CN105825171B publication Critical patent/CN105825171B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30188—Vegetation; Agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Harvesting Machines For Specific Crops (AREA)
  • Spectrometry And Color Measurement (AREA)
  • Manipulator (AREA)

Abstract

The invention discloses the quick recognition positioning methods of fruit on a kind of tree based on RGB-D, it can the corresponding depth of synchronization gain coordinate and the characteristics of colouring information using RGB-D sensor, the layer-by-layer cutting of depth spherical surface is carried out since the closest approach, wherein the leaf of strip-like features is only obtained non-close curve and disregarded by the cutting of depth spherical surface, and to all closed curves obtained by the cutting of depth spherical surface, limb and bamboo pole of fruit diameter etc. is significantly less than according to its minor axis length to reject, remaining fruit closed curve is further generated to the three-D profile of depth and color point cloud, to complete the identification and positioning of ripening fruits.The present invention utilizes spatial entities fruit and the disparity of sheet leaf and the advantage of RGB-D sensor synchronization gain coordinate, reflectivity, depth information, it fast implements the identification for setting upper ripening fruits and determines method, and regardless of to the occlusion issue for setting the maximum puzzlement of upper fruit identification positioning formation, method is simple and reliable, real-time with it is practical, can be applied to set the quick identification positioning of fruit.

Description

The quick recognition positioning method of fruit on a kind of tree based on RGB-D
Technical field
The present invention relates to field of agricultural robots, in particular to fruit quickly identifies positioning on a kind of tree based on RGB-D Method.
Background technique
In picking robot, the identification Yu positioning of fruit are key technology difficulties on tree.Due under field conditions (factors), passing Unite vision technique exist to light it is too sensitive, vulnerable to background interference, cannot be distinguished advancing coloud nearside fruit leaf and the matching difficulty of positioning etc. Deficiency realizes that fruit identification positioning has been subjected to more and more paying attention to using depth information in recent years.But single depth letter Breath only can be carried out shape recognition and can not obtain color difference, so that mature and immature fruit cannot be distinguished, be still vulnerable to simultaneously Branches and leaves in complicated canopy space interfere with each other；And merging depth and visible light or the information such as infrared, then due to different devices Coordinate transformation and images match between part and become extremely complex, seriously affected its recognition effect and real-time.RGB-D is can The novel consumer level senser element of synchronization gain depth and colouring information, but the existing recognition positioning method based on RGB-D is still main Continue the division processing method of conventional color image, although making to identify and position effect and obtain centainly to change using depth information The problems such as kind, but the overlapping of the upper fruit identification positioning of long-standing problem tree is blocked, is not solved effectively, identification certainty and reality When property is not also able to satisfy the needs of practical application.
Summary of the invention
For the deficiency of fruit recognition positioning method on existing tree, the present invention provides fruit on a kind of tree based on RGB-D Quick recognition positioning method realizes quick and precisely identifying and positioning to the upper mature objective fruit of tree.
In order to solve the above technical problems, the specific technical solution that the present invention uses the following steps are included:
Step 1, RGB-D sensor (1) by real time scan, in front of synchronization gain influences of plant crown according to coordinate one by one Corresponding depth and color data (D, E) │(D,θ)；
Wherein in polar coordinate space, using RGB-D sensor as coordinate origin, the depth data of each coordinate points (D, θ) For D, the color data of each coordinate points (D, θ) is E.
Step 2 first finds the closest approach A of inspected object in influences of plant crown in depth data；
Step 3 utilizes the numerical value D of depth since closest approach A1=DA+ i Δ D (i=1,2 ..., n) is successively cut It cuts, wherein DAIt is the depth spacing successively cut, D for the depth value , ⊿ D of the nearest depth data point A for being detected object1 =DA+ i Δ D is the spherical surface in three-dimensional space.
Fruit (4) and limb (7), bamboo pole (8) etc. have the object of spatial entities feature by depth spherical surface D1It is deep after cutting There are closed curve (2) in degree point cloud；And the leaf (6) of strip-like features is by depth spherical surface D1After cutting, depth point cloud only occurs non- Closed curve (3).Therefore, to depth spherical surface D1Successively cutting, until occur closed curve (2), and all non-close curves (3) It is considered as leaf (6) and disregards；
Step 4 calculates its minor axis length L to all closed curves (2), and is more than limb (7) by all minor axis length L It is considered as fruit (4) with object where the closed curve (2) of the maximum gauge value [L] of bamboo pole (8), and all minor axis length L do not surpass Object where the closed curve (2) for the maximum gauge value [L] for crossing limb (7) and bamboo pole (8) is considered as limb (7) or bamboo pole (8) And it is not further processed；
Step 5 is more than the closed curve of the maximum gauge value [L] of limb (7) and bamboo pole (8) to all minor axis length L (2), 3 dimension profiles of fruit (4) where depth point cloud constructions closed curve (2) are utilized；
Step 6 obtains fruit (4) according to the one-to-one relationship of the color data of closed curve (2) and depth data 3 dimension profile point clouds color data values, thus according to color carry out fruit (4) maturity judge；
Step 7 reaches the ripening fruits (5) that picking requires to maturity, is taken turns using 3 dimensions of coordinate pair ripening fruits (5) Exterior feature carries out space orientation, and then completes picking.
The RGB-D sensor be can synchronization gain depth data, color data Realsense, Kinect etc. appoint It is a kind of.
The fruit is that diameter is noticeably greater than apple, tomato, peach, pears, citrus of limb (7) and bamboo pole (8) diameter etc. It is any.
The present invention has beneficial effect.The present invention utilizes the disparity and RGB-D of spatial entities fruit and sheet leaf The advantage of sensor synchronization gain coordinate, reflectivity, depth information fast implements the identification for setting upper ripening fruits and determines method, and Regardless of to the occlusion issue for setting the maximum puzzlement of upper fruit identification positioning formation, method is simple and reliable, real-time and practicability By force.
Detailed description of the invention
Fig. 1 is flow chart of the method for the present invention.
Fig. 2 is that the influences of plant crown based on RGB-D detects schematic diagram of a scenario.
Fig. 3 is that the spherical surface cutting of fruit obtains closed curve schematic diagram.
Fig. 4 is that the spherical surface cutting of leaf obtains non-close curve synoptic diagram.
Fig. 5 is the minor axis length schematic diagram of closed curve.
In figure: 1.RGB-D sensor, 2. closed curves, 3. non-close curves, 4. fruits, 5. ripening fruits, 6. leaves, 7. limb, 8. bamboo poles.
Specific embodiment
In the following with reference to the drawings and specific embodiments, further details of the technical solution of the present invention.
Fig. 1 illustrates flow chart of the method for the present invention.
Such as Fig. 1 and Fig. 2, it is as follows that specific identification of the invention positions implementation process:
(1) RGB-D sensor 1 passes through real time scan, and influences of plant crown is one-to-one according to coordinate in front of synchronization gain Depth and color data (D, E) │(D,θ)；
Wherein in polar coordinate space, using RGB-D sensor as coordinate origin, the depth data of each coordinate points (D, θ) For D, the color data of each coordinate points (D, θ) is E.
(2) in depth data, the closest approach A of inspected object in influences of plant crown is first found；
(3) since closest approach A, the numerical value D of depth is utilized1=DA+ i Δ D (i=1,2 ..., n) is successively cut, Middle DAIt is the depth spacing successively cut, D for the depth value , ⊿ D of the nearest depth data point A for being detected object1=DA+i Δ D is the spherical surface in three-dimensional space.
Fruit 4 and limb 7, bamboo pole 8 etc. have the object of spatial entities feature by depth spherical surface D1After cutting, depth point cloud Occur closed curve 2 (Fig. 3)；And the leaf 6 of strip-like features is by depth spherical surface D1After cutting, only there is non-close song in depth point cloud Line 3 (Fig. 4).Therefore, to depth spherical surface D1Successively cutting, until there is closed curve 2, and all non-close curves 3 are considered as Leaf 6 and disregard；
(4) such as Fig. 5, to all closed curves 2, calculate its minor axis length L, and by all minor axis length L be more than limb 7 with The 2 place object of closed curve of the maximum gauge value [L] of bamboo pole 8 is considered as fruit 4, and all minor axis length L be no more than limb 7 with The 2 place object of closed curve of the maximum gauge value [L] of bamboo pole 8 is considered as limb 7 or bamboo pole 8 and is not further processed；
(5) closed curve 2 to all minor axis length L more than limb 7 and the maximum gauge value [L] of bamboo pole 8, utilizes depth 3 dimension profiles of 2 place fruit 4 of point cloud constructions closed curve；
(6) according to the one-to-one relationship of the color data of closed curve 2 and depth data, 3 dimension profiles of fruit 4 are obtained The color data values of point cloud, to be judged according to the maturity that color carries out fruit 4；
(7) ripening fruits 5 that picking requires is reached to maturity, is carried out using 3 dimension profiles of coordinate pair ripening fruits 5 empty Between position, and then complete picking.

Claims (3)

1. the quick recognition positioning method of fruit on a kind of tree based on RGB-D, it is characterised in that the following steps are included:
Step 1, RGB-D sensor (1) are corresponded by real time scan, synchronization gain front influences of plant crown according to coordinate Depth data and color data (D, E) │(D,θ)；
Wherein in polar coordinate space, using RGB-D sensor as coordinate origin, the depth data of each coordinate points (D, θ) is D, The color data of each coordinate points (D, θ) is E；
Step 2 first finds the closest approach A of inspected object in influences of plant crown in depth data；
Step 3 utilizes the numerical value D of depth since closest approach A1=DA+ i Δ D is successively cut, wherein DAIt is nearest The depth value , ⊿ D for being detected the depth data point A of object is the depth spacing successively cut, D1=DA+ i Δ D is three-dimensional space Interior spherical surface；I=1,2 ..., n；
Fruit (4) and limb (7), bamboo pole (8) have the object of spatial entities feature by depth spherical surface D1After cutting, depth point cloud Occur closed curve (2)；And the leaf (6) of strip-like features is by depth spherical surface D1After cutting, only there is non-close song in depth point cloud Line (3)；Therefore, to depth spherical surface D1Successively cutting, until there are closed curve (2), and all non-close curves (3) are considered as Leaf (6) and disregard；
Step 4 calculates its minor axis length L to all closed curves (2), and is more than limb (7) and bamboo by all minor axis length L Object where the closed curve (2) of the maximum gauge value [L] of pole (8) is considered as fruit (4), and all minor axis length L are no more than branch Object where the closed curve (2) of the maximum gauge value [L] of dry (7) and bamboo pole (8) be considered as limb (7) or bamboo pole (8) without It gives and being further processed；
Step 5 is more than the closed curve (2) of the maximum gauge value [L] of limb (7) and bamboo pole (8), benefit to all minor axis length L The three-D profile of fruit (4) where depth point cloud constructions closed curve (2)；
Step 6 obtains the three of fruit (4) according to the one-to-one relationship of the color data of closed curve (2) and depth data The color data values ZZ of profile point cloud is tieed up, to judge according to the maturity that color data values ZZ carries out fruit (4)；
Step 7 reaches the ripening fruits (5) that picking requires to maturity, utilizes the three-D profile of coordinate pair ripening fruits (5) Space orientation is carried out, and then completes picking.
2. the quick recognition positioning method of fruit on a kind of tree based on RGB-D according to claim 1, it is characterised in that: The RGB-D sensor is can any one of synchronization gain depth data, Realsense, Kinect of color data.
3. the quick recognition positioning method of fruit on a kind of tree based on RGB-D according to claim 1, it is characterised in that: The fruit is that diameter is noticeably greater than any one of the apple of limb (7) and bamboo pole (8) diameter, tomato, peach, pears, citrus.
CN201610137746.6A 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D Active CN105825171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610137746.6A CN105825171B (en) 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610137746.6A CN105825171B (en) 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D

Publications (2)

Publication Number Publication Date
CN105825171A CN105825171A (en) 2016-08-03
CN105825171B true CN105825171B (en) 2019-02-01

Family

ID=56987778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610137746.6A Active CN105825171B (en) 2016-03-10 2016-03-10 The quick recognition positioning method of fruit on a kind of tree based on RGB-D

Country Status (1)

Country Link
CN (1) CN105825171B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106919901A (en) * 2017-01-20 2017-07-04 江苏大学 A kind of depth ball transversal method of spherefruit identification in strain
CN106875412B (en) * 2017-02-28 2020-06-23 重庆理工大学 Segmentation positioning method for two overlapped fruits
CN108064561B (en) * 2018-01-29 2020-07-14 骆通运 Automatic apple picking device and method
CN108470339A (en) * 2018-03-21 2018-08-31 华南理工大学 A kind of visual identity of overlapping apple and localization method based on information fusion
CN109584292B (en) * 2018-11-14 2022-04-19 南京农业大学 Fruit tree three-dimensional form measuring system based on Kinect is demarcation independently
CN116267226B (en) * 2023-05-16 2023-07-28 四川省农业机械研究设计院 Mulberry picking method and device based on intelligent machine vision recognition of maturity

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103065149A (en) * 2012-12-21 2013-04-24 上海交通大学 Netted melon fruit phenotype extraction and quantization method
CN103727877A (en) * 2013-12-20 2014-04-16 北京农业信息技术研究中心 Fruit identifying and locating method, device and system
CN103824324A (en) * 2014-02-28 2014-05-28 北京农业信息技术研究中心 Fruit tree crown layer leaf and fruit three-dimensional reconstruction method and system
CN104700404A (en) * 2015-03-02 2015-06-10 中国农业大学 Fruit location identification method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103065149A (en) * 2012-12-21 2013-04-24 上海交通大学 Netted melon fruit phenotype extraction and quantization method
CN103727877A (en) * 2013-12-20 2014-04-16 北京农业信息技术研究中心 Fruit identifying and locating method, device and system
CN103824324A (en) * 2014-02-28 2014-05-28 北京农业信息技术研究中心 Fruit tree crown layer leaf and fruit three-dimensional reconstruction method and system
CN104700404A (en) * 2015-03-02 2015-06-10 中国农业大学 Fruit location identification method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《基于Kinect视觉系统的西红柿自动识别与定位》;董建民 等;《中国农机学报》;20140731;第35卷(第4期);第170-171页
《苹果采摘机器人果实识别与定位方法》;司永胜 等;《农业机械学报》;20100930;第41卷(第9期);第149-150页

Also Published As

Publication number Publication date
CN105825171A (en) 2016-08-03

Similar Documents

Publication Publication Date Title
CN105825171B (en) The quick recognition positioning method of fruit on a kind of tree based on RGB-D
Tang et al. Recognition and localization methods for vision-based fruit picking robots: A review
Luo et al. A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard
Kusumam et al. 3D‐vision based detection, localization, and sizing of broccoli heads in the field
CN105718945B (en) Apple picking robot night image recognition method based on watershed and neural network
CN103310218B (en) A kind of overlap blocks fruit precise recognition method
Qiang et al. Identification of fruit and branch in natural scenes for citrus harvesting robot using machine vision and support vector machine
MX2019004247A (en) Automated pruning or harvesting system for complex morphology foliage.
Song et al. Recognition and localization methods of occluded apples based on convex hull theory
CN103544493A (en) Fruit distinguishing and locating method with laser scanning and machine vision combined
CN106951905A (en) Apple identification and localization method on a kind of tree based on TOF camera
CN103295018A (en) Method for precisely recognizing fruits covered by branches and leaves
CN110599507B (en) Tomato identification and positioning method and system
CN110610506A (en) Image processing technology-based agaricus blazei murill fruiting body growth parameter detection method
Ge et al. Three dimensional apple tree organs classification and yield estimation algorithm based on multi-features fusion and support vector machine
Changyi et al. Apple detection from apple tree image based on BP neural network and Hough transform
Changhui et al. Overlapped fruit recognition for citrus harvesting robot in natural scenes
CN103729621B (en) Plant leaf image automatic recognition method based on leaf skeleton model
CN104992448B (en) The automatic positioning method of robot antisitic defect grape-picking
Liu et al. Development of a machine vision algorithm for recognition of peach fruit in a natural scene
CN104614386A (en) Lens defects type identification method
Meng et al. The recognition of overlapping apple fruits based on boundary curvature estimation
Jin et al. Far-near combined positioning of picking-point based on depth data features for horizontal-trellis cultivated grape
Hu et al. Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN
Rahman et al. Identification of mature grape bunches using image processing and computational intelligence methods

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant