CN114793631A - Picking end effector based on visual guidance dynamic control - Google Patents
Picking end effector based on visual guidance dynamic control Download PDFInfo
- Publication number
- CN114793631A CN114793631A CN202210388993.9A CN202210388993A CN114793631A CN 114793631 A CN114793631 A CN 114793631A CN 202210388993 A CN202210388993 A CN 202210388993A CN 114793631 A CN114793631 A CN 114793631A
- Authority
- CN
- China
- Prior art keywords
- clamping arm
- end effector
- clamping
- dynamic control
- fruit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D46/00—Picking of fruits, vegetables, hops, or the like; Devices for shaking trees or shrubs
- A01D46/30—Robotic devices for individually picking crops
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Harvesting Machines For Specific Crops (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a picking end effector based on visual guidance dynamic control, and relates to the field of agricultural machinery. The invention comprises a support, a clamping arm, a driving device, a control system, a depth camera and a pressure sensor; the clamping arms comprise a first clamping arm and a second clamping arm, and the first clamping arm and the second clamping arm are driven by a driving device; the first clamping arm is provided with a cutter, and the second clamping arm is provided with the pressure sensor; the depth camera is arranged on the support and is connected with the driving device through the control system. The fruit harvester can finish harvesting of string-type or individual-type fruits at one time, has high efficiency and compact integral structure, solves the problems of shielding and overlapping, and improves the operation convenience and the safety and reliability.
Description
Technical Field
The invention relates to the field of agricultural machinery, in particular to a picking end effector based on visual guidance dynamic control.
Background
With the increase of the aging population and the reduction of labor force, intelligent automation is urgently needed in agriculture at present. To the automatic harvesting of intelligence of crops fruit, often because the branch and leaf of fruit shelter from and the overlap between the fruit, to fruit detection and fruit stalk snatch and cause very big interference, lead to plucking the failure. In the aspect of vision, most of the existing picking end effectors based on visual guidance detect picking points in front of fruits by utilizing deep learning, for the fruits with shielding and overlapping, the deep learning method cannot effectively detect the picking points, and a large amount of data is often needed for training a deep learning model, so that the calculation cost is greatly increased. In the aspect of picking the end effector, most of researches on the structure and motion path planning of the mechanical arm are carried out, and the end effector mainly comprises an air-suction type, a flexible finger type and a mechanical target holder, and cutting parts such as scissors and an electric saw. The air suction type and the flexible finger type have high requirements on positioning accuracy, and can not dynamically control the positioning accuracy in real time, and the mechanical single-closed grabbing easily damages overlapped fruits by the existing method for visually positioning picking points in front of the fruits. Therefore, there is a need to develop a picking end effector based on visual guidance dynamic control to improve the fruit harvesting efficiency, reduce the harvesting cost and realize timely harvesting of crops.
Disclosure of Invention
In view of the above, the invention provides a picking end effector based on visual guidance dynamic control, which aims to reduce the calculation cost, solve the problems of difficult picking of blocked and overlapped fruits and the like, and overcome the defects that the existing air-suction type, flexible finger type and mechanical end effectors only can perform single grabbing and are easy to grab failure and damage crops.
In order to achieve the purpose, the invention adopts the following technical scheme:
a picking end effector based on visual guidance dynamic control comprises a support, a clamping arm, a driving device, a control system, a depth camera and a pressure sensor; the clamping arms comprise a first clamping arm and a second clamping arm, and the first clamping arm and the second clamping arm are driven by a driving device; the first clamping arm is provided with a cutter, and the second clamping arm is provided with the pressure sensor; the depth camera is arranged on the support and is connected with the driving device through the control system.
Optionally, the driving device comprises a clamping stepping motor, a coupler and a screw rod; the clamping stepping motor is fixed on one side of the bracket, and a power output shaft of the clamping stepping motor is connected with the screw rod through the coupler; the first clamping arm and the second clamping arm are connected to the screw rod.
Optionally, the cutter is slidably connected to the first clamping arm, and the cutter is driven by a sliding rail stepping motor.
Optionally, the slide rail stepping motor on the first clamping arm controls the slide rail trapezoidal screw rod to move, and the slide block is used for moving the slice back and forth.
Optionally, small deep groove ball bearings are mounted at two ends of the sliding rail trapezoidal screw rod.
Optionally, two ends of the screw rod are provided with large deep groove ball bearings.
Optionally, a fan is further provided, and the fan is disposed on one side of the depth camera.
Optionally, the control system obtains the contour information of the bottom of the fruit by using a Canny operator, performs polygonal curve fitting on the contour to obtain edge points, and uses the contour points of the bottom as initial state values for the kalman filter to predict the state value at the next moment.
Optionally, the control system positions the fruit stem picking point by using depth information through a set constraint condition, the fruit stems are clamped by the clamping arms, the clamping force is given by the pressure sensor on the second clamping arm, and when the clamping force reaches a preset value, the driving device drives the cutter on the first clamping arm to move.
Compared with the prior art, the picking end effector based on the visual guidance dynamic control can finish harvesting of string-type or individual-type fruits at one time, is high in efficiency and compact in overall structure, solves the problems of shielding and overlapping, and improves operation convenience and safety reliability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic structural view of an end effector of the present invention;
FIG. 2 is a top view of an end effector of the present invention;
FIG. 3 is a flow chart of the visual guidance dynamic control of the present invention;
FIG. 4 is a flowchart illustrating operation of the end effector of the present invention;
FIG. 5 is a diagram of a control system for an end effector of the present invention;
in the figure: the device comprises a clamping stepping motor 1, a coupler 2, a bearing end cover 3, a support 4, a depth camera 5, a fan 6, a clamping trapezoidal screw 7, a cutting stepping motor 8, a circular saw tooth cutter 9, a sliding block 10, a sliding rail trapezoidal screw 11, a sliding rail stepping motor 12, a right clamping arm 13, a pressure sensor 14, a left clamping arm 15, a large deep groove ball bearing 16 and a small deep groove ball bearing 17.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a picking end effector based on visual guidance dynamic control, which is shown in figures 1-5, wherein the picking end effector comprises a clamping stepping motor 1, a coupler 2, a bearing end cover 3, a support 4, a depth camera 5, a fan 6, a clamping trapezoidal screw rod 7, a cutting stepping motor 8, a circular saw tooth cutter 9, a slide block 10, a slide rail trapezoidal screw rod 11, a slide rail stepping motor 12, a right clamping arm 13, a pressure sensor 14, a left clamping arm 15, a large deep groove ball bearing 16 and a small deep groove ball bearing 17, and specifically comprises the following steps:
(1) the device comprises a support, a depth camera, a left clamping arm, a right clamping arm, a coupler, a fan, a clamping stepping motor, a cutting stepping motor, a sliding rail stepping motor, a sliding block, a clamping trapezoidal screw rod, a sliding rail trapezoidal screw rod, a large deep groove ball bearing, a small deep groove ball bearing, a circular serrated knife and a pressure sensor;
(2) the clamping stepping motor is fixed on the right side of the bracket, a power output shaft of the clamping stepping motor is connected with the clamping trapezoidal screw rod through a coupler, deep groove ball bearings are arranged at two ends of the clamping trapezoidal screw rod, a left clamping arm and a right clamping arm are connected to the clamping trapezoidal screw rod, a rotating pair for clamping the trapezoidal screw rod is converted into a moving pair of the clamping arms, and the trapezoidal screw rod is self-locked, so that the clamping stability can be greatly improved;
(3) the upper end of the support is fixedly provided with a depth camera and a fan; a pressure sensor is arranged at the finger position of the right clamping arm; a slide rail stepping motor on the left clamping arm controls the motion of a slide rail trapezoidal screw rod, and the circular saw blade moves back and forth through a slide block;
(4) the large deep groove ball bearings are arranged at the two ends of the clamping trapezoidal screw rod, and the small deep groove ball bearings are arranged at the two ends of the sliding rail trapezoidal screw rod, so that the transmission stability can be ensured;
(5) the center of the base circle of the circular saw tooth cutter is fixedly connected with an output shaft of the cutting stepping motor; the clamping stepping motor and the cutting stepping motor are controlled by a controller;
(6) the cutting range of the circular serrated knife is slightly larger than the maximum diameter of the main stem of the fruit.
(7) When the visual system of the picking end effector detects the position of the bottom of the fruit, the mechanical arm drives the end effector to be close to the bottom of the fruit, and the safety distance between the support and the fruit is controlled by utilizing the depth information.
(8) The real-time contour prediction algorithm can dynamically control the opening and closing size of the end effector, and the end effector is prevented from damaging target fruits while overlapped fruits are pushed away.
(9) The start of the fan needs whether the real-time contour prediction algorithm finds that the occlusion exists, and when the occlusion of the leaves exists, the fan is started to blow the leaves away.
(10) The sensor detects whether the fruit stalks need to enter the clamping position of the end effector, if the fruit stalks enter the clamping position, the clamping stepping motor rotates to drive the left clamping arm and the right clamping arm to be closed, meanwhile, the pressure sensor detects the pressure on the fruit stalks, and when the pressed fruit stalks reach the preset minimum damage force (the force which does not damage the fruit stalks and ensures that the fruits cannot slide off from the paws under the action of gravity), the clamping stepping motor stops rotating and keeps the current state.
(11) Then the slide rail stepping motor and the cutting stepping motor rotate simultaneously to drive the circular serrated knife to move towards the fruit stalks, and the fruit stalks are cut off. The fruit can be picked without damage.
(12) The lowest point of the fruit is located through the depth information acquired by the depth camera, the end effector is guided to move upwards from the bottom of the fruit, the fruit is generally wide at the top and small at the bottom, so that the bottom is not influenced by overlapping, and the end effector is prevented from damaging the overlapped fruit by starting to move from the bottom.
(13) Real-time contour prediction algorithm: firstly, contour information of the bottom of a fruit is obtained by using a Canny operator, polygonal curve fitting is carried out on the contour to obtain edge points, the contour points at the bottom are used as initial state values to be used for a Kalman filter to predict the state value of the next moment, and the specific principle is shown in formulas (2) to (4).
(14) And (3) predicting the contour in real time by using a Kalman filter, and comparing the contour with the contour obtained by a Canny operator at the same moment so as to judge whether the prediction is abnormal or not and whether the leaf is shielded or not.
(15) The depth camera is arranged above the support, the path of the end effector of the fruit can be planned in advance according to the outline of the fruit through a visual algorithm of real-time outline prediction, the opening and closing size of the clamping arm can be dynamically controlled in real time through a clamping mechanism consisting of a stepping motor and a trapezoidal screw rod at the right end of the support as well as a left clamping arm and a right clamping arm, so that overlapped fruits can be pushed away while crops are prevented from being damaged, and the influence of overlapping factors on picking is avoided;
(16) when the contour prediction is abnormal, the upper part can be judged to be shielded by the leaves, and the fan above the support can blow the leaves open, so that the shielding of the leaves is avoided;
(17) the fruit stem picking points are positioned by utilizing depth information through set constraint conditions, fruit stems are clamped by the clamping mechanism, the clamping force is given by the pressure sensor on the finger of the right clamping arm, when the clamping force reaches a preset value, the sliding rail motor on the left clamping arm drives the sliding block to move forwards, and meanwhile, the cutting motor is started, so that the circular serrated knife cuts off the fruit stems, and picking of fruits is realized.
The working principle of the embodiment is as follows: the fruit picking robot is used for accurately picking overlapped and shielded fruits, the bottom position of the fruit to be picked is detected by a visual system of a depth camera 5 of the picking robot in real time to obtain the depth information of the fruit, and the end effector is driven by the mechanical arm to reach a certain distance below the bottom of the fruit in the range that the support 4 can not collide with the fruit. The gripper is composed of a left clamping arm and a right clamping arm, the depth camera 5 is installed above the support 4, the fruit outline of the gripper can be predicted in advance in real time, the distance between the marginal point and the central axis of the fruit is calculated, the mechanical arm is controlled in real time to drive the end effector to move from bottom to top, and the gripper stepper motor 1 is dynamically guided to dynamically control the opening and closing size of the gripper. When the contour prediction is abnormal, the upper part is judged to be shielded by the leaves, and the fan 6 is started to blow the leaves. When the sudden change of the outline width is detected, the fruit stalks can be positioned by using the formula (1) in combination with the depth information, then the fruit stalks can enter a grabbing state, the clamping stepping motor 1 rotates to control the closing of the paw, the fruit stalk clamping action is completed, when the pressure measured by the pressure sensor 14 reaches the preset minimum fruit stalk damage force, the paw is kept at the current position, then the slide rail stepping motor 12 on the left clamping arm 15 is started to drive the slide block 10 to move towards the fruit stalk direction, and meanwhile, the cutting stepping motor 8 on the slide block 10 drives the circular serrated knife 9 to rotate at a high speed, so that the fruit stalk shearing action is completed. After the shearing is completed, the sliding rail stepping motor 12 resets, the cutting stepping motor 8 stops rotating, the fan 6 stops, the mechanical arm drives the end effector to move to the fruit collecting device, the gripper releases fruits, the fruit picking action is completed, if the fruit picking quantity is not zero, the picking action of the next fruit is started, if the fruit picking quantity is zero, each part resets, and the picking is finished.
D f ＝R+d c ±Δd (1)
Wherein D f The depth range of the fruit stem is shown, R is the maximum radius of the fruit, dc is the nearest distance from the camera to the fruit, and deltad is an error value.
X(k+1)＝A(k)X(k)+T(k)W(k) (2)
Z(k)＝H(k)X(k)+V(k) (3)
Wherein, X (k) represents the state value at the time k, X (k +1) represents the state value at the time k +1, a (k) is a state transition matrix, t (k) is an interference transition matrix, w (k) represents the system state noise of the motion model, z (k) represents the observation vector, h (k) is an observation matrix, v (k) represents the observation noise, (X (k)) represents the observation noise i ，y i ) Representing the position of the actual trace point, (x) i '，y i ') indicates position information of the predicted trace points, and k indicates the number of predicted trace points.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A picking end effector based on visual guidance dynamic control is characterized by comprising a support, a clamping arm, a driving device, a control system, a depth camera and a pressure sensor; the clamping arms comprise a first clamping arm and a second clamping arm, and the first clamping arm and the second clamping arm are driven by a driving device; the first clamping arm is provided with a cutter, and the second clamping arm is provided with the pressure sensor; the depth camera is arranged on the support and is connected with the driving device through the control system.
2. The picking end effector based on visual guidance dynamic control of claim 1, wherein the driving device comprises a gripping stepper motor, a coupling, a lead screw; the clamping stepping motor is fixed on one side of the bracket, and a power output shaft of the clamping stepping motor is connected with the screw rod through the coupler; the first clamping arm and the second clamping arm are connected to the screw rod.
3. The picking end effector based on visual guidance dynamic control of claim 1, wherein the cutter is slidably connected to the first gripper arm, the cutter being driven by a sled stepper motor.
4. The picking end effector based on visual guidance dynamic control according to claim 3, wherein a slide rail stepping motor on the first clamping arm controls the motion of a slide rail trapezoidal screw rod, and the slide block is used for realizing the back and forth movement of the slicing.
5. The picking end effector based on visual guidance dynamic control according to claim 4, characterized in that small deep groove ball bearings are mounted at two ends of a slide rail trapezoidal screw rod.
6. The picking end effector based on visual guidance dynamic control according to claim 1, characterized in that both ends of the screw rod are provided with large deep groove ball bearings.
7. The picking end effector based on visual guidance dynamic control according to claim 1, characterized in that a fan is further provided, and the fan is arranged on one side of the depth camera.
8. The picking end effector based on the vision-guided dynamic control is characterized in that the control system acquires contour information of the bottom of a fruit by using a Canny operator, performs polygonal curve fitting on the contour to obtain edge points, and uses the contour points of the bottom as initial state values to be used by a Kalman filter for predicting the state values at the next moment.
9. The picking end effector based on the vision-guided dynamic control of claim 1, wherein the control system uses depth information to position fruit stem picking points through set constraints, fruit stems are clamped by the clamping arms, the clamping force is given by a pressure sensor on the second clamping arm, and when the clamping force reaches a preset value, the driving device drives the cutter on the first clamping arm to move.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210388993.9A CN114793631B (en) | 2022-04-13 | 2022-04-13 | Picking end effector based on visual guidance dynamic control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210388993.9A CN114793631B (en) | 2022-04-13 | 2022-04-13 | Picking end effector based on visual guidance dynamic control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114793631A true CN114793631A (en) | 2022-07-29 |
| CN114793631B CN114793631B (en) | 2023-04-18 |
Family
ID=82536918
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210388993.9A Active CN114793631B (en) | 2022-04-13 | 2022-04-13 | Picking end effector based on visual guidance dynamic control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114793631B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115413479A (en) * | 2022-09-27 | 2022-12-02 | 江苏大学 | Water shield picking end effector based on depth vision and picking method thereof |
| CN116686545A (en) * | 2023-06-29 | 2023-09-05 | 佛山市中科农业机器人与智慧农业创新研究院 | Litchi picking robot shade removing method based on machine vision control |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2760595A1 (en) * | 1997-03-13 | 1998-09-18 | Tecnoma | Harvesting bunches of grapes |
| CN108401685A (en) * | 2018-05-08 | 2018-08-17 | 湖州佳创自动化科技有限公司 | A kind of spheral fruit picking robot |
| EP3539735A1 (en) * | 2018-03-13 | 2019-09-18 | Soluciones Robóticas Agrícolas S.L. | Robotic arm end effector for fruit harvesting |
| CN112715163A (en) * | 2021-02-02 | 2021-04-30 | 广州大学 | Fruit picking robot |
| CN114175927A (en) * | 2021-11-29 | 2022-03-15 | 季华实验室 | Cherry tomato picking method and cherry tomato picking manipulator |
-
2022
- 2022-04-13 CN CN202210388993.9A patent/CN114793631B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2760595A1 (en) * | 1997-03-13 | 1998-09-18 | Tecnoma | Harvesting bunches of grapes |
| EP3539735A1 (en) * | 2018-03-13 | 2019-09-18 | Soluciones Robóticas Agrícolas S.L. | Robotic arm end effector for fruit harvesting |
| CN108401685A (en) * | 2018-05-08 | 2018-08-17 | 湖州佳创自动化科技有限公司 | A kind of spheral fruit picking robot |
| CN112715163A (en) * | 2021-02-02 | 2021-04-30 | 广州大学 | Fruit picking robot |
| CN114175927A (en) * | 2021-11-29 | 2022-03-15 | 季华实验室 | Cherry tomato picking method and cherry tomato picking manipulator |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115413479A (en) * | 2022-09-27 | 2022-12-02 | 江苏大学 | Water shield picking end effector based on depth vision and picking method thereof |
| CN115413479B (en) * | 2022-09-27 | 2024-03-22 | 江苏大学 | Deep vision-based water shield picking end effector and picking method thereof |
| CN116686545A (en) * | 2023-06-29 | 2023-09-05 | 佛山市中科农业机器人与智慧农业创新研究院 | Litchi picking robot shade removing method based on machine vision control |
| CN116686545B (en) * | 2023-06-29 | 2024-01-16 | 佛山市中科农业机器人与智慧农业创新研究院 | Litchi picking robot shade removing method based on machine vision control |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114793631B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114793631B (en) | Picking end effector based on visual guidance dynamic control | |
| CN107214716B (en) | Tomato fruit string picking end effector and picking method thereof | |
| CN102550216B (en) | End effector of intelligent harvesting robot | |
| CN207476246U (en) | A kind of ridge culture strawberry picking robot | |
| CN109496563A (en) | A kind of fruit picking robot | |
| WO2009128712A1 (en) | Method and device for the removal of a leaf from a crop | |
| US20210368680A1 (en) | Real-time speed adjustment for harvesting crops | |
| CN109352673A (en) | A kind of fruit picking end effector | |
| CN113843810A (en) | Machine vision-based quasi-spherical fruit picking robot and fruit picking method thereof | |
| CN112715163A (en) | Fruit picking robot | |
| JP5407020B2 (en) | Fruit harvesting robot | |
| CN108702926B (en) | Use method of intelligent pineapple picking robot | |
| CN208675820U (en) | A kind of terminal executor of picking robot | |
| EP4140286A1 (en) | Crop harvesting method, crop harvesting device, and crop harvesting system | |
| CN108370740A (en) | A kind of topping end effector and the topping machine containing the actuator | |
| KR102511629B1 (en) | End Effector for Crop Implements | |
| CN109121799B (en) | Reciprocating tool and control method thereof | |
| CN106856849A (en) | A kind of barrier induction installation and avoidance mechanism | |
| CN108064561B (en) | Automatic apple picking device and method | |
| Hu et al. | Design and experiment of a new citrus harvesting robot | |
| CN215530076U (en) | Plum tree fruit picking equipment | |
| CN108370746B (en) | Guide cutter structure for tree obstacle clearing aerial robot and control method | |
| CN210746116U (en) | Fruit stalk clamping and shearing type picking robot end effector | |
| CN219311289U (en) | End of general agricultural picking robot | |
| CN111406502A (en) | Lawn mower with function is collected to grass bits |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |