CN109202848A - Man-machine collaboration picking robot and its control method based on Leap Motion and machine vision - Google Patents
Man-machine collaboration picking robot and its control method based on Leap Motion and machine vision Download PDFInfo
- Publication number
- CN109202848A CN109202848A CN201811059487.5A CN201811059487A CN109202848A CN 109202848 A CN109202848 A CN 109202848A CN 201811059487 A CN201811059487 A CN 201811059487A CN 109202848 A CN109202848 A CN 109202848A
- Authority
- CN
- China
- Prior art keywords
- picking
- machine
- mechanical arm
- camera
- leap motion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J5/00—Manipulators mounted on wheels or on carriages
- B25J5/005—Manipulators mounted on wheels or on carriages mounted on endless tracks or belts
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D45/00—Harvesting of standing crops
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J15/00—Gripping heads and other end effectors
- B25J15/0052—Gripping heads and other end effectors multiple gripper units or multiple end effectors
- B25J15/0066—Gripping heads and other end effectors multiple gripper units or multiple end effectors with different types of end effectors, e.g. gripper and welding gun
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/10—Programme-controlled manipulators characterised by positioning means for manipulator elements
- B25J9/12—Programme-controlled manipulators characterised by positioning means for manipulator elements electric
- B25J9/126—Rotary actuators
Abstract
The invention discloses man-machine collaboration picking robots and its control method based on Leap Motion and machine vision, including walking mechanism, picking mechanical arm, power supply is fixedly installed in the walking mechanism, basket and robot controller are collected in picking, the picking mechanical arm is arranged in walking mechanism, picking mechanical arm is provided on the picking mechanical arm, it further include remote control system, the remote control system includes computer and Leap Motion body-sensing sensor, the Leap Motion body-sensing sensor is connect with dataphone, the computer and robot controller communication connection；The picking mechanical is provided with camera, the camera and robot controller communication connection on hand, and the camera and robot controller are electrically connected to a power source.The present invention can recognize picking object using the outstanding vision system of people under complex background environment, and remotely control picking robot movement by contactless body-sensing sensor and pick to fruit.
Description
Technical field
The present invention relates to modern agriculture equipment technical fields, more particularly to the people based on Leap Motion and machine vision
Machine cooperation picking robot and its control method.
Background technique
In application industry, fruit picking harvest about accounts for the 50% of entire industry workload, due to fruit picking
Complexity in the process, so most of fruit industry area is in fruit picking still based on handwork；For certain kinds
The fruit of class partially realizes mechanized harvesting operation, but automatization level is still lower.Intelligent picking fruit is knot
It closes intelligent and automatic technology and realizes that the intelligent picking of fruit improves operating efficiency to reduce manual labor amount, wherein
Fruit identifies and positions mainly based on machine vision technique.
In intelligent picking fruit technical field, application No. is the patent application documents of CN201710857174.3 to disclose
A kind of intelligent fruit picking robot based on machine vision comprising walking mechanism, picking mechanical arm, vision system, control
System, power-supply system, control system are connect with walking mechanism, picking mechanical arm, vision system, power-supply system respectively, control system
System, power-supply system are fixedly connected with walking mechanism, and picking mechanical arm is four-degree-of-freedom articulated robot and passes through pedestal and row
It walks mechanism to be fixedly connected, the visual sensor of vision system is fixedly installed on above the wrist of picking mechanical arm, vision system
Visual spatial attention module is connect integrally and with visual sensor signal with control system integration, and visual spatial attention module is used to be based on image
The full convolutional network image processing techniques of segmentation carries out picking target fixation and recognition, vision to the image that visual sensor transmission comes
Control module will pick targeting information and be transmitted to control system, and control system control walking mechanism is mobile and/or manipulator is plucked
Water intaking fruit is simultaneously put into fruit collection device.
Above-mentioned patent automatically controls walking mechanism adjustment position and picking mechanical according to the fruit location information of vision system
The picking of hand there is problems although realizing the intelligent picking of fruit: due on fruit tree fruit growth with
Machine and complexity, i.e. fruit may it is single as a result, may also cluster as a result, according to the Machine Vision Recognition algorithm of above-mentioned patent
It is also difficult to differentiate between blade blocks fruit, branch blocks and mutually blocks or be overlapped between fruit and fruit etc. under complex backgrounds
Fruit.
Summary of the invention
In view of this, the object of the present invention is to provide the man-machine collaboration picking machines based on Leap Motion and machine vision
Device people and its control method can recognize picking object using the outstanding vision system of people under complex background environment, and pass through
Contactless body-sensing sensor remotely controls picking robot movement and picks to fruit.
The present invention solves above-mentioned technical problem by following technological means:
Man-machine collaboration picking robot based on Leap Motion and machine vision, including walking mechanism, picking mechanical
Arm, power supply is fixedly installed in the walking mechanism, basket and robot controller are collected in picking, and the picking mechanical arm setting exists
In walking mechanism, it is provided with picking mechanical arm on the picking mechanical arm, wherein further include remote control system, the long-range behaviour
Control system includes computer and Leap Motion body-sensing sensor, and the Leap Motion body-sensing sensor and dataphone connect
It connects, the computer and robot controller communication connection；The picking mechanical is provided with camera, the camera and robot on hand
Controller communication connection, the camera and robot controller are electrically connected to a power source.Pass through Leap Motion body-sensing sensor
Angle information is rolled to palm coordinate information, hand pitch angle information, hand, clicks gesture, horizontal slip gesture, click screen gesture
Etc. information be acquired, palm coordinate information is the coordinate system under Leap Motion body-sensing sensor, according to picking mechanical arm
Motion range, the conversion of two coordinate systems is carried out by the method for ratio enlargement, palm is sensed in Leap Motion body-sensing
Coordinate transformation on device is the coordinate at picking robot manipulator clamping midpoint on picking mechanical arm platform, passes through picking machine
Tool arm inverse kinematics go out the rotation angle of picking mechanical arm.According to turning for the equivalent corresponding control picking mechanical arm of hand pitch angle
Angle rolls the corner of the equivalent corresponding control picking mechanical arm of angle information according to hand.It clicks gesture control picking and unloads fruit integration
Operation, horizontal slip gesture control platform vehicle travel forward, and to adjust fruit number and the position of camera fields of view kind, click screen
Gesture control Leap Motion body-sensing sensor starts to acquire palm coordinate information, and opens the control of picking mechanical arm.
Further, the picking mechanical arm includes rotating platform, large arm and forearm, is fixedly connected on the rotating platform
Being hinged and connected for shoulder joint motor, the shoulder joint motor and large arm, is fixedly connected with elbow joint motor on the forearm, described
The one end of large arm far from rotating platform is hinged and connected with elbow joint motor, and the shoulder joint motor and elbow joint motor are and machine
The electrical connection of people's controller.Rotating platform can make picking mechanical upper-arm circumference rotate around walking mechanism, large arm, forearm, shoulder joint economize on electricity
The setting of machine and elbow joint motor can make picking mechanical arm carry out the tune of height and far and near distance to picking mechanical arm
Section, expands the picking job area of picking mechanical arm.
Further, the rotating platform includes pedestal upper plate and pedestal lower plate, and the pedestal lower plate is fixedly connected on walking
In mechanism, it is fixedly connected with support column between the pedestal upper plate and pedestal lower plate, is fixedly installed with and turns in the pedestal lower plate
Motor is moved, rotatable platform is rotatably connected on the pedestal upper plate, the output shaft of the rotary electric machine passes through pedestal upper plate and turns
Moving platform is fixedly connected, and output shaft and the pedestal upper plate of the rotary electric machine are slidably connected, and the shoulder joint motor is fixedly mounted
On rotatable platform, the rotary electric machine is electrically connected with robot controller.Pedestal upper plate, pedestal lower plate and support column are for branch
Picking mechanical arm is supportted, rotary electric machine is for driving rotatable platform to be rotated, and then the rotation for driving picking mechanical arm to carry out 360 °
Turn.
Further, the picking mechanical arm includes mounting bracket, and the forearm is fixedly connected with wrist far from one end of large arm
Joint motor, the side of the mounting bracket are fixedly connected with wrist connecting plate, and the other side of the mounting bracket is fixedly connected with
Clamping manipulator, the wrist connecting plate are hinged and connected with wrist joint motor；Camera installation is fixedly connected in the mounting bracket
Plate, the camera are mounted in camera installing plate, and the wrist joint motor is electrically connected with robot controller.In picking, folder
Manipulator is held for clamping fruit, wrist joint motor can further adjust the position between clamping manipulator and fruit.
Further, wrist rotary electric machine, the output shaft and peace of the wrist rotary electric machine are fixedly installed on the wrist connecting plate
Dress bracket is fixedly connected, and the wrist rotary electric machine is electrically connected with robot controller.By the setting of wrist rotary electric machine, can make
It obtains picking mechanical arm and carries out rotation inclination, in this way, the fruit picking of different angle can be applicable in.
Further, be fixedly installed with cutting motor in the mounting bracket above the clamping manipulator, cutting motor it is defeated
Shaft is fixedly connected with link block, and the link block is fixed on the link block between cutting motor and clamping manipulator
There are cutter, the cutting motor is electrically connected with robot controller.When picking is such as citrus fruit, generally by cutting
The mode of the carpopodium of disconnected fruit is picked, and cutting motor is arranged in the present invention above clamping manipulator, and cutting motor drive is cut
The rotation of cutter, and then carpopodium is cut off, therefore, the present invention is adapted to the fruit of picking more multi items.
Further, the clamping manipulator includes connecting rod, the first clamping jaw and the second clamping jaw, and the connecting rod is fixedly connected on peace
It fills on bracket, first clamping jaw and the second clamping jaw are slidably connected on connecting rod, are fixedly connected with manipulator on the connecting rod
Driving motor, the robot drives motor are electrically connected with robot controller.When clamping and unloading fruit, robot drives motor
Link motion is driven, connecting rod drives the side-to-side movement on connecting rod of the first clamping jaw and the second clamping jaw, and then controls clamping manipulator
It opens and closes.
Further, the camera installing plate is equipped with annular groove and collar mounting holes, is fixed with camera on the camera
Fixed plate is connected with fastening screw between the camera fixed plate and annular groove and collar mounting holes.When in use, it unclamps tight
Gu screw can be such that camera rotates around annular center hole, thus the visual field of the adjustable camera on ground.
Further, the walking mechanism is track-type work platform truck.Walking mechanism uses caterpillar operation platform cart,
So that work capacity of the robot under the rough intricately condition such as mountain area, hills greatly improves.
The control method of the picking robot, includes the following steps,
S1, camera acquire fruit tree picture in real time and are identified and handled to fruit, and Leap Motion body-sensing passes
Sensor identifies that palm hand gesture carries out track-type work platform truck position and adjusts；
S2, computer are acquired palm coordinate on Leap Motion body-sensing sensor and palm hand gesture information, pass through machine
Tool arm inverse kinematics go out picking mechanical arm rotary electric machine, shoulder joint motor, elbow joint motor, wrist joint motor, wrist rotation
The rotation angle of motor；
S3, the movement of picking mechanical arm, camera carry out template matching, the Clamping Center of picking mechanical arm to present viewing field fruit
Point coordinate is overlapped with fruit center point coordinate, and picking mechanical arm picks fruit；
S4, picking mechanical arm control picking mechanical arm and move to picking collection basket position from fruit position is currently plucked, and unload fruit.
Beneficial effects of the present invention:
(1) present invention is by man-machine collaboration, is worked continuously ability using the outstanding vision system of people and machine, can solve to hide
The problem of fruit is difficult under the complex background environments such as gear, overlapping；
(2) present invention use the contactless robot control mode of Leap Motion, reduce motion mode, mostly freely
The complexity for spending robot control can control a set of picking of picking mechanical people completion by the movement of small space hand and gesture and make
Industry；
(3) present invention realizes that target is fixed using Leap Motion control in such a way that machine vision automatic identification combines
Automation picking after level is true reduces artificial later period repetition picking movement so that the intelligence degree of man-machine collaboration can be improved
Intervention；
(4) present invention carries out real-time matching processing to selection target using camera, can when matching target and being determined as fruit
It realizes automatic picking, and completes that machine vision can so be greatly reduced to the place of entire image from human assistance to automatic picking
Reason can reduce the complexity of vision algorithm, improve treatment effeciency；
(5) present invention is manually remotely detected and is manipulated by Leap Motion and realizing of Robot Vision, is made without scene
Industry mitigates the working strength of fruit picking；
(6) present invention is applicable not only to common fruit picking, and applies also for the water picked by cutting off carpopodium
Fruit, the scope of application are wider.
Detailed description of the invention
Fig. 1 is the man-machine collaboration picking robot schematic diagram the present invention is based on leapmotion and machine vision；
Fig. 2 is picking mechanical arm and picking mechanical arm partial enlarged view in Fig. 1；
Fig. 3 is man-machine collaboration picking robot init state position (preparing picking) figure；
Fig. 4 is that man-machine collaboration picking robot unloads fruit state position figure；
Fig. 5 is 3 kinds of control gesture figures of man-machine collaboration picking robot；
Fig. 6 is hand gestures and motion planning and robot control corresponding relationship；
Fig. 7 is man-machine collaboration picking robot motion control flow chart；
Wherein, 1, computer；2, Leap Motion body-sensing sensor；3, robot controller；4, track-type work platform
Vehicle；5, basket is collected in picking；6, picking mechanical arm；7, forearm；8, elbow joint motor；9, large arm；10, shoulder joint motor；11, it rotates
Platform；12, pedestal upper plate；13, rotary electric machine；14, pedestal lower plate；15, camera；16, camera installing plate；17, cutting motor；
18, cutter；19, link block；20, clamping manipulator；21, robot drives motor；22, wrist rotary electric machine；23, wrist connects
Plate；24, wrist joint motor；25, support column；26, connecting rod；27, the first clamping jaw；28, the second clamping jaw；29, mounting bracket；30, ring
Shape mounting hole, 31, camera fixed plate.
Specific embodiment
Below with reference to attached drawing, the present invention is described in detail:
As shown in figs. 1-7:
Man-machine collaboration picking robot based on Leap Motion and machine vision, including track-type work platform truck 4,
Remote control system and picking mechanical arm 6 are equipped with generator by fastening bolt on track-type work platform truck 4 and (do not mark
Out), basket 5 is collected in picking and robot controller 3, generator are electrically connected with robot controller 3.
Remote control system includes computer 1 and Leap Motion body-sensing sensor 2, Leap Motion body-sensing sensor 2
GPRS communication connection is passed through by data line communication connection, computer 1 and robot controller 3 with computer 1.
Picking mechanical arm 6 includes pedestal upper plate 12, pedestal lower plate 14, large arm 9 and forearm 7, and pedestal lower plate 14 passes through fastening
Bolt is fixedly connected on track-type work platform truck 4, and support column 25, base are welded between pedestal upper plate 12 and pedestal lower plate 14
Rotary electric machine 13 is fixedly installed with by fastening bolt in seat lower plate 14, rotatable platform 11 is rotatably connected on pedestal upper plate 12,
The output shaft of rotary electric machine 13 passes through pedestal upper plate 12 and is fixedly connected with rotatable platform 11 by flat key, the output of rotary electric machine 13
Axis is slidably connected with pedestal upper plate 12, is fixedly installed with shoulder joint motor 10, shoulder joint by fastening bolt on rotatable platform 11
Motor 10 and the lower end of large arm 9 are hinged and connected, and are fixedly connected with elbow joint motor 8 by fastening bolt on forearm 7, large arm 9
Upper end is hinged and connected with the elbow joint motor 8 on forearm 7, and the upper end of forearm 7 is fixedly connected with wrist joint electricity by fastening bolt
Machine 24, is hinged with wrist connecting plate 23 on wrist joint motor 24, is fixedly connected with wrist rotation by fastening bolt on wrist connecting plate 23
Motor 22, the output shaft of wrist rotary electric machine 22 pass through wrist connecting plate 23 and are fixedly connected with mounting bracket 29 by fastening bolt.
It is fixedly connected with connecting rod 26 in mounting bracket 29, the first clamping jaw 27 and the second clamping jaw are slidably connected on connecting rod 26
28,26 lower part of connecting rod is fixedly connected with robot drives motor 21 by fastening bolt, in the mounting bracket 29 of 26 top of connecting rod
It is fixedly installed with cutting motor 17, the output shaft of cutting motor 17 is fixedly connected with link block 19, link block by fastening bolt
19 two sides are fixed with cutter 18 using screw.
Integrally formed with camera installing plate 16 in the mounting bracket 29 of 17 top of cutting motor, camera installing plate 16 is equipped with
Annular groove and collar mounting holes 30, camera fixed plate 31 is fixed on camera 15, and camera 15 and the communication of robot controller 3 connect
It connects, is connected with fastening screw between camera fixed plate 31 and annular groove and collar mounting holes 30.Shoulder joint motor 10, elbow joint
Motor 8, rotary electric machine 13, wrist joint motor 24, wrist rotary electric machine 22, cutting motor 17, robot drives motor 21 are and machine
Device people controller 3 is electrically connected.
Control method and fruit picking process of the invention is as follows:
By Leap Motion body-sensing sensor 2 to palm coordinate information, hand pitch angle information, hand roll angle information,
It clicks the information such as gesture, horizontal slip gesture, click screen gesture to be acquired, palm coordinate information is in Leap Motion
Coordinate system under body-sensing sensor 2 carries out two seats by the method for ratio enlargement according to the motion range of picking mechanical arm 6
Coordinate transformation of the palm on Leap Motion body-sensing sensor 2 is in picking robot manipulator clamping by the conversion for marking system
Coordinate of the midpoint on 6 platform of picking mechanical arm, goes out mechanical arm rotary electric machine 13, shoulder joint by mechanical arm inverse kinematics
The rotation angle of motor 10 and elbow joint motor 8.According to turning for the equivalent corresponding control mechanical wrist joint motor 24 of hand pitch angle
Angle rolls the corner of the equivalent corresponding control mechanical wrist rotary electric machine 22 of angle information according to hand.It clicks gesture control picking and unloads
Fruit institutional operation, horizontal slip gesture control platform vehicle travel forward, to adjust fruit number and the position in 15 visual field of camera
It sets, clicks screen gesture control Leap Motion body-sensing sensor 2 and start to acquire palm coordinate information, and open picking mechanical
Arm 6 controls.
The control flow of each link is not as shown in fig. 7, preparing the picking stage, i.e. track-type work platform truck 4 moves also
When to designated position, track-type work platform truck 4 in the ranks travels forward along orchard, while picking mechanical arm 6 is in initial position,
It is and vertical with 4 direction of motion of track-type work platform truck as shown in figure 3, mechanical arm faces fruit tree, camera 15 to fruit tree picture into
Row acquires in real time and is identified and handled to fruit, when fruit object pixel quantity is greater than first threshold, illustrates camera 15
There is sufficient amount of fruit in the visual field, it is possible to implement picking operation, track-type work platform truck 4 stops at this time, and opens gesture
Detection, the position that operator judges that track-type work platform truck 4 stops according to the real-time pictures that camera 15 passes on computer 1 are
It is no to can be used as picking point.
When needing to adjust picking robot picking position forward, operator uses horizontal slip gesture, such as Fig. 5 a, control
Robot processed travels forward corresponding distance.When operator thinks that this point can be picked, implements to click screen gesture, such as scheme
5c, computer 1 starts to acquire palm coordinate information on Leap Motion body-sensing sensor 2 at this time, and passes through coordinate transformation and machinery
Arm the computation of inverse- kinematics is converted into the rotation angle of mechanical arm rotary electric machine 13, shoulder joint motor 10 and elbow joint motor 8, will
Angle information is sent to robot controller 3, and robot controller 3 controls rotary electric machine 13, shoulder joint motor 10 and elbow joint
8 corresponding sports of motor.
While picking mechanical arm is close to crop, camera 15 carries out template matching to present viewing field fruit, works as matching degree
Greater than matching threshold, illustrate that camera 15 can identify current picking object completely, then calculates fruit coordinate, and automatically control and adopt
Robot alignment fruit is plucked, simultaneously closes off Leap Motion body-sensing sensor 2 to palm information collection, and to artificially to adopting
6 motion control of mechanical arm is plucked, when the Clamping Center of picking mechanical arm point coordinate is overlapped with fruit center point coordinate, illustrates picking machine
Fruit movement can be picked and be unloaded at this time to tool hands movement in place, and the sequence for picking and unloading fruit movement is robot drives motor 21
Rotation, the first clamping jaw 27 and the second clamping jaw 28 closure, wrap fruit, and cutting motor 17 drives cutter 18 to rotate, cuts off fruit
Stalk.
After having picked, picking mechanical arm 6 controls picking mechanical arm and moves to picking collection basket 5 from fruit position is currently plucked
It sets, as shown in figure 4, robot drives motor 21 rotates, the first clamping jaw 27 and the second clamping jaw 28 are opened, and fruit falls into picking and collects
In basket 5, last picking mechanical arm 6 returns to initial position, which completes the picking to a fruit, and subsequent picking is according to this
Picking process recycles single picking.In picking, sponge can also be all provided in the inside of the first clamping jaw 27 and the second clamping jaw 28
Layer, can be to avoid the upper fruit rinds of folder.
Wherein, it will appear camera 15 in picking process and identify the case where matching degree of fruit is consistently less than matching threshold, this
When the Clamping Center point of picking mechanical arm can be essentially coincided with fruit center by artificial hand mobile operating completely, then lead to
Click gesture is crossed, as shown in Figure 5 b, issues picking fruit order, realization is picked and unloaded fruit automatically, simultaneously closes off Leap Motion
Body-sensing sensor 2 is to palm information collection, and to 6 motion control of picking mechanical arm.
The above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although referring to preferred embodiment to this hair
It is bright to be described in detail, those skilled in the art should understand that, it can modify to technical solution of the present invention
Or equivalent replacement should all cover without departing from the objective and range of technical solution of the present invention in claim of the invention
In range.Technology not described in detail in the present invention, shape, construction portion are well-known technique.
Claims (10)
1. the man-machine collaboration picking robot based on Leap Motion and machine vision, including walking mechanism, picking mechanical arm,
Power supply, picking collection basket and robot controller, the picking mechanical arm are installed to be arranged in vehicle with walking machine in the walking mechanism
On structure, picking mechanical arm is provided on the picking mechanical arm, it is characterised in that: it further include remote control system, it is described long-range
Control system includes computer and Leap Motion body-sensing sensor, and the Leap Motion body-sensing sensor and dataphone connect
It connects, the computer and robot controller communication connection；The picking mechanical is provided with camera, the camera and robot on hand
Controller communication connection, the camera and robot controller are electrically connected to a power source.
2. the man-machine collaboration picking robot according to claim 1 based on Leap Motion and machine vision, feature
Be: the picking mechanical arm includes rotating platform, large arm and forearm, and shoulder joint economize on electricity is fixedly connected on the rotating platform
Machine, the shoulder joint motor and large arm are hinged, elbow joint motor are fixedly connected on the forearm, the large arm is far from rotary flat
One end of platform and elbow joint motor are hinged, and the shoulder joint motor and elbow joint motor are electrically connected with robot controller.
3. the man-machine collaboration picking robot according to claim 2 based on Leap Motion and machine vision, feature
Be: the rotating platform includes pedestal upper plate and pedestal lower plate, and the pedestal lower plate is fixedly connected in walking mechanism, described
It is connected with support column between pedestal upper plate and pedestal lower plate, rotary electric machine, the pedestal upper plate are installed in the pedestal lower plate
On be rotatably connected to rotatable platform, the output shaft of the rotary electric machine passes through pedestal upper plate and is fixedly connected with rotatable platform, described
The output shaft of rotary electric machine is slidably connected with pedestal upper plate, and the shoulder joint motor is fixedly mounted on rotatable platform, and described turn
Dynamic motor is electrically connected with robot controller.
4. the man-machine collaboration picking robot according to claim 3 based on Leap Motion and machine vision, feature
Be: the picking mechanical arm includes mounting bracket, and the forearm is fixedly connected with wrist joint motor, institute far from one end of large arm
The side for stating mounting bracket is fixedly connected with wrist connecting plate, and the other side of the mounting bracket is fixedly connected with clamping manipulator,
The wrist connecting plate is hinged and connected with wrist joint motor；Camera installing plate, the camera are fixedly connected in the mounting bracket
It is mounted in camera installing plate, the wrist joint motor is electrically connected with robot controller.
5. the man-machine collaboration picking robot according to claim 4 based on Leap Motion and machine vision, feature
It is: is fixedly installed with wrist rotary electric machine on the wrist connecting plate, the output shaft of the wrist rotary electric machine is fixed with mounting bracket
Connection, the wrist rotary electric machine are electrically connected with robot controller.
6. the man-machine collaboration picking robot according to claim 5 based on Leap Motion and machine vision, feature
It is: is fixedly installed with cutting motor in the mounting bracket above the clamping manipulator, the output shaft of cutting motor is fixed is connected
It is connected to link block, the link block is fixed with cutter, institute between cutting motor and clamping manipulator on the link block
Cutting motor is stated to be electrically connected with robot controller.
7. the man-machine collaboration picking robot according to claim 6 based on Leap Motion and machine vision, feature
Be: the clamping manipulator includes connecting rod, the first clamping jaw and the second clamping jaw, and the connecting rod is fixedly connected in mounting bracket,
First clamping jaw and the second clamping jaw are slidably connected on connecting rod, and robot drives motor is fixedly connected on the connecting rod,
The robot drives motor is electrically connected with robot controller.
8. the man-machine collaboration picking robot according to claim 7 based on Leap Motion and machine vision, feature
Be: the camera installing plate is equipped with annular groove and collar mounting holes, is fixed with camera fixed plate on the camera, described
Fastening screw is installed between camera fixed plate and annular groove and collar mounting holes.
9. the man-machine collaboration picking robot according to claim 8 based on Leap Motion and machine vision, feature
Be: the walking mechanism is track-type work platform truck.
10. the control of the man-machine collaboration picking robot according to claim 9 based on Leap Motion and machine vision
Method, it is characterised in that: include the following steps,
S1, camera acquire fruit tree picture in real time and are identified and handled to fruit, Leap Motion body-sensing sensor
Identify that palm hand gesture carries out track-type work platform truck position and adjusts；
S2, computer are acquired palm coordinate on Leap Motion body-sensing sensor and palm hand gesture information, pass through mechanical arm
Inverse kinematics go out picking mechanical arm rotary electric machine, shoulder joint motor, elbow joint motor, wrist joint motor, wrist rotary electric machine
Rotation angle；
S3, the movement of picking mechanical arm, camera carry out template matching to present viewing field fruit, and the Clamping Center point of picking mechanical arm is sat
Mark is overlapped with fruit center point coordinate, and picking mechanical arm picks fruit；
S4, picking mechanical arm control picking mechanical arm and move to picking collection basket position from fruit position is currently plucked, and unload fruit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811059487.5A CN109202848A (en) | 2018-09-12 | 2018-09-12 | Man-machine collaboration picking robot and its control method based on Leap Motion and machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811059487.5A CN109202848A (en) | 2018-09-12 | 2018-09-12 | Man-machine collaboration picking robot and its control method based on Leap Motion and machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109202848A true CN109202848A (en) | 2019-01-15 |
Family
ID=64983529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811059487.5A Pending CN109202848A (en) | 2018-09-12 | 2018-09-12 | Man-machine collaboration picking robot and its control method based on Leap Motion and machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109202848A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110275610A (en) * | 2019-05-27 | 2019-09-24 | 山东科技大学 | A kind of collaboration gesture control coal mining simulation control method based on LeapMotion motion sensing control device |
| CN111185906A (en) * | 2020-01-10 | 2020-05-22 | 上海大学 | Leap Motion-based dexterous hand master-slave control method |
| CN111531545A (en) * | 2020-05-18 | 2020-08-14 | 珠海格力智能装备有限公司 | Robot control method, robot control system, and computer storage medium |
| CN111688526A (en) * | 2020-06-18 | 2020-09-22 | 福建百城新能源科技有限公司 | User side new energy automobile energy storage charging station |
| CN112692831A (en) * | 2020-12-24 | 2021-04-23 | 深圳市亚博智能科技有限公司 | Mechanical arm control system and mechanical arm |
| CN113997292A (en) * | 2021-11-30 | 2022-02-01 | 国网四川省电力公司南充供电公司 | Mechanical arm operation method based on machine vision, medium and electronic equipment |
| CN114012757A (en) * | 2021-12-02 | 2022-02-08 | 南通科技职业学院 | Movable automatic fruit picking device based on machine vision and method thereof |
| CN115215082A (en) * | 2021-04-16 | 2022-10-21 | 上海玄刃科技有限公司 | Transfer tank transfer method |
| CN116602121A (en) * | 2023-07-20 | 2023-08-18 | 无锡学院 | Picking machine with sorting function |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006013593A1 (en) * | 2004-08-06 | 2006-02-09 | Valente Pali Precompressi Srl | Automated fruit and vegetables picker |
| CN102914967A (en) * | 2012-09-21 | 2013-02-06 | 浙江工业大学 | Autonomous navigation and man-machine coordination picking operating system of picking robot |
| CN107593113A (en) * | 2017-09-21 | 2018-01-19 | 昆明理工大学 | A kind of intelligent fruit picking robot based on machine vision |
| CN107738255A (en) * | 2017-09-26 | 2018-02-27 | 山东工商学院 | Mechanical arm and the Mechanical arm control method based on Leap Motion |
| CN207573956U (en) * | 2017-12-01 | 2018-07-06 | 安徽农业大学 | A kind of orchard puma manipulator picker based on man-machine collaboration |
-
2018
- 2018-09-12 CN CN201811059487.5A patent/CN109202848A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006013593A1 (en) * | 2004-08-06 | 2006-02-09 | Valente Pali Precompressi Srl | Automated fruit and vegetables picker |
| CN102914967A (en) * | 2012-09-21 | 2013-02-06 | 浙江工业大学 | Autonomous navigation and man-machine coordination picking operating system of picking robot |
| CN107593113A (en) * | 2017-09-21 | 2018-01-19 | 昆明理工大学 | A kind of intelligent fruit picking robot based on machine vision |
| CN107738255A (en) * | 2017-09-26 | 2018-02-27 | 山东工商学院 | Mechanical arm and the Mechanical arm control method based on Leap Motion |
| CN207573956U (en) * | 2017-12-01 | 2018-07-06 | 安徽农业大学 | A kind of orchard puma manipulator picker based on man-machine collaboration |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110275610A (en) * | 2019-05-27 | 2019-09-24 | 山东科技大学 | A kind of collaboration gesture control coal mining simulation control method based on LeapMotion motion sensing control device |
| CN110275610B (en) * | 2019-05-27 | 2022-09-30 | 山东科技大学 | Cooperative gesture control coal mining simulation control method based on LeapMotion somatosensory controller |
| CN111185906A (en) * | 2020-01-10 | 2020-05-22 | 上海大学 | Leap Motion-based dexterous hand master-slave control method |
| CN111531545A (en) * | 2020-05-18 | 2020-08-14 | 珠海格力智能装备有限公司 | Robot control method, robot control system, and computer storage medium |
| CN111688526A (en) * | 2020-06-18 | 2020-09-22 | 福建百城新能源科技有限公司 | User side new energy automobile energy storage charging station |
| CN111688526B (en) * | 2020-06-18 | 2021-07-20 | 福建百城新能源科技有限公司 | User side new energy automobile energy storage charging station |
| CN112692831A (en) * | 2020-12-24 | 2021-04-23 | 深圳市亚博智能科技有限公司 | Mechanical arm control system and mechanical arm |
| CN115215082A (en) * | 2021-04-16 | 2022-10-21 | 上海玄刃科技有限公司 | Transfer tank transfer method |
| CN113997292A (en) * | 2021-11-30 | 2022-02-01 | 国网四川省电力公司南充供电公司 | Mechanical arm operation method based on machine vision, medium and electronic equipment |
| CN114012757A (en) * | 2021-12-02 | 2022-02-08 | 南通科技职业学院 | Movable automatic fruit picking device based on machine vision and method thereof |
| CN116602121A (en) * | 2023-07-20 | 2023-08-18 | 无锡学院 | Picking machine with sorting function |
| CN116602121B (en) * | 2023-07-20 | 2023-09-19 | 无锡学院 | Picking machine with sorting function |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109202848A (en) | Man-machine collaboration picking robot and its control method based on Leap Motion and machine vision | |
| Sarig | Robotics of fruit harvesting: A state-of-the-art review | |
| CN109328973B (en) | Intelligent rubber tapping system for rubber trees and control method of intelligent rubber tapping system | |
| CN109392462B (en) | Robot and method for picking daylily | |
| CN207573956U (en) | A kind of orchard puma manipulator picker based on man-machine collaboration | |
| Scarfe et al. | Development of an autonomous kiwifruit picking robot | |
| CN108848889B (en) | Leaf vegetable harvesting mechanical device based on image recognition and control method thereof | |
| CN205902411U (en) | Retractable kiwi fruit fruit picking manipulator arm | |
| CN109220225A (en) | A kind of full-automatic fruit picker | |
| CN111758397B (en) | Intelligent pepper picking device based on visual identification | |
| CN107567823A (en) | Intelligent tea picking robot | |
| CN104584779A (en) | Apple maturity automatic detection and picking robot based on binocular vision | |
| CN204498792U (en) | A kind of ripe apples degree based on binocular vision detects and picking robot automatically | |
| CN107439143A (en) | A kind of greenhouse string tomato harvester and collecting method | |
| CN114080905B (en) | Picking method based on digital twins and cloud picking robot system | |
| CN216058333U (en) | Intelligent movement fruit picking robot | |
| CN209527208U (en) | A kind of strawberry picking machine | |
| CN114679963B (en) | Cross-ridge double-arm cooperation strawberry harvesting robot, control system and picking method | |
| CN108811766A (en) | A kind of man-machine interactive fruits and vegetables of greenhouse harvesting robot system and its collecting method | |
| CN106973631A (en) | One kind automation picker system | |
| CN104584780A (en) | Flexible manipulator for detecting fruit maturity and harvesting | |
| CN115316129B (en) | Self-adaptive bionic picking device based on binocular vision recognition and fruit stringing picking method | |
| CN113057154A (en) | Greenhouse liquid medicine spraying robot | |
| CN209185088U (en) | A kind of fruit picking | |
| CN213290258U (en) | Based on crawler-type multifunctional robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190115 |
|
| RJ01 | Rejection of invention patent application after publication |