CN109729835A - A kind of camellia oleifera fruit picker system and control method based on binocular vision - Google Patents
A kind of camellia oleifera fruit picker system and control method based on binocular vision Download PDFInfo
- Publication number
- CN109729835A CN109729835A CN201910124070.0A CN201910124070A CN109729835A CN 109729835 A CN109729835 A CN 109729835A CN 201910124070 A CN201910124070 A CN 201910124070A CN 109729835 A CN109729835 A CN 109729835A
- Authority
- CN
- China
- Prior art keywords
- picking
- camellia oleifera
- fruit
- oleifera fruit
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 226
- 241000526900 Camellia oleifera Species 0.000 title claims abstract description 143
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000004807 localization Effects 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 20
- 230000033001 locomotion Effects 0.000 claims description 17
- 241001122767 Theaceae Species 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 8
- 241001269238 Data Species 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 4
- 230000007797 corrosion Effects 0.000 claims description 4
- 238000005260 corrosion Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000005484 gravity Effects 0.000 claims description 2
- 230000007246 mechanism Effects 0.000 claims description 2
- 235000013616 tea Nutrition 0.000 claims 6
- 239000003921 oil Substances 0.000 description 12
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 240000001548 Camellia japonica Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000010495 camellia oil Substances 0.000 description 1
- 235000018597 common camellia Nutrition 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- NJPPVKZQTLUDBO-UHFFFAOYSA-N novaluron Chemical compound C1=C(Cl)C(OC(F)(F)C(OC(F)(F)F)F)=CC=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F NJPPVKZQTLUDBO-UHFFFAOYSA-N 0.000 description 1
- 230000000050 nutritive effect Effects 0.000 description 1
- 239000002420 orchard Substances 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P60/00—Technologies relating to agriculture, livestock or agroalimentary industries
- Y02P60/12—Technologies relating to agriculture, livestock or agroalimentary industries using renewable energies, e.g. solar water pumping
Landscapes
- Manipulator (AREA)
Abstract
The present invention provides a kind of camellia oleifera fruit picker system and control method based on binocular vision, and method includes: control device according to the conduct Route Generation travel commands planned in advance, and travel commands are for driving crawler travelling device to advance to picking destination；Binocular vision device acquires camellia oleifera fruit image, and the picking information of camellia oleifera fruit is determined according to camellia oleifera fruit image；Control device generates picking instruction according to picking information, and picking instruction is respectively sent in picking executing agency and mechanical arm.It can walk according to the conduct route of planning, picking fruit is carried out to multiple picking destinations on conduct route, when reaching picking destination, camellia oleifera fruit image is acquired by binocular vision device, camellia oleifera fruit image is handled to obtain picking information, is instructed according to the picking that picking information obtains control picking executing agency and mechanical arm implementation picking, realizes the automatic picking of camellia oleifera fruit, manpower is saved, is improved efficiency.
Description
Technical field
The invention mainly relates to automatic picked technology fields, and in particular to a kind of camellia oleifera fruit picking system based on binocular vision
System and control method.
Background technique
Tea oil has very high health care and nutritive value, and country payes attention to energetically to ensure grain and oil safety and oil tea is helped to produce
Industry development, at present camellia oleifera fruit picking rely primarily on artificial picking, and the amount of labour is big and efficiency is extremely low.Camellia oleifera fruit picking has become oil tea
The most weak ring of industry development is increasingly becoming the bottleneck for hindering camellia oleiferaindustry fast development.
Summary of the invention
The technical problem to be solved by the present invention is in view of the deficiencies of the prior art, provide a kind of oil based on binocular vision
Tea fruit picker system and control method.
The technical scheme to solve the above technical problems is that a kind of camellia oleifera fruit based on binocular vision picks system
System, including workbench, crawler travelling device, picking executing agency, mechanical arm, binocular vision device and control device.
The workbench is hollow cavity structure, and the bottom of the workbench is arranged in the crawler travelling device
Portion, on the working platform at the center on surface, the picking executing agency is arranged in the machinery for the mechanical arm setting
The top of arm, surface, the control device are arranged in the work on the working platform for the binocular vision device setting
In the hollow cavity structure of platform, the binocular vision device is electrically connected with the control device, the control device respectively with
The picking executing agency and crawler travelling device electrical connection.
The control device is used for according to the conduct Route Generation travel commands planned in advance, and the travel commands are for driving
It moves the crawler travelling device and advances to picking destination.
The binocular vision device picks camellia oleifera fruit image in the preset range of destination for acquiring, and according to the oil tea
Fruit image determines the picking information of camellia oleifera fruit.
The control device is also used to generate picking instruction according to the picking information, and picking instruction is sent respectively
To in the picking executing agency and the mechanical arm, the picking instruction is for driving the picking executing agency and the machine
Tool arm carries out camellia oleifera fruit picking.
Another technical solution that the present invention solves above-mentioned technical problem is as follows: a kind of camellia oleifera fruit picking based on binocular vision
Control method includes the following steps:
Control device is according to the conduct Route Generation travel commands planned in advance, and the travel commands are for driving crawler type
Mobile device advances to picking destination.
Binocular vision device acquires camellia oleifera fruit image in the preset range of the picking destination, and according to the camellia oleifera fruit figure
Picking information as determining camellia oleifera fruit.
The control device generates picking instruction according to the picking information, and picking instruction is respectively sent to pick
In executing agency and mechanical arm, the picking instruction is for driving the picking executing agency and the mechanical arm to carry out camellia oleifera fruit
Picking.
The beneficial effects of the present invention are: can walk according to the conduct route of planning, to multiple pickings on conduct route
Destination carries out picking fruit, when reaching picking destination, camellia oleifera fruit image is acquired by binocular vision device, to camellia oleifera fruit figure
As being handled to obtain picking information, control picking executing agency is obtained according to picking information and mechanical arm implements the picking of picking
The automatic picking of camellia oleifera fruit is realized in instruction, is saved manpower, is improved efficiency.
Detailed description of the invention
Fig. 1 is the structural schematic diagram for the camellia oleifera fruit picker system that one embodiment of the invention provides；
Fig. 2 is the flow diagram for the camellia oleifera fruit picking method that one embodiment of the invention provides；
Fig. 3 be another embodiment of the present invention provides camellia oleifera fruit picking method flow diagram；
Fig. 4 is the flow diagram for determining picking information that one embodiment of the invention provides；
Fig. 5 is the flow diagram for the planning conduct route that one embodiment of the invention provides.
In attached drawing, the names of the parts represented by the respective symbols are as follows:
1, executing agency, 2, mechanical arm, 3, binocular vision device, 4, control device, 5, workbench are picked.
Specific embodiment
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the invention.
Fig. 1 is the structural schematic diagram for the camellia oleifera fruit picker system that one embodiment of the invention provides.
As shown in Figure 1, a kind of camellia oleifera fruit picker system based on binocular vision, including workbench 5, crawler-type mobile dress
Set 6, picking executing agency 1, mechanical arm 2, binocular vision device 3 and control device 4.
The workbench 5 is hollow cavity structure, and the workbench 5 is arranged in the crawler travelling device 6
Bottom, the mechanical arm 2 are arranged at the center of 5 upper surface of workbench, and the picking executing agency 1 is arranged described
The top of mechanical arm 2, the setting of binocular vision device 3 exist in 5 upper surface of workbench, the setting of control device 4
In the hollow cavity structure of the workbench 5, the binocular vision device 3 is electrically connected with the control device 4, the control
Device 4 is electrically connected with the picking executing agency 1 and the crawler travelling device 6 respectively.Specifically, the binocular vision dress
It sets 3 to be electrically connected by data line with the control device 4, the control device 4 is executed with the picking respectively by signal wire
Mechanism 1 and the crawler travelling device 6 are electrically connected.
The control device 4 is used for according to the conduct Route Generation travel commands planned in advance, and the travel commands are used for
The crawler travelling device 6 is driven to advance to picking destination.
The binocular vision device 3 is used to acquire camellia oleifera fruit image in the preset range of the picking destination, and according to institute
State the picking information that camellia oleifera fruit image determines camellia oleifera fruit.
The control device 4 is also used to generate picking instruction according to the picking information, and picking instruction is sent out respectively
It send into the picking executing agency 1 and the mechanical arm 2, the picking instruction is for driving 1 He of picking executing agency
The mechanical arm 2 carries out camellia oleifera fruit picking.
It specifically, is 360 ° in the preset range, binocular vision device judges according to the camellia oleifera fruit image that this is acquired
It can pick in range with the presence or absence of oil tea fruits, and if it exists, then control device passes through picking information driving picking executing agency
Picking fruit is carried out with mechanical arm, if it does not exist, control device, which issues control signal, is rotated by 90 ° binocular vision device, and counts
Number is primary, while judging whether counts reach 4 times, if not up to counts 4 times, control device passes through picking information
Driving picking executing agency and mechanical arm carry out picking fruit, if reaching 4 times, control device drives the crawler-type mobile dress
It sets and advances to next picking destination.
In above-described embodiment, it can walk according to the conduct route of planning, to multiple picking destinations on conduct route
Picking fruit is carried out, when reaching picking destination, camellia oleifera fruit image is acquired by binocular vision device, camellia oleifera fruit image is carried out
Processing obtains picking information, obtains the picking instruction that executing agency is picked in control and mechanical arm implements picking according to information is picked,
It realizes the automatic picking of camellia oleifera fruit, saves manpower, improve efficiency.
Optionally, as an embodiment of the present invention, the binocular vision device 3 includes at video camera and visual pattern
Device is managed, there are two the video camera is set, two video camera settings are on the upper surface of the workbench 5 and are located at
The two sides of the mechanical arm 2.
The video camera, for acquiring camellia oleifera fruit image in the preset range.
The visual pattern processor obtains pretreatment camellia oleifera fruit figure for pre-processing to the camellia oleifera fruit image
Picture, and identify multiple camellia oleifera fruit profiles from the pretreatment camellia oleifera fruit image, to the multiple camellia oleifera fruit profile respectively into
The processing of row three-dimensional localization obtains the corresponding three-dimensional localization information of multiple camellia oleifera fruit profiles, and believes obtained multiple three-dimensional localizations
Breath is calculated, and obtains picking information, the picking information is sent in the control device 4.
Specifically, the video camera be can 360 ° of rotations video cameras, described in the video camera is mounted on by rotary shaft
On the upper surface of workbench 5.
Optionally, as an embodiment of the present invention, as shown in figure 4, the visual pattern processor includes:
Image pre-processing unit is handled for carrying out image filtering processing to the camellia oleifera fruit image through image filtering
Camellia oleifera fruit image carry out luminance histogram equalization processing, and to equalisedization processing camellia oleifera fruit image progress binaryzation at
Reason, and corrosion expansion process is carried out to the camellia oleifera fruit image through binary conversion treatment, obtain pretreatment camellia oleifera fruit image.
Three-dimensional localization unit identifies multiple camellia oleifera fruits for carrying out feature extraction to the pretreatment camellia oleifera fruit image
Profile carries out Stereo matching processing to the multiple camellia oleifera fruit profile respectively, to the multiple camellia oleifera fruit wheels handled through Stereo matching
It is wide to carry out three-dimensional reconstruction processing respectively, obtain multiple three-dimensional localization information.
Picking point center of gravity calculation unit, for carrying out polar coordinate transform processing respectively to the multiple three-dimensional localization information,
Multiple polar datas are obtained, and range conversion processing is carried out to the multiple polar data respectively, obtain multiple distance values
Data, and linear fit is carried out to the multiple distance value data respectively, obtain multiple groups picking fruit location information and attitude angle
Information.
Fruit number decision unit, for determining picking fruit number according to the multiple picking fruit location information.
Transmission unit is used for the multiple groups picking fruit location information and attitude angle information and the picking fruit number
It is sent in the control device 4 as the picking information.
In above-described embodiment, more visible camellia oleifera fruit picking can be obtained by pre-processing to camellia oleifera fruit image, convenient for spy
Sign is extracted, and is carried out three-dimensional reconstruction to camellia oleifera fruit and is handled to obtain three-dimensional localization information, to three-dimensional localization information processing, can obtain weight
Heart position and camellia oleifera fruit contour fitting, to further determine that camellia oleifera fruit picking location information and attitude angle information.
Optionally, as an embodiment of the present invention, the control device 4 includes:
Path planning unit, for according to the conduct Route Generation travel commands planned in advance, the travel commands to be used for
The crawler travelling device 6 is driven to advance to picking destination.
Processing unit is controlled, for when reaching the picking destination, Xiang Suoshu binocular vision device 3 to send acquisition and refers to
It enables, the acquisition instructions are for driving the binocular vision device 3 to acquire camellia oleifera fruit figure in the preset range of the picking destination
Picture.
It is also used to when the multiple groups picking fruit location information and attitude angle for receiving the transmission of binocular vision device 3
When information and the picking fruit number, according to the picking fruit number to the multiple groups picking fruit location information and posture
Angle information carry out sequence label processing, and according to through label each group picking fruit location information and attitude angle information obtain machinery
Arm picking instruction and the picking instruction of picking executing agency, and the mechanical arm is sent to the mechanical arm 2 according to label sequence and is adopted
It plucks instruction and sends the picking executing agency picking instruction, the mechanical arm picking instruction to the picking executing agency 1
For driving the mechanical arm 2 to move, the picking executing agency picking instruction is for driving the picking executing agency 1 to execute
Picking movement.
In above-described embodiment, to every group of picking fruit location information and attitude angle information carry out sequence label, and root
Mechanical arm picking instruction and the picking instruction of picking executing agency are sequentially obtained according to this, and mechanical arm and picking executing agency can be made suitable
Sequence executes each instruction, orderly and effective automatic implementation camellia oleifera fruit picking.
Optionally, as an embodiment of the present invention, the control device 4 further includes judging unit.
The judging unit completes to belong to the current each group picking through label for picking destination for judging whether to execute
Instruction, if completed, according to the lower travel commands of conduct Route Generation planned in advance, next travel commands are for driving
It moves the crawler travelling device 6 and advances to next picking destination.
Optionally, as an embodiment of the present invention, the judging unit is also used to, before generating next travel commands,
Judge whether that the conduct route planned in advance is completed in walking according to the workbench current location information of acquisition, if so, to institute
It states control processing unit and sends standby command, if it is not, then generating next travel commands.
In above-described embodiment, camellia oleifera fruit picking, pole can be carried out to each picking point in the conduct route planned in advance
Picking efficiency is improved greatly.
Fig. 2 is the flow diagram for the camellia oleifera fruit picking method that one embodiment of the invention provides.
Optionally, as another embodiment of the invention, as shown in Fig. 2, a kind of camellia oleifera fruit based on binocular vision is adopted
Control method is plucked, is included the following steps:
Control device 4 is according to the conduct Route Generation travel commands planned in advance, and the travel commands are for driving crawler belt
Formula mobile device 6 advances to picking destination.
Camellia oleifera fruit image in the acquisition picking destination preset range of binocular vision device 3, and according to the camellia oleifera fruit image
The picking information is sent in the control device 4 by the picking information for determining camellia oleifera fruit.
The control device 4 generates picking instruction according to the picking information, and picking instruction is respectively sent to adopt
It plucks in executing agency 1 and mechanical arm 2, the picking instruction is for driving the picking executing agency 1 and the mechanical arm 2 to carry out
Camellia oleifera fruit picking.
Specifically, as shown in figure 4, it is as follows to the conduct route setting up procedure planned in advance:
It can planning path according to the determination of pre-set navigational map.It include oil tea fruit tree in camellia oleifera fruit orchard in the navigation map
The geographical location of plantation.
According to can the road width of planning path, link length, road grade and with corner determine path orientation and
Advance maximum magnitude.
Range can be picked according to the motion range determination of mechanical arm, so that it is determined that crawler travelling device walking step-length is big
It is small.
Optionally, as an embodiment of the present invention, the picking that camellia oleifera fruit is determined according to the camellia oleifera fruit image
The process of information includes:
The camellia oleifera fruit image is pre-processed, obtains pretreatment camellia oleifera fruit image, and from the pretreatment camellia oleifera fruit
Multiple camellia oleifera fruit profiles are identified in image, and three-dimensional localization processing is carried out to the multiple camellia oleifera fruit profile respectively, is obtained multiple
The corresponding three-dimensional localization information of camellia oleifera fruit profile, and obtained multiple three-dimensional localization information are calculated, obtain picking information.
Optionally, as an embodiment of the present invention, as shown in figure 5, the mistake for obtaining pretreatment camellia oleifera fruit image
Journey includes:
Image filtering processing is carried out to the camellia oleifera fruit image, brightness is carried out to the camellia oleifera fruit image handled through image filtering
Histogram equalization processing, and binary conversion treatment is carried out to the camellia oleifera fruit image of equalisedization processing, and to through binary conversion treatment
Camellia oleifera fruit image carry out corrosion expansion process, obtain pretreatment camellia oleifera fruit image.
The process for obtaining the corresponding three-dimensional localization information of multiple camellia oleifera fruit profiles includes:
Feature extraction is carried out to the pretreatment camellia oleifera fruit image, multiple camellia oleifera fruit profiles are identified, respectively to described more
A camellia oleifera fruit profile carries out Stereo matching processing, carries out three-dimensional reconstruction respectively to the multiple camellia oleifera fruit profiles handled through Stereo matching
Processing, obtains multiple three-dimensional localization information.
It is described obtain picking information process include:
Polar coordinate transform processing is carried out to the multiple three-dimensional localization information respectively, obtains multiple polar datas, and divide
It is other to the multiple polar data carry out range conversion processing, obtain multiple distance value datas, and respectively to it is the multiple away from
Linear fit is carried out from Value Data, obtains multiple groups picking fruit location information and attitude angle information.
Picking fruit number is determined according to the multiple picking fruit location information, and multiple groups picking fruit position is believed
Breath and attitude angle information and the picking fruit number are as the picking information.
Optionally, as an embodiment of the present invention, the control device 4 generates picking according to the picking information and refers to
The process of order includes:
When reaching the picking destination, Xiang Suoshu binocular vision device 3 sends acquisition instructions, and the acquisition instructions are used
Camellia oleifera fruit image in the preset range of the picking destination is acquired in the driving binocular vision device 3.
When receive the multiple groups picking fruit location information and attitude angle information that the binocular vision device 3 is sent and
When the picking fruit number, according to the picking fruit number to the multiple groups picking fruit location information and attitude angle information
Carry out sequence label processing, and according to through label each group picking fruit location information and attitude angle information obtain mechanical arm picking
Instruction and the picking instruction of picking executing agency, and the mechanical arm picking instruction is sent to the mechanical arm 2 according to label sequence
And the picking executing agency picking instruction is sent to the picking executing agency 1, the mechanical arm picking instruction is for driving
It moves the mechanical arm 2 to move, the picking executing agency picking instruction is dynamic for driving the picking executing agency 1 to execute picking
Make.
Specifically, it is as follows to execute the process that picking instructs for mechanical arm:
Mechanical arm picking instruction is specifically used for the motion profile of planning mechanical arm, i.e. picking fruit track and specified fruit is adopted
Pluck location point.
Control device sends mechanical arm picking instruction into the driving unit of mechanical arm, the driving unit driving machine of mechanical arm
Tool arm is moved according to the motion profile.
Judge whether mechanical arm reaches specified picking fruit position, is weighed if mechanical arm does not reach specified picking fruit position
The motion profile of new planning mechanical arm；If mechanical arm reaches specified picking fruit position, mechanical arm stopping is acted and is sent anti-
Feedback signal is into control device.
Specifically, it is as follows to execute the process that picking instructs for picking executing agency:
Executing agency's picking instruction is picked for driving picking executing agency to execute picking movement.
Control device sends the picking instruction of picking executing agency into the driving unit of picking executing agency, picks execution machine
The driving unit driving picking executing agency of structure carries out movement waiting.
When control device receives the feedback signal of mechanical arm, the feedback signal be reach specified picking fruit position and
It is adjusted to the signal of predetermined picking fruit posture, sends the picking instruction of picking executing agency again to the driving of picking executing agency
Unit, the driving unit driving picking executing agency for picking executing agency carry out primary picking movement.
The driving unit for picking executing agency sends feedback signal into control device, and enters movement next time and wait.
It optionally, as an embodiment of the present invention, further include step when the picking for having executed current index instructs
It is rapid: to judge whether to execute each group picking instruction through label for completing to belong to current picking destination, if completed, according to pre-
The lower travel commands of the conduct Route Generation first planned, next travel commands are for before driving the crawler travelling device
Proceed to next picking destination.
Optionally, as an embodiment of the present invention, it before generating next travel commands, further comprises the steps of: according to acquisition
Workbench current location information judge whether that the conduct route planned in advance is completed in walking, if so, at the control
It manages unit and sends standby command, if it is not, then generating next travel commands.
Optionally, as an embodiment of the present invention, the oil that binocular vision device is acquired according to this is further comprised the steps of:
The judgement of tea fruit image can pick in range with the presence or absence of oil tea fruits, and if it exists, then control device passes through picking information driving
It picks executing agency and mechanical arm carries out picking fruit, if it does not exist, control device, which issues control signal, makes binocular vision device
It is rotated by 90 °, and counts once, while judging whether counts reach 4 times, if not up to counts 4 times, control device
Picking fruit is carried out by picking information driving picking executing agency and mechanical arm, if reaching 4 times, described in control device driving
Crawler travelling device advances to next picking destination.It is 360 ° in the preset range, that is, carries out 4 rotations.
Specifically, binocular vision device further includes counting device, and before executing picking instruction, control device, which is sent, to be reset extremely
Counting device is reset in the time set.
As shown in figure 3, following embodiment provides the entire implementation process of control device control camellia oleifera fruit picking.
Step S0: being powered on and initialized and starts picker system.
Step S1: for control device according to the conduct Route Generation travel commands planned in advance, crawler travelling device is mobile
Picking original position into path planning, and picking destination is marched to according to travel commands.
Step S2: mechanical arm pedestal resets, while the counting device for recording mechanical arm rotation is reset, binocular vision device
Camellia oleifera fruit image in video camera shooting picking destination preset range, carries out camellia oleifera fruit image by visual pattern processor
Reason, and according to the position of treated camellia oleifera fruit image analysis oil tea fruits, obtain multiple groups picking fruit location information and appearance
State angle information and the picking fruit number, and be transmitted in control device.
Step S3: judge to pick in this camellia oleifera fruit image in range with the presence or absence of oil tea fruits, execute if it exists
Step S4；If it does not exist, control device, which issues control signal, is rotated by 90 ° binocular vision device, and counts once, judges simultaneously
Whether counts reach 4 times, if not up to counts 4 times, it again returns to and executes step S2；If reaching counts 4
Secondary execution step S8.
Step S4: control device is according to the picking fruit number to the multiple groups picking fruit location information and attitude angle
Information carry out sequence label processing simultaneously saves, while being planned in each camellia oleifera fruit picking process according to the position of oil tea fruits
Manipulator motion track obtains mechanical arm picking instruction according to the attitude angle information of picking fruit location information and picking fruit
With the picking instruction of picking executing agency.
Step S5: mechanical arm picking instruction is transferred to the driving unit of mechanical arm, the driving list of mechanical arm by control device
Member driving mechanical arm is moved according to predetermined movement track, judges whether to reach specified picking fruit position, if not up to
Again driving mechanical arm is moved according to predetermined movement track, if reaching mechanical arm stopping movement, and executes step S6.
Step S6: control device will pick executing agency's picking instruction and be transferred to picking executing agency, when mechanical arm reaches
It specifies picking fruit position and is adjusted to predetermined picking fruit posture, picking executing agency's execution movement is completed a fruit and adopted
It plucks.
Step S7: judge whether each picking instruction is completed according to label sequence, if completing, execute step S8；If not
It completes, all fruits in picking range is picked, then return to step S5.
Step S8: judging whether that the conduct route planned in advance is completed in walking, if not walked, control device driving is carried out
Belt mobile device is moved at the next picking position for the conduct route planned in advance, and return step S2；If completing conduct
Route, system enters standby mode, and issues the alarm that stops operation, and terminates.
It is apparent to those skilled in the art that for convenience of description and succinctly, the dress of foregoing description
The specific work process with unit is set, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
In several embodiments provided herein, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of unit, only
A kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.
Unit may or may not be physically separated as illustrated by the separation member, shown as a unit
Component may or may not be physical unit, it can and it is in one place, or may be distributed over multiple networks
On unit.It can select some or all of unit therein according to the actual needs to realize the mesh of the embodiment of the present invention
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
It, can if integrated unit is realized in the form of SFU software functional unit and when sold or used as an independent product
To be stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention substantially or
Say that all or part of the part that contributes to existing technology or the technical solution can embody in the form of software products
Out, which is stored in a storage medium, including some instructions are used so that a computer equipment
(can be personal computer, server or the network equipment etc.) executes all or part of each embodiment method of the present invention
Step.And storage medium above-mentioned include: USB flash disk, it is mobile hard disk, read-only memory (ROM, Read-Only Memory), random
Access various Jie that can store program code such as memory (RAM, Random Access Memory), magnetic or disk
Matter.
More than, only a specific embodiment of the invention, but scope of protection of the present invention is not limited thereto, and it is any to be familiar with
Those skilled in the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or substitutions,
These modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be wanted with right
Subject to the protection scope asked.
Claims (10)
1. a kind of camellia oleifera fruit picker system based on binocular vision, which is characterized in that including workbench (5), crawler-type mobile
Device (6), picking executing agency (1), mechanical arm (2), binocular vision device (3) and control device (4),
The workbench (5) is hollow cavity structure, and the crawler travelling device (6) is arranged in the workbench (5)
Bottom, the mechanical arm (2) is arranged at the center of the workbench (5) upper surface, and the picking executing agency (1) sets
The top in the mechanical arm (2) is set, the binocular vision device (3) is arranged in the workbench (5) upper surface, the control
Device (4) processed is arranged in the hollow cavity structure of the workbench (5), the binocular vision device (3) and the control
Device (4) electrical connection, the control device (4) respectively with the picking executing agency (1) and the crawler travelling device (6)
Electrical connection；
The control device (4) is used for according to the conduct Route Generation travel commands planned in advance, and the travel commands are for driving
It moves the crawler travelling device (6) and advances to picking destination；
The binocular vision device (3) picks camellia oleifera fruit image in the preset range of destination for acquiring, and according to the oil tea
Fruit image determines the picking information of camellia oleifera fruit；
The control device (4) is also used to generate picking instruction according to the picking information, and picking instruction is sent respectively
To in the picking executing agency (1) and the mechanical arm (2), the picking instruction is for driving the picking executing agency
(1) and the mechanical arm (2) carries out camellia oleifera fruit picking.
2. camellia oleifera fruit picker system according to claim 1, which is characterized in that the binocular vision device (3) includes taking the photograph
Camera and visual pattern processor, there are two the video camera is set, two video camera settings are in the workbench (5)
On upper surface and it is located at the two sides of the mechanical arm (2)；
The video camera, for acquiring camellia oleifera fruit image in the preset range；
The visual pattern processor obtains pretreatment camellia oleifera fruit image for pre-processing to the camellia oleifera fruit image, and
Multiple camellia oleifera fruit profiles are identified from the pretreatment camellia oleifera fruit image, and three-dimensional is carried out respectively to the multiple camellia oleifera fruit profile
Localization process obtains the corresponding three-dimensional localization information of multiple camellia oleifera fruit profiles, and carries out to obtained multiple three-dimensional localization information
It calculates, obtains picking information, the picking information is sent in the control device (4).
3. camellia oleifera fruit picker system according to claim 2, which is characterized in that the visual pattern processor includes:
Image pre-processing unit, for carrying out image filtering processing to the camellia oleifera fruit image, to the oil handled through image filtering
Tea fruit image carries out luminance histogram equalization processing, and carries out binary conversion treatment to the camellia oleifera fruit image of equalisedization processing,
And corrosion expansion process is carried out to the camellia oleifera fruit image through binary conversion treatment, obtain pretreatment camellia oleifera fruit image；
Three-dimensional localization unit, for identifying multiple camellia oleifera fruit profiles to pretreatment camellia oleifera fruit image progress feature extraction,
Stereo matching processing is carried out to the multiple camellia oleifera fruit profile respectively, the multiple camellia oleifera fruit profiles handled through Stereo matching are distinguished
Three-dimensional reconstruction processing is carried out, multiple three-dimensional localization information are obtained；
Picking point center of gravity calculation unit is obtained for carrying out polar coordinate transform processing respectively to the multiple three-dimensional localization information
Multiple polar datas, and range conversion processing is carried out to the multiple polar data respectively, multiple distance value datas are obtained,
And linear fit is carried out to the multiple distance value data respectively, obtain multiple groups picking fruit location information and attitude angle information；
Fruit number decision unit, for determining picking fruit number according to the multiple picking fruit location information；
Transmission unit, for using the multiple groups picking fruit location information and attitude angle information and the picking fruit number as
The picking information is sent in the control device (4).
4. camellia oleifera fruit picker system according to claim 3, which is characterized in that the control device (4) includes:
Path planning unit, for according to the conduct Route Generation travel commands planned in advance, the travel commands to be for driving
The crawler travelling device (6) advances to picking destination；
Processing unit is controlled, for when reaching the picking destination, Xiang Suoshu binocular vision device (3) to send acquisition and refers to
It enables, the acquisition instructions are for driving camellia oleifera fruit in the binocular vision device (3) acquisition picking destination preset range
Image；
It is also used to when the multiple groups picking fruit location information and attitude angle letter for receiving the binocular vision device (3) transmission
When breath and the picking fruit number, according to the picking fruit number to the multiple groups picking fruit location information and attitude angle
Information carry out sequence label processing, and according to through label each group picking fruit location information and attitude angle information obtain mechanical arm
Picking instruction and the picking instruction of picking executing agency, and the mechanical arm is sent to the mechanical arm (2) according to label sequence and is adopted
It plucks instruction and sends the picking executing agency picking instruction to the picking executing agency (1), the mechanical arm picking refers to
It enables for driving the mechanical arm (2) to move, the picking executing agency picking instruction is for driving the picking executing agency
(1) picking movement is executed.
5. camellia oleifera fruit picker system according to claim 4, which is characterized in that the control device (4) further includes judgement
Unit；
The judging unit refers to for judging whether to execute to complete to belong to the current each group through label for picking destination and pick
It enables, if completed, according to the lower travel commands of conduct Route Generation planned in advance, next travel commands are for driving
The crawler travelling device (6) advances to next picking destination.
6. camellia oleifera fruit picker system according to claim 5, which is characterized in that the judging unit is also used to, under generation
Before one travel commands, judge whether that the conduct road planned in advance is completed in walking according to the workbench current location information of acquisition
Line, if so, standby command is sent to the control processing unit, if it is not, then generating next travel commands.
7. a kind of camellia oleifera fruit based on binocular vision picks control method, which comprises the steps of:
Control device is according to the conduct Route Generation travel commands planned in advance, and the travel commands are for driving crawler-type mobile
Device advances to picking destination；
Camellia oleifera fruit image in binocular vision device acquisition picking destination preset range, and oil is determined according to the camellia oleifera fruit image
The picking information of tea fruit；
The control device generates picking instruction according to the picking information, and picking instruction is respectively sent to picking and is executed
In mechanism and mechanical arm, the picking instruction is adopted for driving the picking executing agency and the mechanical arm to carry out camellia oleifera fruit
It plucks.
8. camellia oleifera fruit according to claim 7 picks control method, which is characterized in that described according to the camellia oleifera fruit image
The process for determining the picking information of camellia oleifera fruit includes:
The camellia oleifera fruit image is pre-processed, obtains pretreatment camellia oleifera fruit image, and from the pretreatment camellia oleifera fruit image
In identify multiple camellia oleifera fruit profiles, three-dimensional localization processing is carried out to the multiple camellia oleifera fruit profile respectively, obtains multiple oil teas
The corresponding three-dimensional localization information of fruit profile, and obtained multiple three-dimensional localization information are calculated, obtain picking information.
9. camellia oleifera fruit according to claim 8 picks control method, which is characterized in that described to obtain pretreatment camellia oleifera fruit figure
The process of picture includes:
Image filtering processing is carried out to the camellia oleifera fruit image, brightness histogram is carried out to the camellia oleifera fruit image handled through image filtering
Figure equalization processing, and binary conversion treatment is carried out to the camellia oleifera fruit image of equalisedization processing, and to the oil through binary conversion treatment
Tea fruit image carries out corrosion expansion process, obtains pretreatment camellia oleifera fruit image；
The process for obtaining the corresponding three-dimensional localization information of multiple camellia oleifera fruit profiles includes:
Feature extraction is carried out to the pretreatment camellia oleifera fruit image, multiple camellia oleifera fruit profiles are identified, respectively to the multiple oil
Tea fruit profile carries out Stereo matching processing, carries out at three-dimensional reconstruction respectively to the multiple camellia oleifera fruit profiles handled through Stereo matching
Reason, obtains multiple three-dimensional localization information；
It is described obtain picking information process include:
Polar coordinate transform processing is carried out to the multiple three-dimensional localization information respectively, obtains multiple polar datas, and right respectively
The multiple polar data carries out range conversion processing, obtains multiple distance value datas, and respectively to the multiple distance value
Data carry out linear fit, obtain multiple groups picking fruit location information and attitude angle information；
Determine picking fruit number according to the multiple picking fruit location information, by the multiple groups picking fruit location information and
Attitude angle information and the picking fruit number are as the picking information.
10. camellia oleifera fruit according to claim 7 picks control method, which is characterized in that the control device is according to
Picking information generates the process that picking instructs
When reaching the picking destination, Xiang Suoshu binocular vision device sends acquisition instructions, and the acquisition instructions are for driving
Move camellia oleifera fruit image in the binocular vision device acquisition picking destination preset range；
When receiving the multiple groups picking fruit location information and attitude angle information that the binocular vision device is sent and described
When picking fruit number, the multiple groups picking fruit location information and attitude angle information are carried out according to the picking fruit number
Sequence label processing, and according to through label each group picking fruit location information and attitude angle information obtain mechanical arm picking instruction
With the picking instruction of picking executing agency, and according to label sequence to the mechanical arm send mechanical arm picking instruction and to
The picking executing agency sends the picking executing agency picking instruction, and the mechanical arm picking instruction is for driving the machine
The movement of tool arm, the picking executing agency picking instruction is for driving the picking executing agency to execute picking movement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910124070.0A CN109729835B (en) | 2019-02-19 | 2019-02-19 | Binocular vision-based oil tea fruit picking system and control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910124070.0A CN109729835B (en) | 2019-02-19 | 2019-02-19 | Binocular vision-based oil tea fruit picking system and control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109729835A true CN109729835A (en) | 2019-05-10 |
| CN109729835B CN109729835B (en) | 2024-01-12 |
Family
ID=66367777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910124070.0A Active CN109729835B (en) | 2019-02-19 | 2019-02-19 | Binocular vision-based oil tea fruit picking system and control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109729835B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861654A (en) * | 2021-01-20 | 2021-05-28 | 浙江理工大学 | Famous tea picking point position information acquisition method based on machine vision |
| CN113099847A (en) * | 2021-05-25 | 2021-07-13 | 广东技术师范大学 | Fruit picking method based on fruit three-dimensional parameter prediction model |
| CN114731845A (en) * | 2022-02-24 | 2022-07-12 | 华南农业大学 | Receiving equipment is picked to portable navel orange |
| CN115316129A (en) * | 2022-08-12 | 2022-11-11 | 华南农业大学 | Self-adaptive bionic picking device based on binocular vision recognition and cluster fruit picking method |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101356877B (en) * | 2008-09-19 | 2012-06-20 | 中国农业大学 | Cucumber picking robot system and picking method in greenhouse |
| CN102165880A (en) * | 2011-01-19 | 2011-08-31 | 南京农业大学 | Automatic-navigation crawler-type mobile fruit picking robot and fruit picking method |
| CN102662399B (en) * | 2012-05-09 | 2014-10-22 | 江苏科技大学 | Method for optimizing path for fruit picking robot facing towards multiple targets |
| CN104700404B (en) * | 2015-03-02 | 2018-03-02 | 中国农业大学 | A kind of fruit positioning identifying method |
| CN108901362B (en) * | 2018-08-15 | 2023-10-27 | 湖南师范大学 | Mung bean picking robot |
| CN209572477U (en) * | 2019-02-19 | 2019-11-05 | 广西师范大学 | A kind of camellia oleifera fruit picker |
-
2019
- 2019-02-19 CN CN201910124070.0A patent/CN109729835B/en active Active
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861654A (en) * | 2021-01-20 | 2021-05-28 | 浙江理工大学 | Famous tea picking point position information acquisition method based on machine vision |
| CN112861654B (en) * | 2021-01-20 | 2024-09-17 | 浙江理工大学 | Machine vision-based famous tea picking point position information acquisition method |
| CN113099847A (en) * | 2021-05-25 | 2021-07-13 | 广东技术师范大学 | Fruit picking method based on fruit three-dimensional parameter prediction model |
| CN113099847B (en) * | 2021-05-25 | 2022-03-08 | 广东技术师范大学 | Fruit picking method based on fruit three-dimensional parameter prediction model |
| CN114731845A (en) * | 2022-02-24 | 2022-07-12 | 华南农业大学 | Receiving equipment is picked to portable navel orange |
| CN115316129A (en) * | 2022-08-12 | 2022-11-11 | 华南农业大学 | Self-adaptive bionic picking device based on binocular vision recognition and cluster fruit picking method |
| CN115316129B (en) * | 2022-08-12 | 2024-03-12 | 华南农业大学 | Self-adaptive bionic picking device based on binocular vision recognition and fruit stringing picking method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109729835B (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109729835A (en) | A kind of camellia oleifera fruit picker system and control method based on binocular vision | |
| CN112243698B (en) | Automatic walnut picking and collecting method based on multi-sensor fusion technology | |
| US11304374B2 (en) | Selective automated blossom thinning | |
| Ling et al. | Dual-arm cooperation and implementing for robotic harvesting tomato using binocular vision | |
| Sarig | Robotics of fruit harvesting: A state-of-the-art review | |
| CN108633482A (en) | A kind of fruit picking aircraft | |
| CN108337915A (en) | Three-dimensional builds drawing method, device, system, high in the clouds platform, electronic equipment and computer program product | |
| Yu et al. | A lab-customized autonomous humanoid apple harvesting robot | |
| CN109862778A (en) | For gathering in and the system and method for diluted unmanned aerial vehicle group management | |
| Zhaoxin et al. | Design a robot system for tomato picking based on YOLO v5 | |
| CN109699300B (en) | Control system and control method of intelligent citrus picking machine | |
| Hu et al. | Simplified 4-DOF manipulator for rapid robotic apple harvesting | |
| CN108858121A (en) | Hot line robot and its control method | |
| CN109910011A (en) | A kind of mechanical arm barrier-avoiding method and mechanical arm based on multisensor | |
| CN107750643A (en) | The vision system of strawberry picking robot | |
| CN110089350A (en) | A kind of Mushroom Picking Robot system and picking method | |
| CN108955645A (en) | Three-dimensional modeling method and device applied to communication iron tower intelligent patrol detection | |
| CN107067018A (en) | A kind of hot line robot bolt recognition methods based on random Hough transformation and SVM | |
| CN110433467A (en) | Picking up table tennis ball robot operation method and equipment based on binocular vision and ant group algorithm | |
| CN107745756A (en) | The traveling method of six sufficient strawberry picking robots | |
| CN216058333U (en) | Intelligent movement fruit picking robot | |
| CN107703956A (en) | A kind of virtual interaction system and its method of work based on inertia capturing technology | |
| Bohg et al. | Grasping familiar objects using shape context | |
| CN114080905A (en) | Picking method based on digital twins and cloud picking robot system | |
| CN107009357A (en) | A kind of method that object is captured based on NAO robots |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |