CN109729835B - Binocular vision-based oil tea fruit picking system and control method - Google Patents
Binocular vision-based oil tea fruit picking system and control method Download PDFInfo
- Publication number
- CN109729835B CN109729835B CN201910124070.0A CN201910124070A CN109729835B CN 109729835 B CN109729835 B CN 109729835B CN 201910124070 A CN201910124070 A CN 201910124070A CN 109729835 B CN109729835 B CN 109729835B
- Authority
- CN
- China
- Prior art keywords
- picking
- oil tea
- fruit
- tea fruit
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 226
- 241001122767 Theaceae Species 0.000 title claims abstract description 152
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000007246 mechanism Effects 0.000 claims abstract description 70
- 230000008569 process Effects 0.000 claims description 14
- 230000009471 action Effects 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000007797 corrosion Effects 0.000 claims description 4
- 238000005260 corrosion Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 230000005484 gravity Effects 0.000 claims description 3
- 239000003921 oil Substances 0.000 description 95
- 230000005611 electricity Effects 0.000 description 4
- 241000209507 Camellia Species 0.000 description 2
- 239000010495 camellia oil Substances 0.000 description 2
- 235000018597 common camellia Nutrition 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P60/00—Technologies relating to agriculture, livestock or agroalimentary industries
- Y02P60/12—Technologies relating to agriculture, livestock or agroalimentary industries using renewable energies, e.g. solar water pumping
Abstract
The invention provides a binocular vision-based oil tea fruit picking system and a control method, wherein the method comprises the following steps: the control device generates a walking instruction according to a pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device to advance to a picking destination; the binocular vision device collects oil tea fruit images and determines picking information of the oil tea fruits according to the oil tea fruit images; the control device generates picking instructions according to the picking information and sends the picking instructions to the picking executing mechanism and the mechanical arm respectively. The automatic picking machine has the advantages that the automatic picking machine can walk according to a planned travel path, fruits are picked at a plurality of picking destinations on the travel path, when the fruits reach the picking destinations, oil tea fruit images are collected through the binocular vision device, picking information is obtained by processing the oil tea fruit images, picking instructions for picking are obtained by controlling the picking actuating mechanism and the mechanical arm according to the picking information, automatic picking of the oil tea fruits is achieved, labor is saved, and efficiency is improved.
Description
Technical Field
The invention mainly relates to the technical field of automatic picking, in particular to a binocular vision-based oil tea fruit picking system and a control method.
Background
At present, the oil tea fruits are picked mainly by manpower, and the labor capacity is large and the efficiency is extremely low. Oil tea fruit picking has become the weakest link in the development of the oil tea industry and gradually becomes a bottleneck for preventing the rapid development of the oil tea industry.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a binocular vision-based oil tea fruit picking system and a control method.
The technical scheme for solving the technical problems is as follows: a binocular vision-based oil tea fruit picking system comprises a working platform, a crawler-type moving device, a picking executing mechanism, a mechanical arm, a binocular vision device and a control device.
The utility model discloses a robot, including work platform, crawler-type mobile device, picking actuating mechanism, binocular vision device, control device, the crawler-type mobile device is in the work platform's bottom, the arm sets up the center department of work platform upper surface, picking actuating mechanism sets up the top of arm, binocular vision device sets up work platform upper surface, control device sets up in the cavity of work platform is in the cavity, binocular vision device with the control device electricity is connected, control device respectively with picking actuating mechanism with crawler-type mobile device electricity is connected.
The control device is used for generating a walking instruction according to a pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device to advance to a picking destination.
The binocular vision device is used for collecting oil tea fruit images within a preset range of a picking destination and determining picking information of the oil tea fruits according to the oil tea fruit images.
The control device is further used for generating picking instructions according to the picking information, sending the picking instructions to the picking actuating mechanism and the mechanical arm respectively, and the picking instructions are used for driving the picking actuating mechanism and the mechanical arm to pick oil tea fruits.
The other technical scheme for solving the technical problems is as follows: a binocular vision-based oil tea fruit picking control method comprises the following steps:
the control device generates a walking instruction according to the pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device to advance to the picking destination.
The binocular vision device collects the oil tea fruit images within the preset range of the picking destination, and determines picking information of the oil tea fruits according to the oil tea fruit images.
The control device generates picking instructions according to the picking information, the picking instructions are respectively sent to the picking executing mechanism and the mechanical arm, and the picking instructions are used for driving the picking executing mechanism and the mechanical arm to pick oil tea fruits.
The beneficial effects of the invention are as follows: the automatic picking machine has the advantages that the automatic picking machine can walk according to a planned travel path, fruits are picked at a plurality of picking destinations on the travel path, when the fruits reach the picking destinations, oil tea fruit images are collected through the binocular vision device, picking information is obtained by processing the oil tea fruit images, picking instructions for picking are obtained by controlling the picking actuating mechanism and the mechanical arm according to the picking information, automatic picking of the oil tea fruits is achieved, labor is saved, and efficiency is improved.
Drawings
Fig. 1 is a schematic structural view of a tea-oil camellia fruit picking system according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a method for picking oil tea fruits according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a method for picking oil tea fruits according to another embodiment of the present invention;
FIG. 4 is a flow chart of determining picking information according to an embodiment of the present invention;
fig. 5 is a flow chart illustrating a planning of a route according to an embodiment of the present invention.
In the drawings, the names of the components represented by the respective marks are as follows:
1. picking actuating mechanism 2, arm, 3, binocular vision device, 4, controlling means, 5, work platform.
Detailed Description
The principles and features of the present invention are described below with reference to the drawings, the examples are illustrated for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
Fig. 1 is a schematic structural diagram of a tea-oil camellia fruit picking system according to an embodiment of the present invention.
As shown in fig. 1, the oil tea fruit picking system based on binocular vision comprises a working platform 5, a crawler-type moving device 6, a picking executing mechanism 1, a mechanical arm 2, a binocular vision device 3 and a control device 4.
The utility model discloses a picking machine, including work platform 5, control device, mechanical arm 2, mechanical arm 1, binocular vision device 3, crawler-type moving device 4, wherein work platform 5 is hollow cavity structure, crawler-type moving device 6 sets up the bottom of work platform 5, mechanical arm 2 sets up the center department of work platform 5 upper surface, picking actuating mechanism 1 sets up the top of mechanical arm 2, binocular vision device 3 sets up work platform 5 upper surface, control device 4 sets up in the hollow cavity structure of work platform 5, binocular vision device 3 with control device 4 electricity is connected, control device 4 respectively with picking actuating mechanism 1 with crawler-type moving device 6 electricity is connected. Specifically, the binocular vision device 3 is electrically connected with the control device 4 through a data line, and the control device 4 is electrically connected with the picking actuator 1 and the crawler-type moving device 6 through signal lines respectively.
The control device 4 is used for generating a walking instruction according to a pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device 6 to advance to a picking destination.
The binocular vision device 3 is used for collecting oil tea fruit images within a preset range of the picking destination and determining picking information of the oil tea fruits according to the oil tea fruit images.
The control device 4 is further used for generating picking instructions according to the picking information, sending the picking instructions to the picking actuating mechanism 1 and the mechanical arm 2 respectively, and the picking instructions are used for driving the picking actuating mechanism 1 and the mechanical arm 2 to pick oil tea fruits.
Specifically, 360 degrees are included in the preset range, the binocular vision device judges whether oil tea fruit fruits exist in the pickable range according to the collected oil tea fruit images, if so, the control device drives the picking actuating mechanism and the mechanical arm to pick the fruits through picking information, if not, the control device sends out control signals to enable the binocular vision device to rotate 90 degrees and count once, meanwhile, whether the count number of times reaches 4 times is judged, if the count number of times does not reach 4 times, the control device drives the picking actuating mechanism and the mechanical arm to pick the fruits through picking information, and if the count number of times reaches 4 times, the control device drives the crawler-type moving device to advance to the next picking destination.
In the above embodiment, the fruit picking device can walk according to the planned travel route, pick fruits at a plurality of picking destinations on the travel route, acquire the image of the oil tea fruit through the binocular vision device when reaching the picking destinations, process the image of the oil tea fruit to obtain picking information, obtain picking instructions for controlling the picking actuating mechanism and the mechanical arm to implement picking according to the picking information, realize automatic picking of the oil tea fruit, save labor and improve efficiency.
Alternatively, as an embodiment of the present invention, the binocular vision apparatus 3 includes two cameras and a vision image processor, the two cameras are disposed on the upper surface of the working platform 5 and are respectively located at both sides of the robot arm 2.
The camera is used for acquiring the oil tea fruit images within the preset range.
The visual image processor is configured to perform preprocessing on the oil tea fruit image to obtain a preprocessed oil tea fruit image, identify a plurality of oil tea fruit contours from the preprocessed oil tea fruit image, perform three-dimensional positioning processing on the plurality of oil tea fruit contours respectively to obtain three-dimensional positioning information corresponding to the plurality of oil tea fruit contours, calculate the obtained three-dimensional positioning information to obtain picking information, and send the picking information to the control device 4.
Specifically, the camera is a 360 ° rotatable camera mounted on the upper surface of the work platform 5 through a rotation shaft.
Alternatively, as an embodiment of the present invention, as shown in fig. 4, the visual image processor includes:
the image preprocessing unit is used for carrying out image filtering processing on the oil tea fruit image, carrying out brightness histogram equalization processing on the oil tea fruit image subjected to the image filtering processing, carrying out binarization processing on the oil tea fruit image subjected to the equalization processing, and carrying out corrosion expansion processing on the oil tea fruit image subjected to the binarization processing to obtain a preprocessed oil tea fruit image.
The three-dimensional positioning unit is used for extracting characteristics of the preprocessed oil tea fruit images, identifying a plurality of oil tea fruit outlines, respectively carrying out three-dimensional matching processing on the plurality of oil tea fruit outlines, respectively carrying out three-dimensional reconstruction processing on the plurality of oil tea fruit outlines subjected to the three-dimensional matching processing, and obtaining a plurality of three-dimensional positioning information.
The picking point gravity center calculating unit is used for respectively carrying out polar coordinate transformation processing on the plurality of three-dimensional positioning information to obtain a plurality of polar coordinate data, respectively carrying out distance transformation processing on the plurality of polar coordinate data to obtain a plurality of distance value data, respectively carrying out linear fitting on the plurality of distance value data to obtain a plurality of groups of fruit picking position information and attitude angle information.
And a fruit number determining unit for determining the number of picked fruits based on the plurality of fruit picking position information.
And the sending unit is used for sending the plurality of groups of fruit picking position information, the attitude angle information and the number of picked fruits as the picking information to the control device 4.
In the above embodiment, the oil tea fruit image is preprocessed to obtain clear oil tea fruit picking, so that feature extraction is facilitated, three-dimensional reconstruction processing is performed on the oil tea fruit to obtain three-dimensional positioning information, and the three-dimensional positioning information processing is performed to obtain the center of gravity position and the oil tea fruit contour fitting so as to further determine the oil tea fruit picking position information and the attitude angle information.
Alternatively, as an embodiment of the present invention, the control device 4 includes:
and the path planning unit is used for generating a walking instruction according to the pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device 6 to advance to a picking destination.
And the control processing unit is used for sending an acquisition instruction to the binocular vision device 3 when the picking destination is reached, wherein the acquisition instruction is used for driving the binocular vision device 3 to acquire the oil tea fruit images within the preset range of the picking destination.
And when the multi-group fruit picking position information and the gesture angle information and the picking fruit number sent by the binocular vision device 3 are received, carrying out sequential label processing on the multi-group fruit picking position information and the gesture angle information according to the picking fruit number, obtaining a mechanical arm picking instruction and a picking executing mechanism picking instruction according to the labeled various groups of fruit picking position information and gesture angle information, sending the mechanical arm picking instruction to the mechanical arm 2 and the picking executing mechanism picking instruction to the picking executing mechanism 1 according to label sequence, wherein the mechanical arm picking instruction is used for driving the mechanical arm 2 to move, and the picking executing mechanism picking instruction is used for driving the picking executing mechanism 1 to execute picking action.
In the above embodiment, the fruit picking position information and the attitude angle information of each group are sequentially labeled, and the mechanical arm picking instruction and the picking executing mechanism picking instruction are obtained according to the sequence, so that the mechanical arm and the picking executing mechanism can sequentially execute the instructions, and the oil tea fruit picking can be orderly and effectively automatically implemented.
Optionally, as an embodiment of the present invention, the control device 4 further includes a judging unit.
The judging unit is configured to judge whether to execute and complete each labeled group of picking instructions belonging to a current picking destination, and if so, generate a next travelling instruction according to a pre-planned travelling route, where the next travelling instruction is used to drive the crawler-type moving device 6 to advance to the next picking destination.
Optionally, as an embodiment of the present invention, before generating the next travel instruction, the determining unit is further configured to determine whether to travel to complete the pre-planned travel route according to the obtained current positioning information of the working platform, if yes, send a standby instruction to the control processing unit, and if no, generate the next travel instruction.
In the embodiment, the oil tea fruits can be picked at each picking point in the pre-planned path, so that the picking efficiency is greatly improved.
Fig. 2 is a schematic flow chart of a method for picking oil tea fruits according to an embodiment of the present invention.
Alternatively, as another embodiment of the present invention, as shown in fig. 2, a binocular vision-based oil tea fruit picking control method includes the steps of:
the control device 4 generates a travel instruction for driving the crawler-type moving device 6 to advance to the picking destination according to the pre-planned travel route.
The binocular vision device 3 collects oil tea fruit images within a preset range of a picking destination, determines picking information of the oil tea fruits according to the oil tea fruit images, and sends the picking information to the control device 4.
The control device 4 generates picking instructions according to the picking information, and sends the picking instructions to the picking executing mechanism 1 and the mechanical arm 2 respectively, wherein the picking instructions are used for driving the picking executing mechanism 1 and the mechanical arm 2 to pick oil tea fruits.
Specifically, as shown in fig. 4, the process of setting the pre-planned travel route is as follows:
and determining a programmable path according to a preset navigation map. The navigation map comprises geographical positions for planting oil tea fruit trees in an oil tea fruit garden.
The path orientation and maximum extent of advancement are determined based on the road width, road length, road grade, and road angle of the programmable path.
The pickable range is determined based on the range of motion of the robotic arm, thereby determining the track travel step size of the crawler.
Optionally, as an embodiment of the present invention, the process of determining picking information of the oil tea fruit according to the oil tea fruit image includes:
preprocessing the oil tea fruit image to obtain a preprocessed oil tea fruit image, identifying a plurality of oil tea fruit outlines from the preprocessed oil tea fruit image, respectively performing three-dimensional positioning processing on the plurality of oil tea fruit outlines to obtain three-dimensional positioning information corresponding to the plurality of oil tea fruit outlines, and calculating the obtained three-dimensional positioning information to obtain picking information.
Alternatively, as an embodiment of the present invention, as shown in fig. 5, the process of obtaining the preprocessed oil tea fruit image includes:
performing image filtering treatment on the oil tea fruit image, performing brightness histogram equalization treatment on the oil tea fruit image subjected to the image filtering treatment, performing binarization treatment on the oil tea fruit image subjected to the equalization treatment, and performing corrosion expansion treatment on the oil tea fruit image subjected to the binarization treatment to obtain a pretreated oil tea fruit image.
The process for obtaining the three-dimensional positioning information corresponding to the outlines of the plurality of oil tea fruits comprises the following steps:
and carrying out feature extraction on the preprocessed oil tea fruit images, identifying a plurality of oil tea fruit contours, respectively carrying out three-dimensional matching treatment on the plurality of oil tea fruit contours, respectively carrying out three-dimensional reconstruction treatment on the plurality of oil tea fruit contours subjected to the three-dimensional matching treatment, and obtaining a plurality of three-dimensional positioning information.
The picking information obtaining process comprises the following steps:
and respectively carrying out polar coordinate transformation processing on the plurality of three-dimensional positioning information to obtain a plurality of polar coordinate data, respectively carrying out distance transformation processing on the plurality of polar coordinate data to obtain a plurality of distance value data, and respectively carrying out linear fitting on the plurality of distance value data to obtain a plurality of groups of fruit picking position information and attitude angle information.
And determining the number of picked fruits according to the plurality of fruit picking position information, and taking the plurality of groups of fruit picking position information, the attitude angle information and the number of picked fruits as the picking information.
Alternatively, as an embodiment of the present invention, the process of generating the picking instruction by the control device 4 according to the picking information includes:
when the picking destination is reached, an acquisition instruction is sent to the binocular vision device 3, and the acquisition instruction is used for driving the binocular vision device 3 to acquire the image of the oil tea fruits within the preset range of the picking destination.
When the multi-group fruit picking position information and the gesture angle information and the picking fruit number sent by the binocular vision device 3 are received, sequential label processing is carried out on the multi-group fruit picking position information and the multi-group fruit picking gesture angle information according to the picking fruit number, a mechanical arm picking instruction and a picking executing mechanism picking instruction are obtained according to each group of labeled fruit picking position information and gesture angle information, the mechanical arm picking instruction is sent to the mechanical arm 2 according to label sequences, the picking executing mechanism picking instruction is sent to the picking executing mechanism 1, the mechanical arm picking instruction is used for driving the mechanical arm 2 to move, and the picking executing mechanism picking instruction is used for driving the picking executing mechanism 1 to execute picking actions.
Specifically, the process of the mechanical arm executing the picking instruction is as follows:
the mechanical arm picking instruction is specifically used for planning a motion track of the mechanical arm, namely a fruit picking track and a designated fruit picking position point.
The control device sends a mechanical arm picking instruction to a driving unit of the mechanical arm, and the driving unit of the mechanical arm drives the mechanical arm to move according to the movement track.
Judging whether the mechanical arm reaches a designated fruit picking position, and if the mechanical arm does not reach the designated fruit picking position, re-planning the motion track of the mechanical arm; if the mechanical arm reaches the designated fruit picking position, the mechanical arm stops acting and sends a feedback signal to the control device.
Specifically, the picking executing mechanism executes the picking instruction as follows:
the picking instruction of the picking executing mechanism is used for driving the picking executing mechanism to execute picking action.
The control device sends a picking instruction of the picking actuating mechanism to a driving unit of the picking actuating mechanism, and the driving unit of the picking actuating mechanism drives the picking actuating mechanism to wait for action.
When the control device receives a feedback signal of the mechanical arm, the feedback signal is a signal which reaches a designated fruit picking position and is adjusted to a preset fruit picking gesture, a picking instruction of the picking actuating mechanism is sent to a driving unit of the picking actuating mechanism again, and the driving unit of the picking actuating mechanism drives the picking actuating mechanism to perform one-time picking action.
The driving unit of the picking executing mechanism sends a feedback signal to the control device and enters the next action waiting mode.
Optionally, as an embodiment of the present invention, when the picking instruction of the current label is executed, the method further includes the steps of: judging whether each group of marked picking instructions belonging to the current picking destination is executed or not, and if so, generating a next walking instruction according to a pre-planned walking route, wherein the next walking instruction is used for driving the crawler-type moving device to advance to the next picking destination.
Optionally, as an embodiment of the present invention, before generating the next travel instruction, the method further includes the steps of: judging whether to walk to complete a pre-planned walking path according to the obtained current positioning information of the working platform, if yes, sending a standby instruction to the control processing unit, and if not, generating a next walking instruction.
Optionally, as an embodiment of the present invention, the method further includes the steps of: the binocular vision device judges whether oil tea fruit exists in a picking range according to the acquired oil tea fruit image, if so, the control device drives the picking actuating mechanism and the mechanical arm to pick fruits, if not, the control device sends a control signal to enable the binocular vision device to rotate 90 degrees and count once, meanwhile, whether the count number reaches 4 times is judged, if not, the control device drives the picking actuating mechanism and the mechanical arm to pick fruits through the picking information, and if so, the control device drives the crawler-type moving device to move forward to the next picking destination. The preset range is 360 degrees, namely 4 times of rotation are performed.
Specifically, the binocular vision device further comprises a counting device, and before the picking instruction is executed, the control device sends zero clearing to the timing device to zero the counting device.
As shown in fig. 3, the following embodiment shows the overall implementation of the control device to control the picking of oil tea fruits.
And S0, powering up to initialize and starting the picking system.
And S1, the control device generates a walking instruction according to a pre-planned walking path, and the crawler-type moving device moves to a picking starting position in the path planning and moves to a picking destination according to the walking instruction.
S2, resetting the mechanical arm base, simultaneously recording the reset of a counting device for the rotation of the mechanical arm, shooting an oil tea fruit image within a preset range of a picking destination by a camera of a binocular vision device, processing the oil tea fruit image by a vision image processor, analyzing the positions of oil tea fruits according to the processed oil tea fruit image, obtaining a plurality of groups of fruit picking position information, attitude angle information and the number of picked fruits, and transmitting the information and the information to a control device.
Step S3: judging whether oil tea fruits exist in the picking range in the current oil tea fruit image, and if so, executing the step S4; if not, the control device sends out a control signal to enable the binocular vision device to rotate 90 degrees, counts once, judges whether the count times reach 4 times, and returns to execute the step S2 again if the count times do not reach 4 times; if the counted number of times is 4, the step S8 is executed.
Step S4: the control device carries out sequential label processing and storage on the plurality of groups of fruit picking position information and attitude angle information according to the number of picked fruits, simultaneously plans a mechanical arm movement track in the picking process of each oil tea fruit according to the positions of the oil tea fruits, and obtains a mechanical arm picking instruction and a picking executing mechanism picking instruction according to the fruit picking position information and the fruit picking attitude angle information.
Step S5: the control device transmits the mechanical arm picking instruction to the driving unit of the mechanical arm, the driving unit of the mechanical arm drives the mechanical arm to move according to the preset movement track, whether the designated fruit picking position is reached or not is judged, if the designated fruit picking position is not reached, the mechanical arm is driven to move according to the preset movement track again, and if the movement of the mechanical arm is stopped, the step S6 is executed.
Step S6: the control device transmits the picking instruction of the picking executing mechanism to the picking executing mechanism, and when the mechanical arm reaches the designated fruit picking position and is adjusted to a preset fruit picking gesture, the picking executing mechanism executes the action to finish one-time fruit picking.
Step S7: judging whether all picking instructions are finished according to the label sequence, if so, executing step S8; if not, picking all fruits in the picking range, and returning to the step S5.
Step S8: judging whether the walking is completed on the pre-planned walking path, if not, driving the crawler-type moving device to move to the next picking position of the pre-planned walking path by the control device, and returning to the step S2; if the route is completed, the system enters a standby mode, and gives an alarm of stopping the operation, and the system ends.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and units described above may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present invention.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The present invention is not limited to the above embodiments, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and these modifications and substitutions are intended to be included in the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (4)
1. A binocular vision-based oil tea fruit picking system is characterized by comprising a working platform (5), a crawler-type moving device (6), a picking actuating mechanism (1), a mechanical arm (2), a binocular vision device (3) and a control device (4),
the automatic picking device comprises a working platform (5), a crawler-type moving device (6), a mechanical arm (2), a picking executing mechanism (1), a binocular vision device (3) and a control device (4), wherein the working platform (5) is of a hollow cavity structure, the crawler-type moving device (6) is arranged at the bottom of the working platform (5), the mechanical arm (2) is arranged at the center of the upper surface of the working platform (5), the picking executing mechanism (1) is arranged at the top end of the mechanical arm (2), the binocular vision device (3) is arranged on the upper surface of the working platform (5), the control device (4) is arranged in the hollow cavity structure of the working platform (5), the binocular vision device (3) is electrically connected with the control device (4), and the control device (4) is electrically connected with the picking executing mechanism (1) and the crawler-type moving device (6) respectively;
the control device (4) is used for generating a walking instruction according to a pre-planned walking route, and the walking instruction is used for driving the crawler-type moving device (6) to advance to a picking destination;
the binocular vision device (3) is used for collecting oil tea fruit images within a preset range of a picking destination and determining picking information of the oil tea fruits according to the oil tea fruit images;
the control device (4) is further used for generating picking instructions according to the picking information, the picking instructions are respectively sent to the picking executing mechanism (1) and the mechanical arm (2), and the picking instructions are used for driving the picking executing mechanism (1) and the mechanical arm (2) to pick oil tea fruits;
the binocular vision device (3) comprises two cameras and a vision image processor, wherein the two cameras are arranged on the upper surface of the working platform (5) and are respectively positioned on two sides of the mechanical arm (2);
the camera is used for acquiring an image of the oil tea fruit in the preset range;
the visual image processor is used for preprocessing the oil tea fruit image to obtain a preprocessed oil tea fruit image, identifying a plurality of oil tea fruit outlines from the preprocessed oil tea fruit image, respectively performing three-dimensional positioning processing on the plurality of oil tea fruit outlines to obtain three-dimensional positioning information corresponding to the plurality of oil tea fruit outlines, calculating the obtained three-dimensional positioning information to obtain picking information, and sending the picking information to the control device (4);
the visual image processor includes:
the image preprocessing unit is used for carrying out image filtering processing on the oil tea fruit image, carrying out brightness histogram equalization processing on the oil tea fruit image subjected to the image filtering processing, carrying out binarization processing on the oil tea fruit image subjected to the equalization processing, and carrying out corrosion expansion processing on the oil tea fruit image subjected to the binarization processing to obtain a preprocessed oil tea fruit image;
the three-dimensional positioning unit is used for extracting characteristics of the preprocessed oil tea fruit images, identifying a plurality of oil tea fruit contours, respectively carrying out three-dimensional matching processing on the plurality of oil tea fruit contours, respectively carrying out three-dimensional reconstruction processing on the plurality of oil tea fruit contours subjected to the three-dimensional matching processing, and obtaining a plurality of three-dimensional positioning information;
the picking point gravity center calculating unit is used for respectively carrying out polar coordinate transformation processing on the plurality of three-dimensional positioning information to obtain a plurality of polar coordinate data, respectively carrying out distance transformation processing on the plurality of polar coordinate data to obtain a plurality of distance value data, and respectively carrying out linear fitting on the plurality of distance value data to obtain a plurality of groups of fruit picking position information and attitude angle information;
a fruit number determining unit for determining a picked fruit number from the plurality of fruit picking position information;
a transmitting unit for transmitting the plurality of sets of fruit picking position information and attitude angle information and the number of picked fruits as the picking information to the control device (4);
the control device (4) comprises:
the path planning unit is used for generating a walking instruction according to a pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device (6) to advance to a picking destination;
the control processing unit is used for sending an acquisition instruction to the binocular vision device (3) when the picking destination is reached, wherein the acquisition instruction is used for driving the binocular vision device (3) to acquire an image of oil tea fruits within a preset range of the picking destination;
and the picking device is also used for carrying out sequential label processing on the plurality of groups of fruit picking position information and posture angle information according to the number of picked fruits when the plurality of groups of fruit picking position information and posture angle information sent by the binocular vision device (3) and the picking device is received, obtaining a mechanical arm picking instruction and a picking executing mechanism picking instruction according to the labeled groups of fruit picking position information and posture angle information, sending the mechanical arm picking instruction to the mechanical arm (2) and the picking executing mechanism picking instruction to the picking executing mechanism (1) according to the label sequence, wherein the mechanical arm picking instruction is used for driving the mechanical arm (2) to move, and the picking executing mechanism picking instruction is used for driving the picking executing mechanism (1) to execute picking action.
2. The oil tea fruit picking system according to claim 1, wherein the control device (4) further comprises a judging unit;
the judging unit is used for judging whether each group of marked picking instructions belonging to the current picking destination is executed or not, and if so, generating a next walking instruction according to a pre-planned walking route, wherein the next walking instruction is used for driving the crawler-type moving device (6) to move forward to the next picking destination.
3. The oil tea fruit picking system according to claim 2, wherein the judging unit is further configured to judge whether to walk to complete a pre-planned route according to the obtained current positioning information of the working platform before generating the next walking instruction, if yes, send a standby instruction to the control processing unit, and if not, generate the next walking instruction.
4. The binocular vision-based oil tea fruit picking control method is characterized by comprising the following steps of:
the control device generates a walking instruction according to a pre-planned walking path, and the walking instruction is used for driving the crawler-type moving device to advance to a picking destination;
the binocular vision device acquires an image of the oil tea fruit within a preset range of a picking destination, and determines picking information of the oil tea fruit according to the image of the oil tea fruit;
the control device generates picking instructions according to the picking information, and sends the picking instructions to a picking executing mechanism and a mechanical arm respectively, wherein the picking instructions are used for driving the picking executing mechanism and the mechanical arm to pick oil tea fruits;
the process for determining picking information of the oil tea fruits according to the oil tea fruit images comprises the following steps:
preprocessing the oil tea fruit image to obtain a preprocessed oil tea fruit image, identifying a plurality of oil tea fruit contours from the preprocessed oil tea fruit image, respectively performing three-dimensional positioning processing on the plurality of oil tea fruit contours to obtain three-dimensional positioning information corresponding to the plurality of oil tea fruit contours, and calculating the obtained three-dimensional positioning information to obtain picking information;
the process for obtaining the preprocessed oil tea fruit image comprises the following steps:
performing image filtering treatment on the oil tea fruit image, performing brightness histogram equalization treatment on the oil tea fruit image subjected to the image filtering treatment, performing binarization treatment on the oil tea fruit image subjected to the equalization treatment, and performing corrosion expansion treatment on the oil tea fruit image subjected to the binarization treatment to obtain a pretreated oil tea fruit image;
the process for obtaining the three-dimensional positioning information corresponding to the outlines of the plurality of oil tea fruits comprises the following steps:
extracting features of the preprocessed oil tea fruit images, identifying a plurality of oil tea fruit contours, respectively carrying out three-dimensional matching treatment on the plurality of oil tea fruit contours, respectively carrying out three-dimensional reconstruction treatment on the plurality of oil tea fruit contours subjected to the three-dimensional matching treatment, and obtaining a plurality of three-dimensional positioning information;
the picking information obtaining process comprises the following steps:
respectively carrying out polar coordinate transformation processing on the plurality of three-dimensional positioning information to obtain a plurality of polar coordinate data, respectively carrying out distance transformation processing on the plurality of polar coordinate data to obtain a plurality of distance value data, and respectively carrying out linear fitting on the plurality of distance value data to obtain a plurality of groups of fruit picking position information and attitude angle information;
determining the number of picked fruits according to the plurality of fruit picking position information, and taking the plurality of groups of fruit picking position information, the attitude angle information and the number of picked fruits as the picking information;
the process of generating picking instructions by the control device according to the picking information comprises the following steps:
when the picking destination is reached, sending an acquisition instruction to the binocular vision device, wherein the acquisition instruction is used for driving the binocular vision device to acquire an image of the oil tea fruit in a preset range of the picking destination;
when the multi-group fruit picking position information and the gesture angle information and the picking fruit number sent by the binocular vision device are received, sequential label processing is carried out on the multi-group fruit picking position information and the multi-group fruit picking gesture angle information according to the picking fruit number, a mechanical arm picking instruction and a picking executing mechanism picking instruction are obtained according to the labeled multi-group fruit picking position information and gesture angle information, the mechanical arm picking instruction is sent to the mechanical arm according to label sequences, the picking executing mechanism picking instruction is sent to the picking executing mechanism, the mechanical arm picking instruction is used for driving the mechanical arm to move, and the picking executing mechanism picking instruction is used for driving the picking executing mechanism to execute picking action.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910124070.0A CN109729835B (en) | 2019-02-19 | 2019-02-19 | Binocular vision-based oil tea fruit picking system and control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910124070.0A CN109729835B (en) | 2019-02-19 | 2019-02-19 | Binocular vision-based oil tea fruit picking system and control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109729835A CN109729835A (en) | 2019-05-10 |
| CN109729835B true CN109729835B (en) | 2024-01-12 |
Family
ID=66367777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910124070.0A Active CN109729835B (en) | 2019-02-19 | 2019-02-19 | Binocular vision-based oil tea fruit picking system and control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109729835B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113099847B (en) * | 2021-05-25 | 2022-03-08 | 广东技术师范大学 | Fruit picking method based on fruit three-dimensional parameter prediction model |
| CN114731845A (en) * | 2022-02-24 | 2022-07-12 | 华南农业大学 | Receiving equipment is picked to portable navel orange |
| CN115316129B (en) * | 2022-08-12 | 2024-03-12 | 华南农业大学 | Self-adaptive bionic picking device based on binocular vision recognition and fruit stringing picking method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101356877A (en) * | 2008-09-19 | 2009-02-04 | 中国农业大学 | Cucumber picking robot system and picking method in greenhouse |
| CN102165880A (en) * | 2011-01-19 | 2011-08-31 | 南京农业大学 | Automatic-navigation crawler-type mobile fruit picking robot and fruit picking method |
| CN102662399A (en) * | 2012-05-09 | 2012-09-12 | 江苏科技大学 | Method for optimizing path for fruit picking robot facing towards multiple targets |
| CN104700404A (en) * | 2015-03-02 | 2015-06-10 | 中国农业大学 | Fruit location identification method |
| CN108901362A (en) * | 2018-08-15 | 2018-11-30 | 湖南师范大学 | A kind of mung bean picking robot |
| CN209572477U (en) * | 2019-02-19 | 2019-11-05 | 广西师范大学 | A kind of camellia oleifera fruit picker |
-
2019
- 2019-02-19 CN CN201910124070.0A patent/CN109729835B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101356877A (en) * | 2008-09-19 | 2009-02-04 | 中国农业大学 | Cucumber picking robot system and picking method in greenhouse |
| CN102165880A (en) * | 2011-01-19 | 2011-08-31 | 南京农业大学 | Automatic-navigation crawler-type mobile fruit picking robot and fruit picking method |
| CN102662399A (en) * | 2012-05-09 | 2012-09-12 | 江苏科技大学 | Method for optimizing path for fruit picking robot facing towards multiple targets |
| CN104700404A (en) * | 2015-03-02 | 2015-06-10 | 中国农业大学 | Fruit location identification method |
| CN108901362A (en) * | 2018-08-15 | 2018-11-30 | 湖南师范大学 | A kind of mung bean picking robot |
| CN209572477U (en) * | 2019-02-19 | 2019-11-05 | 广西师范大学 | A kind of camellia oleifera fruit picker |
Non-Patent Citations (2)
| Title |
|---|
| 基于ADAMS的油茶果采摘机械臂试验设计及优化;张立勇;饶洪辉;罗时挺;余佳佳;刘木华;;农机化研究(第11期) * |
| 基于Matlab的油茶果机械采摘平台设计与试验;张立勇;饶洪辉;黄登昇;陈斌;刘木华;;南方农机(第21期) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109729835A (en) | 2019-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109729835B (en) | Binocular vision-based oil tea fruit picking system and control method | |
| CN112243698B (en) | Automatic walnut picking and collecting method based on multi-sensor fusion technology | |
| EP3811748B1 (en) | A weeding machine and a method for carrying out weeding using the weeding machine | |
| CN104714547B (en) | Autonomous gardens vehicle with camera | |
| CN111178148B (en) | Ground target geographic coordinate positioning method based on unmanned aerial vehicle vision system | |
| WO2015180021A1 (en) | Pruning robot system | |
| CN106965179A (en) | The vision positioning system and method for a kind of industrial robot | |
| CN113982690B (en) | Control method, device, system and tunneling, anchoring and protecting integrated machine based on anchoring operation | |
| CN109528089A (en) | A kind of walk on method, apparatus and the chip of stranded clean robot | |
| CN108801269A (en) | A kind of interior cloud Algorithms of Robots Navigation System and method | |
| CN106871902A (en) | A kind of method of Navigation of Pilotless Aircraft, device and system | |
| CN110433467A (en) | Picking up table tennis ball robot operation method and equipment based on binocular vision and ant group algorithm | |
| CN216058333U (en) | Intelligent movement fruit picking robot | |
| CN108811766A (en) | A kind of man-machine interactive fruits and vegetables of greenhouse harvesting robot system and its collecting method | |
| CN111862200B (en) | Unmanned aerial vehicle positioning method in coal shed | |
| CN208289901U (en) | A kind of positioning device and robot enhancing vision | |
| CN107272037A (en) | A kind of road equipment position, image information collecting device and the method for gathering information | |
| CN113812262B (en) | Tea-oil camellia fruit picking robot based on machine vision | |
| CN112414410A (en) | Path generation method, device operation method and device control system | |
| CN116508493B (en) | Gantry crawler type tea picking robot and picking method thereof | |
| CN111328554B (en) | Automatic strawberry picking robot | |
| CN211240896U (en) | Bionic snake mouth picking mechanism with visual positioning function and unmanned aerial vehicle harvesting device | |
| CN115589845A (en) | Intelligent cotton picking robot and cotton picking operation path planning method thereof | |
| CN116466724A (en) | Mobile positioning method and device of robot and robot | |
| CN109782771A (en) | A kind of orchard mobile robot and edge of a field forward method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |