CN115250749A

CN115250749A - Fruit picking robot and method

Info

Publication number: CN115250749A
Application number: CN202211082270.2A
Authority: CN
Inventors: 陈海波; 沈大农; 薛山
Original assignee: Deep Blue Technology Shanghai Co Ltd
Current assignee: Deep Blue Technology Shanghai Co Ltd
Priority date: 2022-09-06
Filing date: 2022-09-06
Publication date: 2022-11-01

Abstract

The invention discloses a fruit picking robot and a method, wherein the fruit picking robot comprises: the picking device comprises a camera, a control system, a picking bracket and a plurality of picking arms which are supported to move in a three-dimensional space; wherein the camera is arranged on the picking bracket; a plurality of picking arms are all arranged on the picking bracket; the camera collects information of fruits and surrounding environments of the fruits, transmits the information to the control system, and the control system formulates a picking strategy according to the information and controls the picking arms to pick the fruits according to the picking strategy. The invention can realize high-speed picking operation, namely, the high-speed picking operation is finished by a multi-arm union formed by combining a plurality of picking arms, a reasonable picking strategy can be formulated, the efficiency is improved, and the picking accuracy is improved.

Description

Fruit picking robot and method

Technical Field

The invention relates to the technical field of agricultural picking, in particular to a fruit picking robot and a fruit picking method.

Background

The technical routes of apple picking robots at home and abroad mainly include two types: 1) Six-axis robot hands are used for simulating human behaviors for picking. 2) Is in a barrel type structure, and apples are sucked into a pipe for picking. The picking mode of the single mechanical arm is adopted, the efficiency is low, and the good picking efficiency for replacing manpower cannot be achieved.

Disclosure of Invention

The invention mainly aims to provide a picking shear, a fruit picking robot and a method, which are used for realizing high-speed picking operation.

According to a first aspect of the present invention, there is provided a fruit picking robot comprising: the picking device comprises a camera, a control system, a picking bracket and a plurality of picking arms which are supported to move in a three-dimensional space; wherein the camera is arranged on the picking bracket; a plurality of picking arms are all arranged on the picking bracket; the camera collects information of fruits and surrounding environments of the fruits, transmits the information to the control system, and the control system formulates a picking strategy according to the information and controls the picking arms to pick the fruits according to the picking strategy.

Furthermore, picking arm front end is for picking scissors, picking scissors includes the face, certainly a plurality of strengthening ribs that extend from the face to and set up the cutting part between adjacent strengthening rib.

Further, the blade part is connected with the adjacent reinforcing ribs, and the edge of the blade part faces the palm surface and is sunken.

Furthermore, the head of each reinforcing rib, the side edge of the palm surface and the reinforcing rib positioned on the outermost side are all provided with a buffer piece.

Furthermore, the picking arms are arranged in an array.

Furthermore, the camera is a 3D depth camera.

Further, the step of the control system formulating the picking strategy according to the information comprises: the control system distinguishes the type and the position of the fruit to be picked and makes a strategy of layer-by-layer identification.

Further, the layer-by-layer identification strategy includes: from outside to inside and from bottom to top.

Further, the method also comprises the following steps: the movable chassis is arranged on the fixed frame on the movable chassis, and the picking bracket is movably arranged on the fixed frame.

According to a second aspect of the present invention, there is provided a fruit picking method comprising: picking fruit to be picked using a fruit picking robot as described in the first aspect.

Further, the method comprises:

arranging the fruit picking robot beside the fruit tree;

preliminarily adjusting the picking arm to be approximately flush with the crown of the fruit tree;

the camera collects fruit tree information and transmits the fruit tree information to a control system;

the control system formulates a picking strategy according to the fruit tree information; and

and the control system controls the picking arms to pick fruits according to the picking strategies.

Further, the step of the control system formulating the picking strategy according to the fruit tree information comprises: the control system distinguishes the fruits to be picked and the positions of the fruits, and a strategy for layer-by-layer identification is formulated.

Further, the process of controlling the picking arm to pick fruit according to the picking strategy by the control system comprises:

defining a picking area, and dividing the picking area into a plurality of divided areas, wherein each divided area is used as an operation area of a picking arm;

picking by the picking arms corresponding to each segmentation area according to the obtained positions.

Further, the controlling system controlling the picking arm to pick fruit according to the picking strategy comprises:

when the picking arms pick, picking is carried out according to the sequence from outside to inside and/or from bottom to top.

Compared with the prior art, the fruit picking robot provided by the invention comprises: the picking device comprises a camera, a control system, a picking bracket and a plurality of picking arms which are supported to move in a three-dimensional space; wherein the camera is arranged on the picking bracket; the picking arms are arranged on the picking bracket; the camera collects information of fruits and surrounding environments of the fruits, transmits the information to the control system, and the control system formulates a picking strategy according to the information and controls the picking arms to pick the fruits according to the picking strategy. Therefore, the invention can realize high-speed picking operation by means of a plurality of movable picking arms, namely, the multi-arm complex formed by combining a plurality of picking arms is used for finishing the high-speed picking operation; in addition, the invention can formulate a more reasonable picking strategy through the information acquisition of the camera and the analysis and the processing of the control system, thereby improving the efficiency, improving the picking accuracy and reducing the damage to the fruits. The invention can also effectively solve the labor problem.

Drawings

Fig. 1 is a first schematic structural diagram of a fruit picking robot according to a first embodiment of the present invention;

fig. 2 is a schematic structural view of a picking shears in the first embodiment of the present invention;

fig. 3 is a schematic structural diagram of a fruit picking robot according to the first embodiment of the present invention;

fig. 4 is a schematic structural diagram three of a fruit picking robot in the first embodiment of the invention;

FIG. 5 is a schematic view of a fruit picking process in an embodiment of the present invention;

fig. 6 is a flow chart of a fruit picking method in the second embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate orientations or positional relationships based on orientations or positional relationships shown in the drawings or orientations or positional relationships that the present product is conventionally placed in use, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.

Furthermore, the terms "horizontal", "vertical" and the like do not imply that the components are required to be absolutely horizontal or pendant, but rather may be slightly inclined. For example, "horizontal" merely means that the direction is more horizontal than "vertical" and does not mean that the structure must be perfectly horizontal, but may be slightly inclined.

In the description of the present invention, it should also be noted that, unless otherwise explicitly stated or limited, the terms "disposed," "mounted," "connected," and "connected" are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.

The following describes the present invention in detail.

Example one

Referring to fig. 1 to 4, in a first embodiment of the present invention, a fruit picking robot is provided, including: a camera 8, a control system (not shown), a picking bracket 4 and a plurality of picking arms supported for movement in a three-dimensional space; wherein the camera 8 is arranged on the picking bracket 4, and the picking arms 7 are all arranged on the picking bracket 4; the camera 8 collects information of fruits and surrounding environments of the fruits and transmits the information to the control system, and the control system formulates a picking strategy according to the information and controls the picking arm 7 to pick the fruits according to the picking strategy.

The control system is at least a device with AI analysis capability, which can be connected with the camera 8, picking arm 7 and other components (such as movable chassis 2, picking carriage 4, etc.) in a wired or wireless manner for signal reception and command issue.

The control system can realize independent control of a single picking arm and can also realize simultaneous control of a plurality of picking arms.

As shown in fig. 2 and 4, the picking carriage 4 is provided with the camera 8.

As an example, the picking bracket 4 is provided with a 3D depth camera in front, and can capture a 3D visual image of a fruit tree, the captured 3D visual image of the fruit tree is transmitted to the control system, the control system identifies the fruit to be picked in the fruit tree and locates the position of the fruit, and then three-dimensional coordinates (or relative coordinates) of the target picking position are given, the coordinates are transmitted to the mechanical arm through the control system to drive and control the mechanical arm, so that the picking shears are sent to the target picking position, and fruit stalks at the target picking position are sheared through the picking shears, thereby completing the whole picking operation.

In the embodiment of the invention, the control system obtains the target picking position coordinates of all picked fruits through AI analysis, enables the picking arms to execute the picking strategies from outside to inside and from bottom to top according to the strategy of layer-by-layer identification, and can select a proper number of picking arms according to the distribution condition of the fruits to realize high-speed and high-efficiency fruit picking control. The picking strategy from outside to inside refers to that aiming at fruits in a fruit tree, firstly picking the fruits which are positioned on the outer side of a crown, and then picking the fruits which are positioned on the inner side of the crown; the bottom-up picking strategy refers to that aiming at fruits in fruit trees, the fruits at the lower part of the crown are picked firstly, and then the fruits at the upper part of the crown are picked. The strategy can effectively avoid the influence on other unpinned fruits when the fruits are picked, for example, the outer fruits are picked firstly, the inner fruits are not easily influenced, the outer fruits are easily scratched by a picking arm and damaged when the inner fruits are picked firstly, or the outer fruits block the advancing path of the picking arm and are not easily picked; the fruit of lower part is picked earlier, can effectively avoid picking earlier upper portion fruit and the phenomenon of bumping with lower part fruit when fruit whereabouts, improves the integrality of picking fruit.

In addition, the control system in the embodiment of the invention can also consider the distribution conditions of the tree branches, the fixed frame and other fruits, so that the picking arm has an obstacle avoidance function.

As shown in fig. 2, the front end of the picking arm is a picking scissors 1, for example, the picking scissors 1 includes: a palm surface 101, a plurality of reinforcing ribs 102 extending from the palm surface, and blade portions 103 between adjacent reinforcing ribs 102; the blade part 103 between the reinforcing ribs 102 of the picking shears 1 can adapt to the positioning error of larger fruit stalks and the shearing operation with certain flexibility, and the reinforcing ribs 102 increase the overall rigidity of the picking shears 1 and enable the shearing operation to be more stable.

In the embodiment of the invention, the overall shape of the picking shears is preferably a duck palm shape. The duck-palm-shaped picking scissors are beneficial to obtaining a larger butt joint range of fruits, and the strength of the picking scissors can be improved. It is understood that the shape of the picking shears includes, but is not limited to, a duck's palm.

It should be noted that the duck's palm shape in the present invention is not completely close to the duck's palm, but is substantially similar, for example, the number of the reinforcing ribs illustrated in fig. 2 is 5.

Wherein, the quantity of strengthening rib can be set for according to actual need.

Further, the head of the reinforcing rib 102 (i.e. the end far away from the palm surface 101) is provided with a buffer member, for example, in the form of an encapsulation (rubber or latex, etc.), or a mosaic structure, etc. is used to wrap the head of the reinforcing rib 102, so as to avoid the tip of the head of the reinforcing rib 102 from damaging the fruit during fruit picking.

Furthermore, the side edge of the palm surface 101 and the reinforcing rib 102 at the outermost side can also be provided with the buffer member, that is, the buffer member is arranged at the edge of the picking shears except the cutting part, so that the damage to the fruits caused by the sharp end of the edge of the picking shears 1 is reduced, and the integrity of the picked fruits is improved.

In one embodiment, the shape of the palm surface 101 may be a shape enclosed by a trapezoid and an arc. It is understood that the shape of the palm surface 101 includes, but is not limited to, the above shapes, and the shapes herein are merely preferred embodiments of the shape of the palm surface 101 of the present embodiment.

The ribs 102 may be arranged in a diverging manner, for example, extending from multiple points of the arc.

The blade portion 103 connects adjacent reinforcing ribs 102 and is recessed toward the palm surface 101. In the embodiment of the invention, the cutting part 103 is specially designed, so that the fruit stalks to be picked can be cut more accurately, and the cutting part is prevented from scratching other fruits or other parts of fruit trees.

The blade 103 is generally thin so as to reduce the weight of the shears and also to help obtain a relatively sharp cutting edge to facilitate cutting of fruit stalks.

On the other hand, the blade 103 may have a gradually-changing thickness, i.e., a gradually-changing thickness (similar to a blade) away from the palm surface 101.

In the embodiment of the invention, a plurality of arc-shaped blade parts 103 are distributed at the front end of the palm surface 101, so that the cutting device can adapt to the positioning error of larger fruit stalks and has certain flexible cutting operation; the reinforcing ribs 102 increase the overall rigidity of the picking shears and enable the shearing operation to be more stable. The structure of the invention can adapt to the operation requirement of picking, shearing and cutting fruit stalks quickly, stably and efficiently.

Referring to fig. 5, the picking shears according to the embodiment of the invention can be used for picking fruits conveniently.

The fruit picking robot of this embodiment still includes following structure:

the picking device comprises a movable chassis 2 and a fixing frame 3 arranged on the movable chassis 2, wherein a picking bracket 4 is movably arranged on the fixing frame 3, a plurality of picking arms 7 are arranged on the picking bracket 4, each picking arm 7 can move in a three-dimensional space, and the front ends of the picking arms 7 are picking scissors 1.

The movable chassis 2 can be a movable trolley, for example, so as to be conveniently moved to the front of fruits and vegetables to be picked.

The fixed frame 3 is mainly used for mounting the picking bracket 4. The fixed frame 3 may be movable, for example in fig. 2, by means of a moving device 21, which may be moved on the movable chassis 2.

Wherein the moving device 21 is arranged on the movable chassis 2.

In one example, the movable chassis 2 and the fixed frame 3 may be connected by a sliding rail (not shown), so that the fixed frame 3 and the movable chassis 2 can move relatively under the driving of the moving device 21.

The moving device 21 can be driven by, but not limited to, electric, pneumatic, hydraulic, etc., and servo-controlled.

Relative movement between the fixed frame 3 and the picking carriage 4 is also possible, for example, the picking carriage 4 can be moved up and down on the fixed frame 3 by the lifting device 5.

In one example, the picking carriage 4 may be provided with a first sliding chute, the fixing frame 3 has a first sliding block 31, the first sliding block 31 is arranged in the first sliding chute, and the picking carriage 4 can slide up and down on the fixing frame 3 when the lifting device 5 is driven.

The lifting device 5 can adopt, but is not limited to, electric, pneumatic, hydraulic and other driving modes, and a servo control mode.

As an example, the plurality of picking arms 7 are arranged in an array. And because every picking arm 7 all can be controlled alone, therefore every picking arm 7 can independently work to picking arm 7 that the array form was arranged can effectively improve picking efficiency.

The picking arms have a moving mechanism 6 so that each picking arm 6 can move in three dimensions to achieve better picking capacity.

Specifically, as shown in fig. 3, the movable mechanism 6 includes: a Y-direction fixing frame 61, a Y-direction linear track 62, an X-direction fixing frame 63, an X-direction linear track 64 and a Z-direction push rod. The X/Y/Z directions are vertical two by two.

The Y-fixing frame 61 may be fixed to the picking carriage 4, specifically, on a side away from the fixing frame 3.

For example, the Y-mount 61 may be fixed to the picking carriage 4 by welding, bolts, insertion, or the like.

The Y-direction linear rail 62 may include a first rail member disposed on the Y-direction fixing frame 61 and a first sliding member disposed on the X-direction fixing frame 63, and the first sliding member is sleeved on the first rail member, so that the X-direction fixing frame 63 moves along the Y-direction.

The driving method of the Y-direction linear rail 62 can be, but not limited to, electric, pneumatic, hydraulic, and servo control.

The X-direction fixing frame 63 is connected with the push rod 65 through an X-direction linear rail 64.

The X-direction linear track 64 may include a second track member disposed on the X-direction fixing frame 63 and a second sliding member disposed on a connecting plate at the end of the push rod 65, and the second sliding member is sleeved on the second track member, so as to realize the movement of the push rod 65 along the X direction.

The driving method of the X-direction linear rail 64 can be, but not limited to, electric, pneumatic, hydraulic, etc., and servo control.

The push rod 65 can be extended or retracted, and the driving method can adopt and is not limited to electric, pneumatic, hydraulic and other driving methods, and a servo control method.

Further, the end of the push rod 5 may also be provided with a rotating mechanism, so as to realize the rotation of the push rod around the Rz direction.

Therefore, in the embodiment of the invention, the picking arm 7 can realize multi-direction and multi-degree-of-freedom design by means of the arrangement of the movable mechanism 6, the position of the picking shears can be adjusted according to the position of the fruit, and flexible picking is facilitated.

In addition, due to the array arrangement, a large picking area (such as the whole side surface of a fruit tree) can be operated, effective picking area division can be formed, and a picking arm is used for picking fruits in the division area in each division area, so that multiple picking arms can work simultaneously to carry out picking operation; in addition, each picking arm is only responsible for picking tasks in a relatively small partitioned area, and the XYZ movement distance is short, so that the movement time is short, and the picking tasks in the area can be completed quickly.

The invention can realize high-speed picking operation, namely, the operation of high-speed picking is completed by combining a plurality of mechanical arms into a multi-arm union.

In the invention, for example, N picking arms are combined into a multi-arm combined body to complete picking operation, the efficiency can be improved by N compared with that of single picking arm ² And (4) doubling.

Example two

The second embodiment of the invention provides a fruit picking method which can be realized based on the first embodiment and has the corresponding beneficial effects of the first embodiment.

In one example, as shown in fig. 6, the fruit picking method of the present invention comprises:

s1, arranging the fruit picking robot beside a fruit tree; this step can be implemented by controlling the movement of the robot by the control system.

S2, initially adjusting the picking arm to be approximately flush with the crown of the fruit tree; this step can be implemented by the control system. In this step, the picking arms to be worked are usually multiple, so that the multiple picking arms are generally flush with the crown of the fruit tree as a whole, for example, may be slightly higher than, may be slightly lower than, or may be just flush with the crown of the fruit tree, for example, the reference position may be adjusted by using the middle position of the crown as a reference, or may be adjusted according to the general distribution of the fruits. In the step, the picking arms to be worked are mainly close to the crown, so that the subsequent picking operation is facilitated, and each picking arm can still move during the subsequent picking, so that the picking arms are approximately flush in the step.

S3, collecting fruit tree information by a camera, and transmitting the fruit tree information to a control system; in this step, at least information of the fruit and its surrounding environment is collected, for example, the collected information may be 3D information, where the surrounding environment includes the tree trunk, the fixing frame, the rest of the fruit, and the like.

S4, the control system makes a picking strategy according to the fruit tree information; in this step, the control system identifies the fruit to be picked and its location and formulates a strategy for layer-by-layer identification.

And S5, controlling the picking arms to pick fruits by the control system according to the picking strategy. In this step, the method comprises the following steps:

picking arms corresponding to each divided area pick according to the obtained position (such as the coordinates of the target picking position).

Furthermore, when picking is carried out according to the established picking strategy, picking is carried out according to the sequence from outside to inside and/or from bottom to top. From this, can realize high-efficient harvesting, prevent to omit to avoid the damage to fruit.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present invention, and various changes and modifications can be made to the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and the present invention is covered thereby. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. A fruit picking robot, comprising: the picking device comprises a camera, a control system, a picking bracket and a plurality of picking arms which are supported to move in a three-dimensional space; wherein the camera is arranged on the picking bracket; the picking arms are arranged on the picking bracket; the camera collects information of fruits and surrounding environments of the fruits, transmits the information to the control system, and the control system formulates a picking strategy according to the information and controls the picking arms to pick the fruits according to the picking strategy.

2. The fruit picking robot of claim 1, wherein the picking arm forward end is a picking shears, the picking shears including a palm surface, a plurality of reinforcing ribs extending from the palm surface, and a blade portion disposed between adjacent reinforcing ribs.

3. The fruit picking robot of claim 1, wherein the blade portion connects adjacent ones of the reinforcing ribs and an edge of the blade portion is recessed toward the palm surface.

4. The fruit picking robot of claim 1, wherein the head of the reinforcing rib, the side of the palm surface and the outermost reinforcing rib are provided with a buffer.

5. The fruit picking robot of claim 1, wherein the plurality of picking arms are arranged in an array.

6. The fruit picking robot of claim 1, wherein the camera is a 3D depth camera.

7. A fruit picking robot as claimed in claim 1 or claim 6, in which the control system formulating a picking strategy in dependence on the information comprises: the control system distinguishes the type and the position of the fruit to be picked and makes a strategy of layer-by-layer identification.

8. The fruit picking robot of claim 7, wherein the layer-by-layer identification strategy comprises: from outside to inside and from bottom to top.

9. The fruit picking robot of claim 1, further comprising: the movable chassis is arranged on the fixed frame on the movable chassis, and the picking bracket is movably arranged on the fixed frame.

10. A fruit picking method, comprising: picking fruit in need of picking with a fruit picking robot according to any of claims 1-9.

11. The fruit picking process of claim 10, comprising:

arranging the fruit picking robot beside the fruit tree;

12. The fruit picking method of claim 11, wherein the control system formulating a picking strategy based on the fruit tree information comprises: the control system distinguishes the fruits to be picked and the positions of the fruits, and a strategy for layer-by-layer identification is formulated.

13. The fruit picking method of claim 12, wherein the controlling system controlling the picking arm for fruit picking according to the picking strategy comprises:

picking is carried out by the picking arm corresponding to each partition area according to the obtained position.

14. The fruit picking method of claim 13, wherein the control system controlling the picking arm to perform fruit picking according to the picking strategy comprises: