Automated harvesting apparatus
Field of the invention
The invention generally relates to an agricultural harvesting apparatus, and more specifically to devices for automatically picking fruit. Background of the invention
Mechanization of harvesting of fruits and vegetables is highly desirable for European countries due to the decreasing availability of seasonal labour and the increasing economic and social pressures. Thanks to the rapid developments in robotics, sensing and machine learning, the automated robotic system is regarded as a potential replacement to release humans from the monotonous fruit harvesting task. The research on harvesting robots has started more than 20 years ago and has resulted in various prototypes. For example, many harvesting robots for e.g. tomatoes, eggplants, lettuces etc. are known in the art. However, the developed prototypes still cannot meet the challenging requirements tor practical applications: combination of high speed and accuracy with the ability to deal with the complex, dynamic and continuously changing tasks and environment for an acceptable cost. In order to fulfill these requirements, three critical tasks need to be solved in developing a robotic system for fruit or vegetable harvesting: (1) The guidance of the robot (platform and manipulator) through the crop, (2) location and characterization of the fruit or vegetable on the tree, and (3) grasping and detachment of each piece. The task of grasping and detaching each piece, which involves a lot of domain knowledge from the picker and which requires a special end-effector for individual produce, determines the success of a robotic harvesting system. As the products are different in their shape, size, physical properties and requirements in harvesting, individual products demand a special design of the end-effector and defined harvesting method. Since the 1980s, many end-effectors have been designed for each specific type of fruit and/or vegetable, which can be categorized as mechanical grippers with two or more fingers and vacuum suction pads. However, some fruit and/or vegetables are sensitive and bruise easily while harvesting with such grippers. One proposed solution for instance is the spoon-shaped end-effector which was designed by Zhao et al as described in Biosystems Engineering, 110 (2), 112-122 (2011) to pick and thus harvest the fruit by cutting the stalk. The end-effector contains a gripper to grasp the fruit and an electric cutting device to separate the fruit from the stalk. The success rate of apple harvesting by this robot known in the art was 77%, but partial removal of the stalk resulted in a decreased market value of the harvested fruit, as it reduces the storage potential. Therefore,
there is a need for a novel approach and device for harvesting or picking fruit and/or vegetables.
Summary of the invention
A need still exists for an improved system and method for harvesting fruit and/or vegetables. More specifically, an improved system and method for picking fruits and/or vegetables hanging on a tree or on a plant and detaching these from the tree/plant while keeping both the fruit/vegetable as well as the tree/plant intact.
These objectives are met by the device according to the independent claims of the present invention. The dependent claims relate to preferred embodiments.
The present invention relates to a device for harvesting a fruit and/or vegetable from an environment, wherein the device comprises
- a manipulator allowing the controlling movement of a harvesting end and a means for providing an initial shear force and/or moment,
- a harvesting end, said harvesting end comprising receiving means for receiving said fruit and/or vegetable, whereby said receiving means comprises an opening for receiving said object, and
- a means for providing an initial shear force and/or moment, when said fruit or vegetable is received by said receiving means, to an abscission zone of the fruit or vegetable, for instance by pressure at the stalk or stem, such that said fruit or vegetable is detached from said environment at the abscission zone without detaching the fruit or vegetable from its stalk or stem.
It is an advantage of embodiments of the present invention that the system may comprise a vision system comprising at least a camera for identifying the presence of the fruit or vegetable and for determining the position of the harvesting end. The vision system may allow to accurately position the means for providing an initial shear force and/or moment with respect to the abscission zone of the fruit or vegetable.
The means for providing an initial shear force and/or moment may be such that an initial shear force and/or moment is applied to an abscission zone of the fruit or vegetable by applying a pressure at the end of the stalk or stem close to the abscission zone.
In preferred embodiments the means for providing an initial shear force and/or moment to an abscission zone of an object preferably provides a perpendicular force to the stalk or stem, which results in a shear force on the abscission zone to preferably detaches the stalk from the
branch (and not the fruit from its stalk or stem). Accordingly, embodiments of the present invention do not provide the same effect as cutting the stalk or stem, even in the vicinity of the abscission zone.
The device may be adapted for applying an initial shear force and/or moment such that upon detachment of the fruit or vegetable, the stalk or stem remains intact and attached to the fruit or vegetable.
In some embodiments the means for providing an initial shear force and/or moment to an abscission zone of an object may be a surface which is movable with respect to the fruit or vegetable, whereby the fruit or vegetable does not move substantially and is preferably held motionless by the receiving means, therefore enabling provision of an initial shear force and/or moment on the abscission zone of the object in a most optimal way. In preferred embodiments said device may be positioned on an end portion of a movable arm. It is an advantage of embodiments of the present invention that the fruit or vegetables are not detached from their stem when harvesting, only that an initial shear force and/or moment is provided to the abscission zone, for instance stem, therefore detaching the fruit or vegetable together with the stem still intact on the fruit or vegetable. One of the major advantages of such an approach, i.e. harvesting a fruit or vegetable with its stem intact on the fruit or vegetable, is that the fruit or vegetable can be preserved for a longer period of time, as the water loss and bacterial intrusion through the intact stem is minimal. It is an advantage of embodiments of the present invention that the movement applied is not a twisting movement, as the latter also typically results in detachment of the fruit or vegetable from the stem.
In some embodiments said receiving means comprising an opening may be provided by at least two gripping fingers, more preferably at least two linear fingers, providing a U-shape such to receive said object in the open portion. In other embodiments said receiving means comprising an opening may be provided by a gripping container like for instance a fruit intake case or a sucking case, to receive and substantially hold the fruit or vegetable still with respect to the movable surface such that a shear force can be provided to the abscission zone.
In further embodiments said receiving means comprising an opening may be provided by a suction grabber for picking and hooding a fruit, comprising for instance a funnel-like suction cup allowing partial enclosure of a fruit and means for generating an internal vacuum upon enclosure of a fruit in said cup.
In a second aspect, the present invention provides devices for harvesting an object comprising a stalk or stem, wherein the device comprises:
- receiving means, whereby said receiving means comprises at least two gripping fingers; - at least one movable surface;
- a first drive mechanism that can be actuated so as to cause the at least two gripping fingers to take up the object and immobilize said object relative to the device
- a second drive mechanism that can be actuated so as to cause at least one movable surface to move relative to the at least two gripping fingers until said at least one surface is in contact with said stalk or stem of said object, thereby creating a shear force and/or moment in the abscission zone.
For some objects this shear force and/or moment is adequate to remove or detach the stalk or stem from its environment, i.e. branch. However, it is possible that for other varieties an additional force is need. In these embodiments the device further may comprise a manipulator, wherein said manipulator is adapted to nod or pivot the device ( ) around said point of contact, increasing said created shear force and/or moment in the abscission zone of said object. As may be noted, this movement approximates that of a human picker. In preferred embodiments said nod or pivotation does not comprise imparting a twisting motion, since the twisting motion results in separation of the fruit or vegetable from its stem.
The second drive mechanism may be such that it can be actuated so as to cause at least one movable surface to move relative to the at least two gripping fingers until said at least one surface is in contact with said stalk or stem close to the abscission of the object.
In preferred embodiments the relative movement of the at least one surface is a linear movement.
In other preferred embodiments the device further may comprise means for positioning the device. Preferably, the means for positioning is a Time of Flight (TOF) camera or an ultrasonic distance sensor. In preferred embodiments the device further may comprise at least one object detecting sensor. Preferably, the at least one object detecting sensor is a RGB camera. In preferred embodiments the at least two gripping fingers may comprise protecting means, whereby said protecting means are positioned such as to provide protection between said object and the at least two gripping fingers. In further preferred embodiments the protecting means may further comprise friction means, for instance be textured, such as to enhance the grip of the at least two gripping fingers. Embodiments of the invention comprising at least two gripping means such as linear gripping fingers, and whereby the first drive mechanism is a grasping cylinder and the second drive mechanism is a pushing cylinder. In preferred
embodiments the object is a fruit or vegetable. In preferred embodiments the device further may comprise a ripeness sensor.
In a third aspect the present invention provides methods for detaching or removing a stem or stalk of an object from its environment, the method comprising using a device according to embodiments of a device according to the second aspect, whereby said method comprises:
- taking up the object with the at least two gripping fingers and immobilizing said object relative to the device;
- moving the at least one movable surface relative to the at least two gripping fingers until the at least one surface is in contact with said stalk resulting in a point of contact, and creating a shear force and/or moment in an abscission zone of said object and detaching the stalk from its environment at the abscission zone.
In a fourth aspect, the present invention provides devices for harvesting an object comprising a stalk, wherein the device comprises:
- a tubular body having a harvesting end, the tubular body being configured so as to allow said object to pass there through, whereby said harvesting end further comprises at least two pivoting surfaces, said at least two pivoting surfaces configured to bring the harvesting end in an open or closed position;
- at least one movable surface, whereby said movable surface has an active and an inactive position;
- a manipulator that can be actuated so as to bring the device in a harvesting position;
- a first drive mechanism that can be actuated so as to bring the harvesting end in an open position such as to position the object in said tubular body and then to bring the harvesting end in a closed position such as to bring the pivoting surfaces in contact with the object, pushing the object in the tubular body and to bring the movable surface in its inactive position in contact with the stalk, thereby creating a first shear force and/or moment in an abscission zone of said object. For some objects this first shear force and/or moment is adequate to remove or detach the fruit with its stalk from its environment, i.e. branch. However, it is possible that for other varieties an additional force is needed. In these embodiments the device further may comprise a second drive mechanism that can be actuated so as to bring the movable surface to its active position, whereby the movable surface is adapted to move relative to the tubular body, creating a second shear force and/or moment in an abscission zone of said object.
In preferred embodiments the tubular body may be an elongated tubular body.
In preferred embodiments the relative movement of the at least one movable surface is a linear movement.
In alternative preferred embodiments the relative movement of the at least one movable surface is a rotational movement.
In preferred embodiments the device further may comprise means for positioning the device. Preferably, said means for positioning is a Time of Flight (TOF) camera or an ultrasonic distance sensor.
In preferred embodiments the device further may comprise at least one object detecting sensor. Preferably said at least one object detecting sensor is a RGB camera. In preferred embodiments the object is a fruit or vegetable. In preferred embodiments the device may further comprise a ripeness sensor.
In a fifth aspect the present invention provides methods for detaching or removing a stalk of an object from its environment, the method comprising using a device according to embodiments of a device according to the fourth aspect, the method comprising:
- bringing the harvesting end in an open position and the tubular body in a harvesting position;
- positioning the object in the harvesting end and bringing the harvesting end in a closed position pushing the object further in the tubular body resulting in contact of the stalk with the moveable surface in its inactive position and resulting in a first shear force and/or moment in an abscission zone of said object;
In preferred embodiments the method further may comprise activating the movable surface and moving the movable surface relative to the tubular body, creating a second shear force and/or moment in an abscission zone of said object.
In a sixth aspect the present invention provides devices for harvesting an object comprising a stalk, wherein the device comprises:
- a tubular body having a harvesting end, the tubular body being configured so as to allow said object to pass there through, whereby said harvesting end further comprises at least two pivoting surfaces, said at least two pivoting surfaces configured to bring the harvesting end in an open or closed position;
- at least one surface;
- a manipulator that can be actuated so as to bring the device in a harvesting position;
- a drive mechanism that can be actuated so as to bring the harvesting end in an open position such as to position the object in said tubular body and then to bring the harvesting end in a closed position such as to bring the pivoting surface in contact with the object, pushing the
object in the tubular body and to bring the surface of the harvesting end in contact with the stalk, thereby creating a shear force in an abscission zone of said object.
In preferred embodiments the harvesting end further comprises a recess, whereby the stalk is brought in contact with said recess when the two pivoting surfaces are driven in a closed position.
In preferred embodiments the device further may comprise means for positioning the device. Preferably, said means for positioning is a Time of Flight (TOF) camera or an ultrasonic distance sensor.
In preferred embodiments the device further may comprise at least one object detecting sensor. Preferably, said at least one object detecting sensor is an RGB camera. In preferred embodiments the object is a fruit or vegetable. In further preferred embodiments the device may further comprise a ripeness sensor.
In a seventh aspect the present invention provides methods for detaching or removing an object with its intact stalk from its environment, the method comprising using a device according to embodiments of a device according to the sixth aspect, the method comprising:
- bringing the harvesting end in an open position and the tubular body in a harvesting position;
- positioning the object in the harvesting end and bringing the harvesting end in a closed position pushing the object further in the tubular body resulting in contact of the stalk with the moveable surface in its inactive position and resulting in a first shear force and/or moment in an abscission zone of said object.
Embodiments of the present invention, advantageously guarantee detachment at the abscission zone, where the fruit or vegetable would also be naturally released at a later stage, the stalk will stay on the fruit giving maximal storage potential and commercial quality while the reproductive parts for the next year which are already present on the tree branch will also stay intact thus protecting next year's yield potential. This will be obtained by detaching the fruit through a shear force on the abscission zone which forms the connection of the fruit stalk with the tree branch rather than a tensile force.
The present invention also relates to a fruit and/or vegetable picking device which can receive an object, such as a fruit or vegetable, and which introduces a shear force and/or moment at the abscission zone between the stalk and the tree branch. This shear force and/or moment can be introduced by bringing a surface in contact with the abscission zone. The latter can be done using several embodiments of the invention. For instance by pushing at least one surface, i.e. a movable surface like a third component or finger, on the abscission zone resulting in a
shear force and/or moment. In alternative embodiments one can push the fruit or vegetable stem against a fixed surface, such that the abscission zone is pushed against a surface. The at least one surface, for instance provided by a movable finger, pushes against the flexible stem of a fruit or vegetable, resulting in bending of the flexible stem. As a result, one may provide both a shear force as well as a tension on one side and a pressure on the other side of the abscission zone.
For some fruit or vegetable varieties, this provision of a shear force and/or moment results in detachment of the fruit or vegetable. However, in embodiments when the shear force and/or moment is not strong enough to detach and harvest the fruit or vegetable a supplementary shear force and/or moment can be created. In embodiments of the invention this supplementary shear force and/or moment can be created by keeping the at least one surface in contact with the stem and providing a rotational movement of the device around the abscission zone, while the third component or finger is pushed to the stalk. This is in contrast with classical picking devices known in the art that use only grippers to grab the fruit or vegetable and pull the latter for detachment. Targeting the stalk and creating a shear force and/or moment advantageously increases the probability of detaching the fruit without damaging the stalk and the branch to which it is attached. Particular and preferred aspects of the invention are set out in the accompanying independent and dependent claims. Features from the dependent claims may be combined with features of the independent claims and with features of other dependent claims as appropriate and not merely as explicitly set out in the claims.
Brief description of the drawings
Further features of the present invention will become apparent from the examples and figures, wherein:
Fig. 1 schematically illustrates a device for harvesting an object comprising a stalk or stem, according to an embodiment of the invention whereby the device comprises at least two gripping fingers and at least one movable surface.
Fig. 2 schematically illustrates the device in Figure 1 from a different perspective.
Figs. 3a-3c schematically illustrate the harvesting device as illustrated in Figure 1 used for harvesting an object.
Fig. 4 illustrates a picture of a device according to specific embodiments of the invention. Fig. 5 schematically illustrates a device for harvesting an object comprising a stalk or stem, according to an embodiment of the invention, whereby the device comprises an elongated
tubular body and at least a movable surface according to specific embodiments of the invention.
Figs. 6a-6c schematically illustrates the harvesting device as illustrated in Fig. 5 used for harvesting an object.
Figs. 7a and 7b schematically illustrate a device for harvesting an object comprising a stalk or stem, according to an embodiment of the invention, whereby the device comprises an elongated tubular body and a recess at the harvesting end.
Figs. 8a and 8b schematically illustrate a device for harvesting an object comprising a stalk or stem, according to another embodiment of the invention, whereby the device comprises an elongated tubular body and a recess at the harvesting end.
Definitions
The term "shear force" used in this text may relate to a force acting on an object in a direction perpendicular to the extension of said object. The term "abscission zone" used in this text may relate to a zone between the stalk of a fruit or vegetable and a branch. More specifically, it may relate to a zone which is composed of a top layer that has cells with weak walls, and a bottom layer that expands, breaking the weak walls of the cells in the top layer allowing a fruit or vegetable to be shed from a branch. An abscission zone or separation zone may be different for pit containing fruit (like for instance cherries, apples, pears, etc.) than for instance for citrus fruit (like for instance oranges, lemons, etc.) as the latter may be dependent on the fruit forming processes.
Where in embodiments of the present invention reference is made to pressure on the stalk or stem close to the abscission zone, reference may be made to a position on the stalk or stem, such that the distance to the abscission zone is less than 50% advantageously less than 30% of the length of stalk or stem. Nevertheless, in other embodiments also at a further distance. References
[1] device
[20] object
[21] stalk
[22] abscission zone
[30] pushing cylinder - drive mechanism for third finger
[31] pushing element
[32] pushing surface
[40] grasp cylinder - drive mechanism for grasping fingers
[50] TOF camera
[51] RGB camera
[100] receiving means
[101, 102] grasping fingers
[201, 202] pivoting surfaces
[150] drive mechanism for pivoting surfaces
[160] four linkages
[120, 121] finger padding
[200] ripeness sensor
[300] recess at the harvesting end
[500] manipulator - guiding tube
Detailed description of preferred embodiments
The present invention will be described with respect to particular embodiments and with reference to certain drawings. However, the invention is not limited thereto, but only by the claims. The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated and not drawn on scale for illustrative purposes. Where the term "comprising" is used in the present description and claims, it does not exclude other elements or steps. Where an indefinite or definite article is used when referring to a singular noun e.g. "a" or "an", "the", this includes a plural of that noun unless something else is specifically stated. The term "comprising", used in the claims, should not be interpreted as being restricted to the means listed thereafter; it does not exclude other elements or steps. Thus, the scope of the expression "a device comprising means A and B" should not be limited to devices consisting only of components A and B. It means that with respect to the present invention, the only relevant components of the device are A and B. Furthermore, the terms first, second, third and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein. Moreover, the terms top, bottom, over, under and the like in the description and the claims are used for descriptive purposes and not necessarily for describing relative positions.
It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other orientations than described or illustrated herein. In the drawings, like reference numerals indicate like features; and, a reference numeral appearing in more than one figure refers to the same element. The drawings and the following detailed descriptions show specific embodiments of devices for gripping, clamping, taking up and/or delivering objects.
According to embodiments of the present invention, a device for harvesting a fruit or vegetable from an environment is described. The device comprises a manipulator allowing the controlling movement of a harvesting end and a means for providing an initial shear force and/or moment. The device also comprises a harvesting end, said harvesting end comprising receiving means for receiving said fruit and/or vegetable, whereby said receiving means comprises an opening for receiving said object. The device further comprises a means for providing an initial shear force and/or moment, when said fruit or vegetable is received by said receiving means, to an abscission zone of the fruit or vegetable, for instance the stalk or stem, such that said fruit or vegetable is detached from said environment without detaching the fruit or vegetable from its stalk or stem.
By way of illustration, the invention not being limited thereto, a number of standard and optional features will be described with reference to exemplary embodiments and drawings. The main components of a device for harvesting objects according to some embodiments of the invention are depicted in Figure 1. The latter provides one specific embodiment of the device 1 of the invention, and further in the text several alternative embodiments of a device for harvesting objects according to the invention (e.g. using a tube design or elongated tubular body) are provided. In this specific embodiment the device 1 comprises a receiving means 100 in the form of at least two fingers 101, 102 for instance grasping fingers, whereby the at least two fingers 101, 102 are configured such to grip an object 20, for instance a piece of fruit or vegetable which is attached to the environment, i.e. a branch, at its abscission zone 22 through a stalk or stem 21. By the gripping movement, the object is fixed using at least two fingers 101, 102 so to immobilize the object, e.g. fruit, with respect to the device. Embodiments of devices according to the present invention may be used as an end-effector for an automatic robot arm. A device according to further embodiments of the invention further may comprise at least one movable surface 32, i.e. provided by an expandable finger providing said movable surface i.e. a pushing element or surface, whereby the movable surface 32 is adapted to target and contact
the stalk 21 of the fruit, resulting in a shear force and/or moment in the abscission zone 22. A movable surface 32, for instance may be an additional finger which is pushed out and which by creating a shear force and/or moment in the stalk causes detachment of said object from the branch. This detachment ideally takes place in the natural abscission zone of the stalk. In order to accurately position the device and to optimize abscission of fruit with stalk from the branch, in embodiments of the device, imaging equipment may be preferably added to the gripper design. The combination for instance of a Time of Flight (TOF) camera 50 to gain depth information and a RGB camera 51 to discriminate colors and so to better recognize the fruit components is preferred in this context. Figure 2 schematically illustrates a specific embodiment where the TOF 50 and RGB camera 51 are positioned under an angle (a) with respect to a drive mechanism of the device according to the invention. By positioning the imaging means in such a way, an improved detection and localization of the object advantageously can be made. Having an optimal position of the movable surface 32, i.e. pushing element, is an important part of the design. Hereto, the position of the movable surface 32, in the form of for instance a pushing or third finger is preferably adjustable with respect to the distance and height to the object i.e. fruit (e.g. apple) target. This was tested by for instance harvesting two types of apple varieties in Chile, for instance the Gala variety having a long stalk and the Fuji variety having a short stalk, the "optimal" position of the third finger has been proposed based on those measurements. A person skilled in the art may provide other optimal positions of the third finger for other specific fruit or vegetable varieties. The distance of the third or pushing finger to the target apple is not so important since the contact pads for holding the apple are fairly long and the third finger is active. The height of the third finger can be important as the third finger should be able to push the stalk at or near the abscission zone to have an effective detachment of the fruit with stalk from the branch. Figures 3a-3b illustrate a picking strategy using a device comprising two grasping fingers according to some embodiments of the invention. In this strategy of picking an object, in this example an apple, with a device according to some embodiments of the invention, comprises preferably these three steps, illustrated respectively in Figures 3 (a) to (c): (1) Grasping the object 20, whereby said grasping comprises holding the object i.e. fruit firmly by for instance a parallel gripper 100, which provides two engaging surfaces. (2) Pushing of the stalk with at least one movable surface 21 and thus providing a shear force and/or moment on said stalk, whereby said stalk is attached to for instance a branch. In order to enable said pushing of the stalk, a third movable finger moves out and creates contact with the stalk with the pushing surface 32
and thus creates an initial shear force and/or moment to the abscission zone 22. (3) In some cases where the initially provided shear force and/or moment by the movable surface 32 is not large enough to provide detachment of the object, an optionally additional shear force and/or moment can be applied by an additional movement of the device. The device is then rotated while grasping the object, around the contact point of the third finger with the stalk over an angle . This rotation increases the provided shear force and/or moment resulting in detachment of the apple stalk at the abscission zone. This rotational movement can be provided with an angle R between 30 to 90°. The objectives and advantages of the above mentioned strategy may be the following:
- Embodiments of the present invention provide harvested objects, such as fruit or vegetables like i.e. pears, cherries, apples, etc. whereby the stalk is kept intact on the fruit. As a result, the storage potential of the harvested fruit will be maximally retained.
- Embodiments of the present invention imitate how a person would harvest a single piece of fruit. A person would normally push a finger on the stalk close to the abscission zone, which is the zone where the stalk is connected to the mother branch. Therefore, the right strategy to harvest an apple or pear is creating a shear force and/or moment in that zone to detach the stalk from the branch. The general idea explained above was translated into different realizations of the present invention in order to give an idea of how such a device could look like. The first embodiment provides a linear gripper, which is described in detail below. The other embodiments of a device, i.e. gripper, according to the invention are schematically described.
Example 1: Linear gripper, for instance as illustrated in Figure 1, with active third finger
In this example, the hardware components of a linear gripper according to embodiments of the invention may be the following: (1) a grasping cylinder 40 which is able to move at least two grasping fingers 101, 102, whereby both fingers may be fixed or whereby one finger is fixed and the other movable; (2) a pushing cylinder 30 which enables the device to move the third (detaching) finger providing a movable surface 32; (3) at least two grasping fingers 101, 102; (4) a pushing element 31 providing a movable surface 32; and (5) finger padding 120, 121 to grab and/or protect the object i.e. fruit while grasping; Besides the above mentioned elements, following optional components may make the picking operation more efficient, such for instance a color camera 51, at least one light source (e.g. LEDs), and a Time of Flight (TOF) camera 50. The grasping cylinder 40 in this example may preferably comprise the following components: a Festo parallel gripper HGPL-14-60-A (double acting), two proximity sensors
SMT-10M-24VE (included with the parallel gripper), one position sensor SMAT-8M-U-E-0,3- M8D (mounted in one finger for instance). This specific example of a grasping cylinder may have following functions: moving of two grasping fingers in an open or closed position, by use of for instance a pneumatic cylinder. Preferably, no adaptive grasping is used and one always grasps in the center. The two proximity sensors preferably detect the status of the gripper resulting in an action, whereby the status can be for instance the following: status open, resulting in that the piston is put in the open position, status grasp something, whereby the piston not in the open nor close position, and status grasp nothing, whereby the piston is put in the close position. The device in this example also may have a power supply, which may be pneumatic powered, whereby its input pressure is adjusted to not damage the grasped object while grasping. The pushing cylinder in this example may comprise the following components: a Festo compact cylinder AEN-16-P-A and one proximity sensor SMT-10M-24V-E (included with the cylinder). The functions of these components are to move the surface of the pushing element to push the stalk and to provide a shear force. The proximity sensor may also detect the status of the pushing cylinder. The pushing cylinder also may have a power supply which is pneumatically powered. The grasping fingers in this example preferably comprise the following components: two aluminum bars, which are configured such that they are mounted on the mounting point of the parallel gripper and whereby the aluminum bars comprise protecting means in the form of soft padding. The gripping fingers 101, 102 and/or protecting means 120, 121, like for instance finger padding, may have a structured surface to enhance the grip of the gripper or to diminish the damage of the fruit from the gripping device. In further embodiments, as the contact between tip point of the finger to the object potentially may damage or bruise the object, one may provide protecting means on the movable surface as well, for instance by providing a padding to cover the tip point of the finger i.e. the movable surface. The protecting means are preferably many small plastic balls with a diameter of to 2mm, covered by a flexible membrane tissue covering. Such protecting means advantageously results in a soft, but firm grasping. In this example, the movable surface is provided by a pushing finger. Preferably, said pushing finger may comprise a custom made pushing element mounted on the piston of the pushing cylinder and can be made of plastic. Such a pushing finger is configured to push the stalk while the object (e.g. an apple) is grasped, creating a shear force in the stalk and the abscission zone, in order to detach the stalk from the branch. The pushing surface can be provided by a wide, flat plastic with the guiding head increasing the contact area with the stalk and thus the provided shear force. In further embodiments the gripper may
comprise imaging systems, for instance comprising a TOF camera. An example of such an imaging system is provided in Figure 4. Such an imaging system preferably comprises the following: (1) a PMD camboard nano, which is a mini TOF camera and which provides a depth, distance, and/or amplitude image from a close distance (from 5cm - 50 cm), (2) an RGB color VRMagic camera, which provides RGB color images, and (3) an LED ring, which provides artificial light so that the device can operate in very different conditions. The imaging systems have several functions such as: detecting the orientation of the object or stalk from close distance, detecting any other obstacles, such as leaves, from close distance and detecting and tracking a target. The gripper according to embodiments further comprises an area or surface to mount different elements of the above mentioned imaging system, such as the mini TOF PMD nano camboard and 2 VRmagic cameras. Images provided by such a camera are provided in Figure 4. These images are acquired from the mini ToF which shows that this camera is suitable to work outdoor and gave promising results for detecting or tracking apple fruit. As mentioned above, for some applications the linear gripper according to specific embodiments of the invention has its limitations, and several alternative harvesting devices according to the invention were designed that answer those limitations. The main difference is that these embodiments use an elongated tubular body as a receiving means 100, i.e. a thin-walled cylinder ("tube") on which pivotable surfaces 201, 202, i.e. fingers, are mounted. However, in this case the fingers are not adapted to grasp the object, but to push the object.
Example 2: A harvesting device according to embodiments of the invention comprising an elonggted tubulgr body, comprising two pivotgble surf gees gnd o surf gee, sgid surf gee provided by gn gctive finger, whereby sgid finger provides o shegr force on the stglk.
Figure 5 provides a schematic drawing of a harvesting device according to alternative embodiments of the invention. The harvesting device in this embodiment comprises an elongated tubular body, whereby the diameters of such tubular body preferably has a diameter suitable such that the targeted fruit or vegetable can pass through said tubular body . The tubular body in addition can guide the harvested fruit or vegetable to a harvesting bin. The tubular body is configured such that components, such as for instance driving mechanisms or imaging means, can be attached thereon. At the harvesting end or receiving end 100, the tubular body further comprises pivotable surfaces or fingers 201, 202, which open and close like a window, resulting in an open or closed positing of the harvesting or receiving end 100. The pivotable surfaces 201, 202 are adapted to separate the target fruit or vegetable and when put in a closed position push the fruit or vegetable inside the tubular body. The pivotable
surfaces or fingers are preferably driven or activated by a four linkage mechanism using a pneumatic or a motor as a power source. However, a power source is preferred as this advantageously enables position control. The elongated tubular body in this embodiment further may comprise an additional active finger and thus surface 32 which may provide a shear force and/or moment on the abscission zone at the end of the stalk 21 of the fruit or vegetable 20 positioned in the harvesting end 100. In some further embodiments, the harvesting devices of the present invention may further comprise a ripeness sensor 200 providing a smart harvesting device: a non-contact ripeness sensor, for instance an NIR spectrophotometer is preferred, with an optical fiber attached to the tube element at the harvesting end which guides the signal from the fruit to the sensor. Such a ripeness sensor advantageously may determine if the fruit or vegetable is ripe enough to be harvested. Moreover, it enables the selective harvesting and automatic sorting of fruit while harvesting. In the embodiment comprising a harvesting device having a tubular body, an imaging or camera system can also be provided. Such a camera is preferably mounted inside the tube element (not shown), for instance in the center of the back (could for instance be a mini-ToF 50 and RGB camera 51 like the example above with the gripping fingers). Such an imaging system can advantageously detect the target fruit (stalk or orientation) and possible obstacles. In further embodiments a harvesting device according to embodiments of the invention may further comprise means for blowing leaves or other obstacles away. The latter is preferably mounted near the camera system in the back of the tube. Such means for blowing obstacles can be a pressured air blowing device which enables one to blow the leaves or other similar object, to make the stalk (more) visible and remove the obstacle while approaching. The picking strategy using the harvesting device having a tubular body comprising a movable pushing surface is similar to the linear gripper described above. The improvement and advantages of the tubular body design lies on the pivoting fingers and the tubular shape, for example: (1) The pivotable fingers providing pivotable surfaces 201, 202 are thin as compared to the grasping fingers. Therefore, it has less collision to other branch and fruit/vegetables while approaching the target object. (2) The pivoting fingers, in this embodiment, do not grasp the fruit or vegetable, but are adapted to push the later inside the tube which has some advantages, like for instance the fruit is rolled inside the tube which makes the detaching movement itself and thus does not have to be provided by the manipulator, and no grasping is provided, such that there is less risk for the fruit or vegetable. (3) The pivoting fingers or surfaces 201, 202 optionally may have position control in embodiments using the motor actuator that can enable the adaptive opening of the
finger which only opens enough for the target fruit or vegetable to get in. (4) The harvesting device may further comprise a ripeness sensing means 200 in front and the camera inside (the blower is optional). All components can be packed inside the tube, which results in a compact design. (5) The tubular body itself in addition provides a guiding tube, which can guide the harvested fruit or vegetable to roll towards the harvesting bin. This provides a solution for collecting the harvested fruit. Moreover, it could enable autonomous sorting by guiding different harvested fruits to different bins based on the ripeness sensor decision. In this example, use of a pushing surface 32 provided by a pushing finger 31, the use of a tubular harvesting device according to the present invention comprises three main steps, respectively illustrated in Figures 6 (a)-(c), namely: - Step 1: Approach and ripeness sensing as illustrated in Fig. 6 (a) - Step 2: Moving the harvesting device to the harvesting position as illustrated in Fig. 6 (b), whereby the pivoting surfaces 201, 202 are in an open position. - Step 3: Provide the pivoting surfaces 201, 202 in a closed position resulting in pushing of the object, i. e. apple, inside the tube and providing a pushing surface 32 to the stalk as illustrated in Fig. 6 (c). In Figure 6 (a) the manipulator 500, using obtained positioning data from the imaging system, is driven such that the harvesting device is approaching a target object at its harvesting end. When approaching the object, the pivotable surfaces 201, 202 are driven in the open position and the pushing element or third movable surface 32 is in the idle position, i.e. non-moving or retracted. In an optional step, the object is then preferably sensed by a ripeness sensor 200 in order to evaluate if the object 20 is ready for harvesting. When the object is ripe enough, the harvesting device is put into the picking or harvesting position. The picking or harvesting position of a device according to the present invention is the position where the stalk of the object is in the pushing area of the pushing element or third movable surface and where the object is almost inside the tubular body, as illustrated in Figure 6 (b). After the object has been brought in this picking or harvesting position, the pivotable surfaces 201, 202 are driven in the closed position, pushing the object further inside the tubular body, which is illustrated in Figure 6 (c). This pushing of the object by the pivotable surfaces in addition pushes the stalk to the movable surface 32 of the harvesting end, creating a shear force in the stalk. As the object is now positioned inside the tubular body, the stalk is in the pushing area of the third finger. If the object has not been detached yet by the applied shear force and/or moment, the third finger is activated from its idle or non-moving position and a moving pushing surface is provided on the stalk area creating an additional shear force and/or moment in the abscission zone, breaking or detaching the stalk in its abscission zone and rendering a detached object
from its environment. The object is then guided through the tubular body to a harvesting bin or storage box. In alternative embodiments, the pushing surface can be provided by a rotating third finger as illustrated in Figs. 8 (a) and (b).The rotating circular finger now is configured to provide a rotational movement as compared to the linear movement of the embodiment above, which advantageously covers a greater contact area with the stalk and thus may provide a higher shear force. With the rotating third finger, the working protocol is similar as the linearly moving third finger. The advantage of this design is that the cover area of the third finger is larger when it is active. So, the stalk of the object only has to be positioned inside this larger area, making it more robust, thus requiring less accuracy for the stalk detection and/or positioning of the device.
Example 3: A harvesting device according to embodiments of the invention comprising an elongated tubular body, comprising two pivotable surfaces and a surface provided by the hgrvesting end of the tubulgr body, whereby sgid surf gee provides o shegr force on the stglk. The major difference between this example and the previous, is that there is no movable surface or third finger which is actively driven towards the stalk. In this example the edge of the harvesting end which comprises a recess 300 of the tubular element is used as a pushing surface 32. In this embodiment, when the pivotable surfaces 201, 202 are driven to the closed position, these pivotable surfaces push the object deeper inside the tubular element as a result of a recess in the harvesting end, such as to provide an adequate shear force and/or moment on the stalk in order to enable detachment of the object from its environment. In this example, the approaching of the object and moving to the picking position is similar to the example above. However, when driving the pivotable surfaces 201, 202 in the closed position, the object is pushed deeper inside the tubular body because of the recess. The latter results in that the stalk, as it is already brought in contact with the harvesting end, is pushed to a recess of said harvesting end, creating a perpendicular force on the stalk, which results in a shear force on the abscission zone between the stalk and the branch, thus detaching the stalk from the branch. It is to be understood that this invention is not limited to the particular features of the means and/or the process steps of the methods described, as such means and methods may vary. It is also to be understood that the terminology used herein is for purposes of describing particular embodiments only, and is not intended to be limiting. It must be noted that, as used in the specification and the appended claims, the singular forms "a" "an" and "the" include singular and/or plural referents unless the context clearly dictates otherwise. It is also to be understood that plural forms include singular and/or plural referents unless the context clearly
dictates otherwise. It is moreover to be understood that, in case parameter ranges are given which are delimited by numeric values, the ranges are deemed to include these limitation values.