WO2010063075A1

WO2010063075A1 - Crop picking device and method

Info

Publication number: WO2010063075A1
Application number: PCT/AU2009/001578
Authority: WO
Inventors: Rudi Bartels; Ray Daniels; Phillip Palk
Original assignee: Magnificent Pty Ltd
Priority date: 2008-12-03
Filing date: 2009-12-03
Publication date: 2010-06-10

Abstract

A device for selectively picking a crop from a plant which includes at least one sensor that receives information from a region of interest, including multiple aspects of the crop, and converts the information to an electronic output. At least one diagnostic unit receives the electronic output and processes the electronic output to determine location of the crop and suitability of the crop for picking. The device also includes a gripper activated by the at least one diagnostic unit to hold the crop or a stem of the crop and a picker activated by the at least one diagnostic unit to sever the crop from the plant.

Description

TITLE

Crop Picking Device and Method FIELD OF THE INVENTION

The invention relates to a device and method for picking crops. In particular, although not exclusively, the invention relates to a robotic device and a method of using said robotic device for picking crop from a plant. BACKGROUND TO THE INVENTION

Picking crops can be a very labour intensive operation. For specific crops, each plant must be visually inspected to locate the crop, which must then be assessed to determine if it is ready to be harvested. Due to the seasonal availability of the work, as well as the physical demands and relatively uninteresting work, a reliable human picking workforce can be difficult to find and retain.

A wide range of automatic harvesters are in commercial use but they tend to be suited to more robust crops that maintain their value despite rough handling. These crops can be strip harvested (e.g. grain header).

Where selective harvesting of a crop is required, crop harvest by automatic harvesters generally relies on the fact that ripe crop tends to separate more easily from the plant than unripe crop when the plant is agitated. Discrimination of the crop on an individual basis does not occur, and thus the harvest tends to include over-ripe and immature crop. The falling crop is often bruised and therefore has a shorter shelf life or is only suitable for the processed market. Additionally, the falling crop may be lost onto the ground. Automatic harvesting by agitation is not appropriate for all crops, as not all crops (for example, strawberries or broccoli) will easily separate from the plant when it is shaken, even when ripe. This is especially important for crops that need to be picked fresh and that do not ripen uniformly.

Robotic harvesters can reduce the problems of debris collection and damage to the plant and crop, and can provide individual assessment of each piece of crop. However, the picking rates can be slow and difficulties arise in locating and assessing the crop, giving a low recovery of crops. Assessment of the readiness of the crop for harvest must be undertaken without damage to the crop or the stem of the crop, as if the crop is not ready and must remain on the plant, any stem or crop damage may prevent the crop from reaching a harvest-ready state.

JP 2000-092952A (Iseki & Co, Ltd) describes a harvesting robot, particularly for selective harvesting of strawberries. A camera detects a strawberry and from the location and position of the strawberry, estimates the location of the strawberry's stem. A hook mechanism is extended to hook the stem and pull the strawberry upwards. As the strawberry stem is pulled to its limit, the action of the hook causes the stem to break at the boundary of the stem and the strawberry. The strawberry then falls into a receiving box.

JP 2001-095348 also describes a strawberry harvesting robot. A camera detects a strawberry and estimates the position of the stem. Fingers then grip the stem and sever the strawberry from the plant. Both of these prior art devices only observe the strawberry from one side, and thus crops that are under ripe on a bottom surface will be picked prematurely. Additionally, location of the stem is not direct, but is estimated from the location of the strawberry. Thus "false" grabs of the stem may occur, which lowers the picking rate. Research findings regarding the above prior art devices also report the loss of a high percentage of crop after being gripped by the stem gripping mechanism.

The use of robotic equipment in crop farming is significantly more efficient if the equipment can operate autonomously without an operator to guide its movements relative to a raised crop bed or crop row.

Previously used guidance systems are directed towards steering an implement which is being towed or is mounted behind a tractor. Other previous systems are designed for use with heavy vehicles and therefore require complex hydraulic systems or a steering system that interfaces with the existing vehicle steering mechanisms.

Commercial crop production requires careful management of the crop from the initial planting through to the final harvest. Indirect factors such as the markets, supply chain, food safety, machinery, accessibility, and available pest/disease treatments also impact on crop production.

Previous methods of collecting and interpreting crop production information include static sensing or infrequent scanning at low resolution. Otherwise, information is gathered by the crop manager simply walking in the fields and gathering personal information. This may result in a small sample of information captured in the field, is time consuming and may be inaccurate as they are reliant on human observation skills.

OBJECT OF THE INVENTION

It is an object of the invention to overcome or at least alleviate one or more of the above problems and/or provide a useful or commercial choice.

DISCLOSURE OF THE INVENTION

In one form, although it need not be the only or indeed the broadest form, the invention resides in a device for selectively picking a crop from a plant comprising: at least one sensor that receives information in a region of interest, including multiple aspects of the crop, and converts the information to an electronic output; at least one diagnostic unit that receives the electronic output and processes the electronic output to determine location of the crop and suitability of the crop for picking; a gripper activated to hold the crop or a stem of the crop; and a picker activated to sever the crop from the plant.

In one embodiment at least one of the diagnostic units may receive the electronic output from the sensors and process the electronic output to determine location of obstacles. Preferably, the electronic output from two or more sensors is combined by sensor fusion.

In another embodiment the picker is incorporated into the gripper. The gripper preferably includes a gripper sensor, to confirm an object has been gripped, and/or a gathering funnel. Surfaces of the gripper suitably include gripping material.

The device for picking a crop from a plant may also comprise a mechanism for moving obstructions from the path of a sensor or a gripper or a cutter.

Additionally, illuminating means to provide light in the region of interest may be included. Preferably the light is polarized. In a particularly preferred embodiment the illuminating means provides polarized light and at least one of the sensors includes a cross polarization filter.

The device may also include a trimmer to sever excess stem from a crop once the crop is severed from the plant.

Suitably, one or more sensors for grading picked crop are included in the device, and a crop storage device is also preferable.

In one embodiment, the mechanism for moving obstructions is the gripper.

In a further embodiment, at least one sensor detects infra-red light.

Suitably, the picker is located on a picking head, which is preferably moved by a manipulator. The picking head may also include a gripper, and/or a sensor and/or illuminating means, and/or a mechanism for moving obstructions.

In a preferred embodiment, the gripper further comprises a rotation actuator to rotate the crop or the stem of the crop.

The device preferably includes guidance means to guide movement of the device along a pre-determined pathway. Preferably the guidance means includes one or more guidance sensors. In a particularly preferred embodiment the guidance sensor comprises a pair of horizontal wheels, each horizontal wheel connected to a tensile member and a steering wheel connected to each tensile member.

In another form, the invention resides in a method for picking a crop from a plant including the steps of: observing a region of interest with at least one sensor to provide information in electronic format to a diagnostic unit; processing the information in the diagnostic unit to determine location of crop and provide instructions to a gripper; activating the gripper to move to and grip the crop; observing aspects of the crop to provide information in electronic format to the diagnostic unit by changing the orientation of the crop relative to at least one sensor; processing the information in the diagnostic unit to determine suitability of crop for picking and provide instructions to a picker; and activating the picker to sever the crop from the plant.

The method for picking a crop from a plant may also include one or more of the steps of: activating an obstacle mover to remove obstacles from; a region of interest; a path of the gripper; or a path of the picker; or a path of a crop picking device. illuminating a region of interest with illuminating means; utilizing a rotation actuator to rotate crop of a stem of a crop; utilizing a trimmer to trim excess stem from a crop severed from the plant by the picker; utilizing the gripper to place crop severed from the plant into a crop processing mechanism; activating at least one sensor to grade the crop severed from the plant; placing the crop severed from the plant into a crop storage device; or guiding a crop picking device to a crop on a plant.

Preferably, the method also includes guiding a crop picking device along a raised bed containing crop by mechanical guiding means.

Further features of the present invention will become apparent from the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

To assist in understanding the invention and to enable a person skilled in the art to put the invention into practical effect preferred embodiments of the invention will be described by way of example only with reference to the accompanying drawings, wherein:

FIG 1 shows a general schematic of an embodiment of the crop picking device;

FIG 2 shows an embodiment of a frame of a crop picking device including sensors and illumination means; FIGS 3A-D shows cross section and side views of an embodiment of a gripper of the crop picking device, both in open and closed states;

FIG 4 shows an embodiment of a gripper with a scissor design;

FIG 5 shows an embodiment of a gripper with a parallel finger design;

FIG 6 shows an embodiment of a gripper with a guillotine design;

FIG 7 shows an embodiment of a gripper with a snagging design;

FIG 8 shows an embodiment of a gripper including a gathering funnel;

FIG 9 shows an embodiment of a gripper with gripper surface modifications;

FIG 10 shows an embodiment of a gripper including a rotation actuator;

FIG 11 shows an embodiment of a gripper including a solenoid as rotation actuator;

FIG 12 shows an embodiment of a gripper including a convey and pulley mechanism as rotation actuator;

FIG 13 shows an embodiment of a gripper including a motor and non-captive threaded shaft mechanism as rotation actuator;

FIG 14 shows a variety of embodiments of gripper and picker designs in cross section view;

FIGS 15A-C shows an embodiment of a picker with reciprocating saw blades to sever a stem; FIG 16A-16D show an embodiment of a gripper and picker in operation;

FIG 17 shows an embodiment of an obstruction mover using a blower;

FIG 18 shows an embodiment of an obstruction mover using a rotary brush;

FIGS 19A-E shows a variety of embodiments of dragline-type obstruction movers;

FIG 20 shows an embodiment of a multiple dragline-type obstruction mover;

FIG 21 shows an embodiment of a multi-directional dragline-type obstruction mover;

FIG 22 shows a side view of an embodiment of a mechanical guidance means utilizing horizontal wheels;

FIG 23 shows a plan view of the mechanical guidance means of FIG 22;

FIG 24 shows a plan view of a second embodiment of a mechanical guidance means using horizontal wheels;

FIG 25 shows a plan view of a third embodiment of a mechanical guidance means using guide rails;

FIGS 26A-33B show various embodiments of mechanical guidance means, each shown in side view and plan view;

FIG 34A, 34B shows a schematic of an embodiment of a gripper and sensor in operation; and FIG 35A, 35B shows a schematic of a second embodiment of a gripper and sensor in operation.

DETAILED DESCRIPTION OF THE INVENTION

A device for picking crop from a plant has been developed which locates each piece of crop individually for assessment and picking if it is assessed to be suitable. The crop may be any part of a plant which requires harvesting, including a fruit (e.g. apple), a truss of fruit (e.g. grape truss), a flower (e.g. rose flower), a leaf (e.g. lettuce leaves), a stem (e.g. sugar cane), part of a plant (e.g. a strawberry runner) or a root (e.g. potato tuber). The crop may also include parts of a plant which are required to be removed such as excess flowers, damaged or diseased parts, excess leaves or stems.

To assist with understanding of the invention a number of specific embodiments will be described with particular reference to a robotic harvester for strawberries. It is anticipated that the invention will have particular application to strawberry harvesting due to the characteristics of the fruit. Nonetheless, the invention is not limited only to harvesting of strawberries.

The place where the crop is grown is herein referred to as a bed. The bed may be a patch of soil, an area of soil raised above the surrounding area which may be bounded by solid edges or a container such as those utilized in hydroponic systems.

The crop picking device 100, as generally depicted in FIG 1 , comprises a picking head 110, a manipulator 120, a diagnostic unit 130, a crop processing unit 140, a chassis 150 and guidance means 160.

The picking head 110 comprises at least one sensor 111 , illumination means 112, a gripper 113, a picker 114 and an obstruction mover 115.

The one or more sensors 111 collect information regarding the location of the crop, including determining the distance between the picking head 110 and the crop, determining the position of any obstacles including the ground, and to confirm capture of the crop or stem by the gripper 113. Sensors 111 can include cameras (including a set of multiple cameras which provide stereo vision), electromagnetic sensors, infra-red distance sensors, ultrasound distance sensors, laser scanners or radars. Proximity sensors such as limit switches, Hall Effect sensors, capacitive sensors or pressure sensors can provide supplementary information to determine if the picking head 110 is close to or touching the crop, stem or an obstacle.

One or more sensors 111 may also be used to determine readiness of the crop for picking. For example, colour or size of the crop detected by a camera can assist in the determination of ripeness or an infrared camera sensor can determine the presence of bruising.

Each of the one or more sensors 111 may operate individually or in combination. Additionally, information from multiple sensors may be combined by sensor fusion methods to increase measurement reliability.

A preferred location of the one or more sensors 111 is on the picking head 110. However, information from sensors 111 on the picking head 110 may be combined with information from sensors that are located on other parts of the crop picking device. Additionally, viewing of a region of interest by a sensor may be undertaken remotely, using mirrors, optical fibre or the like. Thus a single sensor may view an object from multiple angles. Alternatively, the sensor may be located away from a potentially damaging environment whilst still viewing the region of interest.

In another preferred embodiment, as shown in FIG 2, a multiplicity of sensors 111 are located on a frame 170 which is separate from the picking head 110. The frame 170 is kept at the fixed height relative to a bed 180 containing the crop 190. Suitably, a multiplicity of illumination means 112 are also located on the frame 170. As the frame 170 is at a fixed height relative to the bed 180, the level of illumination provided to the sensors 111 is more constant compared to where a sensor 111 and illumination means 112 are located on the picking head 110.

Additionally, when a multiplicity of sensors 111 is located on a frame 170, the potential to detect crop underneath foliage is increased, due to a low viewing angle of the sensors. Multiple views of the crop, together with crop detection without the need to move obstructing foliage, can increase the speed of detection and location of the crop. When crops are grown by hydroponic methods, the bed containing crop is commonly located above the ground. In this situation one or more sensors 111 may be located underneath the crop.

Illumination means 112 are provided to give a consistent lighting environment for the sensors 111 with minimal shadows or reflections. Preferably a ring of lights surround a sensor 111 , directed along the viewing direction of the sensor 111. Light emitting diodes are particularly preferable as illumination means 112. The illumination means may provide visible light or other forms of electromagnetic illumination, for example infra-red light.

In a preferred embodiment, polarization of light from the illumination means 112 with cross polarization at the sensor 111 serves to reduce glare entering the sensor 111. Polarization and cross polarization may be carried out by the use of polarizing filters.

The gripper 113 holds the crop either directly or indirectly by holding the stem that is proximal to -the crop. This allows manipulation of the crop such that the sensors 111 may collect information from multiple aspects of the crop. The gripper 113 may also be utilized to move obstacles from a region of interest or from any region that might obstruct normal operation of the crop picking device 100. The gripper 113 may also be used to pick up crop which has fallen from the plant.

The gripper 113 may include sensors 111 to confirm that an object has been grasped. In a preferred embodiment, as shown in FIGS 3A-D, the gripper sensor consists of an infra-red emitter 210 which provides an infrared light beam from one side of the grasping area of the gripper 113. A receiver 220 detects the infra-red light beam at an opposite side of the grasping area of the gripper 113. When the infra-red light beam is interrupted by the presence of a stem/crop 230, the receiver 220 does not detect an infra-red light beam, which informs the diagnostic unit 130 that a stem/crop 230 is within the gripper 113 and the gripper 113 is instructed to close on the stem/crop 230.

The preferred design of the gripper 113 includes multiple surfaces which move towards each other to capture the crop/stem. One embodiment is shown in FIG 4, where the gripper 113 is a scissor design consisting of two fingers 310 rotating about a pivot point 320. Movement of one or more of the fingers may be controlled by a gripper actuator 340. When the fingers 310 close a stem 330 between them is gripped. An alternative design is shown in FIG 5, and includes parallel gripping fingers 410. Parallel gripping fingers 410 can more effectively adapt to variations in the diameter of the object to be gripped than a scissor design of two fingers 310.

As shown in FIG 5, the gripper 113 may also include a pressure sensor 420 to detect the presence of a stem/crop within the gripper 113 and to control the force placed on the stem/crop to prevent stem/crop damage. The pressure sensors may include load sensors in the gripper actuator 340 to sense the load applied by the gripper 113. In a preferred embodiment the pressure sensor 420 is located on the gripping surface. Alternatively, the gripper 113 may include a physical stop to prevent closure of the gripper 113 further than a pre-determined distance.

A guillotine gripping mechanism 500, as shown in FIG 6 includes a guillotine action which is achieved by moving a gripper hook 510 and a gripper guillotine 520 co-linearly in the axis of the gripping mechanism 500. Either the gripper hook 510 or the gripper guillotine 520 or both may move relative to the picking head 110 to grasp the crop or stem 530 within the guillotine gripping mechanism 500. A guillotine gripping mechanism 500 operates with movement only on one axis and thus requires less space to operate than grippers 113 such as a scissor design.

Other gripping mechanisms such as a snag mechanism 600 may be utilized, as shown in FIG 7, where a snagging component 610 moves relative to one or more gripping fingers 620 to capture the crop/stem 630.

A suction gripper may be used wherein the crop is sucked by a moving air stream against a porous material. This is advantageous for delicate crops or crops that are oddly shaped such that a gripper comprising fingers would have difficulty holding the crop.

The gripping elements of the gripper 113 may be tapered to allow the gripping elements to easily fit between other crop and/or obstacles without damaging them. Preferably the "fingers" or gripping elements are tapered in both cross section and longitudinal section. Alternatively, the gripping elements may be designed to include step tapered gripping elements, a bend in the tapered gripping elements, very thin fingers or a combination of these.

Any of the grippers 113 described above may be modified to include a gathering funnel 700, as shown in FIG 8. The gathering funnel 700 is formed by extensions 710 to portions of the gripper 113 to assist in gathering the stem or crop into the gripper 113. A gathering funnel 700 allows for lower accuracies in the locating sensors, particularly on the axis that is perpendicular to the gripper approach and/or perpendicular to the axis of a stem. The gathering funnel 700 may be manufactured from a pliable material to reduce the risk of damage to the crop or stem. In an alternative embodiment (not shown), the gathering funnel 700 may consist of a multiple of extensions to the gripping mechanisms or a moving surface, such as a rotating wheel or conveyor belt, to feed the crop or stem into the gripper 113.

The gripper 113 is preferably provided with gripping material 720 on the surfaces of the gripper 113 which come into contact with the crop/stem. The gripping material is soft and preferably resilient such that the force exerted on the crop/stem is less than the force required to crush or damage the crop/stem. The gripping material 720 may be rubber, foam, silicone, thermoplastic or plastic, either alone or in combination with each other. In one embodiment the gripping material 720 includes voids so that the gripping material 720 can more easily flex around the crop/stem. Voids which are of a similar size to the crop/stem are particularly preferred.

The profile of the surfaces of the gripper 113 may be modified to suit the target crop or stem. For example, slotted gripping surfaces 810, as shown in FIG 9, more firmly grip large diameter crops or woody stems 820. Other surface profiles include saw-tooth, semi-circle, wave or V-notch shapes. The surfaces may also be beveled, multi-faceted or curved.

The gripper 113 may also include a rotation actuator 910, as shown in FIG 10, to move the fingers 911 and 912 of the gripper 113 such that the crop/stem 920 is rotated whilst being held by the gripper 113. FIG 11 shows one embodiment in which the rotation actuator 910 is in the form of a solenoid 1710. The solenoid 1710 provides a pulling or pushing force on one gripper finger 911 in a direction parallel to the gripper finger 911 surface, thus rotating the crop/stem 920.

In another embodiment, as shown in FIG 12, one gripper finger 911 of the gripper is connected to a convey and pulley mechanism 1810 which enables movement of gripper finger 911 relative to a second gripper finger 912.

A further embodiment of the rotation actuator 910, as shown in FIG 13, is a motor 1910 which drives a non-captive threaded shaft 1920 which is connected to gripper finger 911. Activation of the motor 1910 moves the gripper finger 911 parallel to a second gripper finger 912. Other embodiments of the rotation actuator 910 include a cable mechanism, pneumatic mechanisms or the use of servo motors.

More than one rotation actuator 910 may be utilized to minimize lateral movement of the crop/stem. For example, one rotational actuator 910 may be located on each of first and second gripper fingers. The first gripper finger 911 is moved an amount in one direction, and the second gripper finger 912 is moved an equal amount in the opposite direction.

The picker 114 serves to sever the crop from the plant. The severance action may include bending, cutting, twisting, crushing, pulling or a combination thereof. The picker 114 is preferably incorporated within the gripper 113, but it may be provided separately.

The picker 114 is preferably a blade or sharp edge as part of the gripper 113, which cuts the stem of the crop when the stem is pulled against it. Various gripper 113 cross sections including sharp edges are shown in FIG 14.

Alternatively, the picker 114 may include a separate stem cutting mechanism such as a simple rotational blade, a scissor blade, a retractable thin hook, a flexible material moving at high speed, a saw tooth edged blade, a high velocity air knife, an hydraulic knife (where water at high speed severs the stem), a flame or hot edge, or a guillotine. One embodiment is shown in FIG 15A-C, wherein the crop 1010 is held by a suction gripper 1020. Reciprocating saw blades 1030 move down the axis of the suction gripper 1020 to sever the stem of the crop 1011. The reciprocating saw blades 1030 consist of a stationary saw blade 1031 and a rotating saw blade 1032.

A trimmer may be also incorporated within the gripper 113 to trim excess stem from the crop once the picker 114 has severed the crop from the plant. The trimmer is preferably located on the other side of the gripper 113 from where the stem of the crop was severed. In a preferred embodiment the trimmer cuts off the remaining stem between the gripper 113 and the crop after the crop has been placed in the crop processing unit 140.

The picking and trimming action is shown in FIGS 16A-16D. Firstly, FIG 16A shows the stem 1110. of the crop grasped by the gripper 113. The gripper 113 bends the stem 1110, FIG 16B, to contact the stem 1110 with a sharp edge picker 1120, FIG 16C. After severing the stem 1110 on the sharp edge, 1120, the gripper 113 is closed further, FIG 16D, to contact the edges of a trimmer 1130 to trim excess stem 1110 from the crop. As crop to be picked is frequently hidden from view behind leaves, unripe crop or stems of the plant, an obstruction mover 115 may be incorporated into the picking head 110. Alternatively, the obstruction mover 115 may be separately located on the crop picking device 100. In a preferred embodiment the obstacle mover 115 rotates about the centre of a plant to move foliage. This allows the movement of foliage with minimal obstruction of the picking head 110. In a particularly preferred embodiment one or more sensors may be mounted on the obstacle mover 115.

The obstruction mover 115 may be in the form of a stream of air from a blower 1210, as shown in FIG 17. Preferably, the air flow 1211 is directed along the axis of the sensor 111. Alternatively, a stream of liquid, such as water, may be directed to move obstructions from the path of the sensors 111 and/or gripper 113.

In another form, the obstruction mover 115 may operate to mechanically move obstructions. One example of a mechanical obstruction mover 115 is a brush. Preferably the brush is a rotary brush 1310, as shown in FIG 18, or a set of rotary brushes to clear the path to the crop for the gripper. Other mechanical obstruction movers 115 include a dragline consisting of a flexible tensile member such as a chain or cord, a rigid or semi-rigid finger-type device, or the picking head 110 itself. In one preferred embodiment the obstruction mover 115 is a solid resilient sheet suspended from a beam. The solid resilient sheet is preferably folded rubber which surrounds a weight, such as a chain suspended from the beam. Examples of dragline-type obstruction movers 115 are shown in FIG 19A-E. The obstruction mover 115 may include one or more lifters, such as a rolling ball 1410,- small wheel or disk 1420, to lift the obstruction mover 115 above ground level. The option of a rigid section 1430 of the obstruction mover 115 reduces snagging of the dragline. In another form, an obstruction mover 115 consists of a dragline with a rotary brush 1440 or rolling tube 1450 which can lift and also press down foliage. In this form, the rolling tube 1450 may be made from a soft flexible material to reduce damage to the crop.

The obstruction mover 115 may be moved by simply dragging with the motion of the picking device 100 or it may include mechanisms to allow coordinated motions to optimize crop exposure tailored to the crop morphology. In one embodiment, as shown in FIG 20, a multiple of obstruction movers 115 are mounted on a crop picking device 100 and dragged with the forwards motion 1510 of the crop picking device 100. In another embodiment, as shown in FIG 21 , a multiple of obstruction movers 115 are mounted on mechanisms on the crop picking device 100 which move in different directions. A first mechanism 1610 moves in a first direction 1611 , a second mechanism 1620 moves in a second direction 1621 and a third mechanism 1630 moves in a third direction 1631.

In another embodiment, a control mechanism may adjust the position of the obstruction mover 115 relative to the edge of the plant canopy, the centerline of the plant, the crown of the plant or the trunk of the plant.

Height sensors may be used to determine the height of a bed of the crop, to enable the obstruction mover 115 to operate close to the ground. In a preferred embodiment, stereo vision cameras are utilized to determine the height of the top of crop foliage, and then infer the height of the bed.

Any of the forms of an obstruction mover 115 may be used in combination with each other according to the type of crop being harvested. The obstruction mover 115 may continually operate or alternatively it may be controlled by the diagnostic unit 130. Based on the known morphology and characteristics of the crop, movement of the obstruction mover 115 may be programmed and instructed by the diagnostic unit 130 to give optimum movement of obstructions. In another embodiment, the obstruction mover 115 is operated by a robotic actuator with at least one degree of freedom.

The manipulator 120 moves the picking head 110 to and from the crop. The manipulator 120 may include a robotic articulated arm, (including a Selective Compliant Assembly Robot Arm or Selective Compliant Articulated Robot Arm (SCARA) manipulator, parallel arm (Delta) robotics), or gantry or Cartesian robotics.

In a preferred embodiment the manipulator 120 is moved, whilst the crop picking device 100 is in motion, such that the speed of the manipulator 120 relative to the ground is very low or nil.

The diagnostic unit 130 receives information from the sensors 111 regarding the plant and the crop and processes the information according to pre-determined guidelines encoded in software. According to the results of the processed information, the diagnostic unit 130 then instructs the manipulator 120 and instruments on the crop picking head 110 as to motions required.

Calibration of the diagnostic unit 130 may be carried out by providing samples. Sensors 111 on the crop picking device 100 are exposed to the samples and the responses are calibrated to indicate the feature demonstrated by the sample. For example, crop samples at each end of a desired colour range or a per cent size range from a single crop sample allows calibration under similar environmental conditions as the crop to be harvested.

The crop processing unit 140 may include one or more sprayers for washing the picked crop and/or applying other chemicals. One or more trimming cutters may be provided for trimming off excess plant matter and/or unwanted parts of the crop. Alternatively, the one or more trimming cutters may be located on the picking head or incorporated into the cutter.

The crop processing unit 140 may also include means for grading the crop by size, colour, ripeness, imperfections or the like, as are well known in the field.

The final potential element of the crop processing unit 140 is packaging the crop. Packaging boxes, punnets, trays or bags may be filled immediately following picking by the manipulator 120 moving the picking head 110 holding the picked crop to the packaging and placing it inside the packaging. Alternatively, the manipulator 120 may move the picking head 110 to place the picked crop into the crop processing unit 140, which then processes the crop and deposits the processed crop into packaging. The manipulator 120 and/or the picking head 110 may also be utilized within the crop processing unit 140 for other tasks such as moving packaging material, either filled or unfilled, or closing the packaging once filled.

The chassis 150 supports the crop picking device 100 and propels the crop picking device 100 to and from a plant bearing crop. The chassis 150 may include wheels for independent movement down a row of plants, or the chassis 150 may be mounted on a track. Alternatively, the crop picking device 100 may be stationary, and the plants moved to the crop picking device 100 either manually or by a conveyor.

The crop picking device 100 may also include guidance means 160 to guide the crop picking device 100 to follow an agronomic target. The agronomic target may include a crop row, a ridge of earth, a raised bed irrigation pipe, a fixed rail, a heating or cooling pipe, a cable, support members for hydroponically grown crops, or other physical target that is aligned with a row of a crop. The guidance means 160 controls both the directional heading of the crop picking device 100 as well as reducing any offset distance between the actual path taken by the crop picking device 100 and the desired path.

The guidance means 160 includes one or more guidance sensors and a steering wheel. Suitable guidance sensors include a camera, an infra-red sensor, a proximity sensor, a sensor for determining absolute position (for example Global Positioning Sensor (GPS), inertial sensor or compass) of the crop picking device 100, a sensor for determining the relative heading of the crop picking device 100 (such as a compass), a sensor for determining the relative position or movement of the crop picking device 100 (such as ultrasound or mechanical feeler sensors), or a mechanical sensor to determine locations of the agronomic target by tactile sensing.

The mechanical sensor could be a rod, slide, bumper, feeler or wand, as are known in the field, used to provide location information. A chain or other tension member can be dragged to track an agronomic target, particularly when the agronomic target is a furrow.

As well as sensing the location of the agronomic target, it is also preferable to detect where the agronomic target ends in order to instruct the crop picking device 100 to stop, turn around or move to a different agronomic target or location. Where the agronomic target is a raised bed, the end of the agronomic target can be detected by sensing the presence or lack thereof of the raised bed.

In one embodiment a guidance sensor is placed on the chassis 150 of the crop picking device 100 to detect the presence or lack thereof of the raised bed. The guidance sensor is suitably one or more of a photoelectric sensor, an ultrasound sensor or a mechanical switch. In a further embodiment, the guidance sensor used to determine the location of the agronomic target may also be used to determine if the agronomic target is no longer there. For example, if a mechanical sensor is in physical contact with a raised bed, a measurement of the magnitude of the force of the raised bed on the mechanical sensor can be used to determine if the bed is no longer present. Alternatively, a photo interrupter sensor may be used to determine the presence or absence of the agronomic target.

The guidance sensor transmits the required directional heading to the steering wheel. The steering wheel is a wheel of the crop picking device 100 which, when turned, alters the heading of the crop picking device 100.

Information from the guidance sensor, particularly non-mechanical guidance sensors may be transmitted to a control device for interpretation. The control device then transmits the required directional heading to the steering wheel.

A proportional-integral-derivative (PID) controller is a particularly preferred control device. The control device may make use of vehicle dynamics modeling to improve estimation of the position and heading of the crop picking device 100. Information from a multiple of guidance sensors may be combined by the control device. For example, measurements from one guidance sensor placed at the front of the crop picking device 100 are combined with measurements from a second guidance sensor placed at the back of the crop picking device 100 to provide information as to the location and/or heading of the crop picking device 100. A Kalman filter, as is known in the art, may be used when combining information from more than one guidance sensor.

The guidance sensor may also be combined with a guidance illumination device. In one embodiment the agronomic target is illuminated by a focused beam of electromagnetic radiation, such as a laser beam. The laser beam may suitably be focused in a line pattern. A camera is used to sense the position of the laser beam. A complimentary metal-oxide semiconductor (CMOS) camera and laser beam illumination combination is particularly preferred.

Alternatively, the sensor may use stereo vision to estimate the position of the agronomic target relative to the crop picking device 100. In one embodiment the planar shape of an edge of a raised bed containing crop is determined using a stereo matching algorithm such as stereo edge detection followed by a determination of a disparity map from the edges.

When the agronomic target is symmetrical or nearly symmetrical, such as a raised bed, guidance sensors may be placed to view each side of the agronomic target. The differences in the estimates of the guidance sensors on each side are used by the control device to provide guidance information. Thus absolute estimates of the position and heading of the crop picking device 100 is not required.

The use of vision based guidance sensors may be enhanced by the addition of specific colors or patterns onto the agronomic target. In one embodiment lines may be printed onto edge materials used in raised bed farming.

It is preferable to filter noise from information gathered by the guidance sensor or alternatively to filter noise prior to the guidance sensor gathering the information. In one embodiment a narrow bandpass filter is placed in front of a camera sensor. Using this configuration noise from other sources of electromagnetic radiation can be effectively excluded. Alternatively, filters may be applied to the gathered information using computer software to filter noise or average readings. In a preferred embodiment a Hough Transform, as is known in the art, is applied so that the position of the laser line in an image gathered by a guidance sensor can be estimated more accurately.

Mechanical guidance sensors are suitably coupled to an axle, fork or other framework of the crop picking device 100. The coupling serves to directly control the steering wheel of the crop picking device 100 to alter its directional heading, according to pressure exerted on the mechanical sensor.

In agricultural environments the agronomic target may vary with time, due to changing weather and soil conditions, as well as the effects of farming operations throughout the growing cycle of the crop. For example, the height and width of a raised bed may alter due to natural slumping. In order to account for this variability, the mechanical sensors suitably include adjustment means. The adjustment means may be in the form of a slide mechanism with a lock nut, or alternatively automatic adjustment means controlled by the control device.

In a preferred embodiment, as shown in side view in FIG 22 and plan view in FIG 23, a mechanical sensor consists of a first horizontal wheel 2010 connected to a first tensile member 2011 that urges the first horizontal wheel 2010 towards an adjacent raised bed 2020. The first tensile member 2011 is adjusted so that the first horizontal wheel 2010 makes positive contact with a side of a first raised bed 2020. A second horizontal wheel 2030 is connected to a second tensile member 2031. The second tensile member 2031 is adjusted so that the second horizontal wheel 2030 makes positive contact with a side of a second raised bed 2040. The first tensile member 2011 and the second tensile member 2031 are suitably one or a combination of springs, pneumatic device, hydraulic pistons or the like.

The first tensile member 2011 and the second tensile member 2031 are connected to the steering wheel 2050

The forces exerted by the first raised bed 2020 onto the first horizontal wheel 2010 and second raised bed 2040 onto the second horizontal wheel 2030 and balancing of the resultant tension forces of the first tensile member 2011 and the second tensile member 2031 serve to maintain the steering wheel 2040 in a central location between the raised beds 2020.

A further mechanical sensor comprising a third horizontal wheel 2060, third tensile member 2061 and fourth horizontal wheel 2070 and fourth tensile member 2071 may also be connected to the steering wheel 2050 to provide additional guidance means 160.

In another preferred embodiment, as shown in FIG 24, the crop picking device 100 (not shown in this figure) is located to straddle a raised bed 2110. The horizontal wheels 2120 make positive contact with the sides of the raised bed 2110 to maintain the crop picking device 100 centrally located above the raised bed 2110.

In an alternate embodiment, as shown in FIG 25, the mechanical sensor is in the form of a guide rail 2210, which is placed under tension by a tensile member 2220 and urged against the side of a raised bed 2230. The guide rails 2210 are connected to a steering wheel 2240. The guide rails 2210 may also be fitted with wheels or suitable sliding material to limit any disturbance to the crops. Similarly to the above, balancing of the tensile forces against the force exerted by the raised bed 2230 serves to centre the steering wheel 2240 between raised beds 2230.

Various configurations of horizontal wheels, guide rails and the like may be used according to the configuration, chassis 150 and weight distribution of the crop picking device 100. The configuration may also be varied according to the height and width of the raised beds and the spacing between adjacent raised beds. Some possible configurations are shown in FIG 26A-33B, in both side view and plan view.

When the mechanical sensors are in use near to the ground, they may be subject to large loads (for example, if a rock is encountered or if a wheel falls into a rut). The mechanical sensor may be protected against ground surface irregularities by a wide variety of protecting mechanisms. The protecting mechanisms include one or more of the following mechanisms:

1. The mechanical sensors can be allowed to pivot or float freely in any direction perpendicular to the sensing direction. The bottom limit of the pivot may be limited by any combination of the following mechanisms: a. a mechanical stop or b. the sensor may be designed so as to slide on the ground with the ground limiting the travel of the sensor or c. the sensor may have a ground wheel.

2. The mechanical sensor can have a spring mechanism to absorb the shock of unexpected loads.

3. The mechanical sensor can have a damping pneumatic or hydraulic mechanism.

4. The mechanical sensor can have a shear bolt or deliberately weakened element that allows the sensor to swivel back.

5. The mechanical sensor can have spring or pneumatic or hydraulic trip mechanism such that the sensor trips back if an excessive load is encountered.

Similar mechanisms can be used to protect other sensors that are fitted in exposed locations.

In operation, the chassis 150 moves the crop picking device 100 towards the plant containing crop. The manipulator 120 moves the picking head 110 towards the crop, as instructed by the diagnostic unit 130 which acts according to information received by the sensors 111. An obstruction mover 115 clears a path such that the sensors 111 may view the crop and provide information to the diagnostic unit 130. The diagnostic unit 130 continues to process the information received from the sensors 111 and instruct the manipulator 120 to move the picking head 110 until the crop is within reach of the gripper 113.

If the crop is grown in a container, the container may be tilted so that crop which is located to the side or under the container is more easily available to the gripper 113. Tilting of the container also allows a larger space for the crop picking device 100 to operate. The container may be tilted by the manipulator 120 of the crop picking device, or by a separate mechanism. After the crop has been picked, the container may be tilted in another direction, or returned to its initial position.

As shown in FIG 34A and 34B, the gripper 113 then grips the crop and moves it such that the sensor 111 can view multiple aspects of the crop. The rotation actuator 910 may be used to rotate the crop for viewing by the sensor 111. Alternatively, as shown in FIG 35A and 35B, the gripper 113 holds the crop and the sensor 111 is moved to view multiple aspects of the crop.

Once the crop is determined by the diagnostic unit 130, from the information provided by the sensors 111 , to be suitable for picking, the picker 114 acts to sever the crop from the plant. The picker 114 may operate in concert with movement of the gripper 113. Once the crop is picked, the gripper 113 is moved by the manipulator 120 to place the picked crop either in packaging or into a crop processing unit 140.

By individually assessing multiple aspects of each piece of crop prior to picking, a better determination of ripeness or suitability for picking may be made. Unripe crop may be left on the plant to mature further for later picking. Additionally, crop that is unsuitable, i.e. damaged crop, may be left on the plant, severed to fall to the ground or picked for removal.

Individual handling of the crop results in less damage to the crop or the plant, giving crop that is of a higher quality and a plant which can The crop picking device 100 is able to pick crop throughout the day and night without requiring a break. Thus a good picking rate compared to a human worker results.

In a particularly preferred embodiment the crop picking device is configured as a strawberry harvester. The preferred sensors located on the picking head include electromagnetic sensors, specifically complementary metal oxide semiconductor (CMOS) image sensors. Additionally, a laser sensor is included as a supplementary means of determining distance. Illumination means, including polarizing filters, are provided to give a constant light to the region of interest being viewed by the sensors.

Information from the sensors is processed in a diagnostic unit by methods including machine vision algorithms and/or monoSLAM (single sensor vision-based Simultaneous Localisation And Mapping) and/or stereo vision techniques and/or other sensors such as a laser scanner. In a preferred embodiment the distance to the target is estimated by a triangulation technique. The target is approached in a linear motion and the change in apparent height of the target is used to calculate the distance to the target.

The preferred gripper has two fingers, each finger having a single tine. The tines of the fingers have a surface recess that conforms to the shape of the stem. Preferably the surface recess is a single notch to grip the stem once it is in place within the fingers. Additionally, the gripper fingers are shaped to lift stem from ground as the stem is grasped. One embodiment of the preferred gripper is shown in FIG 4. An independent infra-red sensor is integrated into the gripper to confirm that a stem is grasped.

The preferred picker is a sharp edge located on each gripper finger. A 90° rotation and pulling of the stem against the finger edge severs the stem.

A trimmer is also incorporated into the preferred picker to sever excess stem after the strawberry is severed from plant.

In the preferred embodiment, the picking head rotates and tilts in order to access each fruit and allow observation of multiple aspects of the fruit by the sensors. A rotation actuator is also used to rotate the fruit for observation. An obstacle mover is also incorporated into the picking head.

The crop picking device 100, or another autonomous robotic vehicle equipped with one or more of the sensors described above, may also be used to collect information regarding a crop for crop management purposes.

Visual information sensors can be configured and calibrated to provide information including sample images of the crop and surrounding area, or high resolution images and/or infra-red images of the crop.

Interpretation of the images by a control device suitably uses machine vision processing and algorithms. The interpretation can be calibrated to provide information regarding the type and/or number of pests present on a crop, the type and/or presence of fungus on a crop, plant stress, the number of flowers on a crop (further interpreted to future yields), a count of ripe and unripe crop or the size, shape, color, odor, solidity, firmness or texture of a crop.

In one embodiment, the locations of each insect detected is recorded and presented as a distribution map of insect pest pressures by total insects, or by specific species (e.g. rutherglen bug) or by category (e.g. beneficial versus pest insects).

The information gathered may be combined with external information such as harvesting time, and tracking of the picked crop to allow origin tracing of a crop. For example, a strawberry punnet may be traced to the plant where it was harvested using the recorded GPS data with an image of each strawberry that was harvested and put into the punnet. Bar code or radio frequency identification (RFID) systems may also be incorporated into tracking systems.

Crop management information may be collected in conjunction with a crop picking device as described above. In one embodiment the sensed information may be interpreted by the control device which then instructs the crop picking device to remove one or more flowers, leaves, stems, or unripe fruit. Runners may be removed or fruiting stems separated from leaf stems. This can reduce yield peaks, and/or extend the crop season or provide better quality fruit later in the season.

The collected crop management information can be analyzed by the autonomous robotic vehicle, or it may be transmitted to a different location. In one embodiment the information is transmitted to one or more remote locations for viewing and interpretation by experts such as an experienced farmer, agronomist, entomologist, agricultural chemist, agricultural economist, or an accountant. Said experts or other operators may in a preferred embodiment be able to provide control instructions from a remote location to the autonomous robotic vehicle to optimize production and/or minimize environmental impacts.

Throughout the specification the aim has been to describe the invention without limiting the invention to any one embodiment or specific collection of features. Persons skilled in the relevant art may realize variations from the specific embodiments that will nonetheless fall within the scope of the invention.

It will be appreciated that various other changes and modifications may be made to the embodiment described without departing from the spirit and scope of the invention.

Claims

Claims:

1. A device for selectively picking a crop from a plant comprising: at least one sensor that receives information from a region of interest, including multiple aspects of the crop, and converts the information to an electronic output; at least one diagnostic unit that receives the electronic output and processes the electronic output to determine location of the crop and suitability of the crop for picking; a gripper activated to hold the crop or a stem of the crop; and a picker activated to sever the crop from the plant.

2. The device of claim 1 , wherein at least one of the diagnostic units receives the electronic output from the at least one sensor and processes the electronic output to determine location of obstacles.

3. The device of claim 1 , wherein the electronic output from two or more sensors is combined by sensor fusion.

4. The device of claim 1 , wherein the picker is incorporated into the gripper.

5. The device of claim 1 further comprising a mechanism for moving obstructions from the path of a sensor or a gripper or a cutter.

6. The device of claim 1 , wherein the gripper includes a gripper sensor to confirm an object has been gripped.

7. The device of claim 1 , further including a gathering funnel attached to the gripper.

8. The device of claim 1 , wherein at least some surfaces of the gripper include gripping material.

9. The device of claim 1 further comprising illuminating means to provide light in the region of interest.

10. The device of claim 9 wherein light emitted by the illumination means is polarized.

11. The device of claim 9 wherein light emitted by the illumination means is polarized and at least one of the sensors includes a cross polarization filter.

12. The device of claim 1 further comprising a trimmer to sever excess stem from a crop once the crop is severed from the plant.

13. The device of claim 1 further comprising one or more sensors for grading picked crop.

14. The device of claim 1 further comprising a crop storage device.

15. The device of claim 5 wherein the mechanism for moving obstructions is the gripper.

16. The device of claim 5 further including at least two mechanisms for moving obstructions.

17. The device of claim 5 wherein a height sensor is utilized to locate the mechanism for moving obstructions.

18. The device of claim 1 wherein at least one sensor detects infra-red light.

19. The device of claim 1 wherein the picker is located on a picking head.

20. The device of claim 19 wherein the picking head is moved by a manipulator.

21. The device of claim 20 wherein the manipulator is a robotic articulated arm.

22. The device of claim 19 wherein at least one illuminating means is located on the picking head.

23. The device of claim 19 wherein the gripper is located on the picking head.

24. The device of claim 19 wherein at least one sensor is located on the picking head.

25. The device of claim 19 wherein at least one mechanism for moving obstructions is located on the picking head.

26. The device of claim 1 , wherein at least one sensor is located on a frame which is stationary relative to movement of the gripper.

27. The device of claim 26, wherein at least one illuminating means is also located on the frame.

28. The device of claim 1 wherein the gripper has a first gripping surface which is moveable relative to at least a second gripping surface, such that a crop or a stem of the crop held by the gripper is rotated.

29. The device of claim 28, wherein the gripper further comprises a rotation actuator to rotate the crop or the stem of a crop.

30. The gripper of claim 29, wherein the rotation actuator controls movement of a first gripping surface.

31. The device of claim 29 wherein a rotation actuator is associated with each of a first gripping surface and a second gripping surface.

32. The gripper of claim 29, wherein the rotation actuator is a solenoid.

33. The device of claim 1 , further comprising guidance means to guide movement of the device along a pre-determined pathway.

34. The device of claim 33, wherein the guidance means comprises one or more guidance sensors.

35. The device of claim 34, wherein the guidance sensor is comprised of; a pair of horizontal wheels, each horizontal wheel connected to a tensile member; and a steering wheel which is connected to each tensile member.

36. The device of claim 34, wherein the guidance sensor comprises; a guidance illumination device which illuminates an agronomic target; a camera which senses the illuminated agronomic target and outputs location information; a control device which receives the output information and creates a steering instruction; and a steering wheel which receives the steering instruction.

37. A method for picking a crop from a plant including the steps of: observing a region of interest with at least one sensor to provide information in electronic format to a diagnostic unit; processing the information in the diagnostic unit to determine location of crop and provide instructions to a gripper; activating the gripper to move to and grip the crop; observing aspects of the crop to provide information in electronic format to the diagnostic unit by changing the relative orientation between the crop and at least one sensor; processing the information in the diagnostic unit to determine suitability of crop for picking and provide instructions to a picker; and activating the picker to sever the crop from the plant.

38. The method of claim 37, also including activating an obstacle mover to remove obstacles from a region of interest.

39. The method of claim 37, also including activating an obstacle mover to remove obstacles from a path of the gripper.

40. The method of claim 37, also including activating an obstacle mover to remove obstacles from a path of the picker.

41. The method of claim 37, also including activating an obstacle mover to remove obstacles from a path of a crop picking device.

42. The method of claim 37 also including illuminating a region of interest with illuminating means.

43. The method of claim 37 also including utilizing the gripper to place crop severed from the plant into a crop processing mechanism.

44. The method of claim 37 also including utilizing a rotation actuator to rotate crop or a stem of the crop.

45. The method of claim 37 also including utilizing a trimmer to trim excess stem from a crop severed from the plant by the picker.

46. The method of claim 37 also including activating at least one sensor to grade the crop severed from the plant.

47. The method of claim 37 also including placing the crop severed from the plant into a crop storage device.

48. The method of claim 37 also including guiding a crop picking device to a crop on a plant.

49. The method of claim 37 also including guiding a crop picking device along an agronomic target by mechanical guiding means.