US20150151933A1

US20150151933A1 - Adaptable container handling system

Info

Publication number: US20150151933A1
Application number: US14/550,232
Authority: US
Inventors: Charles M. Grinnell; Joseph L. Jones; Paul E. Sandin; Clara Vu
Original assignee: Harvest Automation Inc
Current assignee: Harvest Automation Inc
Priority date: 2008-02-21
Filing date: 2014-11-21
Publication date: 2015-06-04
Also published as: US20090214324A1; US8915692B2; EP2244910A2; EP2244910B1; WO2009105211A3; EP2244910A4; WO2009105211A2

Abstract

An adaptable handling system featuring a boundary subsystem and one or more robots. Each robot typically includes a chassis, a container lift mechanism moveable with respect to the robot chassis for transporting at least one container, a drive subsystem for maneuvering the chassis, a boundary sensing subsystem, a container detection subsystem, and a controller. The controller is responsive to the boundary sensing subsystem and the container detection subsystem and is configured to control the drive subsystem to follow a boundary once intercepted until a container is detected and turn until another container is detected. The controller then controls the container lift mechanism to place a transported container proximate the second detected container.

Description

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 12/378,612 filed on Feb. 18, 2009 entitled ADAPTABLE CONTAINER HANDLING SYSTEM, which claims priority from U.S. Provisional Patent Application No. 61/066,768, filed on Feb. 21, 2008 entitled POT HANDLING ROBOT PROJECT, both of which are hereby incorporated by reference.

FIELD OF THE INVENTION

The subject invention relates to nursery and greenhouse operations and to an adaptable container handling system including a robot which is able to pick up and transport containers such as plant containers to a specified location.

BACKGROUND OF THE INVENTION

Nurseries and greenhouses regularly employ workers to reposition plants such as shrubs and trees in containers on plots of land as large as thirty acres or more. Numerous, for example hundreds or even thousands of containers may be brought to a field and then manually placed in rows at a designated spacing. Periodically, the containers are respaced, typically as the plants grow. Other operations include jamming, (e.g., for plant retrieval in the fall), consolidation, and collection. See U.S. Patent Application No. 2005/0135912 incorporated herein by this reference.
The use of manual labor to accomplish these tasks is both costly and time consuming. But, attempts at automating such container handling tasks have met with limited success.
Large-scale or hard automation is practiced at some greenhouses especially in Europe. The “Walking Plant System” (WPS Horti System, BV) is one prominent greenhouse automation scheme. In this method, conveyor belts, tables on tracks, and other centrally controlled mechanisms maneuver containers throughout the entire production cycle—plants are never placed on an outdoor floor or on un-actuated surfaces. Thus, from seed to sale, virtually no direct human labor is needed.
Such hard automation methods can be cost effective where land and labor prices are high and in operations where only one variety of plant is produced. But, in other circumstances (more typical of the U.S.), this form of automation loses some of its advantage. Because the automation components are fully integrated into the greenhouse, hard automation installations cannot easily be scaled up or down in response to demand. Further, the initial costs of a hard automation system is very high—growers essentially purchase a plant factory.
Visser International Trade and Engineering, BV has automated one container manipulation task known as “spacing.” Spacing involves positioning containers on an outdoor field or greenhouse floor with a surrounding buffer space so that plants have room to grow. A forklift-sized device called the “Space-O-Mat” uses multiple actuated forks to position plants in a regular pattern as the device backs up under autonomous control. The disadvantages of the Space-O-Mat become apparent when it is used on real-world field. If the surface is somewhat soft or uneven, the Space-O-Mat may fail to deposit containers properly. Also, many greenhouse are too tightly configured to accommodate Space-O-Mat.
A research project begun in 2000 and sponsored by NASA, Carnegie Mellon University, and the American Nurseries and Landscape Association developed a semi-autonomous device called Junior also intended to perform container spacing. Junior, a tractor-sized mechanism, automated part of the spacing task. Two workers move containers from a wagon to a conveyor on Junior that reaches across the wagon. The conveyor transports the plants to a mechanism with about a dozen grippers mounted on an arm. When each of the grippers has been loaded with a container, the arm activates. The arm maneuvers the plants onto the ground, spacing them relative to plants already on the ground. Then Junior backs up positioning itself to accept the next group of plants from the wagon.
Junior has not been developed into a commercial product. Were Junior to be offered for sale, it would likely have a high price. Junior is also a very complex piece of equipment and might require special expertise to operate. Because of its large size Junior cannot be used inside many greenhouses.

BRIEF SUMMARY OF THE INVENTION

It is therefore an object of the subject invention to provide a new robot for container handling.
It is a further object of the subject invention to provide a robot which is low cost, and simple and intuitive to operate.
It is a further object of the subject invention to provide such a robot which is simple in design and reliable.
It is a further object of the subject invention to provide such a robot which is small enough to maneuver indoors such as inside a typical greenhouse.
It is a further object of the subject invention to provide such a robot which eliminates much of the manual labor associated with nursery, greenhouse, and other growing operations.
It is a further object of the subject invention to provide such a robot which reliably operates even in adverse conditions.
It is a further object of the subject invention to provide such a robot which can operate for many hours and even work throughout the night.
It is a further object of the subject invention to provide such a robot which is able to place containers in their designated configuration with better accuracy than human workers.
It is a further object of the subject invention to provide a complete container handling system.
It is a further object of the subject invention to provide such a system which is scalable.
It is a further object of the subject invention to provide such a system which is safe.
The subject invention, however, in other embodiments, need not achieve all these objectives and the claims hereof should not be limited to structures or methods capable of achieving these objectives.
The subject invention features an adaptable container handling system including a boundary subsystem and one or more robots. Each robot typically includes a chassis, a container lift mechanism moveable with respect to the robot chassis for transporting at least one container, a drive subsystem for maneuvering the chassis, a boundary sensing subsystem, and a container detection subsystem. A robot controller is responsive to the boundary sensing subsystem and the container detection subsystem and is configured to control the drive subsystem to follow a boundary once intercepted until a container is detected and turn until another container is detected. The controller then controls the container lift mechanism to place a transported container proximate the second detected container.
In one preferred embodiment, the controller controls the drive subsystem to return the robot to a prescribed container source location and controls the container lift mechanism to retrieve another container at the source location. In one example, a transmitting beacon is located at the source transmitting a signal and each robot further includes a receiver for receiving the transmitted beacon signal. The boundary subsystem typically includes reflective tape which may include non-reflective portions denoting distance.
In one design, the boundary sensing subsystem includes at least one infrared emitter and at least one infrared detector and the container detection subsystem includes a linear array of infrared emitters and infrared detectors. In another design the container detection subsystem includes a camera based subsystem. One subsystem includes a laser source which emits a beam intersecting the field of view of the camera between x_minand x_max. The detection subsystem determines the distance between the robot and a container as a function of the angle of the beam. The camera is preferably mounted such that pixels are read out as columns rather than rows. The beam may be turned off between read outs of adjacent columns and then laser off columns are subtracted from laser on columns.
The container detection subsystem may also includes a system configured to detect if a container is present in the container lift mechanism. One system includes an infrared source and an infrared detector.
The preferred drive subsystem includes a pair of driven wheels one on each side of the chassis. One container lift mechanism includes a rotatable yoke, a pair of spaced forks extending from the yoke for grasping a container, and a drive train driven in one direction to rotate the yoke to simultaneously extend and lower the forks and driven in the opposite direction to simultaneously retract and raise the forks. One preferred drive train includes a first sprocket rotatable with respect to the yoke, a second sprocket rotatable with respect to the chassis, a moving link associated with the yoke interconnecting the first and second sprockets, and a means for driving the second sprocket. The system also includes a motor driving a gearbox, a driver sprocket driven by the gearbox, a third sprocket driven by the driver sprocket and fixed to the second sprocket, and a fork extending from the moving link. A forward skid plate may be mounted to the robot chassis.
A typical user interface for the controller includes an input for setting the width of a bed, an input for setting a desired container spacing, an input for setting the desired spacing pattern, and an input for setting the desired container diameter
The controller logic is typically configured to detect a container/boundary condition and to operate the drive subsystem and the container lift mechanism in response based on the condition. A first condition is no containers are detected before three boundaries are detected and a response to the first condition is to locate the third boundary or obstruction and place a transported container proximate the boundary or obstruction to place the first container in the first row. A second condition is no container is detected before two boundaries are detected and a response to the second condition is to follow the second boundary, detect a container, and place a transported container proximate the detected container to fill the first row with containers. A third condition is a first boundary is followed, a container is detected, the robot turns, but no additional containers are detected before a second boundary or obstacle is detected. A response to the third condition is to place a transported container proximate the second boundary or obstruction to fill a row with its first container.
One container handling robot in accordance with the subject invention may include a chassis; a drive subsystem for maneuvering the chassis; a container detection subsystem; and a container lift mechanism. There is a rotatable yoke, a pair of spaced forks extending from the yoke for grasping a container, and a drive train driven in one direction to rotate the yoke to simultaneously extend and lower the forks and drivable in an opposite direction to simultaneously retract and raise the forks.
A preferred version of the robot further includes a controller responsive to the container detection subsystem and configured to control the drive subsystem until a container is detected and to control the container lift mechanism to place a container carried by the forks proximate the detected container.
One adaptable plant container handling robot in accordance with the subject invention features a chassis; a container lift mechanism moveable with respect to the robot chassis for retrieving at least one container in a field at a first location, a drive subsystem for maneuvering the chassis to transport the container to a second location, a boundary sensing subsystem, a container detection subsystem, and a controller. The controller is responsive to the boundary sensing subsystem and the container detection subsystem and is configured to control the drive subsystem to follow a boundary once intercepted until a container at the second location is detected and turn until another container is detected, and control the container lift mechanism to place a transported container proximate the second detected container.
The subject invention also features a plant container handling robot including a chassis, a drive subsystem for maneuvering the chassis, and a container detection subsystem including a subsystem which detects containers on the ground, and a subsystem which detects a container carried by the robot. A container lift mechanism includes a rotatable yoke, a pair of spaced forks extending from the yoke for grasping a container, and a drive train which simultaneously extends and lowers the forks and simultaneously retracts and raises the forks. A controller is responsive to the container detection subsystem and configured to control the drive subsystem to maneuver the robot until a container is detected, control the container lift mechanism drive train to place a container carried by the forks proximate the detected container, control the drive subsystem to return the robot to a prescribed source location, and control the container lift mechanism drive train to retrieve another container at the source location.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Other objects, features and advantages will occur to those skilled in the art from the following description of a preferred embodiment and the accompanying drawings, in which:

FIG. 1 is a schematic aerial view of a typical nursery operation;

FIG. 2 is a highly schematic three-dimensional top view showing several robots in accordance with the subject invention repositioning plant containers in a field;

FIG. 3 is a block diagram depicting the primary subsystems associated with a container handling robot in accordance with an example of the subject invention;

FIGS. 4A-4B are schematic three-dimensional front views showing an example of one container handling robot design in accordance with the subject invention;

FIGS. 5A-5B are schematic three-dimensional rear views showing the primary components associated with the container lift mechanism of the robot shown in FIG. 4;

FIG. 6A-6D are highly schematic depictions showing the main algorithms carried out by the controller of the robot shown in FIGS. 3 and 4;

FIGS. 7A-7D are highly schematic depictions showing four different tasks which can be carried out by the robots of the subject invention;

FIG. 8 is a schematic front view showing one example of a user interface for the robot depicted in FIGS. 3 and 4;

FIG. 9 is a schematic view depicting how a robot is controlled to properly space containers in a field;

FIGS. 10A-10B are highly schematic views explaining a camera-based container detection system useful in connection with the subject invention;

FIG. 11 is a highly schematic view of another container detection system including infrared ranging sensors;

FIG. 12 is a flow chart depicting the primary steps associated with an algorithm for picking up containers in accordance with the subject invention;

FIGS. 13A-D are views of a robot maneuvering to pick up a container according to the algorithm depicted in FIG. 12;

FIG. 14 is a block diagram depicting the primary subsystems associated with precision container placement techniques associated with the subject invention; and

FIG. 15 is a schematic front view of a robot in accordance with the subject invention configured to transport two containers.

DETAILED DESCRIPTION OF THE INVENTION

Aside from the preferred embodiment or embodiments disclosed below, this invention is capable of other embodiments and of being practiced or being carried out in various ways. Thus, it is to be understood that the invention is not limited in its application to the details of construction and the arrangements of components set forth in the following description or illustrated in the drawings. If only one embodiment is described herein, the claims hereof are not to be limited to that embodiment. Moreover, the claims hereof are not to be read restrictively unless there is clear and convincing evidence manifesting a certain exclusion, restriction, or disclaimer.
FIG. 1 shows a typical container farm where seedlings are placed in containers in building 10. Later, the plants are moved to greenhouse 12 and then, during the growing season, to fields 14, 16 and the like where the containers are spaced in rows. Later, as the plants grow, the containers may be repositioned (re-spacing). At the end of the growing season, the containers may be brought back into greenhouse 12 and/or the plants sold. The Background section above delineates other typical nursery and greenhouse operations. The use of manual labor to accomplish these tasks is both costly and time consuming. And, as noted in the Background section above, attempts at automating these tasks have been met with limited success.
In accordance with the subject invention, autonomous robots 20, FIG. 2 transport plant containers from location A where the containers are “jammed” to location B where the containers are spaced apart in rows as shown. Similarly, robots 20 can retrieve containers from offloading mechanism 22 and space the containers apart in rows as shown at location C. Boundary marker 24 a, in one example, denote the separation between two adjacent plots where containers are to be placed. Boundary marker 24 b denotes the first row of each plot. Boundary marker 24 c may denote the other side of a plot. Or, the plot width is an input to the robot. In one embodiment, the boundary markers include retro-reflective tape laid on the ground. The reflective tape could include non-reflective portions denoting distance and the robots could thereby keep track of the distance they have traveled. Other markings can be included in the boundary tape. Natural boundary markers may also be used since many growing operations often include boards, railroad ties, and other obstacles denoting the extent of each plot and/or plot borders. Typically, at least main boundary 24 a is a part of the system and is a length of retro-reflective tape. Other boundary systems include magnetic strips, visible non-retroreflective tape, a signal emitting wire, passive RFID modules, and the like.
Each robot 20, FIG. 3 typically includes boundary sensing subsystem 30 for detecting the boundaries and container detection subsystem 32 which typically detects containers ready for transport, already placed in a given plot, and being carried by the robot.
Electronic controller 34 is responsive to the outputs of both boundary sensing subsystem 30 and container detection subsystem 32 and is configured to control robot drive subsystem 36 and container lift mechanism 38 based on certain robot behaviors as explained below. Controller 34 is also responsive to user interface 100. The controller typically includes one or more microprocessors or equivalent programmed as discussed below. The power supply 31 for all the subsystems typically includes one or more rechargeable batteries located in the rear of the robot.
In one particular example, robot 20, FIGS. 4A-4B includes chassis 40 with opposing side wheels 42 a and 42 b driven together or independently by two motors 44 a and 44 b and a drive train, not shown. Yoke 46 is rotatable with respect to chassis 40. Spaced forks 48 a and 48 b extend from yoke 46 and are configured to grasp a container. The spacing between forks 48 a and 48 b can be manually adjusted to accommodate containers of different diameters. In other examples, yoke 46 can accommodate two or even more containers at a time. Container shelf 47 is located beneath the container lifting forks to support the container during transport.
A drive train is employed to rotate yoke 46, FIGS. 5A-5B. As best shown in FIG. 5B, gearbox 60 a is driven by motor 62 a. Driver sprocket 63 a is attached to the output shaft of gearbox 60 a and drives large sprocket 64 a via belt or chain 65 a. Large sprocket 64 a is fixed to but rotates with respect to the robot chassis. Sprocket 66 a rotates with sprocket 64 a and, via belt or chain 67 a, drives sprocket 68 a rotatably disposed on yoke link 69 a interconnecting sprockets 64 a and 68 a. Container fork 48 a extends from link 71 a attached to sprocket 68 a. FIGS. 4A, 4B, and 5A show that a similar drive train exists on the other side of the yoke. The result is a yoke which, depending on which direction motors 62 a and 62 b turn, extends and is lowered to retrieve a container on the ground and then raises and retracts to lift the container all the while keeping forks 48 a and 48 b and a container located therebetween horizontal.
FIGS. 4A-4B also show forward skid plate 70 typically made of plastic (e.g., UHMW PE) to assist in supporting the chassis. Boundary sensor modules 80 a and 80 b each include an infrared emitter and infrared detector pair. The container detection subsystem in this example includes linear array 88 of alternating infrared emitter and detection pairs, e.g., emitter 90 and detector 92. This subsystem is used to detect containers already placed to maneuver the robot accordingly to place a carried container properly. This subsystem is also used to maneuver the robot to retriever a container for replacement. The container detection subsystem typically also includes an infrared emitter detector pair 93 and 95 associated with fork 48 a aimed at the other fork which includes reflective tape. A container located between the forks breaks the beam. In this way, controller 34 is informed whether or not a container is located between the forks. Other detection means may be used. Thus, container detection subsystem 32, FIG. 3 may include a subsystem for determining if a container is located between forks 48 a and 48 b, FIGS. 4-5. Controller 34, FIG. 3 is responsive to the output of this subsystem and may control drive subsystem 36, FIG. 3 according to one of several programmed behaviors. In one example, the robot returns to the location of beacon 29, FIG. 2 and attempts to retrieve another container. If the robot attempts to retrieve a container there but is unsuccessful, the robot may simply stop operating. In any case, the system helps ensure that if a container is present between forks 48 a and 48 b, FIG. 4, controller 34 does not control the robot in a way that another container is attempted to be retrieved.
In one preferred embodiment, controller 34, FIG. 3 is configured, (e.g., programmed) to include logic which functions as follows. Controller 34 is responsive to the output of boundary sensing subsystem 30 and the output of container detection subsystem 32. Controller 34 controls drive subsystem 36, (e.g., a motor 44, FIG. 4 for each wheel) to follow a boundary (e.g., boundary 24 a, FIG. 2) once intercepted until a container is detected (e.g., container 25 a, FIG. 2 in row 28 a). Controller 34, FIG. 3 then commands drive subsystem 36 to turn to the right and maneuver in a row (e.g., row 28 b, FIG. 2) until a container in that row is detected (e.g., container 25 b, FIG. 2). Based on a prescribed container spacing criteria (set via user interface 100 FIG. 3, for example), the robot then maneuvers and controller 34 commands lift mechanism 38, FIG. 3 to place container 25 c (the present container carried by the robot) in row 28 b, FIG. 2 proximate container 25 b.
Controller 34, FIG. 3 then controls drive subsystem 36 to maneuver the robot to a prescribed container source location (e.g., location A, FIG. 2). The system may include radio frequency or other (e.g., infrared) beacon transmitter 29 in which case robot 20, FIG. 3 would include a receiver 33 to assist robot 20 and returning to the container source location (may be based on signal strength). Dead reckoning, boundary following, and other techniques may be used to assist the robot in returning to the source of the containers. Also, if the robot includes a camera, the source of containers could be marked with a sign recognizable by the camera to denote the source of containers.
Once positioned at the container source location, controller 34 controls drive subsystem 36 and lift mechanism 38 to retrieve another container as shown in FIG. 2.
FIG. 6 depicts additional possible programming associated with controller 34, FIG. 3. FIG. 6A shows how a robot is able to place the first container in the first row in a given plot. Here, no containers are detected and the robot follows boundaries 24 a and 24 b. In this case, when boundary 24 c is detected, controller 34, FIG. 3 commands the robot to place container 27 a proximate boundary 24 c in the first row. Note that boundaries 24 a through 24 c may be reflective tape as described above and/or obstructions typically associated with plots at the nursery site. Any boundary could also be virtual, (e.g., a programmed distance). In FIG. 6B, the robot follows boundary 24 a and arrives at boundary 24 b and detects no container. In response, controller 34, FIG. 3 commands the robot to follow boundary 24 b until container 27 a is detected. The container carried by the robot, in this case, container 27 b, is then deposited as shown. In a similar fashion, the first row is filled with containers 27 a-27 d as shown in FIG. 6C. To place the first container in second row, container 27 e, the container 27 d in the first row is detected before boundary 24 b is detected and the robot turns in the second row but detects boundary 24 c before detecting a container in that row. In response, controller 34, FIG. 3 commands the robot to maneuver and place container 27 e, FIG. 6C in the second row proximate boundary 24 c.
Thereafter, the remaining rows are filled with properly spaced containers as shown in FIG. 6D and as explained above with reference to FIG. 2. FIG. 6 shows the robot turning 90° but the robot could be commanded to turn at the other angles to create other container patterns. Other condition/response algorithms are possible.
In this way, distributed containers at source A, FIG. 7A, can be “jammed” at location B; distributed containers at location A, FIG. 7B can be respaced at location B; distributed containers at location A, FIG. 7C can be consolidated at location B; and/or distributed containers at location A, FIG. 7D can be transported to location B for collection.
Using multiple fairly inexpensive and simple robots which operate reliably and continuously, large and even moderately sized growing operations can save money in labor costs.
FIG. 8 shows an example of a robot user interface 100 with input 102 a for setting the desired bed width. This sets a virtual boundary, for example, boundary 24 c, FIG. 2. Input 102 b allows the user to set the desired container spacing. Input 102 c allows the user to set the desired spacing pattern. Input 102 d allows the user to set the desired container diameter. The general positioning of features on the robot are shown in FIG. 4 discussed above. The boundary sensor enables the robot to follow the reference boundary; the container sensors locate containers relative to the robot. The preferred container lifter is a one-degree-of-freedom mechanism including forks that remain approximately parallel with the ground as they swing in an arc to lift the container. Two drive wheels propel the robot. The robots perform the spacing task as shown in FIG. 9 in position 1, the robot follows the boundary B. At position 2, the robot's container sensor beams detect a container. This signifies that the robot must turn left so that it can place the container it carries in the adjacent row (indicated by the vertical dashed line). The robot typically travels along the dashed line using dead-reckoning. At position 3, the robot detects a container ahead. The robot computes and determines the proper placement position for the container it carries and maneuvers to deposit the container there. Had there been no container at position 3, the robot would have traveled to position 4 to place its container. The user typically dials in the maximum length, b, of a row. The computation of the optimal placement point for a container combines dead-reckoning with the robot's observation of the positions of the already-spaced containers. Side looking detectors may be used for this purpose.
The determination of the position of a container relative to the robot may be accomplished several ways. In one example, a camera-based container detection system includes a camera and source of structured light mounted on the robot R, FIG. 10A. The field of view of the camera (FOV) intersects with a wedge of light (L) produced by a laser or other source. In the absence of ambient light, the only pixels illuminated in the camera image are those where the light wedge intersects an obstacle that is within the field of view of both camera and light source. The height of a pixel in the image codes for the range of the object. One novel aspect of the subject invention is the way in which ambient light is eliminated from the image even when the robot moves at high speed in full sun.
The structured light container measurement system may be a camera-based system designed to measure the x-y position of containers within the field of view of a camera at a relatively short range. The system is largely unaffected by ambient light, is unaffected by robot speeds of up to two meter per second or more, and is very low in cost compared to more conventional systems.
In this system, the field of view of a camera intersects a flat wedge of laser light, as shown in FIG. 10A. The camera points along the direction of robot travel but angled up; the laser wedge also points forward but tilts downward. In FIG. 10B, the camera, located at the origin, has a field of view of angle ψ. The camera is positioned such that the x-axis of the coordinate system is aligned with the lower edge of the camera's field of view. The laser (L) intersects the camera's field of view between x_minand x_maxand makes an angle θ with the x-axis. A cylindrical lens spreads the beam from the laser into a wedge coming out of the page. The camera is able to detect any object within the range x_minto x_max. When an object is at x the laser intersects the object at y=h. On the image plane of camera (a distance f from the origin) the height of h′ is measured.
From FIG. 1:
h′/f=h/x, and (1)
tan(θ)=h/(x _max −x) (2)
from which the following can be derived:
h′=f tan(θ)(x _max −x)/x, and (3)
x=(f tan(θ)x _max)/(h′+f tan(θ)) (4)
This novel system has the poorest resolution at the greatest distance and improves as the object moves toward the camera. For a typical system, x_maxmay be about 2 m, θ about 5 degrees, and ψ is, 40 degrees. Θ is determined by noting that h at x_minmust approximately match the height of the smallest container the system needs to see, e.g., 6″. At x_max, h′=0, assume that an object is located 1 cm closer than this. In this way, h′/f=4.34×10⁻⁵.
If the object were at x_min(which can be computed to be 18.9 cm) then,
h′/f=0.008. (6)
The ratio of these two values of h′/f is about 181. Thus the system can detect a difference of 1 cm at its maximum range if the camera field of view is 40 degrees and the camera has at least 181 pixels in the vertical dimension. At distances of less than x_max, changes of 1 cm in range produce a greater than one pixel difference in the image.
The container measurement system also employs a novel method to identify the structured light signal in the image. This system is low in cost because it requires minimal computation and storage can use a slow (30 Hz) low-resolution camera. Yet, it is able to deliver rapid, accurate container positioning at all light levels.
The container measurement system uses triangulation and structured light to compute the range to a container. The key challenge in such a system is to identify pixels in the image that are illuminated by the structured laser light and reject pixels that are not. The traditional method for accomplishing this is frame differencing. Here the structured light source is turned on, and an image is recorded, the light source is turned off and a second image is recorded. Next, the second image is subtracted from the first on a pixel-by-pixel basis. If the only change between the two images is the presence of the structure light in the first then the only non-zero pixels in the difference correspond to structured light. Two drawbacks to the use of this method in a robot in accordance with the subject invention are as follows. First, the robots move rapidly and thus the constraint that nothing changes between images except the structured light is not satisfied. Second, the fact that two complete images must be stored and differenced adds storage and computation requirements thus increasing cost.
The inventive container measuring system works as follows. The camera is turned to 90 degrees to its normal mounting position so that pixels normally exposed and read out in rows are read out as columns. The structured light (a plane of laser light) is synchronized to column exposure. That is, the laser is turned on, a row is exposed and read out, then the laser is turned off, and the adjacent row is exposed and read out, and so on. The detail in a typical image can be expected not to change very much from one column to the next. Thus when the laser-off column is subtracted from the adjacent laser-on column, the largest value in the difference will correspond to pixels illuminated by the laser. Further, since cameras typically have more than 100 columns per frame, the robot will move two orders of magnitude less between the processing of two columns than it moves between the time two frames are processed.
Depending on its velocity, the robot may move significantly during the time required to process a frame one column at a time. However, because the speed of the robot is known, it is possible to eliminate the resulting motion distortion from the data. Finally, besides the ability to expose and read out image data one column at a time, the structured light system places no unusual demands on the camera. The camera's built-in facilities for automatic gain adjustment should enable it to operate under all light conditions. If rather than the column-differencing scheme described above, a conventional frame-differencing scheme were used to measure the containers, the standard the camera would have to meet to ensure that the difference between two images was almost entirely due to the modulation of the structured light source rather than the motion of other image features is explained as follows. From the geometry above, the most rapid change in image position occurs when an object is near x_min. If a CIF camera chip with 288 lines is used it turns out that a change of one line in the image corresponds to a motion of 1.15 mm in x when the object is near x_min. At 2.0 m/s it takes the robot just 0.00115/2.0=575 μs to move that far. Thus to cancel the effects of motion the camera would have to grab an image 1739 times per second. Cameras that operate at this high frame rate are considerably more expensive than lower speed cameras. However, for objects near x_max, a standard 30 Hz camera would be sufficient. Thus, an alternate strategy could attempt to detect objects when they are far away using a 30 Hz camera and slow the robot down before the object reaches x_min. This requires that the velocity be about 58 times slower near x_min, than at 2.0 meters or 3.4 cm/s. This speed constraint would limit the utility of the system.
Focused light or an infrared source could be used. The container detection system can also be implemented using discrete components rather than a camera. In FIG. 11, a collection of IR ranging sensors (S) measures the distance from robot to container. Emitted beams (e) and detection paths (d) are shown.
A flowchart of the container centering/pickup method is shown in FIG. 12. FIG. 13 depicts the steps the robot performs. In step 120, the robot servos to within a fixed distance d, FIG. 13A of the container with the forks retracted. The robot is accurately aligned for container pickup when angle Θ is zero. In step 122, FIG. 12, the robot extends the forks and drives forward while servoing to maintain alignment, FIG. 13B. In FIG. 13C, the robot detects the container between its forks and stops its forward motion. In FIG. 13D, the robot retracts the forks by sweeping through an arc. This motion captures the container and moves it within the footprint of the robot.
The preferred system of the subject invention minimizes cost by avoiding high-performance but expensive solutions in favor of lower cost systems that deliver only as much performance as required and only in the places that performance is necessary. Thus navigation and container placement are not typically enabled using, for example a carrier phase differential global positioning system. Instead, a combination of boundary following, beacon following, and dead-reckoning techniques are used. The boundary subsystem of the subject invention provides an indication for the robot regarding where to place containers greatly simplifying the user interface.
The boundary provides a fixed reference and the robot can position itself with high accuracy with respect to the boundary. The robot places containers within a few feet of the boundary. This arrangement affords little opportunity for dead-reckoning errors to build up when the robot turns away from the boundary on the way to placing a container.
After the container is deposited, the robot must return to collect the next container. Containers are typically delivered to the field by the wagonload. By the time one wagonload has been spaced, the next will have been delivered further down the field. In order to indicate the next load, the user may position a beacon near that load. The robot follows this procedure: when no beacon is visible, the robot uses dead-reckoning to travel as nearly as possible to the place it last picked up a container. If it finds a container there, it collects and places the container in the usual way. If the robot can see the beacon, it moves toward the beacon until it encounters a nearby container. In this way, the robot is able to achieve the global goal of spacing all the containers in the field, using only local knowledge and sensing. Relying only on local sensing makes the system more robust and lower in cost.
Users direct the robot by setting up one or two boundary markers, positioning a beacon, and dialing in several values. No programming is needed. The boundary markers show the robots where containers are to be placed. The beacon shows the robots where to pick up the containers.
FIG. 14 depicts how, in one example, the combination of container detection system 32, the detection of already placed containers (130), the use of Baysian statistics on container locations (132), dead reckoning (134), and boundary referencing (136) is used to precisely place containers carried by the robots.
FIG. 15 shows a robot 20′ with dual container lifting mechanisms 150 a and 150 b in accordance with the subject invention. In other embodiments, the lifting mechanism or mechanisms are configured to transport objects other than containers for plants, for example, pumpkins and the like.
Thus, although specific features of the invention are shown in some drawings and not in others, this is for convenience only as each feature may be combined with any or all of the other features in accordance with the invention. The words “including”, “comprising”, “having”, and “with” as used herein are to be interpreted broadly and comprehensively and are not limited to any physical interconnection. Moreover, any embodiments disclosed in the subject application are not to be taken as the only possible embodiments. For example, the robot system disclosed herein may be used for tasks other than those associated with growing operations.
In addition, any amendment presented during the prosecution of the patent application for this patent is not a disclaimer of any claim element presented in the application as filed: those skilled in the art cannot reasonably be expected to draft a claim that would literally encompass all possible equivalents, many equivalents will be unforeseeable at the time of the amendment and are beyond a fair interpretation of what is to be surrendered (if anything), the rationale underlying the amendment may bear no more than a tangential relation to many equivalents, and/or there are many other reasons the applicant can not be expected to describe certain insubstantial substitutes for any claim element amended.
Other embodiments will occur to those skilled in the art and are within the following claims.

Claims

What is claimed is:

1-52. (canceled)

53. An autonomous mobile object handling robot comprising:

a chassis;

an object lift mechanism movably mounted on the chassis for retrieving objects at a source location and depositing the objects at a destination location;

a drive subsystem for maneuvering the chassis to transport objects from the source location to the destination location;

a boundary sensing subsystem for detecting a boundary;

an object detection subsystem for detecting objects; and

a controller receiving outputs from and being responsive to the boundary sensing subsystem and the object detection subsystem, said controller comprising one or more processors programmed with logic instructions, which when executed by the one or more processors, cause the one or more processors to:

control the object lift mechanism to retrieve an object at the source location;

control the drive subsystem to travel from the source location towards the destination location, and to intercept the boundary and to follow the boundary once intercepted; and

control the object lift mechanism to place the object at a given position relative to the boundary in the destination location.

54. The robot of claim 53, wherein the controller further controls the drive subsystem to return the robot to the source location, and controls the object lift mechanism to retrieve another object to be transported to the destination location.

55. The robot of claim 53, wherein the controller, using the object detection subsystem, detects another object proximate the boundary at the destination location, and controls the object lift mechanism to deposit the object carried by the robot at a position relative to the detected object.

56. The robot of claim 53, wherein the boundary comprises reflective tape.

57. The robot of claim 53, wherein the boundary sensing subsystem includes at least one infrared emitter and at least one infrared detector.

58. The robot of claim 53, wherein the object detection subsystem includes a linear array of infrared emitters and infrared detectors.

59. The robot of claim 53, wherein the object detection subsystem includes a camera based subsystem.

60. The robot of claim 59, wherein the camera based subsystem includes a laser source which emits a beam intersecting a field of view of a camera between x_minand x_max.

61. The robot of claim 60, wherein the object detection subsystem determines the distance between the robot and an object as a function of an angle of the beam.

62. The robot of claim 60, wherein the camera is mounted such that pixels are read out as columns rather than rows.

63. The robot of claim 60, wherein the beam is turned off between read outs of adjacent columns and laser off columns are subtracted from laser on columns.

64. The robot of claim 53, wherein the object detection subsystem includes a system configured to detect if an object is present in said object lift mechanism.

65. The robot of claim 53, wherein the object detection subsystem includes an infrared source and an infrared detector.

66. The robot of claim 53, wherein the drive subsystem includes a pair of driven wheels one on each side of the chassis.

67. An automated method of depositing objects in given locations using an autonomous mobile object handling robot comprising a chassis, an object lift mechanism movably mounted on the chassis, a drive subsystem for maneuvering the chassis, a boundary sensing subsystem, an object detection subsystem, and a controller, the method, implemented in one or more processors of the controller, comprising the steps of:

receiving outputs from the boundary sensing subsystem and the object detection subsystem;

controlling the object lift mechanism to retrieve an object at the source location;

controlling the drive subsystem to travel from the source location towards the destination location, and to intercept the boundary and to follow the boundary once intercepted; and

controlling the object lift mechanism to place the object at a given position relative to the boundary in the destination location.

68. The method of claim 67, further comprising controlling the drive subsystem to return the robot to a given object source location, and controlling the object lift mechanism to retrieve another object to be transported at the object source location.

69. The method of claim 67, further comprising detecting another object proximate the boundary at the destination location, and controlling the object lift mechanism to deposit the object carried by the robot at a position relative to the detected object.

70. The method of claim 67, wherein the boundary comprises reflective tape.

71. The method of claim 67, wherein the boundary sensing subsystem includes at least one infrared emitter and at least one infrared detector.

72. The method of claim 67, wherein the object detection subsystem includes a linear array of infrared emitters and infrared detectors.

73. The method of claim 67, wherein the object detection subsystem includes a camera based subsystem.

74. The method of claim 67, further comprising detecting if an object is present in said object lift mechanism.