EP4680892A1

EP4680892A1 - Diffuse illumination for robotic workspaces

Info

Publication number: EP4680892A1
Application number: EP24771772.1A
Authority: EP
Inventors: Surya MURUGAVEL RAVISHANKAR; Kai BIEGUN; Ethan DONION; Noe FONTANA; John HUCKINS; Matthew Pearce; Guang Wen SUN
Original assignee: Pickle Robot Co
Current assignee: Pickle Robot Co
Priority date: 2023-03-15
Filing date: 2024-03-14
Publication date: 2026-01-21
Also published as: WO2024192298A1

Abstract

A robot adapted to interact with objects in an environment comprises a means of manipulation, a machine vision subsystem, and an illumination subsystem, comprising a plurality of lighting elements, each of which may be a directed light source or a diffuse light source and each of which may be angled laterally toward a wall of the environment, upward toward the ceiling of the environment, or downward toward the floor of the environment. The walls, ceiling, and/or floor of the environment act as diffusers which provide indirect illumination of the objects to be manipulated.

Description

DIFFUSE ILLUMINATION FOR ROBOTIC WORKSPACES

I. PRIORITY CLAIM

[0001] The present invention is related to, and claims priority from, United States Provisional Patent Application Ser. No. 63/490,317, filed on March 15, 2023, the disclosure of which is hereby incorporated by this reference in its entirety.

II. FIELD OF INVENTION

[0002] The invention disclosed herein relates to methods and devices for diffuse illumination of a robot workspace.

III. BACKGROUND OF THE INVENTION

[0003] In the warehouse logistics industry, there is a push for the use of robotics guided by artificial intelligence algorithms to move and sort packages. This push is a continuation of a general trend of automation in industry and reflects both economic and social pressures to give physically difficult jobs to machines to perform instead of humans. Within the package handling arena there are various subcategories of automation. The current application is particularly concerned with robots in a warehouse environment that are configured to unload trailers of parcels, but the invention disclosed herein has other applications as will be evident to a person of skill in the art.

[0004] Warehouse logistics robots rely on various sensors to perceive their environment and detect packages that must be picked and handled, including, for example, digital cameras and laser rangefinders. Ambient lighting in the warehouse or trailer environment or other enclosed robotic workspaces is often not optimal for the operation of such sensors. For example, ambient lighting may be insufficient for digital cameras to properly perceive the environment, especially in the context of a trailer unload robot where ambient light is minimal within a partially enclosed trailer. To address that issue, existing robot systems may include lights to illuminate the environment for the robot’s sensors. Conventionally, illumination has been provided by directed light sources.

[0005] Directed light sources, however, may cause problems for package handling robots. For example, packages or other objects in the environment may be partially or totally reflective. Packages often have a glossy faces, are covered with plastic wrapping, or fastened with packing tape, each of which reflect light in a way that may interfere with or degrade sensor operation. When illuminated with one or more directed light source(s) reflective surfaces may prevent digital cameras from perceiving texture and/or depth as well as degrade rangefinder operation. In addition, in some situations operation of a robot’s machine vision system may be improved by illuminating a scene uniformly throughout the field of view, which is generally not accomplished by directed light sources. The invention of this disclosure addresses these and other issues in robotic workspaces.

IV. BRIEF SUMMARY OF THE INVENTION

[0006] Systems and methods of the instant disclosure create an advantageous illumination environment for machine vision, especially in enclosed robotic workspaces, by directing light toward the walls, ceiling or floor of a workspace and thereby diffusely reflecting light within the workspace.

V. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figs. 1 A-C shows a robot employing one embodiment of the instant invention.

[0008] Fig. 2 shows an overhead view of an embodiment of the instant invention comprising a robot working in a trailer workspace.

[0009] Fig. 3 is a logical block diagram of a system in which one embodiment of the instant invention may be implemented.

[0010] Fig. 4 shows an overhead view of an embodiment of the instant invention comprising a robot working in a trailer workspace with a light control barrier installed. VI. DETAILED DESCRIPTION

[0011] Techniques described herein pertain to illumination of a robot workspace environment. Disclosed components may facilitate robot perception of the workspace and prevent or mitigate shortcomings and issues of prior art systems.

[0012] In one embodiment, a robot adapted to enter and unload packages from a tractor trailer comprises a means of manipulation, such as a robot arm with an end-effector, a machine vision subsystem, and illumination subsystem, comprising a plurality of lighting elements. Each lighting element may be a diffuse lighting source or a directed lighting source. Each of the lighting elements may be angled laterally toward a wall of the tailer, upward toward the ceiling of the trailer, or downward toward the floor of the trailer. The walls, ceiling, and/or floor of the enclosed workspace (in this embodiment, a trailer) may act as diffusers which provide indirect illumination of the workspace (in this embodiment, a plurality of packages which are to be unloaded).

[0013] Fig. 1A shows one embodiment of the invention. Robot 100 comprises a mobile base 110, a robot arm 120 affixed to the mobile base 110, a vision subsystem comprising one or more cameras 130 to perceive the workspace, and a lighting subsystem comprising one or more lighting elements 140a, 140b. In some embodiments, lighting elements 140 may be placed at the same height or elevation as one or more cameras 130. In some embodiments, lighting elements 140 may be placed above or below one or more cameras 130. In this embodiment, each lighting element 140a, 140b is directed laterally outward at an angle from the direction of forward travel of the robot such that no lighting element directly illuminates the workspace in front of the robot or in the field of view of a light sensor or other vision system. Light from lighting elements 140a, 140b is directed toward the sides of the workspace. The sides of the workspace act as a light diffuser and reflect light toward the workspace in front of the robot. [0014] Fig. IB shows a frontal view of an embodiment of the invention. Lighting elements 140a and 140b are attached to the mobile base 110 at the same height or elevation as one of the cameras 130. Lighting elements 140c and 140d are attached to the mobile base 110 well above camera 130.

[0015] Fig. 1C shows a frontal view of another embodiment of the invention. Lighting elements 140a and 140b are attached to the mobile base 110 at the same height or elevation as one of the cameras. Lighting element 140c is attached to the mobile base 110 above the camera. Lighting elements 140a, 140b, 140c are angled such that the light is directed primarily at the walls of the workspace.

[0016] One of ordinary skill in the art will understand that the number and placements of the lighting elements are a design choice to implement the teachings of the instant invention and the disclosure of a particular number or particular positions of lighting elements does not limit the scope of the disclosure.

[0017] Fig. 2 shows an overhead view of robot 100 in a workspace 250 with walls 251 and objects to be manipulated 260 within the workspace. Lighting elements 140a, 140b are mounted to robot 100 such that they direct light toward walls 251 of the workspace 250. Light directed toward the walls 251 of the workspace 250 illuminates the interior of the workspace by reflecting from the walls diffusely.

[0018] Diffuse reflection from the walls or other surfaces provides a more advantageous illumination environment for machine vision because the scene is illuminated in a more uniform fashion and specular reflections from objects in the workspace that would otherwise interfere with machine vision operation are minimized or eliminated.

[0019] Fig. 3 shows a simplified block diagram of the system 300 in which some embodiments of the invention may be implemented. The overall system 300 comprises a processing subsystem 301, a movement interface 301, a robot arm interface 303, a vision subsystem 330, and an illumination subsystem 340. The processing subsystem 301 may comprise and be implemented in one or more CPUs, microprocessors, microcontrollers, ASICs, and/or FPGAs. The processing subsystem 301 may be in communication with each of the other subsystems and interfaces. The processing subsystem 301 may command the movement interface 302 to move the robot about the workspace. The movement interface 302 may communicate data from various sensors or other sources related to movement back to the processing subsystem 301. For example, the movement interface 302 may communicate data related to the amount of current required by one or more drive motors. The processing subsystem 301 may command the robot arm interface 303 to move a robot arm, pick, grip, or grasp objects within the workspace, release objects after they are picked, gripped, or grasped. The robot arm interface 303 may receive communication data from various sensors or other sources related to the robot arm or end-effector back to the processing subsystem 301. For example, if the end-effector employs suction to pick, grip, or grasp objects, the robot arm interface may communicate data related to vacuum or pressure at one or more locations within the system. The robot arm interface 303 may also communicate data related to the position of one or more joints comprising the robot arm, a position of the end-effector, or current required by a motor associated with one or more joints comprising the robot arm. The processing system 301 may issue various commands or provide information relating to the workspace environment to the vision subsystem 330. The vision subsystem 330 may communicate data relating to its perception of the workspace environment back to the processing subsystem 301. The processing system 301 may issue commands to the illumination subsystem 340, such as to power on or off one or more lighting elements or change the brightness or another characteristic of one or more lighting elements. The illumination subsystem 340 may communicate data back to the processing subsystem 301, such as fault information or data related to power usage. [0020] In some embodiments, the illumination subsystem 340 may be in communication with vision subsystem 330 directly. In these embodiments, the vision subsystem 330 may, for example, send information relating to performance of the vision subsystem 330 to the illumination subsystem 340 or issue commands to the illumination subsystem 340.

[0021] In some embodiments, the processing subsystem 301 may be in communication with only a subset of the foregoing subsystems. For example, in some embodiments, the processing subsystem 301 may not be in communication with the illumination subsystem 340. Figure 3 is a logical block diagram rather than a physical specification. Aspects or elements of the processing subsystem may be co-located with components of the various other subsystems without departing from the spirit of this disclosure. For example, aspects of the processing subsystem may be implemented in connection with the vision subsystem.

[0022] In some embodiments, one or more qualities or characteristics of the light emitted by the lighting elements may be dynamically adjusted to address different workspace environments. Different workspace environments may require different illumination for a number of reasons. For example, the walls, floor and/or ceiling of the workspace may comprise different materials and different surfaces, which change how incident light reflects. Different surfaces may cause more or less reflection in general and/or different proportions of specular versus diffuse reflection — i.e., more specular reflection and less diffuse reflection or less specular reflection and more diffuse reflection. In the particular environment of a trailer: walls may be flat or corrugated; walls may be wood, such as plywood, or metal, such as aluminum; walls may also be bare or coated with paint or anodization. The amount of ambient light in the workspace may also vary. In the instance of a trailer unload workspace, the light entering the trailer from the trailer door may be different based on the lighting in a loading dock or warehouse. Some trailers also have translucent ceilings or top covers, which allow some light into the interior workspace.

[0023] In these embodiments, the processing subsystem may receive data from the vision subsystem relating to a characteristic of the workspace, such as amount of ambient light, amount of effective illumination of the workspace being provided by the illumination subsystem at a particular time, relative ease or difficulty with which the vision subsystem perceives various objects within the workspace, or color, brightness, or reflectivity of the walls, floor, ceiling, and/or objects within the workspace. The processing subsystem may issue commands to the illumination subsystem based on the characteristics of the workspace. For example, if data reported by the vision subsystem indicates suboptimal lighting within the workspace or within a part of the workspace, the processing subsystem may issue one or more commands to the illumination subsystem to increase or decrease the brightness of all or a subset of lighting elements, depending on whether the data indicate the workspace is too brightly or too dimly lit.

[0024] In some embodiments, the one or more qualities or characteristics of the light emitted by the lighting elements that is dynamically adjusted may enable the system to enhance its model of the workspace and objects therein by means of composite image synthesis. The qualities and/or characteristics of the emitted light may be adjusted to optimize for a plurality of detectable features of the environment. In some embodiments, the spectra emitted by one or more lighting elements may be optimized or tuned to improve detection of features of the workspace by different sensors.

[0025] In some embodiments, one or more lighting elements may be mounted to actuators that are able to control the direction of the lighting element. In these embodiments, the angle of incidence of the light emitted from the lighting element and the walls, ceiling, floor or other object in the environment may be dynamically controlled. In addition, the system may illuminate only a subset of available lighting elements to obtain images of the workspace with different characteristics. In some embodiments, rather than performing composite imaging synthesis, the processing subsystem may run multiple images through a model and post process the resulting outputs.

[0026] In some embodiments, additional elements may be employed to control or attenuate external illumination of the workspace. For example, in some environments, ambient light from outside the workspace may interfere with the illumination subsystem and vision subsystem. This problem may be especially acute when the robot is working at the edge or entry of a workspace, for example as a trailer unload robot is unloading the first packages of a full trailer. Under these circumstances physical structures to block or screen ambient light may be added to the edge or entry of a workspace to reduce or eliminate lighting other than lighting elements controlled by the robot system.

[0027] Fig. 4 shows an overhead view of a trailer unload workspace 250 with robot 100 working at the entry of the workspace. A light barrier 450 may be installed to eliminate or attenuate light entering from outside the workspace and interfering with the illumination subsystem. Light barriers may be constructed from rigid structures or flexible structures, so long as the material or materials are substantially opaque and allow the robot to enter the workspace unimpeded.

* * *

[0028] In the foregoing description, various embodiments have been described. For purposes of explanation, specific configurations and details have been set forth in order to provide a thorough understanding of the embodiments. However, it will also be apparent to one skilled in the art that the embodiments may be practiced without the specific details. Furthermore, well-known features may have been omitted or simplified in order not to obscure the embodiment being described.

Claims

1. A robotic system adapted to enter a workspace and manipulate objects within the workspace comprising: a mobile base with a robot arm affixed to the mobile base; and a processing subsystem in communication with a movement interface configured to receive commands to move the mobile base about the workspace, a robot arm interface configured to receive commands to move the robot arm, pick, grip, or grasp objects within the workspace, and release objects after they are picked, gripped, or grasped, a vision subsystem comprising one or more cameras to perceive the workspace, and an illumination subsystem comprising one or more lighting elements; wherein each lighting element is directed laterally outward at an angle from the direction of forward travel of the mobile base such that no lighting element directly illuminates the workspace in front of the mobile base and one or more physical boundaries of the workspace act as a light diffuser and reflect light toward the workspace in front of the mobile base.

2. The robotic system of claim 1 wherein the one or more lighting elements are located at the same height as at least one of the one or more cameras.

3. The robotic system of claim 1 wherein the one or more lighting elements are located at a height above at least one of the one or more cameras.

4. The robotic system of claim 1 wherein the one or more lighting elements are located at a height below at least one of the one or more cameras.

5. The robotic system of claim 1 wherein the illumination subsystem is in direct communication with the vision subsystem.

6. The robotic system of claim 1 wherein the processing subsystem receives data from the vision subsystem relating to a characteristic of the workspace.

7. The robotic system of claim 6 wherein the characteristic of the workspace comprises one or more of an amount of ambient light, an amount of effective illumination of the workspace being provided by the illumination subsystem at a particular time, a relative ease or difficulty with which the vision subsystem perceives various objects within the workspace, and color, brightness, or reflectivity of the walls, floor, ceiling, and/or objects within the workspace.

8. The robotic system of claim 6 wherein the processing system issues commands to the illumination subsystem to alter one or more characteristics of light emitted by a lighting element based on one or more characteristics of the workspace.

9. The robotic system of claim 1 wherein the processing subsystem issues commands to the illumination subsystem to alter the spectra emitted by one or more lighting elements to improve detection of features of the workspace.

10. The robotic system of claim 1 wherein one or more lighting elements may be physically rotated about an axis such that an angle of incidence of light emitted from such lighting elements is dynamically controllable.

11. The robotic system of claim 1 wherein the illumination subsystem comprises a plurality of lighting elements.

12. The robotic system of claim 11 wherein the processing subsystem issues commands to the illumination subsystem to illuminate only a subset of the plurality of lighting elements.

13. The robotic system of claim 1 wherein a plurality of objects to be manipulated are colocated with one another at a distal location of the workspace and wherein physical boundaries of the workspace are defined by two lateral walls, one distal wall, one ceiling, and one floor.

14. The robotic system of claim 13 wherein each lighting element is directed toward one of the lateral walls such that no lighting element directly illuminates the plurality of objects too be manipulated and the lateral wall acts as a light diffuser and reflects light toward the plurality of objects to be manipulated.

15. The robotic system of claim 13 wherein each lighting element is directed toward one of the ceiling or floor such that no lighting element directly illuminates the plurality of objects too be manipulated and the ceiling or floor acts as a light diffuser and reflects light toward the plurality of objects to be manipulated.