DE112011104645T5 - Mobile human interface robot - Google Patents

Abstract

A mobile robot (100) including a drive system (200) having a forward direction of travel (F), a controller (500) in communication with the drive system, and a volumetric point cloud imaging device (450) over the drive system is fixed and adapted to obtain a point cloud from a volume of space including a ground plane (5) in a moving direction of the mobile robot. A dead zone sensor (490) has a detection area (492) adapted to detect an object in a volume of space (453) that is not detectable by the volumetric point cloud imaging device. The controller receives point cloud signals from the imaging device and detection signals from the dead zone sensor, and at least partially outputs drive commands to the drive system based on the received point cloud and detection signals.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This US patent application claims priority 35 USC §119 (e) to US Provisional Application 61 / 428,717, filed December 30, 2010; US Provisional Application 61 / 428,734, filed December 30, 2010; US Provisional Application 61 / 428,759, filed December 30, 2010; US Provisional Application 61 / 429,863, filed January 5, 2011, US Provisional Application 61 / 445,408, filed February 22, 2011; US Provisional Application 61 / 445,473 filed February 22, 2011; Provisional Patent Application 61 / 478,849 filed on Apr. 25, 2011; and according to 35 USC §120 to U.S. Patent Application Serial No. 13 / 032,312, filed February 22, 2011, and U.S. Patent Application Serial No. 13 / 032,228, filed February 22, 2011. The disclosures of these prior applications are regarded as forming part of the disclosure of this application and are incorporated herein by reference included in their entirety.

TECHNICAL AREA

This disclosure relates to human-interface mobile robots.

GENERAL PRIOR ART

A robot is generally an electromechanical machine controlled by a computer or electronic programming. Mobile robots can move in their environment and are not tied to a physical location. An example of a mobile robot that is widely used today is an automated controlled vehicle or AGV. An AGV is generally a mobile robot that follows markers or cables in the ground or uses a vision system or laser for navigation. Mobile robots are used in industry, military and security environments. In addition, they are available as consumer products for entertainment or to perform certain tasks, such as vacuuming or domestic support.

SUMMARY

One aspect of the disclosure provides a mobile robot including a propulsion system with a forward direction of travel, a controller in communication with the propulsion system, and a volumetric point cloud imaging device mounted above the propulsion system and configured to make a point cloud from a volume of space including a ground plane in a moving direction of the mobile robot. A dead zone sensor has a detection area arranged to detect an object in a volume of space unrecognizable by the volumetric point cloud imaging apparatus. The controller receives point cloud signals from the imaging device and detection signals from the dead zone sensor, and at least partially outputs drive commands to the drive system based on the received point cloud and detection signals.

Embodiments of the disclosure may include one or more of the following features. In some embodiments, the dead zone sensor includes at least one of a volumetric point cloud imaging device, a sonar sensor, a camera, an ultrasonic sensor, LIDAR, LADAR, an optical sensor, and an infrared sensor. The detection range of the dead zone sensor may surround a volume of space that is not detectable by the device for imaging volumetric point clouds (i.e., in a dead zone). In some examples, the volume of space that is not detectable by the volumetric point cloud imaging device is defined by a first angle, a second angle, and a radius (eg, 57 ° x 45 ° x 50 cm). The detection range of the dead band sensor may be arranged between the volumetric point cloud imaging device and a detection area of the volumetric point cloud imaging device. In some examples, the deadband sensor has a field of view that extends outward at least 3 meters from the dead zone sensor. In this example, the deadband sensor can be used for both a relatively short range within the dead zone and as a long range sensor for detecting objects that are relatively far away, to plan the path and avoid obstacles.

In some embodiments, the robot includes an array of dead zone sensors, wherein the detection area of at least one dead zone sensor is arranged to detect an object in the volume of space that is not detectable by the volumetric point cloud imaging device. The array Deadband sensors may be arranged so that the fields of view are aligned along the forward direction of travel or evenly distributed over a vertical center axis defined by the robot.

The imaging device, in some examples, emits light onto a scene around the robot and captures images of the scene along the direction of travel of the robot. The images include at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image. The controller determines a location of an object in the scene based on the images and issues drive commands to the drive system to maneuver the robot based on the object location in the scene. The imaging device may determine a transit time between the emission of the light and the reception of the light reflected from the scene. The controller uses the transit time to determine a distance to the reflective surfaces of the object.

In some embodiments, the imaging device includes a light source for emitting light into the scene and an imager for receiving reflections of the emitted light from the scene. The light source may emit the light in intermittent pulses, for example, at a first energy-efficient frequency and upon receiving a sensor event at a second, active frequency. The sensor event may include a sensor signal indicating the presence of an object in the scene. The imager may include an array of photosensitive pixels.

The imaging device may include first and second portions (eg, portions of a sensor or first and second imaging sensors). The first section is arranged to radiate light substantially to the ground and to receive reflections of the light radiated from the ground. The second section is arranged to emit light into a scene substantially above the ground and to receive reflections of the light radiated from the scene around the robot.

In some implementations, the imaging device includes a speckle emitter that emits a speckle pattern of light along a direction of travel of the robot to a scene, and an imager that receives reflections of the speckle pattern from an object in the scene. The controller stores comparison images of the speckle pattern as reflected from a comparison object in the scene. The comparison images are taken at different distances from the comparison object. The controller compares at least one target image of the speckle pattern, as reflected by a target object in the scene, with the comparison images to determine a distance of the reflective surfaces of the target object. In some cases, the controller determines a primary speckle pattern on the target object and calculates at least one of a cross-correlation or decorrelation between the primary speckle pattern and the speckle patterns of the comparison images.

To increase a lateral field of view, the imaging sensor may scan side-to-side relative to the forward direction of travel. Similarly, the imaging sensor can scan up and down to increase a vertical field of view.

In some implementations, the controller adjusts the use of received point cloud signals for the issuance of drive commands to the drive system after a limit period has been reached after receipt. The controller may override the setting of the use of received point cloud signals if, based on received detection signals from the dead zone sensor, the presence of an object in volume space not detectable by the formation sensor has been detected. Further, the controller may continue setting the use of received point cloud signals after the boundary period after the reception, for example, when it is determined based on detection signals received from the dead zone sensor that the volume of space that is not detectable by the imaging sensor is free of objects.

Another aspect of the disclosure provides a mobile robot including a base and a holonomic drive system carried by the base and defining a vertical axis (Z). The holonomic drive system maneuvers the robot over a work surface of a scene. The robot includes a controller in communication with the drive system, a leg that extends upward from the base, and a body that is supported by the leg. The body rotates relative to the base about the vertical axis. At least one imaging sensor (eg, a device for imaging volumetric point clouds) is disposed on the body and receives a volumetric point cloud (eg, three-dimensional images) of the scene around the robot. The rotating body moves the imaging sensor in a pivoting motion about the vertical axis, providing a 360 ° field of view around the robot.

In some embodiments, the at least one imaging sensor includes an imaging axis that is aimed downwardly along a forward direction of travel of the drive system. The at least one imaging sensor may include a first imaging sensor having an image axis arranged to aim downward along a forward direction of travel of the drive system and a second imaging sensor having an image axis disposed to be parallel to or above the working surface away from the body aim, involve. Further, the at least one imaging sensor may scan side-to-side relative to the forward travel direction to increase a lateral field of view and / or scan up and down to increase a vertical field of view of the imaging sensor.

The at least one imaging sensor may include a speckle emitter that emits a speckle pattern of light to a scene, and an imager that receives reflections of the speckle pattern from an object in the scene. The controller stores comparison images of the speckle pattern as reflected from a comparison object in the scene. The comparison images are taken at different distances from the comparison object. The controller compares at least one target image of the speckle pattern, as reflected by a target object in the scene, with the comparison images to determine a distance of the reflective surfaces of the target object. The imaging sensor can capture images of the scene along a direction of travel of the robot. The images include at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image.

The controller may determine a location of an object in the scene based on the image comparison, and issue drive commands to the drive system to maneuver the robot based on the object location in the scene. In some examples, the controller determines a primary speckle pattern on the target object and calculates at least one of a cross-correlation or decorrelation between the primary speckle pattern and the speckle patterns of the comparison images.

The imaging sensor may be a volumetric point cloud imaging device positioned at a height greater than 60.96 cm (2 feet) above the work surface and configured to obtain a point cloud from a volume of space that is a ground plane in a direction of travel of the robot. The imaging sensor may have a horizontal field of view of at least 45 degrees and a vertical field of view of at least 40 degrees and / or a range of between 1 meter and about 5 meters. In some examples, the imaging sensor has a latency of approximately 44 ms, and the image output of the imaging sensor may receive a timestamp that compensates for the latency.

In some embodiments, the robot includes a dead zone sensor having a detection area arranged to detect an object in a volume of space that is not detectable by the volumetric point cloud imaging device. The dead zone sensor may include at least one of a volumetric point cloud imaging device, a sonar sensor, a camera, an ultrasonic sensor, LIDAR, LADAR, an optical sensor, and an infrared sensor. The detection range of the dead zone sensor may surround a volume of space that is not detectable by the device for imaging volumetric point clouds (i.e., in a dead zone). In some examples, the volume of space that is not detectable by the volumetric point cloud imaging device is defined by a first angle, a second angle, and a radius (eg, 57 ° x 45 ° x 50 cm). The detection range of the dead band sensor may be arranged between the volumetric point cloud imaging device and a detection area of the volumetric point cloud imaging device. In some examples, the deadband sensor has a field of view that extends outward at least 3 meters from the dead zone sensor. In this example, the deadband sensor can be used for both a relatively short range within the dead zone and as a long range sensor for detecting objects that are relatively far away, to plan the path and avoid obstacles.

In some embodiments, the robot includes an array of dead zone sensors, wherein the detection area of at least one dead zone sensor is arranged to detect an object in the volume of space that is not detectable by the volumetric point cloud imaging device. The array of dead zone sensors may be arranged so that the fields of view are aligned along the forward direction of travel or evenly distributed over a vertical center axis defined by the robot.

In some implementations, the controller adjusts the use of received point cloud signals for the issuance of drive commands to the drive system after a limit period has been reached. The controller can do that Canceling the use of received point cloud signals when, based on received detection signals from the dead band sensor, the presence of an object in the volume of space that is not detectable by the formation sensor has been detected. Further, the controller may continue adjusting the use of received point cloud signals after the boundary period, for example, if it has been determined by the deadband sensor based on received detection signals that the volume of space that is not detectable by the imaging sensor is free of objects.

The body may rotate relative to the leg and / or the leg may rotate relative to the base about the vertical axis. In some examples, the height of the leg is variable.

In another aspect, a method for object detection for a mobile robot includes rotating an imaging sensor about a vertical axis of the robot. The imaging sensor emits light onto a scene around the robot and captures images of the scene. The images include at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image. The method further includes determining a location in the scene based on the images, assigning a confidence level for the object location, and maneuvering the robot in the scene based on the object location and the respective confidence level.

In some embodiments, the method includes creating an object-occupancy map of the scene. The confidence level of each object site may be downgraded over time until the respective object site is updated with a newly determined object location. The method may include maneuvering the robot to at least: a) touch the object and follow along a circumference of the object, or b) dodge the object.

In some examples, the method includes detecting an object in a volume of space that is not detectable by the imaging sensor, such as by using a deadband sensor having a detection range disposed to detect an object in the volume of space that is not detectable by the imaging sensor, and adjusting the downgrading of the confidence level of the detected object. The method may include continuing to downgrade the confidence level for the detected object if it has been determined that the volume of space that is not detectable by the imaging sensor is free of objects.

The method may include emitting light to the scene in intermittent pulses, optionally alternating a frequency of the emitted light pulses. The light pulses may be emitted in a first, energy-efficient frequency and upon receiving a sensor event in a second, active frequency. The sensor event may include a sensor signal indicating the presence of an object in the scene.

The method may include constructing the three-dimensional depth image of the scene by radiating a speckle pattern of light onto the scene, receiving reflections of the speckle pattern from the object in the scene, and storing comparison images of the speckle pattern as reflected from a comparison object in the scene. The comparison images are taken at different distances from the comparison object. The method further includes capturing at least one target image of the speckle pattern as reflected from a target object in the scene and comparing the at least one target image with the comparison images to determine a distance of the reflective surfaces of the target object. The method may include determining a primary speckle pattern on the target object and calculating at least one of a cross-correlation or decorrelation between the primary speckle pattern and the speckle patterns of the comparison images. Further, the method may include taking images of reflections of the emitted speckle pattern from surfaces of the target object at a frame rate, e.g. Between about 10 Hz and about 90 Hz, and optionally removing differences between speckle patterns taken in successive images to identify the target object.

The details of one or more of the embodiments of the disclosure are set forth below in the accompanying drawings and the description. Other aspects, features and advantages will become apparent from the description, the drawings and the claims.

DESCRIPTION OF THE DRAWINGS

Show it:

1 a perspective view of an exemplary mobile human interface robot.

2 a schematic view of an exemplary mobile human interface robot.

3 a side perspective view of an exemplary mobile human interface robot.

4A a front perspective view of an exemplary base for a mobile human interface robot.

4B a perspective rear view of the base 4A ,

4C a plan view of the base 4A ,

5A a schematic front view of an exemplary base for a mobile human interface robot.

5B a schematic plan view of an exemplary base for a mobile human interface robot.

6 a front perspective view of an exemplary body for a mobile human interface robot.

7 a front perspective view of an exemplary neck for a mobile human interface robot.

8A - 8G schematic views of an exemplary circuit for a mobile human interface robot.

9 a schematic view of an exemplary mobile human interface robot.

10A a perspective view of an exemplary mobile human interface robot with multiple sensors that are targeted to the ground.

10B a perspective view of an exemplary mobile robot with multiple sensors, which are aligned parallel to the ground.

11 a schematic view of an exemplary imaging sensor that detects an object in a scene.

12 a schematic view of an exemplary arrangement of actions for the operation of an imaging sensor.

13 a schematic view of an exemplary three-dimensional (3D) Specklekamera that detects an object in a scene.

14 a schematic view of an exemplary arrangement of actions for the operation of a 3D-Specklekamera.

15 a schematic view of an exemplary 3D runtime camera that detects an object in a scene.

16 a schematic view of an exemplary arrangement of actions for the operation of a 3D runtime camera.

17A a schematic view of an exemplary occupancy card.

17B a schematic view of a mobile robot with a field of view of a scene in a workspace.

18 a schematic view of a dead zone of an imaging sensor.

19 a perspective view of an exemplary mobile robot with a first imaging sensor, which is arranged to be directed downwards in a forward direction of travel, and a second imaging sensor, which is arranged to be aligned over the ground to the outside.

20 a plan view of an exemplary mobile robot with a body that rotates in relation to its base.

21 a schematic view of an exemplary imaging sensor with a dead zone and a dead zone sensor with a field of view that surrounds the dead zone.

22 a plan view of an exemplary mobile robot with a dead zone sensor, which is arranged to detect objects in a dead zone of an imaging sensor.

23 a plan view of an exemplary mobile robot with an array of dead zone sensors.

24 a plan view of an exemplary mobile robot with long-distance sensors, which are arranged about a vertical axis of the robot.

25 a schematic view of an exemplary control system that is executed by a controller of a mobile human interface robot.

26A an exemplary schematic view of the locally perceived space of a mobile human interface robot while this is stationary.

26B an exemplary schematic view of the locally perceived space of a mobile human interface robot while it moves.

26C an exemplary schematic view of the locally perceived space of a mobile human interface robot while this is stationary.

26D an exemplary schematic view of the locally perceived space of a mobile human interface robot while it moves.

Like reference symbols indicate like elements throughout the several drawings.

DETAILED DESCRIPTION

Mobile robots can interact with people or act as interfaces to provide a variety of services ranging from home-help to commercial and more. In the example of household tasks, a mobile robot can help elderly people in everyday tasks, such as adherence to a medication plan, mobility assistance, communication support (eg videoconferencing, telecommunications, internet access, etc.), monitoring of home or other locations (inside and outside) / or outside), person monitoring and / or providing a personal emergency system, but this listing is not exhaustive. In the commercial field, the mobile robot may provide videoconferencing functions (eg, in a hospital environment), point-of-sale point of contact, interactive information / marketing terminal, etc.

With reference to 1 - 2 includes a mobile robot 100 in some embodiments, a robot body 110 (or a housing) which defines a forward direction of travel F. The robot 100 also includes a drive system 200 , an interface module 300 and a sensor system 400 , each from the robot body 110 be worn and with a control 500 communicate the operation and movement of the robot 100 coordinated. A power source 105 (eg a battery or batteries) may be from the robot body 110 and be in electrical communication with each of these components as needed, and provide them with power. For example, the controller 500 include a computer that can provide> 1000 MAPS (millions of instructions per second) and the power source 1058 Provides a battery sufficient to power the computer for more than three hours.

The robot body 110 includes a base in the illustrated examples 120 , at least one leg 130 that from the base 120 goes up, and a body 140 that of the at least one leg 130 will be carried. The base 120 can at least parts of the drive system 200 wear. The robot body 110 also includes a neck 150 , of the body 140 will be carried. The neck 150 carries a head 160 , the at least one section of the interface module 300 wearing. The base 120 has a sufficient weight (eg by carrying the power source 105 (Batteries)) to provide a low center of gravity CG of the base _B 120 and a deep center of gravity CG _{R of} the robot 100 to maintain mechanical stability.

With reference to 3 and 4A - 4C defines the basis 120 in some embodiments, a three-sided symmetrical shape (eg a triangular shape from the top view). For example, the base 120 a base housing 122 include a base body 124 with a first, a second and a third base body portion 124a . 124b . 124c , each leg of the three-sided shaped base 120 (see eg 4A ). Each base body section 124a . 124b . 124c can move from the base housing 122 be worn in order to respond to contact with an object regardless of the base housing 122 to be able to move. The three-sided symmetrical shape of the base 120 allows bump detection 360 ° around the robot 100 , Each base body section 124a . 124b . 124c may include an associated contact sensor (eg, a capacitive sensor, read switch, etc.) that controls movement of the associated base body portion 124a . 124b . 124c in relation to the base housing 122 detected.

In some versions, the drive system provides 200 omnidirectional and / or holonomic motion control of the robot 100 ready. As used herein, the term "omnidirectional" refers to the ability to move substantially in any planar direction, ie, side-to-side, forward / backward, and rotational. These directions are referred to herein generally as x, y, and θz, respectively. Further, the term "holonom" is used to substantially correspond to the use of the term in the literature, and refers to the ability to move in a planar direction with three levels of freedom of movement, ie, two translations and one rotation , Thus, a holonomic robot has the ability to move in a planar direction at a speed consisting essentially of any proportion of the three plane velocities (forward / backward, lateral, and rotating), as well as the ability to substantially match those proportions continuous way to change.

The robot 100 can be operated using wheels for mobility purposes in human environments (e.g., environments typically designed for biped walking occupants). In some embodiments, the drive system includes 200 a first, second and third drive wheel 210a . 210b . 210c which are arranged at regular intervals (ie symmetrical on three sides) about the vertical axis Z (eg at a distance of 120 degrees); however, other arrangements are possible as well as a holonomic four-wheel drive system. With reference to 5A and 5B can drive the wheels 210a . 210b . 210c define a diagonal, arcuate rolling surface (ie, a curved profile in a direction diagonal or perpendicular to the rolling direction D _R ), indicating the maneuverability of the holonomic drive system 200 could improve. Each drive wheel 210a . 210b . 210c is with a respective drive motor 220a . 220b . 220c coupled to the drive wheel 210a . 210b . 210c independent of the other drive motors 220a . 220b . 220c can drive in forward and / or reverse directions. Every drive motor 220a -C can be a respective donor 212 exhibit ( 8C ), the wheel rotation feedback to the controller 500 provides. In some examples, each of the drive wheels 210a . 210b . 210c attached to or near one of the three points of an equilateral triangle and has a direction of travel (forward and backward directions), which is perpendicular to an angle bisector of each triangle end. Driving the three-sided symmetrical, holonomic base 120 in a forward direction F allows the robot 100 to change to non-forward directions to free yourself from a crowded situation or objects standing around, and then to rotate and / or move the drive after release along the forward direction F.

In the examples 3 - 5B is the first drive wheel 210a is arranged as a leading drive wheel along the forward direction F, with the remaining two drive wheels 210b . 210c follow. To drive forward, the controller can 500 in this arrangement, issue a drive command that includes the second and third drive wheels 210b . 210c caused to travel at a uniform speed in a forward rolling direction, while the first drive wheel 210a slips in the forward directions F. Furthermore, this drive wheel arrangement allows the robot 100 to stop suddenly (eg by a rapid negative acceleration against the forward direction F). This is due to the natural dynamic instability of the three-wheeled design. If the forward travel direction F were along an angle bisector between two forward drive wheels, the sudden stop would generate a torque that would cause the robot 100 would cause it to fall over, tipping over its two "front" wheels. Driving with a drive wheel 210a Front supports the robot 100 on the other hand and Prevents him from tipping over when he has to come to a quick stop. However, if he accelerates from standing, the controller can 500 an inertia moment I of the robot 100 from its overall CG _R focus.

In some versions of the drive system 200 has every drive wheel 210a . 210b . 210c a rolling direction D _R that is radial with a vertical axis Z perpendicular to the X and Y axes of the robot 100 runs, is aligned. The first drive wheel 210a may be arranged as a leading drive wheel along the forward direction of travel F, with the remaining two drive wheels 210b . 210c follow. To drive forward, the controller can 500 in this arrangement, issue a drive command that the first drive wheel 210a caused to move in a forward rolling direction and the second and third drive wheel 210b . 210c causes, with a same speed as the first drive wheel 210a but in the opposite direction to drive.

In other embodiments, the drive system 200 be arranged so that the first and second drive wheel 210a and 210b are positioned so that an angle bisector of an angle between the two drive wheels 210a . 210b with the forward direction F of the robot 100 is aligned. To drive forward, the controller can 500 in this arrangement, issue a drive command that includes the first and second drive wheels 210a . 210b caused to travel in a forward rolling direction and at a same speed, while the third drive wheel 210c in a reverse direction or remains idle and behind the first and second drive wheel 210a . 210b is dragged behind. To turn left or right while driving forward, the controller can 500 issue a command representing the corresponding first or second drive wheel 210a . 210b causes to drive at a faster / slower speed. Also other arrangements of the drive system 200 can be used. The drive wheels 210a . 210b . 210c can define a cylindrical, circular, elliptical or polygonal profile.

With renewed reference to 1 - 3 carries the base 120 at least one leg 130 that is in the Z direction from the base 120 goes up. The leg / legs 130 can be configured to have a variable height for raising and lowering the body 140 in relation to the base 120 exhibit. In some versions, each leg includes 130 a first and second leg portion 132 . 134 moving in relation to each other (eg telescopic, linear and / or angular motion). Instead of hollow bodies with smaller and smaller diameters, which move telescopically in and out of a relatively larger base hollow body, the second leg section moves 134 in the illustrated examples, telescopically over the first leg section 132 , causing other components along the second leg section 134 be placed and potentially with the second leg section 134 relatively close to the base 120 can move. The leg 130 can be a controller arrangement 136 ( 8C ) for moving the second leg section 134 in relation to the first leg section 132 include. The controller arrangement 136 can be a motor drive 138a that with a hoist motor 138b is in communication, and a giver 138c , the position feedback to the controller 500 provides include.

Generally, telescopic assemblies include hollow bodies of ever smaller diameters extending telescopically upwardly and out of relatively larger hollow bodies at the base 120 move out to a center of gravity CG of the entire leg _L 130 to be as deep as possible. Furthermore, stronger and / or larger components can be placed on the ground to cope with stronger torques at the base 120 occur when the leg 130 fully extended. However, this approach causes two problems. First, if the relatively smaller components on top of the leg 130 can be placed, rain, dust or other particles run down or fall into the hollow body and thus penetrate into a space between the hollow bodies and hinder the telescoping of the hollow body. This provides a very big sealing problem, if at the same time the complete mobility / mobility of the leg 130 should be maintained. Second, it may be desirable to have payloads or accessories on the robot 100 to fix. A typical location for attaching accessories is on top of the body 140 , When the second leg section 134 telescoped into and out of the first leg section, accessories and components may only extend above the entire second leg section 134 be attached when they are with the body 140 have to move. Otherwise, any components would be on the second leg section 134 attached, the telescopic movement of the leg 130 limit.

This causes the second leg section 134 telescopic over the first leg section 132 moves, puts the second leg section 134 additional payload attachment points that are in proportion to the base 120 can move vertically. This type of arrangement ensures that water or airborne particles on the outside of each leg section 132 . 134 (eg hollow body) the body 140 run down without getting into a space between the leg sections 132 . 134 penetrate. This will seal all the joints of the leg 130 considerably simplified. Further, payload / attachment attachment features of the body 140 and / or the second leg portion 134 always exposed and available, regardless of how the leg 130 is extended.

With reference to 3 and 6 carries / carry the leg / legs 130 the body 140 , the one flank 142 may be above and above the base 120 runs. In the example shown, the body 140 a downward or lower surface 144 (eg towards the base), which is at least part of the flank 142 forms, and an opposite, upward or upper surface 146 on, with a side surface 148 between them runs. The body 140 may define various shapes or geometries, such as a circular or elliptical shape with a central portion 141 , the leg / legs 130 is worn, and a free peripheral portion 143 which is lateral over a lateral extent of the leg (s) 130 extends and thus provides a projecting portion of the downwardly facing surface 144 Are defined. In some examples, the body defines 140 a polygonal or other complex shape that defines an edge that provides a protruding portion above the base 120 over the leg / legs 130 extends.

The robot 100 can have one accessory port or multiple accessory ports 170 (eg, mechanical and / or electrical connection points) for receiving payloads. The accessory connections 170 can be arranged so that received loads the sensors of the sensor system 400 (eg on the lower surface 144 and / or the upper surface 146 of the body 140 etc.) do not obscure or obstruct. In some versions, as in 6 shown, the body includes 140 on a rear section 149 of the body 140 one accessory port or multiple accessory ports 170 for example, a load in the basket 360 absorb and thus the sensors on a front section 147 of the body 140 or other sections of the robot body 110 not to obscure.

With renewed reference to 1 - 3 and 7 the body carries 140 the throat 150 , which is a pan and tilt of the head 160 in relation to the body 140 provides. In the illustrated examples, the neck includes 150 a turning device 152 and a tilt device 154 , The turning device 152 may provide an angular range of motion θ _R (eg, across the Z axis) between about 90 ° and about 360 °. Other areas are possible. Furthermore, the rotating device includes 152 in some examples electrical connections or contacts that require continuous rotation of the head 160 360 ° in relation to the body 140 allow in an unlimited number of rotations and at the same time the electrical communication between the head 160 and the rest of the robot 100 maintained. The tilting device 154 may include the same or similar electrical connections and contacts that cause rotation of the head 160 in relation to the body 140 allow and at the same time the electrical communication between the head 160 and the rest of the robot 100 maintained. The turning device 152 can a turntable motor 152m include that with a ring 153 (eg, an annular ring gear) is coupled or engages. The tilting device 154 can head independently of the turning device 152 in relation to the body 140 at an angle θ _T (eg, around the Y axis). In some examples, the tilting device includes 154 a tilting motor 155 that the head 160 at an angle θ _T of ± 90 ° relative to the Z axis moves. Other areas are possible, such as ± 45 °, etc. The robot 100 can be configured so that the leg / legs 130 , the body 140 , the neck 150 and the head 160 within a scope of the base 120 stay in place for a stable mobility of the robot 100 maintain. Im in 8F illustrated exemplary circuit diagram includes the neck 150 a pan / tilt arrangement 151 that is a turning device 152 and a tilt device 154 together with associated motor drives 156a . 156b and donors 158a . 158b includes.

8A - 8G represent exemplary diagrams of the circuit for the robot 100 ready. 8A - 8C Make example diagrams of the circuit for the base 120 which are the proximity sensors, such as the sonar proximity sensors 410 and the clip proximity sensors 420 , the contact sensors 430 , the laser scanner 440 , the sonar scanner 460 and the drive system 200 may contain. The base 120 can also control 500 , the power source 105 and the leg adjuster assembly 136 contain. The body 140 can be a microcontroller 145 the microphone (s) 330 , the speaker (s) 340 , an imaging sensor 450 (such as a 3D scanning imaging sensor 450a ) and a body touch sensor system 480 It's the controller 500 allows to receive and respond to user contact or touches (eg by moving the body 140 relative to the base 120 , Panning and / or tilting the neck 150 and / or issuing commands to the drive system 200 in response). The neck 150 can a pan / tilt arrangement 151 included a swivel motor 152 with an associated motor drive 156a and donors 158a and a tilt motor 154 with a associated motor drive 156b and donors 158b may include. The head 160 can one of several webpad (s) 310 (which may be, for example, a remote computing device that is with the robot 100 is in communication) and a camera 320 contain.

The webpad 310 may execute a software application (eg, a tablet-based user interface component / application) that allows a remote user to access an environment or scene 10 around the robot 100 represent and the robot 100 to remotely control. This software application can use hardware, such as an Apple iPad 2 and / or a Motorola Xoom, as a webpad 310 and a PrimeSensor camera (available from PrimeSense, 28 Habarzel St., 4th Floor, Tel-Aviv, 69710, Israel) as an imaging sensor 450 or use any other suitable hardware. The software application may use Apple iOS 4.x, Android 3.0 (also called Honeycomb), OpenGL ES 2.0, or any other suitable operating system or program.

To achieve reliable and stable autonomous motion, the sensor system can 400 with reference to 1 - 4C and 9 include various types of sensors that can be used in conjunction with one another to achieve a perception of the environment of the robot that is sufficient for the robot to do 100 make intelligent decisions about actions in this environment. The sensor system 400 may include one or more sensor types that are from the robot body 110 such as obstacle detection obstacle avoidance (ODOA) sensors, communication sensors, navigation sensors, etc. For example, these sensors may include proximity sensors, contact sensors, three-dimensional (3D) imaging / depth card sensors, a camera (eg, a camera for visible light and / or infrared light), sonar, radar, LIDAR (Light Detection And Ranging), light detection and measurement, which may involve remote optical detection, in which characteristics of scattered Light is measured to obtain range and / or other information about a distant target), LADAR (Laser Detection and Ranging), etc., but this enumeration is not exhaustive. In some embodiments, the sensor system includes 400 Sonar sensors 410 (For example, nine around a perimeter of the base 120 ), Clip proximity detectors 420 , Contact sensors 430 , a laser scanner 440 , one or more 3D imaging / depth sensor (s) 450 and an imaging sonar 450 ,

The placement of sensors on a robot platform poses several challenges. First, the sensors must be placed such that they provide maximum coverage of areas of interest around the robot 100 provide. Second, the sensors must be placed in such a way that the robot 100 the sensors themselves are only minimally blocked; In short, the sensors can not be placed so that they are hidden by the robot itself. Third, the placement and mounting of the sensors should not limit the rest of the industrial design of the platform. In terms of aesthetics, it can be assumed that a robot with inconspicuously attached sensors is "more attractive" than other models. As far as the use is concerned, sensors should be fixed in such a way that they do not disturb the normal operation of the robot (getting caught on obstacles, etc.).

In some embodiments, the sensor system includes 400 a set or array of proximity sensors 410 . 420 that with the controller 500 are in communication and in one or more zones or sections of the robot 100 are arranged (eg on or in the vicinity of the base body section (s)) 124a . 124b . 124c of the robot body 110 attached) to detect nearby or penetrating obstacles. The proximity sensors 410 . 420 For example, converging infrared (IR) emitter sensor elements, sonar sensors, ultrasonic sensors, and / or imaging sensors (eg, 3D depth map imaging sensors) that provide a signal to the controller 500 deploy when an object is within a given range of the robot 100 located.

In the examples 4A - 4C includes the robot 100 an array of sonar proximity sensors 410 which are (eg, at substantially uniform intervals) around the base body 120 and are arranged with an upward field of view. The first, second and third sonar proximity sensors 410a . 410b . 410c are on or near the first (forward) base body section (s) 124a arranged, where at least one of the sonar proximity sensors near a radially outermost edge 125a of the first base body 124a lies. The fourth, the fifth and the sixth sonar proximity sensor 410d . 410e . 410f are on or near the second (right) base body section (s) 124b arranged, wherein at least one of the sonar proximity sensors in the vicinity of a radially outermost edge 125b of the second base body 124b lies. The seventh, eighth, and ninth sonar proximity sensors 410g . 410h . 410i are on or near the third (right) base body section (s) 124c arranged, wherein at least one of the sonar proximity sensors in the vicinity of a radially outermost edge 125c of the third base body 124c lies. This configuration provides at least three coverage areas.

In some examples, the set of sonar proximity sensors is 410 (eg 410a - 410i ) around the base body 120 arranged so that it faces upwards (eg substantially in the Z-direction) and is optionally angled outwardly away from the Z-axis, creating a detection curtain 412 around the robot 100 is produced. Each sonar proximity sensor 410a - 410i can be a frame or a broadcasting guide 414 have the sonar emissions upwards or at least not in the direction of the other sections of the robot body 110 conducts (eg, no movement of the robot body 110 in relation to itself). The broadcasting guide 414 can define a shell or half shell shape. In the example shown, the base body is sufficient 120 lateral over the leg 130 out and the sonar proximity sensors 410 (eg 410a - 410i ) are on the base body 120 (eg, substantially along the circumference of the base body 120 ) around the leg 130 arranged. Further, the upward sonar proximity sensors are 410 spaced at a continuous or substantially continuous sonar detection curtain 412 around the Bern 130 to create. The sonar detection curtain 412 can be used to detect obstacles with increased lateral protruding sections, such as table tops, shelves, etc.

The upward sonar proximity sensors 410 provide the ability to see objects that are primarily in the horizontal plane, such as table surfaces. Due to their aspect ratio, these objects can be used by other sensors of the sensor system, such as the laser scanner 440 or the imaging sensors 450 be easily overlooked and thereby for the robot 100 a problem. The around the scope of the base 120 arranged, upward sonar proximity sensors 410 represent a way to see or capture these types of objects / obstacles. Furthermore, the sonar proximity sensors 410 placed around the widest points of the base perimeter and angled slightly outwards to get away from the body 140 or head 160 of the robot 100 not obstructed or obscured, resulting in false positives by capturing portions of the robot 100 occur yourself. In some embodiments, the sonar proximity sensors are 410 arranged (upwards and outwards) to create a volume around the body 140 outside a field of view of the sonar proximity sensors 410 so that it is free to receive fixed loads or accessories such as the basket 460 , The sonar proximity sensors 410 can in the base body 124 reset in order to achieve optical masking so that no external features are caught on or touch obstacles.

The sensor system 400 may be one or more sonar proximity sensor (s) directed to the rear (eg, opposite to the forward direction F) 410 (eg a rear proximity sensor 410j ) for detecting obstacles when reversing. The rear sonar proximity sensor 410j can be a broadcasting guide 414 involve his sonar detection area 12 align. Furthermore, the rear sonar proximity sensor 410j used for measurement, a distance between the robot 100 and a detected object in the field of view of the rear sonar proximity sensor 410j to be determined (eg as a reverse alarm). In some examples, the rear sonar proximity sensor is 410j reset in the base body 120 fixed so that it does not produce any optical or functional unevenness in the housing shape.

With reference to 3 and 4B includes the robot 100 in some embodiments clip proximity sensors 420 that are near or around the drive wheels (s) 210a . 210b . 210c are arranged to capture a cliff before the drive wheels 210a . 210b . 210c hit a cliff (eg stairs). For example, clip proximity sensors 420 at or near the radially outermost edges 125a -C the base body 124a -C and be located at locations in between. In some cases, cliff detection is accomplished using infrared (IR) proximity detection or actual distance detection using an infrared emitter 422 and an infrared detector 424 implemented, which are angled to one another to produce at a location where a ground is to be expected overlapping radiation and detection areas and therefore a detection zone. IR proximity detection may have a relatively narrow field of view, may depend on surface retroreflectivity in its reliability, and may have varying ranging accuracy from surface to surface. Therefore, several discrete sensors can be around the circumference of the robot 100 be placed to cliffs from different points on the robot 100 from being able to adequately capture. Furthermore, sensors based on IR approximation typically can not exist between a cliff and a safe event, for example after the robot 100 has crossed a threshold.

The clip approach sensors 420 can recognize when the robot 100 is hit on a sloping edge of the floor, as if he gets to a staircase. The control 500 (which executes a control system) can perform behaviors affecting the robot 100 cause an action, such as changing its direction of travel, when an edge has been detected. In some embodiments, the sensor system includes 400 one or more secondary clip sensor (s) (eg other sensors configured for clip detection and optionally for other forms of detection). The cliff detection proximity sensors 420 may be arranged to provide early detection of cliffs, provide data for discrimination between actual cliffs and safe events (such as threshold crossing), and be directed down and out so that their field of view is at least part of the robot body 110 and an area away from the robot body 110 includes. In some versions, the controller performs 500 a cliff detection routine that includes an edge of the supporting work surface (eg, the floor), an increase in the distance over the edge of the work surface, and / or an increase in the distance between the robot body 110 and the work surface identified and recorded. This implementation allows: 1.) early detection of potential cliffs (which can enable faster mobility speeds in unknown environments); 2.) increased reliability of self-mobility, as the controller 500 Cliff image information from cliff detection proximity sensors 420 receives to find out whether a cliff event is actually insecure or can safely be exceeded (eg by climbing up and crossing a threshold); 3.) a reduction in false positives from cliffs (eg due to the use of edge detection as opposed to several discrete IR proximity sensors with narrow fields of view). Additional sensors arranged as "Radabsenkung" can be used for redundancy purposes and to recognize situations in which a distance measurement camera can not reliably detect a certain type of cliff.

The threshold and step detection allows the robot 100 to effectively plan to either exceed an ascendable threshold or avoid a stage that is too high. The same can apply to random objects on the desktop that the robot 100 may or may not pass safely. For obstacles or thresholds for which the robot 100 Having determined that he can climb it, knowing the height enables the robot 100 brake adequately, if deemed necessary, to allow a smooth transition to maximize flatness and minimize any instability due to sudden acceleration. In some implementations, the threshold and step detection is based on an object height above the work surface as well as geometric detection (eg, distinguishing between a threshold or an electrical cable as opposed to a blob such as a sock). Thresholds can be detected by edge detection. The control 500 can capture imaging data from cliff detection proximity sensors 420 (or another imaging sensor on the robot 100 ), execute an edge detection routine, and output a drive command based on results of the edge detection routine. The control 500 can also use pattern recognition to identify objects. The threshold detection allows the robot 100 to change its orientation in relation to the threshold in order to enable a steady climbing of steps.

The proximity sensors 410 . 420 may work alone or alternatively for redundancy purposes in combination with one or more contact sensor (s) 430 (eg bump switches). For example, one or more contact or ground wave sensor (s) may 430 on the robot body 110 capture if the robot 100 physically encounters an obstacle. Such sensors can use a physical property, such as capacity or physical displacement in the robot 100 to determine when he hit an obstacle. In some embodiments, each base body section 124a . 124b . 124c the base 120 an associated contact sensor 430 include (eg, a capacitive sensor, read switch, etc.) that controls movement of the associated base body portion 124a . 124b . 124c in relation to the base housing 122 recognizes (see eg 4A ). For example, each base body may be 124a C radially relative to the Z axis of the base housing 122 move to provide 3-way wave detection.

With reference to 1 - 4C . 9 and 10A includes the sensor system 400 in some versions a laser scanner 440 standing on a front section of the robot body 110 is attached and with the controller 500 is in communication. In the examples shown, the laser scanner 440 on the base body 120 directed forwardly (eg with a field of view along the forward direction of travel F) on or above the first base body (s) 124a attached (eg to have a maximum image coverage along the direction of travel F of the robot). Further, the placement of the laser scanner on or near the front tip of the triangular base 120 in that the outer angle of the robot base (eg 300 degrees) is greater than a field of view 442 of the laser scanner 440 (eg, -285 degrees), which prevents the base 120 the detection field of view 442 of the laser scanner 440 locked or hidden. The laser scanner 440 can be reset as far as possible in the base body 124 be arranged without obstructing its field of view to any sections of the laser scanner that are above the base body 124 protrude, to minimize (eg for aesthetic reasons and to minimize sticking to obstacles).

The laser scanner 440 scans an area around the robot 100 and the controller 500 created using the laser scanner 440 received signals an environment map or object map of the scanned area. The control 500 can use the object map for navigation, obstacle detection and obstacle avoidance. Furthermore, the controller 500 Sensor inputs from other sensors of the sensor system 400 for creating the object map and / or for navigation.

In some examples, the laser scanner is 440 a scanning LIDAR that can use a laser that scans an area quickly in one dimension as a main scan line, and a runtime imaging element that uses a phase difference or similar technique to assign a depth to each pixel created in the line (which gives a two-dimensional depth line in the scan plane). To create a three-dimensional map, the LIDAR can perform an additional scan in a second direction (eg, by "nodding" the scanner). This mechanical scanning technique may use technologies such as "Flash" lidar / LADAR and "Swiss Ranger" focal plane imaging element sensors, techniques that use semiconductor stacks to allow run-time calculations for a complete 2D array of pixels to produce one pixel at each pixel Provide depth or even a range of depths for each pixel (with a coded illuminator or illumination laser) to complement or even support.

The sensor system 400 can use one or more three-dimensional (3D) image sensor (s) 450 include, the one with the controller 500 in communication stands / stands. When the 3D image sensor 450 has a limited field of view, the controller can 500 or the sensor system 400 the 3D image sensor 450a scan from side to side to provide a relatively wider field of view to provide robust ODOA.

With renewed reference to 2 and 4A - 4C can the sensor system 400 an inertial measurement unit (English: Inertial Measurement Unit, IMU) 470 involve with the controller 500 communicates to a moment of inertia of the robot 100 in relation to the overall center of gravity CG _{R of} the robot 100 to measure and monitor.

The control 500 can any deviation of the feedback from the IMU 470 from a threshold signal corresponding to normal, undisturbed operation. For example, when the robot tilts from an upright position, it may be stopped or otherwise restricted by a high obstacle (such as a clothesline) or someone has suddenly added a heavy payload. In these cases, it may be necessary to act quickly (including, but not limited to, evasive maneuvers, recalibration, and / or issuing an audible / visual alert) to ensure safe operation of the robot 100 to ensure.

Because the robot 100 working in a human environment, it can interact with people and operate in spaces designed for people (and regardless of the limitations of robots). The robot 100 can adjust its speeds and accelerations when in a busy, restricted, or highly dynamic environment, such as a cocktail party or a busy hospital. The robot 100 however, it can get into situations where it is safe to drive relatively fast, like in a long, empty hallway, but still able to brake quickly, for example when something crosses the robot's path of movement.

If he accelerates from standing, the controller can 500 an inertia moment of the robot 100 from its overall center of gravity, CG _R , to prevent the robot from tipping over. The control 500 can use a model of his pose, including his current moment of inertia. When carrying loads, the controller can 500 measure a load acting on the center of gravity CG and _R monitor the movement of the moment of inertia of the robot. For example, the body can / can 140 and / or the neck 150 Contain strain gauges to measure the load. If this is not possible, the controller can 500 a test torque command to the drive wheels 210 and IMU 470 measure actual linear and angular acceleration of the robot in order to determine safe limits through experiments.

During a sudden deceleration becomes a predetermined load on the second and third drive wheel 210b . 210c (the rear wheels) decreases while the first drive wheel 210a (the front wheel) slides in the forward direction and the robot 100 supports. When the load of the second and third drive wheel 210b . 210c (the rear wheels) is asymmetrical, the robot can 100 deviate from the course, which reduces the dynamic stability. The IMU 470 (eg, a gyroscope) may be used to detect this deviation and the second and third drive wheels 210b . 210c to instruct the robot 100 to realign.

With reference to 1 - 3 . 9 and 10A includes the robot 100 in some embodiments, a scanning 3D image sensor 450a standing on a front section of the robot body 110 is mounted with a field of view along the forward direction F (eg, to have a maximum imaging coverage along the direction of travel F of the robot). The scanning 3D image sensor 450a can primarily be used for obstacle detection / obstacle avoidance (ODOA). In the example shown, the scanning 3D image sensor is 450a on the body 140 under the flank 142 or on the soil surface 144 attached and in the body 140 embedded (eg flush or from the ground surface 144 ) reset, as in 3 For example, for user contact with the scanning 3D image sensor 450a to prevent. The scanning 3D image sensor 450 may be arranged to be substantially down and away from the robot body 110 to show it a downward field of view 452 in front of the robot 100 for obstruction detection and obstacle avoidance (ODOA) has (eg with obstruction by the base 120 or other sections of the robot body 110 ). The placement of the scanning 3D image sensor 450a on or near the front edge of the body 140 allows the field of view of the 3D image sensor 450 (eg, -285 degrees) to be less than an outside surface angle of the body 140 (eg 300 degrees) in relation to the 3D image sensor 450 is what causes the body 140 is prevented from viewing coverage area 452 of the scanning 3D image sensor 450a to obstruct or obscure. Furthermore, the scanning 3D image sensor 450a (and the associated actuator) in the body 140 be set back as far as possible without obstructing its field of view (eg also for aesthetic reasons and to minimize the risk of being caught by obstacles). The distracting scan motion of the scanning 3D image sensor 450a is not visible to a user, creating a less distracting interaction experience. Unlike a protruding sensor or feature, the recessed 3D scanning image sensor has 450a no unwanted interactions with the environment (getting stuck on people, obstacles, etc.), especially when moving or scanning, as there is virtually no moving part over the outer shell of the body 140 extends.

In some embodiments, the sensor system includes 400 additional 3D image sensors 450 on the base body 120 , the leg 130 , the neck 150 and / or the head 160 are arranged. In the example off 1 includes the robot 100 3D image sensors 450 on the base body 120 , the body 140 and the head 160 , In the example off 2 includes the robot 100 3D image sensors 450 on the base body 120 , the body 140 and the head 160 , In the example off 9 includes the robot 100 3D image sensors 450 on the leg 130 , the body 140 and the neck 150 , Other configurations are possible. A 3D image sensor 450 (eg on the neck 150 and above the head 160 ) can be used for person recognition, gesture recognition and / or videoconferencing while another 3D image sensor 450 (eg on the base 120 and / or the Bern 130 ) can be used for navigation and / or obstacle detection and obstacle avoidance.

One on the neck 150 and / or the head 160 arranged, forward looking 3D image sensor 450 Can be used for person, facial and / or gesture recognition of persons around the robot 100 be used. Using signal inputs from the 3D image sensor 450 on the head 160 can the controller 500 recognize a user by creating a three-dimensional map of the viewed user's face and comparing the generated three-dimensional map with known 3D images of faces of persons and determining a match with one of the known 3D face images. Face recognition can be used to confirm users as authorized users of the robot 100 be used. Further, one or more of the 3D image sensors may 450 used to gesture one in the field of view of the robot 100 and optionally respond based on the particular gesture (s) (eg, pointing by hand, waving, and / or hand signals). For example, the controller 500 output a drive command in response to a detected pointing by hand in a certain direction.

10B represents a schematic view of a robot 900 with a camera 910 , Sonar sensors 920 and a laser rangefinder 930 ready, all on a robot body 905 are fixed and a field of view, which is parallel or substantially parallel to the bottom G, have. In the example, a laser rangefinder detects 930 Objects near the bottom G, a ring of ultrasonic sensors (sonars) 920 detects objects that lie further above ground G and the camera 910 Captures a large portion of the scene from a high point of view. The key feature of this construction is that the sensors 910 . 920 . 930 all are aligned parallel to the ground G. An advantage of this arrangement is that the calculation can be simplified in that one using one or more of the sensors 910 . 920 . 930 the distance to an object is also the distance that the robot 900 can travel before he hits an object in a particular direction. A disadvantage of this arrangement is that many measurement levels are required to provide good coverage of the robot's environment to reach. This can be unfavorable from a cost-related and computational power point of view, which often causes large gaps in a sensor field of view of all these sensors 910 . 920 . 930 of the robot 900 entails.

In some embodiments, the robot includes a sonar scanner 460 for the acoustic imaging of an area containing the robot 100 surrounds. In the examples 1 and 3 is the sonar scanner 460 on a front section of the base body 120 arranged.

With reference to 1 . 3B and 10A the robot uses 100 in some versions the laser scanner or the laser rangefinder 440 for redundant measurements and a backward sonar proximity sensor 410j for safety, both aligned parallel to the ground G. The robot 100 Can first and second 3D image sensors 450a . 450b (Depth cameras) to reliably detect the environment around the robot 100 provide. The first 3D image sensor 450a is on the body 140 attached and directed at a fixed angle down to the ground G. By angling the first 3D image sensor 450a The robot is receiving down 100 a dense sensor cover in an area directly in front of or adjacent to the robot 100 , what a short-distance ride the robot 100 is relevant in the forward direction. The backward sonar 410j provides object detection when the robot is moving backwards. When reversing for the robot 100 typical, the robot can 100 a third 3D image sensor 450 which is directed downwards and backwards to provide a tight sensor coverage in an area directly behind or adjacent to the robot 100 provide.

The second 3D image sensor 450b is upside down 160 attached, over the neck 150 can be tilted or tilted. The second 3D image sensor 450b can be helpful for remote control because it allows a human operator to see where the robot is going 100 emotional. The neck 150 allows the operator to use the second 3D image sensor 450b to tilt and / or pan to see both nearby and distant objects. Panning the second 3D image sensor 450b increases an associated horizontal field of view. During faster rides, the robot can 100 the second 3D image sensor 450b tilt down slightly to see an entire or combined field of view of both 3D image sensors 450a . 450b to enlarge and the robot 100 give enough time to avoid an obstacle (since higher speeds generally mean there is less time to react to obstacles). At slower speeds, the robot can 100 the second 3D image sensor 450b upward or substantially parallel to the ground G tend to pursue a person who is the robot 100 should follow. Furthermore, the robot can 100 when driving at relatively slow speeds, the second 3D image sensor 450b Pan to his field of vision around the robot 100 to enlarge. The first 3D image sensor 450a can remain fixed (eg in relation to the base 120 not moved) when the robot is moving to extend the sensing range of the robot.

The 3D image sensors 450 may be capable of generating the following types of data: (i) a depth map, (ii) a reflectance-based intensity image, and / or (iii) a regular intensity image. The 3D image sensors 450 Such data may be obtained by pattern assignment, measuring the propagation time, and / or phase shifting for light emitted from a source and reflected by a target.

In some embodiments, reasoning or control software used on a processor (eg, the robot controller 500 ), a combination of algorithms using different through the sensor system 400 generated data types is executed. The reasoning software processes those from the sensor system 400 collected data and outputs data to make navigation decisions about where the robot is going 100 can move without, for example, colliding with an obstacle. By gathering imaging data of the environment of the robot over time, the reasoning software in turn can apply effective methods to selected segments of the recognized image (s) to obtain depth measurements of the 3D image sensors 450 to improve. This can include using appropriate temporary and spatial averaging techniques.

The reliability of performing collision-free movements of the robot may be based on: (i) a confidence level built up over time by high-level reasoning; and (ii) a depth-sensing sensor that collects three major types of data for analysis - ( a) a depth image, (b) an active illumination image and (c) an ambient illumination image. Algorithms with three different data types can be used on each of the deep-perception imaging sensor 450 obtained images are executed. The overall data can improve the confidence level compared to a system that uses only one data type.

The 3D image sensors 450 can capture images that capture depth and brightness data from a scene around the robot 100 (eg, a sensor view portion of a room or workspace) containing one or more objects. The control 500 may be configured to determine occupancy data for the object based on the captured reflected light from the scene. Further, the controller gives 500 in some examples, a drive command to the drive system based at least in part on the occupancy data 200 to avoid obstacles (ie the object in the scene). The 3D image sensors 450 can be used for real-time decision-making by the controller 500 repeatedly capture scene depth images to the robot 100 to navigate through the scene without colliding with objects in the scene. For example, the speed or frequency at which the depth image data from the 3D image sensors 450 be obtained by a shutter speed of the 3D image sensors 450 to be controlled. In addition, the controller 500 an event trigger (eg from another sensor component of the sensor system 400 like the proximity sensor 410 . 420 ) received the control 500 indicates a nearby object or danger. The control 500 can in response to the event trigger the 3D image sensors 450 to increase a frequency at which depth images are taken and occupancy information is obtained.

With reference to 11 includes the 3D imaging sensor 450 in some embodiments, a light source 1172 , the light on a scene 10 like the area around the robot 100 (eg a room). The imaging sensor 450 can also have an imager 1174 (eg an array of photosensitive pixels 1174p ) include that of the scene 10 reflected light, including reflected light from the light source 1172 goes out (eg as a scene depth image). In some examples, the imaging sensor includes 450 a light source lens 1176 and / or a detector lens 1178 for manipulating (e.g., speckling or focussing) the emitted or reflected light. The robot controller 500 or a sensor controller (not shown) associated with the robot controller 500 in communication receives light signals from the imager 1174 (eg the pixels 1174p ) based on image pattern assignment and / or a runtime property of the reflected, by the imager 1174 recorded light depth information for an object 12 in the scene 10 to determine.

12 represents an exemplary arrangement 1200 of actions for the operation of the imaging sensor 450 ready. With additional reference to 10A The actions involve the broadcasting 1202 of light on a scene 10 around the robot 100 and receiving 1204 reflections of the light emitted from the scene 10 to an imager (eg, an array of photosensitive pixels). The actions also include receiving 1206 of light detection signals from the imager by the controller 500 , the recognition 1208 one or more features of an object 12 in the scene 10 using image data derived from the light detection signals and tracking 1210 a position of the recognized feature (s) of the object 12 in the scene 10 using the image depth data derived from the light detection signals. The actions can be repeating 1212 the acts of broadcasting 1202 of light, of receiving 1204 of light reflections, of receiving 1206 of light detection signals, detection 1208 object feature (s) and tracking 1210 a position of the object feature (s) to increase a resolution of the image data or image depth data and / or to provide a confidence level.

The repeat act 1212 can be performed at a relatively slow rate (eg, a slower frame rate) for relatively high resolution, medium rate or high rate with a relatively high resolution. The frequency of the repeat action 1212 can from the robot controller 500 be adjusted. In some versions, the controller 500 the frequency of the repeat action 1212 raise or lower on receiving an event trigger. For example, a detected object in the scene may trigger an event that has an increased frequency of retry action 1212 causes a possibly imminent object 12 (eg a door, threshold or cliff) in the scene 10 to recognize. In additional examples, an elapsed time event may be between detected objects 12 cause the frequency of the repeat act 1212 slowed or suspended for a period of time (eg, go to sleep before being woken up by another event). In some examples, the capture action triggers 1208 a feature or multiple features of an object 12 in the scene 10 a feature detection event that has a relatively higher frequency of the retry action 1212 to raise the rate at which image depth data is obtained. A relatively higher acquisition rate of image depth data may allow relatively more reliable feature tracking in the scene.

The actions also include the output 1214 of navigation data for avoiding the object 12 in the scene 10 , In some designs, the controller uses 500 the issued Navigation data to drive commands to the drive system 200 spend to the robot 100 to move in a way that is a collision with the object 12 prevented.

In some versions, the sensor system captures 400 several objects 12 in the scene 10 around the robot 100 and the controller 500 keeps track of the positions of each of the detected objects 12 , The control 500 can be an assignment card of the objects 12 in an area around the robot 100 as the limited area of a room. The control 500 can the image depth data of the sensor system 400 use a scene 10 to compare with a section of the occupancy map and the occupancy map with the location of tracked objects 12 to update.

With reference to 13 includes the 3D imaging sensor 450 in some embodiments, a three-dimensional (3D) speckle camera 1300 which allows image mapping through speckle correlation. The speckle camera 1300 includes a speckle emitter 1310 (eg of infrared, ultraviolet and / or visible light), which adds a speckle pattern to the scene 10 (as a target area) and an imager 1320 taking pictures of the speckle pattern on the surfaces of an object 12 in the scene 10 receives.

The speckle emitter 1310 can be a light source 1312 , like a laser, involve a beam of light into a diffuser 1314 and on a reflector 1316 radiates so that this as a speckle pattern in the scene 10 reflected and thus projected. The imager 1320 can be objective optics 1322 Include the image on an image sensor 1324 focused, which is an array of light detectors 1326 , such as a CCD or CMOS based image sensor. Although the optical axes of the speckle emitter 1310 and the imager 1320 are shown as collinear, such as in a decorrelation mode, the optical axes of the speckle emitter 1310 and the imager 1320 also not collinear, for example when they are in a cross-correlation mode such that an imaging axis is shifted from an emission axis.

The speckle emitter 1310 a speckle pattern shines into the scene 10 off and the imager 1320 takes comparison images of the bacon pattern in the scene 10 in a range of different object distances Z _n from the speckle emitter 1310 (for example, where the z-axis passes through the optical axis of the imager 1320 can be defined). In the illustrated example, the comparison images of the projected speckle pattern are taken in a sequence of planes at different respective distances from the starting point, such as at the fiducial locations marked Z ₁ , Z ₂ , Z ₃ and so on. The distance ΔZ between the comparative images may be set at a limit distance (eg, 5 mm) or by the controller 500 be adjustable (eg in response to triggered events). The speckle camera 1300 archives and catalogs the recorded comparison images with respective emission distances to decorrelate the speckle pattern with distances from the speckle emitter 1310 to allow for distance measurements for objects 12 performed in successive pictures. Assuming that ΔZ roughly corresponds to the distance between adjacent fiducial distances Z ₁ , Z ₂ , Z ₃ , ..., the speckle pattern on the object 12 at location Z _A, for example, with the comparison image of the recorded at Z ₂ Specklemusters be correlated. On the other hand, the speckle pattern on the object 12 at Z _B, for example, to the comparison image at Z ₃ . These correlation measurements provide the approximate distance of the object 12 from the starting point. To the object 12 The Speckle camera can map in three dimensions 1300 or the controller 500 , the information from the speckle camera 1300 receives, using local cross-correlation with the comparison image that has shown the greatest match.

Other details and features of 3D image mapping based on speckle ranging, for example, by speckle cross correlation using triangulation or decorrelation, which are combinable with those described herein can be found in PCT patent application PCT / IL2006 / 000335 ; the contents of which are hereby incorporated by reference in their entirety.

14 represents an exemplary arrangement 1400 of actions for the operation of the Specklekamera 1300 ready. The actions involve broadcasting 1402 a bacon pattern in the scene 10 and recording 1404 of comparison images (eg of a comparison object 12 ) at different distances from the speckle emitter 1310 , The actions also include broadcasting 1406 a speckle pattern on a target object 12 in the scene 10 and recording 1408 of target images of the speckle pattern on the object 12 , The actions also include comparing 1410 the target images (of the stored object) with different comparison images to identify a comparison pattern that most with the speckle pattern on the target object 12 matches, and determining 1412 an estimated distance range of the target object 12 in the scene 10 , This may be determining a primary speckle pattern on the object 12 and finding a comparison image with a speckle pattern most common with the primary Bacon pattern on the object 12 matches. The distance range can be determined by the corresponding distance of the comparison image.

The actions optionally include the creation 1414 a 3D map of the surface of the object 12 by local cross-correlation between the speckle pattern on the object 12 and the identified comparison pattern, for example, a location of the object 12 to determine in the scene. This may be determining a primary speckle pattern on the object 12 and finding respective deviations between the primary speckle pattern on different regions of the object 12 in the target image and the primary speckle pattern in the identified comparison image to derive a three-dimensional (3D) map of the object. The use of solid state components for 3D mapping of a scene provides a relatively inexpensive solution for robotic navigation systems.

Typically, at least some of the various respective distances are axially separated by more than one axial length of the primary speckle pattern at the respective intervals. Comparing the target image with the comparison images may include calculating a respective cross-correlation between the target image and each of the at least some comparison images, and selecting the comparison image having the largest respective cross-correlation with the target image.

The actions can be repeating 1416 of actions 1406 - 1412 or actions 1406 - 1412 and optional action 1414 (eg, continuously) involve the movement of the object 12 in the scene 10 to pursue. For example, the speckle camera 1300 For comparison with the comparison images, record a sequence of target images while the object 12 emotional.

Other details and features of 3D image mapping based on speckle-ranging that can be combined with those described herein can be found in US Pat U.S. Patent 7,433,024 ; U.S. Patent Application No. 2008/0106746, entitled "Depth-varying light fields for three-dimensional sensing"; US Patent Application No. 2010/0118123, entitled "Depth Mapping Using Projected Patterns"; US Patent Application No. 2010/0034457 entitled "Modeling Of Humanoid Forms From Depth Maps"; US Patent Application No. 2010/0020078, entitled "Depth Mapping Using Multi-Beam Illumination"; U.S. Patent Application No. 2009/0185274, entitled "Optical Designs For Zero Order Reduction"; US Patent Application No. 2009/0096783, entitled "Three-Dimensional Sensing Using Speckle Patterns"; U.S. Patent Application 2008/0240502, entitled "Depth Mapping Using Projected Patterns"; and US Patent Application No. 2008/0106746, entitled "Depth-Varying Light Fields For Three Dimensional Sensing"; the contents of which are hereby incorporated by reference in their entirety.

With reference to 15 includes the 3D imaging sensor 450 in some versions, a 3D runtime camera 1500 for capturing depth image data. The 3D runtime camera 1500 includes a light source 1510 , Complementary Metal Oxide Semiconductor (CMOS) Complementary Metal Oxide Sensor 1520 (or Charge-Coupled Device (CCD)), a lens 1530 and a control logic or camera control 1540 with processing resources (and / or the robot controller 500 ), with the light source 1510 and the CMOS sensor 1520 communicate. The light source 1510 For example, a laser or a light emitting diode (LED) having an intensity modulated by a high frequency periodic signal may be used. In some examples, the light source includes 1510 a focusing lens 1512 , The CMOS sensor 1520 can be an array of pixel detectors 1522 or other arrangements of pixel detectors 1522 include, each pixel detector 1522 can capture the intensity and phase of photonic energy impinging upon it. In some examples, each pixel detector points 1522 a dedicated detector circuit 1524 for processing the sense charge output of the associated pixel detector 1522 on. The objective 1530 Focus on a scene 10 containing one or more object (s) 12 of interest, light reflected on the CMOS sensor 1520 , The camera control 1540 provides a sequence of action by the CMOS sensor 1520 obtained pixel data formatted into a depth map and a brightness image. In some examples, the 3D runtime camera includes 1500 also inputs / outputs (IO) 1550 (eg with the robot controller 500 in communication), a memory 1560 and / or a clock 1570 that with the camera control 1540 and / or the pixel detectors 1522 (eg the detector circuit 1524 ) is in communication.

16 represents an exemplary arrangement 1600 of action for the operation of the 3D runtime camera 1500 ready. The actions involve broadcasting 1602 a light pulse (eg of infrared, ultraviolet and / or visible light) into the scene 10 and measuring 1604 a duration of the light pulse (eg by counting the Uhrimpluse the clock 1570 ). The actions involve receiving 1606 reflections of the emitted light from one or more surfaces of an object 12 in the scene 10 , The reflections can be from surfaces of the object 12 be thrown back at different distances Z _n from the light source 1510 lie. The reflections are through the lens 1530 and on pixel detectors 1522 of the CMOS sensor 1520 receive. The actions involve receiving 1608 for each light pulse reflection on each associated pixel detector 1522 of the CMOS sensor 1520 Will be received. During the duration of the round trip of a light pulse, a counter adds the detector circuit 1523 each respective pixel detector 1522 Watches impulses. A larger number of added clock pulses represents a longer term and therefore a greater distance between a light reflection point on the object shown 12 and the light source 1510 , The actions further include determining 1610 a distance between the reflective surface of the object 12 for each received light pulse reflection and optionally constructing 1612 a three-dimensional object surface. In some embodiments, the actions involve repeating 1614 of actions 1602 - 1610 and optional 1612 to track the movement of the object 12 in the scene 10 ,

Other details and features of 3D runtime imaging that can be combined with those described herein can be found in US Pat U.S. Patent No. 6,323,942 entitled "CMOS Compatible 3-D Image Sensor"; U.S. Patent No. 6,515,740 entitled "Methods for CMOS-Compatible Three-Dimensional Image Sensing Using Quantum Efficiency Modulation"; and PCT Patent Application PCT / US02 / 16621 entitled "Method and System to Enhance Dynamic Range Conversion Usable with CMOS Three-Dimensional Imaging", the contents of which are hereby incorporated by reference in their entireties.

In some versions, the 3D imaging sensor provides 450 three types of information: (1) depth information (eg from each pixel detector 1522 of the CMOS sensor 1520 to a corresponding location on the scene 12 ); (2) ambient light intensity at each pixel detector location; and (3) the active illumination intensity at each pixel detector location. The depth information allows the position of the detected object 12 tracked over time, especially in relation to the proximity of the object to the location of the robotic insert. The active illumination intensity and ambient light intensity are different types of brightness images. The active illumination intensity is determined by reflections of an active light (as by the light source 1510 provided), that of the target object 12 is reflected, recorded. The ambient light image shows from the target object 12 reflected ambient light. The two images together provide additional robustness, especially when the lighting conditions are poor (eg, too dark or excessive ambient lighting).

Image segmentation and classification algorithms can be used to determine the position of objects 12 in the scene 10 to classify and record. Information provided by these algorithms and that provided by the imaging sensor 450 obtained distance measurement information can from the robot controller 500 or other processing resources. The imaging sensor 450 can work with the transit time principle, and in particular with discernible phase shifts in a modulated light pattern that is different from the scene 10 including techniques for modulating the sensitivity of photodiodes for filtering ambient light.

The robot 100 can be the imaging sensor 450 for 1) mapping, localization and navigation; 2) object detection and object avoidance (ODOA); 3) Object search (eg to find a person); 4) gesture recognition (eg for accompanying robots); 5) for human and face recognition; 6) persecution of people; 7) Monitoring and manipulation of objects by the robot 100 ; and other suitable applications for autonomous operation of the robot 100 use.

In some embodiments, at least one of the 3D image sensors 450 be a device for imaging volumetric point clouds (such as a speckle or runtime camera) that is more than about 30.5 cm to 61 cm (1 or 2 feet) above the ground on the robot 100 is positioned to a point cloud from a volume of space including a ground plane in a direction of movement of the robot (via the omnidirectional drive system 200 ) to obtain. In the examples 1 and 3 can be the first 3D image sensor 450a on the base 120 be positioned above the ground at a height of more than about 30.5 cm to 61 cm (1 or 2 feet) (or at a height of about 30.5 cm to 61 cm (about 1 or 2 feet) above the ground) and aligned along the forward travel direction F to capture images (eg, a volumetric point cloud) of a volume including the ground (eg, for obstacle detection and obstacle avoidance) during travel. The second 3D image sensor 450b is as upside down 160 (For example, at a height greater than about 91.5 or 122 cm (3 or 4 feet) above the ground) shown attached to a skeleton identifier and definition point clouds from one to the robot 100 to get adjacent room volume. The control 500 can perform skeleton / digital acquisition software to analyze data from recorded volumetric point clouds.

Proper recognition of objects 12 using an imaging sensor 450 despite ambient light conditions can be very important. In many environments, lighting conditions cover a wide range from direct sunlight to glaring neon light and subdued shadows, and can lead to large variations in the surface texture and fundamental reflectance of objects 12 to lead. The lighting can also be within a particular location and by scene 10 to scene 10 vary. In some embodiments, the imaging sensor may 450 used to be people and objects 12 in all situations with relatively little effect of ambient light conditions (eg ambient light suppression) to identify and determine.

In some embodiments, the VGA resolution of the imaging sensor is 450 640 horizontal by 480 vertical pixels; however, other resolutions are possible, such as 320 × 240 (eg for short-range sensors).

The imaging sensor 450 may include a pulse laser and a camera air, which in the time domain act as band filters to only objects 12 within a certain range. A variable iris of the imaging sensor 450 Can be used to objects 12 to recognize at different intervals. Further, for outdoor applications, a higher power pulse laser can be used.

Tabelle 1

Tabelle 2 Table 1 and Table 2 (below) provide exemplary features, parameters, and / or specifications of imaging sensors 450 ready for various applications. The sensor 1 can be used as an imaging sensor 450 be used for general purposes. The sensors 2 and 3 could be used in a human interaction robot, and the sensors 4 and 5 could be used on a cover or cleaning robot.

Table 1

Table 2

Minimal sensor latency ensures that objects 12 can be seen quickly enough to avoid them when the robot 100 emotional. The latency of the imaging sensor 450 may be a factor in real-time response to detected and recognized user gestures. In some examples, the imaging sensor has 450 a latency of about 44 ms. From the imaging sensor 450 captured images may have an assigned time stamp that may be used to determine in which robot pose an image was taken during translation or rotation in space.

An SPI (Serial Peripheral Interface, SPI) bus connected to the controller 500 Being in communication may be for communication with the imaging sensor 450 be used. The use of an SPI interface for the imaging sensor 450 does not restrict its use to multi-node distributed sensor / controller systems and allows connection to an Ethernet-enabled device, such as a field programmable gate array (FPGA) microprocessor that can expose data over the Ethernet and an EtherIO system, such as described in U.S. Patent Application No. 61 / 305,069, filed February 16, 2010, entitled "Mobile Robot Communication System", which is hereby incorporated by reference in its entirety.

Because SPI is a restricted protocol, an interrupt pin may be on the interface to the imaging sensor 450 be available, which scans or progresses when an image capture has been performed. The interrupt pin enables communication with the controller 500 when a picture was taken. As a result, the controller knows 500 that data is read-ready. In addition, the interrupt pin from the controller 500 used to record a timestamp indicating when the image was taken. The image output of the imaging sensor 450 may be timestamped (eg by a global clock of the controller 500 ), which can compensate for a latency. Furthermore, the time-stamped image output may be from multiple imaging sensors 450 (eg from different sections of the scene 10 ) are synchronized and combined (eg merged). Via an EtherIO system, an interrupt time (or interrupt pin) can be picked up and made available to higher level devices and software in the EtherIO system. The robot 100 may include a multi-node sensor / controller system that implements a clock synchronization strategy, such as IEEE1588 pointing at from the imaging sensor 450 recorded data can be applied.

Both the SPI interface and the EtherIO may be memory address driven interfaces. Data in the form of bytes / words / double words may be from the imaging sensor 450 for example, read via the SPI interface and made available in a memory space of the EtherIO system. For example, local directories and memory, such as random access memory (DMA) in an FPGA, can be used to control an EtherIO node of an EtherIO system.

In some cases, the robot needs 100 possibly the imaging sensor 450 scan from side to side and / or up and down (eg, around an object 12 to contemplate or to obstruct 16 to see ( 17A )). For a differently controlled robot 100 This may be the rotation of the robot 100 on the spot with the drive system 200 include; or rotating an imaging sensor 450 associated mirror, prism, variable angle micromirrors or MEMS mirror arrays.

The field of vision 452 of the imaging sensor 450 with an angle of view θv of less than 360 can be increased to 360 degrees by optics such as omnidirectional, fisheye, catadioptric (e.g., parabolic, telecentric), panamorphic mirrors, and objectives. Because the controller 500 the imaging sensor 450 inter alia for distance measurements, but not necessarily for human-visible images or videos (eg, for human communication), the distortion (eg warping) of the illumination of the light source is 1172 and / or image acquisition by the imager 1174 ( 11 ) are acceptable by the optics for the distance measurements (such as with the 3D speckle camera 1300 and / or the 3D runtime camera 1500 ).

In some cases, the imaging sensor can 450 Difficulty detecting and measuring black objects 12 , Surfaces with varying reflectivity, highly reflective objects 12 , extreme 3D structures, self-similar or periodic structures or objects in or just out of the field of view 452 (eg on or outside horizontal and vertical field of view angle). In such cases, other sensors of the sensor system 400 used to be the imaging sensor 450 to supplement or provide redundancy.

In some versions, the light source includes 1172 (eg the 3D Speckle camera 1300 and / or the 3D runtime camera 1500 ) an infrared (IR) laser, IR pattern illuminator or other IR illuminator. A black object, especially black fabric or carpet, can absorb IR and throw back a reflection that is insufficient to pass through the imager 1174 to be recognized. In this case, either a second mode of detection (such as a sonar) or a self-calibration technique may be required for differences in surface retroreflectivity to enhance black object detection.

A highly reflective object 12 or an object 12 with highly specular luminous dots (eg, cylindrical or spherical), determining distances for the imaging sensor 450 considerably more difficult. Likewise, objects can also 12 , the wavelengths of light after which the image sensor 450 seek, absorb to a high degree, pose problems. objects 12 Such as doors or windows made of glass can strongly reflect and, if their distance is to be determined, either appear as free space (infinite distance) or measured as the reflection of the first non-reflecting reflecting surface. That can lead to the robot 100 the object 12 does not see as an obstacle and for this reason collide with the window or the door and possibly damage the robot or the object 12 can cause. To prevent this, the controller can 500 run one or more algorithms that look for discontinuities in surfaces that are sized and shaped (square) correspond to a typical window or door. From this it can be concluded that these surfaces are obstacles or not free spaces. Another embodiment for detecting reflective objects in the path of the robot involves the use of a reflection sensor that detects its own reflection. After careful approach to the obstacle or object 12 For example, the reflection sensor may be used to determine whether there is a specular reflective object in advance or whether the robot can safely occupy the space.

In the case of the 3D-Speckle camera 1300 It may be that the light source 1310 is unable to form a pattern on the surface of a highly reflective object 12 is recognizable or that the imager 1320 unable to make a speckle reflection from the highly reflective object 12 to recognize. In the case of the 3D runtime camera 1500 can be the highly reflective object 12 create a multipath situation in which the 3D runtime camera 1500 a distance to another object 12 captured in the object 12 is reflected (instead of the object itself). To fix the IR failure modes, the sensor system can 400 acoustic runtime, millimeter-wave radar, stereo or other visual techniques that use even small reflections in the scene 10 can use.

network objects 12 can be the distance measurement for the imaging sensor 450 difficult. If there are no objects directly behind mesh with a certain porosity 12 The net material appears as a solid obstacle 12 , However, if there is an object 12 moved past the mesh material and, in the case of the 3D-Speckle camera 1300 , the speckle from the object 12 behind the mesh material, the object appears in the depth map instead of the mesh material, even though it is behind it. If information is available on the items previously attributed to the identification of the net material (before any object 12 moved past), such information could be used to register the location of the network material in future occupancy maps. By receiving information about the probabilistic correlation of the received speckle card at different distances, the controller can 500 the locations of several porous net-like objects 12 in one direction with the imaging sensor 450 determine.

The control 500 can capture data from the imaging sensor 450 for color / size / dimension blob analysis. The identification of discrete objects 12 in the scene 10 allows the robot 100 not only to avoid collisions, but also to objects 12 to search. The human interface robot 100 may need people and target objects 12 recognize against the backdrop of a home or office environment. The control 500 may include one or more color map blob analysis algorithms on the image data of the imaging sensor 450 Perform derived depth map (s) as if the maps were simple grayscale maps, and look for the same "color" (ie, depth continuity) to be persistent objects 12 in the scene 10 to find. The use of color charts to decide how objects 12 would also enhance object mapping, allowing for segmentation in color space as well as in depth space. The control 500 can objects 12 first through the depth and recognize the objects 12 then segment by color. This allows the robot 100 , between two objects 12 be distinguished, which are close to each other or adjacent to each other and have different optical qualities.

In versions in which the sensor system 400 just an imaging sensor 450 (eg a camera) for object detection, the imaging sensor 450 in the absence of scene texture, have problems in imaging surfaces and will not be able to resolve the size ratio of the scene. Furthermore, mirrors and / or specular luminous points of an object can 12 a saturation in a group of pixels 1174p of the imager 1174 (eg, saturation of an associated portion of a captured image); and in color images, the specular luminous dots may look different from different points of view, thereby falsifying the image comparison, as for the speckle camera 1300 ,

Using or assembling two or more sensors for object detection can provide a relatively more robust and redundant sensor system 400 provide. For example, although flash LADARs generally have a low dynamic range and rotating scanners generally have long inspection times, these sensor types may be useful for object detection. In some embodiments, the sensor system includes 400 a flash LADAR and / or a rotating scanner in addition to the imaging sensor 450 (eg the 3D-Speckle camera 1300 and / or the 3D runtime camera 1500 ), with the controller 500 is in communication. The control 500 can capture the detection signals from the imaging sensor 450 and use the flash LADAR and / or a rotating scanner to scan objects 12 to identify a distance from objects 12 from the robot 100 to determine a 3D map of the surfaces of the objects 12 to construct and / or to design a patch card or to update a patch card 1700 , The 3D-Speckle camera 1300 and / or the 3D runtime camera 1500 can be used to resolve any color or stereo camera growth points by initializing a distance determination, filling in areas of low texture, detecting depth discontinuities, and / or anchoring the scale.

In examples, the 3D-Specklekamera 1300 can use that from the speckle emitter 1310 radiated speckle pattern in relation to the imager 1320 be rotationally invariant. Furthermore, an additional camera 1300 (such as color or stereo camera), which together with the 3D Speckle camera 1300 and / or the 3D runtime camera 1500 is registered, employ a feature detector which is a partial or full scale / rotational invariant to handle self-rotation, tilt, perspective, and / or scale (distance). Scale invariant feature transform (or SIFT) is an algorithm for capturing and / or describing local features in images. SIFT can be from the controller 500 (with data from the sensor system 400 ) can be used for object recognition, robotic mapping and navigation, 3D modeling, gesture recognition, video tracking, and matching motion. SIFT, as a scale-invariant, rotation-invariant transformation, allows placement of a signature on features in the scene 10 and can help identify identified features in the scene 10 even if they are further away or rotated. For example, using SIFT on conventional images allows you to detect a moving object 12 (for example, a face or a button or a text) to identify that object 12 has the same luminance or the same color pattern, only larger or smaller or rotated. Other transformations can be used that are affine-invariant and can compensate for twist or distortion to objects 12 to identify from an angle. The sensor system 400 and / or the controller 500 can use scale-invariant feature recognition (eg by a color or stereo camera) using SIFT, RIFT, Affine SIFT, RIFT, G-RIF, SURF, PCA-SIFT, GLOH, PCA-SIFT, SIFT with FAST corner detection and / or or scalable vocabulary tree and / or SIFT with histogram classifications according to irregular orientations.

In some versions, the controller performs 500 a program or routine that uses SIFT and / or other object detection and / or identification transformations. The control 500 can take imaging data from an image sensor 450 , such as a color, black-and-white or IR camera. In some examples, the imaging sensor is 450 a 3D speckle IR camera that can provide image data without the speckle illumination to identify features without the benefit of speckle spacing. The control 500 can be features or objects 12 Identify or tag previously recorded in the Speckle distance determination in the 3D map scene. The depth map can be used to improve the SIFT recognition rate for camera-picked features and / or to simplify scale invariance (since both motion and range change are known and can be scaled-up). SIFT-like transformations may be useful in depth map data that is normalized and / or postponed for positional variation from image to image. Inertia tracking, odometry, proprioception and / or signal reference can be tracked. For example, a scale and rotational invariance transformation may still be effective to detect a localized feature in the depth map if the depth map is indexed by the amount of movement in the direction of the feature.

Other details and features of SIFT-like or other feature descriptors for 3D data that are combinable with those described herein can be found in FIG Se, S .; Lowe, David G .; Little, J. (2001). "Vision-based mobile robot localization and mapping using scale-invariant features". Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). 2. p. 2051 ff. ; or Rothganger, F; S. Lazebnik, C. Schmid and J. Ponce: 2004. 3D Object Modeling and Recognition Using Local Affine Invariant Image Descriptors and Multi-View Spatial Constraints, ICCV ; or Iryna Gordon and David G. Lowe, "What and where: 3D object recognition with accurate pose", Toward Category-Level Object Recognition, (Springer-Verlag, 2006), pp. 67-82 ; the contents of which are hereby incorporated by reference in their entirety.

Other details and features of techniques that are suitable for 3D-SIFT in human action detection, including traps, can be found in Laptev, Ivan and Lindeberg, Tony (2004). "Local descriptors for spatio-temporal recognition". ECCV'04 Workshop on Spatial Coherence for Visual Motion Analysis, Springer Lecture Notes in Computer Science, Issue 3667. pp. 91-103 ; Ivan Laptev, Barbara Caputo, Christian Schuldt and Tony Lindeberg (2007). Local velocity-adapted motion events for spatio-temporal recognition. Computer Vision and Image Understanding 108: 207-229 ; Scovanner, Paul; Ali, S; Shah, M (2007). "A 3-dimensional sift descriptor and its application to action recognition". Proceedings of the 15th International Conference on Multimedia. Pp. 357-360 ; Niebles, JC Wang, H. and Li, Fei-Fei (2006). "Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words ". Proceedings of the British Machine Vision Conference (BMVC). Edinburgh ; the contents of which are hereby incorporated by reference in their entirety.

The control 500 can be the imaging sensor 450 (eg, a depth map sensor) in constructing a 3D map of the surface of an object 12 use to fill holes of depth discontinuities and anchor a metric scale of a 3D model. Structure-out-of-motion, supported by depth map sensor survey data, can be used to estimate sensor poses. A typical structure-out-of-motion conduit may include viewpoint-invariant feature estimation, cross-camera feature comparisons, and grouping customization.

A software solution that features color / stereo cameras with the imaging sensor 450 (eg the 3D-Speckle camera 1300 and / or the runtime camera 1500 ) can include (1) sensor pose estimation, (2) depth map estimation and (3) 3D network estimation. The sensor pose estimation determines the position and orientation of the sensor package of each image capture. In depth map estimation, a high-resolution depth map is obtained for each image. In 3D network estimation, sensor pose estimates and depth maps can be used to identify objects of interest.

In some embodiments, a color or stereo camera 320 ( 9 ) and the 3D-Speckle camera 1300 or the 3D runtime camera 1500 be registered together. A projection distance of 1 meter and a field of view 452 of 45 degrees can provide sufficient switching time and overlap between the views. If at least two pixels are required for 50% detection, at least one 1 megapixel color camera with a 45 degree field of view lens can be used 452 with proportionally larger resolution for a field of view 452 of 60 degrees or wider.

Although a depth card sensor may have relatively low resolution and range accuracy, it can reliably allocate collections of pixels from the color / stereo image to a proper surface. This allows for reduction of stereo vision errors due to lack of texture and can also be achieved by limiting the range, e.g. B. to a distance of 5 cm, the Disparitätsuchbereich and reduce the computational costs.

With renewed reference to 10A can be the first and second 3D image sensor 450a . 450b used to improve the mapping of the environment of the robot to create a robot map because the first 3D image sensor 450a can be used to record nearby objects and the second 3D image sensor 450b can be used to record distant objects.

With reference to 17A and 17B the robot receives 100 in some circumstances an occupancy card 1700 the objects 12 in a scene 10 and / or a workspace 5 or the robot controller 500 generates the occupancy card 1700 based on the imaging sensor 450 (eg, the second 3D image sensor 450b ) image data and / or image depth data received over time (and may update it). In addition to the localization of the robot 100 in the scene 10 (eg the environment around the robot 100 ) the robot can 100 using the sensor system 400 to other points in a connected room (eg the workspace 5 ) drive. The robot 100 can be an imaging sensor 450a short-range (for example, on the bottom of the body 140 attached as shown in Figs. 1 and 3) to a nearby area around the robot 110 record and relatively nearby objects 12 and an imaging sensor 450b (eg on his head 160 is attached, as in 1 and 3 shown) with a long range to a relatively larger area around the robot 100 record and relatively distant objects 12 perceive. The robot 100 can the occupancy card 1700 use to familiar objects 12 in the scene 10 as well as lockups 16 to identify (for example, where an object 12 should or may not be, but this can not be confirmed from the current vantage point). The robot 100 can be an obstruction 16 or a new object 12 in the scene 10 register and try the obstruction 16 or the new object 12 to drive around to the location of the new object 12 or any object 12 in the lock 16 to confirm. Furthermore, the robot can 100 using the occupancy card 1700 the movement of an object 12 in the scene 10 determine and track. For example, the imaging sensor 450 . 450a . 450b a new position 12 ' of the object 12 in the scene 10 capture while holding a mapped position of the object 12 in the scene 10 not recorded. The robot 100 can change the position of the old object 12 as an obstruction 16 register and try the obstruction 16 to avoid the location of the object 12 to confirm. The robot 100 can capture new image depth data with previous image depth data (such as the map 1700 ) and a confidence level of the location of the object 12 in the scene 10 to assign. The site confidence level of an object 12 in the scene 10 may expire or expire after a cut-off period. The sensor system 400 can be location confidence levels of each object 12 after each imaging cycle of the sensor system 400 To update. In some examples, a detected new lock may be detected 16 (eg a missing object 12 in the occupancy card 1700 ) in a lock detection period (eg, less than ten seconds) a "living" object 12 (eg a moving object 12 ) in the scene 10 Show.

In some embodiments, a second, behind a captured first object 12a in the scene 10 located object 12b of interest at first not as an obstruction 16 in the scene 10 be recorded. An obstruction 16 can be an area in the scene 10 be that for the imaging sensor 450 . 450a . 450b not easy to capture or see. In the example shown, the sensor system has 400 (or, for example, a section of it, such as the imaging sensor 450 . 450a . 450b ) of the robot 100 a field of vision 452 with a viewing angle θ _V (which can be any angle between 0 degrees and 360 degrees) around the scene 10 to see. In some examples, the imaging sensor includes 450 omnidirectional optics for a viewing angle θ _V of 360 degrees; while the imaging sensor 450 . 450a . 450b in other examples, has a viewing angle θ _V of less than 360 degrees (eg, between about 45 degrees and 180 degrees). In examples where the viewing angle θ _{V is} less than 360 degrees, the imaging sensor may 450 . 450a . 450b (or components thereof) relative to the robot body 110 rotate to achieve a viewing angle θ _V of 360 degrees. The imaging sensor 450 . 450a . 450b may have a vertical angle of view θ _{VV, VH} θ resembles the horizontal view angle or different from it. For example, the imaging sensor 450 . 450a . 450b have a horizontal field of view θ _vH of at least 45 degrees and a vertical field of view θ _VV of at least 40 degrees. In some embodiments, the imaging sensor may be 450 . 450a . 450b or sections thereof relative to the robot body 110 and / or the drive system 200 move. To the second object 12b to capture, the robot can 100 also the imaging sensor 450 . 450a . 450b by driving in the scene 10 in one or more directions (eg, by translation and / or rotation on the work surface 5 ) to obtain a vantage point that captures the second object 10b [sic] allows. The robot movement or independent movement of the imaging sensor 450 . 450a . 450b or portions thereof may also solve monocular difficulties.

A confidence level can be detected locations or tracked movements of objects 12 in the workspace 5 be assigned to. For example, the controller 500 when creating or updating the occupancy card 1700 every object 12 on the map 1700 assign a confidence level. The confidence level can be directly proportional to a probability that the object is 12 actually in the workspace 5 is, as on the map 1700 displayed. The confidence level can be determined by a number of factors, such as the number and type of sensors used to make the object 12 capture. For example, the contact sensor 430 provide the highest confidence level because the contact sensor 430 actual contact of the robot 100 with the object 12 recognizes. The imaging sensor 450 may provide a different level of confidence that may be higher than that of the proximity sensor 430 , Of more than one sensor of the sensor system 400 Received data may be collected or collected to provide a higher confidence level compared to that of a single sensor.

Odometry is the use of data from the motion of operators to estimate a change in position over time (distance traveled). In some examples, an encoder is on the drive system 200 arranged to Radumdrehungen and therefore the robot 100 measure the distance traveled. The control 500 can use odometry when evaluating a confidence level for an object location. In some embodiments, the sensor system includes 400 an odometer and / or a rotation rate sensor (eg a gyroscope or an IMU 470 ) for measuring one from the robot 100 traveled distance. A gyroscope is a device for measuring or maintaining alignment based on the principles of torque preservation. The control 500 may use odometry and / or gyroscope signals received from the odometer and / or yaw rate sensor to determine a location of the robot 100 in a workspace 5 and / or on an occupancy card 1700 to determine. In some examples, the controller uses 500 Dead reckoning. Dead reckoning is the process of estimating a current position based on a predetermined position and advancing that position based on known or estimated speeds over time and course. If a robot site in the workspace 5 (eg, by odometry, gyroscope, etc.) as well as a recognized location of one or more objects 12 in the workspace 5 (via the sensor system 400 ), the controller can 500 a relatively higher confidence level of a location or movement of an object 12 on the occupancy card 1700 and in the workspace 5 evaluate (versus without the use of odometry or a gyroscope).

Wheel motion based odometry may include electrical noise. The control 500 can image data of the environment or scene 10 around the robot 100 from the imaging sensor 450 receive, regardless of wheel-based odometry of the drive system 200 to calculate robot movement by visual odometry. Visual odometry may involve the use of optical flow to control the motion of the imaging sensor 450 to determine. The control 500 may calculate the calculated movement based on imaging data of the imaging sensor 450 to correct any errors in radar-based odometry, allowing for improved mapping and motion control. Visual Odometry can be used in scenes 10 with low texture or low light intensity limitations are imposed when using the imaging sensor 450 Features in / in the captured image (s) can not track.

Other details and features of odometry and imaging systems that may be combined with those described herein can be found in: U.S. Patent 7,158,317 (which describes a "field depth" imaging system) and U.S. Patent 7,115,849 (which describes wavefront-encoding interference contrast imaging systems), the contents of which are incorporated herein by reference in their entireties.

With reference to 18 has the imaging sensor 450 in some embodiments, an imaging deadband 453 on, which is a volume of space around the imaging sensor 450 where objects are not captured. In some examples, the imaging deadband includes 453 a volume of space defined by a first angle α, a second angle β and a radius R _S of approximately 57 ° × 45 ° × 50 cm, directly to the imaging sensor 450 adjoins and around an imaging axis 455 is centered. The dead zone 453 is between the imaging sensor 450 and a detection area 457 of the imaging sensor 450 within the field of view 452 positioned.

In the example off 19 includes the robot 100 a first and a second imaging sensor 450a . 450b (eg, 3D depth imaging sensors) on the body 140 are arranged. Both imaging sensors 450a . 450b are arranged so that their field of vision 452 along the forward direction F is aligned. The first imaging sensor 450a is arranged to its imaging axis 455 essentially down and away from the robot 100 to aim (eg to be able to see an area on the ground and / or to view a lower portion of the robot) to capture objects before they reach the base 120 or the leg 130 touch. By angling the first 3D imaging sensor 450a down the robot receives 100 a dense sensor cover in an area directly in front of or adjacent to the robot 100 , what a short-distance ride the robot 100 is relevant in the forward direction. The second imaging sensor 450b is with his imaging axis 455 arranged substantially parallel to the ground along the forward travel direction F (eg, to detect objects placed on a middle and / or upper portion of the robot 100 get). In other examples, the second imaging sensor is 450b with his imaging axis 455 arranged above the ground or even pointing up away from the ground.

If the imaging sensors 450a . 450b dead zones 453 it may happen that an object is near or adjacent to the robot (s) 100 can not be detected. In the example off 10A includes the robot 100 an imaging sensor 450 on his head 160 is arranged and over the neck 150 can be tilted and tilted. This allows the robot 100 the imaging sensor 450 move on the head to the dead zones 453 the other imaging sensors 450a . 450b to see what complete or essentially complete fields of view 452 around the robot 100 be provided for object detection. If it is not possible, an imaging sensor 450 on the head 160 to place or if an imaging sensor 450 can not be moved so that he has the dead zones 453 Other techniques can be used to remove the dead zones 453 to see. In addition to the dead zones 453 can do some objects within the field of view 452 of the imaging sensor 450 difficult to detect due to their size, shape, reflectance and / or color. For example, sometimes highly reflective or specular objects are difficult to detect. In other examples, very dark or black objects may be difficult to detect. Furthermore, slender objects (ie having a very thin profile) may be difficult to detect. Hard-to-detect objects can be more easily detected when viewed from multiple angles or detected by multiple sensors.

In the examples 1 . 4C and 10A the robot includes one or more sonar proximity sensor (s) 410 (eg 410a - 410i ) the / so around the base body 120 is / are arranged to face upwards (eg, substantially in the Z direction) and optionally angled outward away from the Z axis, thereby creating a detection curtain 412 around the robot 100 is produced. The sonar proximity sensors 410 can be arranged and aligned so that they are objects within the dead zone 453 every imaging sensor 450 detect.

In some versions, the robot moves 100 (via the controller 500 or the sensor system 400 ) the imaging sensors 450 . 450a . 450b or panning them to the visibility of the corresponding dead zones 453 to increase. An imaging sensor 450 can by moving its associated imaging axis 455 be directed in any direction 360 ° (+/- 180 °).

In some examples, the robot maneuvers 100 even on the ground, around the imaging axis 455 and the associated field of view 452 every imaging sensor 450 to move in order to get a perception of the volume that was previously in a dead zone 453 was. For example, the robot may 100 pivot in place, move holonom laterally, move forwards or backwards, or perform a combination of them. When the imaging sensor 450 a restricted field of view 452 and / or a limited detection area 457 can, the controller can 500 or the sensor system 400 in additional examples, the imaging sensor 450 Press to scan from side to side and / or up and down to create a relatively wider and / or higher field of view to perform robust ODOA. Panning the imaging sensor 450 (by moving the imaging axis 455 ) increases an associated horizontal and / or vertical field of view, which is the imaging sensor 450 allows not just the entire or a section of its deadband 453 to see, but also the dead zone 453 another imaging sensor 450 on the robot 100 ,

In some examples, any imaging sensor may 450 an associated actuator (not shown) containing the imaging sensor 450 moved in the scanning motion. In additional examples, the imaging sensor includes 450 an associated rotating mirror, a prism, a variable angle micromirror, or a MEMS mirror array around the field of view 452 and / or the coverage area 457 of the imaging sensor 450 to enlarge.

In the example off 20 the body pivots 140 on the leg 130 around the z-axis, to the robot 100 to allow one on the body 140 arranged imaging sensor 450 relative to that by the base 120 defined forward direction to move F. In some examples, the leg pivots 130 around the Z-axis, moving the body 140 around the Z axis. In both examples a steller rotates 138 (like a swivel unit) with the controller 500 is in communication, the body 140 in relation to the base 120 (eg by rotating the body 140 in relation to the leg 130 and / or rotating the leg 130 relative to the base 120 ). The rotating body 140 moves the imaging sensor 450 in a pivotal movement about the Z-axis and thus provides a field of view 452 up to 360 ° around the robot 100 ready. The robot 100 can the body 140 in a continuous 360 ° degree or +/- an angle ≤ of 180 ° in relation to the forward direction F pivot.

With reference to 21 includes the robot 100 in some embodiments, a dead zone sensor 490 that with every imaging sensor 450 connected and arranged so that he objects in the dead zone 453 of the associated imaging sensor 450 recognizes. The dead zone sensor 490 may be a sonar sensor, a camera, an ultrasonic sensor, LIDAR, LADAR, an optical sensor, an infrared sensor, etc. In the example shown, the deadband sensor is 490 arranged so that his field of vision 492 the dead zone 453 surrounds or substantially surrounds. 22 represents a top view of a robot 100 with a dead zone sensor 490 on the body 140 adjacent to the imaging sensor 450 and with his field of vision 492 in the dead zone 453 is arranged in sufficient, ready. In the example shown, the deadzone field of view is 492 essentially in the dead zone 453 centered; however, other arrangements (eg, not centered) are also possible.

23 shows an exemplary robot 100 with an array of deadband sensors 490 on a front section 147 of the body 140 is arranged. The array of deadband sensors 490 not only provides coverage of the deadband shown 453 ready but also additional areas around the robot 100 that were not in the field of view of a sensor before (eg, the areas on each side of the field of view 452 of the imaging sensor 450 ). That allows the robot 100 to recognize nearby objects before moving or rotating in them.

In the example off 24 includes the robot 100 at least one long distance sensor 2190 which is arranged and configured to an object 12 that is relatively far from the robot 100 away (eg> 3 meters). The long-distance sensor 2190 can be an imaging sensor 450 (eg, with optics or a zoom lens configured to capture at relatively long distances). In additional examples, the long-distance sensor is 2190 a camera (eg, with a zoom lens), a laser range finder, LIDAR, RADAR, etc. In the example shown, the robot includes 100 four Long-range sensors 2190 with associated fields of view 2192 along the forward, backward, right and left driving directions are arranged. Other arrangements are also possible.

The detection of distant objects enables the robot 100 (via the controller 500 ) Execute navigation routines to avoid the object when viewed as an obstacle or to approach the object when it is viewed as a destination (eg, approaching a person to perform a videoconference session). The perception of objects outside the field of view of the imaging sensor (s) 450 on the robot 100 allows the controller 500 To avoid movements affecting the captured object 12 in a dead zone 453 could place. When a person is out of the field of view of an imaging sensor in person tracking routines 450 moved out, the long-haul sensor can 2190 furthermore, capture the person and give it to the robot 100 allow maneuvering so that he is the person in the field of vision 452 of the imaging sensor 450 can recognize again.

With reference to 25 leads the controller 500 in some versions a control system 510 This is a control arbitration system 510a and a behavioral system 510b involves communicating with each other. The control arbitration system 510a allows dynamic addition and removal of applications 520 to / from the control system 510 and makes it easier to use 520 , the robot 100 to steer, without losing other applications 520 to know. In other words, the control arbitration system 510a provides a simple, prioritized control mechanism between applications 520 and resources 530 of the robot 100 ready. The resources 530 can the drive system 200 , the sensor system 400 and / or any payloads or controllable devices associated with the controller 500 in communication.

The applications 520 can be stored in memory or sent to the robot 100 be communicated to be executed simultaneously (eg on a processor) and the robot 100 to control. The applications 520 can on behavior 600 of the behavioral system 510b access. The independently used applications 520 are dynamically combined during operation to robot resources 530 (eg drive system 200 , Arm (s), head / heads, etc.) of the robot 100 to share. A low level strategy is implemented to control the robot resources 530 during operation dynamically among the applications 520 divide. The strategy determines which application 520 the robot resources 530 controls that for the application 520 required (for example, a hierarchy of priorities among the applications 520 ). The applications 520 can start and end dynamically and run completely independently of each other. The control system 510 also allows complex behavior 600 that can be combined with each other to support each other.

The control arbitration system 510a includes one or more resource controls 540 , a robot manager 550 and one or more control arbiters 560 , These components do not have to be in a conventional process or computer and do not have to be started in a specific order. The resource control component 540 provides an interface with the control arbitration system 510a for applications 520 ready. For every application 520 is there an instance of this component. The resource control 540 abstract and hide the complexities of authentication, distributed resource control arbiter, command buffering, and the like. The robot manager 550 coordinates the prioritization of applications 520 by controlling which application 520 exclusive control over any of the robotic resources at any given time 530 Has. Since this is the central information coordinator, there is only one instance of the robot manager per robot 550 , The robot manager 550 implements a priority strategy, which is a linear prioritized order of resource controls 540 and the resource control arbiter 560 , which provide hardware control, tracked. The control arbiter 560 receives the commands from each application 520 and generates a single command based on the priorities of the applications and publishes them for the associated resources 530 , The control arbiter 560 also receives state feedback from the associated resources 530 and send them back to the applications 520 , The robot resources 530 may be a network of functional modules (eg, actuators, drive systems and groups thereof) with one or more hardware controllers. The commands of the control arbiter 560 are specific to the resource 530 to perform specific actions.

A dynamic model 570 that on the controller 500 can be configured to the center of gravity (CG), moments of inertia and cross products of inertia of different sections of the robot 100 to calculate to assess a current robot state. The dynamics model 570 In addition, it can simulate the shapes, the weight and / or the moments of inertia of these components. In some examples, the dynamics model communicates 570 with an inertia moment unit 470 (English: inertial moment unit, IMU) or sections thereof (eg, accelerometers and / or gyroscopes) placed on the robot 100 is arranged and with the controller 500 is in communication with the different focuses of the robot 100 to calculate. The dynamics model 570 can from the controller 500 in addition to other programs 520 or behavior 600 used to limit operating limits of the robot 100 and its components.

Every application 520 has an action selection engine 580 and a resource control 540 , one or more behavior 600 using the action selection engine 580 is / are connected, and one or more action model (s) 590 using the action selection engine 580 is connected / are. The behavior system 510b Provides predictive modeling and allows behavior 600 to jointly decide on the actions of the robot by evaluating possible results of robotic actions. In some examples is a behavior 600 a plug-in component that provides a hierarchical, full score function that combines sensor feedback from multiple sources with a priori limits and information into feedback feedback on the allowed actions of the robot. Because the behavior 600 in the application 520 "Plugged in" (eg inside or outside the application 520 They can be removed or added without the application 520 or any other part of the control system 510 must / must be modified. Every behavior 600 is an independent strategy. To behavior 600 To make it more powerful, it is possible to output different behavior 600 combine together to enter another, allowing for complex combination functions. The behavior 600 serve to controllable sections of the overall perception of the robot 100 to implement.

The action selection engine 580 is the coordinating element of the control system 510 and performs a fast, optimized action selection cycle (prediction / correction cycle), given the input of all behavior 600 looking for the best act. The action selection engine 580 has three phases: nomination, action selection search and completion. In the nomination phase is every behavior 600 notifies that the action selection cycle has started, and informs about the cycle start time, the cycle state, and the limits of the robot controller space. Every behavior is decided on the basis of internal guidelines or external input 600 whether it wants to participate in this action selection cycle or not. During this phase, a list of active behavioral primitives is generated, the input of which selects on the robot 100 will affect commands to be executed.

In the action selection search phase, the action selection engine generates 580 possible results from the range of available actions, also known as action area. The action selection engine 580 uses the action models 590 to provide a set of possible commands (within constraints) and corresponding results as a result of action simulation of each command at different time intervals with a time horizon in the future. The action selection engine 580 calculated based on the result scores of the behavior 600 a preferred result and sends the appropriate command to the control arbitration system 510a and notifies the action model 590 as feedback about the selected command.

In the completion phase, the commands that correspond to a collaborative best scoring result are combined into a total command that executes on the robot resources 530 to the resource control 540 provided. The best result is as feedback to the active behavior 600 provided to be used in future evaluation cycles.

From the sensor system 400 received sensor signals may interact with one or more behaviors 600 cause to perform actions. For example, the controller selects 500 using the control system 510 For example, an action (or move command) for each robotic component (eg, the motor or actuator) from a corresponding action area (eg, a collection of possible actions or moves for that particular component) in an efficient manner Collisions with yourself or any other objects around the robot 100 of which the robot 100 Has knowledge, avoids to trigger a coordinated movement of each robot component. The control 500 can issue a coordinated command via the robotic network, such as the EtherIO network.

The control system 510 can adaptive speed / acceleration of the drive system 200 provide (for example, one or more behaviors 600 ) to the stability of the robot 100 in different configurations / maximize positions while the robot 100 maneuvered through an area.

In some versions, the controller gives 500 Commands to the drive system 200 out the robot 100 according to a course setting and speed setting. One or more behavior 600 can / can from the sensor system 400 use received signals to evaluate predicted results from possible commands, one of which may be selected for execution (singly or in combination with other commands as a total robotic command) to respond to obstacles. For example, signals from the proximity sensors 410 the control system 510 induce the commanded speed or the course of the robot 100 to change. For example, a signal triggered by a nearby wall may be from the proximity sensor 410 cause the control system 510 Issues a command to decelerate. In another case, a collision signal from / from the contact sensor (s) triggered by an encounter with a chair may affect the control system 510 cause to issue a command to change the course. In other cases, it may happen that the speed setting of the robot 100 is not reduced in response to the contact sensor; and / or that the course setting of the robot 100 in response to the proximity sensor 410 not changed.

The behavior system 510b can be a mapping behavior 600a to create an occupancy card 1700 , an object detection / obstacle avoidance (ODOA) behavior 600b , a speed behavior 600c (eg, a processor executable behavior routine) configured to adjust the speed setting of the robot 100 adapt, and a price behavior 600d , which is configured to the course setting of the robot 100 to change. The speed and price behavior 600c . 600d can be configured to run concurrently and independently. For example, the speed behavior 600c be configured to detect one of the sensors (eg, the set / sets of proximity sensors 410 . 420 ) and price behavior 600d may be configured to interrogate another sensor (eg, the kinetic ground wave sensor).

With reference to 25 and 26A - 26D can the robot 100 in some embodiments, on his ability to perceive his locally perceived space 2100 (ie the space around the robot 100 as he is from the sensor system 400 is detected), and execute an object detection and object avoidance (ODOA) strategy to navigate to a destination location. The sensor system 400 can capture sensor data including three-dimensional depth image data from a device 450 for imaging volumetric point clouds on the robot 100 is positioned to make a point cloud from one to the robot 100 provide adjacent room volume. For example, the device may 450 for imaging volumetric point clouds at a height greater than 2 feet above the ground on the robot 100 be positioned and aligned so that it can detect a point cloud of a volume of space, which is a ground plane G in a direction of movement of the robot 100 includes.

The robot 100 (eg the control system 510 out 25 ) can its locally perceived space 2100 classify into three categories: obstacles (black) 2102 , unknown (gray) 2104 and known free (white) 2106 , obstacles 2102 are observed (for example, detected) points above the ground G, which are below a height of the robot 100 lie and observed points below the ground G (eg holes, steps down, etc.). Free known 2106 corresponds to areas where the 3D sensors 450 can see the ground G. Data from some or all sensors in the sensor system 400 can be combined into a discretized 3D voxel mesh. The 3D grid can then be analyzed and converted into a 2D grid 2101 be converted with three local perceptible space classifications. 26A shows an exemplary schematic view of the locally perceived space 2100 of the robot 100 while this is stationary. The information in the 3D voxel grid is persistent but decays over time if not amplified (eg, by fresh sensor data). When the robot 100 moved, it has a larger known free area due to the resistance 2106 in which he can navigate. The volumetric point cloud data of the 3D imaging sensor 450 can expire after a threshold period, such as milliseconds to seconds, so that volatile or slightly older objects in the environment (eg, walking people, sensor artifacts, etc.) are not used for local path planning.

If the 3D imaging sensor 450 a dead zone 453 ( 20 - 24 ) and the robot 100 next to a non-volatile object 12 maneuvered, which is completely in the dead zone 453 located ( 26A ), it can be the control system 510 allow that non-volatile object 12 associated sensor data expire and the object 12 therefore no longer as an obstacle for navigation purposes 2102 is recognized. Although other sensors of the sensor system 400 how the dead zone sensor (s) 490 to be able to make the object 12 To recognize, the control system can 510 Run an ODOA strategy that checks the data flow for this object 12 in the dead zone 453 suspended sensor data.

For example, the control system 510 the data flow for sensor data, which normally takes place after a boundary period and with obstacles 2102 (eg objects 12 ) in the locally perceptible space 2100 are connected, suspend when the obstacle 2102 when in the dead zone 453 or an area next to the robot 100 is detected. The control system 510 can be the presence of an object 12 that the obstacle 2102 in the dead zone 453 using one or more dead band sensors 490 or another sensor (s) of the sensor system 400 as close sensors determine. The control system 510 can allow that with this object 12 Connected sensor data only expire again or expire after the robot 100 moved away from this location and / or the dead zone sensor (s) 490 recognize / recognize that the object 12 from the dead zone 453 has moved. This allows the control system 510 Perform object detection / obstacle avoidance (ODOA) navigation strategies that reduce the possibility of an obstruction in the dead zone 453 of the robot 100 consider.

An object detection / obstacle avoidance (ODOA) navigation strategy for the control system 510 may include either accepting or rejecting potential robot positions that would result from commands. Potential robot paths 2110 Many levels can be created deep with different commands and resulting robot positions on each level. 26B shows an exemplary schematic view of the locally perceived space 2100 of the robot 100 while it is moving. An ODOA behavior 600b ( 25 ) can be any predicted robot path 2110 rate. These ratings can be from the action selection engine 580 used to determine a preferred result and an associated robot command. For example, the ODOA behavior 600b for every robot position 2120 in the robot path 2110 perform a method of object detection and obstacle avoidance, which involves identifying each cell 2103 in the grid 2101 in a borderline form 2107 (eg collision box, triangle or circle) around a corresponding position 2120 of the robot 100 and receiving a classification of each cell 2103 includes. For every cell 2103 that act as an obstacle 2102 or unknown 2104 is classified, becomes a grid point 2105 , the cell 2103 corresponds to, retrieved and executed a collision check by determining whether the grid point 2105 within a borderline form 2107 (eg a collision circle) around a location 2120 of the robot 100 located. When the grid point 2105 within the borderline 2107 In addition, the method further includes performing a triangle test to determine if the grid point 2105 within a triangular shaped border 2107 (for example, the robot can 100 be designed as a triangle). When the grid point 2105 in the collision triangle 2107 the method includes rejecting the grid point 2105 , When the robot position 2120 in a sensor system view field 405 of parent grid points 2105 on the robot path 2110 lies, become "unknown" grid points 2105 ignored because it is assumed that these grid points 2105 will be known when the robot 100 she reaches.

The method may include determining whether any obstacle collisions in a robotic path area (formed, for example, as a rectangle) between successive robotic positions 2120 in the robot path 2110 involve robotic collisions during the transition from a robot position 2120 to prevent the next.

26C provides a schematic view of the locally perceived space 2100 of the robot 100 and a sensor system view field 405 ready (the control system 510 For robot path determination, I can use certain sensors, such as the first and second 3D image sensors 450a . 450b ). Taking advantage of the holonomic mobility of the drive system 200 can the robot 100 use the durability of the known floor G to enable it to drive in directions that the sensor system visual field 405 not actively covering. If the robot 100 for example, has stood still and the first and second 3D sensor 450a . 450b pointing forward rejects the control system 510 the proposed movement, although the robot 100 is able to drive sideways, as the robot 100 does not know what is at his side, as in the example in 26C indicating an area classified as unknown on the side of the robot 100 shows shown. If the robot 100 moves forward while the first and second 3D image sensor 450a . 450b pointing to the front, the bottom G can be next to the robot 100 as known free 2106 classified as both the first and the second 3D image sensor 450a . 450b see the ground G as clear while the robot 100 moved forward and the consistency of the classification has not yet expired. (See, for example, 26B ). In such situations, the robot can 100 drive sideways.

With reference to 26D can the ODOA behavior 600b Given a large number of potential lanes of holonomic mobility, it will cause the robot to select course paths where it can see (though not up to date) where it is going. For example, the robot 100 the sensor field of view orientations predict the control system 510 allow objects to to capture. Since the robot can rotate during translation, the robot can view the sensor field of view 405 zoom in while driving.

Various implementations of the system and the techniques described herein may be implemented in digital electronic circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. The various embodiments may include implementation in one or more computer programs that are executable and / or interpretable on a programmable system, including at least one programmable processor, which may be special or generic, and at least one memory system an input device and at least one output device is connected to receive data and instructions from these and transmitted to them.

These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and may be implemented in a procedural and / or object oriented programming language and / or assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (eg, magnetic disks, optical disks, memory, programmable logic devices (PLDs) )), which is used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

Embodiments of the subject matter of this application and the functional acts described in this application may be implemented in digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more from that. Embodiments of the subject matter of this description may be implemented as one or more computer program product (s), i. H. one or more modules of computer program instructions encoded for execution by a data processing device on a computer readable medium or to control its operation. The computer-readable medium may be a machine-readable storage device, a machine-readable storage medium, a storage device, a bulk composition that causes a machine-readable propagating signal, or a combination of one or more thereof. The term "data processing device" includes all devices, devices and machines for processing data, including e.g. A programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may include code that creates an execution environment for the particular computer program, e.g. Code providing processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagating signal is an artificially generated signal, e.g. A machine generated electrical, optical or electromagnetic signal generated to encode information for transmission to a suitable receiver device.

A computer program (also known as a program, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted languages, and may be implemented in any form, including as a standalone program or as a module , Component, subroutine or other entity suitable for use in a computer environment. A computer program does not necessarily correspond to a file in a file system. A program may be in a section of a file containing other programs or data (eg, one or more scripts stored in a markup language document) in a single file associated with the particular program is stored in several coordinated files (eg files that store one or more modules, subroutines or code sections). A computer program can be used to run on a computer or on multiple computers located in one location or distributed over multiple sites and connected through a communications network.

The processes and logic operations described in this application may be performed by one or more programmable processor (s) executing one or more computer programs to perform functions by processing input data and generating outputs. be executed. The processes and logic operations may also be handled by specialized logic circuits, e.g. B. one FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) and the device can be implemented as such.

Processors suitable for executing a computer program include, for example, both general and special purpose microprocessors and any one or more processors of any type of digital computer. In general, a processor receives instructions and data from a read-only memory or a memory or both. The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. In general, a computer also includes one or more mass storage devices for storing data, e.g. Magnetic, magneto-optical, or optical disks, or is operably coupled thereto to receive or transmit data therefrom, or both. However, a computer does not necessarily have to have such devices. Furthermore, a computer may be embedded in another device, e.g. A mobile phone, a personal digital assistant (PDA), a mobile audio player, a global positioning system (GPS) receiver, to name but a few. Computer-readable media suitable for storing computer program instructions and data includes all forms of non-volatile memory, media and storage devices, including semiconductor memory devices, e.g. EPROM, EEPROM and flash memory devices; Magnetic disks, z. Internal hard disks or removable disks; magneto-optical discs; and CD-ROMs and DVD-ROMs, but this list is not exhaustive. The processor and memory may be supplemented or integrated with special logic circuits.

Embodiments of the subject matter of this description may be implemented in a computing system that includes a backend component, e.g. A data server, or a middleware component, e.g. An application server, or a front-end component, e.g. A client computer having a graphical user interface or a web browser through which a user may interact with one embodiment of the invention described in this application, or any combination of one or more of such backend, middleware or frontend components. The components of the system may be interconnected by any form or medium of digital data communication, e.g. B. a communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), e.g. For example, the Internet.

The computing system may include clients and servers. A client and a server are generally remote and typically interact over a communications network. The relationship between client and server is created by computer programs that run on each computer and have a client-server relationship.

While this application contains many specific details, these are not to be considered as limitations on the scope of the invention or the claimed subject matter, but as descriptions of features specific to particular embodiments of the invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single implementation. Conversely, various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable subcombination. Although features described above are described as acting in certain combinations and even initially claimed as such, in some instances one or more features of a claimed combination may be further removed from the combination and the claimed combination may be of a sub-combination or refer to variation of a subcombination.

Similarly, while acts in the drawings are presented in a particular order, it should not be construed that such acts must be performed in the specific order or sequence shown or that all acts shown must be performed to achieve desired results. In some circumstances, multitasking and parallel processing may be beneficial. Further, the separation of various system components in the above-described embodiments is not to be understood as requiring such disconnection in all embodiments, and it should be understood that the described program components and systems are generally integrated together into a single software product or into various software Products can be split.

Several versions have been described. It is understood, however, that various modifications can be made without departing from the spirit and scope of the disclosure. Accordingly, other embodiments fall within the scope of the following claims. For example, the acts listed in the claims may be performed in a different order and still achieve the desired results.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• IL 2006/000335 [0120]
• US 7433024 [0125]
• US 6323942 [0128]
• US 6515740 [0128]
• US 7158317 [0165]
• US 7115849 [0165]

Cited non-patent literature

• 35 USC §119 (e) [0001]
• 35 USC §120 [0001]
• IEEE1588 [0139]
• Se, S .; Lowe, David G .; Little, J. (2001). "Vision-based mobile robot localization and mapping using scale-invariant features". Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). 2. pp. 2051 ff. [0153]
• Rothganger, F; S. Lazebnik, C. Schmid and J. Ponce: 2004. 3D Object Modeling and Recognition Using Local Affine Invariant Image Descriptors and Multi-View Spatial Constraints, ICCV [0153]
• Iryna Gordon and David G. Lowe, "What and where: 3D object recognition with accurate pose", Toward Category-Level Object Recognition, (Springer-Verlag, 2006), pp. 67-82 [0153]
• Laptev, Ivan and Lindeberg, Tony (2004). "Local descriptors for spatio-temporal recognition". ECCV'04 Workshop on Spatial Coherence for Visual Motion Analysis, Springer Lecture Notes in Computer Science, Issue 3667. pp. 91-103 [0154]
• Ivan Laptev, Barbara Caputo, Christian Schuldt and Tony Lindeberg (2007). Local velocity-adapted motion events for spatio-temporal recognition. Computer Vision and Image Understanding 108: 207-229 [0154]
• Scovanner, Paul; Ali, S; Shah, M (2007). "A 3-dimensional sift descriptor and its application to action recognition". Proceedings of the 15th International Conference on Multimedia. Pp. 357-360 [0154]
• Niebles, JC Wang, H. and Li, Fei-Fei (2006). "Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words". Proceedings of the British Machine Vision Conference (BMVC). Edinburgh [0154]

Claims (50)

Mobile Robot ( 100 ), comprising: a drive system ( 200 ) with a forward direction (F); a controller ( 500 ) connected to the drive system ( 200 ) is in communication; a machine ( 450 ) for imaging volumetric point clouds located above the propulsion system ( 200 ) and is adapted to form a point cloud from a volume of space that is a ground plane in a moving direction of the mobile robot ( 100 ) to receive; and a dead zone sensor ( 490 . 2190 ) with a detection area ( 492 . 2192 ), which is arranged around an object ( 12 ) in a volume of space ( 453 ), that for the device ( 450 ) for imaging volumetric point clouds is not detectable; where the controller ( 500 ) Point cloud signals from the imaging device ( 450 ) and detection signals from the dead zone sensor ( 490 . 2190 ) and drive commands to the drive system ( 200 ) based at least in part on the received point cloud and detection signals. Mobile Robot ( 100 ) according to claim 1, wherein the dead zone sensor ( 490 . 2190 ) at least one of a device ( 450 ) for imaging volumetric point clouds, a sonar sensor, a camera, an ultrasonic sensor, LIDAR, LADAR, an optical sensor and an infrared sensor. Mobile Robot ( 100 ) according to claim 1 or 2, wherein the detection area ( 492 . 2192 ) of the dead band sensor ( 490 . 2190 ) a volume of space ( 453 ) surrounding the device ( 450 ) for imaging volumetric point clouds is not detectable. Mobile Robot ( 100 ) according to claim 3, wherein the volume of space ( 453 ), that for the device ( 450 ) is not detectable for imaging volumetric point clouds, defined by a first angle (α), a second angle (β) and a radius (R _S ), preferably 57 ° × 45 ° × 50 cm. Mobile Robot ( 100 ) according to one of the preceding claims, wherein the detection area ( 492 ) of the dead band sensor ( 490 ) between the device ( 450 ) for imaging volumetric point clouds and a detection area ( 457 ) of the device ( 450 ) is arranged for imaging volumetric point clouds. Mobile Robot ( 100 ) according to one of the preceding claims, wherein the dead zone sensor ( 2190 ) a field of view ( 2192 ) which is at least 3 meters from the dead zone sensor ( 2190 ) extends to the outside. Mobile Robot ( 100 ) according to any of the preceding claims, further comprising an array of deadband sensors ( 490 . 2190 ) with at least one dead zone sensor ( 490 . 2190 ), whose scope ( 492 . 2192 ) is arranged to an object ( 12 ) in the volume of space ( 453 ), that for the device ( 450 ) is not detectable for imaging volumetric point clouds, the array of dead zone sensors ( 490 ) with the fields of view ( 492 . 2192 ) along the forward direction (F) or over one from the robot ( 100 ) defined vertical center axis (Z) is arranged distributed. Mobile Robot ( 100 ) according to one of the preceding claims, wherein the imaging device ( 450 ) Light on a scene ( 10 ) around the robot ( 100 ) and images of the scene ( 10 ) along the direction of travel (F) of the robot ( 100 wherein the images comprise at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image; where the controller ( 500 ) based on the images, a location of an object ( 12 ) in the scene ( 10 ) and drive commands to the drive system ( 200 ) to the robot ( 100 ) based on the object location in the scene ( 10 ) to maneuver. Mobile Robot ( 100 ) according to claim 8, wherein the imaging device ( 450 ) a transit time between the emission of the light and the reception of the reflected light from the scene ( 10 ) and the controller ( 500 ) uses the runtime to set a distance to the reflective surfaces of the object ( 12 ). Mobile Robot ( 100 ) according to claim 8 or 9, wherein the imaging device ( 450 ) Comprising: a light source ( 1172 . 1310 . 1510 ) for emitting light to the scene ( 10 ), preferably in intermittent pulses, and more preferably the light source ( 1172 . 1310 . 1510 ) emits the light pulses at a first, energy-efficient frequency and emits the light pulses upon receiving a sensor event at a second, active frequency, the sensor event optionally including a sensor signal indicating the presence of an object ( 12 ) in the scene ( 10 ) indicates; and an imager ( 1174 . 1320 . 1520 ) for receiving reflections of the emitted light from the scene ( 10 ); the imager ( 1174 . 1320 . 1520 ) an array of light detection pixels ( 1174p . 1326 . 1522 ). Mobile Robot ( 100 ) according to one of the preceding claims, wherein the imaging device ( 450 ) a first and a second section ( 450a . 450b ), the first section ( 450a ) is arranged to radiate light substantially to the ground and receive reflections of the radiated light from the ground, and the second section (FIG. 450b ) is arranged to light substantially above the ground in a scene ( 10 ) and reflect reflections of the emitted light from the scene ( 10 ) around the robot ( 100 ) to recieve. Mobile Robot ( 100 ) according to one of the preceding claims, wherein the imaging device ( 450 ) Comprising: a speckle emitter ( 1310 ), which along a direction of travel of the robot ( 100 ) a speckle pattern of light on a scene ( 10 ) emits; and an imager ( 1320 ), reflections of the speckle pattern from an object ( 12 ) in the scene ( 10 ) receives; where the controller ( 500 ) Comparison images of the speckle pattern, as shown by a comparison object ( 12 ) in the scene ( 10 ), wherein the comparison images at different distances (Z _n ) from the comparison object ( 12 ) are included; and at least one target image of the speckle pattern as obtained from a target object ( 12 ) in the scene ( 10 ) is compared with the comparison images by a distance (ΔZ) of the reflecting surfaces of the target object ( 12 ). Mobile Robot ( 100 ) according to claim 12, wherein the controller ( 500 ) a primary speckle pattern on the target object ( 12 ) and calculate at least one of a cross-correlation or a decorrelation between the primary speckle pattern and the speckle patterns of the comparison images. Mobile Robot ( 100 ) according to one of the preceding claims, wherein the imaging sensor ( 450 ) scans from side to side relative to the forward direction (F) to form a lateral field of view (FIG. 452 , θ _vH ) and / or _scanning up and down to a vertical field of view ( 452 , θ _vv ) of the imaging sensor ( 450 ) to enlarge. Mobile Robot ( 100 ) according to one of the preceding claims, wherein the controller ( 500 ) after a limit period after the reception, the use of the received point cloud signals for the output of drive commands to the drive system ( 200 ). Mobile Robot ( 100 ) according to claim 15, wherein the controller ( 500 ) the setting of the use of the received point cloud signals in determining the presence of an object ( 12 ) in the volume of space ( 453 ), that for the device ( 450 ) is not detectable for imaging volumetric point clouds, based on the received detection signals from the dead zone sensor ( 490 . 2190 ) picks up. Mobile Robot ( 100 ) according to claim 16, wherein the controller ( 500 ) proceeds with the setting of the use of the received point cloud signals after the limit period after the reception, preferably based on the received dead zone signals from the dead zone sensor ( 490 . 2190 ) it was determined that the volume of space ( 453 ), that for the device ( 450 ) is not detectable for imaging volumetric point clouds, free of objects ( 12 ). Mobile Robot ( 100 ), comprising: a base ( 120 ); a holonomic drive system ( 200 ), that from the base ( 120 ) and defines a vertical axis (Z), the holonomic drive system ( 200 ) the robot ( 100 ) via a desktop ( 5 ) of a scene ( 10 ) maneuvered; a controller ( 500 ) connected to the drive system ( 200 ) is in communication; a leg ( 130 ), that from the base ( 120 ) goes up; a body ( 140 ), off the leg ( 130 ), whereby the body ( 140 ) in relation to the base ( 120 ) vertical axis (Z) rotates; and at least one imaging sensor ( 450 ), on the body ( 140 ) and a volumetric point cloud of the scene ( 10 ) around the robot ( 100 ) receives; wherein the rotating body ( 140 ) the device ( 450 ) for imaging volumetric point clouds in a pivoting movement about the vertical axis (Z) moves and so a field of view ( 452 ) of up to 360 ° around the robot ( 100 ). Mobile Robot ( 100 ) according to claim 18, wherein the at least one imaging sensor ( 450 ) an imaging axis ( 455 ) arranged to travel downwardly along a forward direction (F) of the drive system (FIG. 200 ) to aim. Mobile Robot ( 100 ) according to claim 18, wherein the at least one imaging sensor ( 450 ) a first imaging sensor ( 450 . 450a ) with an imaging axis ( 455 ) arranged to travel along a forward direction (F) of the drive system (Fig. 200 ) and a second imaging sensor ( 450 . 450b ) with an imaging axis ( 455 ), which is arranged to be parallel to or above the working surface ( 5 ) away from the body ( 140 ) to aim. Mobile Robot ( 100 ) according to one of claims 18-20, wherein the at least one imaging sensor ( 450 ) is scanned side-to-side relative to the forward direction (F) to form a lateral field of view (FIG. 452 , θ _vH ) and / or _scanning up and down to a vertical field of view ( 452 , θ _vv ) of the imaging sensor ( 450 ) to enlarge. Mobile Robot ( 100 ) according to one of claims 18-21, wherein the at least one imaging sensor ( 450 ) Comprising: a speckle emitter ( 1310 ), which adds a speckle pattern to light on the scene ( 10 ) emits; and an imager ( 1320 ), reflections of the speckle pattern from an object ( 12 ) in the scene ( 10 ) receives; where the controller ( 500 ) Comparison images of the speckle pattern, as shown by a comparison object ( 12 ) in the scene ( 10 ), wherein the comparison images at different distances (Z _n ) from the comparison object ( 12 ) were recorded; and at least one target image of the speckle pattern as obtained from a target object ( 12 ) in the scene ( 10 ) is compared with the comparison images by a distance (ΔZ) of the reflecting surfaces of the target object ( 12 ). Mobile Robot ( 100 ) according to claim 22, wherein the at least one imaging sensor ( 450 ) Pictures of the scene ( 10 ) along a direction of travel (F) of the robot (FIG. 100 ), the images comprising at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image. Mobile Robot ( 100 ) according to claim 22 or 23, wherein the controller ( 500 ) based on the image comparison a location of an object ( 12 ) in the scene ( 10 ) and drive commands to the drive system ( 200 ) to the robot ( 100 ) based on the object location in the scene ( 10 ) to maneuver. Mobile Robot ( 100 ) according to any one of claims 22-24, wherein the controller ( 500 ) a primary speckle pattern on the target object ( 12 ) and calculate at least one of a cross-correlation or a decorrelation between the primary speckle pattern and the speckle patterns of the comparison images. Mobile Robot ( 100 ) according to one of claims 18-25, wherein the at least one imaging sensor ( 450 ) a machine ( 450 ) for imaging volumetric point clouds located at a height greater than 60.96 cm (2 feet) above the working surface ( 5 ) and is designed to be a point cloud from a volume of space that is a ground plane ( 5 ) in a direction of movement (F) of the robot (F) 100 ) to receive. Mobile Robot ( 100 ) according to one of claims 18-26, wherein the at least one imaging sensor ( 450 ) has a horizontal field of view (θ _vH ) of at least 45 degrees and a vertical field of view (θ _vV ) of at least 40 degrees and / or a range of between 1 meter and about 5 meters. Mobile Robot ( 100 ) according to one of claims 18-27, wherein the imaging sensor ( 450 ) has a latency of about 44 ms, wherein the image output of the imaging sensor ( 450 ) preferably receives a timestamp that balances the latency. Mobile Robot ( 100 ) according to any one of claims 18-28, further comprising a deadband sensor ( 490 . 2190 ) with a detection area ( 492 . 2192 ) arranged to receive an object ( 12 ) in a volume of space ( 453 ) used for the imaging sensor ( 450 ) is not detectable. Mobile Robot ( 100 ) according to claim 29, wherein the dead zone sensor ( 490 . 2190 ) at least one of a device ( 450 ) for imaging volumetric point clouds, a sonar sensor, a camera, an ultrasonic sensor, LIDAR, LADAR, an optical sensor and an infrared sensor. Mobile Robot ( 100 ) according to claim 29 or 30, wherein the detection area ( 492 . 2192 ) of the dead band sensor ( 490 . 2190 ) a volume of space ( 453 ) surrounding the device ( 450 ) for imaging volumetric point clouds is not detectable. Mobile Robot ( 100 ) according to claim 31, wherein the volume of space ( 453 ), that for the device ( 450 ) is not detectable for imaging volumetric point clouds, defined by a first angle (α), a second angle (β) and a radius (R _S ), preferably 57 ° × 45 ° × 50 cm. Mobile Robot ( 100 ) according to any one of claims 29-32, wherein the detection area ( 492 ) of the dead band sensor ( 490 ) between the imaging sensor ( 450 ) and a coverage area ( 457 ) of the imaging sensor ( 450 ) is arranged. Mobile Robot ( 100 ) according to any one of claims 29-33, wherein the dead zone sensor ( 2190 ) a field of view ( 2192 ) located at least 3 meters from the dead zone sensor ( 2190 ) extends to the outside. Mobile Robot ( 100 ) according to any one of claims 29-34, further comprising an array of dead-zone sensors ( 490 . 2190 ) with at least one dead zone sensor ( 490 . 2190 ), whose scope ( 492 . 2192 ) is arranged to an object ( 12 ) in the volume of space ( 453 ) used for the imaging sensor ( 450 ) is undetectable, with the array of deadband sensors ( 490 ) with its fields of view ( 492 . 2192 ) along the forward direction F or evenly across a vertical axis of the robot ( 100 ) defined center axis (Z) is arranged distributed. Mobile Robot ( 100 ) according to any one of claims 29-35, wherein the controller ( 500 ) after a limiting period, the use of the imaging sensor ( 450 ) received point cloud signals for the output of drive commands to the drive system ( 200 ). Mobile Robot ( 100 ) according to claim 36, wherein the controller ( 500 ) the setting of the use of the received point cloud signals in determining the presence of an object ( 12 ) in the volume of space ( 453 ) used for the imaging sensor ( 450 ) is not detectable based on received detection signals from the dead zone sensor ( 490 . 2190 ) picks up. Mobile Robot ( 100 ) according to claim 37, wherein the controller ( 500 ) with the setting of the use of the received point cloud signals after the limit period, preferably based on the received detection signals from dead zone sensor ( 490 . 2190 ) it was determined that the volume of space ( 453 ) used for the imaging sensor ( 450 ) is not detectable, free of objects ( 12 ). Mobile Robot ( 100 ) according to any one of claims 18-38, wherein the body ( 140 ) in relation to the leg ( 130 ) and / or the leg ( 130 ) in relation to the base ( 120 ) is rotated about a vertical axis (Z). Mobile Robot ( 100 ) according to any one of claims 18-39, wherein the leg ( 130 ) has a variable height (H _L ). Method for object detection for a mobile robot ( 100 ), the method comprising: rotating an image sensor ( 450 ) about a vertical axis (Z) of the robot ( 100 ), wherein the imaging sensor ( 450 ) Light on a scene ( 10 ) around the robot ( 100 ) and images of the scene ( 10 wherein the images comprise at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image; Determining a Location of an Object ( 12 ) in the scene ( 10 ) based on the pictures; Assigning a confidence level for the object location; and maneuvering the robot ( 100 ) in the scene ( 10 ) based on the object location and the corresponding confidence level. The method of claim 41, further comprising constructing an object map ( 1700 ) of the scene ( 10 ). The method of claim 40 or 41, further comprising demoting the confidence level of each object location over time until the corresponding object location is updated with a newly determined object location. The method of claim 43, further comprising: detecting an object ( 12 ) in a volume of space ( 453 ) used for the imaging sensor ( 450 ) is not detectable, preferably using a deadband sensor ( 490 . 2190 ) with a detection area ( 492 . 2192 ), which is arranged around an object ( 12 ) in the volume of space ( 453 ) used for the imaging sensor ( 450 ) is not detectable; and adjusting the downgrading of the confidence level of the detected object ( 12 ). The method of claim 44, further comprising continuing to downgrade the confidence level of the detected object. 12 ) when it is detected that the volume of space ( 453 ) used for the imaging sensor ( 450 ) is not detectable, free from the object ( 12 ). The method of any of claims 41-45, further comprising maneuvering the robot ( 100 ) to at least: a) the object ( 12 ) and along a perimeter of the object ( 12 ) or b) the object ( 12 ) to evade. The method of any of claims 41-46, further comprising emitting light to the scene ( 10 in intermittent pulses, optionally alternating a frequency of the emitted light pulses, preferably radiating the light pulses at a first energy efficient frequency, and after receiving a sensor event, emitting the light pulses at a second active frequency, the sensor event preferably including a sensor signal that the presence of an object ( 12 ) in the scene ( 10 ). The method of any of claims 41-47, further comprising creating the three-dimensional depth image of the scene ( 10 ) by: emitting a speckle pattern of light to the scene ( 10 ); Receiving reflections of the speckle pattern from the object ( 12 ) in the scene ( 10 ); Storing comparison images of the speckle pattern, as shown by a comparison object ( 12 ) in the scene ( 10 ), wherein the comparison images at different distances (Z _n ) from the object to be compared ( 12 ) are recorded; Picking up at least one target image of the speckle pattern as seen from a target object ( 12 ) in the scene ( 10 ) is reflected; and compare. of the at least one target image with the comparison images by a distance (ΔZ) of the reflective surfaces of the target object ( 12 ). The method of claim 48, further comprising determining a primary speckle pattern on the target object (16). 12 ) and calculating at least one of a cross-correlation or a decorrelation between the primary speckle pattern and the speckle patterns of the comparison images. A method according to any one of claims 48 or 49, further comprising taking images of reflections of the emitted speckle pattern from surfaces of the target object ( 12 ) at a frame rate, preferably between about 10 Hz and about 90 Hz, and optionally eliminating differences between speckle patterns taken in successive images to obtain the target object (Fig. 12 ) to identify.

Images