WO2020074722A1

WO2020074722A1 - Method and robot system for inputting a work area

Info

Publication number: WO2020074722A1
Application number: PCT/EP2019/077635
Authority: WO
Original assignee: Robert Bosch Gmbh
Priority date: 2018-10-12
Filing date: 2019-10-11
Publication date: 2020-04-16
Also published as: DE102018125266B3; CN112789570A

Abstract

A robot system (15) for inputting a work area (4) in a monitored region (14), wherein: the robot system (15) has, as an input unit, a laser pointer (2) for inputting laser dots (3) by a user (1); the robot system (15) has a first camera (6) for capturing a first image of the monitored region (14) from a first viewing angle; the robot system (15) has a mobile robot unit (9) with a second camera (10) for capturing a second image of the monitored region (14) from a second viewing angle; and the robot system (15) has a first image analysis unit (7) for recognising laser dots (3) in the first image and in the second image, or has a first image analysis unit (7) for recognising laser dots in the first image and a second image analysis unit (11) for recognising laser dots in the second image, characterised in that the robot system (15) has an evaluation unit (13) for calculating a polygon chain (5, 33) that determines the work area, from the recognised laser points (3).

Description

description

title

Method and robot system for entering a work area

The invention is based on a system or method for entering a work area for a mobile robot according to the type of the independent claims.

From DE10215167C1 an arrangement for commanding a semi-autonomous system in a work area is already known. A camera is used to monitor the

Working area and a laser pointer for marking object points in the working area.

From US4887223 a system for material transport is known which controls mobile robots. A visual navigation system for the control of the mobile robots evaluates images that are recorded by means of cameras attached to the mobile robots.

An optical guidance system for mobile robots is already known from the publication “Optical Guidance System for Multiple Mobile Robots' (Proceedings of the 2001 1EEE International Conference on Robotics & Automation, Seoul, Korea, 2001 IEEE). In doing so, a

Laser pointer used, which projects target positions for the mobile robots on the floor.

From the publication “This Far, No Further: Introducing Virtual Borders to Mobile Robots Using a Laser Pointer” (Arxiv preprint arxiv: 1708.06274, 2017), a method is already known in which a work area for a mobile robot is entered by a laser pointer.

DE 10 2016 224 693 A1 relates to a portable hand-held control device for remote control of a manipulator, the portable hand-held control device having an optical radiation source for generating a bundled light beam, and the bundled light beam being set up to couple the portable hand-held control device to a manipulator. Other aspects of the invention relate to a portable hand-held device with a

Jogwheel, a sensor for detecting a bundled light beam from a portable handheld operating device, a coupling display means for visualizing the coupling state of a portable handheld operating device with a manipulator, a system for coupling a portable handheld device with a manipulator and a method for coupling a portable handheld device with a manipulator.

An optoelectronic sensor according to DE 20 2009 012 589 U1 for monitoring and optical detection of a flat or voluminous protective area has a receiving unit for receiving received light beams from the protected area and generating received signals as a function of the received light, and a control and evaluation unit for generating a switching state "Object in the protected area" corresponding object detection signal depending on the

Receive signals. According to the invention, at least one light transmitter is provided with which the switching state “object in the protected area” can be visually recognized in the protected area, the light transmitter for displaying the switching state being a visible one

Emits light beam in the direction in which the object lies.

DE 10 2008 054 863 A1 discloses a method for controlling a robot on the basis of light marks, in which a light mark is generated by means of a light source, a camera system detecting the light mark, keying the light mark with less than or equal to half the refresh rate of the camera system light / dark, and is recognized by evaluating the images captured by the camera system.

AT 512 350 B1 relates to a computer system which comprises a data processing system and a freely movable input device which contains at least one position change sensor, from which information can be sent to the data processing system.

Information about the absolute position of the input device is given by the

Data processing system can be detected by using a light-sensitive

position-sensitive input surface is in data connection and the input device emits a light beam, the impact of which on the input surface can be detected thereby.

US 2008/0 063 400 A1 discloses a system and method for controlling a remote vehicle comprises a manual control with a laser generator for generating a laser beam. The hand control can be manipulated to align and trigger the laser beam to determine a target for the distant vehicle. The distant vehicle detects a reflection of the laser beam and moves towards the desired target. The Manual control enables one-handed operation of the remote vehicle and one or more of its payloads.

TROUVAIN, Boris A .; describes an approach for the integration of a direct multimodal human-robot interaction with a laser pointer and speech into an experimental multi-robot control station. SCHNEIDER, Frank E .;

WILDERMUTH, Dennis. Integrating a multimodal human-robot interaction method into a multi-robot control station. In: Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No. 01TH8591). IEEE, 2001.

Pp. 468-472.

The robot system according to the invention for entering a work area in a

Monitoring area with the features of independent claim 1 has the advantage that it has an evaluation unit for determining a polygon chain determining the working area from laser points that are entered by a user with a laser pointer and that with a first camera from a first angle and one second camera can be recognized from a second angle. Because with the first camera and the second camera, which is connected to a mobile robot, a larger monitoring area can be monitored, in which the work area can be entered.

The method according to the invention for entering a work area for a mobile robot with the method steps of independent claim 2 also has the advantage that a projection from a laser beam into a three-dimensional coordinate space

Point set is determined in the three-dimensional coordinate space, and from the

A polygon chain for determining the work area is determined and evaluated. This is because a user can conveniently enter the work area by drawing with the laser pointer in a monitoring area.

The measures listed in the dependent claims are advantageous

Further developments and improvements of the device specified in the independent claims are possible.

It is advantageous if the method according to the invention for the monitoring area

Occupancy probabilities. Because the mobile robot can do that

Use occupancy probabilities of the monitoring area for navigation planning. It is also advantageous if an evaluation of the occupancy probabilities of the monitoring area results in the work area in a prohibition zone or a permission zone, so that limits of its movement area are defined for the mobile robot.

In addition, it is advantageous if a control point is determined from the laser points in order to determine the work area, the control point being a change in a

Probability of occupancy of the work area. This is because an entry with a laser pointer can be used to define a prohibition zone or permission zone, which are evaluated to determine the work area.

It is also advantageous if the method according to the invention uses a plane model when projecting into the three-dimensional coordinate space. Because this simplifies the calculation of the projection.

Furthermore, it is advantageous if, in the method according to the invention, which continuously captures a first image and a second image, the first image and / or the second image contains depth information. Because position calculations of the laser points are thus improved.

It is particularly advantageous if the method according to the invention performs a grouping of points from the point set when determining the polygon chain. This is because incorrectly recognized or incorrectly entered laser points can be removed for determining the polygon chain.

It is particularly advantageous if the method according to the invention performs a thinning of points from the point set when determining the polygon chain. This is because spatially redundant points can be sorted out for determining the polygon chain.

It is particularly advantageous if the method according to the invention uses a subset of points from the point set when determining the polygon chain. Because this enables the polygon chain to be calculated quickly.

Embodiments of the invention are shown in the drawings and in the

following description explained in more detail. Show it

Fig. 1 is a schematic view of an embodiment of an inventive

Robot systems,

2 shows a method according to the invention, and

Fig. 3 is an exemplary illustration of a method step of the invention

Procedure.

1 shows an embodiment of a robot system according to the invention.

The robot system 15 according to the invention for entering a work area 4 in one

Monitoring area 14 has a laser pointer 2, a first camera 6, a mobile robot 9 with a second camera 10, a first image analysis unit 7, a second

Image analysis unit 11, a first interface 8 for communication, a second interface 12 for communication and an evaluation unit 13. The laser pointer 2 is designed as an input unit for the input of laser points 3 by a user 1.

The user 1 holds the laser pointer 2 as an input unit for the robot system 15 in one hand and illuminates laser points 3 on a floor of the monitoring area 14. When the user 1 uses the laser pointer 2 to draw the laser points 3 on the floor, the user 1 indicates the limits of the Work area 4 in the robot system 15. The user 1 can also enter a control point with the laser pointer 2, which determines on which side of the boundary the work area 4 is located.

The first camera 6 takes a first image of the monitoring area 14 from a first point of view. The first camera 6 is stationary on a wall and takes the first image from a view from above. The first camera 6 is connected to a first one

Image analysis unit 7 for recognizing the laser points 3 in the first image. The

Image analysis unit 7 is connected to first interface 8 for communication, for example via a radio link, in robot system 15. For example, first camera 6 is an RGB camera for color image recording or an RGBD camera for combined color image and depth image recording.

The mobile robot 9 has the second camera 10, the second image analysis unit 11, the second interface 12 and the evaluation unit 13. The second camera 10 is designed to record a second image of the monitoring area 14 from a second point of view. The The second image analysis unit 11 is designed to detect the laser points 3 in the second image and is connected to the second camera 10. The second interface 12 is connected to the second image analysis unit 11 and the evaluation unit 13. The second interface 12 is designed for communication in the robot system 15 , for example via a radio connection. The evaluation unit 13 is designed to determine a polygon chain 5 which determines the working area 4 from the detected laser points 3. For example, the second camera 10 is an RGB camera for color image recording or an RGBD camera for combined color image and depth image recording. 2 shows a method 20 according to the invention for entering a work area for a mobile robot in a monitoring area. The method 20 has six method steps 21, 22, 23, 24, 25, 26.

In the first method step 21, a first image from a first view and a second image from a second view are continuously captured. In the second method step 22, laser spots 3 are recognized in the first image and / or the second image. In the third

Method step 23 is transformed from laser points 3 into a projection

three-dimensional coordinate space a set of points in the three-dimensional

Coordinate space determined. For example, the first image and / or the second image contains gray values, color values and / or depth values. For example, for the projection

Layer model used.

In the fourth method step 24, a polygon chain 5 is determined from the point set.

For example, in the fourth method step 24, a grouping or clustering is carried out with points from the point set, in which points are filtered out which are not assigned to a group with many points. For example, the DBSCAN algorithm is carried out for grouping.

For example, in the fourth method step 24, thinning of points from the set of points is carried out. For example, locally adjacent points are replaced by a point that is formed from their mean values.

For example, in the fourth method step 24, the point set or one becomes

Subset of the point set, which results from the grouping and filtering or thinning, the polygon chain 5 is formed as follows. Starting from a starting point, a Connected to an end point that results from the set of points so that it is closest to the start point. The end point is then the next, current start point and a connection is drawn to a next, current end point, which in turn results from the number of points, is closest to the current start point and has not yet been assigned to polygon chain 5. This end point becomes the next, current starting point and the drawing of connections continues accordingly until all points have been taken into account for the formation of the polygon chain 5. The polygon chain 5 can be open or closed, for example. In a fifth method step 25, the polygon chain 5 is used to determine the

Work area 4 evaluated. For example, a grid with cells with occupancy probabilities is created for the monitoring area 14. Entries for a cell of the grid can indicate whether the cell is free, occupied, or with a certain probability is occupied or free. For example, the closed polygon chain 5 can determine an outer boundary of the working area 4 if the occupancy probabilities of the cells which are delimited by the polygon chain 5 and lie within the polygon chain 5 are marked as free and the occupancy probabilities of the cells which lie outside the polygon chain 5 , are marked as occupied. An area within the polygon chain 5 is thus specified as a permission zone and an area outside the polygon chain as a prohibition zone, so that the working area 4 is defined as the area lying within the polygon chain 5. The mobile robot 5 would only be allowed to move within the work area 4 when performing a task. For another example, the area within the polygon chain 5 could be occupied with the occupancy and be marked free outside, so that the monitoring area 14 without the area within the polygon chain 5 is defined as the work area 4. In a sixth method step 26, a control point is determined from the laser points 3, which causes a change in an occupancy probability of the work area 4.

For example, the checkpoint can use its position to indicate whether an area in which the checkpoint is located is marked as a prohibition zone or as a permission zone, and thus, if necessary, cause a change in the work area 4 in that the permission zone and the prohibition zone could be exchanged.

FIG. 3 illustrates an example of the fifth method step 25. A set of points recognized from laser points and projected into a three-dimensional coordinate space serves as Initial set 30. After clustering and filtering the initial set 30, there is a reduced point set 31. Performing a thinning on the reduced point set 31 leads to a thinned point set 32. A polygonal line 33 is generated from the thinned point set 32.

Claims

Expectations

1. Robot system (15) for entering a work area (4) in a monitoring area

(14),

wherein the robot system (15) has as an input unit a laser pointer (2) for inputting a plurality of laser points (3) for marking the work area (4) by a user (1),

wherein the robot system (15) has a first camera (6) for taking a first image of the monitoring area (14) from a first point of view,

wherein the robot system (15) has a mobile robot unit (9) with a second camera (10) for taking a second image of the monitoring area (14) from a second point of view,

the exact positions of the two cameras (6, 10) in space

Robot system (15) are known, the two images record the same laser points (3) from the laser pointer (2) and

wherein the robot system (15) has a first image analysis unit (7) for the detection of

Laser dots (3) in the first image and in the second image or a first one

Image analysis unit (7) for detecting laser points in the first image and a second image analysis unit (11) for detecting laser points in the second image, characterized in that the robot system (15) has an evaluation unit (13) for determining (24) one Point set (30) in a three-dimensional coordinate space by projecting the detected laser points (3) into the coordinate space and

Evaluation (25) of a polygon chain (5, 33) determining the working area from the point set (30).

2. Method (20) for entering a work area (4) for a mobile robot (9) marked by a user (1) with a laser pointer (2) using a plurality of laser points (3) in a monitoring area (14),

wherein a first image from a first view known to the robot (9) and a second image from a second view known to the robot (9) are continuously acquired (21), the same laser points (3) in the first image and the second image recognized (22),

whereby from the laser points (3) by a projection into a three-dimensional

Coordinate space a point set (30) in which three-dimensional coordinate space is determined (23), characterized in that a polygon chain (5, 33) for determining the working area (4) is evaluated (25) from the point set (30).

3. The method according to claim 2, characterized in that the monitoring area (14) is provided with occupancy probabilities.

4. The method according to claim 3, characterized in that an evaluation of the

Allocation probabilities in a prohibition zone or a permission zone

Work area (4) result.

5. The method according to claim 4, characterized in that for determining the

Working area (4) a control point is determined from the laser points (3), the control point causing a change in the occupancy probability of the working area (4).

6. The method according to claim 2, characterized in that in the projection in the

three-dimensional coordinate space a plane model is used.

7. The method according to claim 2, characterized in that the first image or the second image contains depth information.

8. The method according to claim 2, characterized in that when determining the

Polygon chain (5, 33) a grouping of points from the point set (30) takes place.

9. The method according to claim 2, characterized in that when determining the

Polygon chain (5, 33) a thinning of points from the point set (30) takes place.

10. The method according to claim 2, characterized in that the polygon chain (5, 33) is determined from a subset of the point set (30).