WO2016193058A1

WO2016193058A1 - Method for determining a path point

Info

Publication number: WO2016193058A1
Application number: PCT/EP2016/061667
Authority: WO
Inventors: Jacob Saoumi
Original assignee: Kuka Roboter Gmbh
Priority date: 2015-05-29
Filing date: 2016-05-24
Publication date: 2016-12-08
Also published as: CN107666988A; EP3302895A1; DE102015209900A1

Abstract

The present invention relates to a method for determining a path point for robot programming, comprising the detection of a first temperature profile of a surrounding area and the determination of the path point using the detected first temperature profile. The present invention further relates to a corresponding system for determining a path point for robot programming.

Description

Method for determining a path point

1. Technical area

The present invention relates to a method for determining a path point for a robot programming, and further to a corresponding system for

Determining a path point for a robot programming.

2. Technical background

Robots (and in particular industrial robots) are freely programmable,

program-controlled handling devices. The actual robot mechanism is commonly referred to as a manipulator, which may consist of a plurality of movable members or axles. Targeted control of motors (and in particular servomotors) allows the individual axes to be moved.

During a work process, such as a welding process or seaming process, a corresponding manipulator typically travels a trajectory or path, with the individual points of the path dictated by a robot program or path planning. The term path planning generally describes

Method and apparatus for moving a manipulator. For programming a robot program, the positions of individual track points, which are to be approached by the manipulator, must first be recorded or specified. The remaining points, or the detailed path movement of the manipulator, is calculated by means of an algorithm, for example by means of a point-to-point path planning, or so-called spline path planning. This setting of individual train points, or the "teaching" of track points, or the setting of a trajectory is a time-consuming step in the

Robot programming. In principle, a distinction can be made between two different approaches: online and offline programming.

In offline programming, for example, individual path points can be generated by means of a CAD model or a simulation tool. The manipulator will not required for program creation, because the movements of the

Manipulator can be determined using virtual three-dimensional simulation environments.

In online programming, programming is done directly on the

Manipulator. For this purpose, for example, special programming handheld devices can be used to guide a manipulator directly from one point to another while recording the approached track points and store. However, this procedure is very time consuming and not intuitive. Furthermore, a playback method is known as another variant of online programming. In this case, the programmer moves the manipulator by directly guiding the manipulator along a later trajectory. Meanwhile, individual points

recorded and transferred to a corresponding robot program. For this purpose, however, special safety precautions must be taken so that the

Programmer is not injured by the manipulator. Furthermore, a PdV (programming-by-the-master) method is known as another variant of online programming in which different sensors and machine learning methods are used to map human motion through a corresponding manipulator motion.

From the document US 2014/0201112 Ai is a method for teaching of

Track points known in which a special handling device is used, which is equipped with lighting elements. This method is also disadvantageous, since this special handling device must necessarily be provided, and also an appropriate training of the programmer to use the handling device is required. It is therefore an object of the present invention to provide a method and a system with which work processes of a robot can be defined, wherein the above-mentioned disadvantages are at least partially eliminated. It is a further object of the present invention to provide a method and system with which track points of robot programming can be intuitively and reliably determined. These and other objects, which will become apparent upon reading the following description, are achieved by a method according to claim 1 and a system according to claim 10.

3. Content of the invention The present invention relates to a method for determining a path point for a robot program or for a robot program. The

The inventive method can thus be used to determine one or more track points that can be used for robot programming. The term robot programming here comprises predetermining a sequence of movements of a robot or manipulator corresponding to one or more track points. The specific path points can for example be integrated directly into a robot program or a path planning, or be used to create such. The method can preferably be used during offline or online programming of a manipulator. The method includes detecting a first temperature profile of an environment, and determining a orbit point using the sensed first temperature profile. In this case, preferably more than one track point can be determined, for example if a track consisting of several track points is to be determined. Preferably, the detection of the temperature profile by means of a temperature measuring device and further preferably by means of a

Thermal camera.

Thus, for example, a thermal signature which is present on a surface of the environment can be detected, and this thermal signature used to determine the point of the orbit. A complex simulation or direct guidance of a manipulator is advantageously not necessary.

The term "environment" encompasses objects and objects which

For example, in the robot programming can be included. For example, workpieces can fall under the concept of "environment", which are processed by means of a manipulator (and a corresponding robot program) should. The environment may also include planar or textured surfaces, such as walls, floors, tabletops, molds, and so forth. The environment may also include one or more manipulators.

Preferably, the method further comprises generating a temperature change on the environment, wherein said step of generating is preferably performed by a user and in particular by a human. Further preferably, the temperature change is generated by touching the environment with a finger. In particular, the detected first temperature profile of the environment is preferably characteristic of that generated by the user

Temperature change on the environment.

Thus, the method according to the invention makes it possible to define corresponding track points by touching the surroundings. In doing so, traces of heat can be detected and processed in order to determine a path point. These traces of heat are generated, for example, by a user by touching a surface. Since the body temperature of a human (about 37 ° C) is usually not equal to the ambient temperature, by touching the environment by the user, the environment is locally heated (or locally cooled).

For example, by moving an index finger on any surface of the environment, a user can thus create thermal paths on the basis of which path points can be determined for the robot programming. Thus, the method according to the invention provides a simple and intuitive method

Defining a desired railway line ready, which is to be traversed for example by a manipulator. Alternatively, only individual points in the environment can be touched by a user, which are used for the determination of the corresponding path point. Thus, determining the path point preferably comprises determining the ones generated on the environment

Temperature change. For example, simple detection algorithms can be used to detect traces of heat or single point heat points. Depending on the components used (such as the thermal imager), properties of the touched surface,

Ambient temperature, time between contact and recording the temperature profile, etc., the finger touch should be sufficiently long to allow the most unambiguous determination of the path point.

More preferably, generating the temperature change includes

Generating a symbolic temperature change. The user can

For example, move a certain gesture with his finger on the environment, so that the resulting temperature change reflects this gesture. For example, the user can run a cross sign, a circle sign, stylized arrows or similar symbols or signs with his finger on the environment. The symbolic temperature change can also result from a simultaneous, for example, parallel guidance of two fingers along the environment. Further preferably, determining the path point comprises associating a predefined robotic action. By determining the temperature change thus produced on the environment, the sign or gestures can be recognized and assigned to a corresponding robot action in the robot program. Thus, the user can intuitively define certain robotic actions, such as starting or stopping a measurement.

Preferably, the method further comprises detecting a second temperature profile of the environment. In this case, the determination of the path point preferably takes place using the detected second temperature profile. Preferably, the second temperature profile is different from the first temperature profile, for example due to a user acting on the environment. For example, a first temperature profile of the environment can be detected, followed by a

Temperature change on the environment caused by a user (for example, this user touches a point of the environment with his finger), and then a second temperature profile of the environment are detected. Based on the two recorded temperature profiles or under

Using these temperature profiles, the track point is determined, which corresponds to the location that was touched with the finger. Further preferably, determining the path point comprises comparing the first and second

Temperature profile. By such a comparison, it can be efficiently determined whether and which area of an environment has been touched, for example, by a user, and is to be used for the determination of the point of the path. Preferably, the method further comprises capturing a three-dimensional image of the environment, and determining a three-dimensional structural profile of the environment based on the captured three-dimensional image. Thus, both a temperature profile and a three-dimensional image of the environment is detected. The three-dimensional structure profile contains information about a spatial arrangement of the environment, such as a height profile. Furthermore, the determination of the path point takes place using the determined three-dimensional structure profile of the environment. This makes it possible to detect traces of heat even on surfaces whose geometry is unknown beforehand. With the help of the three-dimensional image can be recognized which three-dimensional

Coordinates of the room or the environment are to be used for the determination of the point of the train. Further preferably, the detection of the three-dimensional image by means of a 3D camera.

Preferably, the method further comprises providing the particular

Track point to a manipulator control. Furthermore, preferably

Method of controlling a manipulator based on the particular point of the track, this control being done by means of the manipulator control. In this case, the particular path point can be provided in particular preferably for spline path planning or point-to-point path planning. Thus, the particular track point can be used directly to control a manipulator accordingly. For example, a user can touch an object, and then, by means of the method according to the invention, this object can be gripped by a manipulator. A complex programming of such a manipulator or

Robot movement is not necessary, but the present invention allows intuitive setting of track points.

Preferably, the method further comprises picking up a voice command, and determining the web-point further using the

recorded voice command. A programmer can thus use his voice to intuitively control the process by, for example, recording a

Temperature profile controls by means of a corresponding voice command. Preferably, the voice command is recorded by means of a microphone. Thus, the method is still intuitive, a lengthy training a programmer is there advantageous not necessary. Preferably, it is possible to switch between two method modes by means of a voice command, wherein preferably a method mode comprises determining individual path points based on, for example, individual ones

Heat points, and another method mode is determining a path based on, for example, a heat trace.

Preferably, the environment has a substantially planar surface, for example a plate or glass surface. Furthermore, the detection of the

Temperature profile preferably detecting a temperature profile of the

Essentially flat surface. For example, a programmer can impose a particular trajectory with his finger on the substantially planar surface, such as by touching the surface with his index finger, and then the heat trace resulting from that contact can be detected and used to calculate the trajectory point be used. This method is thus very robust, since, for example, fragile

Touch-sensitive handling devices for programming are not necessary. The skilled person understands, according to the application, to choose a suitable material with a suitable thermal conductivity. In this case, the skilled person understands in particular that a material with high thermal conductivity, such as steel, allows easy generation of a temperature change by, for example, a finger touch, but the resulting local temperature increase can easily dissipate. Further, one of ordinary skill in the art will understand that a low thermal conductivity material, such as wood, may absorb poor heat by, for example, a finger touch, but a local temperature increase will persist longer. Thus, for example, depending on the application form, a surface (for example a wood panel) with a corresponding coating (eg., Copper coating) are provided to an optimal determination of a

Railway point in the sense of the present invention to allow.

Furthermore, the present invention relates to a system for determining a path point for a robot program or for a robot programming. In this case, the system comprises a temperature measuring device, configured to detect a first temperature profile of an environment, and a controller, which is set up to determine the path point using the means of the Temperature measuring device detected first temperature profile. In this case, the temperature measuring device preferably comprises a thermal imaging camera.

Preferably, the system further comprises a camera configured to capture a three-dimensional image of the environment, and the controller is further configured to determine the orbit point using the captured three-dimensional image, the camera preferably comprising a 3D camera.

Preferably, the system comprises a microphone which is adapted to

Recording a voice command, and the controller is further adapted to

Determining the track point using the recorded voice command. The individual components of the system may be implemented (in part) separately or provided (partially) in a single device.

For example, the controller and microphone may be provided in a single device, such as a portable computer. Furthermore, for example, the controller can be implemented together with a robot controller, and / or the temperature measuring device can be mounted on a corresponding robot or robot.

Be provided manipulator.

The term "temperature profile" in the sense of the present invention may include a temperature change generated by a user on the environment.

In particular, the temperature change produced on the environment may preferably comprise a single heat point, and / or a heat trace consisting of several heat points. The term "heat point" encompasses a single point on the environment, which is distinctive due to its (relative) temperature, for example a heat point on a surface has a higher or lower temperature than the other points on the surface A "heat trace", on the other hand, does not just come from touching one area at a time but, for example, by tracing a line on the surface. Thus, a web line can be determined, which is to be traveled, for example, during a welding process by the manipulator to create a specific weld. 4th embodiments

In the following, the present invention will be described in detail with reference to the accompanying drawings. Showing:

1 shows schematically the determination of a path point according to an embodiment;

Fig. 2 shows schematically the determination of a path point according to another

embodiment;

3 schematically shows the sequence of a method for determining a path point according to an embodiment;

4 schematically shows the sequence of a method for determining a path point according to another embodiment, and

5 schematically shows the sequence of a method for determining a path point according to a further embodiment.

FIG. 1 schematically shows the determination of a path point, according to an embodiment of the present invention. A camera system 10 comprising a thermal imaging camera is directed onto a surface 12 that is substantially planar. To determine a path point, a programmer drives his finger 11 over the surface 12, thereby heating this surface 12 locally. The resulting heat trace 13 is characterized by the

Thermal camera detected, and following appropriate track points for movement of a manipulator are derived from this detected heat track. To the

For example, by detecting the thermal track 13 by the thermal camera, the programmer can depress a shutter button (not shown) when it is ensured that the thermal imager does not detect the programmer himself. Alternatively or additionally, the programmer may also pronounce a voice command which is recorded by a microphone (not shown) and triggers the detection of the heat track 13. In order to detect the heat track 13 precisely automatically, an image can be compared, for example by means of the thermal imaging camera, before and after the contact of the surface 12 by the finger 11. The illustration in FIG. 2 essentially corresponds to that of FIG. 1, but there are two objects 23, 24 on the surface 22. In contrast to the representation from FIG. 1, no path is to be based on a heat track, but only two individual path points based on two individual heat points 25, 26 are determined. For this purpose, the programmer can, for example, press a shift key (not shown) with which it is possible to switch between the modes "Detecting a train line" and "Detecting individual track points". Alternatively, such

Switching can also be controlled by means of a voice command which can be recorded by a microphone (not shown). In addition to a thermal imaging camera, the camera housing 20 additionally comprises a 3D camera.

Camera. The programmer touches with his finger 21 each one point on the objects 23, 24, whereby the objects experience a punctual increase in temperature. The thermal imager captures these locally elevated temperatures, or the

Heat points 25, 26. For this purpose, the 3D camera acquires a three-dimensional image, which makes it possible to determine the three-dimensional position of the points touched on the objects 23, 24. Subsequently, two track points are determined based on the data acquired by the thermal imager and the 3D camera.

FIG. 3 shows a method 30 for determining a path point. In step 31, a temperature profile of an environment is detected. Subsequently, in step 32, a track point is determined using the sensed temperature profile.

FIG. 4 shows a further method 40 for determining a path point. In step 41, a first temperature profile of an environment is detected. Subsequently, at step 42, a user, by touching the environment with his finger, creates a heat trail on that environment. In the following step 43, a second

Temperature profile of the environment detected. The skilled person understands that due to the contact of the environment by the user, this second temperature profile may be different from the first temperature profile. In step 44, the first temperature profile is compared with the second temperature profile, and then, based on this, in step 45, the corresponding path point is determined. FIG. 5 shows a further method 50 for determining a path point. In step 51, a thermal image is taken from an environment by means of a thermal imaging camera. In step 52, a heat trace is detected in this thermal image. In parallel, in step 53, a three-dimensional image is taken of the same environment, and in step 54, a three-dimensional structural profile of that environment is determined based on the acquired three-dimensional image.

Subsequently, in step 55, the track point is determined using the heat track recognized in step 52 and the three-dimensional one determined in step 54

Structure profile determined.

LIST OF REFERENCE NUMBERS

10. 20 camera system

11. 21 fingers

12. 22 area

13 heat track

23, 24 objects

25, 26 heat points

Claims

Claims l. A method of determining a path point for a robot program, the method comprising:

Detecting a first temperature profile of an environment (12, 22, 23, 24), and determining the path point using the detected first

Temperature profile.

2. The method of claim 1, wherein the detection of the temperature profile by means of a temperature measuring device, and preferably by means of a thermal imaging camera takes place.

3. The method of claim 1 or 2, further comprising:

Generating, a temperature change (13, 25, 26) on the environment (12,

22, 23, 24), wherein the generation of the temperature change (13, 25, 26) preferably takes place by touching the environment (12, 22, 23, 24) with a finger by a user, and wherein determining the path point preferably comprises determining the temperature change (13, 25, 26) generated on the environment (12, 22, 23, 24).

4. The method of claim 3, wherein generating the temperature change comprises generating a symbolic temperature change, such as crosses, circles, or arrows, and wherein determining the web point preferably comprises associating a predefined robotic action taking into account the sensed first temperature profile.

5. The method of claim 1, further comprising detecting a second temperature profile of the environment (12, 22, 23, 24), and determining the path point using the detected second

Temperature profile is carried out, wherein the determination of the path point preferably comprises a comparison of the first and second temperature profile.

6. The method according to any one of the preceding claims, further comprising: Capturing a three-dimensional image of the environment (12, 22, 23, 24), and

Determining a three-dimensional structure profile of the environment (12, 22, 23, 24) based on the captured three-dimensional image,

wherein the determination of the path point is made using the determined three-dimensional structure profile of the environment (12, 22, 23, 24).

7. The method of claim 6, wherein detecting the three-dimensional

Structure profile by means of a 3D camera is done.

8. The method according to claim 1, further comprising providing the determined path point to a manipulator control, and preferably further comprising controlling a manipulator based on the

certain train point, wherein the control is done by means of the manipulator control.

The method of any one of the preceding claims, further comprising recording a voice command, and wherein determining the web-point further occurs using the recorded voice command, wherein the recording of the voice command is preferably by means of a microphone.

10. The method of claim 1, wherein the environment comprises a substantially planar surface, and wherein the detection of the temperature profile comprises detecting a temperature profile of the substantially planar surface , 22).

11. System for determining a path point for a robot program, comprising:

A temperature measuring device, configured to detect a first temperature profile of an environment (12, 22, 23, 24),

and a controller configured to determine the path point at

Use of the first detected by means of the temperature measuring device

Temperature profile.

12. The system of claim 11, wherein the temperature measuring device is a

Thermal imaging camera includes.

The system of claim 11 or 12, further comprising a camera 5 adapted to capture a three-dimensional image of the environment (12, 22, 23, 24), and wherein the controller is further configured to determine the orbit point using the detected three-dimensional Image, wherein the camera preferably comprises a 3D camera.

14. The system of claim 11, further comprising a microphone configured to receive a voice command, and wherein the controller is further configured to determine the path point using the voice command

recorded voice command.

The method according to one of claims 1 to 10 or the system according to any one of claims 11 to 14, wherein the temperature profile comprises a temperature change (13, 25, 26) generated by a user on the environment (12, 22, 23, 24) ), wherein the temperature change (13, 25, 26) generated on the environment (12, 22, 23, 24) preferably comprises a heat track (13) consisting of a plurality of heat points (25, 0 26).