WO2011051285A1

WO2011051285A1 - System and method for an interaction between a person and a machine

Info

Publication number: WO2011051285A1
Application number: PCT/EP2010/066163
Authority: WO
Inventors: Javier Massanell; Thomas May
Original assignee: Ifm Electronic Gmbh
Priority date: 2009-10-28
Filing date: 2010-10-26
Publication date: 2011-05-05
Also published as: DE102009046107A1

Abstract

The invention relates to a system for an interaction between a person and a machine, in particular a robot, having marking means disposed on the person at certain positions, preferably end positions of extremities or joints, having at least one 3D-TOF camera for three-dimensionally detecting the interaction region and for detecting the marking means, having an analysis unit designed such that, based on the detected marking means, a model of the person is created and a position and a motion of the person is detected and/or prescribed, having a control unit actuating the machine such that, in case of a safety-critical position and/or motion of the person, the person is not placed at risk.

Description

description

SYSTEM AND METHOD FOR AN INTERACTION BETWEEN ONE

PERSON AND A MACHINE

The invention relates to a system and a method for interaction

between a person and a machine, in particular a robot, in which marking means are arranged on the person at certain positions and can be detected by means of a 3D TOF camera. The machine is controlled by a control unit so that the recognized person is not endangered in their safety.

From DE 10 2005 003 827 B4 is already a device and a

Method of interaction between a human and a human

Robot unit at a robot workstation known in which the

Approaching a person is monitored using two safety thresholds. About a first sensor unit, an approach of a person to the workplace is detected such that at a

Falling below a first safety distance, a first signal and a falling below a second safety distance, a second signal is generated. In the presence of the first signal, it is provided to transfer the robot unit in a predefinable state to a so-called safe operating stop. If the second signal is present, the robot unit is transferred to a so-called emergency stop state, in which the robot unit remains motionless in its current position. Furthermore, it is provided that an additional safety switch is arranged at the workstation of the robot unit, which transfers the robot unit into the state of safe operating stop.

From DE 103 20 343 B4 a method for supervised cooperation between a robot unit and a person is known in which after a predetermined program sequence handling steps within a both of the robot unit as well as by humans

accessible shared workspace. In this case, it is provided that the robot unit is impressed on relative approach to humans, a movement pattern that is determined by the direction of movement, movement speed and the distance of the Characterizes robot unit relative to humans. From a variety of different stored patterns of movement a man individual movement pattern is selected so that the people a sense of safety at work and control over the

Robot unit is taught. Each movement pattern is present at least in a first and second quality level, for example, in their movement speed and / or movement direction

differ. Taking into account a current braking distance of the robot unit and a maximum own

Movement speed of man, the robot unit is braked or brought to a standstill until a

Risk of collision is excluded.

Systems for three-dimensional image acquisition are also known from the prior art, which operate with the aid of active illumination. These include so-called time-of-flight (TOF) or

Runtime measurement systems. These use amplitude modulated or pulsed illumination to illuminate the area to be detected

three-dimensional scenery.

With camera system, in particular, all 3D TOF camera systems should be included, which gain a transit time information from the phase shift of an emitted and received radiation. When

3D TOF cameras are in particular PMD cameras with

Photonic mixer (PMD) suitable, as u.a. in the applications, EP 1 777 747, US 6 587 186 and also DE 197 04 496 described and, for example, by the company, ifm electronic gmbh 'as a frame grabber O3D are available. In particular, the PMD camera allows a flexible arrangement of the light source and the detector, which can be arranged both in a housing and separately.

The object of the invention is to further form a system with a 3D TOF camera for an interaction system between a person and a machine.

The object is achieved in an advantageous manner by the invention

Device and the inventive method of independent Claims solved.

Advantageously, the system according to the invention provides for an interaction

between a person and a machine, in particular a robot, before marking means, which are arranged at certain positions of the person, preferably at end positions of extremities or joints. At least one 3D TOF camera is used for

three-dimensional acquisition of the interaction area and for the detection of the marking means. With the aid of an evaluation unit, based on the detected marking means, a model of the person is created and a position and a movement of the person are detected and / or predetermined. Based on the detected and / or predetermined position and movement of the person, the machine is controlled via a control unit such that in a safety-critical position and / or movement of the person, the person is not further endangered.

Conveniently, the marking means as reflectors

formed, it is advantageously ensured that even with strong ambient light, the marking agent can be clearly and reliably distinguished from the background.

Furthermore, it is useful the markers with different

design optical properties. Thus, it is possible to provide markers with particular optical properties for certain positions. For example, a reflector, which is arranged on the head of the person, have a different optical property than the reflectors which are arranged on the shoulder or on the hand.

In principle, it is conceivable for each position its own optical

To provide property.

[001 1] According to a further embodiment, it is provided, a

To equip working clothes of the interacting person with markers at the specific positions.

Furthermore, it is advantageous to provide more than a 3D TOF camera.

The use of multiple cameras allows in particular, the

Increase the security of the collected data. In addition, it is possible with the help of multiple cameras also if necessary by the person or the machine to cover hidden areas.

In a further embodiment, it is advantageous within the

Interaction area at least two, person-independent

Reference marks, in particular reflectors to provide. The location and spacing of these reference marks are known with respect to at least one 3D TOF camera, so on the basis of these

Reference marks the 3D model of a 3 D-TO F camera can be calibrated.

Furthermore, it is advantageous, a method for operating the

the above-mentioned system, in which the person interacting with the machine is detected in its position and movement by marking means which are arranged at certain points of the person, whereby a model of the person is first determined on the basis of the detected marking means, on the basis of which the others

Calculations done.

The method can be further improved by first before the interaction of the person with the machine in a startup phase

Marking the person are recorded. This can ensure that a suitable model for the further calculations can be deposited for the person.

Appropriately, it is provided that the person in the

Starting phase takes certain positions and / or performs certain gestures. This ensures, for example, that all points of the person's joints are recognized and a maximum range of motion of the person is detected.

Advantageously, the interaction with the machine is only released if all expected marking means were clearly identified in the startup phase.

In a further embodiment, it is provided to control the machine in a recognized or predicted safety critical position and / or movement of the person so that a risk to the person is excluded.

Are the recognized or predicted positions and / or Movement of the person within an expected process area, it is advantageous provided a cooperative process flow between

Maintain machine and the person.

In the drawings:

Fig. 1 shows schematically a system according to the invention,

2 shows a machine working area with a cooperating person, FIG. 3 shows a cooperating person within an interaction area, FIG. 4 shows a cooperating person outside the interaction area in the start phase.

FIG. 1 shows a measurement situation for an optical distance measurement with a 3D TOF camera system, as is known, for example, from DE 197 04 496.

The 3D TOF camera system here comprises a transmitting unit or a

Illumination module 100 with a light source 12 and associated beam shaping optics 50 and a receiving unit or 3 D-TO F camera 200 with a receiving optics 150 and a photosensor 15. Der

Photosensor 15 is preferably as a pixel array, in particular as

PMD sensor, designed. The receiving optics typically consist of improving the imaging properties of a plurality of optical elements. The beam-shaping optical system 50 of the transmitting unit 100 is preferably designed as a reflector. However, it can also diffractive elements or combinations of reflective and diffractive

Elements are used.

The measuring principle of this arrangement is essentially based on that, starting from the phase difference of the emitted and received light, the duration of the emitted and reflected light can be determined. For this purpose, the light source and the photosensor 15 via a modulator 18 together with a specific

Modulation frequency applied to a first phase position a.

In accordance with the modulation frequency, the light source 12 transmits an amplitude-modulated signal having the phase a. In the case shown, this signal or the electromagnetic radiation is reflected by an object 20 and hits due to the distance covered in the photosensor 15, the signal of the first phase position a of the modulator 18 with the received signal, which has meanwhile assumed a second phase position b, mixed and from the

resulting signal determines the phase shift or the object distance.

Figure 2 schematically shows a machine workstation with a robot 30 and a cooperating person 300. The robot 30 is here as a gripper or tool robot with multiple articulated arms 32 and

Joints 33 configured. Between the person 300 and the robot 30 is a workstation 500, which is preferably used cooperatively by the person 300 and the robot 30 together. The entire situation in which the robot 30 and the person 300 are located is detected by a 3D TOF camera 10. For better recognition of the

300 person extremities are at preferred locations like

For example, head, shoulder, wrist, arm and knee joint as well as foot markers, preferably reflectors, arranged. The situation captured by the 3D TOF camera is determined by using a

Evaluation unit or image processing 210 processed and evaluated in particular with regard to position and movement of the person 300. The data of the evaluation unit 210 are preferably made available to a control unit 35 of the robot 30. Based on the position, movement and position data of the person 300 may in relation to the known position and movement plans of the robot a

safety-critical situation of the person 300 are detected. In order to reduce the risk for the person or possibly avert it, it may be provided, for example, the speed of

Reduce robot joints when approaching, so that it can be ensured that the robot arm can be safely brought to a standstill at any time.

In a cooperating interaction between the person 300 and the robot 30, predetermined movement patterns are expected of the person 300. Cooperates the person 300 within these movement patterns, holds the Machine upholds the cooperative process flow. However, if the person 300 deviates from this process, the machine goes

optionally in a security mode or evades the movements of the person 300.

Also for the cooperating process flow, it is provided that the maximum speed of the robot is selected so that it comes to a standstill within a predetermined safety limit in case of danger. For example, it may be provided that the speed of the robot is reduced as the distance of the person 300 from the robot 30 decreases.

FIG. 3 schematically shows the interactive situation in a plan view.

By way of example, three 3D TOF cameras 10 are provided in this embodiment, the within their coverage area E the

Detect interaction area I of the machine 30 completely. Of the

Interaction area I of the machine 30 is spanned primarily by the maximum movement possibilities of the robot and, of course, not only comprises a two-dimensional plane, but also a corresponding volume area. In the illustration, the person 300 is at the machine workstation 500 and, for example, can cooperatively provide the work equipment 510 located on the workstation 500 to the robot 30.

Furthermore, 500 are in a peripheral area on the workstation

Reference marks 250 arranged. Place and distance of this

Reference marks are in relation to at least one

3D TOF camera known, so starting from these

Reference mark, for example, a 3D model of a 3D TOF camera can be calibrated. In the case illustrated, the

Reference marks 250 may be formed, for example, as a reflection strip. Of course, the reference marks 250 can also be formed without special reflection properties, it is decisive that these markings 250 can be clearly detected by at least one 3D TOF camera 10. The

For example, reference marks 250 may be in other locations the detection range E are arranged, in particular it is also conceivable that the reference marks 250 at the boundary of

Interaction area I are arranged. So that both location and position as well as a border of the interaction area I are marked.

FIG. 4 shows a possible starting phase of that shown in FIG

Work situation. In the illustrated case, the person 300 is in a predetermined starting position SP. This start position is preferably located outside the interaction area, so that in this phase first no potential danger emanates from the machine 30. Within the starting phase, it is provided according to the invention that the person 300 before the start of an interaction with the system in the

Starting position SP can be seen. Advantageously, the person assumes a particular posture or gesture to facilitate recognition of the markers to the system or to improve the reliability of the recognition. To further improve recognition, it may also be provided to take multiple positions or gestures. After the person has been safely detected and identified by collecting all expected tagging means, the system releases a possible interaction.

Persons without marking agent in the detection area E

intrusion is classified as potentially endangered. Depending on the speed and direction of the penetrating object, safety-related measures are taken up to the emergency stop of the machine.

This procedure ensures that only according

authorized persons, namely those who by certain

Marking agents are marked, can cooperatively interact with the machine. Unauthorized persons or

Unknown objects are classified as a precautionary safety risk.

Claims

claims

A system for interaction between a person (300) and a machine (30), in particular a robot,

with marking means (400) which are arranged on the person (300) at specific positions, preferably end positions of extremities or joints,

with at least one 3D TOF camera (10) for the three-dimensional detection of the interaction area (I) and for the detection of the marking means (400),

with an evaluation unit (210) which is designed in such a way that, based on the detected marking means (400), a model of the person is created and a position and a movement of the person are detected and / or predetermined,

- With a control unit (35) which controls the machine (30) such that in a safety-critical position and / or movement of the person, the person is not endangered.

2. System according to claim 1, wherein the marking means (400) are formed as reflectors.

3. System according to any one of the preceding claims, wherein the

Marking means (400) are designed with different optical properties

4. System according to any one of the preceding claims, wherein a

Workwear of the Interacting Person (300) has marking means at the designated positions.

5. System according to one of the preceding claims, with at least two 3D TOF cameras (10).

6. System according to any one of the preceding claims, wherein within the interaction area (I) at least two, personunabhängige

Reference mark (250), in particular reflectors are mounted.

7. A method of operating a system according to any one of the preceding claims,

in which a person (300) interacting with the machine (30) is detected in its position and movement by means of marking means (400) which are arranged at specific locations of the person (300), wherein on the basis of the detected marking means (400) a model of the person is initially determined, on the basis of which the further calculations are carried out.

8. The method of claim 7, wherein before the interaction of the person (300) with the machine (30) in a start-up phase, first all marking means of the person (300) are detected.

9. The method according to any one of claims 7 to 8, wherein the person (300) occupies certain positions in the starting phase and / or performs gestures.

10. The method according to any one of claims 7 to 9, wherein the interaction with the machine (30) is released in the start phase only when all expected marking means (400).

1 1. The method according to any one of claims 7 to 10, wherein based on the last positions and / or movements of the person (300) taking into account the determined model, the next movement of the person (300) is predetermined or predicted.

12. The method according to any one of claims 7 to 1 1, wherein when a detected or predicted safety-critical position and / or movement of the person (300), the machine (30) is controlled so that a risk to the person (300) is ruled out.

13. The method of claim 7, wherein the machine maintains a cooperative process with the person when the identified or predicted position and / or movement of the person is within an expected process area.