GB2505046A

GB2505046A - Capacitive sensor system having a safety function and an operating function

Info

Publication number: GB2505046A
Application number: GB1310733.9A
Authority: GB
Inventors: Joachim Frangen
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2012-06-29
Filing date: 2013-06-17
Publication date: 2014-02-19
Anticipated expiration: 2033-06-17
Also published as: GB2505046B; GB201310733D0

Abstract

A method for operating a sensor system, having at least one capacitive sensor element 10 which is attachable to the surface of machines 100 or machine parts, preferably robotics, the electric field lines on the sensor element 10 changing upon an approach and/or a contact of a body/object, and the at least one sensor element 10 being connected to a control device 20 which, owing to the detected change of the field lines of the sensor element 10, initiates a safety function on the machine 100. The sensor element 10 having an operating function, e.g. relating to teaching by gestures, in addition to performing the safety function, the operating function is inferred from at least one sensor element 10, and that the safety function has priority over the operating function, in a normal state of the sensor system.

Description

Method for operating a sensor system and sensor system

Prior art

The invention relates to a method for operating a sensor system and to a sensor system according to the preambles of the two independent claims.

Such a method for operating a sensor system and a sensor system are known from the applicant's DE 10 2009 029 021 Al. It comprises at least one capacitive sensor element, the field lines generated by which clement change upon an approach of a body or an objcct, it bcing possible to detect and cvaluatc the change by means of a control dcvicc.

The known sensor system serves, in particular in the case of handling robots or similar machines or devices, upon approach of a person or an (unexpected) approach of an object to a moving machine part, for example a robot arm, for switching the handling device into a safety mode in which the movement of the machine component concerned is stopped. In particular injuries in the event of a collision of the handling device with a person or damage to objects or the handling device can thereby be avoided or at least minimised. The known sensor system preferably comprises a multiplicity of planar sensor elements which are arranged, for example, over the entire surface of a handling device in the manner of a sensor skin and connected to one another by circuitry.

Applications in which handling devices interact with people in a common working space will bccomc increasingly important in thc ftiturc. Such applications arise, for cxamplc, in product manufacture, e.g. in assembly, welding or coating, in the area of so-called service robots (e.g. for fetch and deliver services), cleaning robots, inspection robots, in the area of autonomous vehicles in the surroundings of people, in the area of medical robots working close to people (e.g. for angiography and tumour treatment) or other applications. The ability of such machines to collaborate is based essentially on two basic functions, safe movement in the surroundings of people and intuitive interaction with people. As regards the ability to interact, this is usually accomplished by classical operating elements such as pushbuttons, switches, keyboards, rotary and slide controls, joysticks, computer mice and touchscreens. Also known are speech recognition systems, which react to spoken commands and also optical systems which recognise the movements of an operator (for example in the form of gestures). Furthermore, to guide robots various interaction technologies are known, for example indirect operation via classical operating elements, or eLse direct operation, in which the operator displaces the robot arm with his or her hands (detection of the operator's wish by measuring the motor current via the drive control or by additional torque sensors in the.ioints of the robot arm).

Disclosure of the invention

Starting from thc prior art dcscribcd, thc objcct on which the invcntion is based is to develop a method for operating a sensor system and a sensor system according to the preamble of the respective independent claim in such a way that this is suitable to be used both as a safety system for avoiding collisions of machines or machine components with people or objects, and as a system for interaction between a machine and an operator.

This object is achieved in the case of a method for operating a sensor system having the features of Claim 1 essentially in that the sensor element has an operating function in addition to a safety function, in that an operating function is inferred from the location of the at least one sensor element and the time profile of the change of the field lines, and in that the safety function has priority ovcr the operating function, in a normal statc of the scnsor system. In othcr words, this means that onc and thc samc sensor ccmcnt of a sensor system can be used in different ways, as an initiator for a safety function and as a switching element for an operating function. For safety reasons, it is provided here that the safety function has priority over the operating function, in a normal state of the sensor system. This means that, for example, it is not permitted to perform any gestures in the immediate vicinity of the moving machine or the moving robot. Should an operator attempt this, the safety function will slow down the machine or the robot safely to a standstill. Only after that is the operator able to call the desired function with a gesmre. In particular, injuries to people and damage to machines or objects are thereby safely avoided.

Advantageous developments of the method according to the invention for operating a sensor system and of a sensor system are set out in the respective subclaims. All combinations of at least two features disclosed in the claims, description and/or figures are within the scope of the invention. In this regard, features disclosed in respect of the method are deemed to be disclosed and claimable in respect of the apparatus, and features disclosed in respect of the apparatus are deemed to be disclosed and claimable at the same time in respect of the method.

To simpliI the operation of a machine or, for example, a robot, it can be provided that the safety function can be tcmporarily put out of operation by actuating a safety switch.

This means that by intentionally pressing a safety switch the machinc or robot is immediately switched into an operating mode in which an approach of a person or an object to a sensor element is not classed as safety-relevant, so that, for example, the movement of the machine or robot is not stopped. In this operating state, the operator assumes the responsibility for the safe operation of the machine or the robot system.

Preferably, the safety switch is designed here such that, if it is not actuated or pressed by an operator, it switches back into its original switching state, in which the sensor clement of the machine or robot is in its safety mode.

A further advantageous configuration of the invention provides that the safety function has three different switching states, in dependence on the detected distance of a body or an object from the machine or the machine part, that in the first switching state called close range, which is assigned to a small distance (for example less than 20 cm) of the body or object to the machine or the machine part, the movement is completely stopped by the safety function, that in a second switching state called medium range, which is assigned to a medium distance (for example between 20 cm and 40 cm) of the body or object to the machine or the machine part, the movement is limited to a safe speed Vi by the safety function, and that in a third switching state called far range, which is assigned to a great distance (for example more than 40cm) of the body or object to the machine or the machine part, the movement is limited to a safe speed V2, which may be greater than the speed Vi, by the safety function. This means the following: in the close range, the safety function ensures safe stopping, so that there is no longer any danger to the operator from the movement of the machine or the machine part and the operator can perform any gestures directly in front of the sensor elements. In the medium range, it is possible only to perform gestures in which the hand approaches no closer than up to a minimum distance of, for example, 20 cm to the moving machine or the machine part.

This switching state can, for example, be utilised to guide a robot contactiessly with one hand in that the robot regulates its position such that its distance and orientation to the hand always remain the same. In the far range, there is no restriction to the performance of gestures.

In a simplified configuration, the mcdium rangc may bc dispensed with, so that therc is only a switching threshold of, for cxamplc, 20 cm which scparatcs thc closc range from the far range.

A fhrthcr advantageous configuration of the method according to the invention provides that the operating function has a p'urality of operating modes, a first operating mode relating to a teaching of a gesture by the sensor system, and a second operating mode relating to a performance of a gesture, taught in the first operating mode, by the machine or a machine part. The first operating mode thus means, for example, that a particular gesture is repeated a plurality of times in succession by an operator, and that the sensor system stores the sensor values detected in each case during the performance of the gesture. From the stored sensor values, it is possible to determine, for example, an "average" signal profile for a particular gesture which, subject to tolerance values, also enables the recognition of gestures which differ from this "average" gesture, but are obviously intended to ithtiate the same function by the control device. As soon as the teaching process or the first operating mode is completed, the sensor system works in the second operating mode. If a gesture is now performed, the control device compares the average signal profiles of stored gestures stored in the control device with the currently detected signal profile of a gesture. If a correspondence is found, a particular gesture is inferred, which results in the control device of the machine or thc robot initiating a particular action, e.g. a screwing process or the like.

A sensor system according to the invention is distinguished in an advantageous configuration in that in the operating mode a plurality of sensor elements form an opcrating pancl, the input values, dctectcd by thc control dcvice, of thc scnsor clemcnts being evaluable by means of a computer unit.

It is very particularly prefcrrcd in this case if the sensor elements forming an operating panel constitute a subset of all the sensor elements arranged on the machine or the machine part, aild if the subset of the sensor elements is selectable from the universal set of the sensor elements. This means the following: a machine or a machine component is covered with a multiplicity of sensor elements. For example, a matrix of 5x5 scnsor clements arranged beside one another forms a operating panel. This operating panel can be configured at any place on the machine surface by a corresponding activation of corresponding sensor elements by the control device. This means that, for example, the opcrating panel conccrned can be formcd either on a robot arm or in the region of a support clcment. By means of such a configuration of the invention, it is possibc to freely adapt an operating panel of the sensor system to a wide variety of machines or machine components and also applications, without having to make additional hardware arrangements to do so. The assignment of the sensor elements to the operating panels takes place purely by software.

According to the invention, it can be provided that the at least one sensor element acts, in the operating modc, as a digital switch or as an analog switch, in thc manncr, for cxample, of a slide control. This means that, in the opcrating modc, for example when a particular threshold value at the sensor element is exceeded, the control device switches a particular machine element on or off, irrespective of the value by which the threshold value has been exceeded. Alternatively to this, however, the absolutc level of the detected measured value of the sensor element may be used, for example be converted into a corresponding speed signal for operatillg a machine element, so that the level of the sensor signal corresponds to a particular analog value (for the speed of the machine component).

In order to send an operator an acknowledgement as to whether, for example, a gesture performed by the operator has been correctly detected or interpreted by the sensor system, it can be provided in a further configuration of the invention that at least one visual, acoustic or other indicating element is provided, by means of which the control device outputs a signal in the operating mode. For example, this indicating element may consist in a monitor, which signals as an acknowledgement by a corresponding visual indication to the operator, the detected gesture or a particular operator's wish.

Alternatively to this, the operator can be informed, for example by an acoustic signal, that the control device has without doubt recognised a certain gesture or else if a gesture has not been recognised. Other kinds of elements, for example LEDs, may serve to provide such an acknowledgement to the operator.

It is, furthermore, very particularly preferred if the at least one sensor element, acting in the operating mode, on the machine or the machine component is visually marked. This signals to the operator the place at which he or she, for example, is to perform a gesture in order that this gesture can be recognised as simply and clearly as possible by the sensor system. For example, it is conceivable to carry out such marking by means of an illumination which is reeognisable only in the operating mode of the machine. However, printed-on markings or the like may also be used in order that the operator recognises the region of the sensor elements via which an input, for example, of gestures is enabled.

Further advantages, features and details of thc invention will become apparent from the following description of preferred exemplary embodiments and with the aid of the drawing.

In the drawing: Fig. 1 shows a schematic representation of a handling robot during the transfer of an object by an operator, Fig. 2 shows the basic structure of an interaction system, in which an operating panel constructed from 24 sensor elements serves for recognising a gesture, and Fig. 3 shows a representation for elucidating an exemplary mode of operation of a virtual operating element for detecting a key depression.

Like elements or elements having the same function are provided with the same reference numerals in the figures.

In Fig. 1, a machine in the form of a handling robot 100 is shown. The handling robot comprises a machine base 101 which carries a robot arm 103 rotatably mounted on a first axis 102. The robot arm 103 has three arm sections 104 to 106 which in turn are pivotably mounted on respectively one axis 107 to 109 in the direction of the double arrows shown. The arm section 106 carries a grasping mechanism 110 for grasping an object 1. Such a handling robot 100 as described so far is suitable for picking up the object I at any location and depositing it at any second location, within the range of the robot arm 103.

At least regions of the handling robot 100 are covered by a sensor skin (not shown specifically) having sensor elements 10 which are each designed as capacitive sensor elements 10. The sensor elements 10 form a sensor system. Preferably, the sensor elements 10 are each of the same design, but they may also be of different design or of different size. As regards a basic structure of such a sensor element, reference is made to the applicant's DE 102009 029 021 Al which to that extent is to form part of this application.

The sensor elements 10 are connected to a control device 20 of the handling robot 100.

The respective locations of the sensor elements 10 on the handling robot 100 are stored in the control device 20 of the handling robot 100. By way of example, some of the sensor elements 10 are, according to the invention, used so that the control of a frmnction or an operation of the handling robot 100 can be effected by them. For this purpose, several virtual operating panels 11 to 14 are generated on the surface of the handling robot 100, the sensor elements 10 assigned to the individual operating panels 11 to 14 being freely assignable, for example via an input unit (not shown), as a subset of all the sensor elements 10 arranged on the handling robot 100. The operating panels 11 to 14 described below are intended here merely to represent possible variants, their number and design being greatly variable.

The first operating panel ills arranged in the region of the third arm section 106 of the robot arm 103 on a lateral surface and acts as a digital switching element, so that when, for example, the hand of an operator 30 approaches, the grasping device 110 is opened or closed. The operating panel 12 is arranged on the second arm section 105 of the robot arm 103 and comprises, by way of example, six sensor elements 10 which serve for sequential control of the operation of the handling robot 100, so that when, for example, the hand of the operator 30 approaches a particular sensor element 10 in the operating panel 12 of the handling robot 100, a stored operating program starts or is executed. The operating panel 13 is arranged on the first arm section 104 of the robot arm 103 and serves as a virtual proximity surface for guiding the robot arm 103. In particular when an identical operating panel 13 is arranged on the robot arm 103 on the opposite side of the fir st arm section 104 parallel to the plane of the drawing of Fig. 1, the approach, for example, of the hand to one or the other operating panel 13 causes the robot arm 103 to rotate on the first axis 102 in the desired direction, in order, for example, to teach a particular movement sequence of the robot arm 103. Finally, an operating panel 14 is arranged on the machine base 101, which panel is in the form of a virtual keyboard for data input, and comprises a multiplicity of sensor elements 10 arranged beside and above one another.

To make the operating panels 11 to 14 easily recognisable to the operator 30, provision may be made to highlight the operating panels 11 to 14 by corresponding markings or visual aids.

The sensor elements 10 in the operating panels 11 to 14 shown in Fig. 1, as well as the sensor elements 10 (not shown in Fig. 1) which are distributed over the surface of the handling robot 100 and not arranged in the operating panels 11 to 14, serve to stop the movement of the handling robot 100 during operation of the handling robot 100 in the so-called safety mode when a collision of the handling robot 100 with a person or an object is imminent, in order to avoid injuries or damage. For this purpose, the control device 20 evaluates the input signals produced by the individual sensor elements 10.

Besides the safety-relevant function of the sensor elements 10, the latter serve, as explained above, in the operating panels 11 to 14 to interact with the operator 30 (operating mode).

The safety mode has priority over the operating mode. This means that, for example, when the operator 30 approaches the handling robot 100, which is detected by the sensor elements 10, firstly the speed of the robot arm 103 is reduced to a safe value, so that contact with the moving robot arm 103 is precluded. Thereafter, the sensor elements 1 0 arranged inside the operating panels 11 to 14 are ready to recognise gestures of the operator 30. If the robot at any time receives a command to travel towards an object or a person, to go below the preset safe distance or to exceed the maximum speed corresponding to the distance, the execution of the command is prevented by the priority of the safety mode.

Manual pressing of an optional safety switch 15, which is connected to the control device 20 of the handling robot 100, by the operator 30 causes the approach of a person or an object to be classed as not safety-relevant, while the recognition of gestures continues to be active. In this operating state, the operator assumes responsibility for the safe operation of the robot system.

As along as the safety switch 15 is pressed, the control device 20 switches the handling robot 100 or its sensor elements 10 from the safety mode into the operating mode, in which, in particular the sensor elements 10 arranged inside the operating panels 11 to 14, serve for recognising gestures of the operator 30 or inputs. It may also be provided that, when the operator 30 approaches the sensor elements 10, the control device 20 firstly switches into a mode in which the sensor elements 10 in the region of the operating panels 11 to 14 serve, for example, for recognising gestures, while at the same time the movement of the handling robot 100 takes place with reduced speed in comparison with the normal operation. Upon further approach of the operator 30 to the sensor elements 10, for example, to below a certain minimum distance of e.g. 10 cm, the control device switches the handling robot 100 into the safety mode, in which any movement of the handling robot 100 or its robot arm 103 is stopped.

In order to send the operator 30 an acknowledgement about inputs made by the operator via the operating panels 11 to 14, a visual, acoustic or other indicating unit 16 can be provided, which receives a corresponding signal from the control device 20 when, for cxample, a particular gesture of thc opcrator 30 has been correctly (or not) recogniscd, or when, for exampic, a gcsturc is to be repeated.

In Fig. 2, an operating panel 18 consisting of twenty-four sensor elements 10 is shown.

The sensor elements 10 are arranged in the form of a 4x6 matrix in the operating panel 18 and are connected to a central processing unit 19, which is part of a computer unit.

The operating panel 18 thus formed detects the gestures of the operator 30. For this purpose, each of the sensor elements 10 delivers a measured value D1 which varies with the distance of the finger of the operator 30 from the respective sensor element 10. The central processing unit 19 performs a cyclic interrogation and processing of the measured values D of all the sensor elements 10 and their output via an interface. A control computer 25 with an integrated software interaction module 27, which is connected to the central processing unit 19, reads in the measured values D of all the sensor elements as a time series and evaluates them, by way of example, in the following manner: -preprocessing of the measured values D -recognition of taught patterns in the data, e.g. by means of correlation ifmnctions -classification of the pattems according to operating elements and gestures -assignment of the recognised gestures to commands and -command execution in step 28 (e.g. by calling functions, outputting signals).

In Fig. 3, by way of example, the mode of operation of a virtual operating element having six sensor elements 10 arranged beside one another is shown. Here, the operating element serves for detecting a key depression. The index finger of an operator 30 is moved here in the X-direction of the arrow shown. The diagram shown below the six sensor elements 10 shows sehematicafly the measured-value profile of all six sensor elements 10, dependent on the X-position of the finger. Each sensor element 10 has a characteristic similar to a Gaussian curve, with the Gaussian curves of adjacent sensor elements 10 typically overlapping one another. From the six measured values D1 to D6, with the aid of interpolation functions, the X-position of the finger of the operator 30 can be derived. Here, the resolution of the finger position is not dependent on the width of the sensor element 10 or the width of the finger.

If adjacent sensor elements 10 are present, different finger positions can be detected by interpolation also within a sensor element 10. This measurement principle is applied to two-dimensionally arranged sensor elements 10, in order to detect the finger position in both spatial directions (i.e. parallel and perpendicular to the plane of the drawing of Fig. 3). In similar fashion, the distance of the finger of the operator 30 can also be detected by means of characteristic sensor data, so that the spatial position of the finger of the operator 30 can be determined.

If gestures are to be recognised by means of the sensor elements 10, these are transformed into a time sequence of sensor data owing to the input signals of the sensor elements 10. One task of the control computer 25 shown in Fig. 2 is to perform an inverse transformation, i.e. to recognise gestures in the time sequence of sensor data.

Furthermore, the module 27 arranged in the control computer 25 has the two operating modes of "set up" and "perform". In the "set up" operating mode, the virtual operating elements (operating panels 11 to 14) are set up by an operator 30 before the handling robot 100 is put into operation. This is carried out, for example, in the following steps: a. visualisation of the operating elements (operating panels 11 to 14) on the surface of the handling robot 100, e.g. by stickers or in the form of LEDs delimiting the operating panels 11 to 14.

b. "demonstration" of the gesture at the operating element by the operator 30. Here, the gesture is stored as a time sequence of characteristic sensor data.

Subsequently, reliable variations of thc gesture arc also demonstrated.

c. teaching of the gesture: The stored sensor data are reduced to characteristic features. With these features, a teachable classificator is trained. The classificator is capable of handling multielasses, i.e. can recognise all taught gestures again and separately assign them. In the case of analog gestures, it is able to determine the analog value of the gesture (e.g. the distance between the hand of the operator and a sensor element 10).

d. linking of thc gestures to flinctions: The operator 30 assigns to each taught gesture a frmnction or sequence of functions which are to be automatically performed upon recognition of the gesture. This may, for example, be a functional sequence on the control computer 25 or a command which is sent to peripheral units via a communication interface.

In the "perform" operating mode, the measured values of the sensor elements 10 are continuously cyclically read in and the characteristic features calculated. The previously taught classificator continuously checks whether a taught gesture has been performed. If yes, the control device 20 initiates the performance of the function linked thereto.

The method described so far for operating the sensor system consisting of the sensors 10 can be modified in a variety of ways, wfthout departing from the inventive concept. In particular, the use of such a sensor system is not necessarily limited to the use in handling robots 100, but may also be employed in other machines or machine components.

Claims

Claims 1. Method for operating a sensor system, having at least one capacitive sensor element (10) which is attachable to the surface of machines (100) or machine parts, the electric field lines on the sensor clement (10) changing upon an approach and/or a contact of a body or of an object, and the at least one sensor clement (10) being connected to a control device (20) which, owing to the detected change of the field lines of the at least one sensor element (10), initiates a safety function on the machine (100) or the machine part, characterised in that the at least one sensor element (10) has an operating function in 1 0 addition to the safety function, in that an operating function is inferred from the location of the at least one sensor element (10) and the time profile of the change of the field lines, and in that the safety function has priority over the operating function, in a normal state of the sensor system.
2. Method according to Claim 1, characterised in that the safety function can be temporarily put out of operation by actuating a safety switch (15).
3. Method according to Claim 1 or 2, characterised in that the safety function has at least two different switching states, in dependence on a detected distance of the body or the object from the machine (100) or the machine part, in that in each switching state which is assigned to a distance range of the body or object to the machine (100) or the machine part, the safety function safely limits the maximum speed, and in that simultaneously the operating function is enabled.
4. Method according to one of Claims I to 3, characterised in that the operating function has a plurality of operating modes, a first operating mode relating to a teaching of a gesture by the sensor system, and a second operating mode relating to a performance of a gesture, taught in the first operating mode, by the machine (100) or a machine part.
5. Sensor system, in particular suitable for carrying out a method according to one of Claims I to 4, having at least one capacitive sensor element (10) which is attachable to the surface of machines (100) or machine parts and is connected to a control device (20), characterised in that the control device (20) is designed to switch the machine (100) or a machine part into a safety mode or an operating mode, the control device (20) switching the machine (100) or the machine part into a safety mode in a normal setting.
6. Sensor system according to Claim 5, characterised in that in the operating mode a plurality of sensor elements (10) form an operating panel (11 to 14, 18), the input values, detected by the control device (20), of the sensor elements (11 to 14, 18) being evaluable by means of a computer unit (19, 25, 27) in order to recognise a gesture.
7. Sensor system according to Claim 6, characterised in that the sensor elements (10) forming an operating panel (11 to 14, 18) are a subset of all the sensor elements (10) arranged on the machine (100) or the machine part, and in that the subset of the sensor elements (10) is formed from the universal set of the sensor elements (10) by assignment.
8. Sensor system according to one of Claims 5 to 7, characterised in that the at least one sensor element (10) acts, in the operating mode, as a digital switch or as an analog input element, for example in the form of a slide control.
9. Sensor system according to one of Claims 5 to 8, characterised in that at least one visual, acoustic or other indicating unit (16) is provided, by means of which the control device (20) outputs a signal in the operating mode.
10. Sensor system according to one of Claims S to 9, characterised in that the at least one sensor element (10), acting in the operating mode, on the machine (100) or the machine component is visually marked.
11. Method for operating a sensor system, substantially as hercinbefore described with reference to the accompanying drawings.
12. Sensor system substantially as hereinbefore described with reference to the accompanying drawings.