EP3444079B1

EP3444079B1 - Method and robotic system for operating a hand-guided robot

Info

Publication number: EP3444079B1
Application number: EP17186599.1A
Authority: EP
Inventors: Holger Mönnich
Original assignee: Siemens Healthcare GmbH
Current assignee: Siemens Healthcare GmbH
Priority date: 2017-08-17
Filing date: 2017-08-17
Publication date: 2022-02-16
Anticipated expiration: 2037-08-17
Also published as: EP3444079A1; US20190054619A1; US11148285B2

Description

The invention relates to a method for modelling an external object with a hand-guided robot and to a corresponding robotic system. The invention further relates to a storage medium containing a corresponding program code and a corresponding data processing device.
It is common practice today to model a behaviour, that is, a movement and a resulting position or pose of a robot using a mathematical model. These models are, however, not perfectly accurate in all situations or scenarios. Typically, robots or the corresponding models are calibrated in a setting where the robot or at least its arm can freely move in all directions to cover a complete range of motion and all possible poses. A pose in this sense as well as in terms of the present invention describes a position as well as an orientation. Some models might take into account the essentially static influence of gravity, which can induce a moment, in particular a moment of force or a torque in the robot, in particular in a joint of its arm. Even these models do, however, not always reliably reproduce the actual position of, for example, the respective robot end effector.
US 2016 / 0 221 189 A1 discloses a method for determining a compliance property associated with a selected axis of a manipulator. Therein, a movable part of the manipulator is clamped to a point in space. While the clamped configuration is maintained, an excitation joint set is actuated, and a quantity related to joint torque and joint position is monitored. The compliance property of the selected axis is then determined based on the monitored quantity.
EP 2 383 624 A2 discloses an apparatus for detecting a contact position where a robot makes contact with an object. The apparatus comprises a probe that is attached to the robot and configured to make a displacement in a direction of making contact with the object in an elastic manner.
US 2016 / 0 089 789 A1 discloses a robot system which performs a machining process for a workpiece by using a machining tool. Therein, the robot is operated along a predetermined motion trajectory, and a target position of an acting point of the machining tool or the workpiece is obtained such that a force acting between the machining tool and the workpiece approaches a predetermined value. A position correction amount of a motion trajectory of the robot and of the actuator is calculated in response to the target position of the acting point.
US 2008 / 297092 A1 discloses a control apparatus for an articulated robot that performs machining operations such as friction welding or grinding on a workpiece.
It is an objective of the present invention to provide means to achieve for an improved positional accuracy of a robot and corresponding uses.
This objective is achieved by the objects of the independent claims. Advantageous embodiments with expedient developments of the present invention are indicated in the other patent claims as well as in the following description and the drawings.
A method according to the present invention is concerned with modelling an external object with a hand-guided robot having a jointed arm and an end effector connected to one end of the arm, wherein the jointed arm has at least one joint. As part of this method a total moment affecting the robot is measured at each joint of the arm while the robot is being hand-guided and subjected to a mechanical tension due to contact with an external object. The external object can for example be a stationary part of an apparatus on which the robot is working. The mechanical tension or tensioning can be a mechanical stress or pressure influencing, in particular hindering, limiting, or restricting a movement or motion of the end effector, the jointed arm, or a part thereof.
Another step of the method comprises determining at least one external moment caused by the mechanical tension and/or a force exerted by a user for hand-guiding the robot (2) and affecting the robot based on the at least one measured moment. The measured moment as well as the external moment may be moments of force, in particular a respective torque.
Another step of the method comprises computing a positional correction factor for a position of the end effector based on the at least one external moment and a simplified internal model of the robot, in particular an absolute accurate model of the robot, wherein the simplified model assumes that the robot can move freely and unhindered,.
Another step of the method comprises computing a current position of the end effector based on a correction model in dependence on the positional correction factor while the robot (2) is still in contact with the external object. The positional correction factor is, in other words, used to specify (and/or minimise) a difference between a calculated or modelled, that is, an assumed position of the end effector and its objective, actual position.
Other steps of the method are: determining the position of the external object based on the computed current position of the end effector, moving the end effector sequentially into contact with the external object in multiple different contact points, computing the respective current position of the end effector in each of the multiple different contact points, and virtually modelling the external object based on the multiple computed respective current positions.
A hand-guided robot is a robot that can be moved or positioned manually. This refers in particular to a position and/or orientation, that is, a pose of the end effector. To allow this hand-guided positioning of the robot or the end effector, the robot features a corresponding specialised control system or feedback control. This enables the robot to follow the manual positioning or guidance in a consistent and jitter free manner. Nevertheless, the robot may be motorised and can comprise its own drive system or system of one or more actuators. If the robot is thus equipped, a user can advantageously guide, that is, move the end effector very precisely and jitter free since the positioning of the end effector can be assisted by the robot. This can be especially advantageous when the robot itself and/or a load the robot is carrying is particularly heavy so that it could ordinarily not be moved easily or precisely by an unassisted user.
Since it cannot be precisely foreseen how a user might guide or interact with the robot, it is in practice not possible to precisely model or predict all forces, moments, and interactions that can occur during operation of the robot. If, for example, a user brings the end effector or a part of the jointed arm in contact with a stationary external object, the robot might not change its position and/or orientation as would be predicted or modelled by a typical conventional model that does not take into account external moments. In conventional models and robotic systems this can lead to significant positioning errors of the robot as a whole and of the end effector in particular. These error can be on the order of several millimetres or even centimetres. The position of the end effector calculated by such an internal model of the robot can therefore differ significantly from its objectively measured actual position with respect to a surrounding reference frame such as a fixed world coordinate system.
It is therefore a particular advantage of the present invention that the mechanical tension and the external moment or moments are taken into account in determining the current position of the end effector since this results in the computed current position more closely matching the objective actual position of the end effector. The improved precision of the computed current position of the end effector is in itself an obvious advantage but can also advantageously form a basis for any number of other applications or uses cases that rely on precise knowledge of the current position of the end effector. A more precise knowledge of the current position of the end effector can, for example, enable a higher degree of automation in complex robot assisted tasks and/or lead to a lowered risk of damaging the robot, a tool or object handled or manipulated by the robot, and/or a surrounding of the robot.
By measuring the acting or effective moment at each joint or axis of the jointed robot arm the external moment or moments can be quickly, easily, and precisely determined from or based on the corresponding measured, that is, the actual values.
The end effector is - in accordance with the usual sense of the word in the field of robotics - the last element, member, or device in a kinematic chain comprising the jointed arm as well as the end effector itself. The end effector can be attached to the robot arm via a robot flange. The jointed arm and the end effector can together also be denoted as a robot manipulator or a manipulator of the robot. The end effector can for example be a gripping device or a tool fitted to or held by a corresponding receptacle. The end effector and the robot flange can be located or arranged at a distal end of the jointed arm. The other end of the jointed arm can be formed by or connected to a foot or base of the robot. This foot or base can in particular be stationary with respect to a surrounding of the robot, while the manipulator, that is, the jointed arm and the end effector, can move around with respect to the foot or base of the robot. The position of the end effector can also be denoted as or refer to a tool centre point (TCP) or a position of this tool centre point.
The external moment can effectively, partly or in total be caused or influenced by the mechanical tension or tensioning and/or by a force exerted by a user for guiding the robot.
Computing the positional correction factor and/or the current position of the end effector can comprise one or more calculations and/or one or more evaluations each. It can also comprise automatically retrieving any needed parameters or respective current values, such as for example a respective joint parameter for each joint of the jointed arm, and/or reading corresponding values from a predetermined provided table or characteristic diagram. The respective computing or computation can advantageously be carried out or executed automatically. It can be especially advantageous if this is done continuously or with a predetermined frequency so that the respective current position of the end effector is computed and therefore known while the robot is moving or being operated. To enable the automatic computing or computation or determination of the at least one external moment, the positional correction factor, and/or the current position of the end effector, a respective predetermined model can be provided.
While established conventional methods and the corresponding models sometimes consider essentially static or constant effects such as Earth's gravity, the present invention can advantageously also take into account dynamic or situational influences or effects. In an advantageous development of the present invention the measured moment includes effects of a centrifugal force and/or a Coriolis force affecting the arm and/or the end effector. These forces or effects can contribute to or influence the current position of the end effector. Taking them into account for computing the current position of the end effector can therefore advantageously result in an improved precision or accuracy. If, for example, the end effector is guided by a user in a horizontal arc, this motion can result in a measurable moment essentially perpendicular to the motion induced by the user. Measuring this moment, which can stretch out the arm, and taking it into account through the positional correction factor in computing the current position of the end effector leads to a more precise reflection or representation of reality by the computed values or coordinates. Since the sizes or values of these forces or effects depend on not only a path along which the robot is guided but also a speed or velocity with which the robot is being guided along its path they are considered dynamic influences or effects.
In an advantageous development of the present invention the end effector or a tool attached thereto is brought into contact with the external object or a second external object and the position of the external object or the second external object, respectively, is determined based on the computed current position of the end effector. By - intentionally or unintentionally - bringing the end effector into physical, that is, into mechanical contact with the respective external object and computing the current position of the end effector while it is still in contact with the external object the robot can, in other words, effectively be used as a spatial mapping system. Determining the position of the external object based on the current position of the end effector can for example be done through a simple mathematical operation by adding an object vector to the computed current position of the end effector. This object vector can be predetermined based on an exact point, such as the tool centre point, of the end effector used as a reference or substitution for the position of the end effector as a whole.
The object vector can also depend on an exact physical form or shape of the end effector which determines where the end effector comes into contact with any external object. If, for example, a position of a centre point of the end effector is used to describe the position of the end effector as a whole, then this centre point cannot necessarily come into direct contact with the external object since the centre point can be surrounded by material of the end effector itself.
The described approach for determining the spatial position of the external object using the robot is an especially advantageous development or extension of the previously described method for computing the current position of the end effector, because the effects resulting from the physical contact between the end effector and the external object are themselves used for determining the position of the external object. Since the robot is or can be hand-guided into contact with the external object, a precise point of the external object where its position is determined can be precisely and easily chosen. It can advantageously also be possible to bring the end effector into contact with the external object at a contact point that is - from an outside perspective - at least partly hidden inside the external object or behind another part of the external object or behind another external object. Therefore, the presently described method offers a distinct advantage over optical systems which are limited to or by their line of sight. The presently described approach can advantageously be used to map a position, shape, or form of a complexly shaped external object. This can advantageously be done independently of a lighting condition and a background noise, which is an advantage of the present method over optical and ultrasound systems.
In another step of the present invention the end effector or the tool is sequentially brought into contact with the respective external object in or at multiple different contact points, that is, at multiple different points of the external object. This is not to be confused with multiple contacts or contact points occurring at for example the tips of multiple individual finger elements of the end effector while gripping the external object when the end effector comprises a multi-finger gripping device. Rather, a single point of the end effector can be brought into contact with the external object, then moved away from the external object and brought into contact with the external object again at a different contact point.
It can also be possible to slide the end effector along a surface of the external object from one contact point to another.
The respective current position of the end effector is then computed in each contact point, that is, for each instance or occurrence of the contact between the end effector and the external object. The respective external object is then virtually modelled based on the multiple computed respective current positions. This means, that by touching the end effector to the external object at different contact points and computing the respective current positions, a shape of the external object can be determined and modelled in three dimensions. This is enabled by the precise determination of the current position of the end effector under and taking into account a mechanical tension and/or external moment. Preferably, the end effector or the tool is brought into contact with the external object at more than two, in particular at more than three, different contact points. A higher number of contact points and respective computed current positions enables a more precise and detailed virtualisation, that is, virtual modelling of the external object.
In a further advantageous development of the present invention the external object or a second external object is held by the end effector. The current position of the end effector is then continuously computed. A path of the held external object is then determined or computed based on the continuously computed current positions of the end effector to track a position and/or a movement of the external object. Since the robot can be hand-guided, meaning that the external object can be individually moved through space on an essentially arbitrarily complex path by guiding the robot correspondingly, this approach provides a fast, simple, and effective way of recording or virtualising the path of the external object. Through this method the robot can, in other words, be used as a tracking or motion capturing system. This has the advantage that no optical system is needed and therefore no line of sight has to be considered. Another advantage is that the path of the external object and therefore the movements of the robot do not have to be programmed in advance, significantly lowering a required time and effort. Since the robot can hold the external object jitter free over arbitrarily long periods of time this approach can especially advantageously be combined with an augmented or virtual reality system to precisely track and position the external object in real space and in virtual space.
Another aspect of the present invention is a storage medium containing a program code designed or configured to perform a method in accordance with at least one embodiment of the method according to the present invention on execution of the program code by a processing unit.
Another aspect of the present invention is a data processing device comprising a processing unit and a storage medium according to the present invention. The processing unit is configured to execute the program code contained within the storage medium. The data processing device may also comprise an input interface for receiving the values of the measured moment as well as, for example, data about a current configuration, a current rotation or state of any joint and/or axis of the robot and/or a control signal sent from a control unit (ECU) to at least one drive or motor of the robot. The data processing device may also comprise an output interface for providing the computed correction factor and/or the computed current position or corresponding coordinates of the end effector and/or the external object to another device or system.
Another aspect of the present invention is a robotic system comprising a hand-guided robot having a jointed arm and an end effector connected to one end of the arm, wherein the jointed arm as at least one joint. The robotic system further comprises a moment sensor, in particular a torque sensor, in or at each joint and/or axis of the arm. These moment sensors are configured to measure a respective moment at each joint and/or axis of the arm, in particular while the robot is being hand-guided and/or subjected to a mechanical tension due to contact with an external object. The robotic system further comprises a data processing device in accordance with the present invention connected to each moment sensor via a respective data link or data connection.
In particular, the data processing device and the robotic system can be configured to execute or perform at least one embodiment of the method according to the present invention.
The data processing device and the robotic system may comprise one or more computer- or microchips for executing the respective program code.
In a further advantageous development of the robotic system the robot is an industrial robot having at least six degrees of freedom (6-DOF).
An example of a tracking system for tracking, in particular for recording, a position and/or a movement of an external object, comprises a robotic system in accordance with the present invention, wherein the program code contained in the storage medium of the robotic system is designed or configured to perform at least one embodiment of the method according to the present invention when executed by the data processing device of the robotic system.
The embodiments and developments of the present invention described herein for at least one aspect of the present invention, that is, for the method, the storage medium, the data processing device and the robotic system as well as the corresponding advantages may be applied to any and all aspects of the present invention.
Further advantages, features, and details of the present invention derive from the following description of preferred embodiments of the present invention as well as from the drawings pertaining to the present invention.
In the drawings

FIG 1: schematically shows a perspective view of a robotic system; and
FIG 2: schematically illustrates an exemplary flow chart of a method for operating the robotic system shown in FIG 1.

FIG 1 schematically shows a perspective view of a robotic system 1. The robotic system 1 comprises a robot 2 and a data processing device 3 which is connected to the robot 2 via a data link. While the robot 2 and the data processing device 3 are shown separately, the data processing device 3 can also be integrated into the robot 2.
In the present example the robot 2 is an industrial robot with a foot or base 4 which can be mounted in a fixed location such as for example a floor of an industrial workspace. Attached to the base 4 is a manipulator 5 of the robot 2. The manipulator 5 is formed by several links 6 and several joints 7 connecting the links 6. Each joint 7 enables rotation of a respective connected link 6 about a corresponding axis 8. In other examples different types of joints or connections of the links 6, such as for example prismatic joints, can also be used as part of the manipulator 5 or the robot 2. The links 6 and joins 7 form a robotic jointed arm 9. As part of the manipulator 5 and arranged at a distal end 10 of the arm 9 the robot 2 further comprises an end effector 11. The end effector 11 can be connected to the arm 9 via a robot flange. Arranged in or at each of the joints 7 or the corresponding axes 8, respectively, is a respective moment sensor 12. Only some of these moment sensors 12 are shown here.
The end effector 11 comprises a gripping or holding mechanism or device which in this example is holding a rod or probe 13. Since the 13 is rigidly held, it can also be considered part of the end effector 11 since it effectively is the last part or element of a kinematic chain that is formed by the manipulator 5 and begins at the base 4.
FIG 2 schematically shows an exemplary flow chart 25 illustrating a method for operating the robotic system 1 shown in FIG 1. Below, the process steps of this method are described with reference to both FIG 1 and FIG 2.
The method starts at a process step S1. Here, the robot 2 and the data processing device 3 can, for example, be activated. Alternatively, the process step S1 can be set to the point in time when a user starts hand-guiding the robot 2 or when the robot 2 is subjected to a mechanical tension.
In a process step S2 the moment sensors 12 continuously measure a respective torque. The continuous measurement is schematically indicated by a loop 26 the measured torque values are transmitted in real time from the moment sensors 12 to the data processing device 3 which receives them through an input interface 18. The continuous measurement of the moment or torque can continue throughout the steps and processes described below.
In the present example the robot 2 and in particular the end effector or 11 is guided so that the probe 13 is brought into contact with a target object 14 at a contact point 15. In the presently described example, the end effector 11 also- intentionally or unintentionally - comes into mechanical contact with a stationary external object 17 during this process.
The stationary external object 17 is fixedly positioned and limits a motion of the robot 2, in particular of the end effector 11. For example, the respective user may exert a force on one of the links 6 to move or position the robot 2 so that the probe 13 comes into mechanical contact with the target object 14 and the contact point 15. If, for example, the user exerts this force on a particular link 19 of the multiple links 6 this could be registered at a particular torque sensor 20 of the multiple moment sensors 12. An internal drive of the robot 2 could then be activated to support or assist with the intended movement of the manipulator 5. To do this the drive can exert or put out a certain drive torque to rotate the link 19 about a corresponding axis 21 of the multiple axes 8. A value of this drive torque can also be transmitted to the data processing device 3. In a process step S3 a processing unit 22 of the data processing device 3 can run an internal model of the robot 2 using this drive torque as an input to calculate a resulting calculated position of the end effector 11. This internal model can be a simplified model of the robot 2 which can assume that the robot 2 or the manipulator 5 can move freely and unhindered.
The force exerted by the user to guide the manipulator 5 and/or by the stationary external object 17, which prevents the end effector 11 from moving further in the direction of the stationary external object 17 upon mechanical contact, can, however, exert or cause an additional external moment which can influence the actual motion of the manipulator 5, in particular of the end effector 11. Since the moment sensors 12 measure a total moment affecting the robot 2 at each joint 7 or axis 8 the external moment or moments affecting the robot 2 can be calculated in a process step S4. To achieve this the processing unit 22 might calculate the external moment as a difference between the measured moments and the assumed or calculated moments which would result from the drive torque and a free unhindered motion or movement of the manipulator 5 based on the simplified internal model. In the process step S4 this data is, in other words, processed by the processing unit 22 of the data processing device 3 to determine the external moment caused by the user and/or the stationary external object 17.
In a process step S5 a correction factor can be computed by the processing unit 22 based on the external moment determined in process step S4 and the internal model used in process step S3. This correction factor can in the simplest case just be the external moment itself. The correction factor can, however, also be a different computed value such as, for example, a change in a position or pose of the manipulator 5 or the end effector 11 that would be caused by the external moment alone. Computing the correction factor can alternatively or additionally, for example, comprise modification or transformation of the corresponding value or values depending on a chosen implementation or reference frame.
In a process step S6 a current position of the end effector 11 is computed based on a correction model in dependence on the correction factor. This computed current position is more accurate than a calculated position output by the above-mentioned simplified internal model which does not take into account effects or influences of the external moment. For example, the external moment or a corresponding change in position or pose the external moment would cause on its own could be computed as the correction factor. The correction model could then subtract the respective corresponding value from the corresponding values used as input for the above-mentioned internal simplified model, which was also used in process step S3.
It is especially advantageous if the moment sensors 12 are arranged at each joint 7 or at each axis 8. If, for example, the drive torque acts on the axis 21, then an actual resulting moment at the end effector 11 can differ from its calculated or modelled value due to numerous factors such as a joint elasticity, a transmission elasticity, a backlash, a hysteresis, a temperature influence, etc. affecting parts of the manipulator 5 arranged between the drive or the axis 21 on the one hand and the end effector 11 on the other hand. The arrangement of the moment sensors 12 at each joint 7 or axis 8 therefore represents a particular advantage since the moments measured by the moment sensors 12 automatically include all of these effects and influences which cannot be reliably and precisely modelled with reasonable effort. Since the computed current position of the end effector 11 is partly based on the moments measured by the moment sensors 12 it also automatically takes into account the above-mentioned effects and influences in a particularly easy and effective manner.
In a process step S7 a current position of the target object 14 is calculated based on the computed current position of the end effector 11. This position of the target object 14 can for example be considered equal to the position of the contact point 15. The position of the contact point 15 can either be considered to be equal to the position of the end effector 11 or it can simply be calculated from the position of the end effector 11 by adding to it the known length of the probe 13.
In a process step S8 the probe 13 is brought into contact with the external target object 14 at multiple additional contact points 16. Each time the probe 13 is in contact with one of these additional contact points 16 the corresponding current position of the end effector 11 and the target object 14 is computed. In this manner multiple positional values are determined for the target object 14 which can advantageously be used to map a surface or shape of the target object 14.
Additionally or alternatively the target object 14 can be rigidly attached to the probe 13 or the end effector 11. This can for example be done after the multiple positional values for the target object 14 have been obtained. The target object 14 can then be moved through space by guiding the robot 2 or the manipulator 5. By computing the respective current position of the end effector 11 and the target object 14 continuously or at multiple different points in time a path of the target object 14 can be determined and recorded.
In a process step S9 the positional data for the target object 14 is processed by the processing unit 22 to automatically generate a virtual model and/or a virtual path of the target object 14. The virtual model and/or the virtual path, respectively, can for example be used in a virtual reality or in an augmented reality application.
To carry out all of the above-mentioned calculations and computations the processing unit 22 can access a storage medium 23 of the data processing device 3. This storage medium 23 can hold or contain the necessary instructions as well as the mentioned models in the form of a program code or electronic data.
Any of the computed, calculated, or measured values or data can also be provided or output to any other system or device or model through an output interface 24 of the data processing device 3 for further processing.

Claims

Method (25) for modelling an external object (14, 17) with a hand-guided robot (2) having a jointed arm (9) with at least one joint (7), and an end effector (11) connected to one end of the arm (9), comprising the following steps:
- measuring a total moment at each joint (7) of the arm (9) affecting the robot (2) while the robot (2) is being hand-guided and subjected to a mechanical tension due to contact with an external object (14, 17), the mechanical tension influencing a motion of the jointed arm (9) or the end effector (11),

- based on the measured at least one moment determining at least one external moment caused by the mechanical tension and/or a force exerted by a user for hand-guiding the robot (2) and affecting the robot (2),

- computing a positional correction factor for a position of the end effector (11) based on the at least one external moment and a simplified model of the robot (2), wherein the simplified model assumes that the robot (2) can move freely and unhindered,

- while the robot (2) is still in contact with the external object (14, 17) computing a current position of the end effector (11) based on a correction model in dependence on the positional correction factor, wherein the correction model uses the positional correction factor to specify a difference between the objective current position of the end effector (11) and a calculated position of the end effector (11),
characterized by the following steps:
- determining the position of the external object (14) based on the computed current position of the end effector (11),

- moving the end effector (11) sequentially into contact with the external object (14, 17) in multiple different contact points (15, 16),

- computing the respective current position of the end effector (11) in each of the multiple different contact points, and

- virtually modelling the external object (14, 17) based on the multiple computed respective current positions.
Method (25) according to claim 1, characterised in that
- the external object (13, 14) or a second external object (17) is held by the end effector (11),

- the current position of the end effector (11) is continuously computed,

- a path of the held external object (13, 14) is determined based on the continuously computed current positions of the end effector (11) to track a position and/or a movement of the external object (13, 14).
Storage medium (23) containing a program code adapted on execution by a processing unit (22) to cause a robotic system comprising a hand-guided robot (2) having a jointed arm (9) with at least one joint (7) and an end effector (11) connected to one end of the arm (9), and a moment sensor (12, 20) in or at each joint (7) of the arm (9), to perform a method (25) according to any of the preceding claims.
Data processing device (3) comprising a processing unit (22) and a storage medium (23) according to claim 3.
Robotic system (1) comprising
- a hand-guided robot (2) having a jointed arm (9) with at least one joint (7) and an end effector (11) connected to one end of the arm (9),

- a moment sensor (12, 20) in or at each joint (7) of the arm (9),

- a data processing device (3) according to claim 4 connected to each moment sensor (12, 20) via a respective data link.
Robotic system (1) according to claim 5, characterised in that the robot (2) is an industrial robot (2) having at least six degrees of freedom.