DE102022213568B3 - Calibrating a controller - Google Patents

Abstract

A method for calibrating a controller (3) for controlling a robot arm (10) and/or a gripper (20, 21, 22) guided by the robot arm comprises the steps: recording (S20) a first image of the robot-guided gripper at a first actual -Robot arm position of the robot arm and a first actual finger position of a first finger (21) of the gripper relative to a base (20) of the gripper using a recording device (4); Determining (S30) a first actual pose of the first finger based on this first image; Determining (S40) a first target pose of the first finger for this first actual finger position and this first actual robot arm position; and calibrating (S80) the control based on this determined first actual pose and first target pose of the first finger.

Description

The present invention relates to a method for calibrating a controller for controlling a robot arm and/or a gripper guided by the robot arm, a method for operating the robot arm and/or gripper guided by the robot arm, and a system or computer program or computer program product for carrying out a procedure described here.

Robotic arms that guide finger grippers are known per se. If the robot arm and/or gripper fingers are controlled based on target values, the actual poses of gripper fingers relative to a base of the gripper and/or relative to a base of the robot arm can deviate from the desired target poses, for example due to deformations, Tolerances or the like.

The DE 10 2022 129 008 A1 relates to a robotic grip generation technique for part picking applications, wherein the part and gripper geometry are provided as inputs, the gripper kinematics are also specified as input, a set of grip candidates is provided using any known preliminary grip generation tool, a point model of the part, and a model the gripper contact surfaces with a clearance margin are used in an optimization calculation made for each of the gripping candidates, resulting in a database of adapted grips, the adapted grips optimize the gripping quality through the use of a virtual gripper surface that represents the actual gripper surface at a small distance away from the part positioned, then a signed distance field calculation is performed for each of the adjusted handles, and those where there is any collision between the gripper and the part are discarded.

The DE 11 2021 004 632 T5 relates to a robot control device capable of facilitating the work of setting up a control center for controlling the operation of a robot, the robot control device controlling a robot manipulator equipped with an end effector, the robot control device comprising: an image processing unit using a feature extraction model for capturing images of the robot manipulator and position/attitude information of the robot manipulator from images in which at least a part of the robot manipulator is captured, captures a position in a three-dimensional space, the designated positions designated on the images, as a position relative to the robot manipulator; and a coordinate system determining unit that sets a control center for controlling the operation of the robot manipulator to the position detected in the three-dimensional space.

The DE 10 2020 209 486 B3 relates to an actuator with a first link and a second link, which can be rotated and / or moved linearly relative to one another via at least one drive, wherein a first pose is determined, which is a pose between a first point that is stationary relative to the first link and a second point, which is stationary relative to the second member, describes relative to one another, wherein to determine this first pose at least one camera is arranged in a fixed spatial relationship to the first member and at least two markers are arranged in a fixed spatial relationship to the second member, whereby the first pose is determined in a computing unit from the image data of the markers.

The DE 10 2020 116 342 A1 relates to a device that can more accurately detect the deviation of a working position of a tool with respect to a target position of a workpiece in accordance with the actual activity, wherein a device for detecting the amount of deviation of the working position of a tool with respect to the target position when performing an activity at the target position of the workpiece by the tool while moving the tool by a moving machine, a camera arranged in a certain positional relationship with respect to the tool and at a first time when the tool is brought to perform an operation for the operation, captures an image of the target position, and a deviation amount obtaining unit that, based on the location of the target position in the image data captured by the camera and information showing the location of the work position in this image data, determines the amount of deviation between the work position and the target position at the first time acquired.

The DE 11 2019 005 484 T5 relates to a method for automatically calibrating a camera robot system with tool offsets.

The DE 11 2016 002 797 B4 relates to a calibration device designed to extract only such an external force generated on a tool part by contact with a work object, to a mechanical device designed to perform force control on the tool part a tip of the mechanical device is mounted and is designed to act on the work object.

The object of the present invention is to improve the operation of robot arms with robot arm-guided grippers.

This task is solved by a method with the features of claims 1 and 8, respectively. Claims 9, 11 protect a system or a computer program or computer program product for carrying out a method described here. The subclaims relate to advantageous further training.

According to one embodiment of the present invention, a robot arm has at least three, in particular at least six, in one embodiment at least seven, joints or (movement) axes, preferably at least three, in particular at least six, in one embodiment at least seven, swivel joints or axes of rotation, as well a (distal) robot base and a (proximal) end flange, to which a gripper, preferably non-destructively detachable or replaceable, is attached, which has a one-part or multi-part gripper base, arranged in one embodiment on the end flange of the robot arm, and a, two or more (gripper) fingers that are adjustable relative to this gripper base, in one embodiment electromagnetically, in particular electromotively, hydraulically and / or pneumatically.

• - Aufnehmen eines ersten Bildes des robotergeführten Greifers bei bzw. in
  • - einer ersten Ist-Roboterarmstellung des Roboterarms und
  • - einer ersten Ist-Fingerstellung eines ersten Fingers des Greifers relativ zu einer Basis des Greifers (Greiferbasis) mithilfe einer Aufnahmevorrichtung;
  • - Ermitteln einer ersten Ist-Pose des ersten Fingers auf Basis dieses ersten Bildes;
  • - Ermitteln einer ersten Soll-Pose des ersten Fingers für diese erste Ist-Fingerstellung und diese erste Ist-Roboterarmstellung vor, bei oder nach dem Aufnehmen des ersten Bildes und vor, bei oder nach dem Ermitteln der ersten Ist-Pose des ersten Fingers; und
  • - Kalibrieren der Steuerung auf Basis dieser ermitteln ersten Ist-Pose und ersten Soll-Pose des ersten Fingers.

According to one embodiment of the present invention, a method for calibrating a controller that is provided or set up or used to control the robot arm and/or the gripper guided by the robot arm comprises the steps of:

• - Taking a first image of the robot-guided gripper at or in
  • - a first actual robot arm position of the robot arm and
  • - a first actual finger position of a first finger of the gripper relative to a base of the gripper (gripper base) using a recording device;
  • - Determining a first actual pose of the first finger based on this first image;
  • - Determining a first target pose of the first finger for this first actual finger position and this first actual robot arm position before, during or after taking the first image and before, during or after determining the first actual pose of the first finger; and
  • - Calibrate the control based on this to determine the first actual pose and the first target pose of the first finger.

In one embodiment, the method includes the step preceding the recording of the first image of bringing at least the first finger, preferably all fingers, and/or the gripper base of the gripper into the field of view or recording field of the recording device, in a further development by appropriately adjusting the Robot arm and/or moving and/or focusing the recording device.

One embodiment of the present invention is therefore based on the idea of using a recording device to optically calibrate the control, in particular in such a way that the calibrated control controls the robot arm and/or the gripper guided by it in such a way that an actual pose is thereby approached or several gripper fingers deviate less from a desired target pose than before or without calibration.

In one embodiment, an (actual robot arm) position of the robot arm is determined by the positions of its joints or axes and/or a pose of its end flange or a stationary reference relative to the robot arm base. In one embodiment, a pose in the sense of the present invention includes a one-, two- or three-dimensional position and/or a one-, two- or three-dimensional orientation. Accordingly, in one embodiment, a target pose and/or an actual pose of a finger of the gripper describes a one-, two- or three-dimensional position and/or a one-, two- or three-dimensional orientation of the finger, preferably relative to the robot arm base or in particular preferably relative to the gripper base, in one embodiment a one-, two- or three-dimensional position and / or a one-, two- or three-dimensional orientation of a fingertip of the finger, preferably relative to the robot arm base or particularly preferably relative to the gripper base. By (determining the) target or actual pose relative to the gripper base, influences of the robot arm position(s) can be reduced, preferably eliminated, in one embodiment, since these influence equally on the pose of the gripper base relative to the environment and the pose of the robot arm or of the finger relative to the environment and can therefore be hidden in one embodiment by (determining the) target or actual pose of the finger or fingers relative to the gripper base. In one embodiment, a target pose is determined on the basis of a commanded and/or, in a further development sensory, detected position of the corresponding finger relative to the gripper base, in particular it can be such a position or can be determined on the basis of a kinematics model of the gripper. For example, if a three-dimensional position of a fingertip is described in a coordinate system fixed to the gripper base by [x(a) y(a) z(a)] ^T with the adjustment path a, a target pose in the sense of the present invention can be dung include the adjustment path a and/or one or more, in particular all, components of the position [x(a) y(a) z(a)] ^T , ie the target pose in particular the commanded or, in a further development, detected by sensors Adjusting path a and / or the position determined from this using forward kinematics, for example x (a), in particular include.

• - Ermitteln einer ersten Soll-Pose eines zweiten Fingers des Greifers für eine erste Ist-Fingerstellung des zweiten Fingers relativ zu der Basis des Greifers und vorzugsweise für die erste Ist-Roboterarmstellung; und
• - Ermitteln einer ersten Ist-Pose des zweiten Fingers auf Basis eines Bildes, welches bei dieser ersten Ist-Fingerstellung des zweiten Fingers und vorzugsweise bei der ersten Ist-Roboterarmstellung aufgenommen worden ist, vorzugsweise auf Basis des ersten Bildes;

wobei die Steuerung (auch) auf Basis dieser ermittelten ersten Ist-Pose und ersten Soll-Pose des zweiten Fingers kalibriert wird.If the gripper has at least two fingers, the method in one embodiment comprises the steps analogous to the steps described above:

• - Determining a first target pose of a second finger of the gripper for a first actual finger position of the second finger relative to the base of the gripper and preferably for the first actual robot arm position; and
• - Determining a first actual pose of the second finger based on an image which was recorded at this first actual finger position of the second finger and preferably at the first actual robot arm position, preferably based on the first image;

wherein the control is (also) calibrated based on this determined first actual pose and first target pose of the second finger.

• - Ermitteln einer ersten Soll-Pose eines dritten Fingers des Greifers für eine erste Ist-Fingerstellung des dritten Fingers relativ zu der Basis des Greifers und vorzugsweise für die erste Ist-Roboterarmstellung; und
• - Ermitteln einer ersten Ist-Pose des dritten Fingers auf Basis eines Bildes, welches bei dieser ersten Ist-Fingerstellung des dritten Fingers und vorzugsweise bei der ersten Ist-Roboterarmstellung aufgenommen worden ist, vorzugsweise auf Basis des ersten Bildes;

wobei die Steuerung (auch) auf Basis dieser ermittelten ersten Ist-Pose und ersten Soll-Pose des dritten Fingers kalibriert wird.If the gripper has at least three fingers, the method in one embodiment comprises the steps analogous to the steps described above:

• - Determining a first target pose of a third finger of the gripper for a first actual finger position of the third finger relative to the base of the gripper and preferably for the first actual robot arm position; and
• - Determining a first actual pose of the third finger based on an image which was recorded at this first actual finger position of the third finger and preferably at the first actual robot arm position, preferably based on the first image;

wherein the control is (also) calibrated based on this determined first actual pose and first target pose of the third finger.

As a result, in one embodiment, the control for multi-fingered grippers can be calibrated particularly advantageously and quickly. If reference is made below to a second finger, the same features can also be implemented in one embodiment for a third finger of the gripper, without this being specifically pointed out.

• - Verstellen des ersten Fingers in wenigstens eine zweite Ist-Fingerstellung, in der der erste Finger relativ zu der Basis des Greifers eine veränderte Stellung aufweist, in einer Weiterbildung unter Beibehaltung der ersten Ist-Roboterarmstellung;
• - Ermitteln einer zweiten Soll-Pose des ersten Fingers für diese zweite Ist-Fingerstellung;
• - Aufnehmen wenigstens eines Bildes des Greifers bei dieser zweiten Ist-Fingerstellung des ersten Fingers und vorzugsweise der beibehaltenen ersten Ist-Roboterarmstellung; und
• - Ermitteln einer zweiten Ist-Pose des ersten Fingers auf Basis dieses Bildes; wobei die Steuerung (auch) auf Basis dieser ermittelten zweiten Ist-Pose und zweiten Soll-Pose des ersten Fingers kalibriert wird.

In one embodiment, the method includes the steps:

• - Adjusting the first finger into at least a second actual finger position, in which the first finger has a changed position relative to the base of the gripper, in a further development while maintaining the first actual robot arm position;
• - Determining a second target pose of the first finger for this second actual finger position;
• - Recording at least one image of the gripper in this second actual finger position of the first finger and preferably in the maintained first actual robot arm position; and
• - Determining a second actual pose of the first finger based on this image; wherein the control is (also) calibrated based on this determined second actual pose and second target pose of the first finger.

• - Verstellen des zweiten Fingers in wenigstens eine zweite Ist-Fingerstellung, in der der zweite Finger relativ zu der Basis des Greifers eine veränderte Stellung aufweist, in einer Weiterbildung zusammen mit dem oben genannten Verstellen des ersten Fingers und/oder unter Beibehaltung der ersten Ist-Roboterarmstellung;
• - Ermitteln einer zweiten Soll-Pose des zweiten Fingers für diese zweite Ist-Fingerstellung; und
• - Ermitteln einer zweiten Ist-Pose des zweiten Fingers auf Basis eines Bildes, welches bei dieser zweiten Ist-Fingerstellung des zweiten Fingers aufgenommen worden ist, vorzugsweise auf Basis des Bildes, auf dessen Basis auch die zweite Ist-Pose des ersten Fingers ermittelt wird;

wobei die Steuerung (auch) auf Basis dieser ermittelten zweiten Ist-Pose und zweiten Soll-Pose des zweiten Fingers kalibriert wird.Additionally or alternatively, the process in further training includes the steps:

• - Adjusting the second finger into at least one second actual finger position, in which the second finger has a changed position relative to the base of the gripper, in a further development together with the above-mentioned adjustment of the first finger and / or while maintaining the first actual robot arm position;
• - Determining a second target pose of the second finger for this second actual finger position; and
• - Determining a second actual pose of the second finger based on an image which was recorded in this second actual finger position of the second finger, preferably based on the image on the basis of which the second actual pose of the first finger is also determined;

wherein the control is (also) calibrated based on this determined second actual pose and second target pose of the second finger.

In addition or as an alternative to the use of two or more actual finger positions of the finger(s), in a preferred development in or at (each) the same robot arm position, two or more actual robot arm positions are used or used in one embodiment for calibration.

• - Verstellen des Roboterarms in wenigstens eine zweite Ist-Roboterarmstellung;
• - Aufnehmen wenigstens eines Bildes des Greifers bei dieser zweiten Ist-Roboterarmstellung; und
• - Ermitteln einer zweiten Ist-Pose des ersten Fingers und in einer Weiterbildung auch einer zweiten Ist-Pose des zweiten Fingers auf Basis dieses Bildes;

wobei die Steuerung (auch) auf Basis dieser ermittelten zweiten Ist-Pose des ersten Fingers und einer für diese zweite Ist-Roboterarmstellung und zweite Ist-Pose des ersten Fingers ermittelten zweiten Soll-Pose des ersten Fingers und gegebenenfalls (auch) auf Basis dieser ermittelten zweiten Ist-Pose des zweiten Fingers und einer für diese zweite Ist-Roboterarmstellung und zweite Ist-Pose des zweiten Fingers ermittelten zweiten Soll-Pose des zweiten Fingers kalibriert wird.Accordingly, in one embodiment the method includes the steps:

• - Adjusting the robot arm into at least a second actual robot arm position;
• - Recording at least one image of the gripper in this second actual robot arm position; and
• - Determining a second actual pose of the first finger and, in a further development, also a second actual pose of the second finger based on this image;

wherein the controller (also) determines on the basis of this determined second actual pose of the first finger and a second target pose of the first finger determined for this second actual robot arm position and second actual pose of the first finger and optionally (also) on the basis of this second actual pose of the second finger and a second target pose of the second finger determined for this second actual robot arm position and second actual pose of the second finger.

By using two or more actual finger positions of the finger(s) and the use of two or more actual robot arm positions, calibration, in particular precision and/or reliability, can be improved in one embodiment.

In one embodiment, two or more images with different actual finger positions of the finger or one or more of the fingers can be recorded in or at one or more actual robot arm positions (each), from which different actual poses of the finger or one or several of the fingers can be determined and used together with the associated target poses for calibration. Since the adjustment of gripper fingers can often be carried out quickly (more than the adjustment of the robot arm) and does not require an adjustment of the robot arm, which often requires a correspondingly larger working space, calibration, in particular precision, reliability, feasibility, safety and / or Speed, can be improved. In addition, errors in the robot arm position do not accumulate in one version.

Additionally or alternatively, in an embodiment with the same or different actual finger position of the finger or one or more of the fingers (each) two or more images can be recorded in or at different actual robot arm positions, from which different actual poses of the finger or one or more of the fingers are determined and used together with the associated target poses for calibration, in particular in or at different actual robot arm positions (each) images are recorded with the same actual finger position of the finger or the same actual finger positions of the fingers. As a result, particularly advantageous actual finger positions, preferably large adjustment ranges of the gripper, can be used in one embodiment. Additionally or alternatively, by (adjusting the robot arm into) different actual robot arm positions in one embodiment, the view of the recording device onto the gripper can be improved, in particular partial obscurations or the like can be at least partially compensated for. In addition, errors resulting from the robot arm positions can be taken into account or averaged out.

Preferably, the two above aspects can be advantageously combined by creating images with the first and second actual finger positions of the first and possibly second fingers in or at the first actual robot arm position and images with at least two different ones in or at the second actual robot arm position Actual finger position of the first and, if applicable, second finger is recorded, actual poses of the finger or fingers are determined from this and used together with the associated target poses for calibration. One of the at least two different actual finger positions at or in the second actual robot arm position can be the first or second actual finger position of the respective finger at or in the first actual robot arm position, whereby the same actual finger position is used several times in one embodiment and thereby in particular effort and/or errors resulting from an adjustment of the respective finger can be reduced. Additionally or alternatively, in one embodiment, different finger positions can also be used in the first and second robot arm positions, whereby in one embodiment even more different constellations can be taken into account during the calibration and these, in particular their precision, validity and / or reliability, can thereby be further improved can.

• - Verstellen des ersten Fingers und in einer Weiterbildung des zweiten Fingers (jeweils) in eine zweite Ist-Fingerstellung, in der der erste bzw. zweite Finger relativ zu der Basis des Greifers eine veränderte Stellung aufweist, in bzw. bei der ersten Ist-Roboterarmstellung und/oder in bzw. bei der zweiten Ist-Roboterarmstellung;
• - Ermitteln (je) einer zweiten Soll-Pose des ersten bzw. zweiten Fingers für diese zweite Ist-Fingerstellung und Ist-Roboterarmstellung;
• - Aufnehmen (je) wenigstens eines Bildes des Greifers bei der ersten Ist-Fingerstellung und bei der zweiten Ist-Fingerstellung des ersten und gegebenenfalls zweiten Fingers bei bzw. in der ersten und/oder zweiten Ist-Roboterarmstellung; und
• - Ermitteln (je) einer zweiten Ist-Pose des ersten und gegebenenfalls zweiten Fingers auf Basis des jeweiligen Bildes;

wobei die Steuerung auf Basis dieser ermittelten zweiten Ist-Posen und zweiten Soll-Posen des ersten und gegebenenfalls zweiten Fingers kalibriert wird.Accordingly, in one embodiment the method includes the steps:

• - Adjusting the first finger and, in a further development, the second finger (in each case) into a second actual finger position, in which the first or second finger has a changed position relative to the base of the gripper, in or at the first actual robot arm position and/or in or at the second actual robot arm position;
• - Determining (each) a second target pose of the first or second finger for this second actual finger position and actual robot arm position;
• - Recording (each) at least one image of the gripper in the first actual finger position and in the second actual finger position of the first and possibly second finger at or in the first and/or second actual robot arm position; and
• - Determining (each) a second actual pose of the first and possibly second finger based on the respective image;

wherein the control is calibrated on the basis of these determined second actual poses and second target poses of the first and, if applicable, second finger.

For a more compact representation, both a pose of a finger for a second actual finger position and a pose of a finger for a second actual robot arm position are each referred to as a second pose of the finger. Accordingly, in particular

• - eine Pose eines Fingers für eine Ist-Fingerstellung A bei einer Ist-Roboterarmstellung B eine zweite Pose dieses Fingers; sowie
• - eine Pose dieses Fingers für eine andere Ist-Fingerstellung A' bei derselben Ist-Roboterarmstellung B und/oder
• - eine Pose dieses Fingers für dieselbe Ist-Fingerstellung A bei einer anderen Ist-Roboterarmstellung B' und/oder
• - eine Pose dieses Fingers für eine andere Ist-Fingerstellung A' bei einer anderen Ist-Roboterarmstellung B'

jeweils eine weitere bzw. andere zweite Pose dieses Fingers im Sinne der vorliegenden Erfindung bezeichnen bzw. mehrere zweite Posen desselben Fingers zum Kalibrieren verwendet werden. Werden entsprechend zwei hier beschriebene Aspekte, in denen jeweils auf eine zweite Pose eines Fingers Bezug genommen ist, miteinander kombiniert, so umfasst dies ausdrücklich

• - eine Kombination der Aspekte, in denen jeweils auf dieselbe zweite Pose des jeweiligen Fingers Bezug genommen ist, und
• - eine Kombination der Aspekte, wobei bei dem einen Aspekt auf eine zweite Pose und bei dem anderen Aspekt auf eine weitere bzw. andere zweite Pose des jeweiligen Fingers Bezug genommen ist.

• - a pose of a finger for an actual finger position A with an actual robot arm position B, a second pose of this finger; as well as
• - a pose of this finger for a different actual finger position A' with the same actual robot arm position B and/or
• - a pose of this finger for the same actual finger position A with a different actual robot arm position B' and/or
• - a pose of this finger for a different actual finger position A' with a different actual robot arm position B'

each designate a further or different second pose of this finger in the sense of the present invention or several second poses of the same finger are used for calibration. If two aspects described here, each of which refers to a second pose of a finger, are combined with one another, this expressly includes

• - a combination of the aspects in which reference is made to the same second pose of the respective finger, and
• - a combination of the aspects, with one aspect referring to a second pose and the other aspect referring to a further or different second pose of the respective finger.

In other words, in a combination of two aspects described here in one embodiment, a second pose of one aspect and a second pose of the other aspect can be one and the same pose (in particular, a second pose of the first finger can be a pose of the first finger in the second actual robot arm position and the second actual finger position of the first finger), or a second pose of one aspect and a second pose of the other aspect can be different poses (in particular, a second pose of the first finger can be a pose of the first finger in the second Actual robot arm position and the first actual finger position of the first finger and a further or different second pose of the first finger can be a pose of the first finger at the first actual robot arm position and the second actual finger position of the first finger).

Calibrating the control includes, in one embodiment, determining, in a further development parameterizing and/or storing, an image between actual poses and target poses of the first finger and/or an image between actual poses and target poses of the second finger of the gripper, in a further development an assignment of target poses of the corresponding finger to (approaching) desired target poses and / or an assignment of actual poses of the corresponding finger to, in particular commanded or detected, target poses. Additionally or alternatively, calibrating the control in one embodiment includes determining, in a further development parameterizing and/or storing, a correction between actual poses and target poses of the first finger and/or a correction between actual poses and target poses of the second finger of the gripper, in a further development a correction of target poses of the corresponding finger for approaching desired target poses and / or a correction of actual poses of the corresponding finger for, in particular commanded or detected, target poses. In a further development, the mapping or correction can be different for different robot arm and/or finger positions, whereby the precision can be increased in one embodiment. Equally, the mapping or correction can also have a constant offset, which can advantageously be determined by averaging or the like.

For example, if a kinematics model of the gripper includes an assignment x(a) = k a + x ₀ with the adjustment path a of a finger relative to the gripper base and a position x of its fingertip relative to the gripper base in the adjustment direction, then, for example, different commanded or detected adjustment paths a ₁ , a ₂ or associated positions x ₁ = k a ₁ + x ₀ , x ₂ = k a ₂ + x ₀ , first or second target poses in the sense of the present invention. Correspondingly, calibration can involve determining the parameters _{k , x 0 in} such _a way that positions x ₁₁ , Positions x ₁₂ , x ₃₂ the deviation |x ₁₁ - k a ₁ + x ₀ | + |x ₂₁ - k a ₂ + x ₀ | + |x ₁₂ - k a ₁ + x ₀ | + |x ₃₂ - k a ₃ + x ₀ | minimal will include. Likewise, a constant offset x _korr can be determined, which corrects the kinematics model accordingly, for example x = k a + x ₀ + x _korr or the like. These simple examples are for illustration purposes only; it will be clear to those skilled in the art that more complex illustrations or Correct ures are possible, in a particularly preferred embodiment different images or corrections for different finger positions and / or robot arm positions.

In one embodiment, a first and/or second actual pose of the first and/or second finger (each) is determined based on a theoretical, preferably kinematic, model of the gripper and/or based on object recognition. The recording device particularly preferably comprises a recording device for recording digital and/or three-dimensional images, and can in particular have at least one 3D camera and/or at least two spatially spaced cameras and/or at least one scanner, preferably for three-dimensional scanning. In one embodiment, an image mentioned here (each) has a point cloud, preferably a three-dimensional point cloud, and can in particular be one or consist of one. Accordingly, in particular, a three-dimensional point cloud recorded using a recording device is referred to as an image (recorded using the recording device), which is generally a three-dimensional image in one embodiment.

• - einen bzw. den Roboterarm und einen bzw. den von dem Roboterarm geführten Greifer;
• - eine bzw. die Aufnahmevorrichtung zum Aufnehmen eines ersten Bildes des robotergeführten Greifers bei einer ersten Ist-Roboterarmstellung des Roboterarms und einer ersten Ist-Fingerstellung eines ersten Fingers des Greifers relativ zu einer Basis des Greifers; und
• - eine bzw. die Steuerung zum Steuern des Roboterarms und/oder Greifers, die ihrerseits aufweist:
  • - Mittel zum Ermitteln einer ersten Ist-Pose des ersten Fingers auf Basis dieses ersten Bildes;
  • - Mitteln zum Ermitteln einer ersten Soll-Pose des ersten Fingers für diese erste Ist-Fingerstellung und diese erste Ist-Roboterarmstellung; und
  • - Mitteln zum Kalibrieren der Steuerung auf Basis dieser ermitteln ersten Ist-Pose und ersten Soll-Pose des ersten Fingers.

According to one embodiment of the present invention, a system for calibrating the control for controlling the robot arm and / or the gripper guided by the robot arm, in a further development a system for controlling the robot arm and / or gripper, in particular hardware and / or software, in particular programmatically, set up to carry out a method described here and/or has:

• - one or the robot arm and one or the gripper guided by the robot arm;
• - a or the recording device for recording a first image of the robot-guided gripper at a first actual robot arm position of the robot arm and a first actual finger position of a first finger of the gripper relative to a base of the gripper; and
• - a control for controlling the robot arm and/or gripper, which in turn has:
  • - Means for determining a first actual pose of the first finger based on this first image;
  • - Means for determining a first target pose of the first finger for this first actual finger position and this first actual robot arm position; and
  • - Means for calibrating the control based on this determine the first actual pose and the first target pose of the first finger.

The control can be distributed or have distributed means, in particular in one embodiment it can have a module for controlling the robot arm and / or gripper and its own module for calibrating this module for control.

• - Mitteln zum Verstellen des ersten Fingers in eine zweite Ist-Fingerstellung, in der der erste Finger relativ zu der Basis des Greifers eine veränderte Stellung aufweist;
• - Mitteln zum Ermitteln einer zweiten Soll-Pose des ersten Fingers für diese zweite Ist-Fingerstellung;
• - Mitteln zum Aufnehmen wenigstens eines Bildes des Greifers bei dieser zweiten Ist-Fingerstellung des ersten Fingers; und
• - Mitteln zum Ermitteln einer zweiten Ist-Pose des ersten Fingers auf Basis dieses Bildes; und
• - Mitteln zum Kalibrieren der Steuerung auf Basis dieser ermittelten zweiten Ist-Pose und zweiten Soll-Pose des ersten Fingers.

In one embodiment, the system or its control or its means has:

• - means for adjusting the first finger into a second actual finger position, in which the first finger has a changed position relative to the base of the gripper;
• - Means for determining a second target pose of the first finger for this second actual finger position;
• - Means for recording at least one image of the gripper in this second actual finger position of the first finger; and
• - Means for determining a second actual pose of the first finger based on this image; and
• - Means for calibrating the control based on this determined second actual pose and second target pose of the first finger.

• - Mitteln zum Ermitteln einer ersten Soll-Pose wenigstens eines zweiten Fingers des Greifers für eine erste Ist-Fingerstellung des zweiten Fingers relativ zu der Basis des Greifers;
• - Mitteln zum Ermitteln einer ersten Ist-Pose des zweiten Fingers auf Basis eines Bildes, welches bei dieser ersten Ist-Fingerstellung des zweiten Fingers aufgenommen worden ist; und
• - Mitteln zum Kalibrieren der Steuerung auf Basis dieser ermitteln ersten Ist-Pose und ersten Soll-Pose des zweiten Fingers.

In one embodiment, the system or its control or its means has:

• - Means for determining a first target pose of at least a second finger of the gripper for a first actual finger position of the second finger relative to the base of the gripper;
• - Means for determining a first actual pose of the second finger based on an image which was recorded with this first actual finger position of the second finger; and
• - Means for calibrating the control based on this determine the first actual pose and the first target pose of the second finger.

• - Mitteln zum Verstellen des zweiten Fingers in wenigstens eine zweite Ist-Fingerstellung, in der der zweite Finger relativ zu der Basis des Greifers eine veränderte Stellung aufweist;
• - Mitteln zum Ermitteln einer zweiten Soll-Pose des zweiten Fingers für diese zweite Ist-Fingerstellung;
• - Mitteln zum Ermitteln einer zweiten Ist-Pose des zweiten Fingers auf Basis eines Bildes, welches bei dieser zweiten Ist-Fingerstellung des zweiten Fingers aufgenommen worden ist; und
• - Mitteln zum Kalibrieren der Steuerung auf Basis dieser ermittelten zweiten Ist-Pose und zweiten Soll-Pose des zweiten Fingers.

In one embodiment, the system or its control or its means has:

• - Means for adjusting the second finger into at least one second actual finger position, in which the second finger has a changed position relative to the base of the gripper;
• - Means for determining a second target pose of the second finger for this second actual finger position;
• - Means for determining a second actual pose of the second finger based on an image which was recorded at this second actual finger position of the second finger; and
• - Means for calibrating the control based on this determined second actual pose and second target pose of the second finger.

• - Mitteln zum Verstellen des Roboterarms in wenigstens eine zweite Ist-Roboterarmstellung;
• - Mitteln zum Aufnehmen wenigstens eines Bildes des Greifers bei dieser zweiten Ist-Roboterarmstellung;
• - Mitteln zum Ermitteln einer zweiten Ist-Pose des ersten Fingers auf Basis dieses Bildes; und
• - Mitteln zum Kalibrieren der Steuerung auf Basis dieser ermittelten zweiten Ist-Pose des ersten Fingers und einer für diese zweite Ist-Roboterarmstellung und zweite Ist-Pose des ersten Fingers ermittelten zweiten Soll-Pose des ersten Fingers.

In one embodiment, the system or its control or its means has:

• - Means for adjusting the robot arm into at least one second actual robot arm position;
• - Means for recording at least one image of the gripper in this second actual robot arm position;
• - Means for determining a second actual pose of the first finger based on this image; and
• - Means for calibrating the control based on this determined second actual pose of the first finger and a second target pose of the first finger determined for this second actual robot arm position and second actual pose of the first finger.

• - Mitteln zum Ermitteln einer zweiten Ist-Pose des zweiten Fingers auf Basis des Bildes, das bei der zweiten Ist-Roboterarmstellung aufgenommen worden ist; und
• - Mitteln zum Kalibrieren der Steuerung auf Basis dieser ermittelten zweiten Ist-Pose des zweiten Fingers und einer für diese zweite Ist-Roboterarmstellung und zweite Ist-Pose des zweiten Fingers ermittelten zweiten Soll-Pose des zweiten Fingers.

In one embodiment, the system or its control or its means has:

• - Means for determining a second actual pose of the second finger based on the image taken at the second actual robot arm position; and
• - Means for calibrating the control based on this determined second actual pose of the second finger and a second target pose of the second finger determined for this second actual robot arm position and second actual pose of the second finger.

In one embodiment, the system or its control or its means has: means for determining, in particular parameterizing and/or storing, an image and/or correction between actual poses and target poses of a finger of the gripper for calibrating the control .

In one embodiment, the system or its control or its means has: means for determining an actual pose of a finger based on an image based on image processing of this image and / or a theoretical model of the gripper, in particular based on object recognition.

A system and/or a means in the sense of the present invention can be designed in terms of hardware and/or software technology, in particular at least one processing unit, in particular a microprocessor unit, preferably connected to a memory and/or bus system with data or signals, in particular a digital processing unit ( CPU), graphics card (GPU) or the like, and/or one or more programs or program modules. The processing unit can be designed to process commands that are implemented as a program stored in a memory system, to detect input signals from a data bus and/or to deliver output signals to a data bus. A storage system can have one or more, in particular different, storage media, in particular optical, magnetic, solid-state and/or other non-volatile media. The program can be designed in such a way that it embodies or is able to carry out the methods described here, so that the processing unit can carry out the steps of such methods and thus in particular can calibrate the control or operate the robot arm with the gripper guided by the robot arm. In one embodiment, a computer program product can have, in particular, a storage medium, in particular a computer-readable and/or non-transitory storage medium, for storing a program or instructions or with a program or with instructions stored thereon. In one embodiment, executing this program or these instructions by a system or a controller, in particular a computer or an arrangement of several computers, causes the system or the controller, in particular the computer or computers, to perform a method described here or to carry out one or more of its steps, or the program or the instructions are set up for this purpose.

In one embodiment, one or more, in particular all, steps of the method are completely or partially implemented by computer or one or more, in particular all, steps of the method are carried out completely or partially automatically, in particular by the system or its means.

• 1: ein System zum Kalibrieren einer Steuerung zum Steuern eines Roboterarms und eines von dem Roboterarm geführten Greifers nach einer Ausführung der vorliegenden Erfindung; und
• 2: ein Verfahren zum Kalibrieren der Steuerung nach einer Ausführung der vorliegenden Erfindung.

Further advantages and features result from the subclaims and the exemplary embodiments. This shows, partly schematized:

• 1 : a system for calibrating a controller for controlling a robot arm and a gripper guided by the robot arm according to an embodiment of the present invention; and
• 2 : a method for calibrating the controller according to an embodiment of the present invention.

1 shows a system for calibrating a controller 3 for controlling a robot arm 10 and a gripper guided by the robot arm according to an embodiment of the present invention.

The robot arm 10 has a robot base 11, six swivel joints, the positions of which are q ₁ ,..., q ₆ are indicated, and an end flange 12, to which a base 20 of the gripper is attached, on which a translationally adjustable first finger 21 and a second finger 22 of the gripper are arranged, the adjustment of which for opening or closing the gripper is indicated by movement double arrows .

The controller 3 communicates with the robot arm 10 and grippers, in particular their drives (not shown), and a 3D recording device 4.

In a first step S10 (cf. 2 ), the robot arm 10 is adjusted so that the gripper can be captured as well as possible by the 3D recording device 4. The two fingers 21, 22 show the example in 1 indicated actual finger positions relative to the gripper base 20.

In a step S20, a first image of the robot-guided gripper is recorded using the 3D recording device 4 in this first actual robot arm position of the robot arm and this first actual finger position of the first finger 21 and second finger 22.

In a step S30, a first actual pose of the first finger 21 and a first actual pose of the second finger 22 are determined relative to the gripper base 20 on the basis of this first image. By way of example, the first actual pose y ₂₁ of the first finger 21 relative to the gripper base 20 is symbolized by an arrow, which indicates a position of the fingertip 21A in a coordinate system K ₂₀ fixed to the gripper base. This determination is carried out in a manner known per se and therefore not explained in more detail here, for example by recognizing edges or corners of the fingers 21, 22 and the gripper base in a point cloud recorded using the 3D recording device, advantageously on the basis of a CAD model of the gripper .

In a step S40, a first target pose of the first finger and a first target pose of the second finger 22 are determined for these first actual finger positions and this first actual robot arm position, as they result from the commanded or approached ones, in one embodiment sensor-detected, actual finger positions of the first finger 21 and second finger 22 each relative to the gripper base 20 and a corresponding kinematics model of the gripper. Again, as an example, the first target pose x ₂₁ of the first finger 21 relative to the gripper base 20 is symbolized by an arrow. The configuration 21 'of the first finger 21 is indicated by dashed lines in accordance with the kinematics model. It can be seen that the actual pose y ₂₁ deviates from the target pose x ₂₁ , so that the fingertip 21A does not have the desired pose, which should be approached by approaching or commanding the corresponding robot arm and finger positions.

Accordingly, based on the (deviation between) actual pose y ₂₁ and target pose x ₂₁ , a correction or image y ₂₁ = y ₂₁ (x ₂₁ ) or x ₂₁ = x ₂₁ (y ₂₁ ) can be determined, which corresponds to this Deviation reduced, preferably minimized. Based on the exemplary embodiment, it becomes clear that the target pose can, for example, only be a one-dimensional position or adjustment path of the finger relative to the gripper base 20 in its adjustment direction, in which case each such position is, for example, the three-dimensional one realized thereby -Position of the fingertip or vice versa, each three-dimensional actual position of the fingertip can be assigned a corresponding one-dimensional position of the finger relative to the gripper base 20 in order to approach it.

The determination of the first target poses or step S40 can also take place before or during the adjustment of the robot arm 10 (step S10), recording the image (step S20) or determining the actual poses (step S30).

In a step S50, in the first actual robot arm position, i.e. H. (still) without adjusting the robot arm 10, the first finger 21 and the second finger 22 are each adjusted to a second actual finger position and, in the manner explained above with reference to steps S20-S40, an image of the robot-guided gripper in the first actual robot arm position and recorded with the second actual finger positions, from which the respective (second) actual pose of the first or second finger is determined and the corresponding (second) target poses from the commanded or approached actual finger positions, which are detected by sensors in one embodiment of the first and second fingers.

Then, in a step S60, the robot arm 10 is adjusted to a second actual robot arm position, ie one or more of the joint positions q ₁ ,..., q ₆ is changed, and in the manner explained above with reference to steps S20-S40 and S50 an image of the robot-guided gripper is recorded in this second actual robot arm position and with the second actual finger positions and from this a (further second) actual pose of the first or second finger is determined. Since, as explained above, the actual poses are determined relative to the gripper base 20, the associated target poses advantageously do not change, so that they were determined in step S50 or the target poses determined in this process are also used as target poses. Poses for the second actual robot arm position and second actual finger positions can be adopted and thereby determined.

Then in a step S70 in the second actual robot arm position, ie without adjusting the robot arm 10, the first finger 21 and the second finger 22 are each adjusted into the first or another actual finger position explained above and in the above with reference to the steps S20-S40, S50 or S60, an image of the robot-guided gripper in the second actual robot arm position and with these first or other actual finger positions is recorded, from which a (different second) actual pose of the first or second finger is determined and the corresponding (second) target poses of the first and second fingers are determined from the commanded or approached actual finger positions, which in one version are detected by sensors.

In a step S80, the controller 3 is calibrated on the basis of these actual and target poses of the two fingers 21, 22 determined for the different robot arm and finger positions. In a very simple example, an offset between the one-dimensional position of the respective finger relative to the gripper base 20 in its adjustment direction and the corresponding position of its fingertip is parameterized in such a way that the absolute deviation between the target and actual poses becomes minimal on average. It is clear to the person skilled in the art that more complex images can advantageously be determined, which assign corresponding actual fingertip poses to commanded positions of the fingers (forward kinematics or transformation) or, conversely, assign finger positions to be commanded to desired fingertip poses (backward kinematics or transformation).

In a step S90, the robot arm 10 and the gripper guided by the robot arm are controlled using the controller calibrated in step S80.

Although exemplary embodiments have been explained in the preceding description, it should be noted that a variety of modifications are possible.

In the first and/or second actual robot arm position, one or more further finger positions and/or additionally one or more further actual robot arm positions can be approached, each with one or more actual finger positions, and corresponding additional actual poses of the or the finger is determined and taken into account or used during calibration. By using two or more actual robot arm positions and the use of two or more actual finger positions, a broader database can advantageously be used for calibration and precision, validity and/or reliability can thereby be improved. It should also be noted that the exemplary statements are merely examples and are not intended to limit the scope of protection, applications and structure in any way. Rather, the preceding description provides the person skilled in the art with a guideline for the implementation of at least one exemplary embodiment, whereby various changes, in particular with regard to the function and arrangement of the components described, can be made without leaving the scope of protection, as it appears the claims and combinations of features equivalent to them.

Reference symbol list

3

steering

4

Recording device

10

Robot arm

11

Robot arm base

12

Robot arm end flange

20

Gripper base

21

first finger

21'

First finger configuration

21A

fingertip

22

second finger

q1 - q6

Robot arm joint position

x21

Target pose

y21

Actual pose

Claims (11)

Method for calibrating a controller (3) for controlling a robot arm (10) and/or a gripper (20, 21, 22) guided by the robot arm, with the steps: - Recording (S20) a first image of the robot-guided gripper at a first actual robot arm position of the robot arm and a first actual finger position of a first finger (21) of the gripper relative to a base (20) of the gripper using a recording device (4); - Determining (S30) a first actual pose of the first finger based on this first image; - Determining (S40) a first target pose of the first finger for this first actual finger position and this first actual robot arm position; and - Calibrate (S80) the control based on this determined first actual pose and first target pose of the first finger. Procedure according to Claim 1 , with the steps: - Adjusting the first finger to a second actual finger position in which the first finger is relative to the Base of the gripper has a changed position; - Determining a second target pose of the first finger for this second actual finger position; - Recording at least one image of the gripper in this second actual finger position of the first finger; and - determining a second actual pose of the first finger based on this image; wherein the control is calibrated based on this determined second actual pose and second target pose of the first finger. Method according to one of the preceding claims, with the steps: - Determining a first target pose of at least a second finger (22) of the gripper for a first actual finger position of the second finger relative to the base of the gripper; and - Determining a first actual pose of the second finger based on an image which was recorded with this first actual finger position of the second finger; wherein the control is calibrated based on this determined first actual pose and first target pose of the second finger. Procedure according to Claim 3 , with the steps: - adjusting the second finger into at least one second actual finger position, in which the second finger has a changed position relative to the base of the gripper; - Determining a second target pose of the second finger for this second actual finger position; - Determining a second actual pose of the second finger based on an image which was recorded with this second actual finger position of the second finger; wherein the control is calibrated based on this determined second actual pose and second target pose of the second finger. Method according to one of the preceding claims, with the steps: - Adjusting the robot arm into at least a second actual robot arm position; - Recording at least one image of the gripper in this second actual robot arm position; and - Determining a second actual pose of the first finger based on this image; wherein the control is calibrated on the basis of this determined second actual pose of the first finger and a second target pose of the first finger determined for this second actual robot arm position and second actual pose of the first finger. A method according to the preceding claim, wherein the method comprises the step: - Determining a second actual pose of the second finger based on the image taken at the second actual robot arm position; wherein the control is calibrated on the basis of this determined second actual pose of the second finger and a second target pose of the second finger determined for this second actual robot arm position and second actual pose of the second finger. Method according to one of the preceding claims, wherein calibrating the control comprises determining an image and/or correction between actual poses and target poses of at least one finger of the gripper; and/or wherein at least an actual pose of a finger is determined based on an image based on object recognition and/or a theoretical model of the gripper. Method for operating a robot arm (10) with a gripper (20, 21, 22) guided by the robot arm using a controller (3), the controller being calibrated according to one of the preceding claims and the robot arm and/or gripper being controlled using the calibrated controller becomes. System for calibrating a controller (3) for controlling a robot arm (10) and/or a gripper (20, 21, 22) guided by the robot arm, the system being set up to carry out a method according to one of the preceding claims. System after Claim 9 which is set up to control the robot arm and/or gripper and/or has: - a robot arm (10) and a gripper (20, 21, 22) guided by the robot arm; - a recording device (4) for recording a first image of the robot-guided gripper at a first actual robot arm position of the robot arm and a first actual finger position of a first finger (21) of the gripper relative to a base (20) of the gripper; and - a controller (3) for controlling the robot arm and/or gripper with: - means for determining a first actual pose of the first finger based on this first image; - Means for determining a first target pose of the first finger for this first actual finger position and this first actual robot arm position; and - means for calibrating the control based on this determined first actual pose and first target pose of the first finger. Computer program or computer program product, wherein the computer program or computer program product, in particular stored on a computer-readable and / or non-volatile storage medium, contains instructions that when executed by one or more computers or a system Claim 9 or 10 cause the computer(s) or system to perform a procedure according to one of the Claims 1 until 8th to carry out.

Images