FR3145085A1 - Method and system for controlling a robotic system - Google Patents

Abstract

Method for controlling a slave robotic system Method for controlling a slave robotic system by a master control system comprising at least one vibrator, in which: - at least one quantity F is determined as a function of at least one force and/ or at least one torque exerted or experienced by the slave robotic system during its interaction with an environment in which it operates; - we calculate from this quantity and a predefined control law a vibration feedback signal which can take at least three levels depending on the value of said quantity; - the vibrator is controlled according to this signal so as to generate at least three corresponding levels of vibration intensity felt by the user. Figure for abstract: Fig. 1

Description

Method and system for controlling a robotic system

The present invention relates to robotic teleoperation.

Robotic teleoperation consists of entrusting control of a robot, called a "slave", to a human operator, by means of a control system, called a "master", the latter being able to be a remote control arm or a motion capture tool such as a joystick, among other devices.

For a number of operations in hazardous (e.g. nuclear) or inaccessible areas, or in complex environments where automation is not cost-effective or is difficult to envisage, teleoperation is an effective solution.

Force feedback teleoperation allows the operator to evaluate the forces applied by the robot, thus enabling them to better manage potential collisions, or expected interaction forces (for assembly or finishing, for example).

US Patent Application 2017/0108929 and the Park et al. article “ A Tele-operation Interface with a Motion Capture System and a Haptic Glove ” disclose haptic glove motion capture systems for virtual reality applications with haptic feedback in the form of vibrations when the system detects that the user is coming into contact with a virtual object.

In the paper Ju et al. “ Teleoperation of Humanoid Baxter Robot Using Haptic Feedback ”, a teleoperation system using a Baxter robot is proposed, which includes haptic feedback enabling so-called bilateral manipulation of the teleoperation system. This paper presents a position-position control strategy with force feedback where the slave robot arm moves according to the master’s motion. A workspace mapping method is proposed, facilitating the possibility of matching the master and slave workspace boundaries. An algorithm ensures the accuracy and position transformation between master and slave. A haptic rendering algorithm is designed for haptic feedback interaction.

A reactive motion capture system that combines a haptic feedback motion capture system and a human-scale virtual environment is proposed in the paper Choi et al. “ A Development and Evaluation of Reactive Motion Capture System with Haptic Feedback ”.

The article Pacchierotti et al. “ Teleoperation of Steerable Flexible Needles by Combining Kinesthetic and Vibratory Feedback ” presents a teleoperation system for steering flexible surgical needles, in which vibrations on a master remote control arm are used to indicate a needle orientation direction in the slave system coordinate system. The proposed teleoperation system tracks the needle position using an ultrasound imaging system and calculates the ideal needle position and orientation to reach a given target. A haptic interface then provides the clinician with mixed kinesthetic and vibratory navigation cues to guide the needle to the calculated ideal orientation and position.

The Springer journal Girbés-Juan et al. “ Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot ” discloses a bimanual teleoperation system with a force-feedback remote control arm.

US Patent Application 2006/0290662 discloses a haptic feedback system with different vibration patterns. A wide variety of actuator types can be used to provide synchronized vibration, including linear actuators, rotary actuators, rotating eccentric mass actuators, and oscillating mass actuators. A controller can send signals to one or more driver circuits to direct the operation of the actuators. The controller can provide direction and amplitude control, vibration control, and frequency control to direct the haptic experience. Parameters such as frequency, phase, amplitude, duration, and direction can be programmed or input as different patterns suitable for use in gaming, virtual reality, and real-world situations.

US Patent 8,190,292 relates to high frequency force feedback in a robotic teleoperation system where the master controller receives force or velocity information. The controller of the telerobotic system is characterized by combining high frequency information with low frequency position or velocity information. The controller is useful for teleoperations with or without delay between the communication channels of the master and slave devices.

Existing systems leave the operator unaware of the intensity of the efforts he applies with the robot on its environment, which creates a difficulty during its teleoperation.

There is therefore a need to facilitate the remote operation of robotic systems and the invention meets this objective by proposing, according to one of its aspects, a method of controlling a slave robotic system by a master control system comprising at least one vibrator, in which:

- at least one quantity F is determined as a function of at least one force F _trans and/or at least one torque T _rot exerted or undergone by the slave robotic system during its interaction with an environment in which it operates;

- from this quantity F and a predefined control law, a vibration feedback signal is calculated which can take at least three levels depending on the value of said quantity F;

- the vibrator is controlled based on this vibration feedback signal so as to generate vibrations with a corresponding level of intensity felt by the user.

The invention makes it possible to deliver force information to the user, namely the intensity of vibration felt, by using said quantity F to modulate the intensity of the vibrations generated, according to the predefined control law. The vibratory feeling of the user can take, depending on the amplitude of the quantity F, at least three levels perceived differently by the user, which facilitates the remote operation of the slave system. It is therefore not a binary feeling, vibrator activated or not, but a feeling that can vary in amplitude between minimum and maximum values with at least one intermediate level.

The invention can make it possible to communicate to the operator an effort or contact concerning the slave robotic system in a few tens of milliseconds, whereas bilateral effort feedback is generally constrained to higher response times.

Preferably, the vibration feedback signal is capable of taking a large number of values, in particular more than three values, for example 2 ⁿ values, where n denotes the number of bits on which this signal is coded. The vibrator can be controlled with as many different intensities as there are values of the vibration feedback signal, but it is also possible to have control levels of the vibrator for respective ranges of values of the vibration feedback signal; the number of different operating modes of the vibrator, corresponding to as many control levels, can thus be between 2 ⁿ /50 and 2 ⁿ , for example. Each operating mode of the vibrator advantageously corresponds to a level of intensity felt different from any other mode.

To vary the level of intensity felt, the vibratory amplitude of the vibrator, at constant or variable frequency, and/or the duration during which the vibrator is operating per unit of time can be varied. For example, at least three different levels of intensity felt can correspond to a zero level (without vibration), a maximum level where the vibrator operates 100% of the time over a period of 1 s and at least one intermediate level where the vibrator operates for example between 20% and 80% of the time over a period of 1 s, for example at 50%, alternating operating periods of 250 ms and rest periods of 250 ms.

Thus, in exemplary embodiments, the vibrator is controlled by modulating its operating time over a given period of time, this modulation causing a modulation of the vibration intensity felt by the user. For example, there is PWM type control (duty cycle modulation).

Preferably, the control law includes a dead zone in which no vibration is triggered as long as said quantity F does not exceed a predefined triggering threshold. This limits the risk of an untimely triggering of the vibrator.

The control law preferably includes a saturation zone from which the image by this law of the quantity F no longer increases or increases less and less with the quantity F so as to reach a predefined ceiling even when said quantity F continues to increase. The saturation zone can be chosen according to a threshold of force or torque exerted or undergone by the robot beyond which the operator's knowledge of the amplitude of the force or torque is no longer as useful for teleoperation as lower values.

The control law may be a ReLu or sigmoid type function, among other possible functions, preferably having a relatively steep slope, which can be adjusted to give a maximum variation in vibration amplitude felt by the user for a given force or torque range.

Preferably, said quantity F is given by the weighted sum of the norms of the components of force F _trans and torque T _rot : |.

The robotic system may include an actuator, such as a gripper, and , L _clamp being a characteristic dimension of the clamp, in particular the length of the lever arm of the clamp, which can correspond substantially to the length of the branches of the clamp.

The characteristic dimension of the clamp can be of the same order of magnitude as the dimensions of the object manipulated.

In one embodiment, , which does not prevent maintaining a feeling consistent with the efforts encountered by the slave system.

The maximum vibration intensity felt is preferentially , c being a scaling coefficient, the control law being preferably calibrated such that the maximum intensity is felt when said quantity F reaches a predefined value F _max .

The master control system may include a user movement capture system, the vibrator then equipping, for example, a remote control device in the form of a joystick called a “controller”, independent, manipulated by the user, the movements of this joystick being detected, for example, optically and/or using an inertial unit or an accelerometer integrated therein.

Alternatively, the master control system includes a master remote control arm or a haptic glove.

In one embodiment, in the event of detection of a coupling control signal originating from a user action on an interface, for example a button present on the aforementioned joystick or arm, the set position of the slave robotic system is reset to its current position.

Control system and teleoperation installation

The invention also relates, according to another of its aspects, to a control system to be connected to a command system comprising a user movement capture system equipped with a vibrator, the control system being intended to control a slave robotic system, the control system being configured to implement the method according to the invention as defined above, and being arranged to:
- determine at least one quantity F as a function of at least one force F _trans and/or at least one torque T _rot exerted or undergone by the slave robotic system during its interaction with the environment in which it operates;
- calculate, from this quantity F and a predefined control law, a vibrational feedback signal, which can take at least three levels depending on the value of said quantity;
- control the vibrator according to this signal, so as to generate vibrations with a corresponding level of intensity felt by the user.

The invention also relates, according to another of its aspects, to a teleoperation installation comprising the control system according to the invention, the control system comprising the vibrator, and the slave robotic system.

The invention may be better understood by reading the detailed description which follows, non-limiting examples of its implementation, and by examining the attached drawing, in which:

There is a schematic view of an example of a teleoperation installation according to the invention; and

there graphically illustrates examples of control laws.

Detailed description

We have illustrated schematically in the an example of a teleoperation installation 1 according to the invention.

This installation 1 comprises a control system 2 and a slave robotic system 3 operated remotely by the control system 2 via a control system 4.

Robotic system

The slave robotic system 3 can be any robot, for example a six-axis robotic arm, or any other number of joints.

The slave robotic system 3 may be equipped with any actuator, effector or gripping mechanism. For example, as illustrated, it includes a gripper 5 that can serve as a gripping member.

The robotic system 3 is equipped with force and/or torque sensors that make it possible to measure the force exerted or experienced by the robotic system on its environment. The robotic system 3 comprises, for example, sensors that make it possible to measure the clamping force of the gripper 5. It may also comprise torque sensors at the joints, which make it possible to know the forces exerted or experienced by the different segments of the arm.

Control system

The control system 2 is used to control the robotic system 3 remotely via the control system 4.

It features a user-operated motion capture system 25.

This capture system 25 may include a controller, as illustrated.

The controller may integrate an inertial unit and/or an optical or radiofrequency detection system to detect its position and movements when manipulated by the user. Adapted controllers, such as those marketed under the HTC VIVE or HP brand, exist on the market, which integrate one or more cameras to detect movements of the controller within their environment; other systems exist which include one or more fixed cameras and light markers on the controller, which are detected by this or these cameras.

The control system 2 may also include a remote control arm, for example of the joystick or remote control arm type.

The invention is not limited to a particular control system and it may be presented for example in a different form, for example a glove worn by the user.

The motion capture system comprises a vibrator 20. This vibrator 20 comprises, for example, a motor and a weight forming an unbalance. The vibrator may also be piezoelectric or any other electromechanical device.

Control system

• une chaîne aller qui permet d’envoyer au robot les informations de commande :
  • un module de mise en forme 200 qui reçoit des informations relatives au mouvement et/ou à la position du système de capture 25,
  • un module opérationnel 201 recevant du module de mise en forme 200 des données traitées relatives à la manipulation du système de capture, ainsi qu’une information de couplage Co,
  • un module de pilotage 202 recevant du module opérationnel 201 des informations relatives aux actions à effectuer par le robot, dont une consigne de position X_cons.
• une chaîne retour qui permet d’assurer le pilotage du vibreur 20, et qui comprend :
  • un module de traitement 300, et
  • un module de conversion 301.

Control system 4 includes:

• a go string that allows you to send the command information to the robot:
  • a shaping module 200 which receives information relating to the movement and/or position of the capture system 25,
  • an operational module 201 receiving from the formatting module 200 processed data relating to the manipulation of the capture system, as well as coupling information Co,
  • a control module 202 receiving from the operational module 201 information relating to the actions to be carried out by the robot, including a position instruction X _cons .
• a return chain which allows the control of the vibrator 20, and which includes:
  • a 300 processing module, and
  • a 301 conversion module.

These modules correspond to functionalities of the control system 4 which can be carried out by any suitable computer system, comprising for example one or more microcomputers and/or microcontrollers and/or specialized circuits, as well as all necessary interfaces.

Several modules can correspond to blocks of a program executed on the same hardware entity.

The control system 4 is shown in the figure as a separate block from the control system 2, but all or part of the control system may, if necessary, be integrated into the controller or other capture system 25 used.

The module 200 receives from the capture system 25 information on the latter's position or speed, in this case its current X _tracker position. This data undergoes calibration, in a manner known per se, in order to match given movements of the capture system 25 with the desired corresponding movements of the robot 3.

The module 201 receives from the capture system 25 a coupling information Co; this information can be a binary information, generated for example by pressing a coupling button. For example, if the information Co is true, which corresponds to the absence of pressing the coupling button for example, the calibrated _tracker X position information is processed by the module 201 which emits the instruction X _cons . This instruction X _cons is transmitted to the robotic control module 202 which ensures that the robot 3 follows these instructions.

If Co is false, which corresponds for example to pressing the coupling button, the setpoint position of the robot 3 is repositioned to the current position of the capture system 25, which can allow the contact forces to be released quickly and intuitively. Indeed, upon contact, forces are generated when the setpoint position is moved beyond the contact surface, the rigidity of the control (the ‘spring’ component) then inducing an interaction force. By resetting this position, the forces are then cancelled (at least in the case of rigid materials). This functionality offers a quick possibility of resetting a contact when the operator is no longer able to evaluate the direction of the interaction forces (which are not represented by the vibration feedback).

The force information transmitted by the sensors of the robotic system 3 to the module 300 relates to at least one force F _trans and/or torque T _rot exerted or undergone by the robotic system 3, in particular a force torque on the gripper 5, during its interaction with the environment.

These data are processed by the module 300, in particular to filter them, because the robot's joint force and/or torque sensors can emit noisy data.

The filtered signal is then converted into a scalar value by the conversion module 301.

This module 301 calculates in step 31 a quantity F which is a function of at least the force F _trans and/or at least the torque T _rot , then calculates in step 32 from this quantity F and a predefined control law a vibration feedback signal.

This vibration feedback signal is processed in step 33 to be transformed into a control signal capable of taking at least three levels depending on the value of said quantity F, so as to generate at least three corresponding levels of vibration intensity felt by the user, F being expressed by:

.

It is preferable to set α=1 and to adapt β according to the dimensions of the clamp 5 (which conditions the torques felt during contacts). For example, we have , L _clamp being the lever arm length of the clamp.

Then the scalar value F is scaled by a coefficient c equal to with the maximum felt vibrational intensity, and the maximum force standard considered to be relevant to distinguish (beyond , saturation blocks the vibration at its maximum intensity). This coefficient c reflects in particular the sensitivity of the haptic feedback and the order of magnitude of the efforts that we seek to express.

The control law can be an activation function, of the Rectified Linear Unit (ReLu) or sigmoid type, depending on the desired feeling.

Several examples of ReLu and sigmoid functions are given in .

The graphs represent on the ordinate the duration of the haptic pulsation, which can correspond to the duration of activation of the vibrator, for example in µs depending on the quantity F, for example expressed in N, the numerical values indicated being purely indicative and may be different depending on the applications.

The first line of graphs illustrates a ReLu function, the second line a sigmoid, the third line a steep sigmoid, and the fourth line a flat sigmoid.

The first column represents each of these functions with a dead zone of 50 and a saturation zone of 60.

The second column represents the ReLu function without the saturation region.

The third column represents the ReLu function without dead zone or saturation and the other sigmoid functions without the dead zone.

The dead zone 50 corresponds to a range of values of the quantity F in which no vibration is triggered as long as said quantity F does not exceed a predefined trigger threshold F _min .

The saturation zone 60 corresponds to a range of values of the quantity F from which the haptic pulsation duration no longer increases or increases less and less so as to remain below a predefined ceiling even when said quantity F continues to increase.

One can adapt an overall gain of the control system (keeping the saturation) to feel low efforts and more subtle variations of efforts. One can increase the gain and introduce an offset from zero in the activation function to obtain a feeling of low variations for higher force values.

Beyond the gain modification, the operator can also learn to gauge equivalences between vibration intensity and real effort. In practice, this is quite intuitive; for example, it is enough to manipulate a known flexible object for a short time while visually observing its deformations.

The signal thus obtained at the output of the conversion module 301 is transmitted to the vibrator 20, giving the operator feedback on the forces applied to the clamp 5.

Vibrator 20 provides a non-binary level of felt intensity such that 0 ≤ ≤ .

For example, when the clamp 5 is not serving any object, the vibrator 20 remains inactive. When the clamp 5 applies the maximum tightening, the vibration felt is maximum. When the tightening torque is intermediate, the level of vibration felt by the user is intermediate.

The invention is not limited to the exemplary embodiments described above, and can be applied to the detection of collisions of any part of the slave robotic system by equipping the robotic system with joint torque sensors, unlike slave robots equipped with force sensors on the effector where only the forces applied by the effector are detected.

Claims (12)

Method for controlling a slave robotic system (3) by a master control system (2) comprising at least one vibrator (20), in which:
- at least one quantity F is determined (31) as a function of at least one force F _trans and/or at least one torque T _rot exerted or undergone by the slave robotic system (3) during its interaction with an environment in which it operates;
- we calculate (32) from this quantity F and a predefined control law a vibrational feedback signal which can take at least three levels depending on the value of said quantity F;
- the vibrator (20) is controlled (33) as a function of this vibration feedback signal, so as to generate vibrations with a corresponding level of intensity felt by the user. Method according to the preceding claim, the control law comprising a dead zone in which no vibration is triggered as long as said quantity F does not exceed a predefined triggering threshold (F _min ). Method according to one of the two preceding claims, the control law comprising a saturation zone from which the image by this law of the quantity F no longer increases or increases less and less with the quantity F so as to reach a predefined ceiling even when said quantity F continues to increase. Method according to any one of the preceding claims, the control law being a ReLu or sigmoid type function. Method according to any one of the preceding claims, the control of the vibrator (2) being carried out by modulating its operating time during a given period of time, this modulation causing a modulation of the vibratory intensity felt by the user. Method according to any one of the preceding claims, said quantity F being given by the weighted sum of the norms of the components of force F _trans and of torque T _rot : |. Method according to the preceding claim, the slave robotic system (3) comprising an actuator, such as a gripper (5), and , L _clamp being a characteristic dimension of the clamp. A method according to any preceding claim, the maximum felt vibration intensity being , c being a scaling coefficient, the control law being preferably calibrated such that the maximum intensity is felt when said quantity F reaches a predefined value F _max . Method according to any one of the preceding claims, the master control system (2) comprising a system (25) for capturing user movement. Method according to any one of the preceding claims, in which in the event of detection of a coupling control signal originating from a user action on an interface, the set position of the slave robotic system (3) is reset to its current position. Control system (4) to be connected to a control system (2) comprising a system (25) for capturing the movement of the user equipped with a vibrator (20), the control system (4) being intended to control a slave robotic system (3), the control system (4) being configured to implement the method according to any one of the preceding claims and being arranged to:
- determine (31) at least one quantity F as a function of at least one force F _trans and/or at least one torque T _rot exerted or undergone by the slave robotic system (3) during its interaction with the environment in which it operates;
- calculate (32), from this quantity F and a predefined control law, a vibrational feedback signal which can take at least three levels depending on the value of said quantity;
- control (33) the vibrator (20) according to this signal so as to generate vibrations with a corresponding level of intensity felt by the user. Teleoperation installation (1) comprising the control system (4) according to the preceding claim, the control system (2) comprising the vibrator (20) and the slave robotic system (3).