FR3130125A1 - Method and system for simulating the mechanical behavior of a body - Google Patents

Abstract

Method and system for simulating the mechanical behavior of a body Method for simulating the mechanical behavior of a body (200), comprising the steps consisting in: configuring from a digital twin (231) a dynamic model (100) comprising a deformable interface (101) linked to a plurality of active support elements (103) whose travel and mechanical response are adjusted to reproduce with the interface (101) the external geometry of the body (200) and to simulate its mechanical behavior , the digital twin (231) having been previously generated by acquiring the external geometry of at least part of said body (200) and its local response to the application of a force (F1, F2, F3, F1', F2', F3') Figure for abstract: Fig. 2

Description

Method and system for simulating the mechanical behavior of a body

The present invention relates to the simulation of the mechanical behavior of a body, in particular but not exclusively with a view to facilitating the tele-operation by the doctor of a probe during the diagnosis of a remote patient.

Requests for tele-operation are increasing every year, especially in the medical field.

This is to meet the need to quickly provide medical access where a doctor is not always available.

The Springer 2021 review “ Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends” provides an overview of the newest robotic ultrasound systems that have emerged over the past five years.

Among these systems, the MELODY system from the French company Adechotech uses, on the patient's side, a bracket that must be manipulated by hand, with an effector at the end, carrying a slave probe to be moved over the patient's body. This effector is slaved to the movements of a master probe on the side of the remote operator. The manipulation of the master probe is done without providing haptic feedback or representativeness of the body being explored. In addition, this system requires the assistance of a person on site next to the patient to manipulate the slave probe, which limits its use to situations where such human assistance is available.

An analog RADIUS system was developed at the Institute of Machinery and Materials of South Korea in 2017, which includes a master device comprising a platform mounted on jacks and capable of recording the movements of a probe manipulated by the doctor, and a slave device provided with handles, to be applied to the patient, comprising a mechanism for moving a slave probe in response to the movements of the master probe. Such a remote diagnosis system does not include haptic feedback and does not offer representativeness of the explored body. Moreover, it also requires an assistant to be near the patient to maintain the slave device (http://koreabizwire.com/s-korean-institute-develops-remote-ultrasound-diagnostic-system/102708).

In order to overcome the difficulties related to the lack of representativeness of the body explored on the master side, research has recently been carried out to offer force feedback to the operator. The work of the University of Catania, published in 2020 in the article Bucolo et al. "Force Feedback Assistance in Remote Ultrasound Scan Procedures " , are the best example. The system developed allows diagnosis to be carried out by a collaborative robot remotely controlled by the movement of a probe by an operator, without the need for a human assistant on the patient side, as well as the measurement by the robot of the mechanical resistance opposed by the patient, and the return of this measurement to the operator.

The system operates in a closed loop using a dynamic surface with a general shape predefined on the operator side, composed of pneumatic chambers controlled by electro-valves, making it possible to model the changes in stiffness of the body explored on the master side.

However, such a system turns out to be relatively unresponsive and lacking in resolution, and in particular does not make it possible to properly simulate hard stops representative of the presence of obstacles such as bones in the human body. Its slowness makes it difficult to reconcile with the time constraints of most interventions.

There therefore remains a need to further improve remote haptic exploration processes and systems, particularly in terms of comfort for the operator, efficiency and speed, in order to facilitate the development of telediagnosis, particularly in the medical field, and more generally tele-operation.

There is also a need to facilitate the training of operators manipulating a probe or any other device on a body, by providing a realistic haptic response.

Method for simulating the mechanical behavior of a body

The invention aims to respond according to a first of its aspects to at least one of these needs, and has as its object a method for simulating the mechanical behavior of a body, comprising the steps consisting of:

• configure from a digital twin a dynamic model comprising a deformable interface linked to a plurality of active support elements, whose travel and mechanical response are adjusted to reproduce with the interface the external geometry of the body and to simulate its behavior mechanical,
• the digital twin having been previously generated by acquiring the external geometry of at least part of said body and its local response to the application of a force.

The invention makes it possible, according to this first aspect, to provide a human operator, in particular a doctor, with a dynamic interface that can be explored in a natural way having the same haptic characteristics as a body to be explored . This can facilitate the work of a teleoperator and can allow him to explore the body from a distance. This can also help in the training of a doctor in particular, by improving the realism of the simulation of the displacement of an object or of a hand in contact with a part of the human body.

The digital twin can be a "4D" digital model consisting of a computer file comprising a 3D map of the external geometry of the part of said body whose mechanical behavior is to be reproduced and a matrix representative of the mechanical resistances of said body at different forces. applied to it at different positions and/or geometrical orientations.

Pre-recorded real cases can be stored as digital twins, featuring the 4D model of the body in question. When the twin is generated, it is also possible to save diagnostic images generated during the passage of the tool in the real case, and then restore them during the simulation.

Preferably, the method for simulating the mechanical behavior comprises the detection of the positioning and/or the orientation relative to the dynamic model of a probe manipulated by a human operator, and the restitution to the operator of information which is function of the positioning and/or of the orientation thus detected(s) of the probe.

For example, in the case of training an operator, it involves pre-recorded ultrasound images on the real case from which the digital twin was generated. Thus, the information delivered to the operator is preferentially representative of that delivered by the probe when positioned and/or oriented identically on the body.

Another object of the invention is to meet the need mentioned above to improve the methods and systems for remote haptic exploration, in particular in terms of comfort for the operator, efficiency and speed, in order to facilitate the development of the telediagnosis, in particular in the medical field, and more generally tele-operation.

Haptic feedback remote control method

The invention thus relates, according to a second of its aspects, to a remote control method with haptic feedback, comprising the steps consisting of:

• generate in an initial phase a digital twin of at least one part of a body by acquiring its external geometry and its local response to the application of a force,
• configure from the digital twin thus generated a dynamic remote model comprising a deformable interface linked to a plurality of active support elements whose travel and mechanical response are adjusted to reproduce with the interface the external geometry of the body and to simulate its behavior mechanical,
• detect, in a remote action phase on the body, the positioning and/or orientation relative to the dynamic model of a master object manipulated by a human operator,
• piloting from the positioning and/or the orientation thus detected(s) a robotic arm carrying a slave object to copy, with the slave object on the body, the positioning and/or the orientation of the master object on the deformable interface.

This process allows a more natural remote interaction between the human operator and the studied body and closer to the usual exploration.

The invention allows the human operator to have a representative model of the geometry and the resistance of the body remotely, allowing a more efficient and finer diagnosis than what is possible with current systems.

The invention offers the possibility of taking into account the temporal delays in the communications inherent in a remote diagnosis, thanks to the digital twin generated in the initial phase, to restore predicted mechanical resistances even before having completed the master-slave loop. - master. This makes it possible to better manage the communication delays which exist most of the time in remote diagnosis, by having an interface which reacts to the movements of the human operator almost in real time, rather than waiting for the return of the object. slave each time, as is the case for the system proposed by the University of Catania mentioned above.

The invention makes it possible to adapt the geometry and resistance of the interface to the exact body being diagnosed, rather than having a generic means poorly adapted to the body to be explored.

The step consisting in detecting on the interface the positioning and/or the orientation relative to the dynamic model of the master object can be carried out through specific sensors (for example magnetic, odometric sensors, etc.) which make it possible to detect and quantify the movement of the master object in space, or relative to the dynamic model.

The step consisting in detecting on the interface the positioning and/or the orientation relative to the dynamic model of the master object can still be carried out alternatively through vision tracking methods (camera tracking with or without visual target on the master object).

The measurement of the pressure applied by the master object on the interface can be carried out through signals recovered in the active support elements.

Tracking the master object's displacement and/or force can be accomplished with a combination of any or all of these techniques, to ensure accurate identification of the master object's position and/or the force it exerts on the interface.

Preferably, the master object at least partially reproduces the shape of the slave object, in particular that of the slave object coming into contact with the body. The slave object and the master object can thus have the same shape, or even be identical.

The remote control method according to this aspect of the invention preferably comprises the acquisition of a mechanical response during a positioning of the slave object imposed by that of the master object, the updating of the digital twin then adjusting the stroke of the active support elements and/or their response based on this acquisition. This real-time update makes it possible to take into account changes in the resistance of said body to the slave object, for example during the displacement of an organ or the encounter with a bone, and thus makes it possible to keep a model dynamic whose properties remain faithful as much as possible to those of the distant body to be explored. This can also make it possible to enrich the digital twin during the exploration, and thus to improve the precision of the simulation of the behavior during the examination; this can also make it possible, if desired, to start with a lower volume of data acquired in the initial phase, and therefore to shorten the duration thereof.

Preferably, the acquisition of the local response of the body to the application of a force is carried out during the initial phase at several points using the slave object, in particular by making it carry out with the robotic arm a predefined trajectory.

During the initial phase, the geometry of the body to be explored can be determined by any suitable means, for example by using one or more cameras, for example stereoscopic, one or more lasers or LIDAR, a feeler arm, or any sensor making it possible to obtain the desired 3D cartography.

The remote control method according to the invention may include a calibration phase between the external geometry of the body and the position of the robotic arm. This calibration can be done by any means, and for example by including the arm with a known position and/or orientation in the field of initial acquisition of the topology of the body.

The slave object can be a probe emitting signals, in particular an ultrasound probe, in particular an echographic probe carried by the robotic arm.

The master object can still be a haptic hand and the slave object a representative effector, such as a robotic hand.

The active support elements preferably include actuators, which are preferably linear, preferably electromechanical jacks, and make it possible to adapt the starting topology of the deformable interface to the geometry of the body.

The active support elements preferably each comprise a magnetorheological brake making it possible to modify the response of the active support element to a mechanical stress, this brake being controlled according to the digital twin. The brake makes it possible to finely and very quickly control the resistance opposed by the active support element when the human operator exerts a thrust on the deformable interface.

The brake may comprise an enclosure containing a magnetorheological fluid.

A magnetorheological brake makes it possible to impose a controllable viscous friction using a magnetic field whose intensity depends on that of the electric current used to generate the field. Indeed, magnetorheological fluids are suspensions of micrometric magnetic particles in non-magnetic liquids. When such a liquid is exposed to a magnetic field, the particles aggregate in the form of chains which greatly increase the resistance to flow.

The magnetic field necessary to control the viscosity of the fluid is provided by an electromagnet.

Each active support element can be provided with a return member, in particular a spring, deformable under the effect of a force exerted by the master object. Controlling the stroke of the aforementioned cylinder makes it possible to reproduce the geometry of the body to be explored; the cylinder can be blocked at the stroke corresponding to this geometry. The presence of the return device allows the cylinder to be able to sink when a force is exerted on the model by the operator, and therefore the model to sink locally. The presence of the brake makes it possible to modulate the depression stroke, for example by blocking the depression from a certain stroke, or to vary the force to be exerted to cause this depression, to simulate for example a hardening of the fabric or the presence of an obstacle that prevents further movement.

Thus, in the case of the use of a magnetorheological brake, the method may include controlling the intensity of a current sent to the brake to modify the viscosity of the magnetorheological fluid as a function, on the one hand, of the displacement of a rod of the active support element under the effect of a force exerted on the interface by the operator and on the other hand of the mechanical response recorded in the digital twin; for example, to simulate a sinking without significant mechanical resistance, the brake is not activated or slightly activated, which allows the rod of the active support element to sink in response to a force exerted by the operator , the return member being compressed during this depression; if after a certain driving stroke, it is appropriate to simulate a stop, linked for example to the presence of an underlying bone in the explored tissues, the current in the brake is increased, which blocks the descent of the rod of the active support element and therefore the sinking of the dynamic model locally; when the operator releases the pressure, this release can be detected, and the current in the brake reduced, to allow the rod of the active support element to return to its initial configuration under the effect of the return device.

The interface preferably includes a layer of viscoelastic material, which can define the surface that the human operator explores. This contributes to the flexibility of the deformable interface and can make it possible to have a haptic feedback closer to reality when exploring a human body thanks to the invention.

System for simulating the mechanical behavior of a body

Another subject of the invention, according to another of its aspects, is a system for simulating the mechanical behavior of a body, in particular for the implementation of a method according to the invention, comprising:

• a dynamic model comprising a deformable interface linked to a plurality of active support elements whose stroke and mechanical response are adjustable,
• a control system for controlling the travel and the mechanical response of the active support elements according to a digital twin of at least a part of said body, this digital twin comprising data relating to the external geometry of said body and its response local to the application of a force, the control of the active support elements being carried out in such a way as to reproduce with the interface the external geometry of the body and to simulate its mechanical behavior.

Preferably, the system according to the invention comprises a system for detecting the positioning and/or the orientation relative to the dynamic model of a master object manipulated by a human operator. This information can be transmitted remotely to control a robot carrying a slave object, as in the case of the method defined above.

Another subject of the invention, according to another of its aspects, is a remote control system with haptic feedback, comprising, in addition to the system according to the invention described above, a system for controlling a robotic arm for controlling, from the positioning and/or orientation thus detected, a robotic arm carrying a slave object and copying, with the slave object on the body, the positioning and/or the orientation of the master object on the deformable interface.

The invention also relates to a remote exploration system with haptic feedback, comprising, in addition to the control system according to the invention, the robotic arm carrying the slave object.

This slave object can advantageously be an echographic probe, as indicated above. Any type of echographic probe can be used in the context of the present invention, including linear and convex probes, or even endocavitary probes.

The robotic arm may be a collaborative robotic arm, or “cobot”.

System uses

The invention also relates, according to another of its aspects, to the use of the haptic feedback remote exploration system according to the invention for medical remote diagnosis, in particular ultrasound.

Another subject of the invention, according to another of its aspects, is the use of the haptic feedback remote control system according to the invention for the non-destructive testing of flexible or semi-rigid components or structures.

The invention also relates, according to another of its aspects, to the use of the system to simulate the mechanical behavior of a body for the purpose of recording case studies and their reproduction on the dynamic model to form new operators.

The invention may be better understood on reading the detailed description which follows, non-limiting examples of its implementation, and on examining the appended drawing, in which:

There is a block diagram of an example of a haptic feedback remote exploration system according to the invention;

there schematically represents the components of the system shown in ; And

there schematically illustrates an example of an actuator used in the system of the .

Claims (17)

Method for simulating the mechanical behavior of a body (200), comprising the steps of:

• configure from a digital twin (231) a dynamic model (100) comprising a deformable interface (101) linked to a plurality of active support elements (103) whose stroke and mechanical response are adjusted to reproduce with interface (101) the external geometry of the body (200) and simulate its mechanical behavior,
• the digital twin (231) having been previously generated by acquiring the external geometry of at least part of said body (200) and its local response to the application of a force (F1, F2, F3, F1 ', F2', F3').

Method according to the preceding claim, comprising the detection of the positioning and/or the orientation relative to the dynamic model of a probe (110) manipulated by a human operator (120), and the restitution to the operator (120) of information which is a function of the positioning and/or of the orientation thus detected(s) of the probe (110). Method according to the preceding claim, the information delivered to the operator (120) being representative of that delivered by the probe (110) when positioned and/or oriented in an identical manner on the body (200). Non-surgical haptic feedback remote control method, comprising the steps of:

• generating in an initial phase a digital twin (231) of at least one part of a body (200) by acquiring its external geometry and its local response to the application of a force (F1, F2, F3, F1', F2', F3'),
• configure from the digital twin (231) thus generated a remote dynamic model (100) comprising a deformable interface (101) linked to a plurality of active support elements (103) whose travel and mechanical response are adjusted to reproduce with the interface (101) the external geometry of the body (200) and simulate its mechanical behavior,
• detect, in a phase of remote action on the body (200), the positioning and/or the orientation relative to the dynamic model (100) of a master object (110) manipulated by a human operator (120),
• piloting from the positioning and/or orientation thus detected a robotic arm (220) carrying a slave object (210) to copy, with the slave object (210) on the body (200), the positioning and/or the orientation of the master object (110) on the deformable interface (101).

Method according to the preceding claim, the master object (110) at least partially reproducing the shape of the slave object (210), in particular that of the slave object (210) coming into contact with the body (200). Method according to one of the two preceding claims, the slave object (210) and the master object (110) having the same shape, or even being identical. Method according to any one of Claims 4 to 6, comprising the acquisition of a mechanical response during a positioning of the slave object (210) imposed by that of the master object (110), the updating of the digital twin (231) then adjusting the stroke of the active support elements (103) and/or their response based on this acquisition. Method according to any one of Claims 4 to 7, the acquisition of the local response to the application of a force (F1, F2, F3, F1', F2', F3') being carried out during the initial phase at several points using the slave object (210), in particular by having it perform a predefined trajectory with the robotic arm (220). Method according to any one of Claims 4 to 8, the slave object (210) being a signal-emitting probe, in particular an ultrasound probe, in particular an echographic probe. Method according to any one of the preceding claims, the active support elements (103) each comprising a magnetorheological brake (1034) making it possible to modify the response of the active support element (103) to a mechanical stress, this brake (1034 ) being driven according to the digital twin (231). A method according to any preceding claim, the active support elements (103) comprising linear actuators (1035). Method according to any one of the preceding claims, the interface (101) comprising a layer of viscoelastic material. System (1) for simulating the mechanical behavior of a body (200), in particular for the implementation of a method according to any one of the preceding claims, comprising:

• a dynamic model (100) comprising a deformable interface (101) linked to a plurality of active support elements (103) whose stroke and mechanical response are adjustable,
• a control system (106) for controlling the travel and the mechanical response of the active support elements according to a digital twin (231) of at least a part of said body (200), this digital twin (231) comprising data relating to the external geometry of said body (200) and its local response to the application of a force (F1, F2, F3, F1', F2', F3'), the control of the active support elements (103 ) being carried out in such a way as to reproduce with the interface (101) the external geometry of the body (200) and to simulate its mechanical behavior.

System according to claim 13, comprising a system (111) for detecting the positioning and/or orientation relative to the model (100) of a master object (110) manipulated by a human operator (120). Remote control system with haptic feedback, further comprising the system according to claim 14, a control system (211) of a robotic arm (220) for piloting, from the position and/or orientation thus detected ( s), a robotic arm (220) carrying a slave object (210) and copying, with the slave object (210) on the body (200), the positioning and/or orientation of the master object (110) on the deformable interface (101). Remote exploration system with haptic feedback, comprising, in addition to the control system according to claim 15, the robotic arm (220) carrying the slave object (210). System according to the preceding claim, the slave object being an echographic probe.