CN115227404A - Control method and system for ultrasonic-assisted scanning surgical robot - Google Patents
Control method and system for ultrasonic-assisted scanning surgical robot Download PDFInfo
- Publication number
- CN115227404A CN115227404A CN202210859548.6A CN202210859548A CN115227404A CN 115227404 A CN115227404 A CN 115227404A CN 202210859548 A CN202210859548 A CN 202210859548A CN 115227404 A CN115227404 A CN 115227404A
- Authority
- CN
- China
- Prior art keywords
- model
- robot
- scanning
- simulation
- ultrasonic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B2034/305—Details of wrist mechanisms at distal ends of robotic arms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Robotics (AREA)
- Animal Behavior & Ethology (AREA)
- Geometry (AREA)
- Computer Hardware Design (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Manipulator (AREA)
Abstract
The invention relates to the technical field of surgical robots, in particular to a control method and a control system of an ultrasonic-assisted scanning surgical robot. The method and the system set up a simulation environment as a robot dynamics simulation platform; a flexible soft model is set up in a simulation environment, and human body respiratory motion simulation is added on the basis of the soft model, so that the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen are truly reflected; carrying out ultrasonic scanning in the built simulation environment, and acquiring the state parameters of the robot in real time; and (4) reinforcement learning training, wherein the trained reinforcement learning model is applied to the robot, so that the accurate control of the robot is reinforced. The invention reduces the influence of respiratory motion on the autonomous ultrasonic scanning process, provides a set of training environment containing respiratory motion simulation, and can realize the ultrasonic autonomous scanning perpendicular to the body surface at constant contact force and constant speed.
Description
Technical Field
The invention relates to the technical field of surgical robots, in particular to a control method and a control system of an ultrasonic-assisted scanning surgical robot.
Background
The medical ultrasonic examination is a medical imaging diagnosis technology based on ultrasonic waves, can enable the size, the structure and the pathological focus of muscles and internal organs to be visualized, and has no radiation in the modern medical application compared with the examination such as CT, MRI and the like, low cost and wide applicability. In addition, the requirements for robot-assisted surgery are increasing at present, and the characteristics also enable ultrasonic examination to play a great role in robot-assisted surgery such as minimally invasive surgery. In the operation process of an entity and cavity intervention operation robot based on ultrasonic scanning, an ultrasonic probe is required to be capable of automatically scanning at constant contact force and constant speed to obtain a stable high-quality ultrasonic image, so that an accurate intervention position is obtained.
But previous ultrasound examinations relied on the operating experience and skill of the physician. The doctor is required to have richer operation experience on the angle, operation force and scanning speed of the probe. However, different doctors may also have different operation modes, and the generated images are not standard and inaccurate due to the difference of individual operations, which brings inconvenience to subsequent diagnosis. On the other hand, the working characteristics of ultrasonic examination also lead to that the doctor works in a fixed posture for a long time, and the risk that the doctor suffers from diseases such as joints and cervical vertebra is increased.
In order to solve the above problems, a large number of ultrasonic scanning robot solutions have been proposed up to now. The publication number CN113057673A, entitled a robot ultrasonic scanning control method, system, device and storage medium, provides a technical solution for calculating an expected speed of an ultrasonic probe by using an expected ultrasonic image characteristic and an actual ultrasonic image characteristic of an actual ultrasonic image displayed by an ultrasonic imager in real time, and calculating an expected joint speed of a cooperative robot according to the expected speed. The invention discloses a publication number CN114041828A, which is an ultrasonic scanning control method, a robot and a storage medium, and provides a robot system applied to breast ultrasonic scanning.
In view of the current design, despite the advantages of ultrasound examination, ultrasound scanning robots are not widely used due to the difficulties of many control techniques. The existing patent application with publication number CN113057673A needs to give the desired ultrasound image characteristics in advance for control, mainly controlling speed and position, but it cannot accurately detect the contact force between the robot end and the human body, nor control the attitude of the end, possibly resulting in instability of the obtained image, and possibly causing excessive mechanical arm acting force during detection, and this contact pressure may cause deformation of human tissue, and finally cause discomfort to the patient. The robot system with publication number CN114041828A builds a simulation platform for reinforcement learning, and obtains actual running state parameters in the process of scanning a target area by the robot, but the technical scheme does not consider the influence of human breathing motion on an ultrasonic scanning image, the system is in accordance with the requirement of breast scanning, and the system has insufficient mobility for scanning other parts.
Aiming at clinical ultrasonic scanning, the prior art has the following technical problems to be solved urgently:
1. the current ultrasonic examination depends on experience and operation skill of doctors, and autonomous scanning by a robot cannot be realized, so that stable scanning with constant contact force cannot be realized, image quality is uneven, and further quantitative analysis cannot be carried out.
2. The influence of human body respiratory motion on the quality of scanned images in the ultrasonic scanning process is great, and the problem cannot be effectively solved by the conventional autonomous scanning.
3. When the practical problems related to scanning control and puncture positioning operation of the surgical robot are solved based on the reinforcement learning algorithm, no reliable simulation environment is provided for training, training efficiency of random sampling samples in a real environment is low, and safety problems also exist.
Disclosure of Invention
The embodiment of the invention provides a control method and a control system for an ultrasonic-assisted scanning surgical robot, which at least solve the technical problem that the conventional surgical robot system which can perform vertical scanning and autonomous ultrasonic scanning at constant contact force and constant speed cannot be realized.
According to an embodiment of the invention, a control method of an ultrasonic-assisted scanning surgical robot is provided, which comprises the following steps:
s100, building a simulation environment as a robot dynamics simulation platform, wherein the simulation environment configuration comprises an environment scene file and a mechanical arm model, and an ultrasonic probe clamping and puncturing positioning mechanism model file;
s200, a flexible soft model is built in a simulation environment, and human body respiratory motion simulation is added on the basis of the soft model to truly reflect the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen;
s300, carrying out ultrasonic scanning in the built simulation environment to obtain the state parameters of the robot in real time;
and S400, performing reinforcement learning training, applying the trained reinforcement learning model to the robot, and reinforcing the accurate control of the robot.
Further, step S100 specifically includes:
a deep reinforcement learning platform based on MuJoCo is adopted to build a simulation environment, and the MuJoCo model file needs an MJCF or URDF file in an XML form; the environment scene file is based on a lift-environment scene model of MuJoCo, environmental scenes of a table, a floor and a wall are defined in the environment scene file, a UR5e mechanical arm URDF model is added, the model file of the ultrasonic probe clamping and puncturing mechanism is converted into an XML description file, and a simulated force and torque sensor is added to the tail end of the probe in the model description file; in the models, triangular patch grids are introduced into an STL file of the model through Mesh elements, a compiler automatically deduces inertia characteristics of the grids, and sets elements of skins and Texture Assets are added into an XML model.
Further, building a flexible software model in the simulation environment includes:
the flexible model is composed of a plurality of rigid bodies abstracted into a mass-spring-impedance model, the appearance of rigid body particles is a sliding joint, each scraping joint is connected to a center centroid, and the sum of the displacements of all joints is constant so as to keep the volume of an object; the software model comprises: and (4) building a physical model, and controlling interaction force.
Further, the physical model building comprises:
the modeling process of the soft object comprises selecting the quantity, the shape, the size and the quality of rigid bodies and the rigidity and the damping parameters of the compound object constraint; designing parameters of a software model by using a composite element in Mujoco; the initial positions of the element bodies form a regular grid in 1D, 2D or 3D, each rigid body having a joint connected to the center; the constrained spatial dynamics in the model are the following equations:
wherein a is 1 Which is representative of the acceleration of the vehicle,
representing the velocity, r the position difference,
and
respectively representing the stiffness and the resistance of an equivalent mass-spring-resistance model, wherein the relationship with a reference acceleration is
And d represents an impedance constraint; the length d of the steel can be adjusted,
the value of (A) realizes control of model rigidity and impedance;
texture grid coverage of the surface is added in the software model by using Texture, and the space position state of the sliding joint is displayed; skin elements are added and textured and subdivided using bicubic interpolation.
Further, the interaction force and control includes:
the flexible body is deformed by changing the space position and the position of the mass center body of the soft model and changing the displacement of the corner points of the triangular surface patch of the flexible surface, and the corresponding corner point deformation force is calculated based on the corner point displacement and the structural parameters, so that the flexible body is controlled.
Further, on the basis of the software model, adding human breathing motion simulation comprises:
the simulation of human body breathing motion is realized by applying an outward force to the surface of a rigid body forming the upper layer of the soft body; applying such a force over a given time interval causes the soft body to repeatedly fluctuate and contract, substantially simulating the breathing motion, setting both the breathing rate and the breathing amplitude within a predetermined range, randomizing the breathing rate and breathing amplitude at the start of each step.
Further, step S300 specifically includes:
the state parameters of the robot can be acquired in real time in a simulation environment, and the state parameters comprise joint positions and postures of joints of a mechanical arm, linear velocity and linear acceleration of the joints, force and moment at the tail end of the mechanical arm, force and moment at the tail end of an ultrasonic probe, contact force in a tangential direction and a vertical direction, and contact force derivative state parameters;
taking each scanning process as a round, judging whether the ultrasonic probe is in contact with the soft body model or not according to the parameters, starting random scanning in a direction vertical to the contact surface of the soft body model at constant contact force and constant speed after normal contact, and ending the scanning round until the ultrasonic probe deviates from the track, or loses contact with the soft body model, or reaches the limit of joints.
Further, step S400 specifically includes:
selecting a PPO algorithm as a reinforcement learning algorithm; the PPO algorithm has two networks which are respectively neural network architectures of an operator and a critic, and both have independent network structures and mainly consist of two full connection layers; the first layer consists of 256 neurons and the second layer consists of 128 neurons; the activation function used after each layer is tanh; the two networks also have a third fully connected layer for mapping the output characteristics of the previous layer to the appropriate dimensions of the operator and critic, respectively.
Further, the reinforcement learning algorithm is configured to:
RewardFunction reward function of PPO algorithm:
r total ＝w p r p +w o r o +w f r f +w d r d +w v r v ；
wherein r is p Is a location reward, r o Is a directional award, r f Is a force reward, r d Is a differential force reward, r v Is a speed reward; and weight w p 、w o 、w f 、w d 、w v The weights of the reward items are respectively;
constant contact force, constant velocity, vertical sweep settings were:
several target values are selected, the target position of the trajectory is extracted from the trajectory generator at each time step, and the target direction quaternion is set to:
q goal ＝(-0.692,0.722,-0.005,-0.11)；
and the contact force, the contact force derivative and the target speed are set as follows:
the controller is configured to:
the controller is an OSC controller that maps the position and attitude of the gripper mechanism tip into the robot's underlying controller as a function of:
wherein Λ p And Λ R Is a 6 x 6 matrix corresponding to the tip position and pose, and J p And J R Respectively corresponding to the end effectorJacobian matrix of
And
proportional and differential gain vectors, which are position and direction, respectively, remain fixed at initialization.
According to another embodiment of the present invention, there is provided an ultrasound-assisted scanning surgical robot control system, including:
the simulation platform building module is used for building a simulation environment as a robot dynamics simulation platform, and the simulation environment configuration comprises an environment scene file and a mechanical arm model, and the environment building configuration of an ultrasonic probe clamping and puncturing positioning mechanism model file;
the soft body model building module is used for building a flexible soft body model in a simulation environment, and adding the mechanical characteristics of human body respiratory motion simulation and real reaction of the contact of the ultrasonic probe and the human body abdomen on the basis of the soft body model;
the ultrasonic scanning module is used for carrying out ultrasonic scanning in the built simulation environment and acquiring the state parameters of the robot in real time;
and the reinforcement learning training module is used for reinforcement learning training, applying the trained reinforcement learning model to the robot and reinforcing the accurate control of the robot.
A processor is used for running a program, wherein the program executes the control method of the ultrasonic auxiliary scanning surgical robot in any one of the above aspects when running.
A processor is used for running a program, wherein the program executes the control method of the ultrasonic auxiliary scanning surgical robot in any one of the above aspects when running.
According to the control method and system for the ultrasonic-assisted scanning surgical robot, a simulation environment is built to serve as a robot dynamics simulation platform, the simulation environment configuration comprises an environment scene file and a mechanical arm model, and the environment building configuration of an ultrasonic probe clamping mechanism model file and a puncture positioning mechanism model file; a flexible soft model is built in a simulation environment, and human body breathing motion simulation is added on the basis of the soft model to truly reflect the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen; carrying out ultrasonic scanning in the built simulation environment, and acquiring the state parameters of the robot in real time; and (4) reinforcement learning training, wherein the trained reinforcement learning model is applied to the robot, so that the accurate control of the robot is reinforced. The invention reduces the influence of respiratory motion on the autonomous ultrasonic scanning process, provides a set of training environment containing respiratory motion simulation, and can realize the autonomous ultrasonic scanning perpendicular to the body surface at constant contact force and constant speed.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention and do not constitute a limitation of the invention. In the drawings:
FIG. 1 is a flow chart of a method of controlling an ultrasound assisted scanning surgical robot in accordance with the present invention;
FIG. 2 is a flowchart of a detailed scanning in a simulation environment in the control method of an ultrasound-assisted scanning surgical robot of the present invention;
FIG. 3 is a flow chart of the simulation of human respiratory motion in the control method of the ultrasound-assisted scanning surgical robot according to the present invention;
FIG. 4 is a flowchart of an ultrasound scanning process in the control method of the ultrasound-assisted scanning surgical robot according to the present invention;
FIG. 5 is a diagram illustrating the architecture of the entire reinforcement learning training network in the control method of the ultrasound-assisted scanning surgical robot according to the present invention;
FIG. 6 is a simulation diagram of the scanning process of the autonomous ultrasound scanning system in the control method of the ultrasound-assisted scanning surgical robot of the present invention;
FIG. 7 is a rendering diagram of the physical structure of the scanning process of the autonomous ultrasonic scanning system in the control method of the ultrasonic-assisted scanning surgical robot according to the present invention;
FIG. 8 is a block diagram of the control system of the ultrasound-assisted scanning surgical robot of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
According to an embodiment of the present invention, there is provided a control method for an ultrasound-assisted scanning surgical robot, referring to fig. 1, including the following steps:
s100, establishing a simulation environment as a robot dynamics simulation platform, wherein the simulation environment configuration comprises three parts of environment scene files, mechanical arm models and ultrasonic probe clamping and puncture positioning mechanism model files;
s200, a flexible soft model is built in a simulation environment, and human body respiratory motion simulation is added on the basis of the soft model to truly reflect the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen;
s300, carrying out ultrasonic scanning in the built simulation environment to obtain the state parameters of the robot in real time;
and S400, performing reinforcement learning training, applying the trained reinforcement learning model to the robot, and reinforcing the accurate control of the robot.
According to the control method of the ultrasonic-assisted scanning surgical robot, a simulation environment is built to serve as a robot dynamics simulation platform, the simulation environment configuration comprises an environment scene file and a mechanical arm model, and the environment building configuration of an ultrasonic probe clamping mechanism model file and a puncture positioning mechanism model file; a flexible soft model is built in a simulation environment, and human body breathing motion simulation is added on the basis of the soft model to truly reflect the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen; carrying out ultrasonic scanning in the built simulation environment, and acquiring the state parameters of the robot in real time; and (4) reinforcement learning training, wherein the trained reinforcement learning model is applied to the robot, so that the accurate control of the robot is reinforced. The invention reduces the influence of respiratory motion on the autonomous ultrasonic scanning process, provides a set of training environment containing respiratory motion simulation, and can realize the ultrasonic autonomous scanning perpendicular to the body surface at constant contact force and constant speed.
Wherein, step S100 specifically includes:
a deep reinforcement learning platform based on Mujoco is adopted to build a simulation environment, and a MuJoCo model file needs an MJCF or URDF file in an XML form; the environment scene file is based on a lift-environment scene model of MuJoCo, environmental scenes of a table, a floor and a wall are defined in the environment scene file, a UR5e mechanical arm URDF model is added, the model file of the ultrasonic probe clamping and puncturing mechanism is converted into an XML description file, and a simulated force and torque sensor is added to the tail end of the probe in the model description file; in the models, triangular patch grids are introduced into an STL file of the model through Mesh elements, a compiler automatically deduces inertia characteristics of the grids, and sets elements of skins and Texture Assets are added into an XML model.
Wherein, building a flexible software model in the simulation environment comprises:
the flexible model is composed of a plurality of rigid bodies abstracted into a mass-spring-impedance model, the appearance of rigid body particles is a sliding joint, each scraping joint is connected to a center centroid, and the sum of the displacements of all joints is constant so as to keep the volume of an object; the software model comprises: and (4) building a physical model, and controlling interaction force.
Wherein, physical model construction includes:
the modeling process of the soft object comprises selecting the quantity, shape, size and quality of rigid bodies and the rigidity and damping parameters of the compound object constraint; designing parameters of a software model by using composition elements in Mujoco; the initial positions of the element bodies form a regular grid in 1D, 2D or 3D, each rigid body having a joint connected to the center; the constrained spatial dynamics in the model are the following equations:
wherein a is 1 Which is representative of the acceleration of the vehicle,
indicating the speed, r the position difference,
and
respectively representing the stiffness and the resistance of an equivalent mass-spring-resistance model, wherein the relation with a reference acceleration is
And d represents an impedance constraint; the length of d can be adjusted,
the value of (b) realizes control of model stiffness and impedance;
texture grid coverage of the surface is added in the software model by Texture, and the space position state of the sliding joint is displayed; skin elements are added and textured and subdivided using bicubic interpolation.
Wherein, interactive force and control includes:
the flexible body is deformed by changing the spatial pose of the mass center body of the soft model and changing the displacement of the corner points of the triangular surface patch of the flexible surface, and the corresponding corner point deformation force is calculated based on the corner point displacement and the structural parameters, so that the flexible body is controlled.
Wherein, on the basis of the software model, adding human breathing motion simulation comprises:
the simulation of human body breathing motion is realized by applying an outward force to the surface of a rigid body forming the upper layer of the soft body; the application of such force over a given time interval causes the soft body to repeatedly fluctuate and contract, substantially simulating the breathing motion, setting both the breathing rate and the breathing amplitude within a predetermined range, randomizing the breathing rate and breathing amplitude at the beginning of each step.
Wherein, step S300 specifically includes:
the state parameters of the robot can be acquired in real time in a simulation environment, and the state parameters comprise joint positions and postures of joints of a mechanical arm, linear velocity and linear acceleration of the joints, force and moment at the tail end of the mechanical arm, force and moment at the tail end of an ultrasonic probe, contact force in a tangential direction and a vertical direction, and contact force derivative state parameters;
taking each scanning process as a round, judging whether the ultrasonic probe is in contact with the soft body model or not according to the parameters, starting random scanning in a direction vertical to the contact surface of the soft body model at constant contact force and constant speed after normal contact, and ending the scanning round until the ultrasonic probe deviates from the track, or loses contact with the soft body model, or reaches the limit of joints.
Wherein, step S400 specifically includes:
selecting a PPO algorithm as a reinforcement learning algorithm; the PPO algorithm has two networks which are respectively neural network architectures of an actor and a critic, and both have independent network structures and mainly consist of two full connection layers; the first layer consists of 256 neurons, while the second layer consists of 128 neurons; the activation function used after each layer is tanh; the two networks also have a third fully connected layer for mapping the output characteristics of the previous layer to the appropriate dimensions of the operator and critic, respectively.
Wherein the reinforcement learning algorithm is configured to:
RewardFunction reward function of PPO algorithm:
r total ＝w p r p +w o r o +w f r f +w d r d +w v r v ；
wherein r is p Is a location reward, r o Is a directional award, r f Is a force reward, r d Is a differential force reward, r v Is a speed reward; and weight w p 、w o 、w f 、w d 、w v The weights of the reward items are respectively;
constant contact force, constant velocity, vertical scan settings were:
several target values are selected, the target position of the trajectory is extracted from the trajectory generator at each time step, and the target direction quaternion is set to:
q goal ＝(-0.692,0.722,-0.005,-0.11)；
and the contact force, the contact force derivative and the target speed are set as follows:
the controller is configured to:
the controller is an OSC controller that maps the position and attitude of the gripper mechanism tip into the robot's underlying controller as a function of:
wherein Λ p And Λ R Is a 6 x 6 matrix corresponding to the tip position and pose, and J p And J R Respectively, jacobian matrices corresponding to the end effectors
And
proportional and differential gain vectors, which are position and orientation, respectively, remain fixed at initialization.
The control method of the ultrasonic-assisted scanning surgical robot of the invention is explained in detail by the following specific embodiments:
the invention aims to design a surgical robot system scheme which can realize the independent ultrasonic scanning with constant contact force, constant speed and vertical scanning. The method specifically comprises the following steps:
1. the surgical robot system for autonomous ultrasonic scanning and surgical puncture positioning is designed, and comprises an ultrasonic scanning control method, a simulation training environment, a robot and a puncture positioning device.
2. A soft body membrane simulation environment capable of simulating human body respiratory motion is designed, and an effective solution for reducing the influence of the respiratory motion on the quality of an autonomous ultrasonic scanning image is provided.
3. An ultrasonic scanning device with constant contact force, constant speed and perpendicular to the surface of the body membrane is designed.
The technical scheme of the invention comprises three modules of modeling, ultrasonic scanning simulation and reinforcement learning training. The method is characterized in that the scanning target of the robot with constant contact force, constant speed and vertical to the scanned object is realized in a simulation environment, and a flow chart of specific scanning in the simulation environment is shown in figure 2. The technical scheme of the invention specifically comprises the following steps:
1. modelling
1. Simulation framework construction
The simulation environment configuration comprises environment construction configuration of an environment scene file, a mechanical arm model and an ultrasonic probe clamping and puncturing positioning mechanism (hereinafter referred to as an ultrasonic probe) model file. At present, muJoCo, gazebo, pyBullet and the like are common platforms used for robot dynamics simulation, but the dynamics simulation of a human body software model is considered to be required to be realized, so the invention adopts a simulation environment built by a deep reinforcement learning platform based on MuJoCo, and the MuJoCo model file needs an MJCF or URDF file in an XML form, thereby being compiled into a model file through a physical engine. The environment scene file is based on a lift-environment scene model of MuJoCo, environment scenes such as table size and specification, floors, walls and the like are defined in the model, a UR5e mechanical arm URDF model is added, the model file of the ultrasonic probe clamping and puncturing mechanism is converted into an XML description file, and a simulated force and moment sensor is added to the tail end of the probe in the model description file to obtain the force and moment applied to the tail end of the probe in the simulation process. In the above models, a triangular patch Mesh is introduced into an STL file of the model through a Mesh element, a compiler automatically infers an inertia characteristic of the Mesh, and Assets elements such as Skin and Texture are added to the XML model, thereby giving visualization effects such as floor color and wall picture to a scene environment.
2. Design of soft body model
In order to truly reflect the mechanical characteristics of the contact of the ultrasonic probe and the human abdomen, a flexible soft model needs to be built in a simulation environment. The flexible model is composed of a plurality of rigid bodies abstracted as a mass-spring-impedance model, the appearance of rigid body particles is a sliding joint, each scraping joint is connected to a center centroid, the sliding joint can deform when being subjected to external force, the displacement of a plurality of rigid bodies reflects the physical property of a software, the purpose of simulating the mechanical property when the software is contacted is achieved, and the sum of the displacement of all the joints is constant so as to keep the volume of an object.
Building a physical model: the modeling process of the soft object includes selecting the number, shape, size and mass of rigid bodies, and the stiffness and damping parameters of the composite object constraints. The parameters of the software model are designed using composition elements within MuJoCo, and the composition object is essentially a particle system, but can be constrained to move together in a way that simulates various flexible objects. The original positions of the elementary bodies form a regular grid in 1D, 2D or 3D, all of which may be the sub-entities of the parent entity. Each rigid body has a joint connected to the center allowing displacement of each rigid body. The parameters determining the degree of softness of the software model are solref and solimp. The constrained spatial dynamics in the model are approximated by the following equation:
wherein a is 1 Which is representative of the acceleration of the vehicle,
representing the velocity, r the position difference,
and
respectively representing the stiffness and the resistance of an equivalent mass-spring-resistance model, wherein the relationship with a reference acceleration is
And d represents the impedance constraint. The length d of the steel can be adjusted,
can control the model stiffness, impedance, and in MuJoCo this can be achieved by adjusting sollamp and solref. In addition, texture mesh covering with Texture added surface is used in the inner side of the software model, and the space position state of the sliding joint is displayed. Skin elements are added that are textured and subdivided using bicubic interpolation, enabling the Skin picture to be attached around the model, which effectively hides the rigid body structure.
Interaction force and control: according to the physical characteristics, the position and the posture of the flexible body can be changed by changing the space position and the posture of the mass center body of the soft body model, the deformation of the flexible body is realized by changing the displacement of the corner points of the triangular patch of the flexible surface, and the corresponding corner point deformation force is calculated based on the corner point displacement and the structural parameters, so that the control of the flexible body is completed.
3. Simulation of human respiratory motion
Based on the soft body model described in point 2 above, in order to further increase the sense of realism of a soft body, it is a necessary step to add a simulation of the breathing motion of the human body, which can be achieved by applying an outward force to the surface of the rigid body constituting the upper layer of the soft body. The application of such force over a given time interval causes the soft body to repeatedly undulate and contract, substantially simulating the breathing motion. However, considering that the breathing model needs to be applied to a reinforcement learning training process, the breathing frequency, the breathing amplitude and the like cannot be changed layer by layer, the breathing frequency and the breathing amplitude are set in a range, and the breathing frequency and the breathing amplitude are randomized at the beginning of each step so as to improve the generalization capability of the training model. The implementation flow of this part in the technical solution of the present invention is shown in fig. 3.
2. Ultrasound scanning simulation
In the simulation environment set up above, the ultrasound scanning flow is shown in fig. 4. The state parameters of the robot can be acquired in real time in the simulation environment, and the state parameters comprise joint positions and postures of joints of the mechanical arm, linear velocity and linear acceleration of the joints, force and moment at the tail end of the mechanical arm, force and moment at the tail end of the ultrasonic probe, contact force in the tangential direction and the vertical direction, contact force derivative and the like. Taking each scanning process as a round, judging whether the ultrasonic probe is in contact with the soft body model or not through the parameters, starting to start random scanning in a direction vertical to the contact surface of the soft body model at a constant contact force and a constant speed after normal contact, and ending the scanning round until the ultrasonic probe deviates from the track, or loses contact with the soft body model, or reaches the joint limit.
3. Reinforcement learning training
Based on the simulation environment and the scanning process, the technical scheme of the invention selects the PPO algorithm as the reinforcement learning algorithm. The PPO algorithm has two networks, namely neural network architectures of an operator and a critic, and the two networks have independent network structures and mainly consist of two full connection layers. The first layer consists of 256 neurons and the second layer consists of 128 neurons. The activation function used after each layer is tanh. The two networks also have a third fully connected layer for mapping the output characteristics of the previous layer to the appropriate dimensions of the operator and critic, respectively. As shown in fig. 5, the architecture of the entire reinforcement learning training network is shown.
1. Realization of constant contact force, constant speed and vertical scanning
(1) And (3) reinforcement learning algorithm configuration:
RewardFunction of the PPO algorithm:
r total ＝w p r p +w o r o +w f r f +w d r d +w v r v ；
wherein r is p Is a location reward, r o Is a directional award, r f Is a force reward, r d Is a differential force reward, r v Is a speed award. And weight w p 、w o 、w f 、w d 、w v The weights of the individual bonus items are adjusted and set according to their importance for completing the task. The most central part of the task is to learn that the manipulator follows the trajectory while applying the required contact force. Therefore, the track term and the force term are given the maximum weight, and the weight set by the technical scheme of the invention is w respectively p ＝5、w o ＝1、w f ＝3、w d ＝2、w v ＝1。
(2) Constant contact force, constant speed, vertical scanning arrangement
The scanning target is easy to realize on the basis of the environment configuration, and when the RewardFunction configuration of the reinforcement learning algorithm is performed, the RewardFunction weight of the contact force is set to be the maximum, so that the contact force can be guaranteed to be constant preferentially. As described in the reward function, several target values need to be selected, the target position of the trajectory is extracted from the trajectory generator at each time step, and the target direction quaternion is set to:
q goal ＝(-0.692,0.722,-0.005,-0.11)；
and the contact force, the contact force derivative and the target speed are set as follows:
(3) Controller configuration
The Action output from the reinforcement learning algorithm is to achieve the control objective through the robot controller, and the controller selected by the design is an OSC (operational space control) controller, which can map the position and posture of the end of the clamping mechanism into the underlying controller of the robot, and the mapping function is:
wherein Λ p And Λ R Is a 6 x 6 matrix corresponding to the tip position and pose, and J p And J R Respectively, a Jacobian matrix corresponding to the end effector
And
proportional and differential gain vectors, which are position and orientation, respectively, remain fixed at initialization. As with the PPO algorithm, the strategy output action of the OSC and the controller output torque are both sent at a frequency of 500 Hz.
The key points of innovation and points to be protected of the invention are at least as follows:
1. the technical scheme of the invention designs a set of surgical robot system for autonomous ultrasonic scanning and surgical puncture positioning, which comprises an ultrasonic scanning control method, a simulation training environment, a robot and a puncture positioning device.
2. The technical scheme of the invention designs a soft body membrane simulation environment capable of simulating human body breathing movement, configures a set of soft body model contact control training environment applied to reinforcement learning, and has great significance for man-machine soft contact.
3. The technical scheme of the invention designs an ultrasonic scanning system, a control mode and a device which are in constant contact force, constant speed and vertical to the surface of a human body.
The technical scheme of the invention has the advantages that:
1. the influence of respiratory motion on the autonomous ultrasonic scanning process is reduced, and a set of training environment containing respiratory motion simulation is provided.
2. The ultrasonic autonomous scanning device can realize the ultrasonic autonomous scanning perpendicular to the body surface at constant contact force and constant speed.
Referring to fig. 6-7, the technical scheme of the invention can realize autonomous ultrasonic scanning through simulation experiments of a MuJoCo simulation platform, and the scanning model has good generalization capability, stable scanning process and constant contact force. The technical scheme of the invention can also be applied to a reinforcement learning environment configuration mode containing respiratory motion simulation.
Example 2
According to another embodiment of the present invention, there is provided an ultrasound-assisted scanning surgical robot control system, see fig. 8, including:
the simulation platform building module 201 is used for building a simulation environment as a robot dynamics simulation platform, and the simulation environment configuration comprises environment scene files, a mechanical arm model and environment building configurations of an ultrasonic probe clamping and puncture positioning mechanism model file;
the software model building module 202 is used for building a flexible software model in a simulation environment, and adding the mechanical characteristics of human body respiratory motion simulation and real reaction of the contact of the ultrasonic probe and the human body abdomen on the basis of the software model;
the ultrasonic scanning module 203 is used for performing ultrasonic scanning in the built simulation environment and acquiring the state parameters of the robot in real time;
and the reinforcement learning training module 204 is used for reinforcement learning training, applying the trained reinforcement learning model to the robot and reinforcing the accurate control of the robot.
According to the control system of the ultrasonic-assisted scanning surgical robot, a simulation environment is built to serve as a robot dynamics simulation platform, the simulation environment configuration comprises an environment scene file and a mechanical arm model, and the environment building configuration of three parts, namely an ultrasonic probe clamping and puncture positioning mechanism model file; a flexible soft model is set up in a simulation environment, and human body respiratory motion simulation is added on the basis of the soft model, so that the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen are truly reflected; carrying out ultrasonic scanning in the built simulation environment, and acquiring the state parameters of the robot in real time; and (4) reinforcement learning training, wherein the trained reinforcement learning model is applied to the robot, so that the accurate control of the robot is reinforced. The invention reduces the influence of respiratory motion on the autonomous ultrasonic scanning process, provides a set of training environment containing respiratory motion simulation, and can realize the ultrasonic autonomous scanning perpendicular to the body surface at constant contact force and constant speed.
The control system of the ultrasound-assisted scanning surgical robot of the present invention is described in detail below with specific embodiments:
the invention aims to design a surgical robot system scheme which can realize the independent ultrasonic scanning with constant contact force, constant speed and vertical scanning. The method specifically comprises the following steps:
1. the surgical robot system for autonomous ultrasonic scanning and surgical puncture positioning is designed, and comprises an ultrasonic scanning control method, a simulation training environment, a robot and a puncture positioning device.
2. A soft body membrane simulation environment capable of simulating human body respiratory motion is designed, and an effective solution for reducing the influence of the respiratory motion on the quality of an autonomous ultrasonic scanning image is provided.
3. An ultrasonic scanning device with constant contact force, constant speed and perpendicular to the surface of the body membrane is designed.
The technical scheme of the invention comprises three modules of modeling, ultrasonic scanning simulation and reinforcement learning training. The method is characterized in that the scanning target of the robot with constant contact force, constant speed and vertical to the scanned object is realized in a simulation environment, and a flow chart of specific scanning in the simulation environment is shown in figure 2. The technical scheme of the invention specifically comprises the following steps:
1. modelling
1. Simulation framework construction
The simulation environment configuration comprises environment construction configuration of an environment scene file, a mechanical arm model and an ultrasonic probe clamping and puncturing positioning mechanism (hereinafter referred to as an ultrasonic probe) model file. At present, muJoCo, gazebo, pyBullet and the like are common platforms used for robot dynamics simulation, but the dynamics simulation of a human body software model is considered to be realized, so the invention adopts a simulation environment built by a deep reinforcement learning platform based on Mujoco, and a MuJoCo model file needs an MJCF or URDF file in an XML form, thereby being compiled into a model file through a physical engine. The environment scene file is based on a lift-environment scene model of MuJoCo, environment scenes such as table size and specification, floors, walls and the like are defined in the model, a UR5e mechanical arm URDF model is added, the model file of the ultrasonic probe clamping and puncturing mechanism is converted into an XML description file, and a simulated force and moment sensor is added to the tail end of the probe in the model description file to obtain the force and moment applied to the tail end of the probe in the simulation process. In the above models, a triangular patch Mesh is introduced into an STL file of the model through a Mesh element, a compiler automatically infers an inertia characteristic of the Mesh, and Assets elements such as Skin and Texture are added to the XML model, thereby giving visualization effects such as floor color and wall picture to a scene environment.
2. Design of soft body model
In order to truly reflect the mechanical characteristics of the contact between the ultrasonic probe and the human abdomen, a flexible software model needs to be built inside the simulation environment. The flexible model is composed of a plurality of rigid bodies abstracted as a mass-spring-impedance model, the appearance of rigid body particles is a sliding joint, each scraping joint is connected to a center centroid, the sliding joint can deform when being subjected to external force, the displacement of a plurality of rigid bodies reflects the physical property of a software, the purpose of simulating the mechanical property when the software is contacted is achieved, and the sum of the displacement of all the joints is constant so as to keep the volume of an object.
Building a physical model: the modeling process of the soft object includes selecting the number, shape, size and mass of rigid bodies, and the stiffness and damping parameters of the composite object constraints. The parameters of the software model are designed using composition elements within MuJoCo, and the composition object is essentially a particle system, but can be constrained to move together in a way that simulates various flexible objects. The initial positions of the elements form a regular grid in 1D, 2D or 3D, all of which may be sub-entities of the parent entity. Each rigid body has a joint connected to the center allowing displacement of each rigid body. The parameters determining the degree of softness of the software model are solref and solimp. The constrained spatial dynamics in the model are approximated by the following equation:
wherein a is 1 Which is representative of the acceleration of the vehicle,
representing the velocity, r the position difference,
and
respectively representing the stiffness and the resistance of an equivalent mass-spring-resistance model, wherein the relationship with a reference acceleration is
And d represents the impedance constraint. The length of d can be adjusted,
the values of (a) can be used to control the model stiffness, impedance, and in Mujoco this can be achieved by adjusting sollamp and solref. In addition, texture mesh covering with Texture added surface is used in the inner side of the software model, and the space position state of the sliding joint is displayed. Skin elements are added that are textured and subdivided using bicubic interpolation, enabling the Skin picture to be attached around the model, which effectively hides the rigid structure.
Interaction force and control: according to the physical characteristics, the position and the posture of the flexible body can be changed by changing the space position and the posture of the mass center body of the soft body model, the deformation of the flexible body is realized by changing the displacement of the corner points of the triangular surface patch of the flexible surface, and the corresponding corner point deformation force is calculated based on the corner point displacement and the structural parameters, so that the control of the flexible body is completed.
3. Simulation of human respiratory motion
Based on the soft body model described in point 2 above, in order to further increase the sense of realism of a soft body, it is a necessary step to add a simulation of the breathing motion of the human body, which can be achieved by applying an outward force to the surface of the rigid body constituting the upper layer of the soft body. The application of such force over a given time interval causes the software to repeatedly undulate and contract, substantially simulating the breathing movements. However, considering that the breathing model needs to be applied to the reinforcement learning training process, the breathing frequency, the breathing amplitude and the like cannot be changed in a layer, the breathing frequency and the breathing amplitude are set in a range, and the breathing frequency and the breathing amplitude are randomized at the beginning of each step, so that the generalization capability of the training model is improved. The implementation flow of this part in the technical solution of the present invention is shown in fig. 3.
2. Ultrasound scanning simulation
In the simulation environment set up above, the ultrasound scanning flow is shown in fig. 4. The state parameters of the robot can be obtained in real time in the simulation environment, and the state parameters comprise the joint position and posture of each joint of the mechanical arm, the linear velocity and linear acceleration of the joint, the force and moment at the tail end of the mechanical arm, the force and moment at the tail end of the ultrasonic probe, contact forces in the tangential direction and the vertical direction, contact force derivatives and the like. Taking each scanning process as a round, judging whether the ultrasonic probe is in contact with the soft body model or not through the parameters, starting to start random scanning in a direction vertical to the contact surface of the soft body model at a constant contact force and a constant speed after normal contact, and ending the scanning round until the ultrasonic probe deviates from the track, or loses contact with the soft body model, or reaches the joint limit.
3. Reinforcement learning training
Based on the simulation environment and the scanning process, the technical scheme of the invention selects the PPO algorithm as the reinforcement learning algorithm. The PPO algorithm has two networks, namely neural network architectures of an operator and a critic, and both networks have independent network structures and mainly consist of two full connection layers. The first layer consists of 256 neurons and the second layer consists of 128 neurons. The activation function used after each layer is tanh. The two networks also have a third fully connected layer for mapping the output characteristics of the previous layer to the appropriate dimensions of the operator and critic, respectively. As shown in fig. 5, the architecture of the entire reinforcement learning training network is shown.
1. Realization of constant contact force, constant speed and vertical scanning
(1) And (3) reinforcement learning algorithm configuration:
RewardFunction of the PPO algorithm:
r total ＝w p r p +w o r o +w f r f +w d r d +w v r v ；
wherein r is p Is a location reward, r o Is a directional award, r f Is a force reward, r d Is a differential force reward, r v Is a speed award. And weight w p 、w o 、w f 、w d 、w v The weights of the individual bonus items are adjusted and set according to their importance for completing the task. The most central part of the task is to learn the trajectory that the manipulator follows while applying the required contact force. Thus, the trajectory term and the force term are given the greatest weight, the inventionThe weight set by the technical scheme is w respectively p ＝5、w o ＝1、w f ＝3、w d ＝2、w v ＝1。
(2) Constant contact force, constant speed, vertical scanning arrangement
The scanning target is easy to realize on the basis of the environment configuration, and when the RewardFunction configuration of the reinforcement learning algorithm is performed, the RewardFunction weight of the contact force is set to be the maximum, so that the contact force can be guaranteed to be constant preferentially. As described in the reward function, several target values need to be selected, the target position of the trajectory is extracted from the trajectory generator at each time step, and the target direction quaternion is set to:
q goal ＝(-0.692,0.722,-0.005,-0.11)；
and the contact force, the contact force derivative and the target speed are set as follows:
(3) Controller configuration
The Action output from the reinforcement learning algorithm is to achieve the control objective through the robot controller, and the controller selected by the design is an OSC (operational space control) controller, which can map the position and posture of the end of the clamping mechanism into the underlying controller of the robot, and the mapping function is:
wherein Λ p And Λ R Is a 6 x 6 matrix corresponding to the tip position and pose, and J p And J R Respectively, a Jacobian matrix corresponding to the end effector
And
proportional and differential gain vectors, which are position and direction, respectively, remain fixed at initialization. As with the PPO algorithm, the strategy output action of the OSC and the controller output torque are both sent at a frequency of 500 Hz.
The key points of innovation and points to be protected of the invention are at least as follows:
1. the technical scheme of the invention designs a set of surgical robot system for autonomous ultrasonic scanning and surgical puncture positioning, which comprises an ultrasonic scanning control method, a simulation training environment, a robot and a puncture positioning device.
2. The technical scheme of the invention designs a soft body membrane simulation environment capable of simulating human body breathing movement, configures a set of soft body model contact control training environment applied to reinforcement learning, and has great significance for man-machine soft contact.
3. The technical scheme of the invention designs an ultrasonic scanning system, a control mode and a device which are in constant contact force, constant speed and vertical to the surface of a human body.
The technical scheme of the invention has the advantages that:
1. the influence of respiratory motion on the autonomous ultrasonic scanning process is reduced, and a set of training environment containing respiratory motion simulation is provided.
2. The ultrasonic autonomous scanning device can realize the ultrasonic autonomous scanning perpendicular to the body surface at constant contact force and constant speed. Referring to fig. 6-7, the technical scheme of the invention can realize autonomous ultrasonic scanning through simulation experiments of a MuJoCo simulation platform, and the scanning model has good generalization capability, stable scanning process and constant contact force. The technical scheme of the invention can also be applied to a reinforcement learning environment configuration mode containing respiratory motion simulation.
Example 3
A storage medium stores a program file capable of implementing any one of the above ultrasound-assisted scanning surgical robot control methods.
Example 4
A processor for executing a program, wherein the program when executed performs any one of the above methods of controlling an ultrasound-assisted scanning surgical robot.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described system embodiments are merely illustrative, and for example, a division of a unit may be a logical division, and an actual implementation may have another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is substantially or partly contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and amendments can be made without departing from the principle of the present invention, and these modifications and amendments should also be considered as the protection scope of the present invention.
Claims (10)
1. A control method of an ultrasonic-assisted scanning surgical robot is characterized by comprising the following steps:
s100, building a simulation environment as a robot dynamics simulation platform, wherein the simulation environment configuration comprises an environment scene file and a mechanical arm model, and an ultrasonic probe clamping and puncturing positioning mechanism model file;
s200, a flexible soft model is built in a simulation environment, and human body respiratory motion simulation is added on the basis of the soft model to truly reflect the mechanical characteristics of the contact of the ultrasonic probe and the human body abdomen;
s300, carrying out ultrasonic scanning in the built simulation environment to obtain the state parameters of the robot in real time;
and S400, performing reinforcement learning training, applying the trained reinforcement learning model to the robot, and reinforcing the accurate control of the robot.
2. The control method of the ultrasound-assisted scanning surgical robot according to claim 1, wherein the step S100 specifically includes:
a deep reinforcement learning platform based on MuJoCo is adopted to build a simulation environment, and a MuJoCo model file needs an MJCF or URDF file in an XML form; the environment scene file is based on a lift-environment scene model of MuJoCo, environmental scenes of a table, a floor and a wall are defined in the environment scene file, a UR5e mechanical arm URDF model is added, the model file of the ultrasonic probe clamping and puncturing mechanism is converted into an XML description file, and a simulated force and torque sensor is added to the tail end of the probe in the model description file; in the above models, a mesh element is firstly used to introduce a triangular patch mesh into an STL file of the model, a compiler automatically infers inertia characteristics of the mesh, and elements of skins and Texture Assets are added to the XML model.
3. The method as claimed in claim 1, wherein building a flexible software model in the simulation environment comprises:
the flexible model is composed of a plurality of rigid bodies abstracted into a mass-spring-impedance model, the appearance of rigid body particles is a sliding joint, each scraping joint is connected to a center centroid, and the sum of the displacements of all joints is constant so as to keep the volume of an object; the software model comprises: and building a physical model, and performing interaction and control.
4. The ultrasound-assisted scanning surgical robot control method of claim 3, wherein the physical model building comprises:
the modeling process of the soft object comprises selecting the quantity, the shape, the size and the quality of rigid bodies and the rigidity and the damping parameters of the compound object constraint; designing parameters of a software model by using a composition element in MuJoCo; the initial positions of the element bodies form a regular grid in 1D, 2D or 3D, each rigid body having a joint connected to the center; the constrained spatial dynamics in the model are the following equations:
wherein
Which is representative of the acceleration of the vehicle,
representing the velocity, r the position difference,
and
respectively representing the stiffness and the resistance of an equivalent mass-spring-resistance model, wherein the relationship with a reference acceleration is
And then
Representing an impedance constraint; is adjustable
The value of (b) realizes control of model stiffness and impedance;
texture grid coverage of the surface is added in the software model by Texture, and the space position state of the sliding joint is displayed; skin elements are added and textured and subdivided using bicubic interpolation.
5. The method of claim 3, wherein interacting forces and controlling comprise:
the flexible body is deformed by changing the spatial pose of the mass center body of the soft model and changing the displacement of the corner points of the triangular surface patch of the flexible surface, and the corresponding corner point deformation force is calculated based on the corner point displacement and the structural parameters, so that the flexible body is controlled.
6. The control method of the robot for ultrasound-assisted scanning surgery according to claim 1, wherein the adding of the human respiratory motion simulation on the basis of the soft body model comprises:
the simulation of human body breathing motion is realized by applying an outward force to the surface of a rigid body forming the upper layer of the soft body; applying such a force over a given time interval causes the soft body to repeatedly fluctuate and contract, substantially simulating the breathing motion, setting both the breathing rate and the breathing amplitude within a predetermined range, randomizing the breathing rate and breathing amplitude at the start of each step.
7. The control method of the ultrasound-assisted scanning surgical robot according to claim 1, wherein the step S300 specifically includes:
the state parameters of the robot can be acquired in real time in a simulation environment, and the state parameters comprise joint positions and postures of joints of a mechanical arm, linear velocity and linear acceleration of the joints, force and moment at the tail end of the mechanical arm, force and moment at the tail end of an ultrasonic probe, contact force in a tangential direction and a vertical direction, and contact force derivative state parameters;
taking each scanning process as a round, judging whether the ultrasonic probe is contacted with the soft body model according to the parameters, starting random scanning at constant contact force and constant speed in a direction vertical to the contact surface of the soft body model after the ultrasonic probe is normally contacted, and ending the scanning round until the ultrasonic probe deviates from the track, or loses contact with the soft body model, or reaches the limit of the joint.
8. The control method of the ultrasound-assisted scanning surgical robot according to claim 1, wherein the step S400 specifically includes:
selecting a PPO algorithm as a reinforcement learning algorithm; the PPO algorithm has two networks which are respectively neural network architectures of an operator and a critic, and both have independent network structures and mainly consist of two full connection layers; the first layer consists of 256 neurons and the second layer consists of 128 neurons; the activation function used after each layer is tanh; the two networks also have a third fully connected layer for mapping the output characteristics of the previous layer to the appropriate dimensions of the operator and critic, respectively.
9. The ultrasound-assisted scanning surgical robot control method of claim 8, wherein the reinforcement learning algorithm is configured to:
RewardFunction reward function of PPO algorithm:
r total ＝W p r p +W o r o +w f r f +w d r d +w v r v ；
wherein r is p Is a location reward, r o Is a directional award, r f Is a force reward, r d Is a differential force reward, r v Is a speed reward; and weight w p 、w o 、w f 、w d 、w v Respectively, the weight of each bonus item;
constant contact force, constant velocity, vertical sweep settings were:
several target values are selected, the target position of the trajectory is extracted from the trajectory generator at each time step, and the target direction quaternion is set to:
q goal ＝(-0.692，0.722，-0.005，-0.11)；
and the contact force, the contact force derivative and the target speed are set as follows:
the controller is configured to:
the controller is an OSC controller that maps the position and attitude of the gripper mechanism tip into the robot's underlying controller as a function of:
wherein Λ p And Λ R Is a 6 x 6 matrix corresponding to the tip position and pose, and J p And J R Respectively, a Jacobian matrix corresponding to the end effector
And
proportional and differential gain vectors, which are position and orientation, respectively, remain fixed at initialization.
10. An ultrasound-assisted scanning surgical robot control system, comprising:
the simulation platform building module is used for building a simulation environment as a robot dynamics simulation platform, and the simulation environment configuration comprises an environment scene file and a mechanical arm model, and the environment building configuration of an ultrasonic probe clamping and puncturing positioning mechanism model file;
the soft model building module is used for building a flexible soft model in a simulation environment, and adding the mechanical characteristics of human body respiratory motion simulation, real reaction of the contact of the ultrasonic probe and the human body abdomen on the basis of the soft model;
the ultrasonic scanning module is used for carrying out ultrasonic scanning in the built simulation environment and acquiring the state parameters of the robot in real time;
and the reinforcement learning training module is used for reinforcement learning training, applying the trained reinforcement learning model to the robot and reinforcing the accurate control of the robot.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210859548.6A CN115227404A (en) | 2022-07-21 | 2022-07-21 | Control method and system for ultrasonic-assisted scanning surgical robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210859548.6A CN115227404A (en) | 2022-07-21 | 2022-07-21 | Control method and system for ultrasonic-assisted scanning surgical robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115227404A true CN115227404A (en) | 2022-10-25 |
Family
ID=83674356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210859548.6A Pending CN115227404A (en) | 2022-07-21 | 2022-07-21 | Control method and system for ultrasonic-assisted scanning surgical robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115227404A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116224829A (en) * | 2023-02-03 | 2023-06-06 | 广东工业大学 | Digital twinning-based surgical robot puncture sampling operation semi-physical simulation method |
| CN117058267A (en) * | 2023-10-12 | 2023-11-14 | 北京智源人工智能研究院 | Autonomous ultrasound scanning system, method, memory and device based on reinforcement learning |
-
2022
- 2022-07-21 CN CN202210859548.6A patent/CN115227404A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116224829A (en) * | 2023-02-03 | 2023-06-06 | 广东工业大学 | Digital twinning-based surgical robot puncture sampling operation semi-physical simulation method |
| CN116224829B (en) * | 2023-02-03 | 2023-10-20 | 广东工业大学 | Digital twinning-based surgical robot puncture sampling operation semi-physical simulation method |
| CN117058267A (en) * | 2023-10-12 | 2023-11-14 | 北京智源人工智能研究院 | Autonomous ultrasound scanning system, method, memory and device based on reinforcement learning |
| CN117058267B (en) * | 2023-10-12 | 2024-02-06 | 北京智源人工智能研究院 | Autonomous ultrasound scanning system, method, memory and device based on reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7493528B2 (en) | Feedback continuous position control of end effector | |
| CN115227404A (en) | Control method and system for ultrasonic-assisted scanning surgical robot | |
| Chatelain et al. | Confidence-driven control of an ultrasound probe | |
| Ning et al. | Autonomic robotic ultrasound imaging system based on reinforcement learning | |
| US20150320514A1 (en) | Surgical robots and control methods thereof | |
| US20080021317A1 (en) | Ultrasound medical imaging with robotic assistance for volume imaging | |
| Jiang et al. | Robotic ultrasound imaging: State-of-the-art and future perspectives | |
| KR20160118295A (en) | Systems and methods for non-rigid deformation of tissue for virtual navigation of interventional tools | |
| WO2002070980A1 (en) | Simulation system for image-guided medical procedures | |
| CN112151169B (en) | Autonomous scanning method and system of humanoid-operation ultrasonic robot | |
| CN112132805B (en) | Ultrasonic robot state normalization method and system based on human body characteristics | |
| Lee et al. | Intensity-based visual servoing for non-rigid motion compensation of soft tissue structures due to physiological motion using 4D ultrasound | |
| CN112270993A (en) | Ultrasonic robot online decision-making method and system with diagnosis result as feedback | |
| CN115741732B (en) | Interactive path planning and motion control method for massage robot | |
| CN112998749A (en) | Automatic ultrasonic inspection system based on visual servoing | |
| Li et al. | Rl-tee: Autonomous probe guidance for transesophageal echocardiography based on attention-augmented deep reinforcement learning | |
| Al-Zogbi et al. | Autonomous robotic point-of-care ultrasound imaging for monitoring of COVID-19–induced pulmonary diseases | |
| Yang et al. | Motion tracking for beating heart based on sparse statistic pose modeling | |
| CN115546287A (en) | Method, system, terminal device and medium for processing transesophageal echocardiogram | |
| US20210145412A1 (en) | Assisted steering of intracardiac echocardiogram catheters | |
| Nadeau et al. | Automatic tracking of an organ section with an ultrasound probe: Compensation of respiratory motion | |
| Zhang et al. | A step towards conditional autonomy-robotic appendectomy | |
| Tokuyasu et al. | Mechanical modeling of a beating heart for a cardiac palpation training system | |
| KR102426925B1 (en) | Method and program for acquiring motion information of a surgical robot using 3d simulation | |
| CN117058267B (en) | Autonomous ultrasound scanning system, method, memory and device based on reinforcement learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |