CN117752424A - Master-slave control method and device of flexible surgical robot, robot and medium - Google Patents

Master-slave control method and device of flexible surgical robot, robot and medium Download PDF

Info

Publication number
CN117752424A
CN117752424A CN202311618916.9A CN202311618916A CN117752424A CN 117752424 A CN117752424 A CN 117752424A CN 202311618916 A CN202311618916 A CN 202311618916A CN 117752424 A CN117752424 A CN 117752424A
Authority
CN
China
Prior art keywords
master
slave control
slave
control mode
operation end
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311618916.9A
Other languages
Chinese (zh)
Inventor
请求不公布姓名
汪振
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Shiwei Medical Technology Co ltd
Original Assignee
Shanghai Shiwei Medical Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Shiwei Medical Technology Co ltd filed Critical Shanghai Shiwei Medical Technology Co ltd
Priority to CN202311618916.9A priority Critical patent/CN117752424A/en
Publication of CN117752424A publication Critical patent/CN117752424A/en
Pending legal-status Critical Current

Links

Abstract

The application relates to a master-slave control method and device of a flexible surgical robot, the robot and a medium. Wherein the flexible surgical robot includes a slave manipulator and a master manipulator controlling movement of the slave manipulator, the method comprising: acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with at least a first portion of the joints; controlling at least a first partial joint to switch from a first master-slave control mode to execute a second master-slave control mode under the condition that the first working environment data meets a first mode switching condition; the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes. Thereby improving the smoothness and continuity of the whole operation process and improving the master-slave control experience of the flexible operation robot.

Description

Master-slave control method and device of flexible surgical robot, robot and medium
Technical Field
The application relates to the technical field of medical instruments, in particular to a master-slave control method and device of a flexible surgical robot, the robot and a medium.
Background
The master control end in a flexible surgical robot is typically a rigid structure, while the slave control end or instrument is typically comprised of at least a plurality of flexible continuum(s) that can bend (or bend unidirectionally, or bend multidirectionally, or bend arbitrarily) or rotate (rotate along an axis) compared to conventional rigid surgical robots, making master-slave control of the flexible surgical robot more difficult.
In the related art, a certain master-slave control mode is generally adopted to uniformly execute all master-slave control related in the operation process of the surgical robot, however, the control mode cannot ensure the smoothness and continuity of the whole operation process, and certain operation safety hidden trouble is brought.
Disclosure of Invention
In order to solve the technical problems, the application provides a master-slave control method and device of a flexible surgical robot, the robot and a medium. The problem of the terminal of flexible apparatus position of opening and shutting accurate control has been solved to this application.
In one aspect, the present application provides a master-slave control method of a flexible surgical robot, the flexible surgical robot including a slave operation end and a master operation end for controlling movement of the slave operation end, the method including:
acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with the at least a first portion of the joints;
Controlling the at least a first partial joint to switch from the first master-slave control mode to execute a second master-slave control mode under the condition that the first working environment data meets a first mode switching condition;
the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes.
In another aspect, the present application provides a master-slave control device for a flexible surgical robot, the flexible surgical robot including a slave operating end and a master operating end controlling movement of the slave operating end; the device comprises:
the first acquisition module is used for acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with the at least a first portion of the joints;
the first switching module is used for controlling the at least a first partial joint to switch from the first master-slave control mode to execute a second master-slave control mode under the condition that the first working environment data meets a first mode switching condition;
the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes.
In another aspect, the present application provides a flexible surgical robot comprising:
from the operating end;
the main operation end is used for controlling the movement of the auxiliary operation end;
the device comprises a processor and a memory, wherein at least one instruction or at least one section of program is stored in the memory, and the at least one instruction or the at least one section of program is loaded and executed by the processor to realize the master-slave control method of the flexible surgical robot in any embodiment.
In another aspect, the present application provides a computer readable storage medium having stored therein at least one instruction or at least one program loaded and executed by a processor to implement a master-slave control method of a flexible surgical robot of any of the above embodiments.
According to the master-slave control method, device, robot and medium for the flexible surgical robot, under the condition that the current first working environment data of the flexible surgical robot meets the first mode switching condition, master-slave control mode switching is conducted on at least a first part of joints, so that the master-slave control mode is not limited to a single master-slave control mode in the master-slave control process, coexistence of multiple master-slave control mapping modes is achieved, smoothness and consistency of the whole surgical operation process are improved, and master-slave control experience of the flexible surgical robot is improved. Meanwhile, as different master-slave control modes correspond to different control errors and interference characteristics, the 'blind area' of a certain master-slave control mode can be avoided through the switching control mode, the potential safety hazard of operation is reduced, and the robustness and the stability of the system are enhanced to a certain extent.
Drawings
In order to more clearly illustrate the technical solutions and advantages of embodiments of the present application or of the prior art, the following description will briefly introduce the drawings that are needed in the embodiments or the prior art descriptions, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art;
FIG. 1 is a schematic illustration of an application scenario of a surgical robotic system, according to an exemplary embodiment;
FIG. 2 is a schematic diagram illustrating the structure of a slave manipulator in a surgical robotic system, according to an exemplary embodiment;
FIG. 3 is a schematic view of a portion of the structure of an instrument tip from an operating device, according to an exemplary embodiment;
FIG. 4 is a flow diagram illustrating a master-slave control method of a flexible surgical robot, according to an exemplary embodiment;
FIG. 5 is a flow diagram illustrating another master-slave control method of a flexible surgical robot according to an exemplary embodiment;
FIG. 6 is a flow diagram illustrating another master-slave control method of a flexible surgical robot according to an exemplary embodiment;
FIG. 7 is a flow chart illustrating a control process for a master-slave control mode based on joint angle mapping, according to an exemplary embodiment;
FIG. 8a is a schematic view of a partial structure of a flexible surgical robot according to an exemplary embodiment;
FIG. 8b is a schematic view of a flexible surgical robot illustrating the configuration of the flexible surgical robot with respect to joint angles, according to an example embodiment;
FIG. 8c is a schematic diagram of an equivalent structure of a flexible surgical robot, according to an example embodiment;
FIG. 9 is a flow diagram illustrating a control process for a master-slave control scheme based on Cartesian pose mapping, according to an exemplary embodiment;
FIG. 10 is a simplified structural schematic diagram of a flexible surgical robot shown according to an exemplary embodiment;
FIG. 11 is a flow diagram illustrating another master-slave control method of a flexible surgical robot according to an exemplary embodiment;
FIG. 12 is a block diagram of a master-slave control device of a flexible surgical robot, according to an exemplary embodiment;
fig. 13 is a block diagram of a hardware configuration of an electronic device of a master-slave control method of a flexible surgical robot according to an exemplary embodiment.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.
It should be noted that the terms "first," "second," and the like in the description and the claims of the embodiments of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to make the objects, technical solutions and advantages disclosed in the embodiments of the present application more apparent, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present application embodiments and are not intended to limit the present application embodiments.
The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.
Fig. 1 shows a schematic view of an application scenario of a surgical robotic system. The surgical robot system of the present embodiment includes a master operation device 100 (i.e., a control means), and a slave operation device 200 (i.e., a surgical robot) controlled by the master operation device 100.
The master operation device 100 has a control input device capable of transmitting a control command to the slave operation device 200 according to the action of the operator's hand and/or foot to drive and adjust the posture of the robot arm assembly 210 of the slave operation device 200 and drive the execution instrument of the robot arm assembly 210 to perform a corresponding operation.
As shown in fig. 2 and 3, the slave manipulator 200 has a robot arm assembly 210 for performing a surgical operation, a driving device 220 for driving the robot arm assembly 210 according to a control command, and a base 230 for supporting the driving device 220, wherein the robot arm assembly 210 includes at least one flexible robot arm 211, and each flexible robot arm 211 may be loaded at an end thereof with a performing instrument for performing a different or the same surgical operation, including but not limited to clamping, cutting, shearing, suturing, electro-cutting, or electro-coagulation, etc. For example, the implement may be any of a variety of implements including, but not limited to, needle-holding forceps, scissors, graspers, and clip appliers. Needle-holding forceps instruments are generally used for realizing operations such as clamping, suturing, knotting and the like, shearing instruments are generally used for realizing operations such as thread shearing, dissection, cutting and the like, grasping forceps instruments are generally used for realizing operations such as grasping, pulling and the like, and clip applier instruments are generally used for ligating in cooperation with ligature clips.
Optionally, as further shown in fig. 1, the surgical robot system further includes an image device 400, where the image device 400 is configured to acquire an image of the surgical field in the cavity (referred to as the body cavity of the patient) captured by the endoscope, and further perform imaging processing on the image of the surgical field, and transmit the image to a first display device of the image device 400 and/or a second display device (not shown in the figure) of the main operating device 100 for displaying, so that the operator can observe the image of the surgical field. The surgical field images include, but are not limited to, the type, number, position and pose of the implement within the body cavity, the morphology of the target organ tissue and surrounding vessels that need to be manipulated, and the like. Further, an endoscope for assisting in capturing images of the surgical field may be loaded from one flexible manipulator 211 in the manipulator assembly 210 of the manipulator 200, and may be displayed by the first display device and/or the second display device. It is to be understood that the image displayed by the image device 400 may be a two-dimensional or three-dimensional image. Endoscopes can include a variety of endoscopes used in surgery, such as thoracoscopes, arthroscopes, nasoscopes, and the like.
Optionally, the surgical robot system further includes a support device 300 (e.g., an operating table) for supporting the surgical object for surgery, and the support device 300 may be replaced with another surgical platform according to the type of surgery, which is not limited in this embodiment.
It should be noted that fig. 1 to 3 are only examples. The surgical robot system is not limited to the device structure or number shown in the above figures, and in other application scenarios, corresponding adjustments may be made, such as adding or subtracting devices in fig. 1, adjusting the number of devices or components, or adjusting the structure of devices or components, etc.
The master control end in a flexible surgical robot is typically a rigid structure, while the slave control end or instrument is typically comprised of at least a plurality of flexible continuum(s) that can bend (or bend unidirectionally, or bend multidirectionally, or bend arbitrarily) or rotate (rotate along an axis) compared to conventional rigid surgical robots, making master-slave control of the flexible surgical robot more difficult.
In the related art, a certain master-slave control manner is generally adopted to uniformly perform all master-slave control involved in the operation of the surgical robot. For example, a control mode of cartesian pose mapping, that is, the change of the end cartesian space pose of the master control end, is synchronously reflected as the change of the end cartesian space pose of the slave control end in a certain mapping mode. Because of the difference of master-slave configuration, when the rigid master control end is used for controlling the flexible slave control end in a whole course by adopting a control mode of Cartesian pose mapping, the small-amplitude change of the terminal space pose of the master control end can correspond to the large-amplitude bending or rotation change of individual or all continuous bodies of the slave control end, or certain requirements are brought to the performance of a brake device of the slave control end and the transmission performance of the slave control end, or the continuity of operation experience is difficult to ensure. Therefore, the existing control mode cannot ensure the smoothness and continuity of the whole operation process, and certain operation safety hidden trouble is brought.
In view of this, the embodiments of the present application provide a master-slave control method, apparatus, robot and medium for a flexible surgical robot, where the flexible surgical robot includes a slave operation end and a master operation end that controls movement of the slave operation end, and in a process of executing a first master-slave control mode by at least a first partial joint in the master operation end and the slave operation end, current first working environment data of the flexible surgical robot is obtained; the first working environment data is associated with at least a first portion of the joints; controlling at least a first partial joint to switch from a first master-slave control mode to execute a second master-slave control mode under the condition that the first working environment data meets a first mode switching condition; the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes. Therefore, under the condition that the current first working environment data of the flexible surgical robot meets the first mode switching condition, the master-slave control mode switching is performed on at least the first partial joint, so that the master-slave control mode is not limited to a single master-slave control mode in the master-slave control process, the coexistence of multiple master-slave control mapping modes is realized, the smoothness and continuity of the whole surgical operation process are improved, and the master-slave control experience of the flexible surgical robot is improved. Meanwhile, as different master-slave control modes correspond to different control errors and interference characteristics, the 'blind area' of a certain master-slave control mode can be avoided through the switching control mode, the potential safety hazard of operation is reduced, and the robustness and the stability of the system are enhanced to a certain extent.
The embodiment of the application provides a master-slave control method of a flexible surgical robot. A specific workflow of a master-slave control method applied to one of the flexible surgical robots of fig. 1 is further described below.
Fig. 4 is a flow diagram illustrating a master-slave control method of a flexible surgical robot according to an exemplary embodiment. As shown in fig. 4, the flexible surgical robot includes a slave manipulator and a master manipulator that controls movement of the slave manipulator. The slave operating end comprises at least a flexible continuum capable of bending or rotating. The number of slave operating terminals may be one or more. The master operation terminal and one of the slave operation terminals can constitute an operation portion for performing master-slave control. The main operation end can drive the corresponding slave operation end to execute corresponding operation through the connected joint under the drive of external force. In the case that the number of the slave operation ends is plural, different master operation ends can drive the associated slave operation ends to execute corresponding operations through joints. The method comprises the following steps:
s401: acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with at least a first portion of the joints;
S403: and controlling at least the first partial joint to switch from the first master-slave control mode to execute the second master-slave control mode under the condition that the first working environment data meets the first mode switching condition.
Wherein the first working environment data may refer to current environment data during execution of the first master-slave control mode by at least a first partial joint in the flexible surgical robot. The first operating environment data is associated with at least a first portion of the joint. The at least first partial joint is at least one joint or all joints of the flexible hand robot corresponding to each operation part for executing master-slave control. The operation parts corresponding to the at least first part of joints all execute the same first master-slave control mode.
Optionally, the first working environment data may include one or more of system state data for characterizing a current state of the flexible surgical robot, sensor data for characterizing pose and/or motion properties of related components in the flexible surgical robot, and image acquisition data for characterizing a spatial state of related components in the flexible surgical robot.
By way of example, the system status data may include, but is not limited to, one or more of readiness status data, intra-operative status data, position movement status data, and the like. The sensor data includes, but is not limited to, one or more of pose, velocity, acceleration, etc., detected by the sensor. Image acquisition data includes, but is not limited to, acquired environmental image data, and the like.
The first mode switching condition refers to a trigger condition for switching from the first master-slave control mode to the second master-slave control mode. The first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes. The first master-slave control mode and the second master-slave control mode may refer to master-slave control modes of the adaptive surgical robot.
Optionally, in the first master-slave control mode and the second master-slave control mode, one of the first master-slave control mode and the second master-slave control mode is a master-slave control mode based on Cartesian pose mapping, and the other is a master-slave control mode based on joint angle mapping. The master-slave control mode based on the Cartesian pose mapping, namely the change of the Cartesian space pose at the tail end of the master control end, is synchronously reflected as the change of the Cartesian space pose at the tail end of the slave control end in a certain mapping mode. The cartesian positions and attitudes include a cartesian position and a cartesian attitude, i.e., both the cartesian position and the cartesian attitude are controlled. The master-slave control mode based on joint angle mapping, namely joint angle change of the master control end, is synchronously reflected as joint angle (namely rotation and bending of the continuum) change of the slave control end in a certain mapping mode.
In practical application, an operator of the flexible surgical robot controls the movement of the slave operating end by manipulating the master operating end, i.e. the flexible surgical robot enters a master-slave control mode. The control mode of the flexible surgical robot may be a single control mode or a hybrid control mode, for example, the single control mode may refer to a master-slave control mode in which all joints performing the master-slave control mode are based on cartesian pose mapping or a master-slave control mode based on joint angle mapping; the hybrid control mode may refer to a master-slave control mode in which part of joints performing the master-slave control mode are master-slave control modes based on Cartesian pose mapping, and the other part of joints are master-slave control modes based on joint angle mapping.
Next, during execution of a first master-slave control mode by at least a first partial joint in the master and slave operating ends, first working environment data of the flexible surgical robot is acquired that is current and related to the at least first partial joint. In the case where the first operating environment data indicates that the first mode switching condition is not satisfied, controlling at least the first partial joint to remain executing the first master-slave control mode. And controlling at least the first partial joint to switch from the first master-slave control mode to execute the second master-slave control mode in the case that the first working environment data indicates that the first mode switching condition is satisfied.
In the master-slave control process, the switching from the first master-slave control mode to the second master-slave control mode is not limited, and the switching from the second master-slave control mode to the first master-slave control mode may be performed after the switching to the second master-slave control mode. Of course, in the master-slave control process, the method is not limited to the first master-slave control mode and the second master-slave control mode, and can be applied to master-slave control modes including other adaptations.
According to the embodiment, under the condition that the current first working environment data of the flexible surgical robot meets the first mode switching condition, the master-slave control mode is switched for at least the first part of joints, so that the master-slave control mode is not limited to a single master-slave control mode in the master-slave control process, coexistence of multiple master-slave control mapping modes is realized, smoothness and continuity of the whole surgical operation process are improved, and master-slave control experience of the flexible surgical robot is improved. Meanwhile, as different master-slave control modes correspond to different control errors and interference characteristics, the 'blind area' of a certain master-slave control mode can be avoided through the switching control mode, the potential safety hazard of operation is reduced, and the robustness and the stability of the system are enhanced to a certain extent.
In an alternative embodiment, as shown in fig. 5, in a case where the first working environment data meets the first mode switching condition, the method further includes, before controlling at least the first partial joint to switch from the first master-slave control mode to execute the second master-slave control mode:
s501: and judging whether the first working environment data meets a first mode switching condition.
The first mode switching condition may relate to a mode type of the first master-slave control mode and a working scenario corresponding to the first working environment data. By way of example, the pattern types may include, but are not limited to, a Cartesian pose map type, a joint angle map type, and the like. The working scene may include, but is not limited to, a scene that is mode switched based on a Cartesian pose map, a scene that is mode switched based on a joint angle map.
Optionally, after the current first working environment data of the flexible surgical robot is acquired, whether the first working environment data meets the first mode switching condition may be judged first, and if it is determined that the first working environment data meets the first mode switching condition, a control mode switching step is performed, that is, at least a first part of joints are controlled to switch from the first master-slave control mode to the second master-slave control mode. If the first working environment data does not meet the first mode switching condition, the switching step of the control mode is not required to be executed, and the first part of joints are kept unchanged in executing the first master-slave control mode.
Specifically, the manner of determining whether the first working environment data satisfies the first mode switching condition, that is, whether the working scenario is suitable for mode switching may include, but is not limited to:
1) Based on the system state corresponding to the system state data, for example, cartesian pose mapping control is adopted in part of the system states, and joint mapping control is adopted in part of the system states, for example, system state 1: moving the flexible instrument to a posture suitable for performing an operation, wherein joint mapping control can be adopted; system state 2: performing surgery, which may employ Cartesian position mapping control; the switching of the control modes can be performed synchronously when the system state is switched.
2) Based on sensor data, such as combining joint angles of a slave operating end and a forward kinematics algorithm of a robot, or combining image acquisition data obtained by a camera and a machine vision algorithm, or acquiring a Cartesian space pose of a slave control end instrument through a magnetic sensor, such as judging that the current pose of the slave control end instrument is near a boundary of a working space and needs to leave the boundary, a suitable control angle leaving the boundary is difficult to find in an intuitive way through a master-slave control mode of Cartesian pose mapping, and the slave control mode of joint mapping can be switched to, so that a user can conveniently control the slave control end to leave the boundary.
As an alternative embodiment, the first mode switch condition may be a condition triggered based on a manual operation, including, for example, but not limited to, pressing a physical button associated with a mode switch, a virtual button on an interface, recognizing a switching voice associated with a mode switch, sensor sensing, etc.
According to the embodiment, the two master-slave control methods are combined, and the master-slave control method for switching is performed based on a specific working scene, so that the influence caused by the defects of the master-slave control method is reduced while the advantages of the master-slave control method are reserved to the greatest extent, and the stability of the system is improved.
In an alternative embodiment, where the first master-slave control mode is a cartesian pose mapping-based master-slave control mode and the second master-slave control mode is a joint angle mapping-based master-slave control mode, the first mode switching condition includes, but is not limited to, at least one of:
the main operation end controls the slave operation end to execute bending motion corresponding to the joint;
the main operation end controls the slave operation end to execute rotary motion corresponding to the joint;
the main operation end controls the slave operation end to execute bending motion corresponding to the instrument carried by the joint;
the main operation end controls the auxiliary operation end to execute rotary motion corresponding to the instrument carried by the joint;
The main operation end controls the boundary from the operation end to leave the working space to the core area of the working space.
Optionally, the first mode switching condition applicable to mapping from the cartesian pose to the joint angle mapping, for example, the working scene corresponding to the mode switching condition may include, but is not limited to, at least one of the following: 1) The master operation end controls the slave operation end to execute bending motion corresponding to the joint, for example, the master control end controls the bending of the slave control end; 2) The master operation end controls the slave operation end to execute rotary motion corresponding to the joint, for example, the master control end controls the slave control end to rotate; 3) The main operation end controls the slave operation end to execute bending motion corresponding to the instrument carried by the joint, for example, the main control end controls the bending of the instrument carried by the slave control end; 4) The main operation end controls the slave operation end to execute rotary motion corresponding to the instrument carried by the joint, for example, the main control end controls the rotation of the instrument carried by the slave control end; 5) The main operator controls the departure of the operator from the boundary of the workspace to the core region of the workspace, e.g., controls the departure of the controller from the boundary of its range of motion back to the core region of the workspace, etc.
According to the embodiment, based on the specific first mode switching condition for determining to switch from the master-slave control mode based on the Cartesian pose mapping to the master-slave control mode based on the joint angle mapping, corresponding mode switching operation is executed, so that the master-slave mode switching is performed quickly in an actual application scene meeting any one of the first mode switching conditions, the calculated amount of the master-slave mode switching is reduced, and the response rate of the mode switching is improved.
In an alternative embodiment, where the first master-slave control mode is a master-slave control mode based on joint angle mapping and the second master-slave control mode is a master-slave control mode based on cartesian pose mapping, the first mode switching conditions include, but are not limited to, at least one of:
the main operation end controls the slave operation end to execute stripping or separating operation corresponding to the instrument carried by the joint;
the main operation end controls the slave operation end to execute obstacle avoidance operation corresponding to the joint or the carried instrument;
the main operation end is moved while maintaining the posture of the end of the sub operation end.
Optionally, the first mode switching condition applicable to mapping from the joint angle to the cartesian pose mapping, for example, the working scene corresponding to the mode switching condition may include, but is not limited to, at least one of the following: 1) The main operation end controls the slave operation end to execute stripping or separating operation corresponding to the instrument carried by the joint, for example, controls the slave operation end to execute fine operations such as stripping, separating and the like; 2) The master operation end controls the slave operation end to perform obstacle avoidance operation corresponding to the joint or the carried instrument, for example, controls the slave control end or the carried instrument to perform obstacle avoidance operation; 3) The master operation end is moved while maintaining the slave operation end tip attitude unchanged, for example, the master control end is moved while maintaining the slave control end tip attitude unchanged, and the like.
According to the embodiment, based on the specific first mode switching condition that the joint angle mapping-based master-slave control mode is determined to be switched to the Cartesian pose mapping-based master-slave control mode, corresponding mode switching operation is executed, so that the master-slave mode switching is performed quickly in an actual application scene meeting any first mode switching condition, the calculated amount of the master-slave mode switching is reduced, and the response rate of the mode switching is improved.
In an alternative embodiment, as shown in fig. 6, in a case where the first working environment data satisfies the first mode switching condition, controlling at least the first partial joint to switch from the first master-slave control mode to execute the second master-slave control mode includes:
s601: generating a switching control instruction under the condition that the first working environment data meets a first mode switching condition;
s603: in response to a triggering operation for the switching control instruction, at least the first partial joint is controlled to switch from the first master-slave control mode to the second master-slave control mode.
Alternatively, in the case where it is determined that the first working environment data satisfies the first mode switching condition, that is, when it is determined that the external scene satisfies the mode switching, a switching control instruction for prompting that the switching control operation can be performed is generated. The switching control instruction can be reflected to an operator in the form of voice, text and signal lamps. Upon receiving a trigger operation for the switching control instruction, for example, an operation in the form of a touch virtual key or a physical button, a rotary button, or the like for the switching control instruction, a mode switch is performed by controlling at least the first partial joint to switch from the first master-slave control mode to the second master-slave control mode in response to the trigger operation.
According to the embodiment, under the condition that the first working environment data meets the first mode switching condition, the switching control instruction is firstly generated, and then at least the first partial joint is controlled to be switched from the first master-slave control mode to the second master-slave control mode based on the triggering operation aiming at the switching control instruction, so that the authority of manual operation control is increased in the master-slave control mode, the accuracy of the master-slave control mode switching is further improved, the mode switching judgment error is avoided, and the operation safety is improved.
In an alternative embodiment, as shown in fig. 7, in the second master-slave control mode, the master-slave control mode is a master-slave control mode based on joint angle mapping, and the method further includes:
s701: acquiring a main operation end structure and a slave operation end structure corresponding to at least a first part of joint;
s703: acquiring a control demand amount for executing master-slave control;
s705: determining a mapping relation between master control and slave control based on a master operation end structure, a slave operation end structure and a control demand;
s707: acquiring a source control quantity of a main operation end in a second master-slave control mode, and determining a target control quantity of a corresponding part of a slave operation end based on the source control quantity and a mapping relation;
S709: based on the target control amount, the slave operating end is controlled to perform the corresponding movement.
Optionally, the second master-slave control mode is a master-slave control mode based on joint angle mapping, and the master-slave control mode is performed through joint angle mapping subsequently. At this time, the master operation end structure and the slave operation end structure corresponding to at least the first partial joint, and the control demand amount for executing the master-slave control may be acquired, and the mapping relationship between the master and slave controls may be determined. Then, the source control quantity of the main operation end in the second master-slave control mode is obtained, the target control quantity of the corresponding component of the auxiliary operation end is determined based on the source control quantity and the mapping relation, and the auxiliary operation end is controlled to execute corresponding movement based on the target control quantity. Therefore, based on the master operation end structure, the slave operation end structure and the control demand, the mapping relation between the master control and the slave control based on the joint angle mapping and the corresponding target control quantity are flexibly determined, so that the master-slave control efficiency and response timeliness are improved, and the operation safety is further improved.
As shown in fig. 8a, 8b and 8c, a flexible surgical robot comprising a flexible continuous body of radially bendable, axially rotatable flexible tubing will be described as an example.
The flexible continuum is defined as l in length, θ in bending angle, and φ in rotation angle. The direction along the axial direction of the flexible continuous body from the base to the tail end of the flexible continuous body is the Y-axis forward direction, the axial direction around which the flexible continuous body is bent is the X-axis forward direction, and the Cartesian space positions of the tail end of the flexible continuous body are as follows:
(1) In the case where the second master-slave control mode is a master-slave control mode based on joint angle mapping, one slave control end composed of N flexible continuum with 2 degrees of freedom of a single continuum is considered.
Assuming that each joint of the main control end has only one degree of freedom, the main control end can adopt a rigid link mechanism with 2N degrees of freedom to control the main control end based on joint angle mapping.
The master-slave control mode based on the joint angle mapping is exemplified, and the input (i.e. source control amount) can be the joint angle of each continuum of the rigid link mechanism, namely epsilon in ＝(q 1 ,q 2 ,…,q 2N ) The output (i.e. target control quantity) is the bending and rotation angle of each continuum of the flexible mechanism, namely epsilon out ＝(φ 1 ,θ 1 ,φ 2 ,θ 2 ,…,φ N ,θ N ). Wherein, the joint angle can be obtained by installing a joint encoder to collect data.
Another example of the control mode based on the joint angle mapping is that the input (i.e. source control amount) can be the increment of the joint angle of each continuum of the rigid link mechanism, namely epsilon in ＝(Δq 1 ,Δq 2 ,…,Δq 2N ) The method comprises the steps of carrying out a first treatment on the surface of the The output (i.e. the target control quantity) is the increment of the bending and rotation angles of each continuum of the flexible mechanism, namely epsilon out ＝(Δφ 1 ,Δθ 1 ,Δφ 2 ,Δθ 2 ,…,Δφ N ,Δθ N ). The increment of the joint angle can be obtained by installing a joint encoder to collect data and making a difference between the current period encoder value and the last period encoder value.
ε in And epsilon out The mapping relation of (2) is a function operation, namely the mapping relation between the master control and the slave control is as follows:
ε out ＝f(ε in )
the mapping relation between the master control and the slave control can be equal proportion mapping, namely phi 1 ＝kq 1 ，θ 2 ＝kq 2 ，…，θ N ＝kq 2N Where k is a fixed constant.
The mapping relationship between the master control and the slave control can be a non-time-varying linear mapping, namely:
ε out ＝Aε in
where A is a time independent constant matrix.
The mapping between the master and slave control may be of other types, such as time-varying, non-linear.
In practical application, which kind of mapping relation between master control and slave control is adopted is related to a master control end structure, a slave control end structure, actual control requirements and the like. If the main control end is a serial connecting rod mechanism, the slave control end is a serial flexible body, an angle increment mapping mode is adopted, the mapping relation can be in an equal proportion mapping mode, and the proportion can be a constant value determined based on the working space of the main hand and the working space of the slave hand; the method can also be a self-defined value in a reasonable range which is set by an operator based on the operation experience; the method can also be a time-varying dynamic value which dynamically changes along with the actual operation scene, for example, the value is properly reduced in the operation, and the control precision of the terminal instrument at the control end is improved; the value is properly increased in the non-operative state, and the operation preparation time is reduced; if the control end moves to the vicinity of the boundary of the working space, the value can be properly reduced, and the operation safety is improved. If the main control end is a parallel mechanism, the slave control end is a serial flexible body, and when joint angle mapping is adopted, because of the coupling relation among partial joint angles of the main operation end, mapping based on a constant matrix is adopted at the moment, A is a decoupling matrix of the parallel mechanism, and the angle or the angle increment after decoupling is mapped to the slave control end.
Alternatively, the slave control end may be a partially flexible, partially rigid hybrid structure, the control method being adapted to control the flexible portion of the slave control end with all or part of the articulation of the master control end.
Alternatively, the rigid linkage mechanism of the main control end can have a redundant structure, and the control mode is also applicable to passive joints without driving.
Alternatively, the input (i.e., source control amount) may be a hybrid input, i.e., the input of a portion of the joint is a joint angle and a portion is a joint angle increment.
In an alternative embodiment, the second master-slave control mode is a master-slave control mode based on cartesian pose mapping, and the method further comprises:
s901: acquiring the current degree of freedom of the slave operation end; the degree of freedom is less than or equal to the maximum degree of freedom of the Cartesian space;
s903: determining a mapping relation between master control and slave control;
s905: acquiring a source control quantity of the tail end of the main operation end in a second master-slave control mode; the pose parameter quantity of the source control quantity is matched with the quantity of the degrees of freedom;
s907: determining a target control amount of a corresponding component from an operation end based on the source control amount and the mapping relation;
s909: based on the target control amount, the slave operating end is controlled to perform the corresponding movement.
Optionally, the second master-slave control mode is a master-slave control mode based on a cartesian pose mapping, and the master-slave control mode is performed subsequently through the cartesian pose mapping. At this time, the current degree of freedom from the operation end, which is less than or equal to the maximum degree of freedom in the cartesian space, may be acquired; determining a mapping relation between master control and slave control; acquiring a source control quantity of the tail end of the main operation end in a second master-slave control mode; the pose parameter quantity of the source control quantity is matched with the quantity of the degrees of freedom; determining a target control amount of a corresponding component from an operation end based on the source control amount and the mapping relation; and further controls the slave operating end to perform a corresponding movement based on the target control amount. Therefore, based on the mapping relation between the current degree of freedom of the slave operation end and the master-slave control, the target control quantity of the corresponding part of the slave operation end in the master-slave control based on the Cartesian pose mapping is flexibly determined, so that the master-slave control efficiency and response timeliness are improved, and the operation safety is further improved.
As shown in fig. 8a and 10, the following description continues with the example of a flexible surgical robot comprising a flexible continuum of radially bendable, axially rotatable flexible tubes.
Generally, the cartesian space has 6 degrees of freedom, and the main control end can adopt a rigid linkage mechanism with at least 6 degrees of freedom to control the main control end based on the cartesian space pose.
The control mode based on the Cartesian pose mapping is exemplified, and the input (i.e. source control quantity) can be Cartesian space at the tail end of the main control endPose, i.e. epsilon in ＝(p m ,R m )，p m For the spatial displacement of the end of the main control end relative to a certain coordinate system, R m The tail end of the main control end rotates relative to the space of a certain coordinate system; the output (i.e. the target control quantity) being the Cartesian space position of the end of the flexible mechanism, i.e. delta out ＝(p s ,R s )，p s For spatial displacement from the control end extremity relative to the same coordinate system, R s For spatial rotation of the master control end tip relative to the same coordinate system.
The manner of obtaining the cartesian space position of the end of the main control end may include, but is not limited to: acquiring joint position by joint encoder, and calculating terminal pose by positive kinematics, namely (p) m ,R m ) =f (q); and identifying the pose of the tail end of the main control end through a depth camera, or combining with a space anchor point or combining with a machine vision algorithm.
The manner of obtaining the cartesian space position from the control end may include, but is not limited to: the motor encoder collects the motor position, firstly converts the motor position into the working space of the flexible body, and then calculates the terminal pose through positive kinematics, namely (p) m ,R m ) =f (q); the pose of the control end is directly measured by a magnetic sensor.
Another example, the control mode based on the Cartesian pose mapping can be the relative increment of the Cartesian space pose of the end of the rigid linkage mechanism, namely epsilon, with the input (namely the source control quantity) in ＝(Δp m ,ΔR m )，Δp m Delta R is the increment of the spatial displacement of the tail end of the main control end relative to a certain coordinate system m The increment of the space rotation of the tail end of the main control end relative to a certain coordinate system can be expressed by a space angle, a quaternion and the like; the output (i.e. target control quantity) being the relative increment of the Cartesian space position of the end of the flexible mechanism, i.e. ε out ＝(Δp s ,ΔR s )，Δp s For the increment of spatial displacement from the control end tip relative to a certain coordinate system ΔR s In increments of spatial rotation from the control end tip relative to a coordinate system.
ε in And epsilon out The mapping relation of the control system is a self-defined function operation, namely the mapping relation between the master control system and the slave control system is as follows:
ε out ＝f(ε in )
the mapping relation between the master control and the slave control can be equal proportion mapping, namely
Δp s ＝kΔp m ,ΔR s ＝kΔR m
Where k is a fixed constant.
The mapping relationship between the master control and the slave control may be other mapping methods described in the control method based on the joint angle mapping.
The cartesian space pose mapping may include only a position mapping or a pose mapping, or a partial position mapping or a partial pose mapping, taking an example of an equal proportion mapping, the mapping relationship may be the following mapping manner:
Δp s ＝kΔp m ,ΔR s ＝0
Or (b)
Δp s ＝0,ΔR s ＝kΔR m
If all degrees of freedom do not need to be controlled or the factors such as redundancy, safety, use comfort and the like are considered, the specific number of degrees of freedom of the main control end can be correspondingly changed.
Alternatively, if the degrees of freedom of the slave control end are less than the cartesian space degrees of freedom 6, a part of the joints of the master control end may be in a locked state while joint angle mapping control is performed, and the remaining movable joints are used to implement joint angle mapping.
In an alternative embodiment, the method further comprises:
acquiring current second working environment data of the flexible surgical robot in the process of executing a second master-slave control mode by at least a second part of joints in the master operation end and the slave operation end; the second working environment data is associated with at least a second portion of the joint;
and controlling at least the second partial joint to switch from the second master-slave control mode to execute the first master-slave control mode under the condition that the second working environment data meets the second mode switching condition.
Alternatively, in the hybrid control mode, in addition to performing the corresponding mode switching on at least the first partial joint as described above, a differentiated mode switching may also be performed on at least the second partial joint other than the first partial joint. The second partial joint is different from the first partial joint in the mode type of the current master-slave control mode.
The hybrid control mode is that part of joints of a master operation end and a slave operation end adopt joint mapping control, and part of joints of the master operation end and the slave operation end adopt Cartesian pose mapping control. For example, joint mapping control is employed from a flexible continuum at the manipulator end, while cartesian mapping control is employed from the cartesian pose (non-position) of the manipulator end instrument. The control mode switching at this time may refer to switching the control mode of the flexible continuum from the operation end, such as switching from joint mapping control to cartesian pose mapping control; switching of the control mode of the distal instrument from the operating end, such as from Cartesian gesture mapping control to joint mapping control, or simultaneous switching of both.
Acquiring current second working environment data of the flexible surgical robot in the process of executing a second master-slave control mode by at least a second part of joints in the master operation end and the slave operation end; the second operating environment data is associated with at least a second portion of the joint. And controlling at least the second partial joint to switch from the second master-slave control mode to execute the first master-slave control mode under the condition that the second working environment data meets the second mode switching condition. The second working environment data is similar to the first working environment data, and the second mode switching condition is similar to the first mode switching condition, and will not be described herein.
According to the embodiment, the master-slave mode switching control different from the first part of joints is performed on the second part of joints in the same flexible surgical robot, so that the hybrid mode switching control is realized, the differential control is conveniently performed on the different joints in the same flexible surgical robot, and the flexibility and the accuracy of the master-slave control are improved.
Fig. 11 is a flow diagram illustrating another master-slave control method of a flexible surgical robot according to an exemplary embodiment.
S1101: acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first operating environment data is associated with at least a first portion of the joints.
S1103: invoking a trained mode switching judging model, processing the first working environment data, and outputting a judging result of whether the first working environment data meets the first mode switching condition;
the mode switching judging model is obtained by training based on working environment sample data, wherein the working environment sample data comprises one or more of system state sample data used for representing the current state of the flexible surgical robot, sensor sample data used for representing the pose and/or motion attribute of related components in the flexible surgical robot and image acquisition sample data used for representing the spatial state of the related components in the flexible surgical robot.
In practical application, the mode switching judgment model can be obtained through training by collecting working environment sample data. The operating environment sample data marks the control pattern. The working environment sample data may be sensor data, including, for example, one or more of the following: 1) System state sample data for characterizing a current state of the flexible surgical robot, such as a current state of the system (e.g., a state in which the system is in operation, is in preparation for operation, is moved from a control end to a surgical site, etc.); 2) Sensor sample data for characterizing pose and/or motion properties of related components in the flexible surgical robot, such as joint angles, joint speeds, joint accelerations acquired from encoders of respective joints of the main control end, input instructions, output moments of respective motors; 3) Image acquisition sample data for characterizing the spatial state of related components in a flexible surgical robot, such as the master control end, slave control end, instrument, relative pose relationship between surgical objects acquired by a camera.
The mode switching judgment model can also be a mode switching strategy for autonomously learning corresponding control modes under various working scenes by a method such as reinforcement learning in a simulation environment or a simplified working environment.
And after the mode switching judging model is trained, processing the first working environment data through the mode switching judging model, judging whether the condition is triggered or not by the trained mode switching judging model, and outputting a judging result of whether the first working environment data meets the first mode switching condition or not. The first operating environment data may be sensor data collected in real-time.
The first mode switching condition may be, for example, a movement from the control side to a boundary of the working space, a need to switch from a cartesian space mapping to a joint angle mapping control. The first mode switching condition may be movement from the control end to a surgical-friendly position, requiring switching from joint angle mapping control to Cartesian space mapping control. The first mode switching condition may be that the operation has been completed, a switch from cartesian space mapping control to joint angle mapping control is required, etc. The specific first mode switching condition is a set of multiple judging conditions, and the content of the first mode switching condition can be determined by the surgical procedure, the surgical scene and the like of the flexible surgical robot.
S1105: and controlling at least the first partial joint to switch from the first master-slave control mode to execute the second master-slave control mode under the condition that the first working environment data meets the first mode switching condition.
Optionally, if the judgment result indicates that the first working environment data meets the first mode switching condition, that is, the model judges the switchable control mode, the mode switching is triggered. Then, a prompt message may be given to the user, and if the user accepts the suggestion, a trigger operation for the mode switching operation is performed, for example, an operation in the form of a virtual key or a physical button, a rotary button, or the like for performing mode switching by touch control is performed. And after the trigger operation for the mode switching operation is performed by the user, controlling at least the first partial joint to switch from the first master-slave control mode to the second master-slave control mode. Or, the control mode may be switched by itself without prompting the user, i.e. at least the first partial joint is controlled to switch from the first master-slave control mode to the second master-slave control mode. For example, the mode switching occurs in the system state switching process, and at this time, the main control end is in a state of position maintenance, so that the use experience of the user is not affected by the self-switching control mode. Or the user may be prompted with information, and after the user performs a corresponding triggering operation, for example, an operation in a form of a touch virtual key, a physical button, a rotary button, or the like, at least the first part of the joints is controlled to switch from the first master-slave control mode to the second master-slave control mode.
According to the embodiment, the first working environment data is processed through the trained mode switching judging model, the judging result of whether the first working environment data meets the first mode switching condition is output, and under the condition that the current first working environment data of the flexible surgical robot meets the first mode switching condition, the master-slave control mode switching is performed on at least a first part of joints, so that the master-slave control process is not limited to a single master-slave control mode, multiple master-slave control mapping modes are coexisted, the judging complexity of the mode switching condition is simplified, and the processing efficiency in the master-slave control process is accelerated. In addition, smoothness and continuity of the whole operation process are improved, and master-slave control experience of the flexible operation robot is improved. Meanwhile, as different master-slave control modes correspond to different control errors and interference characteristics, the 'blind area' of a certain master-slave control mode can be avoided through the switching control mode, the potential safety hazard of operation is reduced, and the robustness and the stability of the system are enhanced to a certain extent.
The embodiment of the application also provides a master-slave control device of the flexible surgical robot. Fig. 12 is a block diagram of a master-slave control apparatus of a flexible surgical robot, according to an example embodiment. The flexible surgical robot includes a slave operation end and a master operation end controlling movement of the slave operation end, as shown in fig. 12, the master-slave control device of the flexible surgical robot may include at least:
A first obtaining module 1210, configured to obtain current first working environment data of the flexible surgical robot during a process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with at least a first portion of the joints;
a first switching module 1220, configured to control at least the first partial joint to switch from the first master-slave control mode to execute the second master-slave control mode when the first working environment data satisfies the first mode switching condition;
the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes.
In an alternative embodiment, in the first master-slave control mode and the second master-slave control mode, one of the first master-slave control mode and the second master-slave control mode is a master-slave control mode based on Cartesian pose mapping, and the other is a master-slave control mode based on joint angle mapping.
In an alternative embodiment, the first working environment data comprises one or more of system state data for characterizing a current state of the flexible surgical robot, sensor data for characterizing pose and/or motion properties of related components in the flexible surgical robot, and image acquisition data for characterizing a spatial state of related components in the flexible surgical robot.
In an alternative embodiment, the apparatus further comprises:
the judging module is used for judging whether the first working environment data meets a first mode switching condition or not;
the first mode switching condition is related to a mode type of the first master-slave control mode and a working scene corresponding to the first working environment data.
In an alternative embodiment, in the case that the first master-slave control mode is a master-slave control mode based on cartesian pose mapping and the second master-slave control mode is a master-slave control mode based on joint angle mapping, the first mode switching condition includes at least one of:
the main operation end controls the slave operation end to execute bending motion corresponding to the joint;
the main operation end controls the slave operation end to execute rotary motion corresponding to the joint;
the main operation end controls the slave operation end to execute bending motion corresponding to the instrument carried by the joint;
the main operation end controls the auxiliary operation end to execute rotary motion corresponding to the instrument carried by the joint;
the main operation end controls the boundary from the operation end to leave the working space to the core area of the working space.
In an alternative embodiment, where the first master-slave control mode is a master-slave control mode based on joint angle mapping and the second master-slave control mode is a master-slave control mode based on cartesian pose mapping, the first mode switching condition includes at least one of:
The main operation end controls the slave operation end to execute stripping or separating operation corresponding to the instrument carried by the joint;
the main operation end controls the slave operation end to execute obstacle avoidance operation corresponding to the joint or the carried instrument;
the main operation end is moved while maintaining the posture of the end of the sub operation end.
In an alternative embodiment, the apparatus further comprises:
the model processing module is used for calling a trained mode switching judging model, processing the first working environment data and outputting a judging result of whether the first working environment data meets the first mode switching condition or not;
the mode switching judging model is obtained by training based on working environment sample data, wherein the working environment sample data comprises one or more of system state sample data used for representing the current state of the flexible surgical robot, sensor sample data used for representing the pose and/or motion attribute of related components in the flexible surgical robot and image acquisition sample data used for representing the spatial state of the related components in the flexible surgical robot.
In an alternative embodiment, the first switching module is further configured to:
generating a switching control instruction under the condition that the first working environment data meets a first mode switching condition;
In response to a triggering operation for the switching control instruction, at least the first partial joint is controlled to switch from the first master-slave control mode to the second master-slave control mode.
In an alternative embodiment, the apparatus further comprises:
the second acquisition module is used for acquiring current second working environment data of the flexible surgical robot in the process of executing a second master-slave control mode by at least a second part of joints in the master operation end and the slave operation end; the second working environment data is associated with at least a second portion of the joint;
and the second switching module is used for controlling at least the second partial joint to switch from the second master-slave control mode to the first master-slave control mode under the condition that the second working environment data meets the second mode switching condition.
In an alternative embodiment, in the second master-slave control mode, the master-slave control mode is a master-slave control mode based on joint angle mapping, the apparatus further includes a first control module, where the first control module is configured to:
acquiring a main operation end structure and a slave operation end structure corresponding to at least a first part of joint;
acquiring a control demand amount for executing master-slave control;
determining a mapping relation between master control and slave control based on a master operation end structure, a slave operation end structure and a control demand;
Acquiring a source control quantity of a main operation end in a second master-slave control mode, and determining a target control quantity of a corresponding part of a slave operation end based on the source control quantity and a mapping relation;
based on the target control amount, the slave operating end is controlled to perform the corresponding movement.
In an alternative embodiment, when the second master-slave control mode is a master-slave control mode based on cartesian pose mapping, the method further includes a second control module, where the second control module is configured to:
acquiring the current degree of freedom of the slave operation end; the degree of freedom is less than or equal to the maximum degree of freedom of the Cartesian space;
determining a mapping relation between master control and slave control;
acquiring a source control quantity of the tail end of the main operation end in a second master-slave control mode; the pose parameter quantity of the source control quantity is matched with the quantity of the degrees of freedom;
determining a target control amount of a corresponding component from an operation end based on the source control amount and the mapping relation;
based on the target control amount, the slave operating end is controlled to perform the corresponding movement.
It should be noted that, the master-slave control device embodiment of the flexible surgical robot provided in the embodiment of the present application and the master-slave control method embodiment of the flexible surgical robot are based on the same inventive concept.
Embodiments of the present application also provide a flexible surgical robot comprising:
from the operating end;
the main operation end is used for controlling the movement of the auxiliary operation end;
the device comprises a processor and a memory, wherein at least one instruction or at least one section of program is stored in the memory, and the at least one instruction or the at least one section of program is loaded and executed by the processor to realize the master-slave control method of the flexible surgical robot in any embodiment.
Embodiments of the present application also provide a computer readable storage medium storing at least one instruction or at least one program loaded and executed by a processor to implement a master-slave control method of a flexible surgical robot as provided in the above method embodiments.
Alternatively, in the present description embodiment, the storage medium may be located in at least one network server among a plurality of network servers of the computer network. Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The memory of the embodiments of the present specification may be used for storing software programs and modules, and the processor executes various functional applications and data processing by executing the software programs and modules stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for functions, and the like; the storage data area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide access to the memory by the processor.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the master-slave control method of the flexible surgical robot provided by the method embodiment.
The master-slave control method of the flexible surgical robot provided by the embodiment of the application can be executed in a terminal, a computer terminal, a server or similar computing devices. Fig. 13 is a hardware block diagram of a server of a master-slave control method of a flexible surgical robot according to an exemplary embodiment. As shown in fig. 13, the server 400 may vary considerably in configuration or performance and may include one or more central processing units (Central Processing Units, CPU) 410 (the central processing unit 410 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA or the like), a memory 340 for storing data, one or more storage mediums 420 (e.g., one or more mass storage devices) for storing applications 424 or data 422. Wherein memory 430 and storage medium 420 may be transitory or persistent. The program stored on the storage medium 420 may include one or more modules, each of which may include a series of instruction operations on a server. Still further, the central processor 410 may be configured to communicate with the storage medium 420 and execute a series of instruction operations in the storage medium 420 on the server 400. The server 400 may also include one or more power supplies 460, one or more wired or wireless network interfaces 450, one or more input/output interfaces 440, and/or one or more operating systems 421, such as Windows ServerTM, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.
The input-output interface 440 may be used to receive or transmit data via a network. The specific example of the network described above may include a wireless network provided by a communication provider of the server 400. In one example, the input-output interface 440 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the input/output interface 440 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.
It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 13 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the server 400 may also include more or fewer components than shown in fig. 13, or have a different configuration than shown in fig. 13.
It should be noted that: the foregoing sequence of the embodiments of the present application is only for describing, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device and server embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and references to the parts of the description of the method embodiments are only required.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, and the relevant program may be stored in a computer readable storage medium, where the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the present application is not intended to be limiting, but rather is intended to cover any and all modifications, equivalents, alternatives, and improvements within the spirit and principles of the present application.

Claims (14)

1. A master-slave control method of a flexible surgical robot, wherein the flexible surgical robot includes a slave operating end and a master operating end controlling movement of the slave operating end, the method comprising:
Acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with the at least a first portion of the joints;
controlling the at least a first partial joint to switch from the first master-slave control mode to execute a second master-slave control mode under the condition that the first working environment data meets a first mode switching condition;
the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
and in the first master-slave control mode and the second master-slave control mode, one of the first master-slave control mode and the second master-slave control mode is a master-slave control mode based on Cartesian pose mapping, and the other is a master-slave control mode based on joint angle mapping.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the first working environment data includes one or more of system state data for characterizing a current state of the flexible surgical robot, sensor data for characterizing pose and/or motion properties of related components in the flexible surgical robot, and image acquisition data for characterizing a spatial state of related components in the flexible surgical robot.
4. The method of claim 1, wherein the controlling the at least a first partial joint to switch from the first master-slave control mode to perform a second master-slave control mode in the event that the first operating environment data satisfies a first mode switch condition, the method further comprises:
judging whether the first working environment data meets a first mode switching condition or not;
the first mode switching condition is related to a mode type of the first master-slave control mode and a working scene corresponding to first working environment data.
5. The method of claim 4, wherein, in the case where the first master-slave control mode is a cartesian pose mapping-based master-slave control mode and the second master-slave control mode is a joint angle mapping-based master-slave control mode, the first mode switching condition comprises at least one of:
the main operation end controls the slave operation end to execute bending motion corresponding to the joint;
the main operation end controls the slave operation end to execute rotary motion corresponding to the joint;
the main operation end controls the instruments carried by the corresponding joints of the auxiliary operation end to execute bending motion;
The main operation end controls the auxiliary operation end to execute rotary motion corresponding to the instrument carried by the joint;
the main operation end controls the slave operation end to leave the boundary of the working space to the core area of the working space.
6. The method of claim 4, wherein, in the case where the first master-slave control mode is a joint angle mapping-based master-slave control mode and the second master-slave control mode is a cartesian pose mapping-based master-slave control mode, the first mode switching condition comprises at least one of:
the main operation end controls the slave operation end to execute stripping or separating operation corresponding to the instrument carried by the joint;
the main operation end controls the slave operation end to execute obstacle avoidance operation corresponding to a joint or a carried instrument;
the master operation end is moved while maintaining the end posture of the slave operation end unchanged.
7. The method of claim 1, wherein the controlling the at least a first partial joint to switch from the first master-slave control mode to perform a second master-slave control mode in the event that the first operating environment data satisfies a first mode switch condition, the method further comprises:
Invoking a trained mode switching judging model, processing the first working environment data, and outputting a judging result of whether the first working environment data meets a first mode switching condition or not;
the mode switching judging model is obtained by training based on working environment sample data, and the working environment sample data comprises one or more of system state sample data used for representing the current state of the flexible surgical robot, sensor sample data used for representing the pose and/or motion attribute of related components in the flexible surgical robot and image acquisition sample data used for representing the spatial state of the related components in the flexible surgical robot.
8. The method according to claim 4 or 7, wherein controlling the switching of the at least first partial joint from the first master-slave control mode to the second master-slave control mode in case the first working environment data satisfies a first mode switching condition comprises:
generating a switching control instruction under the condition that the first working environment data meets a first mode switching condition;
and controlling the at least a first partial joint to switch from the first master-slave control mode to execute a second master-slave control mode in response to a triggering operation for the switching control instruction.
9. The method according to claim 1, wherein the method further comprises:
acquiring current second working environment data of the flexible surgical robot in the process of executing a second master-slave control mode by at least a second part of joints in the master operation end and the slave operation end; the second working environment data is associated with the at least a second partial joint;
and controlling the at least second partial joint to switch from the second master-slave control mode to execute the first master-slave control mode under the condition that the second working environment data meets a second mode switching condition.
10. The method of claim 1, wherein in the second master-slave control mode is a joint angle mapping-based master-slave control mode, the method further comprises:
acquiring a main operation end structure and a slave operation end structure corresponding to at least the first part of joints;
acquiring a control demand amount for executing master-slave control;
determining a mapping relation between master control and slave control based on the master operation end structure, the slave operation end structure and the control demand;
acquiring a source control amount of the main operation end in the second master-slave control mode, and determining a target control amount of a corresponding component of the slave operation end based on the source control amount and the mapping relation;
And controlling the slave operation end to execute corresponding movement based on the target control quantity.
11. The method of claim 1, wherein in the second master-slave control mode is a cartesian pose mapping-based master-slave control mode, the method further comprises:
acquiring the current degree of freedom of the slave operation end; the degree of freedom is less than or equal to the maximum degree of freedom of the Cartesian space;
determining a mapping relation between master control and slave control;
acquiring a source control quantity of the tail end of the main operation end in the second master-slave control mode; the pose parameter quantity of the source control quantity is matched with the quantity of the degrees of freedom;
determining a target control amount of the slave operation end corresponding component based on the source control amount and the mapping relation;
and controlling the slave operation end to execute corresponding movement based on the target control quantity.
12. A master-slave control device of a flexible surgical robot, characterized in that the flexible surgical robot comprises a slave operating end and a master operating end for controlling the movement of the slave operating end; the device comprises:
the first acquisition module is used for acquiring current first working environment data of the flexible surgical robot in the process of executing a first master-slave control mode by at least a first part of joints in the master operation end and the slave operation end; the first working environment data is associated with the at least a first portion of the joints;
The first switching module is used for controlling the at least a first partial joint to switch from the first master-slave control mode to execute a second master-slave control mode under the condition that the first working environment data meets a first mode switching condition;
the first master-slave control mode and the second master-slave control mode are based on different master-slave control mapping modes.
13. A flexible surgical robot, comprising:
from the operating end;
the main operation end is used for controlling the movement of the auxiliary operation end;
a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the master-slave control method of the flexible surgical robot of any one of claims 1-11.
14. A computer readable storage medium, characterized in that at least one instruction or at least one program is stored in the storage medium, the at least one instruction or at least one program being loaded and executed by a processor to implement a master-slave control method of a flexible surgical robot according to any one of claims 1-11.
CN202311618916.9A 2023-11-29 2023-11-29 Master-slave control method and device of flexible surgical robot, robot and medium Pending CN117752424A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311618916.9A CN117752424A (en) 2023-11-29 2023-11-29 Master-slave control method and device of flexible surgical robot, robot and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311618916.9A CN117752424A (en) 2023-11-29 2023-11-29 Master-slave control method and device of flexible surgical robot, robot and medium

Publications (1)

Publication Number Publication Date
CN117752424A true CN117752424A (en) 2024-03-26

Family

ID=90313699

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311618916.9A Pending CN117752424A (en) 2023-11-29 2023-11-29 Master-slave control method and device of flexible surgical robot, robot and medium

Country Status (1)

Country Link
CN (1) CN117752424A (en)

Similar Documents

Publication Publication Date Title
JP6765021B2 (en) Graphical user interface for robotic surgery system
CN111315309B (en) System and method for controlling a robotic manipulator or associated tool
CN107049492B (en) Surgical robot system and method for displaying position of surgical instrument
CN107921623B (en) Robot system
US20210093402A1 (en) Systems and methods for onscreen menus in a teleoperational medical system
JP6541640B2 (en) Software configurable manipulator freedom
US8740882B2 (en) Medical robotic system and method of controlling the same
JP2019048108A (en) Inter-operative switching of tools in robotic surgical system
RU2741469C1 (en) Robotic surgical system
CN112618020B (en) Surgical robot and control method and control device thereof
EP3443928B1 (en) Surgical system
CN113876434A (en) Master-slave motion control method, robot system, device, and storage medium
WO2017098259A1 (en) Robotic system
JP2024020245A (en) Systems and methods for controlled grasping and energy delivery
CN117752424A (en) Master-slave control method and device of flexible surgical robot, robot and medium
EP3954319A1 (en) Surgical operation system and method for controlling surgical operation system
KR102225448B1 (en) Master device for manipulating active steering catheter and catheter system capability controlling bidirection of active steering catheter and master device
CN112888398B (en) Main robot and control method thereof
CN219846789U (en) surgical robot system
EP3829826B1 (en) Systems and methods for controlling a robotic manipulator or associated tool
WO2022172697A1 (en) Manipulator system and control device for manipulator
GB2545291A (en) Robotic system
CN116849818A (en) Surgical robot control method, surgical robot, and readable storage medium
CN117814927A (en) Data processing method and device of flexible surgical robot, robot and medium
CN117503355A (en) Flexible instrument end control method, device, system and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination