WO2024041386A1 - 机械臂的匹配方法、医生控制台和计算机可读存储介质 - Google Patents

机械臂的匹配方法、医生控制台和计算机可读存储介质 Download PDF

Info

Publication number
WO2024041386A1
WO2024041386A1 PCT/CN2023/112022 CN2023112022W WO2024041386A1 WO 2024041386 A1 WO2024041386 A1 WO 2024041386A1 CN 2023112022 W CN2023112022 W CN 2023112022W WO 2024041386 A1 WO2024041386 A1 WO 2024041386A1
Authority
WO
WIPO (PCT)
Prior art keywords
main operating
robotic arm
target
data
operating end
Prior art date
Application number
PCT/CN2023/112022
Other languages
English (en)
French (fr)
Inventor
苗燕楠
彭晓宁
孙晓军
马申宇
王家寅
Original Assignee
上海微创医疗机器人（集团）股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海微创医疗机器人（集团）股份有限公司 filed Critical 上海微创医疗机器人（集团）股份有限公司
Publication of WO2024041386A1 publication Critical patent/WO2024041386A1/zh

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/37—Leader-follower robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/35—Surgical robots for telesurgery
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/77—Manipulators with motion or force scaling
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J3/00—Manipulators of leader-follower type, i.e. both controlling unit and controlled unit perform corresponding spatial movements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/107—Visualisation of planned trajectories or target regions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2068—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis using pointers, e.g. pointers having reference marks for determining coordinates of body points
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/305—Details of wrist mechanisms at distal ends of robotic arms
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • This instruction manual belongs to the technical field of medical devices, and in particular relates to the matching method of the robotic arm, the doctor's console and the computer-readable storage medium.
  • This manual provides the matching method of the robotic arm, the doctor's console and the computer-readable storage medium, which can better avoid the influence of the main operating arm's following movement range on the user's operation accurately, and can accurately control the automatic operation of the main operating end. Adjusting to a posture state that matches the second slave robotic arm and is the same or similar to the configuration that the user has adapted to before allows the user to obtain a better operating experience.
  • This specification provides a matching method for a robotic arm, which includes: receiving a robotic arm switching instruction; wherein the robotic arm switching instruction is used to instruct the main operating end to switch from controlling the first slave robotic arm to controlling the second slave robotic arm. ;
  • the second slave robotic arm is mounted with a target surgical instrument; in response to the robotic arm switching command, obtain the current pose data of the target surgical instrument, the current joint position of the main operating end, and the last time the main operating end controlled the second slave
  • the historical configuration data of the master operating end of the robotic arm corresponding to the second slave robotic arm; based on the current pose data of the target surgical instrument, the current joint position of the master operating end and the historical configuration data, determine the target pose of the master operating end Data; based on the target pose data, control the movement of the master operating end to match the second slave robotic arm.
  • This specification also provides a doctor's console, which at least includes: a matching device between a main operating end and a robotic arm; wherein the main operating end is associated with a slave robotic arm of the patient's operating table; and a matching device for the robotic arm When executed, the relevant steps of the matching method of the robotic arm are implemented.
  • This specification also provides a computer-readable storage medium on which computer instructions are stored.
  • the instructions are executed by the processor, the following steps are implemented: receiving a robotic arm switching instruction; wherein the robotic arm switching instruction is used to instruct The main operating end switches from controlling the first slave robotic arm to controlling the second slave robotic arm; the second slave robotic arm is mounted with a target surgical instrument; in response to the robotic arm switching command, the current posture data of the target surgical instrument is obtained , the current joint position of the main operating end, and the historical configuration data of the main operating end corresponding to the second slave robotic arm when the main operating end last controlled the second slave robotic arm; according to the current posture data of the target surgical instrument, the main operating end's The current joint position and historical configuration data are used to determine the target pose data for the master operating end; based on the target pose data, the movement of the master operating end is controlled to match the second slave robotic arm.
  • the doctor's console receives When instructing the main operating end to switch from controlling the first slave robotic arm to controlling the second slave robotic arm, the current posture data of the target surgical instrument, the current joint position of the main operating end, and the last time can be obtained first.
  • the historical configuration data of the master operator corresponding to the second slave robotic arm when the master operator controls the second slave robotic arm then comprehensively utilize the above data to determine the target of the master operator by constructing and solving an objective function that takes into account multiple factors pose data; and then control the movement of the main operating terminal based on the above target pose data.
  • the same or similar posture status effectively improves the user's comfort and continuity during operation, allowing users to obtain a better operating experience.
  • the gripping characteristics of the end of the handle when the user uses the main operating terminal to control the slave robotic arm are determined; and then based on the Grip characteristics, and make targeted adjustments to the target constraints, so that the objective function can be solved based on the above adjusted target constraints, and target posture data that conforms to the user's personalized grasping habits can be obtained to further improve the user's comfort. degree, so that users can get a relatively better operating experience.
  • the target trajectory path is determined by using a preset planning algorithm to perform path planning based on the target pose data, so that the target trajectory path that meets the requirements can be determined more efficiently, and then the target trajectory path can be quickly and safely controlled based on the above target trajectory path.
  • the master operating end moves to match the newly switched slave robotic arm, which reduces the user's waiting time and further improves the user's operating experience.
  • Figure 1 is a schematic flowchart of a matching method for a robotic arm provided by an embodiment of this specification
  • Figure 2 is a schematic diagram of an embodiment of the structure of a doctor's console in a medical system in an example scenario
  • Figure 3 is a schematic diagram of an embodiment of the structure of the main operating end of the doctor's console in an example scenario
  • Figure 4 is a schematic diagram of another embodiment of the structural composition of the main operating end of the doctor's console in an example scenario
  • Figure 5 is a schematic diagram of an embodiment of the coordinate system of the patient's operating table in the medical system in an example scenario
  • Figure 6 is a graph of changes in joint speeds obtained when path planning is performed using the matching method of the robotic arm provided by the embodiment of this specification in an example scenario;
  • Figure 7 is a scene example showing a change curve of joint acceleration obtained when using the matching method of the robotic arm provided by the embodiment of this specification and using the preset planning algorithm for path planning;
  • Figure 8 is a schematic diagram of an embodiment in which the optimized planning algorithm is used to determine the free space when using the preset planning algorithm for path planning using the matching method of the robotic arm provided by the embodiment of this specification in a scene example;
  • Figure 9 is a schematic diagram of the determination process of using an optimized planning algorithm when using the preset planning algorithm to perform path planning using the matching method of the robotic arm provided by the embodiment of this specification in an example scenario.
  • an embodiment of this specification provides a matching method for a robotic arm. Among them, this method is specifically applied to the side of the doctor's console. When implemented, this method may include the following:
  • S101 Receive a robot arm switching instruction; wherein the robot arm switching instruction is used to instruct the main operating end to switch from controlling the first slave robot arm to controlling the second slave robot arm; the second slave robot arm has a target mounted on it.
  • Surgical Instruments
  • S102 In response to the robot arm switching command, obtain the current posture data of the target surgical instrument, the current joint position of the main operating terminal, and the main operation corresponding to the second slave robotic arm when the main operating terminal last operated the second slave robotic arm. historical configuration data of the terminal;
  • S103 Determine the target posture data for the main operating end based on the current posture data of the target surgical instrument, the current joint position of the main operating end, and the historical configuration data;
  • S104 Based on the target pose data, control the movement of the master operating end to match the second slave robotic arm.
  • the above matching method of the robotic arm can be specifically applied to the doctor's console side in the medical system. Doctor users can control the patient's operating table through the doctor's console to perform surgical operations on patients.
  • the above-mentioned doctor console may at least include a main operating terminal and a display.
  • the main operating end may further include a main operating arm (or main mechanical arm) and a handle end (or wrist module).
  • multiple position joints may also be arranged on the above-mentioned main operating arm, for example, position joint 1 , position joint 2 and position joint 3 .
  • changes in the position and posture of the above-mentioned position joints on the main operating arm will have a relatively obvious impact on the range of motion of the main operating end.
  • At least one posture joint for example, posture joint 4, is arranged at the end of the handle. Furthermore, one or more other posture joints, such as posture joint 5, posture joint 6 and posture joint 7, can be arranged at the end of the handle, so that the end of the handle can have a certain degree of redundant freedom.
  • the above-mentioned attitude joint 5 is specifically used to control the pitch attitude
  • the above-mentioned attitude joint 6 is used to control the yaw attitude
  • the above-mentioned attitude joint 7 is used to control the rotation attitude.
  • the change of the posture joint at the end of the handle has a relatively small impact on the range of motion of the main operating end.
  • the position of the main operating end is mainly determined by the joint position of the main operating arm position joint.
  • the posture of the main operating end is mainly determined by the posture data of the posture joint at the end of the handle, and is also affected by the position joint. For example, there is a corresponding conversion relationship between the joint positions of position joint 1 and position joint 3, and the posture data of posture joint 4.
  • the above-mentioned display is used to display image data during surgery to the doctor user.
  • the above-mentioned doctor's console may also include a foot panel.
  • the above-mentioned foot panel is used to trigger the generation of relevant function control instructions, such as robot arm switching instructions, electrical operation function instructions, etc.
  • the above-mentioned doctor control may also include an armrest to provide support for the doctor user and facilitate the doctor user's operation.
  • At least a plurality of slave robotic arms are arranged on the patient's operating table.
  • the above-mentioned patient operating table may also include structures such as a base and a top plate.
  • the above-mentioned slave robotic arm can be mounted with surgical instruments, endoscopes and other related equipment required for surgery.
  • surgical instruments are mounted on the robot arm 1
  • an endoscope is mounted on the robot arm 5 .
  • the slave robotic arm of the patient's operating table is associated with the master operating end of the doctor's console.
  • the doctor user can control the surgical instruments mounted on the slave robotic arm of the patient's operating table by operating the main operating terminal of the doctor's console to perform specific surgical operations on the patient.
  • corresponding sensors can also be arranged on the main operating end of the doctor's console and the slave robotic arm of the patient's operating table to collect the pose data of the main operating end and the slave robotic arm.
  • a camera can also be set up on the doctor's console to collect image data when the doctor user holds the master operating terminal and controls the slave robotic arm of the patient's operating table.
  • the doctor user controls the slave robotic arm 1 through the main operating terminal to use the mounted surgical instrument 1 to perform surgical operations on the patient.
  • the doctor user can initiate an instruction to the doctor's console through the foot panel to switch the main operating terminal from control to control from the robot arm 1 Arm switching command from arm 2.
  • the doctor's console can first use sensors (or detection equipment such as code disks) to collect the joint positions of the master operating arm's position and joints when the master operating end controls the slave robotic arm 1 at this time, as the structure of the slave robotic arm.
  • Type data can be recorded as configuration data 1 corresponding to slave robot arm 1); and save the configuration data in the database. Then respond to the robotic arm switching command, switch the slave robotic arm 1 controlled by the main operating end to the slave robotic arm 2; and restore the main operating end to a posture state matching the surgical instrument 2 mounted on the slave robotic arm 2 .
  • the doctor user controls the slave robot arm 2 to use the mounted surgical instrument 2 for a period of time through the main operating terminal, when the doctor user needs to use the surgical instrument 2 mounted on the slave robot arm 2 to perform a surgical operation on the patient again, the doctor user A robotic arm switching instruction for instructing the main operating terminal to switch from controlling the slave robotic arm 2 to controlling the slave robotic arm 1 can be initiated to the doctor's console through the foot panel.
  • the doctor's console can first use the sensor to collect the joint positions of the master operating arm's position joints when the master operating end controls the slave robotic arm 2, as the configuration data of the slave robotic arm (which can be recorded as the slave robotic arm's configuration data).
  • Configuration data 2 corresponding to the robot arm 2; and save the configuration data in the database.
  • the robot arm switching command apply the matching method of the robot arm provided in this manual, and collect the current posture data of the surgical instrument 1 mounted on the slave robot arm 1 through the sensor; at the same time, collect and obtain the current joint position of the main operating end , as well as the configuration data 1 of the main operating end when the last time the main operating end controlled the slave robotic arm 1 that was previously recorded; and then the current posture data of the above-mentioned surgical instruments, the current joint position and configuration data 1 of the main operating end can be comprehensively utilized, At the same time, taking into account various factors such as the following movement range of the main operating arm, the comfort and continuity of user operation, etc., the target position of the main operating end that can match the current slave robotic arm 1 and has a good user operating experience is determined.
  • the pose state of the master operating end can be controlled to match the current slave robotic arm 1 through path planning; at the same time, the impact of the excessive following movement range of the master operating arm's main operating arm on the user's operation can be minimized. This allows the user to have better comfort and continuity when holding the main operating terminal, effectively improving the user's operating experience.
  • the above-mentioned robot arm switching instruction can specifically be understood as a kind of instruction data used to instruct the main operating end to switch from controlling the first slave robot arm to controlling the second slave robot arm.
  • the user can initiate the above-mentioned robot arm switching command by triggering the foot panel of the doctor's console.
  • the above-mentioned robotic arm switching instructions can be initiated by issuing relevant voice instructions or triggering the switching button set on the doctor's console.
  • the above-mentioned first slave robotic arm can be specifically understood as the slave robotic arm currently controlled by the master operating end;
  • the above-mentioned second slave machine can specifically be understood as the slave robotic arm that is about to be switched to control by the master operating end;
  • the instrument can specifically be understood as a surgical instrument mounted on the second slave robotic arm.
  • the current posture data of the target surgical instrument can be collected and obtained through detection equipment such as sensors arranged on the second slave robot arm in response to the robot arm switching command; at the same time, through the detection equipment arranged on the main operating end
  • the sensor collects and obtains the current joint position of the master operating end; and, the database can also be queried to obtain the saved historical configuration data of the master operating end corresponding to the second slave robotic arm when the last time the master operating end operated the second slave robotic arm.
  • the current joint position of the main operating end can also be used as the configuration data corresponding to the first slave robotic arm when the main operating end controls the first slave robotic arm. , and save the configuration data in the database.
  • the above-mentioned database may store configuration data when the master operating terminal controls multiple slave robot arms respectively.
  • the above configuration data may also carry the corresponding robot arm identification of the slave robot arm.
  • the above configuration data may also carry time information about the storage time.
  • the above-mentioned acquisition of the current posture data of the target surgical instrument may include:
  • S1 Collect the current pose coordinates of the target surgical instrument in the second base coordinate system based on the second slave robotic arm as the initial pose data of the target surgical instrument;
  • S3 Based on the initial pose data of the target surgical instrument, the first transformation relationship and the second transformation relationship, calculate the current pose coordinates of the target surgical instrument in the endoscopic coordinate system as the current pose data of the target surgical instrument.
  • the pose coordinates (for example, Tob2_ot2) directly collected for the target surgical instrument through the sensor arranged on the second slave robot arm (for example, slave robot arm 2) are often based on the second slave robot arm.
  • the pose coordinates of the surgical instrument in the second base coordinate system of the robotic arm for example, the coordinate system ⁇ Ot2 ⁇ ).
  • This pose coordinate can be recorded as initial pose data and needs to be converted into pose coordinates in the endoscopic coordinate system before it can be used.
  • the pose coordinates (for example, Tob1_ot1) directly collected for the surgical instrument through the sensor arranged on the first slave robot arm (for example, slave robot arm 1) are often based on the first base coordinate of the first slave robot arm.
  • the pose coordinates of the surgical instrument under the system for example, coordinate system ⁇ Ot1 ⁇ ).
  • the coordinate system established based on the endoscopic view angle can be recorded as the endoscopic coordinate system, for example, the coordinate system ⁇ Oe ⁇ .
  • the slave robotic arm equipped with an endoscope on the patient's operating table can be recorded as the reference robotic arm, and the coordinate system established based on the reference robotic arm can be recorded as the reference base coordinate system, for example, the coordinate system ⁇ Obc ⁇ .
  • the first transformation relationship for example, the transformation matrix Tobc_ob2
  • the second transformation relationship for example, the transformation matrix Toe_obc
  • the above-mentioned first conversion relationship and second conversion relationship may be pre- Obtained by calibrating the position of the slave robotic arm of the patient's operating table.
  • the above-mentioned acquisition of the current joint position of the main operating terminal may include:
  • S1 Collect the current position coordinates of each joint of the main operating end in the main coordinate system based on the main operating arm as the initial joint position coordinates of the main operating end;
  • S3 Based on the initial joint position coordinates of the main operating terminal, the third transformation relationship and the fourth transformation relationship, calculate the current position coordinates of each joint of the main operating terminal in the display coordinate system as the current joint position of the main operating terminal.
  • the pose coordinates directly collected for the position joints through the sensors arranged on the main operating end are often based on the position coordinates in the main coordinate system of the main operating arm (for example, the coordinate system ⁇ Ob ⁇ ).
  • This position coordinate can be recorded as the initial joint position coordinate, which needs to be converted into position coordinates in the internal display coordinate system before it can be used.
  • the coordinate system established based on the display perspective of the doctor's console can be recorded as the display coordinate system, for example, the coordinate system ⁇ Od ⁇ .
  • the coordinate system based on the end of the handle can be denoted as the handle coordinate system, for example, coordinate system ⁇ Oh ⁇ .
  • the third transformation relationship of the main coordinate system relative to the handle coordinate system based on the handle end, and the fourth transformation relationship of the handle coordinate system relative to the display coordinate system can be obtained and processed.
  • the initial joint position coordinates are used to obtain the current position coordinates of each joint of the main operating terminal in the display coordinate system as the current joint position of the main operating terminal.
  • the above-mentioned third conversion relationship and the fourth conversion relationship may be obtained by calibrating the position of the main operating end of the doctor's console in advance.
  • the method may further include: determining the current joint position of the main operating end to be the same as when the main operating end controls the first slave robotic arm. 1. Slave the configuration data corresponding to the robot arm, and save the configuration data.
  • the posture state matching the current target surgical instrument, the range of motion of the main operating arm during the adjustment of the main operating end, and the user's comfort when grasping the end of the handle can be considered at the same time.
  • the target posture data for the main operating end is determined.
  • the target posture data for the main operating end is determined based on the current posture data of the target surgical instrument, the current joint position of the main operating end, and historical configuration data.
  • the following content may be included:
  • the reference pose data of the main operating end may include the following: determining the expected pose data of the main operating end based on the current pose data of the target surgical instrument and the master-slave mapping relationship; converting the joint positions and attitude data of the main operating end based on Relationship, using the expected pose data of the main operating end and the current joint position, determine the reference pose data of the main operating end that matches the current joint position.
  • the above-mentioned reference posture data at least includes the reference joint position of the main operating arm.
  • the above master-slave mapping relationship may be determined in advance during debugging of the patient's operating table and the doctor's console.
  • the current pose data of the target surgical instrument can be mapped to the associated master operating end, so that the matching expected pose data of the master operating end can be obtained.
  • the desired pose data of the main operator obtained based on the above method is often not the optimal solution.
  • the historical configuration data can also be used directly (for example, it can be recorded as: Jm_before), or the current posture data of the target surgical instrument, the current joint position of the main operating end and the historical configuration data can be used in combination to Determine the matching reference pose data of the main operator.
  • the desired posture data and the current joint position of the main operating terminal can be used to determine the reference posture data of the main operating terminal that matches the current joint position.
  • the expected position of the main operating arm in the expected pose data of the main operating end can be determined using historical configuration data and can be expressed as: Jm_before.
  • the current joint position of the main operating arm of the main operating end is expressed as: Jm_cur.
  • the reference joint position in the reference pose data of the main operating terminal is expressed as: Jm_exp.
  • position joint 1 and position joint 3 can be equivalent to posture joint 4, in order to make the joint configuration of the main operating end as close as possible to the joint configuration when the main operating end controlled the second slave robotic arm last time, and also make The following movement of the main operating end should be as small as possible.
  • Jm_exp(1) Jm_cur(1)
  • Jm_exp(2) Jm_cur(2)
  • Jm_exp(3) Jm_cur(3).
  • the reference pose data of the attitude joints of the main operating end is calculated.
  • the reference pose data of pose joint 4 can be calculated in the following way:
  • Jm_exp(4) Jm_before(4)-(Jm_cur(1)+Jm_cur(3)-Jm_before(1)-Jm_before(3)).
  • Jm_exp(1), Jm_exp(2), and Jm_exp(3) respectively represent the reference joint positions of position joint 1, position joint 2, and position joint 3 on the main operating arm of the main operating end.
  • the above-mentioned Jm_cur(1), Jm_cur (2) and Jm_cur(3) respectively represent the current joint positions of position joint 1, position joint 2 and position joint 3 on the main operating arm of the main operating end.
  • Jm_exp(4) represents the reference pose data of attitude joint 4 at the end of the handle.
  • Jm_before(1), Jm_before(3), Jm_before(4) represent position joint 1 and position joint 2 respectively.
  • the posture data of posture joint 4 can further be represented by three independent posture joints 5, posture joint 6 and posture joint 7.
  • the reference of posture joint 5, posture joint 6, and posture joint 7 can be determined through inverse derivation. pose data.
  • the complete reference pose data of the main operating terminal can be determined.
  • the main operating end may specifically include a main operating arm and a handle end; wherein the main operating arm is provided with multiple position joints;
  • the reference posture data may specifically include: the reference joint position of the main operating arm, and the reference end position and reference posture data of the handle end.
  • the position of the main operating arm can be controlled through the conversion of joint position and attitude data.
  • the joint position of the joint changes to 0.
  • the main operating arm of the main operating end does not move significantly, only the end position of the handle end and
  • the attitude data changes in a small range, so that the following movement range of the main operating terminal can be controlled to be relatively small.
  • the target posture data may at least include: the target end position of the handle end and target posture data, etc.
  • the above target pose data may also include: the target joint position of the position joint of the main operating arm. wait.
  • the reference joint position of the main operating arm can be recorded as: Jd
  • the reference position of the handle end can be recorded as: Pd
  • the reference attitude data of the handle end can be recorded as: Rd
  • the target end position at the end of the handle can be recorded as: Ptar
  • the target attitude data at the end of the handle can be recorded as: Rtar.
  • a larger range of motion of the joint position of the main operating arm can be converted into a smaller range of motion at the end of the handle, thereby effectively reducing the excessive range of movement of the main operating arm following the main operating arm.
  • the objective function regarding the target pose data is constructed based on the reference pose data of the main operating end, the current joint position of the main operating end, and historical configuration data.
  • the following content may be included:
  • a fourth objective function regarding configuration deviation is constructed.
  • the first objective function regarding the end position deviation can be constructed according to the following formula according to the target end position and the reference end position: Min_perr
  • the second target data about the terminal attitude deviation can be constructed according to the target attitude data and the reference attitude data according to the following formula: Min_rerr
  • represents the absolute value of position deviation
  • represents the absolute value of attitude deviation
  • the solution operation process can be optimized in a direction close to the reference pose data that matches the current pose of the target surgical instrument.
  • the third objective function regarding the joint following deviation can be constructed in the following manner based on the reference joint position and the current joint position of the main operating end: Minj_dis
  • represents the absolute value of the deviation distance of the joint position.
  • the solution operation process can be optimized in a direction such that the following motion range of the main operating end is smaller.
  • the fourth objective function regarding the configuration deviation can be constructed according to the following formula based on the reference joint position and the historical configuration data of the main operating end: Minj_dis
  • represents the absolute value of the deviation distance from the historical configuration.
  • the solution operation process can be optimized in a direction such that the configuration of the main operating end is close to the historical configuration.
  • the first objective function, the second objective function, the third objective function and the fourth objective function can be constructed and used in combination, comprehensively considering the current posture data of the target surgical instrument, the last time the main operation terminal manipulated the
  • the historical configuration data of the second slave robotic arm, as well as the current joint position of the main operating end and other factors, while making the posture data of the main operating end match the target surgical instrument take into account the following motion range of the main operating end, and
  • the configuration of the main operating end is determined to not only match the current target surgical instrument, but also cause the main operating end to follow a smaller range of motion, and can be restored to a historical configuration that is close to the previous operating second slave robotic arm that the user is already familiar with. target pose data.
  • the method may also include the following: Content: Based on the position threshold of the main operating arm's position joint, construct the objective constraints for the objective function;
  • the above-mentioned solution of the objective function to obtain the target pose data for the main operating end may specifically include: solving the objective function based on the target constraint conditions to obtain the target pose data for the main operating end.
  • i represents the joint number of the position joint of the main operating end
  • Ji_negative represents the lower limit of the position threshold of the position joint numbered i
  • Ji_positive represents the upper limit of the position threshold of the position joint numbered i.
  • the above-mentioned position threshold may be determined in advance by conducting a safety test on the main operating terminal.
  • the objective function can be solved by using the target constraints to determine the target pose data that is effective, safe and reliable.
  • the following content may also be included:
  • the target constraints are adjusted based on the gripping characteristics.
  • the above-mentioned determination of the gripping characteristics of the handle end when the user uses the main operating terminal may specifically include: obtaining historical image data when the user last used the main operating terminal to manipulate the second slave robotic arm; according to the history The image data determines the gripping characteristics of the end of the handle when the user uses the master operating terminal to control the second slave robotic arm.
  • the camera can capture the image data of the user using the main operating terminal to control the robotic arm in real time or regularly.
  • the camera can be controlled to take a photo of the user currently using the master operating terminal to control the slave robotic arm as image data, and the image data can be saved. in the database.
  • the shooting time of the image data can be stored in the database together with the number of the robotic arm of the slave robotic arm controlled by the user at the time of shooting.
  • the shooting time and robotic arm number of the image data in the database can be retrieved to find the historical image data when the user last used the master operating terminal to control the second slave robotic arm.
  • the historical image data can be processed by using a preset action recognition model to determine the gripping characteristics of the handle end when the user uses the master operating terminal to control the second slave robotic arm.
  • the above-mentioned hand gripping characteristics may specifically include at least one of the following: gripping manner, gripping position, gripping angle, etc.
  • the above-mentioned preset action recognition model can be specifically understood as a pre-trained neural network model that can identify the user's hand grip characteristics when grasping and using the main operating terminal based on image data.
  • historical image data can be used as input into a preset action recognition model, and the model can be run to obtain the corresponding model output as the recognition result; and then the grasping characteristics can be determined based on the recognition result.
  • the above-mentioned preset action recognition model can be trained in the following manner: obtain sample image data; and mark the grasping characteristics in the sample image data to obtain the labeled sample image data; build an initial classification model; use The annotated sample image data is used to train the initial classification model to obtain a preset action recognition model that meets the requirements.
  • the target constraint conditions can be adjusted according to the grasping characteristics; and then the adjusted target constraint conditions are used to solve the objective function, so that the calculation and solution process can be constrained by the user's hand error habits when using the main operating terminal. , so that the final solved target pose data is consistent with the user's hand grip characteristics when using the master operating terminal to control the second slave robotic arm, so that the solved target pose data can bring better results to the user. operating experience.
  • the above-mentioned solution to the objective function obtains the target pose data for the main operating end.
  • Specific implementation may include: using a preset solving algorithm to solve the objective function to obtain the target pose data for the main operating end; wherein , the preset solution algorithm includes at least one of the following: particle swarm algorithm, genetic algorithm, firefly algorithm and other swarm intelligence algorithms.
  • the above-mentioned control of the movement of the main operating terminal according to the target posture data may include:
  • the maximum joint speed and maximum joint acceleration of the main operating end can be obtained first; where, The above-mentioned maximum joint speed and maximum joint acceleration may be determined in advance by conducting a safety test on the main operating end. Then use the trajectory planning algorithm as the preset planning algorithm, and use the current joint position and target pose data of the main operating terminal to calculate the maximum joint speed and maximum joint acceleration to obtain the time-based joint trajectory as the target trajectory path. ;
  • the trajectory algorithm may specifically include at least one of the following: polynomial interpolation planning algorithm, trigonometric function interpolation planning algorithm, S-type interpolation planning algorithm, etc.
  • the target trajectory path that meets the requirements can be determined in the following way.
  • a 0 , a 1 , a 2 , a 3 , a 4 and a 5 are coefficient parameters to be determined respectively.
  • constraint conditions are set with the maximum joint speed and maximum joint acceleration.
  • constraints can be set as follows:
  • q s is the current joint position of the main operating end
  • q f is the position in the target pose data of the main operating end
  • maximum joint speed is the maximum joint acceleration.
  • the path function q(t) that meets the requirements can be obtained, and the target trajectory path that meets the requirements can be obtained.
  • the specific value of the end time can be flexibly set according to the specific situation and processing requirements.
  • the movement of the main operating end can be controlled according to the target trajectory path, so that the posture of the main operating end matches the target surgical instrument of the newly switched second slave robotic arm; at the same time, the user can also obtain a better operating experience.
  • obstacles encountered during the movement of the main operating terminal can also be considered, and an optimized planning algorithm can be used to solve the target trajectory with better effect, higher accuracy, and safer path.
  • a safe and effective movement space can be determined based on obstacles; wherein the obstacles include one or more of the following: handrails, fences, user's arms, etc. .
  • the obstacles can be determined first by obtaining and based on the image data when the current user uses the main operating terminal; and then the obstacle space can be determined based on the obstacles. Then, determine the effective motion space according to the following formula: Among them, A is the obstacle space, and Jm is the effective movement space of the main operating end.
  • constraint conditions are established based on the maximum joint speed and maximum joint acceleration.
  • the optimized planning algorithm is used as the preset planning algorithm to perform the solution operation to determine the target trajectory path that meets the requirements.
  • the optimized planning algorithm includes at least one of the following: RRT algorithm, artificial potential field method, BUG algorithm, graph search method, etc.
  • an optimized planning algorithm can be used to quickly solve the target trajectory path with relatively good effects.
  • the current joint position of the main operating end can be set as Pcur, and the target position of the joint is Ptar.
  • the free space (or effective motion space) can be set as: C ⁇ (Pcur,Ptar).
  • the obstacle space can also be set to S.
  • S2 Calculate the distance between all current nodes Pn on the growing tree and the random point Ptmp, and select the point closest to the random point Ptmp as Pnearest;
  • the doctor console when the doctor console receives a robotic arm switching instruction instructing the main operating end to switch from controlling the first slave robotic arm to controlling the second slave robotic arm, the doctor console can First obtain the current posture data of the target surgical instrument, the current joint position of the main operating end, and the historical configuration data of the main operating end corresponding to the second slave robotic arm when the main operating end last operated the second slave robotic arm; and then comprehensively utilize Based on the above data, an objective function that takes into account various factors is set and solved to determine the target pose data of the main operating end; and then based on the above target pose data, the movement of the main operating end is controlled.
  • the main operating arm of the end can accurately control the impact of the excessive movement range on the user's operation and automatically adjust the main operating arm to match the second slave robotic arm and be in the same or similar posture state as the configuration that the user has adapted to before. , effectively improving user comfort and continuity during operation, allowing users to obtain a better operating experience.
  • the gripping characteristics of the end of the handle when the user uses the main operating terminal to control the slave robotic arm are determined; and then based on the Grip characteristics, make targeted adjustments to the target constraints, so that the objective function can be solved based on the above adjusted target constraints, and target posture data that conforms to the user's personalized grasping habits can be obtained to further improve the user's comfort.
  • the target trajectory path is determined by using a preset planning algorithm to perform path planning based on the target pose data, so that the target trajectory path that meets the requirements can be determined more efficiently, and then the target trajectory path can be quickly and safely controlled based on the above target trajectory path.
  • the master operating terminal moves to match the newly switched slave robotic arm, reducing the user's waiting time and further improving the user's operating experience.
  • the present embodiment also provides a doctor console, which at least includes: a matching device between a main operating end and a robotic arm; wherein the main operating end is associated with a slave robotic arm of the patient's operating table, and the matching device of the robotic arm
  • the following steps are implemented: receiving a robot arm switching instruction; wherein the robot arm switching instruction is used to instruct the main operating end to switch from controlling the first slave robot arm to controlling the second slave robot arm; the second slave robot arm
  • the target surgical instrument is mounted; in response to the robot arm switching command, the current pose data of the target surgical instrument, the current joint position of the main operating end, and the last time the main operating end manipulated the second slave robotic arm were communicated with the second slave machine.
  • the historical configuration data of the main operating end corresponding to the arm based on the current pose data of the target surgical instrument, the current joint position of the main operating end and the historical configuration data, determine the target pose data for the main operating end; based on the target pose data, Control the movement of the master operating end to match that of the second slave manipulator.
  • a sensor may also be provided at the main operating end for collecting the joint position of the main operating end.
  • the main operating end may also include: a main operating arm and a handle end.
  • Embodiments of this specification also provide a computer device, including a processor and a memory for storing instructions executable by the processor.
  • the processor can perform the following steps according to the instructions: receive a robot arm switching instruction; wherein, The robotic arm switching instruction is used to instruct the main operating end to switch from controlling the first slave robotic arm to controlling the second slave robotic arm; the second slave robotic arm is mounted with a target surgical instrument; in response to the robotic arm switching instruction, obtain The current posture data of the target surgical instrument, the current joint position of the main operating end, and the historical configuration data of the main operating end corresponding to the second slave robotic arm when the main operating end last operated the second slave robotic arm; according to the target surgical instrument The current pose data, the current joint position of the main operating end and the historical configuration data are used to determine the target pose data for the master operating end; based on the target pose data, the movement of the main operating end is controlled to match the second slave robotic arm.
  • embodiments of this specification also provide another specific computer device, wherein the computer device includes a network communication port, a processor and a memory, and the above structures are connected through internal cables so that each Structures enable specific data interactions.
  • the network communication port can be specifically used to receive a robot arm switching instruction; wherein the robot arm switching instruction is used to instruct the main operating end to switch from controlling the first slave robot arm to controlling the second slave robot arm; so The second slave robotic arm carries the target surgical instrument.
  • the processor may be specifically configured to respond to the robot arm switching instruction and obtain the current pose data of the target surgical instrument, the current joint position of the main operating end, and the last time the main operating end manipulated the second slave robotic arm.
  • the history of the main operating end corresponding to the slave robot arm Historical configuration data determine the target pose data for the main operating end based on the current pose data of the target surgical instrument, the current joint position of the main operating end and historical configuration data; control the movement of the main operating end based on the target pose data, To match the second slave robotic arm.
  • the memory may specifically be used to store corresponding instruction programs.
  • the network communication port may be a virtual port bound to different communication protocols so that different data can be sent or received.
  • the network communication port may be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication.
  • the network communication port may also be a physical communication interface or a communication chip.
  • it can be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it can also be a Bluetooth chip.
  • Embodiments of this specification also provide a computer-readable storage medium based on the matching method of the above-mentioned robotic arm.
  • the computer-readable storage medium stores computer program instructions.
  • the following implementation is implemented: receiving the robotic arm. Switching instruction; wherein, the robot arm switching instruction is used to instruct the main operating end to switch from controlling the first slave robot arm to controlling the second slave robot arm; the second slave robot arm is mounted with a target surgical instrument; in response to the The above-mentioned robot arm switching command obtains the current posture data of the target surgical instrument, the current joint position of the main operating end, and the historical configuration of the main operating end corresponding to the second slave robotic arm when the main operating end last controlled the second slave robotic arm.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Robotics (AREA)
  • Molecular Biology (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)

Abstract

一种机械臂的匹配方法、医生控制台和计算机可读存储介质。基于上述方法，医生控制台在接收到指示将主操作端由操控第一从机械臂切换为操控第二从机械臂的机械臂切换指令时，可以先获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；再综合利用上述数据，确定出主操作端的目标位姿数据；再根据目标位姿数据，控制主操作端运动，以与新切换的第二从机械臂相匹配。从而可以避免主操作端的主操作臂跟随运动范围过大对用户操作的影响，精准地控制主操作端自动调整到与第二从机械臂相匹配，且与之前用户已适应的构型相同或相近的位姿状态。

Description

机械臂的匹配方法、医生控制台和计算机可读存储介质
本申请要求2022年08月24日递交的申请号为202211020147.8、发明名称为“机械臂的匹配方法、医生控制台和计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本说明书属于医疗器械技术领域，尤其涉及机械臂的匹配方法、医生控制台和计算机可读存储介质。
背景技术
通常，在医生用户使用医生控制台的主操作端操控患者手术台的从机械臂进行手术操作的过程中，当需要切换所操控的从机械臂时，要先调整主操作端的位姿，以与新切换的从机械臂相匹配。但是，基于现有方法，在调整主操作端的位姿的过程中往往存在主操作端的主操作臂跟随运动范围过大、用户操作体验差等问题。
针对上述问题，目前尚未提出有效的解决方案。
发明内容
本说明书提供了机械臂的匹配方法、医生控制台和计算机可读存储介质，可以较好地避免主操作端的主操作臂跟随运动范围过大对用户操作的影响，能够精准地控制主操作端自动调整到与第二从机械臂相匹配，且与之前用户已经适应的构型相同或相近的位姿状态，使用户可以获得较好的操作体验。
本说明书提供了一种机械臂的匹配方法，包括：接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
本说明书提供了还提供了一种医生控制台，至少包括：主操作端和机械臂的匹配装置；其中，所述主操作端与患者手术台的从机械臂关联；所述机械臂的匹配装置执行时实现所述机械臂的匹配方法的相关步骤。
本说明书还提供了一种计算机可读存储介质，其上存储有计算机指令，所述指令被处理器执行时实现以下步骤：接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
基于本说明书提供的机械臂的匹配方法、医生控制台和计算机可读存储介质，医生控制台在接收 到指示将主操作端由操控第一从机械臂切换为操控第二从机械臂的机械臂切换指令时，可以先获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；再综合利用上述数据，通过构建并求解兼顾多方面因素的目标函数，确定出主操作端的目标位姿数据；再根据上述目标位姿数据，控制主操作端运动。从而可以较好地避免主操作端的主操作臂跟随运动范围过大对用户操作的影响，能够精准地控制主操作端自动调整到与第二从机械臂相匹配，且与之前用户已经适应的构型相同或相近的位姿状态，有效地改善了用户操作时舒适度和连续性，使用户可以获得较好的操作体验。
进一步，还通过采集并根据用户使用医生控制台的主操作端操控从机械臂时的历史影像数据，确定出用户使用主操作端操控该从机械臂时针对手柄末端的抓握特征；再根据该抓握特征，对目标约束条件进行针对性的调整，以便后续可以基于上述调整后的目标约束条件求解目标函数，得到符合该用户个性化抓握习惯的目标位姿数据，以进一步提高用户的舒适度，使用户可以获得相对更好的操作体验。
此外，还通过利用预设的规划算法基于目标位姿数据进行路径规划来确定目标轨迹路径，从而可以较为高效地确定出符合要求的目标轨迹路径，进而可以根据上述目标轨迹路径快速、安全地控制主操作端运动，以与新切换的从机械臂匹配，减少了用户的等待时长，也能进一步提高了用户的操作体验。
附图说明
为了更清楚地说明本说明书实施例，下面将对实施例中所需要使用的附图作简单地介绍，下面描述中的附图仅仅是本说明书中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本说明书的一个实施例提供的机械臂的匹配方法的流程示意图；
图2是在一个场景示例中，医疗系统中的医生控制台的结构组成一种实施例的示意图；
图3是在一个场景示例中，医生控制台的主操作端的结构组成的一种实施例的示意图；
图4是在一个场景示例中，医生控制台的主操作端的结构组成的另一种实施例的示意图；
图5是在一个场景示例中，医疗系统中的患者手术台的坐标系一种实施例的示意图；
图6是在一个场景示例中，应用本说明书实施例提供的机械臂的匹配方法进行路径规划时得到的关节速度的变化曲线图；
图7是在一个场景示例中，应用本说明书实施例提供的机械臂的匹配方法采用预设的规划算法进行路径规划时得到的关节加速度的变化曲线图；
图8是在一个场景示例中，应用本说明书实施例提供的机械臂的匹配方法采用预设的规划算法进行路径规划时采用优化的规划算法确定自由空间的实施例示意图；
图9是在一个场景示例中，应用本说明书实施例提供的机械臂的匹配方法采用预设的规划算法进行路径规划时采用优化的规划算法的确定过程示意图。
具体实施方式
为了使本技术领域的人员更好地理解本说明书中的技术方案，下面将结合本说明书实施例中的附图，对本说明书实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本说明书一部分实施例，而不是全部的实施例。基于本说明书中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都应当属于本说明书保护的范围。
参阅图1所示，本说明书实施例提供了一种机械臂的匹配方法。其中，该方法具体应用于医生控制台一侧。具体实施时，该方法可以包括以下内容：
S101：接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；
S102：响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；
S103：根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；
S104：根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
在一些实施例中，上述机械臂的匹配方法具体可以应用于医疗系统中的医生控制台一侧。医生用户可以通过医生控制台操控患者手术台为患者进行手术操作。
具体的，参阅图2所示，上述医生控制台至少可以包括主操作端和显示器。其中，参阅图3所示，上述主操作端进一步可以包括主操作臂(或称主机械臂)和手柄末端(或称手腕模块)。
进一步，参阅图4所示，在上述主操作臂上还可以布设有多个位置关节，例如，位置关节1、位置关节2和位置关节3。其中，上述主操作臂上的位置关节的位姿变化会对主操作端的运动范围幅度产生相对较为明显的影响。
在上述手柄末端至少布设有一个姿态关节，例如，姿态关节4。进一步，在上述手柄末端还可以布设有一个或多个其他的姿态关节，例如，姿态关节5，姿态关节6和姿态关节7，以使手柄末端可以具有一定冗余自由度。其中，上述姿态关节5具体用于控制俯仰姿态，上述姿态关节6用于控制偏摆姿态，上述姿态关节7用于控制自转姿态。区别于主操作臂上的位置关节，上述手柄末端的姿态关节的位姿变化对主操作端的运动范围幅度影响相对较小。
具体的，主操作端的位置主要由主操作臂的位置关节的关节位置决定。主操作端的姿态主要由手柄末端的姿态关节的姿态数据决定，同时也会受到位置关节的影响。例如，位置关节1、位置关节3的关节位置，与姿态关节4的姿态数据之间存在相应的转换关系。
上述显示器用于向医生用户展示手术过程中的影像数据。
此外，参阅图2所示，上述医生控制台还可以包含有脚踏面板。其中，上述脚踏面板用于触发生成相关的功能控制指令，例如，机械臂切换指令、电操作功能指令等。上述医生控制还可以包含有扶手，用于为医生用户提供支撑，方便医生用户的操作。
具体的，参阅图5所示，在患者手术台上至少布设有多个从机械臂。例如，从机械臂1、从机械臂2、从机械臂3、从机械臂4、从机械臂5。此外，上述患者手术台还可以包含有基座、顶盘等结构。
进一步，在上述从机械臂上可以挂载有手术时所需要使用到的手术器械、内窥镜等相关设备。例如，从机械臂1挂载有手术器械，从机械臂5挂载有内窥镜。
患者手术台的从机械臂与医生控制台的主操作端关联。相应的，医生用户可以通过操作医生控制台的主操作端发来操控患者手术台的从机械臂控制所挂载的手术器械等对患者进行具体的手术操作。
具体实施前，还可以分别在医生控制台的主操作端，以及患者手术台的从机械臂布设相应的传感器，用于采集主操作端和从机械臂的位姿数据。
进一步，还可以在医生控制台设置摄像头，用于采集医生用户手握主操作端操控患者手术台的从机械臂时的影像数据。
基于上述医疗系统，一开始，医生用户通过主操作端操控从机械臂1使用所挂载的手术器械1对患者进行手术操作。当需要切换使用从机械臂2所挂载的手术器械2对患者进行手术操作时，医生用户可以通过脚踏面板向医生控制台发起用于指示将主操作端由操控从机械臂1切换为操控从机械臂2的机械臂切换指令。
这时，医生控制台可以先通过传感器(或者码盘等检测设备)采集此时主操作端操控从机械臂1时主操作端时主操作臂的位置关节的关节位置，作为从机械臂的构型数据(可以记为与从机械臂1对应的构型数据1)；并将该构型数据保存于数据库中。再响应机械臂切换指令，将主操作端所操控的从机械臂1切换成从机械臂2；并将主操作端恢复成与从机械臂2所挂载的手术器械2相匹配的位姿状态。
在医生用户通过主操作端操控从机械臂2使用所挂载的手术器械2一段时间后，当医生用户需要再次使用从机械臂2所挂载的手术器械2对患者进行手术操作时，医生用户可以通过脚踏面板向医生控制台发起用于指示将主操作端由操控从机械臂2切换为操控从机械臂1的机械臂切换指令。
这时，医生控制台可以先通过传感器采集此时主操作端操控从机械臂2时主操作端时主操作臂的位置关节的关节位置，作为从机械臂的构型数据(可以记为与从机械臂2对应的构型数据2)；并将该构型数据保存于数据库中。再响应该机械臂切换指令，应用本说明书提供的机械臂的匹配方法，通过传感器采集挂载于从机械臂1的手术器械1的当前位姿数据；同时，采集并获取主操作端的当前关节位置，以及之前记录保存的上一次主操作端操控从机械臂1时主操作端的构型数据1；进而可以综合利用上述手术器械的当前位姿数据、主操作端的当前关节位置和构型数据1，同时兼顾主操作臂的跟随运动范围，用户操作时的舒适度和连续性等多方面因素，确定出能够与当前的从机械臂1相匹配的，且用户操作体验较好的主操作端的目标位姿数据。进而可以根据该目标位姿数据，通过路径规划，控制主操作端的位姿状态与当前的从机械臂1相匹配；同时尽量减少主操作端的主操作臂跟随运动范围过大对用户操作的影响，使用户手握主操作端操作时具有较好的舒适性和连续性，有效地提高了用户的操作体验。
在一些实施例中，上述机械臂切换指令具体可以理解为一种用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂的指令数据。
具体实施时，用户可以通过触发医生控制台的脚踏面板发起上述机械臂切换指令。此外，用户还 可以通过发出相关的语音指令，或者触发设置于医生控制台的切换按钮来发起上述机械臂切换指令等。
在本实施例中，上述第一从机械臂具体可以理解为主操作端当前所操控的从机械臂；上述第二从机械具体可以理解为主操作端即将切换操控的从机械臂；上述目标手术器械具体可以理解为挂载于第二从机械臂上的手术器械。
在一些实施例中，具体实施时，可以响应机械臂切换指令，通过布设于第二从机械臂上的传感器等检测设备采集获取目标手术器械的当前位姿数据；同时，通过布设于主操作端的传感器采集获取主操作端的当前关节位置；并且，还可以通过查询数据库，获取所保存的上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据。
此外，在通过布设于主操作端的传感器采集获取主操作端的当前关节位置之后，还可以主操作端的当前关节位置作为主操作端操控第一从机械臂时与第一从机械臂对应的构型数据，并将该构型数据保存于数据库中。
其中，上述数据库内可以存储有主操作端操控分别操控多个从机械臂时的构型数据。具体的，上述构型数据还可以携带有所对应的从机械臂的机械臂标识。
此外，在上述数据库内，还可以存储有对应同一个从机械臂的多个构型数据。其中，上述构型数据中还可以携带有关于保存时间的时间信息。
在一些实施例中，上述获取目标手术器械的当前位姿数据，具体实施时，可以包括：
S1：采集目标手术器械当前在基于第二从机械臂的第二基坐标系下的位姿坐标，作为目标手术器械的初始位姿数据；
S2：获取第二基坐标系相对于基于参考机械臂的参考基坐标系的第一转换关系，以及参考基坐标系相对于内窥镜坐标系的第二转换关系；其中，所述参考机械臂为挂载有内窥镜的从机械臂；
S3：根据目标手术器械的初始位姿数据、第一转换关系和第二转换关系，计算出目标手术器械当前在内窥镜坐标系下的位姿坐标，作为目标手术器械的当前位姿数据。
具体的，参阅图5所示，通过布设于第二从机械臂(例如，从机械臂2)的传感器针对目标手术器械所直接采集到的位姿坐标(例如，Tob2_ot2)往往是基于第二从机械臂的第二基坐标系(例如，坐标系{Ot2})下的手术器械的位姿坐标。这种位姿坐标可以记为初始位姿数据，需要转换为内窥镜坐标系下的位姿坐标才能使用。类似的，通过布设于第一从机械臂(例如，从机械臂1)的传感器针对手术器械所直接采集到的位姿坐标(例如，Tob1_ot1)往往是基于第一从机械臂的第一基坐标系(例如，坐标系{Ot1})下的手术器械的位姿坐标。
其中，参阅图5所示，基于内窥镜视角所建立的坐标系可以记为内窥镜坐标系，例如，坐标系{Oe}。患者手术台中布设有内窥镜的从机械臂可以记为参考机械臂，基于参考机械臂所建立的坐标系可以记为参考基坐标系，例如，坐标系{Obc}。
在得到目标手术器械的初始位姿数据后，可以获取第一转换关系，例如，变换矩阵Tobc_ob2，以及第二转换关系，例如，变换矩阵Toe_obc。其中，上述第一转换关系和第二转换关系可以是预先 对患者手术台的从机械臂进行位置标定得到的。
再按照以下算式，计算出目标手术器械在内窥镜坐标系下的位姿坐标，作为该目标手术器械的当前位姿数据：Toe_ot2＝Toe_obc*Tobc_ob2*Tob2_ot2。
在一些实施例中，上述获取主操作端的当前关节位置，具体实施时，可以包括：
S1：采集主操作端的各个关节当前在基于主操作臂的主坐标系下的位置坐标，作为主操作端的初始关节位置坐标；
S2：获取主坐标系相对于基于手柄末端的手柄坐标系的第三转换关系，以及手柄坐标系相对于显示器坐标系的第四转换关系；
S3：根据主操作端的初始关节位置坐标、第三转换关系和第四转换关系，计算出主操作端的各个关节当前在显示器坐标系下的位置坐标，作为主操作端的当前关节位置。
具体的，通过布设于主操作端的传感器针对位置关节所直接采集到的位姿坐标往往是基于主操作臂的主坐标系(例如，坐标系{Ob})下的位置坐标。这种位置坐标可以记为初始关节位置坐标，需要转换为内显示器坐标系下的位置坐标才能使用。
其中，基于医生控制台的显示器视角所建立的坐标系可以记为显示器坐标系，例如，坐标系{Od}。基于手柄末端的坐标系可以记为手柄坐标系，例如，坐标系{Oh}。
类似于确定目标手术器械的当前位姿数据，可以获取并使用主坐标系相对于基于手柄末端的手柄坐标系的第三转换关系，以及手柄坐标系相对于显示器坐标系的第四转换关系来处理初始关节位置坐标，得到主操作端的各个关节当前在显示器坐标系下的位置坐标，作为主操作端的当前关节位置。其中，上述第三转换关系和第四转换关系可以是预先对医生控制台的主操作端进行位置标定得到的。
在一些实施例中，在获取主操作端的当前关节位置之后，所述方法具体实施时，还可以包括：将所述主操作端的当前关节位置确定为主操作端操控第一从机械臂时与第一从机械臂对应的构型数据，并保存该构型数据。
在一些实施例中，具体实施时，可以同时考虑与当前的目标手术器械相匹配的位姿状态，以及调整主操作端过程中主操作臂的运动范围，以及用户抓握手柄末端时的舒适度等多方面的因素，综合利用目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据。
在一些实施例中，上述根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据，具体实施时，可以包括以下内容：
根据目标手术器械的当前位姿数据、主操作端的当前关节位置，确定出相匹配的主操作端的参考位姿数据；
根据主操作端的参考位姿数据、主操作端的当前关节位置和历史构型数据，构建关于目标位姿数据的目标函数；
求解目标函数，得到针对主操作端的目标位姿数据。
具体实施时，上述根据目标手术器械的当前位姿数据、主操作端的当前关节位置，确定出相匹配 的主操作端的参考位姿数据，可以包括以下内容：根据目标手术器械的当前位姿数据和主从映射关系，确定出主操作端的期望位姿数据；根据主操作端的关节位置与姿态数据的转换关系，利用主操作端的期望位姿数据和当前关节位置，确定出与当前关节位置相匹配的主操作端的参考位姿数据。其中，上述参考位姿数据至少包括主操作臂的参考关节位置。
其中，上述主从映射关系具体可以是预先在对患者手术台和医生控制台进行调试时所确定的。
基于上述主从映射关系，可以将目标手术器械的当前位姿数据映射到所关联的主操作端，从而可以得到相匹配的主操作端的期望位姿数据。
但是，基于上述方式得到的主操作端的期望位姿数据往往不是最优解。直接基于上述期望位姿数据调整主操作端时，很容易出现主操作端的主操作臂运动范围过大，进而影响用户的操作体验。
正是注意到上述问题，在确定出主操作端的期望位姿数据后，还可以考虑缩小主操作端的主操作臂的运动范围，有针对性地将主操作臂的位置关节的位置变化转换为手柄末端的姿态变换，减少对用户的操作体验的影响。
此外，在一些情况下，也可以直接使用历史构型数据(例如，可以记为：Jm_before)，或者组合使用目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，来确定相匹配的主操作端的参考位姿数据。
具体的，可以根据主操作端的关节位置与姿态数据的转换关系，利用主操作端的期望位姿数据和当前关节位置，确定出与当前关节位置相匹配的主操作端的参考位姿数据。
例如，参阅图4所示。主操作端的期望位姿数据中的主操作臂的期望位置可以使用历史构型数据确定，可以表示为：Jm_before。主操作端的主操作臂的当前关节位置表示为：Jm_cur。主操作端的参考位姿数据中的参考关节位置表示为：Jm_exp。
由于位置关节1和位置关节3的运动可通过姿态关节4等效，为使得主操作端的关节构型尽量与上一次主操作端操控第二从机械臂时的关节构型相近，且还要使得主操作端的跟随运动尽量少，考虑可以尽量将主操作端的主操作臂的位置关节的参考位置设置为各个位置关节的当前关节位置，以减小主操作端的运动幅度。
具体的，例如，按照以下方式分别设置主操作臂上的位置关节1、位置关节2、位置关节3的参考关节位置：Jm_exp(1)＝Jm_cur(1)；Jm_exp(2)＝Jm_cur(2)；Jm_exp(3)＝Jm_cur(3)。
再根据主操作端的关节位置与姿态数据的转换关系，利用主操作端的期望位姿数据和主操作臂各个位置关节的参考位置，计算出主操作端的姿态关节的参考位姿数据。
例如，可以按照以下方式，计算出姿态关节4的参考位姿数据：
Jm_exp(4)＝Jm_before(4)-(Jm_cur(1)+Jm_cur(3)-Jm_before(1)-Jm_before(3))。
其中，上述Jm_exp(1)、Jm_exp(2)、Jm_exp(3)分别表示主操作端的主操作臂上的位置关节1、位置关节2、位置关节3的参考关节位置，上述Jm_cur(1)、Jm_cur(2)、Jm_cur(3)分别表示主操作端的主操作臂上的位置关节1、位置关节2、位置关节3的当前关节位置，Jm_exp(4)表示手柄末端的姿态关节4的参考位姿数据，Jm_before(1)、Jm_before(3)、Jm_before(4)分别表示位置关节1、位置关节2 和姿态关节4的期望姿态数据。
而姿态关节4的位姿数据进一步又可以通过3个分别独立的姿态关节5、姿态关节6和姿态关节7进行表示。相应的，可以根据所确定的位置关节1、位置关节2、位置关节3，和姿态冠杰的参考位姿数据，通过逆解推导，确定出姿态关节5、姿态关节6和姿态关节7的参考位姿数据。从而可以确定出完整的主操作端的参考位姿数据。
在一些实施例中，主操作端具体可以包括主操作臂和手柄末端；其中，所述主操作臂设置有多个位置关节；
相应的，所述参考位姿数据具体可以包括：主操作臂的参考关节位置，以及手柄末端的参考末端位置和参考姿态数据。
在一些情况下，可以通过关节位置和姿态数据的转换，控制主操作臂的位置关节的关节位置变化为0，这时主操作端的主操作臂相对没有发生明显运动，只有手柄末端的末端位置和姿态数据发生小范围的变化，从而可以将主操作端跟随运动范围控制得相对较小。
相应的，所目标位姿数据至少可以包括：手柄末端的目标末端位置和目标姿态数据等。
当然，在另一些情况下，可能无法将主操作端的主操作臂的位置关节的关节位置变化控制到0，这时，上述目标位姿数据还可以包括：主操作臂的位置关节的目标关节位置等。
其中，主操作臂的参考关节位置可以记为：Jd，手柄末端的参考位置可以记为：Pd，手柄末端的参考姿态数据可以记为：Rd。手柄末端的目标末端位置可以记为：Ptar，手柄末端的目标姿态数据可以记为：Rtar。
并且，主操作臂的参考关节位置，与手柄末端的参考位置、手柄末端的参考姿态数据之间所具有的主操作端的关节位置与姿态数据的转换关系可以表示为以下形式：Pd＝Fpos(Jd),Rd＝Frot(Jd)。
基于上述实施例，可以有针对性地将主操作臂的关节位置的较大幅度范围运动，转换成手柄末端较小幅度范围的运动，从而可以有效地减少主操作端的主操作臂跟随运动范围过大对用户操作的影响。
在一些实施例中，上述根据主操作端的参考位姿数据、主操作端的当前关节位置和历史构型数据，构建关于目标位姿数据的目标函数，具体实施时，可以包括以下内容：
根据目标末端位置和参考末端位置构建关于末端位置偏差的第一目标函数；根据目标姿态数据和参考姿态数据构建关于末端姿态偏差的第二目标数据；
根据参考关节位置和主操作端的当前关节位置，构建关于关节跟随偏差的第三目标函数；
根据参考关节位置和主操作端的历史构型数据，构建关于构型偏差的第四目标函数。
具体实施时，可以根据目标末端位置和参考末端位置，按照以下算式，构建关于末端位置偏差的第一目标函数：Min_perr||Ptar-Pd||；
可以根据目标姿态数据和参考姿态数据，按照以下算式，构建关于末端姿态偏差的第二目标数据：Min_rerr||Rtar-Rd||。
其中，perr||Ptar-Pd||表示位置偏差的绝对值，rerr||Rtar-Rd||表示姿态偏差的绝对值。
基于上述第一目标函数、第二目标函数，可以使得求解运算过程向靠近与当目标手术器械的当前位姿相匹配的参考位姿数据的方向优化。
具体实施时，可以根据参考关节位置和主操作端的当前关节位置，按照以下方式，构建关于关节跟随偏差的第三目标函数：Minj_dis||Jd-Jm_cur||。
其中，j_dis||Jd-Jm_cur||表示关节位置的偏差距离的绝对值。
基于上述第三目标函数，可以使得求解运算过程向使得主操作端的跟随运动范围较小的方向优化。
具体实施时，可以根据参考关节位置和主操作端的历史构型数据，按照以下算式，构建关于构型偏差的第四目标函数：Minj_dis||Jd-Jm_before||。
其中，j_dis||Jd-Jm_before||表示与历史构型的偏差距离的绝对值。
基于上述第四目标函数，可以使得求解运算过程向使得主操作端的构型靠近历史构型的方向优化。
基于上述实施例，可以通过构建和组合使用上述第一目标函数、第二目标函数、第三目标函数和第四目标函数，综合考虑目标手术器械的当前位姿数据、主操作端上一次操控第二从机械臂时的历史构型数据，以及主操作端的当前关节位置等多方面因素，在使得主操作端的位姿数据与目标手术器械相匹配的同时，兼顾考虑主操作端的跟随运动范围，以及主操作端的构型，确定出既与当前的目标手术器械相匹配，又导致主操作端的跟随运动范围较小，且能够恢复到接近用户已经熟悉的之前操控第二从机械臂是的历史构型的目标位姿数据。
在一些实施例中，在根据主操作端的参考位姿数据、主操作端的当前关节位置和历史构型数据，构建关于目标位姿数据的目标函数之后，所述方法具体实施时，还可以包括以下内容：根据主操作臂的位置关节的位置阈值，构建针对目标函数的目标约束条件；
相应的，上述求解目标函数，得到针对主操作端的目标位姿数据，具体可以包括：基于所述目标约束条件，求解所述目标函数，得到针对主操作端的目标位姿数据。
具体实施时，可以根据主操作臂的位置关节的位置阈值，按照以下算式，构建针对目标函数的目标约束条件：Ji_negative<＝Jd<＝Ji_positive，i＝1,2,……n。
其中，i表示主操作端的位置关节的关节编号，Ji_negative表示编号为i的位置关节的位置阈值中的下限值，Ji_positive表示编号为i的位置关节的位置阈值中的上限值。上述位置阈值具体可以是预先通过对主操作端进行安全性测试所确定的。
基于上述实施例，可以通过利用目标约束条件求解目标函数，确定出效果较好且安全可靠的目标位姿数据。
在一些实施例中，所述方法具体实施时，还可以包括以下内容：
确定用户使用主操作端操控第二从机械臂时针对手柄末端的抓握特征；其中，所述抓握特征包括抓握位置和/或抓握方式；
根据所述抓握特征，调整所述目标约束条件。
在一些实施例中，上述确定用户使用主操作端时针对手柄末端的抓握特征，具体可以包括：获取用户上一次使用主操作端操控第二从机械臂时的历史影像数据；根据所述历史影像数据，确定出用户使用主操作端操控第二从机械臂时针对手柄末端的抓握特征。
具体实施时，可以通过摄像头实时或定时地拍摄用户使用主操作端操控机械臂的影像数据。
例如，可以在检测到用户切换操控从机械臂指定时间段(例如，10秒)时，控制摄像头拍摄用户当前使用主操作端控制该从机械臂时的照片作为影像数据，并将该影像数据保存于数据库中。具体保存时，可以将该影像数据的拍摄时间，以及拍摄时用户所操控的从机械臂的机械臂编号一起存入数据库中。
相应的，具体实施时，可以通过检索数据库内影像数据的拍摄时间和机械臂编号，以找到用户上一次使用主操作端操控第二从机械臂时的历史影像数据。
具体实施时，可以通过利用预设的动作识别模型处理所述历史影像数据，确定出用户使用主操作端操控第二从机械臂时针对手柄末端的抓握特征。
其中，上述手握特征具体可以包括以下至少之一：抓握方式、抓握位置、抓握角度等。
上述预设的动作识别模型具体可以理解为预先训练得到的能够基于影像数据识别出用户抓取和使用主操作端时的手握特征的神经网络模型。
具体实施时，可以将历史影像数据作为输入，输入至预设的动作识别模型中，并运行该模型，得到对应的模型输出，作为识别结果；再根据识别结果确定出抓握特征。
具体实施前，可以按照以下方式训练得到上述预设的动作识别模型：获取样本影像数据；并标注出样本影像数据中的抓握特征，得到标注后的样本影像数据；构建初始的分类模型；利用标注后的样本影像数据训练初始的分类模型，得到符合要求的预设的动作识别模型。
具体实施时，可以根据抓握特征，调整所述目标约束条件；再利用上述调整后的目标约束条件，求解目标函数，可以使得运算求解过程受用户的使用主操作端时的手误习惯的约束，从而可以使得最终求解得到的目标位姿数据是符合该用户之前使用主操作端操控第二从机械臂时的手握特征的，使得所求解得到的目标位姿数据可以给用户带来更好的操作体验。
在一些实施例中，上述求解目标函数，得到针对主操作端的目标位姿数据，具体实施时，可以包括：利用预设的求解算法，求解目标函数，得到针对主操作端的目标位姿数据；其中，所述预设的求解算法包括以下至少之一：粒子群算法、遗传算法、萤火虫算法等群体智能算法。
具体实施时，除了使用上述所列举的群体智能算法外，还可以利用基于神经网络的算法来求解上述目标函数，以快速得到针对主操作端的目标位姿数据。
在一些实施例中，上述根据目标位姿数据，控制主操作端运动，具体实施时，可以包括：
利用预设的规划算法，根据主操作端的当前关节位置和目标位姿数据，通过路径规划，确定出符合要求的目标轨迹路径；
根据所述目标轨迹路径，控制主操作端运动。
在一些实施例中，具体实施时，可以先获取主操作端的最大关节速度和最大关节加速度；其中， 上述最大关节速度和最大关节加速度可以是预先通过对主操作端进行安全性测试所确定的。再采用轨迹规划算法作为预设的规划算法，并利用主操作端的当前关节位置、目标位姿数据，以最大关节速度和最大关节加速度，通过进行求解计算，得到基于时间的关节轨迹作为目标轨迹路径；其中，所述轨迹算法具体可以包括以下至少之一：多项式插值规划算法、三角函数插值规划算法、S型插值规划算法等。
具体的，以五次多项式规划算法作为预设的规划算法为例，可以按照以下方式确定出符合要求的目标轨迹路径。
首先，基于五次多项式规划算法构建主操作端的关节位置q关于时间t的路径函数，具体可以表示为以下形式：q(t)＝a5t5+a4t4+a3t3+a2t2+a1t+a0。其中，a0、a1、a2、a3、a4、a5分别为待确定的系数参数。
再根据主操作端的当前关节位置、目标位姿数据，以最大关节速度和最大关节加速度，设置约束条件。
具体的，可以按照以下方式设置约束条件：
起始时间的主操作端的关节位置：q(0)＝qs；
结束时间的主操作端的关节位置：q(tf)＝qf；
起始时间的主操作端的关节速度：
结束时间的主操作端的关节速度：
起始时间的主操作端的关节加速度：
结束时间的主操作端的关节加速度：
最大关节速度限制：
最大关节加速度限制：
其中，qs为主操作端的当前关节位置，qf为主操作端的目标位姿数据中的位置，为最大关节速度，为最大关节加速度。具体可以参阅图6和图7所示。
基于上述约束条件，代入路径函数q(t)，可以计算出各个系数参数的具体数值：
从而可以得到满足要求的路径函数q(t)，得到了符合要求的目标轨迹路径。其中，结束时间的具体数值可以根据具体情况和处理需求灵活设置。
进而可以根据目标轨迹路径，控制主操作端运动，以使得主操作端的位姿与新切换的第二从机械臂的目标手术器械相匹配；同时，也可以使得用户获得较好的操作体验。
在一些实施例中，具体进行路径规划时，还可以考虑主操作端运动过程中所遇到的障碍物，并采用优化的规划算法来求解得到效果更好、精度更高、更加安全的目标轨迹路径。
具体实施时，参阅图8所示，首先，可以根据障碍物，确定出安全的有效运动空间；其中，所述障碍物包括以下所列举的一种或多种：扶手、围栏、用户的手臂等。
例如，可以先通过获取并根据当前用户使用主操作端时的影像数据来确定出障碍物；再根据障碍物确定出障碍物空间。然后，按照以下算式，确定出有效运动空间：其中，A为障碍物空间，Jm为主操作端的有效运动空间。
同时，根据最大关节速度和最大关节加速度，建立约束条件。
例如，按照以下算式，建立约束条件：|Vj|<＝vmax,|amax|<＝amax，其中，vmax为最大关节速度，amax为最大关节加速度。此外，在建立约束条件时，还可以引入用户使用主操作端操控第二从机械臂时针对手柄末端的抓握特征。
然后，采用优化的规划算法作为预设的规划算法，来进行求解运算，以确定出符合要求的目标轨迹路径。其中，所述优化的规划算法包括以下至少之一：RRT算法、人工势场法、BUG算法、图搜索法等。
基于上述实施例，由于考虑到了主操作端运动过程中可能碰到的障碍物，并根据障碍物有针对性地建立有效运动空间；再基于该有效运动空间，进行路径规划，从而可以有效地避免后续在控制主操端按照目标轨迹路径运动的过程中与障碍物发送碰撞。
在一些实施例中，具体的，可以采用一种优化的规划算法来快速地求解得到效果相对较好的目标轨迹路径。
可以设主操作端的当前关节位置为Pcur，该关节的目标位置为Ptar，根据优化的规划算法，可以设置自由空间(或有效运动空间)为：C∈(Pcur,Ptar)。此外，还可以设置障碍物空间为S。参阅图9所示，利用优化的规划算法，具体求解时，可以包括以下步骤：
S1：在自由空间随机生成一个随机点Ptmp；
S2：计算生长树上所有当前节点Pn与该随机点Ptmp间的距离，并选择与该随机点Ptmp最近的点为Pnearest；
S3：由最近点Pnearest向目标点Ptmp以步长u生成生长点Pgrowing，其中，上述步长u可为固定值或随机值，且步长u小于该关节最大关节速度，即u<＝vmax；
S4：判断若该关节运动至该生长点Pgrowing时是否会发生碰撞；
具体进行S4时，若发生碰撞，舍弃该生长点，重复上述步骤S1、S2、S3、S4；若不发生碰撞，则将该生长点Pgrowing作为生长树的新节点Pn，触发进行步骤S5；
S5：计算新节点Pn与目标节点Ptar间的距离是否小于步长u；
具体进行S5时，若小于u，则规划结束，依次查找出相关父节点即可得到完整路径，作为目标轨迹路径；若大于等于u，则重复上述步骤S1、S2、S3、S4、S5。
由上可见，本说明书实施例提供的机械臂的匹配方法，医生控制台在接收到指示将主操作端由操控第一从机械臂切换为操控第二从机械臂的机械臂切换指令时，可以先获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；再综合利用上述数据，设置并求解兼顾多方面因素的目标函数，以确定出主操作端的目标位姿数据；再根据上述目标位姿数据，控制主操作端运动。从而可以较好地避免主操作 端的主操作臂跟随运动范围过大对用户操作的影响，能够精准地控制主操作端自动调整到与第二从机械臂相匹配，且与之前用户已适应的构型相同或相近的位姿状态，有效改善了用户操作时舒适度和连续性，使用户可以获得较好的操作体验。进一步，还通过采集并根据用户使用医生控制台的主操作端操控从机械臂时的历史影像数据，确定出用户使用主操作端操控该从机械臂时针对手柄末端的抓握特征；再根据该抓握特征，对目标约束条件进行针对性的调整，以便后续可以基于上述调整后的目标约束条件求解目标函数，得到符合该用户的个性化抓握习惯的目标位姿数据，进一步提高用户的舒适度。此外，还通过利用预设的规划算法基于目标位姿数据进行路径规划来确定目标轨迹路径，从而可以较为高效地确定出符合要求的目标轨迹路径，进而可以根据上述目标轨迹路径快速、安全地控制主操作端运动，以与新切换的从机械臂匹配，减少了用户的等待时长，进一步提高了用户的操作体验。
本说明实施例还提供了一种医生控制台，至少包括：主操作端和机械臂的匹配装置；其中，所述主操作端与患者手术台的从机械臂关联，所述机械臂的匹配装置执行时实现以下步骤：接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
在一些实施例中，主操作端处还可以设置有传感器，用于采集主操作端的关节位置。进一步，主操作端还可以包括：主操作臂和手柄末端。
本说明书实施例还提供一种计算机设备，包括处理器以及用于存储处理器可执行指令的存储器，所述处理器具体实施时可以根据指令执行以下步骤：接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
为了能够更加准确地完成上述指令，本说明书实施例还提供了另一种具体的计算机设备，其中，所述计算机设备包括网络通信端口、处理器以及存储器，上述结构通过内部线缆相连，以便各个结构可以进行具体的数据交互。
其中，所述网络通信端口，具体可以用于接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械。
所述处理器，具体可以用于响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历 史构型数据；根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
所述存储器，具体可以用于存储相应的指令程序。
在本实施例中，所述网络通信端口可以是与不同的通信协议进行绑定，从而可以发送或接收不同数据的虚拟端口。例如，所述网络通信端口可以是负责进行web数据通信的端口，也可以是负责进行FTP数据通信的端口，还可以是负责进行邮件数据通信的端口。此外，所述网络通信端口还可以是实体的通信接口或者通信芯片。例如，其可以为无线移动网络通信芯片，如GSM、CDMA等；其还可以为Wifi芯片；其还可以为蓝牙芯片。
本说明书实施例还提供了一种基于上述机械臂的匹配方法的计算机可读存储介质，所述计算机可读存储介质存储有计算机程序指令，在所述计算机程序指令被执行时实现：接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
在本实施例中，该计算机可读存储介质存储的程序指令具体实现的功能和效果，可以与其它实施方式对照解释，在此不再赘述。

Claims (15)

  1. 一种机械臂的匹配方法，其特征在于，包括：
    接收机械臂切换指令；其中，所述机械臂切换指令用于指示将主操作端由操控第一从机械臂切换为操控第二从机械臂；所述第二从机械臂挂载有目标手术器械；
    响应所述机械臂切换指令，获取目标手术器械的当前位姿数据、主操作端的当前关节位置，以及上一次主操作端操控第二从机械臂时与第二从机械臂对应的主操作端的历史构型数据；
    根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据；
    根据目标位姿数据，控制主操作端运动，以与第二从机械臂相匹配。
  2. 根据权利要求1所述的机械臂的匹配方法，其特征在于，获取目标手术器械的当前位姿数据，包括：
    采集目标手术器械当前在基于第二从机械臂的第二基坐标系下的位姿坐标，作为目标手术器械的初始位姿数据；
    获取第二基坐标系相对于基于参考机械臂的参考基坐标系的第一转换关系，以及参考基坐标系相对于内窥镜坐标系的第二转换关系；其中，所述参考机械臂为挂载有内窥镜的从机械臂；
    根据目标手术器械的初始位姿数据、第一转换关系和第二转换关系，计算出目标手术器械当前在内窥镜坐标系下的位姿坐标，作为目标手术器械的当前位姿数据。
  3. 根据权利要求1所述的机械臂的匹配方法，其特征在于，获取主操作端的当前关节位置，包括：
    采集主操作端的各个关节当前在基于主操作臂的主坐标系下的位置坐标，作为主操作端的初始关节位置坐标；
    获取主坐标系相对于基于手柄末端的手柄坐标系的第三转换关系，以及手柄坐标系相对于显示器坐标系的第四转换关系；
    根据主操作端的初始关节位置坐标、第三转换关系和第四转换关系，计算出主操作端的各个关节当前在显示器坐标系下的位置坐标，作为主操作端的当前关节位置。
  4. 根据权利要求3所述的机械臂的匹配方法，其特征在于，在获取主操作端的当前关节位置之后，所述方法还包括：
    将所述主操作端的当前关节位置确定为主操作端操控第一从机械臂时与第一从机械臂对应的构型数据，并保存该构型数据。
  5. 根据权利要求1所述的机械臂的匹配方法，其特征在于，根据目标手术器械的当前位姿数据、主操作端的当前关节位置和历史构型数据，确定出针对主操作端的目标位姿数据，包括：
    根据目标手术器械的当前位姿数据、主操作端的当前关节位置，确定出相匹配的主 操作端的参考位姿数据；
    根据主操作端的参考位姿数据、主操作端的当前关节位置和历史构型数据，构建关于目标位姿数据的目标函数；
    求解目标函数，得到针对主操作端的目标位姿数据。
  6. 根据权利要求5所述的机械臂的匹配方法，其特征在于，根据目标手术器械的当前位姿数据、主操作端的当前关节位置，确定出相匹配的主操作端的参考位姿数据，包括：
    根据目标手术器械的当前位姿数据和主从映射关系，确定出主操作端的期望位姿数据；
    根据主操作端的关节位置与姿态数据的转换关系，利用主操作端的期望位姿数据和当前关节位置，确定出与当前关节位置相匹配的主操作端的参考位姿数据。
  7. 根据权利要求5所述的机械臂的匹配方法，其特征在于，主操作端包括主操作臂和手柄末端；其中，所述主操作臂设置有多个位置关节；
    相应的，所述参考位姿数据包括：主操作臂的参考关节位置，以及手柄末端的参考末端位置和参考姿态数据；
    所目标位姿数据至少包括：手柄末端的目标末端位置和目标姿态数据。
  8. 根据权利要求7所述的机械臂的匹配方法，其特征在于，根据主操作端的参考位姿数据、主操作端的当前关节位置和历史构型数据，构建关于目标位姿数据的目标函数，包括：
    根据目标末端位置和参考末端位置构建关于末端位置偏差的第一目标函数；根据目标姿态数据和参考姿态数据构建关于末端姿态偏差的第二目标数据；
    根据参考关节位置和主操作端的当前关节位置，构建关于关节跟随偏差的第三目标函数；
    根据参考关节位置和主操作端的历史构型数据，构建关于构型偏差的第四目标函数。
  9. 根据权利要求8所述的机械臂的匹配方法，其特征在于，在根据主操作端的参考位姿数据、主操作端的当前关节位置和历史构型数据，构建关于目标位姿数据的目标函数之后，所述方法还包括：
    根据主操作臂的位置关节的位置阈值，构建针对目标函数的目标约束条件；
    相应的，求解目标函数，得到针对主操作端的目标位姿数据，包括：
    基于所述目标约束条件，求解所述目标函数，得到针对主操作端的目标位姿数据。
  10. 根据权利要求9所述的机械臂的匹配方法，其特征在于，所述方法还包括：
    确定用户使用主操作端操控第二从机械臂时针对手柄末端的抓握特征；其中，所述抓握特征包括抓握位置和/或抓握方式；
    根据所述抓握特征，调整所述目标约束条件。
  11. 根据权利要求10所述的机械臂的匹配方法，其特征在于，确定用户使用主操作端时针对手柄末端的抓握特征，包括：
    获取用户上一次使用主操作端操控第二从机械臂时的历史影像数据；
    根据所述历史影像数据，确定出用户使用主操作端操控第二从机械臂时针对手柄末端的抓握特征。
  12. 根据权利要求5所述的机械臂的匹配方法，其特征在于，求解目标函数，得到针对主操作端的目标位姿数据，包括：
    利用预设的求解算法，求解目标函数，得到针对主操作端的目标位姿数据；其中，所述预设的求解算法包括以下至少之一：粒子群算法、遗传算法、萤火虫算法。
  13. 根据权利要求1所述的机械臂的匹配方法，其特征在于，根据目标位姿数据，控制主操作端运动，包括：
    利用预设的规划算法，根据主操作端的当前关节位置和目标位姿数据，通过路径规划，确定出符合要求的目标轨迹路径；
    根据所述目标轨迹路径，控制主操作端运动。
  14. 一种医生控制台，其特征在于，至少包括：主操作端和机械臂的匹配装置；其中，所述主操作端与患者手术台的从机械臂关联；所述机械臂的匹配装置执行时实现权利要求1至13中任一项所述方法的步骤。
  15. 一种计算机可读存储介质，其特征在于，其上存储有计算机指令，所述指令被处理器执行时实现权利要求1至13中任一项所述方法的步骤。
PCT/CN2023/112022 2022-08-24 2023-08-09 机械臂的匹配方法、医生控制台和计算机可读存储介质 WO2024041386A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211020147.8A CN117695016A (zh) 2022-08-24 2022-08-24 机械臂的匹配方法、医生控制台和计算机可读存储介质
CN202211020147.8 2022-08-24

Publications (1)

Publication Number Publication Date
WO2024041386A1 true WO2024041386A1 (zh) 2024-02-29

Family

ID=90012515

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/112022 WO2024041386A1 (zh) 2022-08-24 2023-08-09 机械臂的匹配方法、医生控制台和计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN117695016A (zh)
WO (1) WO2024041386A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118319502B (zh) * 2024-06-07 2024-09-27 苏州视通医疗科技有限公司 手术机器人的主从运动控制方法及相关设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020082612A1 (en) * 1998-11-20 2002-06-27 Intuitive Surgical, Inc. Arm cart for telerobotic surgical system
CN103533909A (zh) * 2011-05-13 2014-01-22 直观外科手术操作公司 用于控制工具运动的标架的位置和定向的估计
CN105431104A (zh) * 2013-07-26 2016-03-23 奥林巴斯株式会社 医疗用系统及其控制方法
CN109806002A (zh) * 2019-01-14 2019-05-28 微创（上海）医疗机器人有限公司 一种用于手术机器人的成像系统及手术机器人
US20200100852A1 (en) * 2017-06-08 2020-04-02 Medicaroid Corporation Remote control apparatus and remote control system
CN113876434A (zh) * 2020-07-01 2022-01-04 北京术锐技术有限公司 主从运动的控制方法、机器人系统、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020082612A1 (en) * 1998-11-20 2002-06-27 Intuitive Surgical, Inc. Arm cart for telerobotic surgical system
CN103533909A (zh) * 2011-05-13 2014-01-22 直观外科手术操作公司 用于控制工具运动的标架的位置和定向的估计
CN105431104A (zh) * 2013-07-26 2016-03-23 奥林巴斯株式会社 医疗用系统及其控制方法
US20200100852A1 (en) * 2017-06-08 2020-04-02 Medicaroid Corporation Remote control apparatus and remote control system
CN109806002A (zh) * 2019-01-14 2019-05-28 微创（上海）医疗机器人有限公司 一种用于手术机器人的成像系统及手术机器人
CN113876434A (zh) * 2020-07-01 2022-01-04 北京术锐技术有限公司 主从运动的控制方法、机器人系统、设备及存储介质

Also Published As

Publication number Publication date
CN117695016A (zh) 2024-03-15

Similar Documents

Publication Publication Date Title
CN111315309B (zh) 用于控制机器人操纵器或相关工具的系统和方法
JP7275204B2 (ja) 遠隔操作医療システムにおけるオンスクリーンメニューのためのシステム及び方法
WO2021047522A1 (zh) 手术机器人及其末端器械的控制方法、控制装置
US20210068910A1 (en) Robotic surgical system control scheme for manipulating robotic end effctors
KR101891162B1 (ko) 도구의 제어 움직임에 사용되는 프레임의 위치 및 방향을 추정하는 장치 및 방법
JP6567558B2 (ja) 外科手術ロボットシステムにおいてカメラ位置を制御するためのシステムおよび方法
US8315720B2 (en) Method for graphically providing continuous change of state directions to a user of a medical robotic system
US8002694B2 (en) Master-slave manipulator system
US9632573B2 (en) Medical manipulator and method of controlling the same
JP2020532383A (ja) 手術ロボットシステムのためのカメラ制御
US20200261160A1 (en) Robotic surgical systems and methods and computer-readable media for controlling them
Stroppa et al. Human interface for teleoperated object manipulation with a soft growing robot
WO2022002155A1 (zh) 主从运动的控制方法、机器人系统、设备及存储介质
CN113876434A (zh) 主从运动的控制方法、机器人系统、设备及存储介质
KR20140126301A (ko) 오퍼레이터 작용을 구별하는 방식을 이용하는 로봇 시스템 작동 모드의 사용자 선택
US11478312B2 (en) Unmatching/matching UID to robot grasper for engaging teleoperation
WO2024041386A1 (zh) 机械臂的匹配方法、医生控制台和计算机可读存储介质
Devine et al. Real time robotic arm control using hand gestures with multiple end effectors
CN113876433A (zh) 机器人系统以及控制方法
WO2023066253A1 (zh) 机械臂控制方法、装置以及机械臂
CN114074328B (zh) 机器人系统及其控制方法
CN113069208B (zh) 手术导航方法、装置、电子设备及存储介质
CN116763450A (zh) 包括弯曲工具的机器人系统及其主从运动控制方法
EP4164538A1 (en) Surgical robotic system with motion integration
CN118042999A (zh) 具有不同长度的进入端口的外科机器人系统的操作系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23856476

Country of ref document: EP

Kind code of ref document: A1