WO2023202291A1

WO2023202291A1 - Surgical robot system and control device apparatus thereof

Info

Publication number: WO2023202291A1
Application number: PCT/CN2023/082193
Authority: WO
Inventors: 王深辉; 王牌; 高元倩
Original assignee: 深圳市精锋医疗科技股份有限公司
Priority date: 2022-04-23
Filing date: 2023-03-17
Publication date: 2023-10-26
Also published as: CN116965937A

Abstract

The present application relates to a surgical robot system and a control apparatus thereof. The system comprises a controller, and a driving arm and an operation portion coupled with the controller. The controller is configured to: apply, when the tail end of a first device exceeds the boundary of a safety space of the tail end of the first device, a virtual force to the tail end of the first device; convert a virtual force of the tail end of the first device in a first coordinate system into a real force of the tail end of a second device in a second coordinate system, wherein the first device comprises one of the driving arm and the operation portion, and the second device comprises the other one of the driving arm and the operation portion; convert the real force of the tail end of the second device in the second coordinate system into a target joint driving force of a joint assembly in the second device; and control the joint assembly in the second device to output the target joint driving force so as to ensure safety or reliability.

Description

Surgical robot system and its control device

This application claims priority to the Chinese patent application filed with the China Patent Office on April 23, 2022, with the application number CN 202210431915.2 and the application name "Surgical Robot System and Control Device", the entire content of which is incorporated into this application by reference. middle.

Technical field

The present application relates to the field of medical devices, and in particular to a surgical robot system and its control device.

Background technique

Minimally invasive surgery refers to a surgical method that uses modern medical instruments such as laparoscope and thoracoscope and related equipment to perform surgery inside the human cavity. Compared with traditional surgical methods, minimally invasive surgery has the advantages of less trauma, less pain, and faster recovery.

With the advancement of science and technology, minimally invasive surgical robot system technology has gradually matured and been widely used. The surgical robot system includes a master operating console and slave operating equipment. The slave operating equipment includes a plurality of medical instruments with terminal instruments. These medical instruments include imaging instruments with imaging terminal instruments and surgical instruments with operating terminal instruments. The main operating console includes a display and an operating section. The doctor operates the operating part to control the movement of the imaging instrument or the surgical instrument under the field of view provided by the imaging instrument displayed on the monitor.

However, since the range of motion of the operating part and the range of motion of the terminal instrument are usually inconsistent, when one of the operating part and the terminal instrument reaches the boundary of the movement range, in the overbound direction, due to its continued manipulation or manipulation, it is easy to Causing problems that compromise safety or reliability to occur.

Contents of the invention

Based on this, it is necessary to provide a surgical robot system and its control device that can ensure safety or reliability.

On the one hand, this application provides a surgical robot system, including: a driving arm; an operating part, and The driving arm has motion correlation; a controller is coupled to the driving arm and the operating part, and is configured to: when the end of the first device exceeds the boundary of its safe space, move toward the end of the first device. applying a virtual force; converting the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system, the first device including one of the driving arm and the operating part , the second device includes the other one of the driving arm and the operating part; convert the real force of the end of the second device in the second coordinate system into the target joint driving force of the joint assembly in the second device; control the second The joint component in the device outputs the target joint driving force so that the operator can feel the resistance at the second device.

Optionally, the drive arm includes a manipulator assembly, the operating portion is configured to operate the manipulator assembly, the manipulator assembly includes a medical instrument, the first device includes the manipulator assembly, The end of the manipulator assembly includes the end instrument of the medical device, and converting the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system includes: The virtual force of the terminal instrument in the first coordinate system is converted into a virtual force in the first intermediate coordinate system; the requirements of the operating part in the second intermediate coordinate system are determined based on the virtual force of the terminal instrument in the first intermediate coordinate system. The applied real force; convert the real force of the operating part in the second intermediate coordinate system into the real force in the second coordinate system; wherein the first coordinate system includes the base coordinate system of the medical device, the first intermediate The coordinate system includes an endoscope coordinate system, the second intermediate coordinate system includes a display coordinate system, and the second coordinate system includes a base coordinate system of the operating part.

Optionally, applying a virtual force to the end of the first device includes: obtaining a position point where the end of the first device exceeds the boundary of its safe space; determining a target direction for applying the virtual force based on the position point; and applying the virtual force to the end of the first device. The tip of the first device exerts the virtual force in the target direction.

Optionally, the safe space includes a cylindrical space, and the target direction includes a direction perpendicular from the position point to a central axis of the cylindrical space.

Optionally, the safe space includes two or more safe spaces defined based on different conditions. When the end of the first device exceeds the boundaries of different safe spaces, the magnitude of the virtual force applied to the end of the first device is different.

Optionally, the magnitude of the virtual force is the same as the magnitude of the real force; the real force is between 3N～10N.

Optionally, the drive arm includes a manipulator assembly, the operating portion is configured to operate the manipulator assembly, the manipulator assembly includes a medical instrument, the first device includes the manipulator assembly, The end of the manipulator assembly includes a terminal instrument of the medical device, and the controller is further configured to: obtain first information that the terminal instrument exceeds the boundary of its safe space, where the first information includes an out-of-bounds position, an out-of-bounds At least one of the number of times and the out-of-bounds time; obtaining the surgical procedure associated with the safe space of the end instrument; and determining the level of the doctor's operating proficiency under the surgical procedure by combining the first information and the surgical procedure. .

Optionally, the surgical robot system is coupled to a hospital management system, and the controller is further configured to: send doctor information associated with the surgical procedure and having a level of the doctor's operating proficiency to The hospital management system is configured to match a suitable doctor to a patient based on the doctor information.

Optionally, the drive arm includes an interconnected robotic arm and a manipulator assembly, the operating portion is configured to operate the manipulator assembly, the manipulator assembly includes a medical instrument, and the first device includes the The manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical device, the distal end of the robotic arm is equipped with a puncture device, and multiple medical instruments are inserted into the living body through the same puncture device. body, different safety spaces of the terminal instrument are associated with the orientation of the puncturer, and the controller is further configured to: obtain the safety space of the terminal instrument when the doctor performs the operation under the current orientation of the puncturer. The first information of the boundary of the space, the first information includes at least one of the exceeding position, the number of exceeding times and the exceeding time; predicting the target orientation of the puncture device based on the obtained first information.

Optionally, different orientations of the trocar are associated with different surgical procedures, and the controller is further configured to: generate visual information and/or auditory information including the target orientation of the trocar and play it; or, Based on the target orientation of the trocar, visual information and/or auditory information of a target surgical procedure associated with the target orientation of the trocar is generated and played.

Optionally, the puncture tool passes through a telecentric fixed point. When the puncture tool is connected to the living body, the telecentric fixed point corresponds to the position where the puncture tool is connected to the living body, and the controller further is configured To: control the movement of the joint assembly in the mechanical arm according to the target orientation of the puncturer to make the puncturer move around the distal fixed point, and make the orientation of the puncturer reach the target of the puncturer orientation.

Optionally, the drive arm includes an interconnected robotic arm and a manipulator assembly, the operating portion is configured to operate the manipulator assembly, the manipulator assembly includes a medical instrument, and the first device includes the The manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical device, the distal end of the robotic arm is equipped with a puncture device, and multiple medical instruments are inserted into the living body through the same puncture device. body, the puncture device passes through a telecentric fixed point, and when the puncture device is connected to the living body, the telecentric fixed point corresponds to the position where the puncture device is connected to the living body, and the controller is also configured To: Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device. The first information includes one of the out-of-bounds position, the number of out-of-bounds times, and the out-of-bounds time. Above; determine the target center point based on the first information; control the movement of the joint assembly in the robotic arm to move the puncturer around the distal fixed point and align the orientation of the puncturer with the target center point .

Optionally, the medical device includes an imaging device, and determining the target center point based on the first information includes: obtaining an operation image captured by the imaging device; determining the center point in the operation image by combining the first information and the operation image. the target organ; determining the target center point based on the target organ.

Optionally, the controller is further configured to highlight the target organ and/or the target center point in the operation image.

Optionally, the medical instrument includes a surgical instrument, and the controller is further configured to: control the manipulation in response to a change in the orientation of the trocar when the trocar moves around the distal fixed point. The joint components in the instrument assembly move to maintain the position or posture of the terminal instrument.

Optionally, the drive arm includes an interconnected robotic arm and a manipulator assembly, the operating portion is configured to operate the manipulator assembly, the manipulator assembly includes a medical instrument, and the first device includes the The manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical device, the distal end of the robotic arm is equipped with a puncture device, and multiple medical instruments are inserted into the living body through the same puncture device. body, the puncture device passes through the telecentric fixed point, and the puncture device is connected to the living body Then, the telecentric fixed point corresponds to the position where the puncture device is connected to the living body, and the controller is further configured to: obtain the end instrument during the operation performed by the doctor under the current orientation of the puncture device. The first information that exceeds the boundary of its safe space, the first information includes more than one of the over-bound position, the number of over-bounds, and the over-bound time; switch the first operation mode to the second operation mode, the first operation mode includes the above The operating part operates the manipulator assembly, and the second operating mode includes the operating part operating the robotic arm.

Optionally, the second operation mode includes: the orientation of the puncture tool changes following the change in the orientation of the operating part; or, the orientation of the puncture tool changes following the change in the position of the operating part.

Optionally, the controller is further configured to: determine a first section on the boundary of the safe space of the terminal instrument based on the first information; generate and display an image model of the first section.

On the other hand, the present application provides a control method for a surgical robot system. The surgical robot system includes a driving arm and an operating part, and has motion correlation with the driving arm. The control method includes: in the first device When the end of the first device exceeds the boundary of the safe space of the end of the first device, a virtual force is applied to the end of the first device, and the direction of the virtual force is opposite to the direction of the end of the first device beyond the boundary. ; Convert the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system. The first device includes the drive arm and the operating part. One, the second device includes the other of the drive arm and the operating part; convert the real force of the end of the second device in the second coordinate system into the target of the joint assembly in the second device Joint driving force; controlling the joint assembly in the second device to output the target joint driving force so that the operator can feel resistance at the second device.

On the other hand, the present application provides a computer-readable storage medium that stores a computer program, and the computer program is configured to be loaded and executed by a processor to implement any one of the above embodiments. the steps of the control method described above.

On the other hand, the present application provides a control device for a surgical robot system, including: a memory for storing a computer program; and a processor for loading and executing the computer program; wherein the computer program is configured as Loaded and executed by the processor to implement any of the above The steps of the control method described in the embodiment.

The surgical robot system and its control device of the present application have the following beneficial effects:

When the end of the first device exceeds the boundary of the safe space, a virtual force is applied to the end and the virtual force is converted into a real force of the second device, thereby controlling the joint component in the second device to output a target related to the real force. The joint driving force, even when the end of the first device does not and/or cannot be provided with a force sensor that senses exceeding the boundary, can also enable the operator to have a force sensor at the second device when the end of the first device exceeds the boundary of the safe space. Obvious force sense tactile sensation, that is, you can feel the resistance caused when the end of the first device exceeds its boundary, avoiding excessive manipulation of the end of the first device; at the same time, since there is no need to set a force sensor at the end of the first device, it can reduce cost, and the structure of the end can also be simplified.

Description of the drawings

Figure 1 is a schematic structural diagram of the main operating console of an embodiment of the surgical robot system of the present application;

Figure 2 is a schematic structural diagram of a slave operating device of an embodiment of the surgical robot system of the present application;

Figure 3 is a schematic structural diagram of the manipulator assembly of an embodiment of the slave operating device shown in Figure 2;

Figure 4 is a partial schematic diagram of the surgical robot system of the present application in an operation state;

Figure 5 is a partial schematic diagram of the surgical robot system of the present application in another operating state;

Figure 6 is a flow chart of an embodiment of the control method of the surgical robot system of the present application;

Figure 7 is a flow chart of another embodiment of the control method of the surgical robot system of the present application;

Figure 8 is a state schematic diagram of an embodiment of the control method of the surgical robot system of the present application;

Figure 9 is a schematic structural diagram of the safe space of an embodiment of the manipulator assembly of the present application;

Figure 10 is a schematic diagram of the transboundary direction of the end instrument of an embodiment of the surgical robot system of the present application;

Figure 11 is a partial schematic diagram of the surgical robot system of the present application in another operating state;

Figure 12 is a flow chart of another embodiment of the control method of the surgical robot system of the present application;

Figures 13 to 22 are respectively schematic diagrams of the display interface related to the out-of-bounds section in the surgical robot system of the present application;

Figure 23 is a schematic structural diagram of a control device of a surgical robot system according to an embodiment of the present application.

Figure 24 is a schematic structural diagram of a slave operating device of another embodiment of the surgical robot system of the present application.

Detailed ways

In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present application are shown in the accompanying drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described in this application. Rather, these embodiments are provided to provide a thorough and comprehensive understanding of the disclosure of the present application.

It should be noted that when an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present. When an element is said to be "connected" to another element, it can be directly connected to the other element or there may also be intervening elements present. When an element is said to be "coupled" to another element, it can be directly coupled to the other element or intervening elements may also be present. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this application are for illustrative purposes only and do not represent the only implementation manner. The terms "distal" and "proximal" used in this application are directional terms, which are commonly used terms in the field of interventional medical devices, where "distal" means the end far away from the operator during the operation, and "proximal" means The end closest to the operator during surgery. The terms "first/second" etc. used in this application may refer to one component and a type of two or more components having common characteristics.

Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by a person skilled in the technical field of this application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. The terms "each" and "plurality" used in this application include one or more than two.

The surgical robot system includes a master operating table and slave operating equipment. As shown in FIG. 1 , the main operating console 200 includes an operation part 210 and a display, and the display may be selected from one of a 2D display and a 3D display. As shown in Figure 2, the slave operating device 100 includes a drive arm. The drive arm may be configured to move under manipulation (i.e., teleoperation or control) of the operating portion 210, such movement including repositioning and/or reorientation, That is, changing position and/or orientation (i.e. posture). The driving arm includes a robotic arm 110 and a manipulator assembly 120. As shown in Figure 3, the manipulator assembly 120 includes a manipulator 130 movably assembled on the distal end of the robotic arm 110 and a medical instrument 140 detachably assembled on the manipulator 130. The medical instrument 140 can be moved by the manipulator 130 . As an example, the manipulator 130 and the robotic arm 110 are connected through a joint assembly, and the joint assembly includes a rotation joint assembly or a translation joint assembly. As an example, the manipulator 130 and the robotic arm 110 are connected through a translation joint assembly, and then The movement between the two can be realized. For example, the translation joint assembly includes a translation mechanism and a manipulator 130 slidably disposed on the guide rail. For example, the translation mechanism includes a driving mechanism and a screw pair coupled to each other. The screw pair includes The screw and the sliding table are slidably disposed on the screw. The manipulator 130 is coupled with the sliding table. The controller drives the sliding table to slide relative to the screw by controlling the driving mechanism to drive the movement of the manipulator 130 .

As shown in FIG. 4 , the medical instrument 140 includes an imaging instrument 141 (also called an endoscope) and a surgical instrument 142 . The medical instrument 140 includes a terminal instrument 150 . The terminal instrument of the imaging instrument 141 includes an image terminal instrument 151 . The terminal instrument of the surgical instrument 142 The instrument includes an operating tip instrument 152. The medical device 140 usually needs to be inserted into the body of a living being (including a human or an animal) to work. The image terminal instrument 151 of the imaging instrument 141 is used to capture operating images, and the operating terminal instrument 152 of the surgical instrument 142 is used to perform surgical operations such as shearing, suturing, cutting, burning, cleaning, and aspirating air and liquid.

In some embodiments, the operating portion 210 is configured to operate any component of the drive arm, including any joint assembly of the drive arm. For example, the operation part 210 can operate the end of the driving arm by manipulating multiple joint components at the proximal end of the end. The end of the drive arm can be freely defined, for example, the end of the drive arm includes the end of the first part of the drive arm. For example, when the driving arm includes multiple arm sections (such as the robot arm 110 and the manipulator assembly 120), any section of the arm body can be manipulated as the first part, and the end of any arm body can be selectively configured as the end of the driving arm. Moreover, the operating part 210 may be configured to manipulate the movement of the end of the arm body by manipulating the movement of the joint assembly in the arm body.

In some embodiments, the tip of the drive arm is configured to include tip instrument 150 . The end of the driving arm may include an imaging end instrument 151 or an operating end instrument 152 . For example, when the end includes the image end instrument 151, the doctor observes the image captured by the end instrument 151 through the display. The operating image is then manipulated through the operating part 210 to manipulate the arm movement associated with the image terminal instrument 151 to reposition and/or reorient the image terminal instrument 151 . For another example, when the end includes the operating end instrument 152, the doctor observes the operating image captured by the image end instrument 151 through the display, and then uses the operating part 210 to manipulate the arm movement associated with the operating end instrument 152 to re-operate the operating end instrument 152. Target and/or redirect.

For example, when the end of the driving arm includes the end instrument 150, any joint component at the proximal end of the end instrument 150 in the driving arm can be configured to be manipulated by the operating part 210 to achieve repositioning and/or repositioning of the end instrument 150 as needed. Orientation, wherein these joint components configured to be manipulated can originate from joint components in any one or more than two sections of the arm body. For example, the manipulable joint assembly may originate from the manipulator 130 so that the tip instrument 150 can be repositioned and/or reoriented by manipulating the manipulator 130 . As another example, the manipulated joint assembly may be derived from the medical device 140 so that the end device 150 can be repositioned and/or reoriented by manipulating the medical device 140 . As another example, the manipulable joint assembly may originate from the manipulator assembly 120 so that the end instrument 150 can be repositioned and/or reoriented by manipulating the manipulator assembly 120 (including the manipulator 130 and the medical instrument 140). As another example, the manipulated joint assembly may originate from the robotic arm 110 , and the terminal instrument 150 may be repositioned and/or reoriented by manipulating the robotic arm 110 . As another example, the manipulable joint assembly can be derived from the robotic arm 110 and the manipulator assembly 120, and then the terminal instrument 150 can be repositioned by manipulating the robotic arm 110 and the manipulator assembly 120 (including the manipulator 130 and the medical instrument 140). and/or redirect.

In some embodiments, the end of the drive arm is configured to include the end of the robotic arm 110 . Continuing to refer to FIG. 4 , the end of the exemplary robotic arm 110 includes a puncturer 160 that is detachably connected to the distal end of the robotic arm 110 . The puncturer 160 is used to connect with an incision or natural cavity of a living body to form an airtight channel. The medical device 140 is inserted into the living body through the airtight channel. Referring to FIG. 5 , during surgery, it is usually necessary to keep the position of the connection point between the puncture device 160 and the living body unchanged to avoid stress tearing the incision or natural orifice caused by changes in the position of the puncture device 160 . For example, when the end of the driving arm includes the puncturer 160, the doctor can reorient the puncturer 160 by manipulating the movement of the arm body associated with the puncturer 160, that is, the mechanical arm 110, through the operating part 210. The reorientation of the puncturer 160 includes puncture. 160mm Movement around a remote center of motion. Of course, when the puncturer 160 is not connected to the living body, the puncturer 160 can also be repositioned and/or reoriented by manipulating the movement of the robotic arm 110 .

In some embodiments, the operation part 210 and the driving arm may be configured to have motion correlation. That is, movement of one of the operating portion 210 and the drive arm may be configured to cause a corresponding movement of the other.

In some embodiments, the surgical robot system includes a master-slave following mode. The master-slave following mode includes the doctor operating the operating part 210 to manipulate the driving arm. For example, the master-slave following mode includes the doctor operating the operating part 210 to manipulate the movement of the manipulator assembly 120 to achieve repositioning and/or reorientation of the terminal instrument 150 .

In some embodiments, the number of operating parts 210 includes more than one, and the target objects manipulated by the operating parts 210 include more than one. In the slave operating device 100 as shown in FIG. 3 , the driving arm includes more than two manipulator assemblies 120 , and these manipulator assemblies 120 are installed at the distal end of the robotic arm 110 . For example, when a doctor controls the movement of the manipulator assembly 120 by operating the operating part 210, for the terminal instrument 150 of the same manipulator assembly 120, the doctor can operate one operating part 210 to operate the terminal instrument 150, or he can operate two terminal instruments 150 at the same time. The terminal instrument 150 is operated by an operating part 210.

In some embodiments, the surgical robot system includes a master-slave alignment mode. The master-slave alignment mode includes the operation part 210 moving in alignment with the movement of the driving arm. By way of example, the master-slave alignment mode includes that the doctor can manipulate the movement of the operating part 210 by operating the driving arm. For example, the master-slave alignment mode includes the doctor operating the operating part 210 to perform corresponding movements by dragging the end of the robotic arm 110 such as the movement of the puncture tool 160 . For example, the doctor can operate one puncture device 160 and manipulate one or two operating parts 210 .

In some embodiments, continuing to refer to FIGS. 1 and 2 , the operating portion 210 and the driving arm each include a plurality of joint components. These joint components may be selected from at least one of a rotational joint component and a translation joint component. For example, these joint components include active joint components; for another example, these joint components include active joint components and driven joint components. The active joint components can be driven, but the driven joint components cannot be driven and move under the drive of the active joint components. . Usually, the active joint assembly includes an electronically controllable driving mechanism. For example, the electronically controllable driving mechanism includes more than one of a motor, an electromagnet, and a pressure cylinder with an electronically controllable valve body. Such an active joint assembly Joint drives can usually be output Force is conducive to realizing force control, such as force balance, force feedback, etc.

In some embodiments, the surgical robot system further includes a controller. For example, the controller may be deployed at the main console 200. For another example, the controller may be deployed on the slave operating device 100 . As another example, the controller can be deployed in the cloud. For another example, the controller includes a first controller and a second controller. The first controller is deployed on the main operating console 200 and the second controller is deployed on the slave operating device 100; or the first controller is deployed on the cloud and the second controller is deployed on the cloud. The controller is deployed on the master operating station 200 or the slave operating device 100 . For another example, the controller includes a first controller, a second controller and a third controller. The first controller is deployed on the main operating console 200 , the second controller is deployed on the slave operating device 100 , and the third controller is deployed on the cloud. . The controller includes one or more processors, and multiple steps can be executed in one processor or multiple processors.

The controller is typically coupled to the electronically controllable components. For example, the controller is coupled to the operating part 210, the display and the driving arm respectively, and the controller is coupled to the driving arm, including the controller to the robotic arm 110, the manipulator 130 and the medical instrument 140. For convenience of description, one of the operating part 210 and the driving arm may be regarded as the first device, and the other of the operating part 210 and the driving arm may be regarded as the second device. For example, when the first device includes a driving arm, the second device includes the operating part 210; for another example, when the first device includes the operating part 210, the second device includes a driving arm. Wherein, any configuration of the driving arm may be described as the device. For example, the robot arm 110 and/or the manipulator assembly 120 may be described as the device.

When the first device and the second device have motion correlation, as shown in Figure 6, the controller can be configured to:

Step S11: When the end of the first device exceeds the boundary of the safe space of the end of the first device, a virtual force is applied to the end of the first device.

The term "exceed" includes the meaning of "reach" and/or "beyond".

This step does not actually apply force to the end of the first device, that is, this step does not control the actual output joint driving force of the corresponding joint component in the first device, but virtually applies a force to the end of the first device (for example, resistance), assuming that the end of the first device is subject to a force (e.g. resistance).

Wherein, the direction of the virtual force is opposite to the end of the first device and exceeds the boundary of the safe space. direction.

Step S12: Convert the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system.

The first coordinate system exemplarily includes the base coordinate system of the first device, and the second coordinate system exemplifies the base coordinate system of the second device.

Usually, the conversion between virtual force and real force can be achieved with more than one coordinate conversion.

For example, the magnitude of the real force may be equal to the magnitude of the virtual force. For example, when the virtual force is configured as 10N, the converted real force is also 10N.

For example, the size of the real force may not be equal to the size of the virtual force. For example, there is a linear amplification or reduction relationship between the size of the real force and the size of the virtual force. For example, when the virtual force is configured to 10N, when the amplification is set When the coefficient is 2, the real force is configured to 20N, and when the reduction coefficient is set to 2, the real force is configured to 5N; or, there is a specific offset value between the size of the real force and the size of the virtual force, for example, this offset The value is fixed at +2N. When the virtual force is configured to 5N, the real force is configured to 7N. For example, when the offset value is fixed to -2N, when the virtual force is configured to 5N, the real force is configured to 3N. .

In some embodiments, in a scenario where the first device includes a driving arm and the second device includes an operating part 210, when the real force obtained after conversion is in the range of 3N to 10N, it will not increase the burden on the doctor to operate the operating part 210. It can also be clearly felt that the end of the driving arm exceeds the boundary. Better, when the real force obtained after conversion is in the range of 5N ~ 8N, it usually has a better ergonomic experience.

Step S13: Convert the real force of the end of the second device in the second coordinate system into the target joint driving force of the joint assembly in the second device.

For example, the real force of the end of the second device in the second coordinate system can be converted into the target joint driving force of the joint component in the second device according to a dynamic method. For example, the real force of the end of the second device in the second coordinate system can also be converted into the target joint driving force of the joint component in the second device according to the kinematic method. For example, the real force of the end of the second device in the second coordinate system can be converted according to the Jacobian in the force domain. The method converts the real force of the end of the second device in the second coordinate system into the target relationship of the joint component in the second device. section driving force. It will be appreciated by those skilled in the art that the various "forces" described herein include forces and/or moments. For example, the target joint driving force includes a target joint driving force and/or a target joint driving torque.

Step S14, control the joint component in the second device to output the corresponding target joint driving force.

Among them, when the joint component includes a motor, it is the corresponding motor that outputs the target joint driving force; when the joint component includes a pressure cylinder, it is the corresponding pressure cylinder that outputs the target joint driving force; when the joint component includes an electromagnet, it is the corresponding pressure cylinder that outputs the target joint driving force. Corresponding electromagnet.

Through the above steps S11 to S14, even if the end of the first device does not and/or cannot be equipped with a force sensor that senses an out-of-boundary, it is also possible to enable the operator to stop the second device when the end of the first device exceeds the boundary of the safe space. There is an obvious sense of force (i.e. resistance) at the device, that is, the resistance caused when the end of the first device exceeds its boundary can be felt to avoid excessive manipulation of the end of the first device; at the same time, since there is no need to Providing a force sensor at the end can not only reduce costs, but also simplify the structure of the end.

In some embodiments, the first device includes, for example, the manipulator assembly 120 . The end of the first device includes the end of the manipulator assembly 120 . The end of the manipulator assembly 120 includes the end instrument 150 , and the end is, for example, a terminal on the end instrument 150 . Feature points or feature areas. The second device includes an operating part 210, and the end of the operating part 210 includes, for example, a wrist 220. The end is, for example, a certain feature point or feature area on the wrist 220. In the master-slave following operation mode, the operating part 210 realizes the repositioning and/or reorientation of the end instrument 150 by manipulating the movement of the manipulator assembly 120. The above steps S11 to S14 can be explained using this example. As shown in Figure 7, especially in the above step S12, the virtual force of the end of the first device in the first coordinate system is converted into the real force of the end of the second device in the second coordinate system, including:

Step S121: Convert the virtual force of the terminal instrument in the first coordinate system into the virtual force in the first intermediate coordinate system.

Step S122: Determine the real force that needs to be applied to the operating part in the second intermediate coordinate system based on the virtual force of the terminal instrument in the first intermediate coordinate system.

Step S123, convert the real force of the operating part in the second intermediate coordinate system into the force in the second coordinate system. True power.

The first coordinate system exemplarily includes the base coordinate system of the medical device 140. For example, the base coordinate system of the medical device 140 is configured as an end surface connected to the manipulator 130 at the proximal end of the medical device 140; an example of the first intermediate coordinate system Specifically includes the tool coordinate system of the image end instrument 151. The tool coordinate system of the image end instrument 151 is sometimes also called the endoscope coordinate system. For example, the endoscope coordinate system is configured to be on the distal surface of the image end instrument 151; The second intermediate coordinate system exemplarily includes the display coordinate system, for example, the display coordinate system is configured on the display surface of the display; the second coordinate system exemplarily includes the base coordinate system of the operation part 210 , for example, the base coordinate system of the operation part 210 The coordinate system is arranged at a position connected to the main console 200 at the proximal end of the operation unit 210 .

The real force obtained through the above steps S121 to S123 can provide the doctor with intuitive force feedback during operation in the master-slave operation mode. Among them, "intuition" includes that the movement direction of the terminal instrument 150 observed by the doctor on the display is consistent with the movement direction of the operating part 210 operated by the doctor. For example, referring to FIG. 8 , the operating part 210 controls the movement of the terminal instrument 150 . When the terminal instrument 150 reaches the boundary of its safe space, the force F of the terminal instrument 150 that the doctor's eyes see in the operating image displayed on the monitor is consistent with the doctor's operation. The direction of the force F' felt by the operation part 210 is basically the same, and the experience is close to intuition.

In some embodiments, the above-mentioned safe space is configured to include more than one set space. For example, the safe space includes more than one of a first setting space, a second setting space, a third setting space and a fourth setting space. For example, as shown in Figure 9, the first set space includes an accessible space based on hardware limits of the joint components in the first device. Illustratively, the second setting space includes an accessible space based on the barrier-free working space of the joint components in the first device. The second setting space is less than or equal to the first setting space, mainly considering objective environmental factors such as substance. The existence of walls and other obstacles to the end of the first device. For example, the third setting space includes an accessible space based on software limits of the joint components in the first device. Exemplarily, the fourth set space includes an accessible space based on the field of view space of the imaging instrument.

When the safe space is configured to include more than two setting spaces, in order to ensure relative safety, an exemplary safe space that can be adopted includes the overlapping space (ie, intersection area) between these setting spaces. domain), the boundary of the safe space corresponds to the boundary of the overlapping space.

In some embodiments, for the case where the safe space includes multiple setting spaces, the priority of the setting space that considers physical condition restrictions is relatively higher than the priority that considers software condition restrictions. Taking the above-mentioned first to fourth setting spaces as an example, the first setting space and the second setting space take into account the physical condition restrictions, and the third setting space and the fourth setting space take into account the software condition restrictions, so the The priorities of the first setting space and the second setting space are relatively higher than the priorities of the third setting space and the fourth setting space. Among them, a setting space with a higher priority generally does not allow the end of the first device to exceed its boundary.

For situations where the safe space includes multiple set spaces, it is usually desirable to prohibit the end of the first device from operating beyond the boundaries of any set space. Although a safe space with multiple set spaces is beneficial in most cases, When the safe space includes multiple setting spaces, the restriction effect on the range of movement of the end of the first device is also obvious. Therefore, when multiple setting spaces have different priorities, allowing the end of the first device to appropriately exceed the boundary of the setting space with a lower priority to appropriately expand the range of movement of the end of the first device mainly includes allowing it to appropriately Beyond the boundaries of the setting space that considers software condition limitations, such as the boundaries of the above-mentioned third setting space and/or the fourth setting space. When the end of the first device reaches the boundary of the lower-priority setting space, for example, it can be determined through human-computer interaction whether to allow exceeding the boundary of the setting space; for another example, the setting can be allowed/transcended by default. The boundaries of space.

In some embodiments, when the safe space includes multiple setting spaces, especially when these setting spaces have different priorities, it may be possible to allow the end of the first device to exceed the boundary of the setting space with a lower priority. Same or different virtual forces (or real forces). For example, it can be configured such that the lower the priority, the smaller the corresponding virtual force (or real force).

In some embodiments, the above-mentioned safe space can be constructed into one of a variety of three-dimensional structural spaces according to the structural characteristics of the first device (including link parameters and joint parameters). The above-mentioned safe space can also be combined with the composition of the first device. Features and software conditions are constructed into one of a variety of three-dimensional structured spaces. These three-dimensional structure spaces include regular or irregular three-dimensional structure spaces. Preferably, these three-dimensional structure spaces include regular three-dimensional structure spaces, for example, including one of a sphere space, a cone space, and a cylinder space. For example, for a first device including manipulator assembly 120, the first device's The end is the end instrument 150. For example, the safety space of the end instrument 150 can be constructed as a cylindrical space, which helps to determine the transboundary direction of the end instrument 150 more easily.

In some embodiments, the above step S11, that is, applying a virtual force to the end of the first device, includes:

Obtain a position point where the end of the first device exceeds the boundary of the safe space of the end of the first device; determine a target direction for applying a virtual force based on the position point; apply a virtual force to the end of the first device in the target direction.

For example, the target direction includes a direction from the location point to the center of the safe space. For example, referring to Figures 9 and 10, when the safe space includes a cylindrical space, the target direction includes a direction from the location point to the central axis of the cylindrical space. For example, the target direction includes a direction from the location point to the central axis of the cylindrical space. Vertical points in the direction of the central axis of the cylindrical space.

Wherein, the virtual force exerted on the end of the first device includes a virtual resultant force, and the virtual resultant force includes at least two virtual component forces. For example, continuing to refer to Figure 10, assume that the end of the first device, such as the terminal instrument 150, exceeds the boundary of its safe space in the xy plane of Cartesian space, and the obtained position point when exceeding the boundary includes P(x, y), According to the trigonometric function of tanθ=Py/Px, the angle between the virtual component force Fx and the virtual resultant force F, such as θ, can be determined. Furthermore, for example, the virtual component force Fx=Fcosθ on the positional degree of freedom x, and the virtual component force on the positional degree of freedom x can be determined. The virtual component force Fy=Fsinθ on degree y, that is, F(Fcosθ,Fsinθ).

In the subsequent processing, it includes coordinate transformation of these virtual components to obtain the corresponding real components, and then using dynamics or kinematics, such as the Jacobian in the force domain in kinematics to calculate these real components. The target joint driving force of the joint component in the second device.

In some embodiments, during surgery, in the master-slave follower mode, the doctor uses the operating part 210 to manipulate the manipulator assembly 120. Therefore, the first device can be configured as the manipulator assembly 120, and then the end of the first device includes a terminal instrument. 150. Medical instruments 140 in manipulated manipulator assembly 120 include imaging instruments 141 and/or surgical instruments 142 . The controller can be configured to:

First information that the terminal instrument exceeds the boundary of the safe space of the terminal instrument is obtained, and the doctor's operating proficiency level is evaluated based on the first information. Feedback to the doctor on his or her operating proficiency can be provided during or after the operation through at least one of visual, auditory, and tactile means. level.

The first information includes at least one of the out-of-bounds position, out-of-bounds times, and out-of-bounds time. The out-of-bounds position can be calculated using forward kinematics. The number of overruns includes the number of overruns associated with the overrun position and/or the total number of overruns. The total number of out-of-bounds attempts can be obtained directly by counting once for each out-of-bounds position, or by calculating the sum of the number of out-of-bounds attempts at each out-of-bounds position. The out-of-bounds time includes the residence time of the end device associated with the out-of-bounds position, including a single out-of-bounds time and/or the total out-of-bounds time, where the total out-of-bounds time includes the total out-of-bounds time in an out-of-bounds position. of out-of-bounds time and/or the total out-of-bounds time in all out-of-bounds locations. During a period of surgery, the doctor's operating proficiency can be judged based on the out-of-bounds position. For example, if the out-of-bounds position appears in an area or position that should not appear during normal surgery, it can be determined that the doctor's operating proficiency is poor; if the out-of-bounds position does not appear in an area or position that should not occur during normal surgery, it can be Make sure the doctor's operating proficiency is better.

During a period of surgery, the doctor's operating proficiency can be judged based on the total number of exceedances. For example, if the number of exceedances does not reach the first threshold, it can be determined that the doctor's operational proficiency is excellent; if the number of exceedances reaches the first threshold but does not reach the second threshold, it can be determined that the doctor's operational proficiency is good; if If the number of exceedances reaches the second threshold but does not reach the third threshold, the doctor's operational proficiency can be determined to be qualified; if the number of exceedances reaches the third threshold, the doctor's operational proficiency can be determined to be unqualified. Wherein, the first threshold, the second threshold and the third threshold increase step by step. Of course, you can also configure fewer levels of threshold levels to simply grade doctors' operating proficiency; you can also configure more levels of threshold levels to conduct detailed grading of doctors' operating proficiency.

During a period of surgery, the doctor's operating proficiency can be judged based on the number of boundary crossings associated with the boundary crossing positions. For example, if the out-of-bounds position appears in an area or position that should not occur during normal surgery and the number of times it exceeds the threshold does not reach the first threshold, it can be determined that the doctor's operating proficiency is excellent; if the out-of-bounds position appears in a region that should not occur during normal surgery If the number of out-of-bounds occurrences in an area or position reaches the first threshold but does not reach the second threshold, it can be determined that the doctor's operating proficiency is good; if the out-of-bounds position appears in an area or position that should not appear during normal surgery, the number of out-of-bounds exceeds If the second threshold value and the third threshold value are not reached, it can be determined that the doctor's operational proficiency is qualified; if the out-of-bounds position appears in an area or position that should not appear during normal surgery and the number of out-of-bounds exceeds the third threshold, it can be determined that the doctor's operational proficiency is qualified. Unsatisfactory degree. Wherein, the first threshold, the second threshold and the third threshold increase step by step. Of course, you can also configure fewer levels of threshold levels to simply grade doctors' operating proficiency; you can also configure more levels of threshold levels to conduct detailed grading of doctors' operating proficiency.

During a period of surgery, the surgeon's operating proficiency can be judged based on the out-of-bounds time associated with the out-of-bounds position. For example, assuming that the out-of-bounds time (such as a single out-of-bounds time at more than one out-of-bounds position) does not reach the first threshold, it can be determined that the doctor's operating proficiency is excellent; if the out-of-bounds time reaches the first threshold and does not reach The second threshold can determine that the doctor's operational proficiency is good; if the over-time time reaches the second threshold but does not reach the third threshold, it can be determined that the doctor's operational proficiency is qualified; if the over-time time reaches the third threshold, the doctor's operational proficiency can be determined Operational proficiency is unqualified. Wherein, the first threshold, the second threshold and the third threshold increase step by step.

In addition, it can also be configured to include more evaluation factors to conduct a more detailed assessment of the doctor's operational proficiency. For example, the further evaluation factors include surgical procedures, and different surgical procedures are usually associated with different safety spaces corresponding to the end instruments. Evaluating a doctor's proficiency in a specific surgical procedure will make the evaluation results more targeted and referenced.

For ease of understanding, the examples of surgical procedures are divided into major categories, including otolaryngology surgery, prostate surgery, nephrology surgery, gastrointestinal surgery, hepatobiliary surgery, and thoracic surgery. , gynecological surgery procedures, cardiac surgery procedures. Furthermore, examples of surgical procedures are divided into subcategories, taking hepatobiliary surgery as an example, including liver transplantation, liver lobectomy, cholecystectomy, and pancreaticoduodenectomy. type, splenectomy surgery, etc. For example, for different subcategories of surgical procedures covering hepatobiliary surgery, they are often associated with different safe spaces corresponding to end instruments.

In some embodiments, the doctor's operational proficiency can be evaluated based on the current surgical procedure associated with the safe space and the first information of the terminal instrument exceeding the boundary of the safe space. The evaluation results include the following: operational proficiency.

Wherein, the first information includes at least one of the out-of-bounds position, out-of-bounds times, and out-of-bounds time. The number of overruns includes the number of overruns associated with the overrun position and/or the total number of overruns. Total number of overruns The number can be obtained directly by counting once for each out-of-bounds position, or by calculating the sum of the number of out-of-bounds times at each out-of-bounds position.

For example, the doctor's operating proficiency can be judged based on the total number of exceedances. For example, during a period of surgery corresponding to the surgical procedure, if the number of exceedances does not reach the first threshold, it can be determined that the doctor's surgical proficiency under the surgical procedure is excellent; if the number of exceedances reaches If the first threshold value reaches the second threshold value but does not reach the second threshold value, it can be determined that the doctor's surgical proficiency in this surgical procedure is good; if the number of exceedances reaches the second threshold value but does not reach the third threshold value, it can be determined that the doctor's surgical proficiency in this surgical procedure is good. The operation proficiency of the operation under the formula is qualified; if the number of exceedances reaches the third threshold, it can be determined that the doctor's operation proficiency in the operation under the operation formula is unqualified.

In some embodiments, the setting of the above threshold level, for example, the setting of the first to fourth thresholds, may be based on multiple operation data of the same doctor in the same safe space associated with the same surgical procedure, and/or different doctors. Single or multiple operation data to conduct big data analysis and determination. Wherein, the operation data includes the out-of-bounds position and/or the number of out-of-bounds times. Optionally, the operation data may also include more than one of operation time, operation success, operation interruption time, and operation interruption times.

Based on the first information obtained above, the controller can not only evaluate the doctor's operating proficiency during surgery, but can also implement other functions related to the surgical procedure and/or the safe space.

The above embodiments are not only applicable to the surgical robot system having the first slave operating device 100 as shown in Figure 2, but also to the surgical robot system having the second slave operating device 100' as shown in Figure 24. The surgical robot system associated with Figure 1 includes a single-hole surgical robot system, and the surgical robot system associated with Figure 24 includes a multi-hole surgical robot system. . The driving arm of the first slave operating device 100 and the driving arm of the second slave operating device 100' have substantially the same structure. The second slave operating device 100' also includes a robot arm 110' and a manipulator assembly 120'.

Wherein, whether it is the first slave operating device 100 or the second slave operating device 100', for the manipulator assembly 120 (120'), the robotic arm 110 (110') includes a plurality of shared joint components. The robotic arm 110 Movement of these shared joint components in (110') will cause common movement of multiple manipulator components 120 (120'). For example, when performing surgery in the master-slave follower mode, the doctor can use the operating part 210 to manipulate the manipulator assembly 120 (120') and then manipulate the terminal instrument to perform the surgery. The joint components between different manipulator assemblies 120 (120') are independent of each other, so the movements between different manipulator assemblies 120 (120') are relatively independent, that is, the movement of one of the manipulator assemblies 120 (120') does not Movement of other manipulator components 120 (120') may result.

The difference between the first slave operating device 100 and the second slave operating device 100' is reflected in:

Compared with the robotic arm 110' of the second slave operating device 100', the robotic arm 110 of the first slave operating device 100 includes more effective degrees of freedom of joint components. For example, the robotic arm 110 of the first slave operating device 100 includes five Effective degrees of freedom, while the robotic arm 110' of the second slave operating device 100' includes three effective degrees of freedom.

Compared with the manipulator 130' of the second slave operating device 100', the manipulator 130 of the first slave operating device 100 includes joint components with fewer degrees of freedom. For example, the manipulator 130 of the first slave operating device 100 includes a joint assembly that connects and drives The power mechanism of the medical instrument 140 provides an effective degree of freedom along the axial direction of the puncture device 160; and the manipulator 130' of the second slave operating device 100' includes an RC (Remote Center) arm and A power mechanism is slidably disposed at the distal end of the RC arm. The power mechanism connects and drives the medical instrument 140. The power mechanism provides an effective degree of freedom along the axial direction of the puncture device 160'. The RC arm provides an axis around the puncture device 160'. At least two effective degrees of freedom (for example, including pitching degrees of freedom and yaw degrees of freedom) that rotate toward (for example, around the remote center of motion in the axial direction of the puncture device 160'), and the RC arm is constructed using the principle of a parallelogram mechanism. Compared with the medical instrument 140' of the second slave operating device 100', the medical instrument 140 of the first slave operating device 100 includes joint components with more effective degrees of freedom. For example, the medical instrument 140 of the first slave operating device 100 includes other than the The medical instrument 140' of the second slave operating device 100' includes three effective degrees of freedom in addition to the three effective degrees of freedom provided by its manipulator 130'. degrees of freedom. The second operating device may also include a plurality of adjustment arms 170' that have different functions from the robotic arm 110' and the manipulator assembly 120', are independent and connected between the robotic arm 110' and the plurality of manipulators 130', and are suitable for The manipulator assembly 120' makes a reasonable range of precise adjustments. In addition, one puncturer 160 in the first slave operating device 100 is connected to the distal end of the robotic arm 110, and multiple puncturers 160' in the second slave operating device 100' are respectively connected to the distal end of the RC arm.

Among them, the description of the effective degrees of freedom is usually based on the Cartesian space description. Please refer to the content recorded in Chinese Patent Publication No. CN110463379A, which can be fully cited in this application.

Another difference between the first slave operating device 100 and the second slave operating device 100' is that multiple medical instruments 140 in the first slave operating device 100 are inserted through the same puncture device 160 connected to the living body. In the living body, it requires fewer incisions or natural orifices for the puncture device 160 to connect, which is conducive to the patient's postoperative recovery; the plurality of medical instruments 140' in the second slave operating device 100' respectively pass through different punctures connected to the living body. The instrument 160' is inserted into a living body, which requires relatively more incisions or natural orifices for the puncture instrument 160' to connect.

In some embodiments, the manipulator assembly 120 in the first slave operating device 100 can independently provide a relatively sufficient effective degree of freedom for the terminal instrument 150. "More sufficient" includes meeting the requirements for the implementation of the corresponding surgical procedure, and does not mean that it is actually effective. The number of degrees of freedom serves as the only measure. For example, the translation of the manipulator 130 (also referred to as the power mechanism) relative to the robotic arm 110 provides the end instrument 150 with an effective degree of freedom, which includes a degree of freedom consistent with the axial direction of the puncture device 160; the medical instrument 140 itself The joint assembly provides the end instrument 150 with multiple additional effective degrees of freedom that are different from those provided by the manipulator 130 , such as an additional three, four, or five effective degrees of freedom.

Since the manipulator assembly 120 can independently provide sufficient effective degrees of freedom for the end instrument 150 to meet the implementation of various surgical techniques, for example, when the manipulator assembly 120 can independently provide six effective degrees of freedom for the end instrument 150, therefore , in most scenarios, the posture of the puncturer 160 can be locked, that is, the position and orientation of the puncturer 160 can be maintained, and only the manipulator assembly 120 can be manipulated, which can effectively reduce the stress that may be generated by the movement of the puncturer 160 on the living body. s damage.

In some embodiments, for the different surgical procedures illustrated above, multiple different types of surgical procedures may be allowed to be performed through the same incision or natural orifice of the living body. For example, it may be possible to allow renal surgery, gastric surgery, etc. Intestinal surgery and hepatobiliary surgery are performed through the same incision or natural orifice; it may also allow multiple different small types of surgeries in the same large type of surgery to be performed through the same incision or opening of the body. Surgery performed at the natural orifice, such as liver transplantation, liver lobectomy, and cholecystectomy may be permitted in hepatobiliary surgery. Surgery, pancreaticoduodenectomy, and splenectomy are performed through the same incision or natural orifice.

Continuing to refer to FIG. 5 , since the puncturer 160 is connected to the living body, when it is necessary to switch between surgical procedures related to the same incision or the same natural orifice, the puncturer 160 is allowed to move around the remote movement center at most, that is, the puncturer is maintained. 160 position, movement to change the orientation of the puncture device 160. When the puncturer 160 is connected to the robotic arm 110, the puncturer 160 passes through the remote movement center, that is, the remote movement center can usually be configured on the puncturer 160. For example, the remote movement center is configured to be between the puncturer 160 and the robot arm 110. The position where the incision or the natural orifice is connected is such that when the puncture tool 160 moves around the remote movement center, the incision or the natural orifice will not be damaged.

In some embodiments, the safe space of the end of the first device includes a range of motion of the end, and the range of motion is closely related to the composition of the first device. When the translation and/or rotation of the safe space relative to the reference coordinate system in Cartesian space is not considered, the safe space is a safe space in an absolute sense. The safe space is a safe space in a relative sense, taking into account the translation and/or rotation of the safe space relative to the reference coordinate system in Cartesian space. Since the safe space in this application often translates and/or rotates compared to the reference coordinate system, the safe space described in this application mainly includes safe space in a relative sense.

In some embodiments, taking the manipulator assembly 120 as the first device as an example, in an absolute sense, the end of the manipulator assembly 120, that is, the safe space of the end instrument 150, is related to the composition of the manipulator assembly 120; in a relative sense, the manipulation The end of the instrument assembly 120, that is, the safe space of the end instrument 150 is not only related to the composition of the manipulator assembly 120, but also related to the translation and/or rotation of the safe space relative to the reference coordinate system in Cartesian space. For example, the safe space of the end instrument 150 may be determined based on the base coordinate system of the manipulator assembly 120 . In an absolute sense, for example, different end instruments 150 may be configured to have different safety spaces. For example, the safety spaces of different end instruments 150 are determined based on different base coordinate systems associated with the manipulator assembly 120; for example, different The end instruments 150 may be configured to have the same safety space. For example, the safety spaces of different end instruments 150 are determined based on the same base coordinate system associated with multiple manipulator assemblies 120. For example, with continued reference to FIG. 3, in the first In the operating device, the same base coordinate system can be configured as multiple manipulators 130 When in the initial position, the proximal ends of the multiple manipulators 130 are on the same plane. For example, the origin of the base coordinate system is the center point of the same plane.

In some embodiments, in the scenario where the manipulator assembly 120 is used as the first device, please refer to FIG. 11 . For simplicity of expression, the manipulator assembly 120 is intentionally omitted and only the puncturer 160 is retained for illustration. As shown in FIG. 11 , since different surgical procedures usually require different orientations of the puncture device 160 , in a relative sense, the safe space of the end instrument 150 has differences related to the orientation of the puncture device 160 , that is, the safety space of the puncture device 160 is different. When the orientation changes relative to a certain reference coordinate system, the safety space of the end instrument 150 also has associated changes relative to the reference coordinate system. In other words, corresponding to different surgical procedures, the terminal instrument 150 usually has different safety spaces. For example, the reference coordinate system includes the base coordinate system of the robotic arm 110 .

For example, please continue to refer to FIG. 9 , the safe spaces of multiple different end instruments 150 inserted into the same puncture device 160 can be configured to have the same safe space. The same safe space can be, for example, the intersection space of the safe spaces of multiple different terminal instruments 150 . The same safe space is illustratively associated with the orientation axis of the trocar 160, which is sometimes also called the RC axis or the central axis. When the same safe space includes, for example, a cylindrical safe space, its central axis includes, for example, the orientation axis of the puncturer 160 . Therefore, there is usually a correlation between the orientation of the puncture device 160 and the surgical procedure. That is, a change in the orientation of the puncture device 160 will cause a change in the surgical procedure, or a change in the surgical procedure will cause a change in the orientation of the puncture device 160 . .

In some embodiments, when the doctor operates the operating part 210 to manipulate the manipulator assembly 120 to perform surgery, the controller is configured to: record the surgical procedure associated with the surgical procedure, and at the same time, evaluate and record the doctor's operating proficiency under the surgical procedure. Spend.

In some embodiments, considering the different relative posture relationships between the first slave operating device 100 and the operating bed on which the patient lies, and/or the differences in the patient's individual characteristics (including body shape, gender, age, etc.), it is possible to By continuously training a model including the associated surgical procedures and the orientation of the corresponding puncture device 160 , such as using a convolutional neural network model to train, a specific posture corresponding to the first slave operating device 100 and the operating table is obtained. relationship and/or a more accurate surgical procedure with specific individual signs and the corresponding orientation of the puncture device 160 .

In some embodiments, the controller may also be configured to:

Obtain patient information. The patient information includes the patient's identity information and information about the target surgical procedure to be accepted. The target surgical procedure may be pre-evaluated. For example, the target surgical procedure is cholecystectomy.

Get doctor information. Among them, the doctor information includes identity information and operation proficiency information of doctors associated with multiple surgical procedures.

Combining patient information and doctor information to recommend appropriate doctors to patients. This includes matching from multiple surgical procedures a doctor whose operating proficiency in relation to the target surgical procedure reaches a preset level. For example, it is assumed that the operating proficiency includes four levels from low to high: unqualified, qualified, good and excellent. The preset level can be configured as good, for example, so the operating proficiency under the target surgical procedure can be reached to good and excellent. Excellent doctors recommend to patients. When there are multiple recommended doctors, it is preferable to recommend them in order from high to low based on the level of operational proficiency.

In some embodiments, the patient information also includes the patient's surgery time, and the doctor information also includes the doctor's schedule time. When recommending a doctor to perform the surgery for the patient, the controller can also be configured to:

From multiple surgical procedures, a doctor whose operating proficiency related to the target surgical procedure reaches a preset level and whose shift time is consistent with the time to receive the surgery is matched.

For example, the recommendation can still be given priority in order of operating proficiency levels from high to low; for another example, the patient's surgery time and/or the doctor's scheduling time can be the first factor, and the doctor's operating proficiency can be the third factor. Two factors are comprehensively ranked to make recommendations. In the first factor, the patient's operation time and the doctor's schedule should not conflict, that is, the two times should match. More specifically, in the patient's operation time, the doctor's schedule The shift time is free without other important schedules, such as no other surgical schedules. Of course, when there is a conflict between the recommended doctor's schedule and the patient's surgery time, the patient's surgery time can be rescheduled based on the above patient information and doctor information.

The above recommendations are usually implemented before preoperative planning, and the recommended information can be provided, for example, to a hospital management system (Hospital Information System) that can communicate with the surgical robot system. HIS), so that the hospital can arrange (i.e. configure) doctors and/or patients through the HIS and draw on the above recommended information, for example, arrange doctors, including allocating patients, allocating receiving surgery time and allocating surgical robot systems. More than one type; for another example, arranging patient arrangements includes allocating doctors, allocating surgery time, and allocating more than one surgical robot system.

Further, multiple surgical robot systems communicate with the HIS, and the controller in the surgical robot system can be configured to: obtain the information associated with a certain patient configured by the HIS system, which information includes the assigned doctor and the assigned receiving surgery time. and more than one of the assigned surgical robotic systems. The controller is further configured to: for the assigned surgical robot system, only the assigned doctor is allowed to log in to the surgical robot system during the assigned operation time to operate the surgical robot system. Such a setting can prevent other doctors from occupying a specific surgical robot system at a specific time and affecting the performance of operations on specific patients.

In some embodiments, the controller can also predict the orientation of the puncture device 160 based on the doctor's manipulation of the manipulator assembly 120 through the operating part 210 (equivalent to predicting the surgical procedure). Therefore, as shown in Figure 12, the controller can also be configured to perform:

Step S21: Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device.

The first information includes at least one of the out-of-bounds position, out-of-bounds times, and out-of-bounds time. The number of overruns includes the number of overruns associated with the overrun position and/or the total number of overruns. The total number of out-of-bounds attempts can be obtained directly by counting once for each out-of-bounds position, or by calculating the sum of the number of out-of-bounds attempts at each out-of-bounds position. The out-of-bounds time includes the residence time of the end device associated with the out-of-bounds position, including a single out-of-bounds time and/or the total out-of-bounds time, where the total out-of-bounds time includes the total out-of-bounds time in an out-of-bounds position. of out-of-bounds time and/or the total out-of-bounds time in all out-of-bounds locations.

Step S22: Predict the target orientation of the puncture device based on the obtained first information.

Different orientations of the trocar 160 are generally associated with different surgical procedures, and the target orientation is usually different from the current orientation.

In some embodiments, predicting the target orientation of the puncturer based on the obtained first information includes: in response to obtaining the first instruction, predicting the target orientation of the puncturer based on the obtained first information.

In some embodiments, the first instruction may include a first instruction processed and obtained by the controller, that is, the prediction of the orientation of the puncturer 160 is automatically triggered. For example, during the operation during the operation under the current orientation of the puncture device 160, the controller may determine, based on the first information, a first section in which the end instrument 150 frequently exceeds the boundary of its safe space. In one section, the controller can obtain the first instruction. The first section includes an area composed of more than one point on the boundary. Since the positions of these points themselves can be determined based on kinematics, and the first section is a set of points, the positions of all points covered by it can also be determined. of.

For example, the boundary can be divided into multiple sections in advance. The determination of the first section includes determining whether the number of boundary violations in the multiple divided sections reaches a threshold based on statistics. If the threshold is reached, determining the corresponding One section is the first section, where, during the operation period, the number of times exceeding the threshold in multiple sections may include more than 1, so the first section may include more than 1, such as 1 , 2 or more. For example, assuming that the boundary of the safe space (such as the boundary associated with the xy plane) is circular, the boundary can be divided into more than 2 sections, such as 2, 3, 4, 5, 6...120 sections A segment, of course, can also contain more segments. These sections may be configured to be equally divided, or partially equally divided, or not equally divided, for example, these sections may be configured to be equally divided sections. Generally, the more divided sections, the more beneficial it is to accurately predict the target orientation of the trocar 160 . As shown in Figure 13, for example, the boundary can be divided into sections 1 to 8 in advance. The end device 150 has exceeded the boundary in sections 1 to 8. According to statistics, the number of boundary violations in section 1 has occurred the most. Therefore, section 1 as shown in Figure 14 can be determined as the first section.

In addition, the determination of the first section may also include determining whether the out-of-bounds time in the plurality of statistically divided sections reaches a threshold. If the threshold is reached, determining the corresponding section as the first section.

For example, it is not necessary to divide the boundary into multiple sections in advance. The determination of the first section includes generating a warning on the boundary of the safe space and associated with it when the total number of boundary violations in a certain period reaches the first threshold. The normal distribution curve of the over-bound position and the number of over-bounds, and based on the normal distribution curve, determine the target interval where the over-bound probability reaches the second threshold, and use the position on the boundary of the safe space associated with the endpoint of the target interval, and then you can Determine the first section. In addition, after determining the target interval, you can Statistics are made on the number of exceedances within the target interval. When the number of exceedances reaches the third threshold, the first section is determined using the position on the boundary of the safe space associated with the endpoint of the target interval. As shown in Figure 15, there is no need to divide the boundary into sections in advance. The densest interval beyond the boundary is first determined as the target interval, and then the endpoint of the target interval is used to determine the first section as shown in Figure 16 on the boundary.

In addition, the determination of the first section may also include dividing the boundary into multiple sections without having to divide it into multiple sections in advance. The determination of the first section may include generating a signal in the safe space when the total out-of-bounds time within a certain period reaches the first threshold. on the boundary, a normal distribution curve associated with the over-bound position and over-bound time, and based on the normal distribution curve, determine the target interval where the over-bound probability reaches the second threshold, and use the boundary of the safe space associated with the endpoint of the target interval position, and then the first segment can be determined.

In addition, after the target interval is determined, statistics can be made on the number of out-of-bounds times or out-of-bound time times in the target interval. When the number of out-of-bounds times or out-of-bound time times reaches the third threshold, the safe space associated with the endpoints of the target interval can be used. The position on the boundary determines the first segment.

In some embodiments, the first instruction may include a first instruction input by a doctor to the controller through an input device, that is, the prediction of the orientation of the puncturer 160 may be manually triggered. The doctor can actively input this first instruction through any means. Exemplarily, the input device for inputting the first instruction includes at least one of the following devices, for example, a touch screen coupled to the controller, another example, a voice recognition device coupled to the controller, another example, and The operating part 210 coupled to the controller is, for example, a foot pedal coupled to the controller, an action recognition device (such as a gesture recognition device) coupled to the controller, or a brain coupled to the controller. Radio wave identification devices, and other devices that enable input.

In some embodiments, predicting the target orientation of the trocar based on the acquired first information includes: determining a first section based on the first information, and predicting the target orientation of the lancer based on the first section.

The method for determining the first section includes the methods mentioned above, which will not be described again here.

Exemplarily, predicting the target orientation of the puncture device based on the first section includes: matching from multiple surgical procedures a surgical procedure in which the safe space of its associated terminal instrument is located in the transboundary direction on one side of the first section as Target surgical procedure, and then determine the target orientation of the puncture device based on the target surgical procedure. Such target surgical procedure or the safe space of the end instrument or the target orientation of the puncture device can basically reflect the target surgical procedure. It reflects the doctor’s expectation for the adjustment direction of the safe space of the terminal instrument, which basically reflects the doctor’s expectation for the adjustment direction of the puncture device. For example, as shown in Figure 17, it is assumed that the safe spaces of the end instruments associated with multiple surgical procedures include the safe spaces A to E of the end instruments. If the safe spaces A, B, and C of the end instruments are located in the super-border direction, If the safe spaces D and E of the terminal instrument on one side of the first section are not located in the transboundary direction and are on the side of the first section, then the orientation of the puncture device associated with the safe spaces A, B, and C of the terminal instrument can be matched as the target. orientation.

Due to the orientation of the puncture device, the surgical technique and the safe space of the terminal instrument, any two are interrelated. Based on this association relationship, the surgical procedure associated with the safe spaces A, B, and C of the end device can be matched as the target surgical procedure, and/or the safe spaces A, B, and C of the end device can be matched as the target safe space. .

Exemplarily, predicting the target orientation of the puncture device based on the first section includes: matching the safe space of its associated end instrument from multiple surgical procedures to at least partially cover the surgical procedure of the first section as the target operation. The surgical procedure is used to determine the target orientation of the puncture device according to the target surgical procedure. For example, a surgical procedure that only needs to cover a part of the points of the first section may be regarded as the target surgical procedure. For another example, a surgical procedure whose proportion of points covering the first section reaches the first threshold can be used as the target surgical procedure. The predicted target surgical procedure may include more than one surgical procedure. For example, as shown in Figure 18, assuming that multiple surgical procedures include the safe spaces A' to D' of the terminal instruments associated with the surgical procedures, if the proportion of the safe space A' covering the first section reaches 100%, the safety The proportion of space B' covering the first section reaches 80%, the proportion of safe space C' covering the first section reaches 40%, and the proportion of safe space D' covering the first section is 0. For example, safe space A can be The surgical procedures associated with ', B', and C' are all used as target surgical procedures. For another example, a surgical procedure with a coverage ratio greater than 50% can be used as a target surgical procedure. At this time, the safe spaces A' ~ C' Among the related surgical procedures, only the surgical procedures related to the safe spaces A' and B' can be used as target surgical procedures.

For example, in order to predict the target orientation of the trocar 160 more accurately, predicting the target orientation of the trocar based on the first section may include: obtaining the first section; obtaining the operation image collected by the imaging instrument; identifying the operation image. of the organ; predict puncture by combining the first segment and the identified organ The target orientation of the device. Exemplary organs include heart, liver, spleen, lungs, stomach, gallbladder, pancreas, kidney, bladder, large intestine, duodenum, etc. Organs also include smaller features, such as the lobes in the liver. As shown in FIG. 19 , when the liver lobe, kidney, and duodenum are identified from the operation image, for example, the orientation of the puncture device corresponding to the liver lobe and kidney can be predicted as the target orientation based on the first section.

In some embodiments, predicting the target orientation of the trocar in conjunction with the first segment and the identified organ includes:

A first surgical procedure associated with the first section is matched from a plurality of surgical procedures; a second surgery associated with the safe space of the end instrument and the identified organ is matched from the first surgical procedure The surgical procedure is used as the target surgical procedure.

Exemplarily, matching the first surgical procedure associated with the first section from the multiple surgical procedures includes: matching the safe space of its associated terminal instrument from the multiple surgical procedures and located in the transboundary direction. The surgical procedure on one side of the first section is regarded as the first surgical procedure.

Exemplarily, matching the first surgical procedure associated with the first section from the plurality of surgical procedures includes: matching the safe space of its corresponding end instrument from the plurality of surgical procedures, which can at least partially cover the first surgical procedure. A section of surgical procedure is used as the first surgical procedure.

From the first surgical procedure, match the second surgical procedure whose associated safe space of the terminal instrument is associated with the identified organ as the target surgical procedure. In other words, exclude the surgical procedure associated with the unrecognized organ. formula, that is, the remaining surgical technique is used as the second surgical technique. The organ not being recognized includes that the organ is not within the operation image (that is, it is not within the field of view of the imaging device) and/or the organ is within the operation image but is not recognized due to incomplete features and other factors. Furthermore, by combining image recognition with the screening of the first surgical procedure, the prediction accuracy of the second surgical procedure can be improved and can better reflect the doctor's operating intention.

In some embodiments, after predicting the target surgical procedure and/or the target orientation of the puncturer, the controller may be configured to recommend the target surgical procedure and/or the target orientation of the puncturer to the doctor. For example, a user interface as shown in FIG. 20 may be generated to present corresponding recommendation information to the doctor. In the interface shown in FIG. 20 , the target surgical procedure related to the liver lobe and/or the target orientation of the puncture device is recommended to the doctor. Furthermore, the doctor can manually adjust the orientation of the puncture device according to the recommended target surgical procedure and/or the target orientation of the puncture device. The adjustment of the orientation of the puncture device includes adjusting the orientation axis (sometimes also called the central axis or center axis) of the puncture device. RC axis) to basically match the requirements of the target surgical procedure and/or the target orientation of the puncture device on the orientation of the orientation axis of the puncture device.

There are various means for the controller to recommend the target surgical procedure and/or the target orientation of the puncture device to the doctor. For example, a voice device or a display device coupled to the controller may be provided, and the information associated with the second surgical procedure and/or the target orientation of the puncture device may be played through the voice device, and/or the display device may be used to play the information. Display of information related to the second surgical procedure and/or the target orientation of the puncture device. Among them, for doctors, the recommended target surgical procedure may usually be easier for doctors to understand. Of course, for doctors who are skilled in using surgical robot systems, it is also acceptable to directly recommend the target orientation of the puncture device.

For example, the voice device includes speakers or headphones provided independently of the main operating console 200 and the slave operating device 100 . For another example, the voice device includes a speaker integrated in the main operating console 200 and/or provided in the slave operating device 100 .

For example, the display device includes a display provided independently of the master operating console 200 and the slave operating device 100 , such as a display on an image cart coupling the master operating console 200 and the slave operating device 100 . For another example, the display device includes a display integrated in the main operating console 200 and/or provided in the slave operating device 100 .

In some embodiments, after predicting the target orientation of the puncturer, the controller may be configured to: control the movement of the joint assembly in the robotic arm according to the target orientation to move the puncturer around the remote motion center and allow the puncturer to reach the target orientation. Wherein, making the trocar reach the target orientation includes making the orientation axis of the trocar reach the target orientation. Exemplarily, the target orientation of the puncture device includes a target orientation based on the base coordinate system of the robotic arm.

In some embodiments, after predicting the target surgical procedure, the controller may be configured to: obtain the orientation of the puncturer associated with the target surgical procedure as the target orientation, and control the movement of the joint components in the robotic arm according to the target orientation to make the puncturer Move around the remote center of motion and bring the orientation of the trocar to the target orientation.

Among them, controlling the movement of the joint components in the robotic arm according to the target orientation includes: converting the target orientation into joint variables of the joint components in the robotic arm through inverse kinematics; and then controlling the movement of the joint components in the robotic arm according to the corresponding joint variables to make the puncture device The orientation reaches the target orientation.

In some embodiments, when the second surgical procedure as the target surgical procedure includes multiple, the final determination of the target surgical procedure may include multiple implementations, wherein the final determination of the target surgical procedure includes selecting one of the second surgical procedures. The surgical procedure serves as the target surgical procedure.

For example, the final determination of the target surgical procedure may include the doctor's selection of the second surgical procedure. For example, the selection can be made through voice recognition performed by a voice recognition device. For example, when it is recognized that the information associated with the second surgical procedure includes, for example, a name, a number, etc., the second surgical procedure is determined as the target surgical procedure. For another example, information related to the second surgical procedure can be generated and displayed on the interface of the display, and the corresponding second surgical procedure is determined as the target by touching or squeezing the corresponding input device such as a touch screen, a button, a foot pedal, etc. Surgical procedures. Of course, there can also be other methods, such as using the brain wave identification device to perform brain wave recognition to make the selection. For example, when information related to the second surgical procedure is recognized, the second surgical procedure is determined as the target surgical procedure.

For example, the final determination of the target surgical procedure may include automatic selection of the second surgical procedure by a surgical robot system, such as a controller. For example, the surgical procedure with the highest correlation among the second surgical procedures can be used as the target surgical procedure by default. When the second surgical procedure includes one surgical procedure, the surgical procedure is unique and therefore has a higher degree of correlation. That is the highest. For another example, assuming that the closer the safe space associated with the terminal instrument is to the first section, the higher the degree of correlation, the second surgical procedure with the center of the safe space associated with the terminal instrument closest to the first section can be defined. as the target surgical technique to reduce the range of movement of the puncture device when it is adjusted.

In some embodiments, controlling the movement of the joint components in the robot arm according to the target orientation includes: after reaching the delay time, controlling the movement of the joint components in the robot arm according to the target orientation. The delay time can be configured from 0 to 120 seconds, for example. For example, when the delay time is configured to 0 seconds, once the target surgical procedure and/or the target orientation of the puncturer is determined, the movement of the joint component in the robotic arm can be immediately controlled to adjust the orientation of the puncturer. For example, when the delay time is configured to 30 seconds, after the target surgical procedure and/or the target orientation of the puncture device is determined and the delay reaches 30 seconds, the control robot arm The joint components move to adjust the orientation of the puncture device, and the configuration of the delay time allows the doctor to have sufficient time to re-determine the target surgical procedure. Of course, after the target surgical procedure and/or the target orientation of the puncture device is determined and the delay time is not reached, after the controller obtains the confirmation instruction sent by the doctor through an interactive method for immediately adjusting the orientation of the puncture device, that is, The robot arm movement can be controlled immediately without delaying the expiration of time.

In some embodiments, when the terminal instrument 150 frequently exceeds the boundary at the boundary, even in a scenario where the target surgical procedure and/or the target orientation is not required or cannot be predicted, automatic adjustment of the orientation of the puncture device can be configured. Or adjust manually.

In some embodiments, the controller may be configured to perform:

Step S31: Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device.

Wherein, the first information includes at least one of the out-of-bounds position, out-of-bounds times, and out-of-bounds time.

Step S32: Determine the target center point based on the obtained first information.

Determining the target center position based on the acquired first information includes: determining the position of a feature point on the first section as the target center point based on the determined first section. The feature point includes, for example, the center point of the first section. The determination of the first section includes any of the methods mentioned above, which will not be described again here.

Step S33, control the movement of the joint assembly in the robotic arm to make the puncturer move around the remote motion center and align the orientation of the puncturer with the target center point.

Wherein, the movement of the puncturer around the remote movement center includes the rotational movement of the puncturer around the remote movement center, which usually only moves on the attitude freedom. The orientation of the trocar to the target center point includes the orientation axis of the trocar passing through the target center point.

Among them, step S33 includes: obtaining the target orientation that is expected to be reached by the movement of the orientation axis of the puncture device, determining the target joint variables of the joint components in the robotic arm according to the target orientation, and then controlling the movement of the corresponding joint components in the robotic arm according to the target joint variables to enable puncture. The instrument performs RC movement and aligns the orientation of the puncture instrument with the target center point.

For example, the target orientation can be obtained in the following manner. The controller is configured to execute: get The orientation of the line formed by the remote motion center and the target center point is taken as the target orientation. The target orientation may be, for example, the target orientation in the base coordinate system of the robot arm.

Aligning the orientation axis of the puncture device with the target center point includes aligning the orientation axis of the puncture device with a line formed by the remote movement center and the target center point. Among them, the target can be determined based on the position of the target center point in the image end instrument coordinate system of the imaging instrument (sometimes also called the endoscope coordinate system) and the conversion relationship between the endoscope coordinate system and the base coordinate system of the robotic arm. The position of the center point in the base coordinate system helps to obtain the target orientation.

For example, the above target joint variables can be obtained in the following way. The controller is configured to: determine target joint variables for the joint components in the robotic arm using a combination of target orientation and inverse kinematics.

As shown in FIG. 21 , the dotted line image model of the safe space represents before the orientation of the puncturer 160 is adjusted, and the solid line image model of the safe space represents the adjustment of the orientation of the puncturer 160 until it is aligned with the target center point.

Through the above-mentioned steps S31 to S33, the orientation of the puncture device 160 can be automatically adjusted to prevent the terminal instrument 150 from frequently crossing the boundary of the safe space when the doctor operates under the current orientation of the puncture device 160. Furthermore, by continuously repeating the above steps S31 to S33, the orientation of the puncture device 160 can be continuously adjusted to finally achieve the orientation of the puncture device 160 desired by the doctor.

In some embodiments, the controller may be configured to perform:

Step S41: Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device.

Step S42: Determine the first section based on the first information, and determine the target organ in the operation image by combining the first section and the operation image.

For the sake of brevity, please refer to the previous section for a description of the first section.

Exemplarily, combining the first section and the operation image to determine the target organ in the operation image may include:

All organs within the operation image are identified, and a target organ is determined from the identified organs based on the first segment. That is, first identify all the organs in the operation image, and then select the The organ related to the first section is determined as the target organ. Among them, determining the target organ in combination with the first section mainly includes assigning a reasonable range to determine the target organ. For example, the range includes the range of the first section located on the side away from the orientation axis of the current puncture device. Based on this range, the target organ is determined. The corresponding organ is determined as the target organ.

For example, combining the first section and the operation image to determine the target organ in the operation image may also include:

An organ associated with the first segment within the operation image is identified and used as the target organ. That is, instead of identifying all the organs in the operation image, the organ related to the first section is directly identified as the target organ. This can reduce the amount of image processing and thereby increase the speed of image processing.

Step S43: Determine the target center point based on the identified target organ.

For example, the geometric center of the target organ can be determined as the target center point based on the contour information of the target organ.

Step S44, control the movement of the joint assembly in the robotic arm to make the puncturer move around the remote motion center and align the orientation of the puncturer with the target center point.

Among them, step S44 includes: obtaining the target orientation of the orientation axis movement of the expected puncture device, determining the target joint variables of the joint components in the robotic arm according to the target orientation, and then controlling the movement of the corresponding joint components in the robotic arm according to the target joint variables to make the puncture device Make an RC motion and align its orientation with the target center point.

For example, the target orientation can also be obtained by obtaining the orientation of the line formed by the remote motion center and the target center point as the target orientation. In some embodiments, the target organ may include more than one, so the target center point may include more than one, and the controller may be configured to determine one of the multiple target center points as the target center point according to the interactive instructions between the doctor and the surgical robot system. . Of course, the controller can also determine one of multiple target center points as the target center point by default according to preset rules. The preset rules exemplarily include: acquiring multiple targets between the remote motion center and multiple target center points. Orientation, determine the target center point associated with the current orientation of the puncture device and the one with the smallest difference among multiple target orientations as the target center point.

As shown in Figure 22, when the target organ includes the liver lobe, the center of the liver lobe is used as the target center point. The line image model of the safe space represents before the orientation of the puncturer 160 is adjusted, and the solid line image model of the safe space represents the adjustment of the orientation of the puncturer 160 until it is aligned with the target center point.

Through the above-mentioned steps S41 to S44, the orientation axis of the puncture device 160 can be aligned with the target center point of the target organ, so as to facilitate the operation on the target organ.

It is worth noting that when the orientation axis of the puncturer 160 is manually adjusted or automatically adjusted to move around the remote movement center, it is better to consider the safety that the end instrument 150 may bring when the medical instrument 140 follows the movement of the puncturer 160. risk.

In order to avoid the above safety risks as much as possible, for example, the plurality of medical instruments 140 passing through the trocar 160 can be returned to a safe position before adjusting the orientation axis of the trocar 160 to move around the remote motion center. For example, when the orientation axis of the puncturer 160 is adjusted to move around the remote movement center, in response to the change in the orientation of the puncturer 160 when it moves around the remote movement center, the joint components in the manipulator assembly 120 can be controlled to move cooperatively to maintain the terminal instrument. 150 position or posture.

In the medical instrument 140 , since the operating end instrument 152 of the surgical instrument 142 is more harmful to the living body than the image end instrument 151 of the imaging instrument 141 , in some embodiments, the puncture device 160 can be adjusted. When the orientation axis moves around the remote motion center, only the joint components in the manipulator assembly 120 associated with the surgical instrument 142 are controlled to move cooperatively to maintain the position and/or posture. At the same time, since the imaging instrument 141 is not controlled, the imaging instrument 141 follows the puncture. Movement of the puncture device 160 can create a new field of view to facilitate observation of changes in the orientation of the puncture device 160 .

Of course, in other embodiments, if it is necessary to maintain the current surgical field of view, the joint components in the manipulator assembly 120 associated with the surgical instrument 142 can also be controlled to move cooperatively to maintain the position and/or posture, and at the same time control the imaging instrument associated with it. The joint assemblies in manipulator assembly 120 of 141 move cooperatively to maintain position and/or posture.

In some embodiments, the target object manipulated by the operating part can also be changed according to the doctor's manipulation of the manipulator assembly by operating the operating part. Controllers can be configured to execute:

During the operation performed by the doctor under the current orientation of the puncture device, when the number of times the end instrument exceeds the boundary of its safe space in a certain period reaches the target threshold, the first operation mode is switched to the second operation mode.

The area may include an area on the boundary of a predefined safe space. This section of the region may also include a section of the pre-divided regions in which the number of exceedances reaches a set threshold. This section of the area may also include a section of the area determined by a method such as counting the number of out-of-bounds times and the normal distribution of out-of-bounds locations during a period of surgery. Continuing to refer to Figure 13, for example, within a certain period, such as 10 seconds, when the number of times the terminal instrument exceeds the boundary of section 1 of its safe space reaches the target threshold, such as 3 times, the first operation mode can be switched to Second operating mode.

Wherein, the first operating mode includes manipulation of the manipulator assembly by the operating part. The second operating mode includes the manipulation of the robotic arm by the operating part, that is, the manipulation of the distal end of the robotic arm, that is, the puncturer, wherein the manipulation of the puncturer is more preferably the manipulation of the remote movement center of the puncturer, Exemplarily, the manipulation of the trocar by the operating part includes manipulating the trocar to move around the remote motion center. Furthermore, by switching the operating mode, it is convenient for the doctor to quickly manually adjust the orientation of the orientation axis of the puncture device through the operating part. The doctor can adjust the orientation of the puncture device by operating the operating part without leaving the operating position. In particular, since there is obvious force feedback when the terminal instrument exceeds the boundary, it can clearly reflect the doctor's intention to switch operating modes and provide a good user experience.

For example, in the second operating mode, the orientation of the trocar may be configured to change following the orientation change of the operating part. For example, the orientation of the puncture device changes in the first orientation degree of freedom (such as the yaw degree of freedom) following the change of the first orientation freedom (such as the yaw degree of freedom) of the operating part, and the orientation of the puncture device changes in the first orientation degree of freedom. (such as pitching degree of freedom) changes following the change of the first orientation degree of freedom (such as pitching degree of freedom) of the operating part, and the orientation of the puncture device follows the first orientation of the operating part at the first orientation degree of freedom (such as rolling degree of freedom) Changes in degrees of freedom (such as rolling degrees of freedom).

For example, in the second operating mode, the orientation of the puncture tool may also be configured to change following the position change of the operating part. For example, the movement of the operating part in the first positional degree of freedom (such as the horizontal degree of freedom) is converted into the movement of the orientation of the puncture tool in the first orientation degree of freedom (such as the yaw degree of freedom), and the movement of the operating part in the third degree of freedom is converted. The movement of the two positional degrees of freedom (such as the vertical degree of freedom) is converted into the movement of the orientation of the puncture device in the second orientation degree of freedom (such as the pitching degree of freedom), and the operating part is converted into a third positional degree of freedom (such as the front and rear degrees of freedom). The movement in the orientation degree of freedom (direction degree of freedom) is converted into the movement of the orientation of the puncture tool in a third orientation degree of freedom (such as the rolling degree of freedom).

In the above embodiments, usually when it is desired to adjust the orientation of the puncture device, the controller needs to obtain an instruction. The instruction can be input by the doctor through interaction (such as voice, movement, brain waves, etc.), or the controller can configure a delay time and generate the instruction after the delay time expires. In response to obtaining the instruction, the controller controls the movement of the joint assembly in the robotic arm to adjust the orientation of the puncturer.

In the above embodiments, by adjusting the orientation of the puncture device, the operating space of the terminal instrument of the medical device relative to the reference coordinate system of the robotic arm can be greatly improved, thereby avoiding the inconvenience caused by the frequent occurrence of out-of-bounds problems.

In some embodiments, the controller may be configured to generate at least an image model of the first section where the terminal instrument frequently exceeds the bounds of its safe space determined above and display it on any of the aforementioned displays, for example, see FIG. 14, The image model shown in any one of Figures 16 to 20. By viewing the image model displayed on the monitor, the doctor can clearly understand his/her operation situation and/or determine the operation intention he wants to achieve. In other embodiments, the controller can also be configured to generate an image model of the boundary of the safe space of the end device, and highlight the determined first section, including through color, brightness, lines (including line shape, thickness) , strobe, etc. to highlight the differences. In some embodiments, in order not to affect the doctor's observation of the operation image, the controller may be configured not to display the above-mentioned image model when the first section cannot be determined.

In some embodiments, in order to facilitate the doctor's understanding of the predicted target surgical procedure and/or the target orientation of the puncture device, the controller may be configured to display the identified organ and/or the determined first section on the operating image. Feature points such as center points are highlighted, including highlighting the outline of the organ, highlighting the organ and/or the target center point on the first section on the operating image, so as to facilitate the doctor to determine his or her operating intention based on the information displayed in the auxiliary image.

In some embodiments, the above image model also includes the first part of the manipulator assembly 120 being manipulated. The first part exemplarily includes the terminal instrument 150. Of course, the first part may also include other components, such as those included in the medical instrument 140. The connecting component of the terminal instrument 150 (consisting of multiple joint components). The terminal instrument 150 may appear in various representations in the image model, for example, as an arrow, aperture, or an icon with almost the same structure as the terminal instrument 150 . End instrument 150 The position in the image model can be calculated based on forward kinematics. By generating an image model including the end instrument 150, it is more beneficial for the doctor to grasp the situation of the end instrument 150 and the boundary, for example, when it is desired to switch operating modes.

For example, when the doctor wants to adjust the orientation of the puncture device, combining the above-mentioned visual feedback and the force feedback of the operating part when exceeding the limit can help the doctor clarify his operation intention and reduce the possibility of misoperation.

In one embodiment, the present application also provides a control method for a surgical robot system. The control method includes: when the end of the first device exceeds the boundary of its safe space, applying a virtual force to the end of the first device; The virtual force of the end of the device in the first coordinate system is converted into the real force of the end of the second device in the second coordinate system. The first device includes one of the driving arm and the operating part, and the second device includes the driving arm and the operating part. the other; convert the real force of the end of the second device in the second coordinate system into the target joint driving force of the joint component in the second device; control the joint component in the second device to output the target joint driving force.

In one embodiment, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program is configured to be loaded by the processor and executed to implement the following steps: exceeding the end of the first device. When the boundary of its safe space is reached, apply a virtual force to the end of the first device; convert the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system, and the first device including one of the driving arm and the operating part, and the second device including the other of the driving arm and the operating part; converting the real force of the end of the second device in the second coordinate system into the target joint drive of the joint assembly in the second device Force; controls the joint component in the second device to output the target joint driving force.

In one embodiment, the present application also provides a control device for a surgical robot system. As shown in Figure 23, the control device may include: a processor (processor) 501, a communications interface (Communications Interface) 502, a memory (memory) 503, and a communication bus 504.

The processor 501, the communication interface 502, and the memory 503 complete communication with each other through the communication bus 504.

The communication interface 502 is used to communicate with other devices such as various sensors or motors or solenoid valves or other network elements of clients or servers.

The processor 501 is configured to execute the program 505. Specifically, it can execute the relevant steps in the above method embodiment.

Specifically, program 505 may include program code including computer operating instructions.

The processor 505 may be a central processing unit CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present application, or a graphics processor GPU (Graphics Processing Unit). ). The one or more processors included in the control device can be the same type of processor, such as one or more CPUs, or one or more GPUs; or they can be different types of processors, such as one or more CPUs and One or more GPUs.

Memory 503 is used to store programs 505. The memory 503 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 505 can be specifically used to cause the processor 501 to perform the following steps: when the end of the first device exceeds the boundary of its safe space, apply a virtual force to the end of the first device; position the end of the first device in the first coordinate system. The virtual force is converted into a real force at the end of the second device in the second coordinate system. The first device includes one of the driving arm and the operating part, and the second device includes the other of the driving arm and the operating part; The real force of the end in the second coordinate system is converted into the target joint driving force of the joint component in the second device; the joint component in the second device is controlled to output the target joint driving force.

The technical features of the above-described embodiments can be combined in any way. To simplify the description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, All should be considered to be within the scope of this manual.

The above-described embodiments only express several implementation modes of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all fall within the protection scope of the present application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

A surgical robot system, characterized by including:

drive arm;

The operating part has motion correlation with the driving arm;

A controller, coupled to the driving arm and the operating part, and configured to:

applying a virtual force to the end of the first device when the end of the first device exceeds the boundary of its safe space;

Convert the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system. The first device includes one of the driving arm and the operating part, and the second device including the other of the driving arm and the operating portion;

Convert the real force of the end of the second device in the second coordinate system into the target joint driving force of the joint component in the second device;

The joint assembly in the second device is controlled to output the target joint driving force so that the operator can feel resistance at the second device.
The surgical robot system of claim 1, wherein the drive arm includes a manipulator assembly, the operating portion is configured to manipulate movement of the manipulator assembly, the manipulator assembly includes a medical instrument, and the The first device includes the manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical device, and the virtual force of the end of the first device in the first coordinate system is converted into the end of the second device in The real force in the second coordinate system includes:

Convert the virtual force of the end instrument in the first coordinate system into a virtual force in the first intermediate coordinate system;

Determine the real force that needs to be applied to the operating part in the second intermediate coordinate system according to the virtual force of the terminal instrument in the first intermediate coordinate system;

Convert the real force of the operating part in the second intermediate coordinate system into the real force in the second coordinate system;

Wherein, the first coordinate system includes the base coordinate system of the medical device, the first intermediate coordinate system includes the endoscope coordinate system, the second intermediate coordinate system includes the display coordinate system, and the second coordinate system includes the base coordinates of the operating part. Tie.
The surgical robot system according to claim 1, wherein applying a virtual force to the end of the first device includes:

Obtain the position point where the end of the first device exceeds the boundary of its safe space;

Determine the target direction for applying the virtual force according to the position point;

The virtual force is applied to the end of the first device in the target direction.
The surgical robot system according to claim 3, wherein the safe space includes a cylindrical space, and the target direction includes a direction perpendicular from the position point to the central axis of the cylindrical space.
The surgical robot system according to claim 1, wherein the safe space includes more than two safe spaces defined based on different conditions. When the end of the first device exceeds the boundaries of different safe spaces, it moves toward the first device. The ends of the device exert different amounts of virtual force.
The surgical robot system according to claim 1, wherein the magnitude of the virtual force is the same as the magnitude of the real force; the real force is between 3N and 10N.
The surgical robot system of claim 1, wherein the drive arm includes a manipulator assembly, the operating portion is configured to manipulate movement of the manipulator assembly, the manipulator assembly includes a medical instrument, and the The first device includes the manipulator assembly, a distal end of the manipulator assembly includes an end instrument of the medical device, and the controller is further configured to:

Obtain first information that the end device exceeds the boundary of its safe space, and the first information includes more than one of the out-of-bounds position, the number of out-of-bounds times, and the out-of-bounds time;

Obtain the surgical procedure associated with the safe space of the end instrument;

The level of the doctor's operating proficiency under the surgical technique is determined based on the first information and the surgical technique.
The surgical robot system according to claim 7, wherein the surgical robot system is coupled with a hospital management system, and the controller is further configured to:

Send doctor information associated with the surgical procedure and with the doctor's operating proficiency level to the hospital management system, so that the hospital management system can match a suitable doctor for the patient based on the doctor information. .
The surgical robot system of claim 1, wherein the drive arm includes an interconnected robotic arm and a manipulator assembly, the operating portion is configured to manipulate the movement of the manipulator assembly, and the manipulator assembly Including a medical instrument, the first device includes the manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical instrument, the distal end of the robotic arm is equipped with a puncture device, and a plurality of the medical The instrument is inserted into the living body through the same puncturer, and different safe spaces of the end instrument are associated with the orientation of the puncturer. The controller is also configured to:

Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device. The first information includes at least one of the out-of-bounds position, the number of out-of-bounds times, and the out-of-bounds time;

A target orientation of the trocar is predicted based on the acquired first information.
The surgical robot system according to claim 10, wherein different orientations of the puncture device are associated with different surgical techniques, and the controller is further configured to:

Generate visual information and/or auditory information including the target orientation of the trocar and play it; or,

Based on the target orientation of the trocar, visual information and/or auditory information of a target surgical procedure associated with the target orientation of the trocar is generated and played.
The surgical robot system according to claim 9, characterized in that the puncture tool passes through a telecentric fixed point, and when the puncture tool is connected to the living body, the telecentric fixed point corresponds to the distance between the puncture tool and the living body. Where the biological body is connected, the controller is further configured to:

The motion of the joint assembly in the robotic arm is controlled according to the target orientation of the trocar to make the trocar move around the distal fixed point, and to make the orientation of the trocar reach the target orientation of the trocar.
The surgical robot system of claim 1, wherein the drive arm includes an interconnected robotic arm and a manipulator assembly, the operating portion is configured to manipulate the movement of the manipulator assembly, and the manipulator assembly Including a medical device, the first device includes the manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical device, and the distal end of the robotic arm is A puncture device is provided, and multiple medical instruments are inserted into the living body through the same puncture device. The puncture device passes through a telecentric fixed point. When the puncture device is connected to the living body, the telecentric fixed point is not fixed. The point corresponds to the position where the puncture device is connected to the living body, and the controller is further configured to:

Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device. The first information includes at least one of the out-of-bounds position, the number of out-of-bounds times, and the out-of-bounds time;

Determine the target center point based on the first information;

Control the motion of the joint assembly in the robotic arm to move the puncturer around the distal fixed point and align the orientation of the puncturer with the target center point.
The surgical robot system according to claim 12, wherein the medical instrument includes an imaging instrument, and determining the target center point based on the first information includes:

Obtain the operation image captured by the imaging device;

Determine the target organ in the operation image by combining the first information and the operation image;

The target center point is determined based on the target organ.
The surgical robot system according to claim 13, wherein the controller is further configured to:

The target organ and/or the target center point are highlighted in the operation image.
The surgical robot system according to claim 12 or 13, wherein the medical instrument includes a surgical instrument, and the controller is further configured to:

In response to changes in the orientation of the trocar when the trocar moves around the distal fixed point, movement of a joint component in the manipulator assembly is controlled to maintain the position or posture of the terminal instrument.
The surgical robot system of claim 1, wherein the drive arm includes an interconnected robotic arm and a manipulator assembly, the operating portion is configured to manipulate the movement of the manipulator assembly, and the manipulator assembly Including a medical instrument, the first device includes the manipulator assembly, the end of the manipulator assembly includes the end instrument of the medical instrument, the distal end of the robotic arm is equipped with a puncture device, and a plurality of the medical The instrument is inserted into the living body through the same puncture device, and the puncture device passes through a telecentric fixed point. When the puncture device is connected to the living body, the telecentric fixed point corresponds to At the position where the puncture device is connected to the living body, the controller is further configured to:

Obtain the first information that the terminal instrument exceeds the boundary of its safe space during the operation performed by the doctor under the current orientation of the puncture device. The first information includes at least one of the out-of-bounds position, the number of out-of-bounds times, and the out-of-bounds time;

The first operation mode is switched to a second operation mode, the first operation mode includes the manipulation of the manipulator assembly by the operation part, and the second operation mode includes the manipulation of the robot arm by the operation part.
The surgical robot system according to claim 16, wherein the second operating mode includes:

The orientation of the puncture device changes following the orientation change of the operating part; or,

The orientation of the puncture tool changes following the position change of the operating part.
The surgical robot system according to claim 16, wherein the controller is further configured to:

Determine a first section on the boundary of the safe space of the terminal instrument based on the first information;

Generate an image model of the first section and display it.
A computer-readable storage medium is characterized in that it is suitable for a surgical robot system. The surgical robot system includes a driving arm and an operating part, and has motion correlation with the driving arm. The computer-readable storage medium stores a computer A program, the computer program configured to be loaded and executed by the processor, implements:

When the end of the first device exceeds the boundary of the safe space of the end of the first device, apply a virtual force to the end of the first device, the direction of the virtual force being opposite to the direction in which the end of the first device exceeds the boundary;

Convert the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system. The first device includes one of the driving arm and the operating part, and the second device including the other of the driving arm and the operating portion;

Convert the real force of the end of the second device in the second coordinate system into the target joint driving force of the joint component in the second device;

Control the joint component in the second device to output the target joint driving force so that the operator can Resistance can be felt at the device.
A control device for a surgical robot system, which is characterized in that it is suitable for a surgical robot system. The surgical robot system includes a driving arm and an operating part, and has motion correlation with the driving arm. The control device includes:

Memory, used to store computer programs;

and a processor for loading and executing the computer program;

Wherein, the computer program is configured to be loaded and executed by the processor to implement:

applying a virtual force to the end of the first device when the end of the first device exceeds the boundary of its safe space;

Convert the virtual force of the end of the first device in the first coordinate system into the real force of the end of the second device in the second coordinate system. The first device includes one of the driving arm and the operating part, and the second device including the other of the driving arm and the operating portion;

Convert the real force of the end of the second device in the second coordinate system into the target joint driving force of the joint component in the second device;

The joint assembly in the second device is controlled to output the target joint driving force so that the operator can feel resistance at the second device.