CN219846790U - surgical robot system - Google Patents

Abstract

The present disclosure relates to the field of medical instruments, and discloses a surgical robot system, comprising: a main manipulator comprising a robotic arm and a handle disposed at a distal end of the robotic arm, the handle being movable in at least one degree of freedom; a display device for displaying at least one graphical element on the display interface, the at least one graphical element representing movement of the handle in at least one degree of freedom; and a control device in communication with the display device and the main operator, the control device configured to periodically receive the motion parameter of the handle in the first degree of freedom and to control updating the first display characteristic of the at least one graphical element presented on the display interface based on the motion parameter of the first degree of freedom.

Description

Surgical robot system

Technical Field

The present disclosure relates to the field of medical devices, and more particularly to a surgical robotic system.

Background

Laparoscopic surgical robotic systems are advanced medical devices that assist surgeons in performing surgery by controlling surgical tools. The surgeon can teleoperate the surgical tool through the main manipulator to complete the surgical procedure and view the surgical procedure in real time through the high definition visual display.

The advantage of teleoperational control is its high precision, stability and flexibility, which helps the surgeon to achieve more accurate, safe and efficient surgical procedures.

In actual surgery, the current movement of the handle of the primary manipulator is typically perceived by the operator himself during the surgery, lacking visual cues.

Disclosure of Invention

In some embodiments, the present disclosure provides a surgical robotic system comprising: a main manipulator comprising a robotic arm and a handle disposed at a distal end of the robotic arm, the handle being movable in at least one degree of freedom; a display device for displaying at least one graphical element on the display interface, the at least one graphical element representing movement of the handle in at least one degree of freedom; and a control device in communication with the display device and the main operator, the control device configured to periodically receive the motion parameter of the handle in the first degree of freedom and to control updating the first display characteristic of the at least one graphical element presented on the display interface based on the motion parameter of the first degree of freedom.

In some embodiments, the control device is further configured to control the updating of the display characteristic of the at least one graphical element to the alert characteristic in response to the movement parameter of the handle in the first degree of freedom reaching the limit condition.

In some embodiments, the control device is further configured to control the display interface to display a critical area in response to the motion parameter of the handle in the first degree of freedom reaching a limit condition, the critical area corresponding to the limit condition.

In some embodiments, the first degree of freedom is a horizontal degree of freedom, the motion parameter of the handle of the primary manipulator in the first degree of freedom comprises a position of the handle of the primary manipulator in the horizontal degree of freedom, the control device is configured to update the display characteristic of the at least one graphical element such that the at least one graphical element moves horizontally following a change in the position of the handle of the primary manipulator in the horizontal degree of freedom; or the first degree of freedom is a vertical degree of freedom, and the movement parameter of the handle of the main operator in the first degree of freedom comprises a position of the handle of the main operator in the vertical degree of freedom, the control means being configured to update the display characteristics of the at least one graphical element such that the at least one graphical element grows or shrinks following a change in the position of the handle of the main operator in the vertical degree of freedom.

In some embodiments, the control device is further configured to periodically receive a motion parameter of the handle of the main operator in the second degree of freedom, and to control updating a feature associated with the second degree of freedom of the at least one graphical element presented on the display interface based on the motion parameter of the second degree of freedom.

In some embodiments, the at least one graphical element includes a first linear graphical element corresponding to a vertical degree of freedom direction and a second linear graphical element corresponding to a horizontal degree of freedom direction;

the control means is configured to control the first linear graphical element to grow or shrink following a change in position of the handle of the main operator in a vertical degree of freedom and/or to control the movement of the first linear graphical element following the handle of the main operator in a horizontal degree of freedom to move along the second linear graphical element.

In some embodiments, the second line-type graphic element represents a reference of a vertical degree of freedom direction, and the first line-type graphic element grows or shrinks above or below the second line-type graphic element.

In some embodiments, the control device is configured to control the display of an actual image of the surgical instrument at the display interface, the movement of the surgical instrument being controlled by the handle of the primary manipulator; the actual image is displayed in combination with at least one graphical element to demonstrate movement of the handle of the primary manipulator in at least one degree of freedom and coordinated movement of the surgical instrument.

In some embodiments, the control device is configured to determine a position and/or an attitude of the handle of the primary manipulator and to determine a parameter of the handle of the primary manipulator in the first degree of freedom based on the position and/or the attitude of the handle of the primary manipulator.

In some embodiments, the handle comprises at least one joint, the control device is configured to periodically receive joint information of at least the joint, and determine the position and/or posture of the handle based on the joint information of the at least one joint.

According to the surgical robot system, the dynamic movement process of the handle in the first degree of freedom can be represented or reflected by updating the first display characteristic of at least one graphic element displayed on the display interface, and then the movement condition of the handle in the first degree of freedom is visually and intuitively prompted to an operator.

Drawings

Fig. 1 illustrates a schematic block diagram of a surgical robotic system according to some embodiments of the present disclosure;

fig. 2 illustrates a schematic view of a surgical robotic system according to some embodiments of the present disclosure;

FIG. 3 illustrates a schematic view of a surgical trolley according to some embodiments of the present disclosure;

FIG. 4 illustrates a schematic view of a surgical tool according to some embodiments of the present disclosure;

FIG. 5 illustrates a schematic diagram of an imaging tool according to some embodiments of the present disclosure;

FIG. 6 illustrates a schematic view of a distal flexible section of an operating arm according to some embodiments of the present disclosure;

FIG. 7 illustrates a schematic structural view of an operating arm according to some embodiments of the present disclosure;

FIG. 8 illustrates a schematic diagram of a drive module and a positioning arm according to some embodiments of the present disclosure;

fig. 9 illustrates a schematic diagram of a master trolley according to some embodiments of the present disclosure;

FIG. 10 illustrates a schematic diagram of a primary operator according to some embodiments of the present disclosure;

FIG. 11 illustrates a schematic block diagram of a control device according to some embodiments of the present disclosure;

FIG. 12 illustrates a schematic diagram of a display interface according to some embodiments of the present disclosure;

FIG. 13A illustrates a schematic diagram of a graphical element in a first state according to some embodiments of the present disclosure;

FIG. 13B illustrates a schematic diagram of a graphical element in a second state according to some embodiments of the present disclosure;

FIG. 13C illustrates a schematic diagram of a graphical element in a third state according to some embodiments of the present disclosure;

FIG. 13D illustrates a schematic diagram of a graphical element in a constraint according to some embodiments of the present disclosure;

FIG. 13E shows a schematic view of a graphical element in a constraint according to further embodiments of the present disclosure;

fig. 13F illustrates a schematic diagram of a graphical element in a fourth state according to some embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, and those skilled in the art will appreciate that the scope of the present disclosure is not limited to only these embodiments. Various modifications and variations of the present disclosure can be made on the basis of the following embodiments. Such modifications and variations are intended to be included within the scope of the present disclosure. Like reference numerals designate like parts among the various embodiments shown in the drawings of the present disclosure.

In this disclosure, the terms "first," "second," and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In this disclosure, the end proximal to the operator (e.g., physician) is defined as proximal, or posterior, and the end proximal to the surgical patient is defined as distal, or anterior, anterior. In this disclosure, the term "pose" may include a "position" or a "pose". The term "position" refers to the positioning of an object (e.g., a master manipulator) or a portion of an object (e.g., a handle of a master manipulator) in three dimensions (e.g., three translational degrees of freedom, such as along the cartesian X-axis, Y-axis, and Z-axis, respectively, may be described using cartesian X, Y and Z-coordinate changes). In this disclosure, the term "pose" refers to a rotational setting of an object or a portion of an object (e.g., three rotational degrees of freedom may be described using roll, pitch, and yaw).

Fig. 1 illustrates a schematic block diagram of a surgical robotic system 100 according to some embodiments of the present disclosure. As shown in fig. 1, the surgical robotic system 100 includes: a control device 110, a main operator 120, and a display device 130. In some embodiments, the master manipulator 120 comprises: a robotic arm (e.g., multi-degree of freedom robotic arm 1010 shown in fig. 10) and a handle (e.g., handle 1020 shown in fig. 10) disposed at a distal end of the robotic arm, the handle being capable of movement in at least one degree of freedom. In some embodiments, display device 130 is configured to present at least one graphical element (e.g., graphical element 1230 or graphical element 1240 shown in fig. 12 or graphical element 1300 shown in fig. 13A) on a display interface (e.g., display interface 1200 shown in fig. 12), the at least one graphical element representing movement of the handle in at least one degree of freedom. In some embodiments, the control device 110 is communicatively coupled to the display device 130 and the main operator 120, the control device 110 being configured to periodically receive a motion parameter of the handle in the first degree of freedom and to control updating the first display characteristic of the at least one graphical element presented on the display interface based on the motion parameter of the first degree of freedom.

In some embodiments, surgical robotic system 100 also includes at least one surgical instrument 140. The movement of the surgical instrument 140 is controlled by the handle of the main manipulator 120. For example, the control device 110 receives a motion control signal of a handle of the main operator 120 and generates a motion control instruction to the surgical instrument 140 according to the received motion control signal to control the motion of the surgical instrument 140.

In some embodiments, the control device 110 is, for example, a control device 1100 as shown in fig. 11. In some embodiments, the control device 110 may be integrated on a master or surgical trolley, for example, or in a distributed configuration. In some embodiments, the control device 110 is, for example, a control device on a master trolley and/or a surgical trolley. In some embodiments, the master trolley is, for example, master trolley 220 shown in fig. 2 or master trolley 900 shown in fig. 9. In some embodiments, the surgical trolley is, for example, surgical trolley 210 shown in fig. 2 or surgical trolley 300 shown in fig. 3. In some embodiments, surgical instrument 140 is, for example, surgical instrument 230 shown in fig. 2 or surgical instrument 830 shown in fig. 8. In some embodiments, surgical instrument 140 may include, for example, a surgical tool and/or an imaging tool. In some embodiments, the surgical tool is, for example, surgical tool 2301 as shown in fig. 2 or surgical tool 400 as shown in fig. 4. In some embodiments, the imaging tool is, for example, imaging tool 2302 as shown in fig. 2 or imaging tool 500 as shown in fig. 5.

Fig. 2 illustrates a schematic diagram of a surgical robotic system 200 according to some embodiments of the present disclosure. In some embodiments of the present disclosure, referring to fig. 2, surgical robotic system 200 may include a surgical trolley 210, a master trolley 220, and a surgical instrument 230. One or more surgical instruments 230 are mounted on the surgical trolley 210. In some embodiments, surgical instrument 230 may include an operating arm and a tip instrument disposed at a tip of the operating arm. In some embodiments, surgical instrument 230 is, for example, a surgical tool 2301 or an imaging tool 2302. The master cart 220 is communicatively coupled to the surgical cart 210 for controlling the surgical tool 2301 to perform a surgical procedure or for controlling the imaging tool 2302 to perform an intra-operative image acquisition procedure. In some embodiments, the master trolley 220 is connected to the surgical trolley 210 by a wired or wireless transmission. For example, the master cart 220 and the operation cart 210 may be connected by a cable. In some embodiments, surgical trolley 210 is, for example, surgical trolley 300 shown in fig. 3. In some embodiments, master trolley 220 is, for example, master trolley 900 shown in fig. 9.

In some embodiments, the surgical tool 2301 may include an operating arm and an end effector disposed at an end of the operating arm. In some embodiments, the imaging tool 2302 may include an operating arm and an imaging module disposed at an end of the operating arm. The surgical tool 2301 or the imaging tool 2302 may be provided on the surgical trolley 210 and driven by a corresponding drive module. The image of the operating arm of the surgical tool 2301 and its end effector acquired by the imaging module may be transmitted to the master trolley 220. In some embodiments, portions of the surgical tool 2301 or portions of the imaging tool 2302 may act as slave tools. In some embodiments, the master trolley 220 includes a master manipulator 2201, the master manipulator 2201 being used to teleoperate the surgical tool 2301 or the imaging tool 2302. In some embodiments, surgical tool 2301 is, for example, surgical tool 301 shown in fig. 3 or surgical tool 400 shown in fig. 4. Imaging tool 2302 is, for example, imaging tool 304 shown in fig. 3 or imaging tool 500 shown in fig. 5. In some embodiments, the primary operator 2201 is, for example, the primary operator 901 shown in fig. 9 or the primary operator 1000 shown in fig. 10.

Operation trolley

Fig. 3 illustrates a schematic diagram of an operating table vehicle 300 of some embodiments of the present disclosure. In some embodiments of the present disclosure, referring to fig. 3, a surgical trolley 300 includes: control devices (the control devices may be disposed in computer equipment and provided inside the surgical cart 300), a surgical cart chassis 302, a surgical cart housing 303, a system status display 305, a main column 306, a main beam 307, a positioning arm 308, a drive module 309, and the like. The surgical trolley chassis 302 is used to perform the movement and positioning functions of the surgical trolley 300. The surgical dolly case 303 is used to integrate surgical dolly electric parts inside. System status display 305 is used to display a user interface of the surgical trolley system and to receive user inputs. The main column 306 is vertically movable, and its top end is fixed to the main beam 307. The end of the main beam 307 has a beam cradle head, and the lower end of the beam cradle head is connected with a plurality of positioning arms 308. The positioning arm 308 carries a driving module 309, and the driving module 309 is used for loading the surgical tool 301 or the imaging tool 304 (the imaging tool 304 is, for example, a 3D electronic endoscope). In some embodiments, the surgical trolley 300 integrates a plurality of positioning arms 308, each positioning arm 308 having a plurality of motion joints. In some embodiments, the positioning arm 308 is, for example, the positioning arm 810 shown in fig. 8. In some embodiments, the drive module 309 is, for example, the drive module 820 shown in fig. 8. In some embodiments, the surgical trolley 300 is integrated with a plurality of surgical tools 301 and imaging tools 304, with portions of the operating arms 3011 and end effectors 3012 of the plurality of surgical tools 301 and portions of the operating arms 3041 and imaging modules 3042 of the imaging tools 304 entering the workspace through the sheath tube assembly 310.

Surgical tool

Fig. 4 illustrates a schematic diagram of a surgical tool 400 in accordance with some embodiments of the present disclosure. In some embodiments of the present disclosure, referring to fig. 4, a surgical tool 400 includes a drive transmission 490, an operating arm 440, and an end effector 460 disposed at the end of the operating arm. In some embodiments, the drive transmission 490 may receive drive from a drive module (e.g., drive module 309) to drive the movement of the operating arm 440. The driving transmission device 490 is used for being connected with the driving module, and the driving force of the driving module is transmitted to the operation arm 440 through the driving transmission device 490, so as to drive the operation arm 440 to realize the motion with multiple degrees of freedom. The drive module may also control the end effector 460 to perform surgical operations. In some embodiments of the present disclosure, end effector 460 may include, but is not limited to, a bipolar curved split-jaw effector, a bipolar elbow grasper effector, a monopolar curved scissors effector, a monopolar electric hook effector, a bipolar grasper effector, a needle holder effector, and a tissue grasper effector. In some embodiments, surgical tool 400 may be mounted, for example, in surgical trolley 210 shown in fig. 2 or surgical trolley 300 shown in fig. 3. In some embodiments, surgical tool 400 may be loaded onto drive module 820 shown in fig. 8.

Imaging tool

Fig. 5 illustrates a schematic diagram of an imaging tool 500 of some embodiments of the present disclosure. In some embodiments of the present disclosure, referring to fig. 5, an imaging tool 500 includes a drive transmission 590, an operating arm 540, and an imaging module 560 disposed at the end of the operating arm. In some embodiments, imaging tool 400 may be mounted, for example, in surgical trolley 210 shown in fig. 2 or surgical trolley 300 shown in fig. 3. In some embodiments, the imaging tool 500 may be loaded onto the drive module 820 shown in fig. 8.

Operating arm

In some embodiments, the surgical instrument includes an operating arm, such as a flexible operating arm. In some embodiments, the manipulator arm (e.g., manipulator arm 440 or manipulator arm 540) may comprise one or more distal flexible segments, and may have multiple degrees of freedom of manipulator arm, such as a manipulator arm that may achieve 6 degrees of freedom of motion. The flexible segments may be implemented by a variety of suitable structures, such as a continuum, a serpentine structure, a rod plus multi-joint structure, and the like. In the present disclosure, a continuous body is described as an example, but this does not constitute a limitation of the present disclosure.

Fig. 6 illustrates a schematic view of a distal flexible section 600 of an operating arm according to some embodiments of the present disclosure. The manipulator arm may comprise at least one deformable distal flexible section 600. As shown in fig. 6, the flexible distal section 600 includes a distal fixation disc 610 and a plurality of distal structural bones 620. The distal ends of the plurality of distal structural bones 620 are fixedly coupled to the distal fixation disk 610 and the proximal ends are coupled to a drive transmission (e.g., drive transmission 490). In some embodiments, distal fixation disc 610 may be of various shapes including, but not limited to, annular structures, disk-like structures, etc., and may be circular, rectangular, polygonal, etc. in cross-section. The drive transmission deforms the distal flexible segment 600 by driving the distal structural bone 620. For example, the drive transmission may place the distal flexible section 600 in a curved state as shown in fig. 6 by driving the distal structural bone 620. In some embodiments, a second end of the plurality of distal structural bones 620 is coupled to a drive transmission through the base plate 630. In some embodiments, similar to distal fixation disc 610, base disc 630 may be of various shapes including, but not limited to, annular structures, disc-like structures, etc., and may be circular, rectangular, polygonal, etc. in cross-section. In some embodiments, a distal spacer disk 640 is also included between the distal fixation disk 610 and the base disk 630, with a plurality of distal structural bones 620 passing through the distal spacer disk 640.

Fig. 7 illustrates a schematic structural view of an operating arm 700 according to some embodiments of the present disclosure. As shown in fig. 7, the manipulation arm 700 is a flexible manipulation arm that is deformable, and the manipulation arm 700 may include a manipulation arm tip 710 and a manipulation arm body 720. The manipulation arm body 720 may include one or more distal flexible segments, such as a first distal flexible segment 721 and a second distal flexible segment 722. In some embodiments, the first and second distal flexible segments 721, 722 may be similar in structure to the distal flexible segment 600 shown in fig. 6. In some implementations, as shown in fig. 7, the operating arm body 720 further includes a first straight rod section 723 between the first distal flexible section 721 and the second distal flexible section 722. The first straight shaft section 723 is connected at a first end to the base plate of the second distal flexible section 722 and at a second end to the fixed plate of the first distal flexible section 721. In some implementations, as shown in fig. 7, the lever body 720 further includes a second straight rod segment 724, the first end of the second straight rod segment 724 being connected to the base plate of the first distal flexible segment 721.

Driving module

Fig. 8 illustrates a schematic diagram of a drive module 820 and a positioning arm 810 according to some embodiments of the present disclosure. In some embodiments, the driving module 820 includes: a motor group unit (not shown in fig. 8), and a driving gear is driven by the motor group unit. For example, the linear motion mechanism is driven by a motor group unit. Referring to fig. 8, in some embodiments, drive module 820 also includes a motor housing 821, a connecting barrel 822, and an adapter barrel 823. In some embodiments, the motor pack unit is disposed within a motor pack housing 821, the distal end of the motor pack housing 821 is fixedly connected to the proximal end of a connecting barrel 822, the distal end of the connecting barrel 822 is detachably connected to an adapter barrel 823, and the proximal end of the drive transmission 831 of the surgical instrument 830 is detachably connected to the distal end of the adapter barrel 823. In some embodiments, the output shaft of the motor in the motor pack unit is coupled to the drive transmission 831 via the coupling of the adapter 823, and the drive transmission 831 converts the rotational motion input of the motor to a linear motion output. In some embodiments, the driving module 820 further includes a linear module 840. In some embodiments, the linear die set 840 is disposed at the end of the positioning arm 810. For example, a bracket 842 with a sliding rail of the linear module 840 is disposed at the end of the fourth link 814 of the positioning arm 810. In some embodiments, the motor housing 821 or the connecting barrel 822 is coupled to the linear module 840 to enable the advancement or retraction of the surgical instrument 830. In some embodiments, the connecting barrel 822 is connected to a slider 841 of the linear module 840. In some embodiments, a screw is rotatably disposed on the bracket body 842, and a screw driving motor is disposed at one end of the bracket body 842, and an output shaft of the screw driving motor is fastened to the screw through a coupling. The slider 841 is in threaded fit with the lead screw and is slidably disposed on the slide rail.

Main control trolley

Fig. 9 illustrates a schematic diagram of a master trolley 900 of some embodiments of the present disclosure. In some embodiments of the present disclosure, referring to fig. 9, a master trolley 900 includes: control means (the control means may be provided on a computer device, provided inside the console car 900), a main operator 901, a console car display (e.g., displays 902-904), and pedals (e.g., pedals 905-907). The control device is respectively in communication connection with the main operator 901, the main control trolley display and the pedal, and is used for carrying out signal interaction with the main operator 901, the main control trolley display and the pedal, and generating corresponding control instructions based on the collected control information. In some embodiments, the control device of the master trolley 900 is also in communication with a surgical trolley, such as the surgical trolley 300 shown in fig. 3 (e.g., in communication with the control device of the surgical trolley 300), for controlling the surgical tool 301 to perform a surgical operation or the imaging tool 304 to operate based on the operation signal of the master manipulator 901. In some embodiments, the control device of the master cart 900 may specifically be, for example, a master computer disposed inside the master cart 900. In some embodiments, the master trolley display includes a stereoscopic display 902, a master external display 903, a master touch display 904. The stereoscopic display 902 displays the operation part image and the system state prompt, the main control external display 903 is used for assisting in displaying the operation part image and the system state prompt, and the touch display 904 displays a software user interface of the main control trolley 900. In some embodiments, the images displayed by the stereoscopic display 902 or the master external display 903 may be determined based on the images acquired by the imaging module. In some embodiments, the master trolley pedal is used to collect input from both feet of a medical staff, including the structure of an electro-cutting pedal 905, an electro-coagulation pedal 906, a clutch pedal 907, and the like.

Main manipulator

In some embodiments, the main operator 901 generally includes a left main operator (e.g., for controlling a first operating arm) and a right main operator (e.g., for controlling a second operating arm) corresponding to left-handed operation of a medical staff member, respectively. In a practical scenario, the main operator 901 is used to collect operation inputs of a medical staff, which controls the movement of a surgical tool or an imaging tool in an operation area by teleoperation of the main operator 901, so as to realize medical operation. In some embodiments, the primary manipulator 901 comprises at least one joint. In some embodiments, the master manipulator 901 includes a multi-degree of freedom mechanical arm 9011, a master manipulator sensor is disposed at each joint on the multi-degree of freedom mechanical arm 9011, and joint information (e.g., joint angle data) is generated by the master manipulator sensor of each joint. In some embodiments, the multiple degree of freedom robotic arm 9011 has six degrees of freedom. In some embodiments, the main manipulator 901 further comprises a clamp 9012, the clamp 9012 being operable to control the opening and closing angle of the end instrument. In some embodiments, the primary manipulator 901 may be specifically the primary manipulator 1000 shown in fig. 10, as described in detail below.

Fig. 10 illustrates a schematic diagram of a primary operator 1000 of some embodiments of the present disclosure. As shown in fig. 10, in some embodiments, the main manipulator 1200 includes a multiple degree of freedom manipulator 1010 and a handle 1020, the multiple degree of freedom manipulator 1010 including a plurality of joints (10101-10107). In some embodiments, the joints of the multiple degree of freedom robotic arm 1010 include coupled joints and uncoupled joints. The coupling joint may refer to a joint for adjusting the position and posture of the main manipulator. The uncoupled joint may refer to a joint that can only be used to adjust the position or posture of the main manipulator. In some embodiments, the joints of the multi-degree of freedom robotic arm 1010 include a positional joint as well as a gestural joint as an orientation module of the master manipulator 1000, from which the pose of the handle 1020 may be determined. The position joints, by which the position of the handle 1020 can be determined, serve as positioning modules for the main manipulator 1000. For example, fig. 10 shows a main manipulator 1000 in which a first joint 10101, a second joint 10102, and a third joint 10103 are positional joints, the first joint 10101, the second joint 10102, the fifth joint 10105, the sixth joint 10106, and the seventh joint 10107 are posture joints, the first joint 10101 and the second joint 10102 are coupling joints that can adjust both the position of the main manipulator 1000 and the posture of the main manipulator 1000, and the fifth joint 10105 and the sixth joint 1016 and the seventh joint 10107 are uncoupled posture joints that can adjust only the posture of the main manipulator 1000.

In some embodiments, the base coordinate system of the main manipulator is set to b (not shown in the figure), and the coordinate system of the handle is set to d (not shown in the figure). In some embodiments, the base coordinate system b is a coordinate system established with the base virtual as a point, the orientation of which can be determined based on its physical configuration. Similarly, the coordinate system d of the handgrip is a coordinate system established with the handgrip virtually as a point, the direction of which can be determined based on its physical configuration. In some embodiments, the origin of the coordinate system d of the handle may coincide with the origin of the coordinate systems of the fifth, sixth, seventh joints. It will be appreciated by those skilled in the art that the position and/or attitude of the coordinate system d of the handgrip with respect to the base coordinate system b of the main manipulator may be determined by some or all of the joint information of the first to seventh joints. For example, the position and/or posture of the handle is calculated from the joint information and a mathematical structural model of the main manipulator.

In some embodiments, a primary manipulator sensor is used to obtain joint information q of the primary manipulator _{j_mp} (j is the number of the joint). In some embodiments, the jth joint information q _{j_mp} May include the angle value theta of the corresponding joint _{j_mp} . For example, acquiring joint information q of a first joint _{1_mp} Joint information q of the second joint _{2_mp} Joint information q of third joint _{3_mp} Joint information q of fourth joint _{4_mp} Joint information q of fifth joint _{5_mp} Joint information q of the sixth joint _{6_mp} Joint information q of seventh joint _{7_mp} . In some embodiments, the fourth joint is a slave joint of the third joint, and the joint angle of the fourth joint is in the same direction as the absolute value of the joint angle of the third joint. In some embodiments, the first, second and third joints are positional joints, corresponding joint information q _{1_mp} 、q _{2_mp} 、q _{3_mp} The position of the handle of the main operator is determined. The first joint, the second joint, the fifth joint, the sixth joint and the seventh joint are gesture joints, and corresponding joint information q _{1_mp} 、q _{2_mp} 、q _{5_mp} 、q _{6_mp} 、q _{7_mp} The posture of the handle is determined.

In some embodiments, the mathematical structural model of the main manipulator may be constructed based on a D-H parametric method or an exponential product representation. For example, a D-H matrix corresponding to a joint of the primary manipulator is determined, and a mathematical structural model of the primary manipulator is determined based on the D-H matrix of the joint. The D-H matrix of each joint of the main manipulator is expressed as formula (1).

The correspondence between the D-H matrix and the joint information is shown in Table 1.

Table 1 correspondence between D-H matrix and joint information

In equation (29), rot (x, α) _{j_mp} ) To rotate alpha about the x-axis _{j_mp} Angle, rot (z, θ) _{j_mp} ) To rotate theta around z-axis _{j_mp} Angle, trans (x, a) _{j_mp} ) Move a in x direction _{j_mp} ，Trans(z,d _{j_mp} ) Move d in z direction _{j_mp} . The main manipulator 1000 shown in fig. 10 has a z-axis as the axis of rotation of the joint and an x-axis pointing to the next joint, and the y-axis direction can be determined according to the left/right hand law of the cartesian coordinate system. Rot (x, alpha) _{j_mp} )、Trans(x,a _{j_mp} ) The fourth order matrix represents rotation about a direction by a certain angle or translation along a direction by a certain distance.

In some embodiments, the mathematical structural model of the primary manipulator is described by D-H matrix multiplication of all joints, as in equation (2):

⁰ T _{7_mp} ＝ ⁰ T _{1_mp} · ^1_mp T _{2_mp} · ^2_mp T _{3_mp} · ^3_mp T _{4_mp} · ^4_mp T _{5_mp} · ^5_mp T _{6_mp} · ^6_mp T _{7_mp} (2)

in some embodiments, the D-H matrix for the joint in equation (2) may be determined based on equation (1).

Display device

In some embodiments, the display device (e.g., display device 130 shown in fig. 1) may include a liquid crystal display device, a field emission display device, an organic light emitting diode display device, and the like. In some embodiments, the display device is, for example, a two-dimensional image display (e.g., master external display 903) or a three-dimensional stereoscopic imaging display (e.g., stereoscopic display 902). In some embodiments, a display device is used to present at least one graphical element at a display interface (e.g., display interface 1200 shown in fig. 12). In some embodiments, the display device is used to display an actual image of the surgical instrument (e.g., actual image 1210 shown in fig. 12) at the display interface. In some embodiments, the actual image of the surgical instrument comprises an image taken by an imaging tool. In some embodiments, the display interface is, for example, a screen of a display. In some embodiments, the display interface may also be an area that is a virtual image.

Control device

In some embodiments, the control device (e.g., control device 110 shown in fig. 1) includes at least one integrated circuit unit. The at least one integrated circuit unit is configured to implement a predetermined function of the control device. For example, receiving information, performing calculations, sending instructions, etc. For example, the at least one integrated circuit unit is configured to periodically receive a motion parameter of the handle in the first degree of freedom and to control updating the first display characteristic of the at least one graphical element presented on the display interface based on the motion parameter of the first degree of freedom. Fig. 11 illustrates a schematic block diagram of a control device 1100 according to some embodiments of the present disclosure. Referring to fig. 11, in some embodiments, a control device 1100 includes: a signal receiving integrated circuit unit 1101 and a display control integrated circuit unit 1102 communicatively coupled to the signal receiving integrated circuit unit 1101. Wherein the signal receiving integrated circuit unit 1101 is configured to periodically receive a motion parameter of a handle (e.g., the handle 1020 shown in fig. 10) in a first degree of freedom. The display control integrated circuit unit 1102 is configured to control updating a first display characteristic of at least one graphical element presented on the display interface based on the motion parameter of the first degree of freedom. In some embodiments, the integrated circuit unit may be, for example, a circuit board or a circuit unit integrated on the circuit board for implementing a predetermined function.

Fig. 12 illustrates a schematic diagram of a display interface 1200 according to some embodiments of the present disclosure. Referring to fig. 12, in some embodiments, a control device (e.g., control device 110 shown in fig. 1 or control device 1110 shown in fig. 11) is configured to control the display of an actual image 1210 of a surgical instrument at display interface 1200, the movement of the surgical instrument being controlled by a handle of a primary manipulator. The actual image 1210 is displayed in combination with at least one graphical element (1230, 1240) to demonstrate movement of the handle of the primary manipulator in at least one degree of freedom and coordinated movement of the surgical instrument. In some embodiments, the actual image 1210 includes: an image of an end effector (1250, 1260) of the surgical tool and an image of the surgical site 1220. In some embodiments, a control device is communicatively coupled to an imaging tool (e.g., imaging tool 2302 shown in fig. 2, imaging tool 304 shown in fig. 3, or imaging tool 500 shown in fig. 5), the control device being configured to receive an actual image 1210 captured by the imaging tool and to control display of the actual image 1210 on display interface 1200. In some embodiments, graphical elements 1230 and 1240 shown in fig. 12 correspond to a handle of a left-hand-operated left-main operator (e.g., for controlling end effector 1250) and a handle of a right-hand-operated right-main operator (e.g., for controlling end effector 1260), respectively, of a medical staff member. In some embodiments, the graphical element 1230 or graphical element 1240 shown in fig. 12 is, for example, the graphical element 1300 shown in fig. 13A-F.

In some embodiments, the motion parameter of the degree of freedom of the handle is, for example, the motion position of the handle in the degree of freedom. For example, the motion parameter of the degree of freedom of the handle is, for example, the position coordinates of the handle corresponding to the degree of freedom. In some embodiments, the graphical elements include any of linear graphical elements, polygonal graphical elements, shaped graphical elements, and the like. In some embodiments, the display characteristics of the graphical element include: any one of the colors, shapes, sizes, and the like of the graphic elements.

In some embodiments, the control updates the first display characteristic of the at least one graphical element presented on the display interface, for example, based on a correspondence of the first display characteristic and the motion parameter of the first degree of freedom, to update the display characteristic corresponding to the different motion parameter. For example, according to different motion parameters, the control device controls the color, shape or size of the graphic element corresponding to the display interface. In some embodiments, the dynamic course of motion of the handle in the first degree of freedom can be represented or reflected by updating the first display characteristic of the at least one graphical element presented on the display interface.

Fig. 13A illustrates a schematic diagram of a graphical element 1300 in a first state according to some embodiments of the present disclosure. Fig. 13B illustrates a schematic diagram of a graphical element 1300 in a second state according to some embodiments of the present disclosure. Fig. 13C illustrates a schematic diagram of a graphical element 1300 in a third state according to some embodiments of the present disclosure.

In some embodiments, the first degree of freedom is a vertical degree of freedom, and the motion parameter of the handle of the main operator in the first degree of freedom comprises a position of the handle of the main operator in the vertical degree of freedom (e.g., a position in a vertical direction), the control device being configured to update the display characteristic of the at least one graphical element such that the at least one graphical element grows or shrinks following a change in the position of the handle of the main operator in the vertical degree of freedom. In some embodiments, the at least one graphical element comprises: first line-type graphic elements (1310 a,1310b, and 1310 c) corresponding to the vertical degree of freedom directions. The control means is configured to control the first linear graphic element to grow or shrink following a change in the position of the handle of the main operator in the vertical degree of freedom. For example, following the rise of the handle in the vertical degree of freedom, the first linear graphic element 1310a is updated to the first linear graphic element 1310b. Alternatively, the first linear graphic element 1310a is updated to the first linear graphic element 1310c following the descent of the handle in the vertical degree of freedom. In some embodiments, the first degree of freedom is a horizontal degree of freedom, and the motion parameter of the handle of the main operator in the first degree of freedom comprises a position of the handle of the main operator in the horizontal degree of freedom (e.g., a position in a horizontal direction), the control device being configured to update the display characteristic of the at least one graphical element such that the at least one graphical element moves horizontally following a change in the position of the handle of the main operator in the horizontal degree of freedom. In some embodiments, the at least one graphical element further includes a second linear graphical element 1320 corresponding to the horizontal degree of freedom direction. The control means is configured to control the first linear graphic element to move along the second linear graphic element following the movement of the handle of the main operator in the position of the horizontal degree of freedom. Fig. 13F illustrates a schematic diagram of a graphical element 1300 in a fourth state according to some embodiments of the present disclosure. Referring to fig. 13F, following the movement of the handle in the horizontal degree of freedom (arrow direction), the first linear graphic element 1310b moves along the second linear graphic element 1320 to the position of the first linear graphic element 1310F. In some embodiments, the second linear graphic element 1320 may represent a reference to a vertical degree of freedom direction, and the first linear graphic element 1310a-f may grow or shrink above (e.g., 1310a,1310b, 1310d, 1310e, 1310 f) or below (e.g., 1310 c) the second linear graphic element 1320.

In some embodiments, the control device is configured to control the updating of the display characteristic of the at least one graphical element to the alert characteristic in response to the movement parameter of the handle in the first degree of freedom reaching the limit condition. In some embodiments, the limitation of the handle is the boundary of the range of motion of the handle in physical space, e.g., the highest or lowest height of motion of the handle in a vertical degree of freedom (e.g., vertical direction), the furthest distance in a horizontal degree of freedom (e.g., horizontal direction). In some embodiments, the constraint of the handle may also be a range within the boundaries of the range of motion. For example, the constraint of the handle is a safety margin near the limit range of motion boundary of the handle. In some embodiments, the operator can be alerted to the movement of the corresponding degree of freedom of the handle reaching the limit by controlling the updating of the display feature to the alert feature. For example, the operator is alerted that the motion in a horizontal degree of freedom (e.g., horizontal direction) reaches a safe boundary in the horizontal direction. In some embodiments, the alert feature is a distinguishing feature from the display feature. For example, the alert feature is a different color, shape, line style, etc. than the display feature. Fig. 13D illustrates a schematic diagram of a graphical element 1300 in a constraint according to some embodiments of the present disclosure. Referring to fig. 13D, the motion parameters of the handle in the vertical degrees of freedom reach the limit condition, and the first line graphic element is updated to the display feature exhibited by the first line graphic element 1310D. The first type of graphic element 1310d may be colored or sized differently from the first type of graphic element 1310a, 1310b or 1310c to indicate that the motion parameter is about to exceed the range of motion.

In some embodiments, the control device is configured to control the display interface to display a critical area in response to the motion parameter of the handle in the first degree of freedom reaching a limit condition, the critical area corresponding to the limit condition. In some embodiments, the vicinity is a mapped area of the defined condition on the display interface. For example, the adjacent area is the boundary of the range of motion or the area corresponding to the safety boundary on the display interface. In some embodiments, when the motion parameter of the handle in the first degree of freedom reaches the limit condition, the display area can alert the operator that the motion of the corresponding degree of freedom of the handle reaches the limit condition. Fig. 13E shows a schematic diagram of a graphical element 1300 in a constraint according to further embodiments of the present disclosure. Referring to fig. 13D, the motion parameters of the handle in the vertical degree of freedom reach the limit condition, the display adjacent area 1330 is displayed, and the display characteristic of the first line-shaped graphic element 1310e may be kept unchanged or updated as the reminding characteristic.

In some embodiments, when the handle reaches a preset limit in a certain degree of freedom, the operator or the control device may perform a corresponding control. In some embodiments, the control device is configured to lock movement of the handle in a current degree of freedom, or disconnect the control chain of the handle to the surgical instrument in the current degree of freedom, when the handle reaches a preset limit in a certain degree of freedom. In some embodiments, the control device is configured to reestablish the control chain of the handle to the surgical instrument in the current degree of freedom when the handle is restored to its normal range of motion in a certain degree of freedom (e.g., the operator will move the handle to within the boundaries) by a preset limit.

In some embodiments, the control device is configured to periodically receive a motion parameter of the handle of the main operator in the second degree of freedom and to control updating a feature associated with the second degree of freedom of the at least one graphical element presented on the display interface based on the motion parameter of the second degree of freedom. In some embodiments, the dynamic course of motion of the handle in the second degree of freedom can be represented or reflected by updating the feature associated with the second degree of freedom of the at least one graphical element presented on the display interface. In some embodiments, the second degree of freedom is a different degree of freedom than the first degree of freedom. For example, the first degree of freedom is a vertical degree of freedom and the second degree of freedom is a horizontal degree of freedom, and vice versa. In some embodiments, the feature associated with the second degree of freedom is displayed, for example, according to a motion parameter of a different second degree of freedom, and the control device controls the display of the first display feature at a different position in the direction of the second degree of freedom on the display interface. In other embodiments, the feature associated with the second degree of freedom is displayed, for example, in accordance with a different motion parameter of the second degree of freedom, and the control means controls the motion parameter itself to display the second degree of freedom on the first display feature on the display interface. In other embodiments, the feature associated with the second degree of freedom is displayed, for example, in accordance with a motion parameter of a different second degree of freedom, and the control device controls the display of the second display feature of the at least one graphical element on the display interface. In some embodiments, the second display feature is similar to the first display feature. In some embodiments, the second display feature comprises: any one of the colors, shapes, sizes, and the like of the graphic elements.

In some embodiments, the control device is configured to determine a position and/or an attitude of the handle of the primary manipulator and to determine a parameter of the handle of the primary manipulator in the first degree of freedom based on the position and/or the attitude of the handle of the primary manipulator. In some embodiments, the control device is configured to periodically receive joint information of the at least one joint and determine the position and/or posture of the handle of the primary manipulator based on the joint information of the at least one joint. In some embodiments, the control device is communicatively coupled to the sensors of the primary manipulator, the control device being configured to periodically receive joint information acquired by the primary manipulator sensors of the at least one joint and calculate the position and/or pose of the handle based on the received joint information and the mathematical structural model of the primary manipulator. In some embodiments, the control device is configured to periodically receive joint information of the first joint, the second joint, and the third joint, and calculate the position of the handle. In some embodiments, the control device is configured to periodically receive joint information of the first joint, the second joint, the fifth joint, the sixth joint, and the seventh joint, and calculate the pose of the handle.

While particular embodiments of the present disclosure have been illustrated and described, it would be obvious to those skilled in the art that various other changes and modifications can be made without departing from the spirit and scope of the disclosure. Accordingly, it is intended to include in the appended claims all such changes and modifications that are within the scope of this disclosure.

Claims (10)

1. A surgical robotic system, comprising:

a main manipulator comprising a robotic arm and a handle disposed at a distal end of the robotic arm, the handle being movable in at least one degree of freedom;

display means for presenting at least one graphical element on a display interface, the at least one graphical element representing movement of the handle in at least one degree of freedom; and

and a control device in communication with the display device and the main operator, the control device configured to periodically receive a motion parameter of the handle in a first degree of freedom and to control updating a first display characteristic of the at least one graphical element presented on the display interface based on the motion parameter of the first degree of freedom.

2. The surgical robotic system of claim 1, wherein the control device is further configured to control the update of the display characteristic of the at least one graphical element to a reminder characteristic in response to the motion parameter of the handle in the first degree of freedom reaching a limit condition.

3. The surgical robotic system of claim 1, wherein the control device is further configured to control the display interface to display a critical area in response to the motion parameter of the handle in the first degree of freedom reaching a limit condition, the critical area corresponding to the limit condition.

4. The surgical robotic system of claim 1, wherein,

the first degree of freedom being a horizontal degree of freedom, the motion parameter of the handle of the main operator in the first degree of freedom comprising a position of the handle of the main operator in the horizontal degree of freedom, the control means being configured to update the display characteristic of the at least one graphical element such that the at least one graphical element moves horizontally following a change in the position of the handle of the main operator in the horizontal degree of freedom; or alternatively

The first degree of freedom is a vertical degree of freedom, the movement parameter of the handle of the main operator in the first degree of freedom comprises a position of the handle of the main operator in the vertical degree of freedom, the control means being configured to update the display characteristic of the at least one graphical element such that the at least one graphical element grows or contracts following a change in the position of the handle of the main operator in the vertical degree of freedom.

5. The surgical robotic system of claim 4, wherein the control device is further configured to periodically receive a motion parameter of a handle of the primary manipulator in a second degree of freedom and to control updating a feature associated with the second degree of freedom of the at least one graphical element presented on the display interface based on the motion parameter of the second degree of freedom.

6. The surgical robotic system of claim 5, wherein,

the at least one graphic element comprises a first linear graphic element corresponding to the vertical degree of freedom direction and a second linear graphic element corresponding to the horizontal degree of freedom direction;

the control means is configured to control the first linear graphical element to grow or shrink following a change in position of the handle of the main operator in the vertical degree of freedom and/or to control the movement of the first linear graphical element following the handle of the main operator in the horizontal degree of freedom along the second linear graphical element.

7. The surgical robotic system of claim 6, wherein the second linear graphical element represents a reference to a vertical degree of freedom direction, the first linear graphical element growing or shrinking above or below the second linear graphical element.

8. The surgical robotic system of any one of claims 1-7, wherein the control device is configured to control display of an actual image of a surgical instrument at the display interface, movement of the surgical instrument being controlled by a handle of the primary manipulator; the actual image is displayed in combination with the at least one graphical element to demonstrate movement of the handle of the primary manipulator in the at least one degree of freedom and coordinated movement of the surgical instrument.

9. The surgical robotic system of claim 8, wherein the control device is configured to determine a position and/or pose of the handle of the primary manipulator and determine a parameter of the handle of the primary manipulator in the first degree of freedom based on the position and/or pose of the handle of the primary manipulator.

10. The surgical robotic system of claim 9, wherein the handle includes at least one joint, the control device is configured to periodically receive joint information of at least a joint, and determine a position and/or pose of the handle based on the joint information of the at least one joint.