CN114848152A - Computer-readable storage medium, electronic device, and surgical robot system - Google Patents

Computer-readable storage medium, electronic device, and surgical robot system Download PDF

Info

Publication number
CN114848152A
CN114848152A CN202110152671.XA CN202110152671A CN114848152A CN 114848152 A CN114848152 A CN 114848152A CN 202110152671 A CN202110152671 A CN 202110152671A CN 114848152 A CN114848152 A CN 114848152A
Authority
CN
China
Prior art keywords
surgical
surgical instrument
point
arm
motion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110152671.XA
Other languages
Chinese (zh)
Inventor
王家寅
何超
张毅成
王超
江磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Microport Medbot Group Co Ltd
Original Assignee
Shanghai Microport Medbot Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Microport Medbot Group Co Ltd filed Critical Shanghai Microport Medbot Group Co Ltd
Priority to CN202110152671.XA priority Critical patent/CN114848152A/en
Priority to PCT/CN2022/075224 priority patent/WO2022166929A1/en
Publication of CN114848152A publication Critical patent/CN114848152A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/32—Surgical robots operating autonomously
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • A61B2034/254—User interfaces for surgical systems being adapted depending on the stage of the surgical procedure

Abstract

The present invention relates to a computer-readable storage medium, an electronic device, and a surgical robot system, the computer-readable storage medium having a program stored thereon, the program, when executed, performing the steps of: judging whether the surgical instrument is in the surgical field according to the surgical field information provided by the image acquisition device; when the surgical instrument is located outside the surgical field of view, planning a motion scheme, and enabling an image arm connected with the image acquisition device and/or a tool arm connected with the surgical instrument to execute the motion scheme so that the surgical instrument returns to the surgical field of view; wherein the program further performs safety measures to bring the image acquisition device and/or the surgical instrument into a safe state before or during the execution of the motion profile by the image arm and/or the tool arm. The invention can automatically restore the surgical instruments to the surgical field, thereby improving the safety and controllability of the surgery.

Description

Computer-readable storage medium, electronic device, and surgical robot system
Technical Field
The invention relates to the technical field of medical instruments, in particular to a computer-readable storage medium, electronic equipment and a surgical robot system.
Background
Adopt robot operation system to carry out minimal access surgery, patient's wound is little on the one hand, and wound infection risk is low, and the postoperative resumes soon, on the other hand can reduce doctor's the operation degree of difficulty and the tired degree of operation. However, during the robotic surgery, the surgical instruments are not in the surgical field of view due to the movement of the endoscope or the movement of the surgical instruments, which is an invisible surgical blind area for the surgeon. When the surgical instrument is not in the surgical field, the doctor cannot directly control the surgical instrument or the surgical field, and the surgical operation is very easy to damage human tissues if the surgical operation is directly performed.
Some devices capable of improving the safety of the operation appear in the prior art, for example, a medical imaging system in the prior art achieves the purpose of prompting a doctor about the current position of the surgical instrument by adding prompting information such as directionality on a nurse display screen, but the medical imaging system only roughly positions the orientation of the surgical instrument and cannot ensure the safety of the operation.
Therefore, it is an urgent need to solve the above-mentioned problems to design a surgical robot system and a control method thereof, which can automatically return surgical instruments to the surgical field to improve the safety and controllability of the surgical robot system during the use process.
Disclosure of Invention
An object of the present invention is to provide a computer-readable storage medium, an electronic device, and a surgical robot system that can improve the safety and controllability of a surgical operation.
To achieve the above object, the present invention provides a computer-readable storage medium having a program stored thereon, which when executed, performs the steps of:
judging whether the surgical instrument is in the surgical field according to the surgical field information provided by the image acquisition device;
when the surgical instrument is located outside the surgical field of view, planning a motion scheme, and enabling an image arm connected with the image acquisition device and/or a tool arm connected with the surgical instrument to execute the motion scheme so that the surgical instrument returns to the surgical field of view;
wherein the program further performs safety measures to bring the image acquisition device and/or the surgical instrument into a safe state before or during the execution of the motion profile by the image arm and/or the tool arm.
Optionally, before the image arm and/or the tool arm executes the motion profile, the program performs the following steps to execute the safety measure;
and judging whether the tail end of the image acquisition device and/or the tail end of the surgical instrument are/is positioned in the corresponding stamp card, if not, driving the tail end of the image acquisition device and/or the tail end of the surgical instrument to move into the corresponding stamp card.
Optionally, after the image arm and/or the tool arm performs the motion profile, the program further performs the steps of:
and driving the end of the image acquisition device and/or the end of the surgical instrument to move and extend out of the corresponding stamp card so as to enable the surgical instrument to return to the surgical field.
Optionally, the program performs the following steps to perform the safety measures during the execution of the motion profile by the image arm and/or the tool arm:
judging whether the characteristic value of the image arm and/or the tool arm is within a safety threshold value, if not, driving the image acquisition device and/or the surgical instrument to move a preset distance along the direction facing the outside of the body;
wherein the characteristic values comprise at least one of joint moments, joint positions or joint movement speeds of the image arm and/or the tool arm.
Optionally, during the execution of the motion profile by the image arm and/or the tool arm, the program performs the following steps to perform the safety measure;
judging whether the characteristic value of the image arm and/or the tool arm is within a safety threshold value, if not, stopping the image arm and/or the tool arm to execute the motion scheme;
wherein the characteristic values comprise at least one of joint moments, joint positions or joint movement speeds of the image arm and/or the tool arm.
Optionally, the program performs the following steps to plan the motion profile:
planning a target position of the surgical instrument within the surgical field of view;
planning the motion plan according to the current position of the surgical instrument and the target position.
Optionally, when the image acquisition device and/or the surgical instrument move such that the surgical instrument is not in the surgical field of view, the target position refers to a position that enables the surgical instrument to return to the surgical field of view and corresponds to the surgical instrument before leaving the surgical field of view.
Optionally, the program performs the following steps to obtain the target position: and acquiring the coordinate of the central point C of the operation visual field, and taking the central point C of the operation visual field as the target position.
Optionally, the program performs the following steps to obtain the target position: acquiring coordinates of a central point C of the surgical field;
and acquiring a spherical surface with the central point C as a spherical center, and taking any point in the sphere and on the spherical surface as the target position, wherein the radius of the spherical surface is the length of an end effector of the surgical instrument.
Optionally, the program performs the following steps to obtain a target position of the surgical instrument located outside the surgical field of view:
acquiring coordinates of a center point C of the surgical field and coordinates of a terminal point T of the surgical instrument located in the surgical field;
and calculating the coordinate of the middle point Z of a connecting line TC of the central point C and the tail end point T of the surgical instrument positioned in the surgical visual field, and taking the middle point Z as the target position of the surgical instrument positioned outside the surgical visual field.
Optionally, the program performs the following steps to obtain a target position of the surgical instrument located outside the surgical field of view:
acquiring coordinates of a central point C of the operation visual field and coordinates of a terminal point T of a surgical instrument positioned in the operation visual field;
calculating coordinates of a midpoint Z of a line TC connecting the center point C and a terminal point T of the surgical instrument located in the surgical field;
acquiring a spherical surface with a terminal point T of the surgical instrument positioned in the surgical field as a spherical center, wherein the radius of the spherical surface is the length of an end effector of the surgical instrument positioned in the surgical field;
judging whether the distance from the midpoint Z to a terminal point T of the surgical instrument positioned in the surgical field is larger than the radius of the spherical surface, if so, taking the midpoint Z as the target position; if not, selecting a point A on the connecting line TC as the target position, wherein the point A is positioned between the middle point Z and the central point C, the distance from the point A to a terminal point T of the surgical instrument positioned in the surgical field is m times of the radius of the spherical surface, and m is larger than 1.
Optionally, the program performs the following steps to acquire the coordinates of the center point C of the surgical field:
obtaining the coordinate of the central point C according to the coordinates of the terminal point N and the reference point M of the image acquisition device and the depth of field H of the image acquisition device; wherein the reference point M is a point which is fixed in position during the operation and is located on the axis of the image acquisition device.
Optionally, the program performs the following steps to obtain the coordinates of the center point C:
obtaining the coordinates N (x) of the terminal point N 1 ，y 1 ，z 1 ) And the coordinates M (x) of said reference point M 2 ，y 2 ，z 2 )；
Calculating a direction vector in the axial direction of the image pickup device as
Figure BDA0002932509330000051
Calculating the coordinate C (x) of the center point C of the surgical field 3 ，y 3 ，z 3 ) Comprises the following steps: c (x) 3 ，y 3 ，z 3 )＝N(x 1 ，y 1 ，z 1 )+H×E NM 。
Optionally, the program performs the following steps to plan the motion profile:
obtaining a first motion trail equation according to a first motion trail of the surgical instrument planned by one person; the starting point of the first motion track is the current position of the surgical instrument, and the ending point of the first motion track is the target position;
selecting a preset motion trail equation matched with the first motion trail to serve as a second motion trail equation;
performing combined optimization on the first motion trail equation and the second motion trail equation to obtain a third motion trail equation;
acquiring the relative position relation between the surgical instrument and the surgical visual field;
determining the motion scheme according to the relative position relation of the surgical instrument and the surgical field; the motion profile is defined by the first motion trajectory equation, or by the second motion trajectory equation, or by the third motion trajectory equation.
Optionally, the program performs the following steps to plan the motion profile:
acquiring a terminal point E of the image acquisition device 1 And the stationary point R of the image arm 1 The coordinates of (a);
obtaining a tip point T of the tool arm 1 The coordinates of (a);
calculating straight line E 1 T 1 And a straight line T 1 R 1 The included angle theta is formed 1 ；
Determining the motion scheme, the motion scheme comprising: the image arm is at the fixed point R of the image arm 1 Rotate the center of rotation in a first direction by theta 1 And (4) an angle.
Optionally, the program further performs the steps of:
obtaining a stationary point R of the tool arm 2 The coordinates of (a);
the motion scheme further comprises: the image arm is at the fixed point R of the image arm 1 Rotate theta in the second direction for the center of rotation 2 Angle, and the tool arm with a stationary point R on the tool arm 2 Rotate theta in the second direction for the center of rotation 2 An angle to maintain the surgical instrument within the surgical field of view.
Optionally, at least one of the surgical instruments is located within the surgical field of view, at least one surgical instrument is located outside the surgical field of view; the tool arm used for mounting the surgical instrument positioned in the surgical visual field is a first tool arm, and the tool arm used for mounting the surgical instrument positioned outside the surgical visual field is a second tool arm;
the program performs the following steps to plan the motion profile:
acquiring a terminal point E of the image acquisition device 1 And the stationary point R of the image arm 1 The position of (a);
obtaining a stationary point R of the first tool arm 3 And a distal point T of a surgical instrument located outside the surgical field of view 1 The position of (a);
calculating straight line E 1 T 1 And a straight line T 1 R 1 The included angle theta is formed 1 ；
Determining the motion scheme comprising the image arm at its motionless point R 1 Rotate the center of rotation in a first direction by theta 1 Angle, and the first tool arm with a stationary point R on the first tool arm 3 Rotate the center of rotation in a first direction by theta 1 Angle to the surgical instrument mounted on the second tool armWhile the instrument is returned to the surgical field, the surgical instrument mounted on the first tool arm remains in the surgical field.
To achieve the above object, the present invention also provides an electronic device comprising a processor and a computer-readable storage medium as described in any of the preceding, the processor being configured to execute a program stored on the computer-readable storage medium.
To achieve the above object, the present invention also provides a surgical robot system including:
the image acquisition device is used for providing an operation visual field;
a tool arm for mounting a surgical instrument for performing a surgical procedure within the surgical field; and the number of the first and second groups,
a control unit configured to execute a program stored on a computer readable storage medium as in any of the previous items.
Optionally, the exercise apparatus further comprises an input device, and the control unit plans the exercise scheme according to instructions input by the input device.
Optionally, the surgical robotic system comprises the electronic device of claim 18, the control unit comprising the processor.
Compared with the prior art, the computer-readable storage medium, the electronic device and the surgical robot system have the following advantages:
a first, aforementioned computer-readable storage medium has a program stored thereon, which when executed, performs the steps of: judging whether the surgical instrument is in the surgical field according to the surgical field information provided by the image acquisition device; when the surgical instrument is located outside the surgical field of view, planning a motion scheme, and enabling an image arm connected with the image acquisition device and/or a tool arm connected with the surgical instrument to execute the motion scheme so that the surgical instrument returns to the surgical field of view; wherein the program further performs safety measures to bring the image acquisition device and/or the surgical instrument into a safe state before or during the execution of the motion profile by the image arm and/or the tool arm. When the computer readable storage medium is applied to a surgical robot system and the surgical robot system is utilized to perform surgical operation, once the surgical instrument is positioned outside a surgical field of view, a corresponding program can be executed so as to ensure that the surgical instrument returns to the surgical field of view without damaging human tissues and improve the controllability and safety of the surgical operation.
Secondly, when the program plans a motion scheme and controls the tool arm to execute the motion scheme so as to enable the surgical instrument to return to the surgical visual field, a plurality of methods for planning the target position of the surgical instrument can be provided, so that the method is suitable for different surgical scenes and improves the universality of a surgical robot system.
Thirdly, the motion scheme can also relate to the motion scheme of the image arm, the motion of the image arm is controlled so as to enable the surgical instrument to return to the surgical field, and the motion of the image arm and/or the tool arm is utilized to combine to form different adjustment strategies, so that the redundant selection in the surgical process is ensured, and the surgical instrument is ensured to return to the surgical field.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic block diagram of a surgical robotic system provided in accordance with an embodiment of the present invention;
FIG. 2 is a control flow diagram of a surgical robotic system provided in accordance with an embodiment of the present invention;
FIG. 3 is a schematic view of a surgical instrument in a surgical robotic system having a tool arm performing a motion profile in accordance with an embodiment of the present invention;
FIG. 4 is a schematic view of a surgical instrument in a surgical robotic system having a tool arm performing a motion profile in accordance with another embodiment of the present invention;
FIG. 5 is a block diagram of the control principles of the tool arm of the surgical robotic system as it executes a motion profile in accordance with an embodiment of the present invention;
FIG. 6a is a diagrammatic view of a surgical instrument of a surgical robotic system within a surgical field of view with one of the surgical instruments in accordance with an embodiment of the present invention;
FIG. 6b is a diagrammatic view of a surgical instrument of a surgical robotic system within a surgical field of view with two surgical instruments in view according to an embodiment of the present invention;
FIG. 6c is a diagrammatic view of a surgical instrument of a surgical robotic system within a surgical field of view showing three surgical instruments in accordance with an embodiment of the present invention;
FIG. 7 is a diagrammatic view of a surgical robotic system during surgery with two surgical instruments therein, one of the surgical instruments in a surgical field of view and the other of the surgical instruments out of the surgical field of view in accordance with an embodiment of the present invention;
FIG. 8a is a schematic view of a target position of a surgical instrument planned by a control unit in a surgical robotic system according to an embodiment of the present invention, wherein the second surgical instrument is shown in phantom outside the surgical field of view and in solid lines back inside the surgical field of view;
FIG. 8b is a schematic diagram of a control unit planning a target position of a surgical instrument in a surgical robotic system provided in accordance with another embodiment of the present invention;
fig. 8c is a schematic view of a target position of a surgical instrument planned by a control unit in a surgical robotic system provided in accordance with yet another embodiment of the present invention;
FIG. 9 is a functional block diagram of a control unit in a surgical robotic system planning a tool arm movement plan in accordance with an embodiment of the present invention;
fig. 10a is a diagram of the position of a surgical instrument versus time when a control unit of a surgical robotic system is configured to plan a motion profile of a tool arm using a T-trajectory planning method according to an embodiment of the present invention;
FIG. 10b is a graph of the velocity of the surgical instrument versus time for a control unit of the surgical robotic system using T-trajectory planning to plan the movement of the tool arm, in accordance with one embodiment of the present invention;
FIG. 10c is a graph of acceleration versus time for a surgical instrument when a control unit of a surgical robotic system configured according to an embodiment of the present invention employs a T-trajectory planning method to plan a motion profile of a tool arm;
FIG. 11a is a schematic diagram of a control unit of a surgical robotic system planning a motion profile of an image arm in accordance with an embodiment of the present invention;
FIG. 11b is a schematic view of a control unit of the surgical robotic system controlling the movement of an endoscope with a second surgical instrument positioned in a surgical field according to an embodiment of the present invention;
fig. 11c is a schematic diagram of the control unit of the surgical robotic system controlling the motions of the endoscopic and second surgical instruments according to an embodiment of the present invention.
[ reference numerals are described below ]:
10-a doctor console;
11-a display;
20-an image display device;
30-a surgical manipulation device;
31-image arm, 32-tool arm, 33-endoscope, 33' -surgical field, 34-surgical instrument, 34 a-first surgical instrument, 34 b-second surgical instrument, 35-stab card;
40-operating table;
50-a tool placement device;
61-track identification module, 62-selection module, 63-storage module, 64-position calculation unit and 65-track calculation unit.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the present embodiment are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Furthermore, each of the embodiments described below has one or more technical features, and thus, the use of the technical features of any one embodiment does not necessarily mean that all of the technical features of any one embodiment are implemented at the same time or that only some or all of the technical features of different embodiments are implemented separately. In other words, those skilled in the art can selectively implement some or all of the features of any embodiment or combinations of some or all of the features of multiple embodiments according to the disclosure of the present invention and according to design specifications or implementation requirements, thereby increasing the flexibility in implementing the invention.
As used in this specification, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. As used in this specification, the term "or" is generally employed in its sense including "and/or" unless the content clearly dictates otherwise, and the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either fixedly connected, detachably connected, or integrally connected. Either mechanically or electrically. Either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
A core idea of the present invention is to provide a computer-readable storage medium having a program stored thereon, which when executed, performs the steps of determining whether a surgical instrument is within a surgical field based on surgical field information provided by an image acquisition device; when the surgical instrument is located outside the surgical field of view, planning a motion scheme, and enabling an image arm connected with the image acquisition device and/or a tool arm connected with the surgical instrument to execute the motion scheme so that the surgical instrument returns to the surgical field of view; wherein the program further performs safety measures to bring the image acquisition device and/or the surgical instrument into a safe state before or during the execution of the motion profile by the image arm and/or the tool arm. The computer readable storage medium is applied to a surgical robot system, and once a surgical instrument is located outside a surgical field during a surgical operation performed by the surgical robot system, the surgical robot system may execute a corresponding program to return the surgical instrument to the surgical field. In other words, in the surgical robot system, when the surgical instrument leaves the surgical field, the surgical instrument returns to the surgical field by executing corresponding operation, so that a doctor is prevented from performing surgical operation in a blind field area, and the safety and controllability of the surgery are improved.
Further, an embodiment of the present invention also provides an electronic device including the computer-readable storage medium, and a surgical robot system including a control unit that executes a program stored on the computer-readable storage medium.
To further clarify the objects, advantages and features of the present invention, a more particular description of the invention will be rendered by reference to the appended drawings. It is to be noted that the drawings are in a very simplified form and are not to precise scale, which is merely for the purpose of facilitating and distinctly claiming the embodiments of the present invention. The same or similar reference numbers in the drawings identify the same or similar elements.
The surgical robot system provided by the embodiment of the invention can be a master-slave robot system to be operated, or other surgical robot systems. The surgical robot system can be used for various minimally invasive surgical operations. For convenience, the following description will be given by taking the surgical robot system as a master-slave robot system and taking the surgical robot system to perform laparoscopic surgery as an example, but those skilled in the art should understand that the invention should not be limited thereto.
The surgical robot system provided by the embodiment of the invention can be a teleoperated master-slave robot system, and can also be other surgical robot systems. The surgical robot system can be used for various minimally invasive surgical operations. In the following description, for convenience of understanding, the surgical robot system is taken as a master-slave robot system, and the surgical robot system performs laparoscopic surgery as an example, but those skilled in the art should understand that the invention should not be limited thereto.
Referring to fig. 1, the surgical robot system includes a control end including a doctor console 10, and an execution end including an image display device 20, a surgical operation device 30, an operating table 40, and a tool placement device 50. The surgical robot system is mainly used for performing minimally invasive surgical treatment on a patient on the operating table 40.
With continued reference to fig. 1, the surgical device 30 includes at least one image arm 31 and at least one tool arm 32, and the image acquisition device is hung on the image arm 31. For convenience of description, the image acquiring device is, for example, an endoscope 33 or an ultrasound probe, and the image acquiring device is, for example, an endoscope 33. The tool arm 31 is used to carry a surgical instrument 34, and the endoscope 33 and the surgical instrument 34 are passed into the patient through a wound in the patient's body, in particular through a stab card (not shown in fig. 1) provided at the wound and into the patient's body. Thereafter, the endoscope 33 can acquire the human tissue information, the surgical instrument 34 information in the human body, and the surgical environment information, i.e., the endoscope 33 provides a surgical field in which the surgical instrument 34 performs a surgical operation.
In this embodiment, the doctor console 10 includes a main manipulator (also called a master hand), the control unit is connected in communication with the main manipulator, the image arm 31, the tool arm 32 and the surgical instrument 34, and the main manipulator and the tool arm 32 and the surgical instrument 34 form a master-slave control relationship. That is, the control unit is configured to control the tool arm 32 to move according to the movement of the main manipulator during the operation, and control the surgical instrument 34 to execute the movement instruction related to the main manipulator. Further, the surgeon console 10 further includes a display 11, and the display 11 is used for displaying the condition inside the patient and the movement condition of the surgical instrument 34. Further, when the surgical instrument is not within the surgical field of view, the control unit is configured to receive an operation instruction according to the received resume, disconnect the master-slave control relationship of the master manipulator with the tool arm 32 and the surgical instrument 34, and then may perform a resume operation to return the surgical instrument 34 within the surgical field of view. In this embodiment, the triggering manner of the command is not limited, alternatively, the doctor may transmit the command to the control unit by triggering an input device in the surgical robot system, for example, a switch, where the switch may be a virtual key on the interactive interface of the display 11, a voice-operated switch, or an electrical hardware switch, where the electrical hardware switch may be disposed on the supporting beam or the main control arm of the doctor console 10, or the command may be triggered by a foot switch or a different control mode of the master hand pinch control of the existing surgical robot system, or the like.
In performing the recovery operation, the control unit is configured to: planning an exercise scheme; controlling the image arm 31 and/or the tool arm 32 to execute the motion scheme such that the surgical instrument 34 is returned to the surgical field after the image arm 31 and/or the tool arm 32 executes the motion scheme and while the tip of the endoscope 33 and/or the tip of the surgical instrument 34 are outside the respective stab cards. It is understood that reference herein to the distal end of the endoscope 33 and/or the distal end of the surgical instrument 34 being outside the respective badge refers to the situation where the endoscope 33 and/or the surgical instrument 34 are inside the patient and outside the badge.
Furthermore, the control unit is configured to perform safety measures before or during the execution of the motion profile by the image arm 31 and/or the tool arm 32 to bring the endoscope 33 and/or the surgical instrument 34 into a safe state. As used herein, the term "safe state" means that the endoscope 33 and/or the surgical instrument 34 do not cause damage to non-target body tissue during a surgical procedure.
Fig. 2 shows a control flow chart of the surgical robot system, and referring to fig. 2, the control flow of the surgical robot system is as follows:
step S1: the doctor judges whether the surgical instrument is in the surgical field, if so, the surgical operation is continued, and if not, the step S2 is executed.
Step S2: the surgeon determining whether an adjustment to the relative position of the surgical instrument and the endoscope is required to bring the surgical instrument back into the surgical field; if not, the surgical operation is continued, and if so, step S3 is executed.
Step S3: the surgeon triggers the instructions to cause the control unit to break the master-slave control relationship of the master manipulator with the tool arm and the surgical instrument.
Step S4: the control unit plans a movement plan.
Step S5: the control unit controls the image arm and/or the tool arm to execute the motion profile.
Step S6: the control unit executes the safety measures so that the image arm and/or the tool arm do not pose a threat to the safety of human tissue during execution of the motion program.
Step S7: the doctor judges whether the surgical instrument returns to the surgical visual field, if so, the master-slave control relation between the master manipulator and the tool arm and the surgical instrument is restored, and the surgical operation is continued; if not, the process returns to step S4.
Wherein the step S6 is performed before the step S5, or the step S6 is performed in synchronization with the step S5. And step S7, in which the doctor determines whether or not the surgical instrument is returned to the surgical field, when the distal end of the surgical instrument 34 and the distal end of the endoscope 33 are located outside the respective stab cards.
In this embodiment, the control unit executes the safety measure to avoid unnecessary damage to human tissue by the endoscope and/or the surgical instrument during execution of the motion profile by the image arm and/or the tool arm.
As can be appreciated by those skilled in the art, for a non-master-slave controlled surgical robotic system, the control unit directly plans the motion scheme upon receiving the instruction, controls the image arm and/or the tool arm to execute the motion scheme and the safety measures to return the surgical instrument 34 to the surgical field of view. That is, the surgical robot system not controlled by master and slave does not need to perform the step S3.
Optionally, in some embodiments, the step S6 is performed before the step S5, when the safety measure includes driving the endoscope 33 and/or the surgical instrument 34 to move until the tip of the endoscope 33 and the tip of the surgical instrument 34 are located in the corresponding stab card 35. Specifically, if the surgical instrument 34 performs the motion scheme alone, in the step S6, the tip of the surgical instrument 34 is moved into the card 35 of the surgical instrument 34 (as shown in fig. 3). Alternatively, if the endoscope 33 executes the motion program separately, in step S6, the distal end of the endoscope 33 is moved into a poke card of the endoscope 33. Alternatively, if the surgical instrument 34 and the endoscope 33 jointly execute the motion scheme, in step S6, the distal end of the surgical instrument 34 is moved into the stab card of the surgical instrument 34, and the distal end of the endoscope 33 is moved into the stab card of the endoscope 33. In this way, when the image arm 31 and/or the tool arm 32 performs the motion program, the distal end of the endoscope 33 and/or the distal end of the surgical instrument 34 are always located within the poke card, which does not contact human tissue, thereby avoiding injury to the human tissue. After the image arm 31 and/or the tool arm 32 perform the motion scheme, it is also necessary to control the movement of the tip of the endoscope 33 and/or the tip of the surgical instrument 34 to follow the respective stampWhich protrudes into the body to facilitate determining whether the surgical instrument 34 is back in the surgical field 33'. It will be appreciated by those skilled in the art that the operations herein to actuate the movement of the distal end of the endoscope 33 and/or the distal end of the surgical instrument 34 to protrude from the respective tamp card may be actuated automatically by a control unit, or may be performed manually by a medical professional, preferably by the control unit. And, before performing the safety measure, if the distance from the end of the endoscope 33 and/or the end of the surgical instrument 34 to the end of the corresponding badge is L 0 The control unit then controls the endoscope 33 and/or the surgical instrument 34 to move L in a direction outside the body when the safety measure is executed 0 And after the image arm 31 and/or the tool arm 32 performs the motion scheme, the endoscope 33 and/or the surgical instrument 34 is moved L in an in vivo-oriented direction 0 The distal end of the endoscope 33 or the distal end of the surgical instrument 34 is extended from the inside of the badge into the human body to determine whether the surgical instrument 34 is returned to the surgical field. The end of the card is the end of the card that is positioned in the body. Accordingly, the proximal end of the card is referred to hereinafter as the end of the card that is positioned outside the body.
In other embodiments, the step S6 is executed synchronously with the step S5, and the safety measure includes: and judging whether the characteristic value of the image arm 31 and/or the tool arm 32 is within a safety threshold value, and if not, driving the endoscope 33 and/or the surgical instrument 34 to move towards the direction outside the human body by a preset distance. The "direction toward the outside of the human body" refers to a direction toward the proximal end of the stab card along the distal end of the stab card. In this way, when the image arm 31 and/or the tool arm 32 perform the motion scheme, the distal end of the endoscope 33 and/or the distal end of the surgical instrument may be outside the corresponding badge 35, but without any force or even at a distance from the body tissue (as shown in fig. 4). Here, the "characteristic value" is a joint moment, a joint position, or a joint movement speed of the image arm 31 and/or the tool arm 32.
Referring to fig. 5, the specific implementation process of the safety measure is described herein by taking the case that the control unit controls the tool arm to implement the motion scheme, the characteristic value is the joint torque of the tool arm, and the safety threshold is the torque threshold.
Step a: the control unit monitors the joint moment tau of the tool arm in real time 1 . The joint moment may be acquired by a sensor provided at a joint of the tool arm.
Step b: the control unit judges tau 1 Whether the torque is greater than a torque threshold value tau prestored in the control unit 0 If not, returning to the step a; if yes, executing step c.
Step c: the control unit controls the endoscope and/or the surgical instrument to move towards the direction outside the body D 0 。
And repeating the steps a to c until the tool arm finishes executing the motion scheme.
After the tool arm executes the motion scheme, the control unit further controls the surgical instrument to move p × D in a direction toward the inside of the body 0 And p is the number of times step c is performed. The "direction toward the inside of the body" as referred to herein means a direction toward the distal end of the stab card along the proximal end of the stab card.
It will be appreciated by those skilled in the art that if the characteristic value is a joint position, the control unit monitors in real time a deviation of an actual position of the joint from a predetermined position, and if the deviation is not within a safety threshold, controls the endoscope and/or the surgical instrument to move a predetermined distance in a direction toward the outside of the body. And if the characteristic value is the joint moving speed, the control unit monitors the deviation of the actual speed and the preset speed of the joint in real time.
Or, in an alternative embodiment, the security measures include: the control unit judges whether the characteristic value of the image arm and/or the tool arm is within a safety threshold value, and if not, the control unit controls the image arm and/or the tool arm to suspend executing the motion scheme. And then manually adjusting the endoscope and/or the surgical instrument by a medical staff so that the endoscope and/or the surgical instrument moves for a preset distance along the direction towards the outside of the body, and then executing the motion scheme.
In this embodiment, the control unit may plan different motion schemes and control at least one of the tool arm and the image arm to execute the motion schemes. In the following, the method by which the control unit plans the movement plan will be described in detail.
As shown in fig. 6a to 6b, at least one endoscope 33 and at least one surgical instrument 34 are required for the surgical robotic system to complete a surgical operation. Fig. 6a shows a schematic view of the surgical robot system comprising one endoscope 33 and one surgical instrument 34, fig. 6b shows a schematic view of the surgical robot system comprising one endoscope 33 and two surgical instruments 34, and fig. 6c shows a schematic view of the surgical robot comprising one endoscope 33 and three surgical instruments 34. Of course, in other embodiments, the surgical robotic system may also include more than one surgical instrument 34 and more than two endoscopes 33. The endoscope 33 is any image acquiring device for acquiring image information of tissues in the human body.
For ease of understanding, the manner in which the control unit plans the motion profile according to the invention is described in more detail herein by way of example in which the surgical robotic system includes one endoscope 33 and two surgical instruments 34. For convenience, as shown in fig. 6b, two surgical instruments 34 are respectively referred to as a first surgical instrument 34a and a second surgical instrument 34b, and accordingly, a tool arm for mounting the first surgical instrument 34a is referred to as a first tool arm 32a, and a tool arm for mounting the second surgical instrument 34b is referred to as a second tool arm 32 b. Those skilled in the art may modify the following description to accommodate situations where the surgical robotic system includes more than two endoscopes and more than three surgical instruments.
Referring to fig. 6b, during the operation, when the two surgical instruments 34 are located in the surgical field 33' provided by the endoscope 33, the surgeon can perform normal surgical operations. But is not conducive to surgical procedures when at least one of the surgical instruments 34, such as the second surgical instrument 34b, is positioned outside of the surgical field 33' (as shown in fig. 7). If the surgeon determines that further adjustment of the relative position of the second surgical instrument 34b and the endoscope 33 is required, the command is triggered to cause the control unit to disconnect the master-slave control relationship between the master manipulator and the tool arm 32 and the surgical instrument 34 and to adjust the relative position relationship of the surgical instrument 34 and the endoscope 33 to bring the second surgical instrument 34b back into the surgical field 33'.
In one embodiment, the control unit controls the second tool arm 32b to move to bring the second surgical instrument 34b back into the surgical field. At this point, the control unit is configured to plan the target position of the second surgical instrument 34b within the surgical field 33' prior to planning the motion profile.
The target location may be planned using different methods for different surgical environments. For example, as shown in fig. 8a, in one implementation, when the second surgical instrument 34b is not within the surgical field 33 ' due to movement of the endoscope 33 and/or the second surgical instrument 34b, the target position of the second surgical instrument 34b refers to a position that enables the second surgical instrument 34b to be restored within the surgical field 33 ' and corresponds to its position before exiting the surgical field 33 '. That is, in this case, the position of the second surgical instrument 34b within the surgical field 33 'at the previous time is taken as the target position with reference to the coordinate system in which the surgical field 33' is located. The "previous time" mentioned here refers to a specified time before the second surgical instrument 34b leaves the surgical field 33'.
Specifically, when the surgical field 33 'is displaced due to the adjustment of the attitude of the endoscope 33, the second surgical instrument 34b is moved out of the surgical field 33', the last time is the time before the adjustment of the attitude of the endoscope 33. At this time, the control unit is configured to: the position of the second surgical instrument 34b before the endoscope 33 is adjusted in posture is recorded as the target position. The position of the second surgical instrument 34b can be obtained by a robotic kinematics method (DH method). As will be appreciated by those skilled in the art, in this manner, the coordinate system of the surgical field 33 'moves synchronously with the endoscope 33 when the endoscope 33 is adjusted in pose, to translate the movement of the endoscope 33 into a movement of the second surgical instrument 34b relative to the coordinate system of the surgical field 33'.
Alternatively, when the second surgical instrument 34b leaves the surgical field 33' due to its own movement, the control unit is configured to: recording the position of the second surgical instrument 34b in real time; the moment when the second surgical instrument 34b leaves the surgical field 33' is determined and the position of the second surgical instrument 34b at the previous moment is taken as the target position. For example, the control unit records the position of the second surgical instrument 34b every predetermined time, and at the i-th time, the second surgical instrument 34b is in the surgical field 33 ', and at the i + 1-th time, the control unit determines that the second surgical instrument 34b is away from the surgical field 33', and then the position of the second surgical instrument 34b at the i-th time is taken as the target position. The control unit may obtain the position of the second surgical instrument 34b from a robotic kinematics method or may obtain the position of the second surgical instrument 34b by monitoring an identifier provided on the second surgical instrument 34 b. The identifier may be a visualization element or any other directional indicia indicative of the second surgical instrument 34 b.
As another example, as shown in fig. 8b, in another implementation, the control unit is configured to acquire a position of a center point C of the surgical field, and use the center point C as the target position. The control unit, when acquiring the position of the center point C, is specifically configured to:
in a reference coordinate system, the position N (x) of the terminal point N of the endoscope 33 is obtained according to a robot kinematics method 1 ，y 1 ，z 1 ) And obtaining the position M (x) of a reference point M 2 ，y 2 ，z 2 ). The reference point M is a point whose position is constant throughout the procedure, and is on the axis of the endoscope 33. Typically, in laparoscopic surgery, the reference point M is the point at which the endoscope 33 is positioned at the patient's abdominal wound (commonly referred to as the belly point), or the reference point M is a point on a stab card. The reference coordinate system is an artificially established coordinate system, such as a geodetic coordinate system.
Then, according to the direction vector calculation method, the direction vector in the axial direction of the endoscope 33 is calculated as
Figure BDA0002932509330000241
Finally, calculating the coordinate C (x) of the center point C of the operation visual field 3 ，y 3 ，z 3 ) Comprises the following steps: c (x) 3 ，y 3 ，z 3 )＝N(x 1 ，y 1 ，z 1 )+H×E NM And H is the depth of field of the endoscope. As will be appreciated by those skilled in the art, during endoscopic surgery, the field of view will only be clear to a reasonable extent within the depth of field of the endoscope.
Further, please continue to refer to fig. 6b, in this implementation, the target position may be not only the center point C, but also other points in a sphere and on a sphere with the center point C as a sphere center. As such, the control unit is further configured to: and acquiring a spherical surface with the central point C as the center of the sphere, and taking any point in the sphere and on the spherical surface as the target position. The radius of the sphere may be the length of the end effector of the second surgical instrument 34b, or the radius of the sphere may be determined by the surgeon based on the size of the body tissue, as would be reasonably understood by one skilled in the art. As such, when the target position is on the spherical surface, the coordinates D (x) of the target position 4 ，y 4 ，z 4 ) The coordinate of the center point C and the radius of the ball satisfy the following relation:
r 3 ＝(x 3 -x 4 ) 2 +(y 3 -y 4 ) 2 +(z 3 -z 4 ) 2
as yet another example, in yet another implementation, as shown in fig. 8C, the target location is determined using the first surgical instrument 34a and a center point C of the surgical field 33'. In detail, the control unit is configured to:
first, the position C (x) of the center point C of the operation field is obtained in a reference coordinate system by the method described above 3 ，y 3 ，z 3 ) And acquiring the end point T (x) of the first surgical instrument 34a using a robotics approach 5 ，y 5 ，z 5 ) Position T (x) 5 ，y 5 ，z 5 )；
Next, an averaging algorithm is used to calculate the position of the midpoint Z of the line connecting the center point C and the end point T of the first surgical instrument 34 a.
In this implementation, the midpoint Z may be directly used as the target position without considering possible interference between the first surgical instrument 34a and the second surgical instrument 34 b. Considering that the second surgical instrument 34b may interfere with the first surgical instrument 34a when returning to the surgical field 33', as shown in fig. 8c, the control unit is further configured to:
a sphere centered at the end point T of the first surgical instrument 34a is obtained, the radius r of which is the length of the end effector of the first surgical instrument 34 a.
Judging whether the distance from the midpoint Z to the terminal point T of the first surgical instrument 34a is greater than the radius r of the spherical surface, if so, taking the midpoint Z as the target position; if not, selecting a point A (not marked in the figure) on a connecting line TC as the target position, wherein the point A is positioned between the middle point Z and the central point C, the distance from the point A to the tail end point T of the first surgical instrument is m times of the radius r of the sphere, and m is larger than 1. Thus, the coordinates A (x) of the point A 6 ，y 6 ，z 6 ) Can be calculated by the following formula:
Figure BDA0002932509330000261
wherein the content of the first and second substances,
Figure BDA0002932509330000262
is the direction vector of the TC connection.
The control unit has a plurality of planning methods of the target position, so that a doctor can select the most appropriate method to determine the target position of the second surgical instrument 34b according to the actual situation of the operation, and the adaptability and the universality of the surgical robot system are improved.
The control unit then plans the movement scheme.
In an alternative implementation, as shown in fig. 9, the control unit may include a trajectory recognition module 61, a selection module 62, a storage module 63, a position calculation unit 64, and a trajectory calculation unit 65, where the storage module 63 stores a preset motion trajectory equation. The display device 11 of the surgical robot system further has a touch screen handwriting function, and the display device 11 is further configured to display a motion trajectory of the second tool arm 32b (i.e., a motion trajectory of the second tool arm when the second surgical instrument moves from the current position to the target position) manually planned by a medical staff as a first motion trajectory. That is, the medical staff can draw the first movement trace of the second surgical instrument 34b on the display device 11 artificially according to the condition inside the patient body displayed by the display device. At the same time, the trajectory identification module 61 identifies the first motion trajectory and generates a first motion equation. Next, the selecting module 62 selects a preset motion trail equation matched with the first motion trail equation in the storage module 63 to serve as a second motion trail equation. It should be understood that "a predetermined trajectory equation matching the first trajectory equation" herein means a predetermined trajectory equation closest to the first trajectory equation. And the trajectory calculation unit 65 combines and re-optimizes the first motion trajectory equation and the second motion trajectory equation to obtain a third motion trajectory equation. Next, the position calculation unit 64 is used to calculate the relative position relationship between the second surgical instrument 34b and the surgical field 33'. Finally, the trajectory calculation unit 65 is configured to determine the movement scheme according to the relative position relationship between the second surgical instrument 34b and the surgical field 33'. According to the actual situation, the motion scheme may be defined by the first motion trajectory equation, or defined by the second motion trajectory equation, or defined by the third motion trajectory equation. Here, the motion scheme may be defined by the first motion trail equation or the second motion trail equation or the third motion trail equation, that is, the motion of the second surgical instrument 34b defined by the corresponding motion trail equation is solved as the motion of the joint of the second tool arm 32b by the robot inverse kinematics algorithm, so that the master motion scheme may be obtained. In this embodiment, a motion scheme is preliminarily planned through a first motion trajectory which is artificially planned (that is, the motion scheme defined by the first motion trajectory equation and the motion scheme defined by the second motion trajectory equation are rough motion schemes and not necessarily final motion schemes), and then an optimal motion scheme is obtained by using a trajectory calculation unit, so that the second surgical instrument 34b can move smoothly when executing the motion scheme.
Optionally, the control unit comprises a first control unit and a second control unit, wherein the first control unit may be disposed on the image display device 20 or the doctor console 10 and comprises the trajectory recognition module 61, the selection module 62 and the storage module 63. The second control unit may include the microneedle calculation unit 64 and the trajectory calculation unit 65, and the second control unit may be provided on the surgeon console 10 or on the surgical operation device 30. In other implementations, the first control unit and the second control unit may also be integrated. That is, the present embodiment does not particularly limit how the control unit is specifically provided.
Furthermore, those skilled in the art will appreciate that in alternative implementations, the process of obtaining the first and second trajectory equations is not required. In other words, the control unit may directly calculate the position relationship between the second surgical instrument 34b and the surgical field 33 'by using the position calculation unit, and then the trajectory calculation unit may plan the motion scheme of the second tool arm 32b according to the position relationship between the second surgical instrument 34b and the surgical field 33'.
In this embodiment, the control unit (specifically, the trajectory calculation unit) may determine the motion scheme by using any one of a conventional polynomial trajectory design, such as an nth-order polynomial spline difference method (n ≧ 5), an S-type trajectory design, or a T-type trajectory design. The following description will take an example in which the control unit determines the movement of the second tool arm 32b by using a T-shaped trajectory design method.
In the motion scheme designed by the T-shaped trajectory design method, the motion scheme of the second tool arm 32b includes: a uniform acceleration motion stage, a uniform velocity motion stage and a uniform deceleration motion stage. In the course of executing the exercise program, the position, speed, acceleration of the second surgical instrument 34b are plotted against time, please refer to fig. 10a, 10b and 10 c. And, when the second surgical instrument 34b is in a uniform acceleration phase of motion while executing the motion profile, the position of the second surgical instrument 34b may be calculated by the following equation:
Figure BDA0002932509330000291
the velocity of the second surgical instrument 34b may be calculated by the following equation:
Figure BDA0002932509330000292
the acceleration of the second surgical instrument 34b is:
Figure BDA0002932509330000293
when the second surgical device 34b is in the uniform motion stage, the position of the second surgical device 34b can be calculated by the following formula:
Figure BDA0002932509330000294
the speed of the second surgical instrument 34b is:
Figure BDA0002932509330000295
the acceleration of the second surgical instrument 34b is:
Figure BDA0002932509330000296
when the second surgical instrument 34b is in the uniform deceleration movement stage, the position of the second surgical instrument 34b can be calculated by the following formula:
Figure BDA0002932509330000297
the velocity of the second surgical instrument 34b is calculated by the following equation:
Figure BDA0002932509330000298
the acceleration of the second surgical instrument 34b is:
Figure BDA0002932509330000299
in the formula, q 0 Is the position of the second surgical instrument 34b at the beginning,
Figure BDA00029325093300002910
is the second manual tool34b at t c The position of the moment of time is,
Figure BDA0002932509330000302
is that the second surgical instrument 34b is at t j The location of the time of day. t is t c Is the moment at which the second surgical instrument 34b reaches a maximum speed, t j Is the moment at which the second surgical instrument 34b starts to decelerate, t f Is the moment when the velocity of the second surgical instrument 34b is zero.
In another embodiment, the control unit may also control the movement of the image arm 31 and/or the tool arm 32 to return the second surgical instrument 34b to the surgical field. Referring to fig. 11a, when the second surgical instrument 34b is positioned outside the surgical field 33', the control unit is configured to:
acquiring the end point E of the image arm 31 according to a robot kinematics method 1 And a stationary point R on the image arm 31 1 The position of said stationary point R 1 On the axis of said endoscope 33, at a stationary point R 1 Which may be a point on the badge of the endoscope 33 corresponding to the wound in the abdomen of the patient (commonly referred to as the belly point) or a point on the image arm 31 corresponding to the belly point.
Acquiring the end point T of the second tool arm 32b according to a robot kinematics method 1 And a stationary point R on the second tool arm 32b 2 The position of (a). Those skilled in the art will know how to make the stationary point R on the second tool arm 32b 2 I.e. a stationary point R on the second tool arm 32b, as is known to the person skilled in the art 2 Is clear.
Calculating straight line E 1 T 1 And a straight line T 1 R 1 The included angle theta is formed 1 Comprises the following steps:
Figure BDA0002932509330000301
thus, as shown in fig. 11b, the motion scheme determined by the control unit may include: the endoscope 33 andstationary point R on the image arm 31 1 Rotate the center of rotation in a first direction by theta 1 And (4) an angle.
Since the robotic surgical system further comprises a first surgical instrument (not shown in fig. 11a and 11 b), to ensure that the first surgical instrument does not leave the surgical field, the motion scheme further comprises: the first surgical instrument has a stationary point R of a first tool arm of the first surgical instrument 3 (shown in both FIGS. 11a and 11 b) is rotated by a rotation center in the first direction 1 And (4) an angle. Such that both the first and second surgical instruments 34b may be positioned within the surgical field 33'. Of course, if only the endoscope 33 is rotated in the first direction θ 1 After the angle, the first surgical instrument is still within the surgical field 33', and the first surgical instrument may also not rotate.
Or, the motion scheme determined by the control unit includes: the endoscope 33 first uses the stationary point R on the image arm 31 1 Rotate the center of rotation in a first direction by theta 1 An angle; thereafter, as shown in fig. 11c, the endoscope 33 moves to a stationary point R on the image arm 31 1 Rotate theta in the second direction for the center of rotation 2 Angle, and stationary point R of the second surgical instrument 34b on the tool arm 32 of the second surgical instrument 34b 2 Rotate theta in the second direction for the center of rotation 2 Angled to maintain the second surgical instrument 34b within the surgical field 33'. In order to ensure that the first surgical instrument 34a and the second surgical instrument 34b are both within the surgical field 33' in the presence of the first surgical instrument 34a, the second direction is opposite to the first direction, which is clockwise, as indicated by the arrow in fig. 11a, and counterclockwise, as indicated by the arrow in fig. 11c, taking the orientation shown in fig. 11a to 11c as an example. In so doing, the first surgical instrument 34a and the second surgical instrument 34b are both within the surgical field 33'. If it is desired to return operative field 33' completely to its original position, then there is θ 2 Is equal to theta 1 If the operative field 33' does not completely return toOriginal position, θ 2 May not be equal to theta 1 . However, if the first surgical instrument 34a is not present, the second direction may be the same as the first direction.
Further, an embodiment of the present invention also provides a computer-readable storage medium on which a program is stored, which, when executed, performs corresponding operations performed by the control unit as described above.
Further, the present invention also provides an electronic device comprising a processor and the computer-readable storage medium, wherein the processor is configured to execute the program stored on the computer-readable storage medium.
Although the present invention is disclosed above, it is not limited thereto. Various modifications and alterations of this invention may be made by those skilled in the art without departing from the spirit and scope of this invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (21)

1. A computer-readable storage medium having a program stored thereon, wherein when the program is executed, the steps of:
judging whether the surgical instrument is in the surgical field according to the surgical field information provided by the image acquisition device;
when the surgical instrument is positioned outside the surgical field of view, planning a motion scheme, and enabling an image arm connected with the image acquisition device and/or a tool arm connected with the surgical instrument to execute the motion scheme so as to enable the surgical instrument to return to the surgical field of view;
wherein the program further performs safety measures to bring the image acquisition device and/or the surgical instrument into a safe state before or during the execution of the motion profile by the image arm and/or the tool arm.
2. The computer-readable storage medium of claim 1, wherein prior to the image arm and/or the tool arm performing the motion profile, the program performs the following steps to perform the security measures;
and judging whether the tail end of the image acquisition device and/or the tail end of the surgical instrument are/is positioned in the corresponding stamp card, if not, driving the tail end of the image acquisition device and/or the tail end of the surgical instrument to move into the corresponding stamp card.
3. The computer-readable storage medium according to claim 2, characterized in that after the image arm and/or the tool arm executes the motion scheme, the program further performs the steps of:
and driving the end of the image acquisition device and/or the end of the surgical instrument to move and extend out of the corresponding stamp card so as to enable the surgical instrument to return to the surgical field.
4. The computer-readable storage medium of claim 1, wherein during execution of the motion profile by the image arm and/or the tool arm, the program performs the following steps to perform the security measures:
judging whether the characteristic value of the image arm and/or the tool arm is within a safety threshold value, if not, driving the image acquisition device and/or the surgical instrument to move a preset distance along the direction facing the outside of the body;
wherein the characteristic values comprise at least one of joint moments, joint positions or joint movement speeds of the image arm and/or the tool arm.
5. The computer-readable storage medium of claim 1, wherein during execution of the motion profile by the image arm and/or the tool arm, the program performs the following steps to perform the security measures;
judging whether the characteristic value of the image arm and/or the tool arm is within a safety threshold value, if not, stopping the image arm and/or the tool arm to execute the motion scheme;
wherein the characteristic values comprise at least one of joint moments, joint positions or joint movement speeds of the image arm and/or the tool arm.
6. The computer-readable storage medium of claim 1, wherein the program performs the steps to plan the motion profile:
planning a target position of the surgical instrument within the surgical field of view;
planning the motion plan according to the current position of the surgical instrument and the target position.
7. The computer-readable storage medium of claim 6, wherein the target position is a position that enables the surgical instrument to be restored into the surgical field and corresponds to its position before exiting the surgical field when the image acquisition device and/or the surgical instrument are moved such that the surgical instrument is not within the surgical field.
8. The computer-readable storage medium according to claim 6, wherein the program performs the steps of: and acquiring the coordinate of the central point C of the operation visual field, and taking the central point C of the operation visual field as the target position.
9. The computer-readable storage medium according to claim 6, wherein the program performs the steps of:
acquiring coordinates of a central point C of the surgical field;
and acquiring a spherical surface with the central point C as a spherical center, and taking any point in the sphere and on the spherical surface as the target position, wherein the radius of the spherical surface is the length of an end effector of the surgical instrument.
10. The computer-readable storage medium of claim 6, wherein the program performs the following steps to obtain a target position of the surgical instrument located outside the surgical field of view:
acquiring coordinates of a center point C of the surgical field and coordinates of a terminal point T of the surgical instrument located in the surgical field;
and calculating the coordinate of the middle point Z of a connecting line TC of the central point C and the tail end point T of the surgical instrument positioned in the surgical visual field, and taking the middle point Z as the target position of the surgical instrument positioned outside the surgical visual field.
11. The computer-readable storage medium of claim 6, wherein the program performs the following steps to obtain a target position of the surgical instrument located outside the surgical field of view:
acquiring coordinates of a central point C of the operation visual field and coordinates of a terminal point T of a surgical instrument positioned in the operation visual field;
calculating coordinates of a midpoint Z of a line TC connecting the center point C and a terminal point T of the surgical instrument located in the surgical field;
acquiring a spherical surface with a terminal point T of the surgical instrument positioned in the surgical field as a sphere center, wherein the radius of the spherical surface is the length of an end effector of the surgical instrument positioned in the surgical field;
judging whether the distance from the midpoint Z to a terminal point T of the surgical instrument positioned in the surgical field is larger than the radius of the spherical surface, if so, taking the midpoint Z as the target position; if not, selecting a point A on the connecting line TC as the target position, wherein the point A is positioned between the middle point Z and the central point C, the distance from the point A to a terminal point T of the surgical instrument positioned in the surgical field is m times of the radius of the spherical surface, and m is larger than 1.
12. The computer-readable storage medium according to any one of claims 8-11, wherein the program performs the steps of:
obtaining the coordinate of the central point C according to the coordinates of the terminal point N and the reference point M of the image acquisition device and the depth of field H of the image acquisition device; wherein the reference point M is a point which is fixed in position during the operation and is located on the axis of the image acquisition device.
13. The computer-readable storage medium according to claim 12, wherein the program performs the steps of:
obtaining the coordinates N (x) of the terminal point N 1 ,y 1 ,z 1 ) And the coordinates M (x) of said reference point M 2 ,y 2 ,z 2 )；
Calculating a direction vector in the axial direction of the image pickup device as
Figure FDA0002932509320000051
Calculating the coordinate C (x) of the center point C of the surgical field 3 ,y 3 ,z 3 ) Comprises the following steps: c (x) 3 ，y 3 ，z 3 )＝N(x 1 ,y 1 ,z i )+H×E NM 。
14. The computer-readable storage medium of claim 6, wherein the program performs the steps to plan the motion profile:
obtaining a first motion trail equation according to a first motion trail of the surgical instrument planned by one person; the starting point of the first motion track is the current position of the surgical instrument, and the ending point of the first motion track is the target position;
selecting a preset motion trail equation matched with the first motion trail to serve as a second motion trail equation;
performing combined optimization on the first motion trail equation and the second motion trail equation to obtain a third motion trail equation;
acquiring the relative position relation between the surgical instrument and the surgical visual field;
determining the motion scheme according to the relative position relation of the surgical instrument and the surgical field; the motion profile is defined by the first motion trajectory equation, or by the second motion trajectory equation, or by the third motion trajectory equation.
15. The computer-readable storage medium of claim 1, wherein the program performs the steps to plan the motion profile:
acquiring a terminal point E of the image acquisition device 1 And the stationary point R of the image arm 1 The coordinates of (a);
obtaining a tip point T of the tool arm 1 The coordinates of (a);
calculating straight line E 1 T 1 And a straight line T 1 R 1 The included angle theta is formed 1 ；
Determining the motion scheme, the motion scheme comprising: the image arm is at the fixed point R of the image arm 1 Rotate the center of rotation in a first direction by theta 1 And (4) an angle.
16. The computer-readable storage medium according to claim 15, wherein the program further performs the steps of:
obtaining a stationary point R of the tool arm 2 The coordinates of (a);
the motion scheme further comprises: the image arm is at the fixed point R of the image arm 1 Rotate theta in the second direction for the center of rotation 2 Angle, and the tool arm with a stationary point R on the tool arm 2 Rotate theta in the second direction for the center of rotation 2 An angle to maintain the surgical instrument within the surgical field of view.
17. The computer-readable storage medium of claim 1, wherein at least one of the surgical instruments is located within the surgical field of view, at least one surgical instrument is located outside the surgical field of view; the tool arm used for mounting the surgical instrument positioned in the surgical visual field is a first tool arm, and the tool arm used for mounting the surgical instrument positioned outside the surgical visual field is a second tool arm;
the program performs the following steps to plan the motion profile:
acquiring a terminal point E of the image acquisition device 1 And the stationary point R of the image arm 1 The position of (a);
obtaining a stationary point R of the first tool arm 3 And a distal point T of a surgical instrument located outside the surgical field of view 1 The position of (a);
calculating straight line E 1 T 1 And a straight line T 1 R 1 The included angle theta is formed 1 ；
Determining the motion scheme comprising the image arm at its motionless point R 1 Rotate the center of rotation in a first direction by theta 1 Angle, and the first tool arm with a stationary point R on the first tool arm 3 Rotate the center of rotation in a first direction by theta 1 An angle such that the surgical instrument mounted on the first tool arm remains within the surgical field while the surgical instrument mounted on the second tool arm is returned within the surgical field.
18. An electronic device comprising a processor and a computer-readable storage medium according to any of claims 1-16, the processor configured to execute a program stored on the computer-readable storage medium.
19. A surgical robotic system, comprising:
the image acquisition device is used for providing an operation visual field;
a tool arm for mounting a surgical instrument for performing a surgical procedure within the surgical field; and the number of the first and second groups,
a control unit configured to execute a program stored on the computer readable storage medium of any one of claims 1-17.
20. A surgical robotic system as claimed in claim 19, further comprising an input device, the control unit planning the motion profile in accordance with instructions input by the input device.
21. A surgical robotic system as claimed in claim 19, comprising the electronic device of claim 18, wherein the control unit comprises the processor.
CN202110152671.XA 2021-02-03 2021-02-03 Computer-readable storage medium, electronic device, and surgical robot system Pending CN114848152A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110152671.XA CN114848152A (en) 2021-02-03 2021-02-03 Computer-readable storage medium, electronic device, and surgical robot system
PCT/CN2022/075224 WO2022166929A1 (en) 2021-02-03 2022-01-30 Computer-readable storage medium, electronic device, and surgical robot system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110152671.XA CN114848152A (en) 2021-02-03 2021-02-03 Computer-readable storage medium, electronic device, and surgical robot system

Publications (1)

Publication Number Publication Date
CN114848152A true CN114848152A (en) 2022-08-05

Family

ID=82622735

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110152671.XA Pending CN114848152A (en) 2021-02-03 2021-02-03 Computer-readable storage medium, electronic device, and surgical robot system

Country Status (1)

Country Link
CN (1) CN114848152A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117204955A (en) * 2023-11-06 2023-12-12 华东交通大学 Force control system of flexible surgical robot and device thereof

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117204955A (en) * 2023-11-06 2023-12-12 华东交通大学 Force control system of flexible surgical robot and device thereof

Similar Documents

Publication Publication Date Title
US11931123B2 (en) Robotic port placement guide and method of use
US20190209262A1 (en) Medical robotic system providing an auxiliary view of articulatable instruments extending out of a distal end of an entry guide
EP4245246A2 (en) Systems for monitoring patient motion during a medical procedure
US8918207B2 (en) Operator input device for a robotic surgical system
US8583274B2 (en) Method for graphically providing continuous change of state directions to a user of medical robotic system
EP2349053B1 (en) Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the positioning and orienting of its tip
EP3092968B1 (en) System for hand presence detection in a minimally invasive surgical system
EP2498711B1 (en) Apparatus for hand gesture control in a minimally invasive surgical system
CN116035699A (en) System and method for constraining a virtual reality surgical system
US9844416B2 (en) Medical manipulator and method of controlling the same
US10111678B2 (en) Medical manipulator and method of controlling the same
EP3503834A1 (en) Systems and methods for monitoring patient motion during a medical procedure
US20140194699A1 (en) Single port surgical robot and control method thereof
US20230110890A1 (en) Systems and methods for entering and exiting a teleoperational state
JP7469120B2 (en) Robotic surgery support system, operation method of robotic surgery support system, and program
EP2442743A1 (en) Virtual measurement tool for minimally invasive surgery
US11950869B2 (en) System and method for providing on-demand functionality during a medical procedure
CN114848152A (en) Computer-readable storage medium, electronic device, and surgical robot system
JPH08117238A (en) Surgical manipulator
Bihlmaier et al. Endoscope robots and automated camera guidance
KR20200108124A (en) Surgical robot system for minimal invasive surgery and drive method thereof
Yang et al. Enhancement of spatial orientation and haptic perception for master-slave robotic Natural Orifice Transluminal Endoscopic Surgery
CN114848153A (en) Computer-readable storage medium, electronic device, and surgical robot system
WO2022166929A1 (en) Computer-readable storage medium, electronic device, and surgical robot system
US20210212773A1 (en) System and method for hybrid control using eye tracking

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination