CN115869069A - Surgical robot control method, device, equipment, medium and system - Google Patents
Surgical robot control method, device, equipment, medium and system Download PDFInfo
- Publication number
- CN115869069A CN115869069A CN202111145634.2A CN202111145634A CN115869069A CN 115869069 A CN115869069 A CN 115869069A CN 202111145634 A CN202111145634 A CN 202111145634A CN 115869069 A CN115869069 A CN 115869069A
- Authority
- CN
- China
- Prior art keywords
- instrument
- pose
- robot
- endoscope
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000000007 visual effect Effects 0.000 claims abstract description 56
- 230000008859 change Effects 0.000 claims abstract description 47
- 230000008569 process Effects 0.000 claims abstract description 26
- 230000033001 locomotion Effects 0.000 claims description 64
- 230000009466 transformation Effects 0.000 claims description 35
- 230000009471 action Effects 0.000 claims description 11
- 238000009434 installation Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 11
- 230000003287 optical effect Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000012546 transfer Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000001356 surgical procedure Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000026058 directional locomotion Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Landscapes
- Manipulator (AREA)
Abstract
The embodiment of the invention discloses a surgical robot control method, a surgical robot control device, a surgical robot control medium and a surgical robot control system. When the method detects that the end endoscope of the endoscope holding robot moves, the endoscope reference pose of the end endoscope under the endoscope holding robot base coordinate system is determined, the target pose of the end instrument of the instrument robot under the instrument robot base coordinate system is determined based on the endoscope reference pose, and then the actual pose of the end instrument is adjusted based on the target pose, so that the pose change of the end instrument in the display visual field of the end endoscope meets the preset change condition in the moving process of the end endoscope, the automatic adjustment of the instrument pose in the moving visual field is realized, the operating position of the handle corresponds to the position of the instrument in the display visual field, an operator does not need to frequently switch among the operation, the visual field moving and the clutch process to adjust the instrument pose, the adjustment efficiency of the instrument pose is improved, and the safety risk caused by operation errors when the instrument pose is manually adjusted is reduced.
Description
Technical Field
The embodiment of the invention relates to the technical field of surgical robots, in particular to a surgical robot control method, device, equipment, medium and system.
Background
The operator needs to adjust the position of the endoscope frequently during the operation so as to keep a clear view of the operation area. Such adjustments include displacements and rotations. The action of adjusting the visual field is usually a specific action which is executed by the two hands of an operator in a matching way, and correspondingly, the endoscope holding robot can move the endoscope according to the action executed by the operator and the preset rule in the controller. After the endoscope moves, the visual field changes relatively in the screen, so that the hand action of an operator is not matched with the instrument posture of the instrument robot any longer, and the accurate operation action is difficult to continue. Therefore, the operator needs to actively disconnect the master-slave mapping operation, manually adjust the handle to the position convenient for operation, ensure the positions of the two-hand operation match with the positions of the instrument in the visual field, and then return to the master-slave state, i.e. the clutch operation.
Disclosure of Invention
The embodiment of the invention provides a control method, a control device, a control medium and a control system of a surgical robot, which are used for realizing automatic adjustment of the pose of an instrument during moving a visual field, improving the adjustment efficiency of the pose of the instrument and reducing the safety risk caused by misoperation during manual adjustment of the pose of the instrument.
In a first aspect, an embodiment of the present invention provides a surgical robot control method, where the method includes:
if the movement of a terminal cavity mirror of the robot holding the mirror is detected, determining a cavity mirror reference pose of the terminal cavity mirror under a base coordinate system of the robot holding the mirror;
determining a target pose of a terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose;
and adjusting the actual pose of the end instrument based on the target pose so that the pose change of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
In a second aspect, an embodiment of the present invention further provides a surgical robot control apparatus, including:
the reference pose determining module is used for determining the reference pose of the end endoscope under the base coordinate system of the mirror holding robot if the movement of the end endoscope of the mirror holding robot is detected;
the target pose determination module is used for determining a target pose of a terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose;
and the instrument pose adjusting module is used for adjusting the actual pose of the end instrument based on the target pose so that the change of the pose of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
In a third aspect, an embodiment of the present invention further provides a surgical robot control system, where the system includes a mirror holding robot, at least one instrument robot, a handle, a display, and a controller, where the mirror holding robot includes an end scope, and the instrument robot includes an end instrument; wherein,
the display is used for displaying the display visual field of the end cavity mirror;
the controller is configured to adjust an actual pose of the end instrument based on the surgical robot control method provided in any embodiment of the present invention, so that a change in the pose of the end instrument in the display field of the end scope satisfies a preset change condition during the movement of the end scope.
In a fourth aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:
one or more processors;
a storage device to store one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a surgical robot control method as provided by any of the embodiments of the invention.
In a fifth aspect, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the surgical robot control method according to any of the embodiments of the present invention.
The embodiment of the invention has the following advantages or beneficial effects:
when the movement of the end endoscope of the endoscope holding robot is detected, the endoscope reference pose of the end endoscope under the endoscope holding robot base coordinate system is determined, the target pose of the end instrument of the instrument robot under the instrument robot base coordinate system is determined based on the endoscope reference pose, and then the actual pose of the end instrument is adjusted based on the target pose, so that the pose change of the end instrument in the display visual field of the end endoscope meets the preset change condition in the movement process of the end endoscope, the automatic adjustment of the instrument pose in the movement visual field is realized, the operating position of the handle corresponds to the position of the instrument in the display visual field, an operator does not need to frequently switch among the operation, the visual field movement and the clutch process to adjust the instrument pose, the adjustment efficiency of the instrument pose is improved, and the safety risk caused by operation errors in the manual adjustment of the instrument pose is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, a brief description is given below of the drawings used in describing the embodiments. It should be clear that the described figures are only views of some of the embodiments of the invention to be described, not all, and that for a person skilled in the art, other figures can be derived from these figures without inventive effort.
Fig. 1A is a schematic flowchart of a surgical robot control method according to an embodiment of the present invention;
FIG. 1B is a schematic diagram of the movement of an end cavity mirror according to one embodiment of the present invention;
fig. 2 is a schematic flowchart of a surgical robot control method according to a second embodiment of the present invention;
fig. 3A is a schematic structural diagram of a surgical robot control system according to a third embodiment of the present invention;
fig. 3B is a schematic structural diagram of another surgical robot control system according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a surgical robot control device according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a schematic flow chart of a surgical robot control method according to an embodiment of the present invention, which is applicable to automatically adjust the pose of an instrument when an operator adjusts the position of an endoscope to perform a surgical operation on another position, so that the instrument does not have a relative change in the pose in a moving display field of view, and the method may be implemented by a surgical robot control apparatus, which may be implemented by hardware and/or software, and specifically includes the following steps:
and S110, if the movement of the end cavity mirror of the mirror-holding robot is detected, determining the cavity mirror reference pose of the end cavity mirror under the base coordinate system of the mirror-holding robot.
Wherein, the mirror holding robot can be a surgical robot with an endoscope at the tail end of a mechanical arm; the end scope of the scope holding robot may specifically be a lens such as an endoscope. Specifically, the end endoscope of the endoscope holding robot can shoot the surgical part; in the process of shooting the surgical site by the end endoscope, the visual field of the end endoscope usually also comprises the end instruments of the instrument robot, wherein the instrument robot can be a surgical robot with the end of a mechanical arm holding surgical instruments, so that an operator can observe the surgical site and perform surgical operation by controlling the end instruments of the instrument robot. The mirror holding robot and the instrument robot can be independent robots respectively or can be robots integrated together.
Illustratively, the operator controls the end instrument to perform the surgery by manipulating the handle; the handle can drive the mechanical arm of the instrument robot to carry out master-slave motion, and the moving mechanical arm drives the tail end instrument to move. When an operator needs to operate another part in the operation process, the end endoscope of the endoscope holding robot can be controlled to move so as to adjust the visual field of the end endoscope to the other part. Illustratively, the movement of the end cavity mirror includes a pointing movement of the lens in space, a positional movement of the lens in space, a rotation of the lens about a shooting axis, and the like. .
In this embodiment, if the end scope of the scope holding robot moves, the end instrument of the instrument robot is stationary, and therefore the position of the end instrument of the instrument robot in the display field of the end scope changes relatively. Illustratively, when the visual field of the end endoscope moves to the left, the position of the end instrument in the visual field moves to the right relatively; when the visual field of the end endoscope rotates clockwise, the position of the end instrument in the visual field can rotate anticlockwise relatively. Accordingly, when the position of the end instrument in the display visual field of the end endoscope changes relatively, the operation position of the handle does not correspond to the position of the end instrument in the display visual field. As shown in fig. 1B, a schematic diagram of the movement of the end scope is shown, in which the position of the end instrument in the field of view is relatively moved to the left and down when the end scope is moved to the right, and the operating position of the handle does not match the position of the end instrument in the field of view.
Therefore, in this embodiment, if the movement of the end endoscope of the endoscope-holding robot is detected, the instrument robot needs to be controlled to perform corresponding movements to ensure that the position of the end instrument in the display field of the end endoscope does not change, and further ensure that the operating position of the handle corresponds to the position of the end instrument in the display field. Specifically, if the movement of the end endoscope of the endoscope-holding robot is detected, the endoscope reference pose of the end endoscope in the endoscope-holding robot base coordinate system needs to be determined, so as to further determine the target pose of the end instrument of the instrument robot in the instrument robot base coordinate system based on the endoscope reference pose. Specifically, the base coordinate system of the mirror holding robot may be a base coordinate system with a center point of the mirror holding robot; the reference pose of the endoscope can be described by a homogeneous transfer matrix of the end endoscope under a base coordinate system of the endoscope holding robot.
And S120, determining the target pose of the terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose.
Wherein, the instrument robot base coordinate system can be a coordinate system with the instrument robot as a central point; the target pose may be a pose to which the end instrument needs to be adjusted, and specifically, the target pose may be a transformation relationship from the end instrument to the instrument robot. In this embodiment, the transformation relationship from the distal end instrument to the instrument robot can be determined according to the transformation relationship from the pose of the distal end endoscope to the pose of the endoscope holding robot.
In an alternative embodiment, the determining the target pose of the end instrument of the instrument robot in the instrument robot base coordinate system based on the endoscope reference pose includes: determining a first transformation relation of the instrument robot base coordinate system under the mirror-holding robot base coordinate system, a second transformation relation from the pose of the preset target point of the end endoscope under the installation coordinate system of the end endoscope to the pose of the preset target point in the display image of the end endoscope, and an instrument display pose of the end instrument of the instrument robot under a display visual field coordinate system; and determining the target pose of the end instrument in the instrument robot base coordinate system based on the first transformation relation, the second transformation relation, the instrument display pose and the endoscope reference pose.
The first transformation relation of the instrument robot base coordinate system under the mirror holding robot base coordinate system can be a transformation relation from the instrument robot to the mirror holding robot. The preset target point may be randomly set or determined by internal parameters of the lens, and specifically, the second transformation relation may be a conversion relation from the pose of the preset target point in the installation coordinate system of the end endoscope to the pose of the preset target point in the display image. The display field of view coordinate system may be a coordinate system centered on a display interface displaying the end lens field of view.
In this alternative embodiment, the target pose of the end instrument in the base coordinate system of the instrument robot may be determined according to the endoscope reference pose of the end endoscope in the base coordinate system of the endoscope robot, the transfer relationship (first transfer relationship) from the instrument robot to the endoscope robot, the instrument display pose of the end instrument of the instrument robot in the display field coordinate system, and the transfer relationship (second transfer relationship) from the pose of the preset target point in the installation coordinate system of the end endoscope to the pose of the preset target point in the display image. As shown in fig. 1B, the target pose may be the pose of the end instrument expected to move as shown in the right drawing of fig. 1B.
Illustratively, the target pose of the end instrument in the instrument robot base coordinate system is determined based on the coordinate system pose, the endoscope lens pose, the instrument display pose and the endoscope reference pose, and the following formula is satisfied:
wherein,
showing the target pose of the end instrument under the instrument robot base coordinate system,
indicating instrument display pose, T camera Representing a second transformation relation, T scope Indicating the reference pose of the endoscope, T registration Representing a first transformation relationship. Both the coordinates and the transformation can be described by a homogeneous transformation matrix.
Specifically, in the formula, the target pose of the end instrument in the instrument robot base coordinate system can be obtained by multiplying the instrument display pose, the transposed matrix of the second transformation relation, the endoscope reference pose, and the transposed matrix of the first transformation relation. By the mode, the target pose can be accurately determined, so that the pose of the terminal instrument in the display visual field is not changed at the time before and after the terminal endoscope moves, and the matching between the operating position of the handle and the position of the terminal instrument in the display visual field is ensured.
S130, adjusting the actual pose of the end instrument based on the target pose so that the pose change of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
In this embodiment, after the target pose of the end instrument in the instrument robot base coordinate system is determined, the actual pose of the end instrument may be adjusted based on the target pose, so that the change of the pose of the end instrument in the display field of the end endoscope satisfies the preset change condition. The preset change condition may be that the pose change value is zero, or the pose change value does not exceed a set threshold. Specifically, in the embodiment, the actual pose of the end instrument can be adjusted by taking the fact that the pose of the end instrument in the display visual field does not change at the moment before and after the movement of the end endoscope as the target.
It should be noted that the number of the instrument robots in the present embodiment may be one or more, for example, the number of the instrument robots is two. If the number of the instrument robots is multiple, when the movement of the tail end lens is detected, the actual pose of the tail end instrument of each instrument robot can be adjusted according to the target pose of the tail end instrument of each instrument robot, so that the change of the pose of each tail end instrument in the display visual field meets the preset change condition, and an operator can conveniently control the tail end instrument through the operating handle.
According to the technical scheme of the embodiment, when the movement of the end endoscope of the endoscope-holding robot is detected, the endoscope reference pose of the end endoscope under the endoscope-holding robot base coordinate system is determined, the target pose of the end instrument of the instrument robot under the instrument robot base coordinate system is determined based on the endoscope reference pose, and then the actual pose of the end instrument is adjusted based on the target pose, so that the change of the pose of the end instrument in the display visual field of the end endoscope meets the preset change condition in the movement process of the end endoscope, the operating position of the handle is ensured to correspond to the position of the instrument in the display visual field, the automatic adjustment of the instrument pose in the movement visual field is realized, an operator does not need to frequently switch to adjust the instrument pose in the operation, visual field movement and clutch processes, the adjustment efficiency of the instrument pose is improved, and the safety risk caused by operation errors in the manual adjustment of the instrument pose is reduced.
Example two
Fig. 2 is a schematic flow chart of a surgical robot control method according to a second embodiment of the present invention, where on the basis of the foregoing embodiment, optionally, the present embodiment further includes: and if the action of a preset part on the handle is detected, determining that the movement of the end endoscope of the endoscope holding robot is detected, wherein the preset part is used for controlling the end endoscope of the endoscope holding robot. Wherein explanations of the same or corresponding terms as those of the above embodiments are omitted. Referring to fig. 2, the surgical robot control provided in this embodiment includes the following steps:
s210, if the action of a preset component on the handle is detected, the movement of the terminal cavity mirror of the robot holding the mirror is determined to be detected, wherein the preset component is used for controlling the terminal cavity mirror of the robot holding the mirror.
The preset component can be an operation rocker on the handle, or an adjusting button on the handle. Specifically, the preset component in this embodiment may be a component for controlling a mechanical arm of the mirror holding robot, and when the component is preset by an operator to operate, the mechanical arm of the mirror holding robot may drive the end cavity mirror to move correspondingly.
Illustratively, an operator controls the endoscope holding robot to realize the movement of the end endoscope by operating two operation rocking rods on the handle through fingers, and the movement mode of the end endoscope comprises the directional movement of a lens in a space, the movement of the position of the lens in the space, the rotation of the lens around a shooting axis and the like; the degrees of freedom included in the above manner are distributed on the operation of the two operation rocking bars, for example, the left operation rocking bar shakes the spatial direction of the visual field of the corresponding end cavity mirror, and the right operation rocking bar shakes the movement of the corresponding end cavity mirror along a single direction and the rotation of the cavity mirror around the axis of the movement direction.
Optionally, the process that the preset component controls the end endoscope of the endoscope holding robot to move satisfies the following formula:
wherein, T scope Represents the pose of the end endoscope under the base coordinate system of the endoscope-holding robot, T registration A first transformation relation showing that the base mark of the instrument robot is under the base mark of the mirror-holding robot,
represents the pose, or position, of the end instrument in the instrument robot base coordinate system>
Representing the pose, T, of the tip instrument in a display field coordinate system control Representing the control space orientation, T, of said preset component camera And a second transformation relation representing the pose of the preset target point of the end cavity mirror in the installation coordinate system of the end cavity mirror to the pose of the preset target point in the display image of the end cavity mirror. Illustratively, if the predetermined component is an operating rocker, T control The spatial orientation can be controlled by operating a rocker.
According to the formula, when the action of the preset part controlled by the user is detected, the pose of the end endoscope under the base coordinate system of the robot holding the endoscope, namely the pose of the end endoscope needing to be moved, can be calculated according to the action (control space direction) of the preset part, and the end endoscope is controlled to move to the pose. Through this optional implementation, the preset component can be realized controlling the accurate movement of the end cavity mirror.
S220, if the movement of the end cavity mirror of the mirror-holding robot is detected, determining the cavity mirror reference pose of the end cavity mirror under the base coordinate system of the mirror-holding robot.
And S230, determining the target pose of the terminal instrument of the instrument robot in the instrument robot base coordinate system based on the endoscope reference pose.
And S240, adjusting the actual pose of the end instrument based on the target pose so that the pose of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
Optionally, the adjusting the actual pose of the end instrument based on the target pose includes: and generating a control signal based on the target pose, and sending the control signal to the instrument robot so as to control the instrument robot to adjust the actual pose of the tail end instrument based on the control signal.
Wherein the control signal may contain information of the degree of freedom of movement of the robotic arm of the instrument robot. Specifically, the instrument robot can control the mechanical arm to drive the tail end instrument to move according to the control signal so as to adjust the actual pose of the tail end instrument.
In an optional embodiment, after the adjusting the actual pose of the end instrument based on the target pose, the method further includes: judging whether the position of a control mechanical arm corresponding to the handle changes or not, and if not, starting a master-slave control mode between the handle and the instrument robot; the control mechanical arm is used for controlling the pose of the handle, the handle is a master device in the master-slave control mode, the instrument robot is a slave device, and the master device controls the slave device.
Wherein, the control mechanical arm can be a mechanical device for controlling the movement of the handle; the control arm may be mounted on a handle console. Specifically, the manipulator on the control mechanical arm can grasp the handle, if an operator unintentionally triggers the movable handle in the moving process of the end endoscope, the handle can drive the control mechanical arm connected with the handle to move, the control mechanical arm can acquire the motion information of the handle, at the moment, the operation position of the handle does not correspond to the position of the end instrument in the display visual field, and the actual position and posture of the handle need to be adjusted again so that the operation position of the handle corresponds to the position of the end instrument in the display visual field, or the position of the end instrument in the display visual field is adjusted according to the actual position and posture of the handle so that the operation position of the handle corresponds to the position of the end instrument in the display visual field. Of course, if the position of the control mechanical arm corresponding to the handle does not change, it indicates that the position of the handle does not change in the process of moving the end endoscope and adjusting the instrument pose of the end instrument, and the adjusted position of the end instrument in the display visual field is matched with the operation position of the handle, so that a master-slave control mode between the handle and the instrument robot can be started; the handle is a master device, the instrument robot is a slave device, and the master device controls the slave device. It should be noted that, according to the surgical robot control method provided in this embodiment, before the actual pose of the end instrument is adjusted based on the target pose, the master-slave control mode between the handle and the instrument robot may be disconnected until the actual pose of the end instrument is adjusted.
In the optional implementation mode, whether the position of the control mechanical arm corresponding to the handle changes or not is judged to ensure that the position of the handle does not change when the actual pose of the tail end instrument is adjusted, so that the position of the adjusted tail end instrument in the display field corresponds to the operating position of the handle, the accuracy of adjusting the pose of the instrument is improved, the situation that the position of the handle does not correspond to the position of the tail end instrument due to movement misoperation of an operator on the handle is avoided, and the safety risk of operation errors caused by position non-correspondence is reduced.
According to the technical scheme of the embodiment, if the action of the preset component on the handle is detected, the movement of the terminal cavity mirror of the mirror-holding robot is determined, the cavity mirror reference pose of the terminal cavity mirror under the base coordinate system of the mirror-holding robot is further determined, the target pose of the terminal instrument of the instrument robot under the base coordinate system of the instrument robot is determined based on the cavity mirror reference pose, and the actual pose of the terminal instrument is further adjusted based on the target pose, so that the change of the pose of the terminal instrument in the display visual field of the terminal cavity mirror meets the preset change condition in the moving process of the terminal cavity mirror, the automatic adjustment of the instrument pose in the moving visual field is realized, an operator does not need to frequently switch among the operation, the visual field moving and the clutch process to adjust the instrument pose, the adjustment efficiency of the instrument pose is improved, and the safety risk caused by misoperation in manual adjustment of the instrument pose is reduced.
EXAMPLE III
Fig. 3A is a schematic structural diagram of a surgical robot control system according to a third embodiment of the present invention, which is applicable to a situation where an operator adjusts a field of view during a surgical procedure to operate other surgical sites or tissues, and specifically includes a mirror holding robot 31, at least one instrument robot 32, a handle 33, a display 34, and a controller 35, where the mirror holding robot 31 includes an end scope 310, and the instrument robot 32 includes an end instrument 320; wherein, the display 34 is used for displaying the display field of view of the end cavity mirror 310; the controller 35 is configured to adjust the actual pose of the end instrument 320 based on the surgical robot control method provided in each of the above embodiments, so that the change in the pose of the end instrument 320 in the display field of the end scope 310 during the movement of the end scope 340 meets a preset change condition.
Optionally, the present embodiment further provides another surgical robot control system, and as shown in fig. 3B, a schematic structural diagram of the another surgical robot control system provided in the present embodiment is shown. Referring to fig. 3B, the surgical robot system includes an instrument robot 1, a surgical instrument 1-1, an instrument robot 2, a surgical instrument 2-1, a scope holding robot 3, an endoscope 3-1, a handle console 4, a control robot 5, a handle 6, a handle rocker 6-1, a controller 7, and a display device 8.
Wherein, the surgical instrument 1-1 is arranged at the tail end of the instrument mechanical arm 1, the surgical instrument 1-2 is arranged at the tail end of the instrument mechanical arm 2, and the endoscope 3-1 is arranged at the tail end of the endoscope holding mechanical arm 3; the control mechanical arm 5 is arranged on the handle control platform 4 and used for controlling the handle 6; the number of control robot arms 5 may be 2. The number of the handle rockers 6-1 can be 2, and the handle rockers are respectively arranged on the handle 6 and operated by the operator through finger shaking. The controller 7 may be installed in other devices to stand alone, or may be installed on the handle console 4. A display device 8 may be mounted on the handle console 4 for displaying the field of view of the endoscope 3-1.
Specifically, in the operation process, an operator drives the instrument mechanical arm 1 and the instrument mechanical arm 2 to move through an operation handle and a master-slave control mode of the handle, the operator controls the motion of the hand of the handle to drive the control mechanical arm 5 to move, the six-degree-of-freedom motion of the hand of the operator on the handle 6 is collected by the control mechanical arm 5, the collected motions of the two hands are used as motion instructions and sent to the controller 7, the controller 7 performs kinematic coordinate transformation according to the motion instructions, and the instrument mechanical arm 1, the instrument mechanical arm 2, the surgical instrument 1-1 installed at the tail end and the surgical instrument 2-1 are respectively driven to perform corresponding motions in space.
When an operator needs to move the visual field of the endoscope, the operator can actively close the master-slave control mode of the handle 6, namely, the mechanical arm 5 is controlled not to drive the mechanical arm 1 and the mechanical arm 2 to do master-slave motion, and meanwhile, the operator controls the endoscope 3-1 to realize the operation of moving the visual field by operating the two handle rockers 6-1 on the handle through fingers. At this time, when the controller 7 can detect the motion of the handle rocker 6-1, the endoscope reference pose of the endoscope 3-1 in the base coordinate system of the endoscope holding mechanical arm 3 is determined, the target pose of the surgical instrument 1-1 in the base coordinate system of the instrument robot 1 is further determined based on the endoscope reference pose, or the target pose of the surgical instrument 2-1 in the base coordinate system of the instrument robot 2 is further determined, and a pose adjusting instruction is further sent to the instrument robot 1 or the instrument robot 2 based on the target pose, so that the instrument robot 1 or the instrument robot 2 controls the mechanical arm to drive the surgical instrument 1-1 or the surgical instrument 1-2 to move.
The surgical robot control system provided by the embodiment can realize automatic adjustment of the pose of the instrument when the visual field is moved, and an operator does not need to frequently switch among the surgical operation, the visual field movement and the clutch process to adjust the pose of the instrument, so that the adjustment efficiency of the pose of the instrument is improved, and the safety risk caused by misoperation is reduced.
Example four
Fig. 4 is a schematic structural diagram of a surgical robot control apparatus according to a fourth embodiment of the present invention, which is applicable to automatically adjust the pose of an instrument when an operator adjusts the position of an endoscope to perform a surgical operation on another position, so that the instrument does not generate a relative change in the pose in a moving display field, and the apparatus specifically includes: a reference pose determination module 410, an object pose determination module 420, and an instrument pose adjustment module 430.
A reference pose determining module 410, configured to determine, if movement of an end endoscope of the mirror-holding robot is detected, a reference pose of the end endoscope under a base coordinate system of the mirror-holding robot;
the target pose determining module 420 is used for determining a target pose of the end instrument of the instrument robot in the instrument robot base coordinate system based on the endoscope reference pose;
an instrument pose adjusting module 430, configured to adjust an actual pose of the end instrument based on the target pose, so that a change in the pose of the end instrument in the display field of the end endoscope during the movement of the end endoscope meets a preset change condition.
Optionally, the target pose determination module 420 includes a first determination unit and a second determination unit, where the first determination unit is configured to determine a first transformation relationship between the base coordinate system of the instrument robot and the base coordinate system of the scope-holding robot, a second transformation relationship between the pose of the preset target point of the end endoscope in the installation coordinate system of the end endoscope and the pose of the preset target point in the display image of the end endoscope, and an instrument display pose of the end instrument of the instrument robot in the display view coordinate system; the second determination unit is configured to determine a target pose of the end instrument in an instrument robot base coordinate system based on the first transformation relation, the second transformation relation, the instrument display pose and the endoscope reference pose.
Optionally, the second determining unit is specifically configured to determine the target pose of the end instrument in the instrument robot base coordinate system according to the following formula:
wherein,
representing the target pose of the end instrument under the base coordinate system of the instrument robot,
indicating instrument display pose, T camera Representing a second transformation relation, T scope Indicating the reference pose of the endoscope, T registration Representing a first transformation relationship.
Optionally, the device includes a cavity mirror movement detection module, where the cavity mirror movement detection module is configured to determine that the movement of the end cavity mirror of the robot is detected if the movement of a preset component on the handle is detected, where the preset component is configured to control the end cavity mirror of the robot.
Optionally, the instrument pose adjusting module 430 is specifically configured to generate a control signal based on the target pose, and send the control signal to the instrument robot, so that the instrument robot is controlled based on the control signal to adjust the actual pose of the end instrument.
Optionally, the apparatus further includes a handle detection module, where the handle detection module is configured to determine whether a position of a control mechanical arm corresponding to the handle changes after the actual pose of the end instrument is adjusted based on the target pose, and if not, start a master-slave control mode between the handle and the instrument robot; the control mechanical arm is used for controlling the pose of the handle, the handle is a master device in the master-slave control mode, the instrument robot is a slave device, and the master device controls the slave device.
Optionally, the process that the preset component controls the end endoscope of the endoscope holding robot to move satisfies the following formula:
wherein, T scope Represents the pose of the end endoscope under the base coordinate system of the endoscope holding robot, T registration Showing a first transformation relation of the base mark of the instrument robot under the base mark of the mirror holding robot,
represents the pose, or position, of the end instrument in the instrument robot base coordinate system>
Representing the pose, T, of the distal instrument in a display field of view coordinate system control Representing the control space orientation, T, of said preset component camera And the second transformation relation from the pose of the preset target point of the end cavity mirror to the pose of the preset target point in the display image of the end cavity mirror is represented in the installation coordinate system of the end cavity mirror.
In the embodiment, when the movement of the end endoscope of the endoscope-holding robot is detected, the endoscope reference pose of the end endoscope under the endoscope-holding robot base coordinate system is determined through the reference pose determination module, the target pose of the end instrument of the instrument robot under the instrument robot base coordinate system is determined through the target pose determination module based on the endoscope reference pose, and then the actual pose of the end instrument is adjusted through the instrument pose adjustment module based on the target pose, so that the pose change of the end instrument in the display visual field of the end endoscope in the movement process of the end endoscope meets the preset change condition, the automatic adjustment of the instrument pose in the movement visual field is realized, an operator does not need to frequently switch in the operation, visual field movement and clutch process to adjust the instrument pose, the adjustment efficiency of the instrument pose is improved, and the safety risk caused by operation errors in the manual adjustment of the instrument pose is reduced.
The surgical robot control device provided by the embodiment of the invention can execute the surgical robot control method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
It should be noted that, the units and modules included in the system are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the embodiment of the present invention.
EXAMPLE five
Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention. FIG. 5 illustrates a block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 5 is only an example and should not bring any limitation to the function and the scope of use of the embodiment of the present invention. The device 12 is typically an electronic device that assumes the control functions of the surgical robot.
As shown in FIG. 5, electronic device 12 is embodied in the form of a general purpose computing device. The components of electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a memory 28, and a bus 18 that couples the various components (including the memory 28 and the processing unit 16).
The processor 16 executes various functional applications and data processing by running the program stored in the memory 28, for example, to implement the surgical robot control method provided by the above-described embodiment of the present invention, including:
if the movement of the end endoscope of the endoscope-holding robot is detected, determining an endoscope reference pose of the end endoscope under a base coordinate system of the endoscope-holding robot;
determining a target pose of a terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose;
and adjusting the actual pose of the end instrument based on the target pose so that the pose change of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
Of course, those skilled in the art can understand that the processor can also implement the technical solution of the surgical robot control method provided by any embodiment of the present invention.
Example six
A sixth embodiment of the present invention further provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the surgical robot control method steps provided in any of the embodiments of the present invention, the method comprising:
if the movement of a terminal cavity mirror of the robot holding the mirror is detected, determining a cavity mirror reference pose of the terminal cavity mirror under a base coordinate system of the robot holding the mirror;
determining the target pose of the terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose;
and adjusting the actual pose of the end instrument based on the target pose so that the pose change of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
Computer storage media for embodiments of the present invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A surgical robot control method, characterized in that the method comprises:
if the movement of the end endoscope of the endoscope-holding robot is detected, determining an endoscope reference pose of the end endoscope under a base coordinate system of the endoscope-holding robot;
determining a target pose of a terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose;
and adjusting the actual pose of the end instrument based on the target pose so that the change of the pose of the end instrument in the display visual field of the end endoscope meets a preset change condition in the moving process of the end endoscope.
2. The method of claim 1, wherein determining a target pose of an end instrument of the instrument robot in an instrument robot base coordinate system based on the endoscopic reference pose comprises:
determining a first transformation relation of the instrument robot base coordinate system under the mirror-holding robot base coordinate system, a second transformation relation from the pose of the preset target point of the end cavity mirror under the installation coordinate system of the end cavity mirror to the pose of the preset target point in the display image of the end cavity mirror, and an instrument display pose of the end instrument of the instrument robot under a display visual field coordinate system;
and determining the target pose of the end instrument in the instrument robot base coordinate system based on the first transformation relation, the second transformation relation, the instrument display pose and the endoscope reference pose.
3. The surgical robot control method of claim 2, wherein the determining the target pose of the end instrument in an instrument robot base coordinate system based on the coordinate system pose, the endoscopic lens pose, the instrument display pose, and the endoscopic reference pose satisfies the following equation:
wherein,
represents the target pose of the tail end instrument under the base coordinate system of the instrument robot, and is matched with the instrument robot>
Indicating instrument display pose, T camera Representing a second transformation relation, T scope Indicating the reference pose of the endoscope, T registration Representing a first transformation relationship.
4. The surgical robot control method according to claim 1, further comprising:
and if the action of a preset part on the handle is detected, determining that the movement of the end endoscope of the endoscope holding robot is detected, wherein the preset part is used for controlling the end endoscope of the endoscope holding robot.
5. The surgical robot control method of claim 1, wherein the adjusting the actual pose of the tip instrument based on the target pose comprises:
and generating a control signal based on the target pose, and sending the control signal to the instrument robot so as to control the instrument robot to adjust the actual pose of the tail end instrument based on the control signal.
6. The surgical robot control method of claim 4, further comprising, after the adjusting the actual pose of the tip instrument based on the target pose:
judging whether the position of a control mechanical arm corresponding to the handle changes or not, and if not, starting a master-slave control mode between the handle and the instrument robot;
the control mechanical arm is used for controlling the pose of the handle, the handle is a master device in the master-slave control mode, the instrument robot is a slave device, and the master device controls the slave device.
7. The surgical robot control method according to claim 4, wherein the preset component controls the movement of the end endoscope of the endoscope holding robot according to the following formula:
wherein, T scope Represents the pose of the end endoscope under the base coordinate system of the endoscope holding robot, T registration Showing a first transformation relation of the base mark of the instrument robot under the base mark of the mirror holding robot,
represents the pose of the terminal instrument in the instrument robot base coordinate system, based on the instrument robot base coordinate system>
Representing the pose, T, of the distal instrument in a display field of view coordinate system control Representing the control space orientation, T, of said preset component camera And the second transformation relation from the pose of the preset target point of the end cavity mirror to the pose of the preset target point in the display image of the end cavity mirror is represented in the installation coordinate system of the end cavity mirror.
8. A surgical robot control apparatus, characterized in that the apparatus comprises:
the reference pose determining module is used for determining the reference pose of the end endoscope under the base coordinate system of the mirror holding robot if the movement of the end endoscope of the mirror holding robot is detected;
the target pose determining module is used for determining a target pose of a terminal instrument of the instrument robot under the instrument robot base coordinate system based on the endoscope reference pose;
and the instrument pose adjusting module is used for adjusting the actual pose of the end instrument based on the target pose so that the change of the pose of the end instrument in the display visual field of the end endoscope meets the preset change condition in the moving process of the end endoscope.
9. A surgical robot control system is characterized by comprising a mirror holding robot, at least one instrument robot, a handle, a display and a controller, wherein the mirror holding robot comprises an end endoscope, and the instrument robot comprises an end instrument; wherein,
the display is used for displaying the display visual field of the end cavity mirror;
the controller is configured to adjust the actual pose of the end instrument based on the surgical robot control method according to any one of claims 1 to 7, so that the change in the pose of the end instrument in the display field of the end scope during the movement of the end scope satisfies a preset change condition.
10. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the surgical robot control method of any of claims 1-7.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a surgical robot control method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111145634.2A CN115869069A (en) | 2021-09-28 | 2021-09-28 | Surgical robot control method, device, equipment, medium and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111145634.2A CN115869069A (en) | 2021-09-28 | 2021-09-28 | Surgical robot control method, device, equipment, medium and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115869069A true CN115869069A (en) | 2023-03-31 |
Family
ID=85763670
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111145634.2A Pending CN115869069A (en) | 2021-09-28 | 2021-09-28 | Surgical robot control method, device, equipment, medium and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115869069A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116509449A (en) * | 2023-07-03 | 2023-08-01 | 深圳华大智造云影医疗科技有限公司 | Pose information determining method and device of mechanical arm and electronic equipment |
-
2021
- 2021-09-28 CN CN202111145634.2A patent/CN115869069A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116509449A (en) * | 2023-07-03 | 2023-08-01 | 深圳华大智造云影医疗科技有限公司 | Pose information determining method and device of mechanical arm and electronic equipment |
| CN116509449B (en) * | 2023-07-03 | 2023-12-01 | 深圳华大智造云影医疗科技有限公司 | Pose information determining method and device of mechanical arm and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107414842B (en) | Control device, robot, and robot system | |
| EP3342561B1 (en) | Remote control robot system | |
| JP6816364B2 (en) | Controls, robots, and robot systems | |
| US20150273689A1 (en) | Robot control device, robot, robotic system, teaching method, and program | |
| WO2022002159A1 (en) | Master-slave motion control method, robot system, device, and storage medium | |
| US20190060008A1 (en) | Positioning arm for a surgical navigation system | |
| JP6897396B2 (en) | Control devices, robot systems and control methods | |
| JP2017019068A (en) | Teaching device, robot, and robot system | |
| CN111599459A (en) | Control method and control device for remote surgery and surgery system | |
| JP2018153874A (en) | Presentation device, presentation method, program and work system | |
| CN115869069A (en) | Surgical robot control method, device, equipment, medium and system | |
| JP6488571B2 (en) | Teaching apparatus and robot system | |
| CN115703227A (en) | Robot control method, robot, and computer-readable storage medium | |
| CN116459007A (en) | Method, device and equipment for determining mechanical arm configuration of surgical robot system | |
| CN115869068A (en) | Pose adjusting method, device, equipment, medium and surgical robot system | |
| CN113384347B (en) | Robot calibration method, device, equipment and storage medium | |
| KR20230065881A (en) | Robot teaching system | |
| CN117084788A (en) | Method and device for determining target gesture of mechanical arm and storage medium | |
| CN115213894A (en) | Robot image display method, display system, and recording medium | |
| JP2017159429A (en) | Robot control device, information processing device, and robot system | |
| WO2023040897A1 (en) | Method and system for calculating spatial registration pose of surgical robot | |
| JP2019111588A (en) | Robot system, information processor, and program | |
| JP2024044796A (en) | Image generating device | |
| JP7340069B1 (en) | Marker position registration program, marker position registration device, marker position registration method, and marker used for the method | |
| JP2018001321A (en) | Robot, robot control device and robot system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |