WO2023274099A1

WO2023274099A1 - Alignment method, surgical robot, and computer storage medium

Info

Publication number: WO2023274099A1
Application number: PCT/CN2022/101377
Authority: WO
Inventors: 李思平; 程陈; 何超
Original assignee: 上海微创医疗机器人（集团）股份有限公司
Priority date: 2021-06-30
Filing date: 2022-06-27
Publication date: 2023-01-05
Also published as: CN113456220A; CN113456220B

Abstract

An alignment method, a surgical robot, and a computer storage medium, for use in aligning robotic arms (11) disposed on a patient operating end (10) with trocars (12). The method comprises: acquiring current relative poses of a patient operating end (10) and trocars (12) and establishing a virtual three-dimensional model; and according to the virtual three-dimensional model of the current relative poses, planning set configurations of robotic arms (11) that have been aligned with the trocars (12), so as to guide the robotic arms (11) to perform an alignment operation with the trocars (12). The method further improves the alignment efficiency and convenience of the robotic arms (11) and the trocars (12) while improving the alignment accuracy of the robotic arms (11) and the trocars (12).

Description

Alignment method, surgical robot and computer storage medium

technical field

The present application relates to the technical field of medical devices, in particular to an alignment method, a surgical robot and a computer storage medium.

Background technique

Robotic technology for minimally invasive surgery can significantly improve the ability of surgeons to perform surgery. During minimally invasive surgery, it is necessary to use a guide tube (trocar) to provide a channel for surgical instruments to enter the cavity. Before the operation, the alignment of the guide tube and the mechanical arm requires a lot of preoperative preparation time, which affects the accuracy of the operation. efficiency.

In view of the above problems, in an existing positioning device and positioning method of a surgical robot mechanical arm, the positioning device of the surgical robot mechanical arm includes a poking card holder and an auxiliary positioning unit, which are used to determine the relationship between the mechanical arm and the surgical poking card ( That is, the accurate position of the connection of the guide tube); in this positioning device and positioning method, a mechanical transfer piece is used to achieve the purpose of improving the alignment accuracy of the guide tube and the mechanical arm to a certain extent, but it cannot be better. The convenience and quickness of the operation still require a long preoperative preparation time.

Therefore, it is necessary to propose an alignment method, a surgical robot, and a computer storage medium to improve the efficiency and convenience of aligning the guide tube with the robotic arm.

Contents of the invention

The purpose of the present application is to provide an alignment method, a surgical robot and a computer storage medium, so that while improving the alignment accuracy of the robotic arm and the guiding tube, the alignment efficiency and convenience of the robotic arm and the guiding tube are also improved.

To achieve the above purpose, the present application provides an alignment method for aligning the robotic arm with the guide tube, including:

Obtaining the current relative pose of the patient operating end and the guide tube and establishing a virtual three-dimensional model, the mechanical arm is set on the patient operating end;

According to the virtual three-dimensional model of the current relative pose, a set configuration of the mechanical arm aligned with the guide tube is planned, so as to guide the mechanical arm to perform an alignment operation with the guide tube.

Optionally, the alignment method also includes:

performing virtual-real fusion registration of the set configuration of the robotic arm and the current configuration of the robotic arm, and displaying it in an augmented reality device; and,

According to the set configuration of the mechanical arm displayed in the augmented reality device, guide the mechanical arm to perform an alignment operation with the guide tube.

Optionally, the step of obtaining the current relative pose of the patient operation end and the guide tube includes: establishing a coordinate mapping relationship between the coordinate system of the patient operation end and the coordinate system of the guide tube by using a positioning device.

Optionally, the positioning device is separated from the augmented reality device, and the step of using the positioning device to establish a coordinate mapping relationship between the coordinate system of the patient's operating end and the coordinate system of the guide tube includes:

In the world coordinate system, establish a coordinate mapping relationship between the coordinate system of the positioning device and the coordinate system of the patient operation terminal and the coordinate system of the augmented reality device; and,

In the world coordinate system, a coordinate mapping relationship is established between the coordinate system of the positioning device and the coordinate system of the guide tube and the coordinate system of the augmented reality device, so that the coordinate system of the patient operation end is consistent with the coordinate system of the guide The coordinate system of the pipe establishes the coordinate mapping relationship.

Optionally, the positioning device is arranged on the augmented reality device, and the step of using the positioning device to establish a coordinate mapping relationship between the coordinate system of the patient's operating end and the coordinate system of the guide tube includes:

In the world coordinate system, establish a coordinate mapping relationship between the coordinate system of the augmented reality device and the coordinate system of the patient operation terminal; and,

In the world coordinate system, establish a coordinate mapping relationship between the coordinate system of the augmented reality device and the coordinate system of the guide tube, so that the coordinate system of the patient operation end and the coordinate system of the guide tube establish a coordinate mapping relationship, Wherein, the coordinate system of the positioning device establishes a coordinate mapping relationship with the coordinate system of the augmented reality device through a mechanical position, and establishes a coordinate mapping relationship between the coordinate system of the positioning device and the world coordinate system through a rotation matrix and a translation vector.

Optionally, the step of planning the set configuration of the robotic arm after being aligned with the guide tube includes: according to the coordinate mapping relationship established between the coordinate system of the patient's operating end and the coordinate system of the guide tube and the obtained The current configuration of the robotic arm is planned, and the set configuration of the aligned robotic arm is obtained by planning the motion path of the robotic arm.

Optionally, the mechanical arm includes a plurality of joints, and by planning the movement path of each joint, the set configuration of each joint of the mechanical arm after being aligned with the guide tube is obtained.

Optionally, the steps of performing virtual-real fusion registration and guiding the robotic arm to align with the guide tube include:

Step S41, judging the alignment sequence of the robotic arms to be aligned in the augmented reality device, and selecting the current robotic arm to be aligned;

Step S42, merging the set configuration of the robotic arm to be aligned with the robotic arm to be aligned in the augmented reality device;

Step S43 , displaying the set configuration of the Nth joint of the robotic arm to be aligned currently on the augmented reality device, so as to guide the alignment operation of the Nth joint.

Optionally, the step of judging the alignment sequence of the robotic arms to be aligned includes: according to the relative position of the augmented reality device and each of the robotic arms, select the distance from the augmented reality device from the closest to the furthest. The farthest robotic arm in turn serves as the currently to-be-aligned robotic arm.

Optionally, the steps of performing virtual-real fusion registration and guiding the robotic arm to align with the guide tube further include:

Step S44, during the process of guiding the alignment operation of the Nth joint, detect in real time whether the actual position of the Nth joint after alignment coincides with the ideal position;

If not, repeat step S43 until the actual position of the aligned Nth joint coincides with the ideal position;

If it coincides, the alignment operation of the Nth joint is completed, then the step S42 to the step S43 are repeatedly executed in a loop, and N is corrected to N+1 in repeating the step S43 until the currently to-be-aligned Alignment of all joints of the robotic arm is complete.

Step S45, judging whether the alignment operation of all the robotic arms to be aligned is completed, if not, repeating step S41 to step S44 in a loop, until prompting for the alignment of all the robotic arms to be aligned The alignment operation is complete.

The application also provides a surgical robot, including:

a patient-operated end provided with at least one robotic arm; and,

A controller is connected in communication with the robotic arm and is configured to execute the alignment method, so as to align the robotic arm with the guide tube.

Compared with the prior art, the technical solution of the present application has the following beneficial effects:

The alignment method, surgical robot, and computer storage medium of the present application obtain the current relative pose of the patient's operating end and the guide tube and establish a virtual three-dimensional model, and plan the setting of the mechanical arm aligned with the guide tube according to the virtual three-dimensional model The configuration realizes the alignment of the mechanical arm and the guide tube, so that while improving the alignment accuracy of the mechanical arm and the guide tube, the alignment between the mechanical arm and the guide tube can also be improved. Alignment efficiency and convenience, thereby improving the surgical efficiency and surgical effect of the surgical robot.

Description of drawings

FIG. 1 is a flowchart of a method for aligning a guide tube and a robotic arm according to an embodiment of the present application;

Fig. 2 is a schematic diagram of the process of aligning the guide tube and the robotic arm according to an embodiment of the present application;

Fig. 3 is a schematic diagram of the alignment of the guide tube and the mechanical arm according to an embodiment of the present application;

Fig. 4 is a schematic structural view of a mechanical arm according to an embodiment of the present application;

Fig. 5 is a schematic diagram of obtaining the current relative pose of the operating end of the patient and the guide tube according to an embodiment of the present application;

Fig. 6 is a schematic diagram of establishing a coordinate mapping relationship between the coordinate system of the patient's operating end and the coordinate system of the guide tube in an embodiment of the present application;

Fig. 7 is a schematic diagram of the separation of the positioning device and the augmented reality device according to an embodiment of the present application;

Fig. 8 is a schematic diagram of establishing a coordinate mapping relationship between the coordinate system of the patient's operating end and the coordinate system of the guide tube in another embodiment of the present application;

FIG. 9 is a schematic structural diagram of an augmented reality device with a positioning device according to an embodiment of the present application;

FIG. 10 is a schematic diagram of the coordinate mapping relationship between the augmented reality device with the positioning device and the positioning device shown in FIG. 9;

Fig. 11 is a schematic diagram of the principle of binocular vision positioning according to an embodiment of the present application;

Fig. 12 is a schematic diagram of the movement direction of each joint of the adjustment arm according to an embodiment of the present application;

Fig. 13 is a schematic diagram of the movement direction of each joint of the tool arm according to an embodiment of the present application;

Fig. 14 is a schematic diagram of the movement direction of each joint of the robotic arm according to an embodiment of the present application;

Fig. 15 is a schematic diagram of a real-time planning scene of a robot arm positioning path according to an embodiment of the present application;

Fig. 16 is a schematic diagram of joint-by-joint guiding alignment of a robotic arm according to an embodiment of the present application;

Fig. 17 is a schematic diagram of an operation scene of aligning a guide tube with a robotic arm in an augmented reality device according to an embodiment of the present application;

Fig. 18 is a schematic diagram of the surgical robot provided by an embodiment of the present application when it is working.

Wherein, the reference numerals of accompanying drawings 1 to 18 are explained as follows:

10-patient operating end; 11-mechanical arm; 111-adjustment arm; 1111-horizontal movement joint; 1112-first rotary joint; 1113-second rotary joint; 1114-third rotary joint; 112-tool arm; 1121- Fourth rotary joint; 1122-fifth rotary joint; 12-guiding tube; 13-target object; 20-positioning device; 30-augmented reality device; 40-ideal position; 41-actual position; 60-doctor control terminal; 70 - auxiliary equipment; 80 - tool trolley; 90 - image trolley.

detailed description

In order to make the purpose, advantages and features of the present application clearer, the alignment method, surgical robot and computer storage medium proposed in the present application will be further described in detail below. It should be noted that the drawings are all in very simplified form and use imprecise scales, and are only used to facilitate and clearly assist the purpose of illustrating the embodiments of the present application.

An embodiment of the present application provides an alignment method for aligning the robotic arm with the guide tube. Refer to FIG. 1. FIG. 1 is a flowchart of a method for aligning the guide tube and the robotic arm according to an embodiment of the present application. Alignment methods described include:

Step S1, obtaining the current relative pose of the patient operating end and the guide tube and establishing a virtual three-dimensional model, the mechanical arm is set on the patient operating end;

Step S2. According to the virtual three-dimensional model of the current relative pose, plan the set configuration of the robotic arm aligned with the guide tube, so as to guide the alignment operation between the robotic arm and the guide tube .

The method for aligning the guide tube and the mechanical arm provided by this embodiment will be described in more detail below.

According to step S1, the current relative pose (position and posture) of the patient operating end and the guide tube is obtained and a virtual three-dimensional model is established. The guide tube may already be inserted into the target subject.

3 and 4, the patient operation end 10 is provided with at least one mechanical arm 11, the mechanical arm 11 includes a connected adjustment arm 111 and a tool arm 112, and one end of the adjustment arm 111 is connected to the On the patient operation end 10, the other end of the adjustment arm 111 is connected to the front end of the tool arm 112, and the end of the tool arm 112 is provided with an interface for connecting with one end of the guide tube 12, so that the The tool arm 112 is connected after being aligned with the guide tube 12, and the other end of the guide tube 12 is inserted into the inside of the target object 13, and the number of the guide tube 12 is the same as that of the mechanical arm 11 to be aligned. same amount. The target object 13 may be a patient, and the target object 13 shown in FIG. 3 is a simulated human body model.

The step of obtaining the current relative pose of the patient operation end and the guide tube includes: referring to FIG. 5 , using a positioning device 20 to establish a coordinate mapping between the coordinate system of the patient operation end 10 and the coordinate system of the guide tube 12 relationship, the positioning device 20 may be a binocular vision positioning device or other modeling devices. Wherein, referring to FIG. 7, the positioning device 20 is separated from the augmented reality device 30. At this time, the positioning device 20 may be a trolley provided with a left camera and a right camera; or, referring to FIG. 9, the positioning device The device 20 is disposed on the augmented reality device 30, and at this time, the positioning device 20 may be a left and right module with a camera function. Preferably, the left camera and the right camera and the left module and the right module of the positioning device 20 are located on the same horizontal plane. The current relative pose of the patient operation end and the guide tube can be acquired through the external camera function on the positioning device and the target or feature value on the patient operation end and the guide tube.

Specifically, referring to FIG. 6, the positioning device 20 is separated from the augmented reality device 30, and the positioning device 20 is used to establish a coordinate mapping between the coordinate system of the patient operation end 10 and the coordinate system of the guide tube 12 The steps of a relationship include:

In the world coordinate system (X0, Y0, Z0), the coordinate system (X51, Y51, Z51) of the positioning device 20 is respectively connected with the coordinate system (X53, Y53, Z53) of the patient operating end 10 and the The coordinate system (X54, Y54, Z54) of the augmented reality device 30 establishes a coordinate mapping relationship; and,

In the world coordinate system (X0, Y0, Z0), the coordinate system (X51, Y51, Z51) of the positioning device 20 and the coordinate system (X52, Y52, Z52) of the guide tube 12 and the enhanced The coordinate system (X54, Y54, Z54) of the reality device 30 establishes a coordinate mapping relationship, so that the coordinate system (X53, Y53, Z53) of the patient operation end 10 and the coordinate system (X52, Y52, Z52) Establish coordinate mapping relationship.

Or, referring to FIG. 8, the positioning device 20 is arranged on the augmented reality device 30, and the coordinate system of the patient operation end 10 and the coordinate system of the guide tube 12 are established by using the positioning device 20 to map coordinates. The steps of a relationship include:

In the world coordinate system (X0, Y0, Z0), coordinate mapping is established between the coordinate system (X54, Y54, Z54) of the augmented reality device 30 and the coordinate system (X53, Y53, Z53) of the patient operation terminal 10 relationship; and,

In the world coordinate system (X0, Y0, Z0), establish a coordinate mapping relationship between the coordinate system (X54, Y54, Z54) of the augmented reality device 30 and the coordinate system (X52, Y52, Z52) of the guide tube 12 , so that the coordinate system (X53, Y53, Z53) of the patient operating end 10 and the coordinate system (X52, Y52, Z52) of the guide tube 12 establish a coordinate mapping relationship.

Wherein, continue to refer to FIG. 10, since the positioning device 20 is arranged on the augmented reality device 30, so that the relative coordinate relationship between the positioning device 20 and the augmented reality device 30 is fixed, then the positioning device 20 The coordinate system (X51, Y51, Z51) can establish a coordinate mapping relationship with the coordinate system (X54, Y54, Z54) of the augmented reality device 30 through the mechanical position. A known fixed relative position between devices 30 . The coordinate system (X51, Y51, Z51) of the positioning device 20 and the world coordinate system (X0, Y0, Z0) can establish a coordinate mapping relationship through the rotation matrix R and the translation vector t, the formula is as follows:

Wherein, P(x _c , y _c , z _c ) in FIG. 10 is the coordinate position of the patient operation end 10 or the guide tube 12; (x _c , y _c , z _c in the above formula (1) ) is the coordinates of the patient operation end 10 or the guide tube 12, if (x _c , y _c , z _c ) is the coordinates of the patient operation end 10, then (x _w , y _w , z _w ) is The coordinates of the guide tube 12, if (x _c , y _c , z _c ) are the coordinates of the guide tube 12, then (x _w , y _w , z _w ) are the coordinates of the patient operation end 10, M1 as a vector.

In addition, referring to FIG. 11 , the above-mentioned principle of using the positioning device to obtain the coordinate system of the patient's operating end, the coordinate system of the guide tube, and the coordinate system of the augmented reality device is as follows:

The positioning device includes a left camera and a right camera. In the coordinate system of the positioning device, the distance between the left camera and the right camera on the X axis is b; P1 and P2 are two positions with known coordinates point, taking the position of the left camera as the coordinate origin, the coordinates of the position point P1 are (x _l , y _l , f), taking the position of the right camera as the coordinate origin, the coordinates of the position point P2 are (x _r , y _r ,f); P(x,y,z) is the feature point on the patient operation end, the guide tube or the augmented reality device, the position point P1 is located in the left camera and the feature point P(x,y , z), the position point P2 is located on the connection line between the right camera and the feature point P(x, y, z), then, according to Figure 11, the formula is obtained:

The coordinates (x, y, z) of the following formula (3) of the feature point P (x, y, z) are obtained after conversion of the above formula (2), thus obtaining the coordinate system of the patient's operating end, the guide tube The coordinate system of and the coordinate system of the augmented reality device:

The augmented reality device may be AR glasses, AR headgear, etc., and the augmented reality devices shown in FIGS. 5 to 11 are AR glasses.

According to step S2, according to the virtual three-dimensional model of the current relative pose, a set configuration of the mechanical arm aligned with the guide tube is planned.

The set configuration is the ideal set position of each feature point on the mechanical arm after alignment, and the set configuration is also a virtual three-dimensional model.

Each of the mechanical arms includes a plurality of joints, and the step of planning the set configuration of the mechanical arm after being aligned with the guide tube includes: according to the coordinate system of the patient's operating end obtained in the step S1 and The coordinate mapping relationship established by the coordinate system of the guide tube and the current configuration of the robotic arm are used to obtain the set configuration of the aligned robotic arm by planning the movement path of the robotic arm. Wherein, the motion path of the robotic arm includes a rotation path and/or a translation path (ie, a positioning path) of each joint.

The adjustment arm may include a plurality of joints, for example, at least one horizontal movement joint and at least one rotation joint connected, and the tool arm includes at least one rotation joint, by planning the translation path of the horizontal movement joint and the rotation The path of rotation of the joints to obtain a set configuration for each joint of the robotic arm aligned with the guide tube. Taking the embodiment shown in Figures 12 to 14 as an example, the mechanical arm 11 includes a connected adjustment arm 111 and a tool arm 112, and the adjustment arm 111 includes a horizontal movement joint 1111, a first rotation joint 1112 connected in sequence , a second rotary joint 1113 and a third rotary joint 1114, the tool arm 112 includes a fourth rotary joint 1121 and a fifth rotary joint 1122 connected in sequence, the third rotary joint 1114 is connected to the fourth rotary joint 1121 . By planning the forward and backward translation of the horizontal movement joint 1111, the left and right rotation of the first rotary joint 1112, the up and down rotation of the second rotary joint 1113, the left and right rotation of the third rotary joint 1114, and the fourth rotation The left and right rotation of the joint 1121 and the up and down rotation path of the fifth rotary joint 1122 are used to obtain the set configuration of each joint of the robotic arm 11 after alignment. Wherein, each joint in the robotic arm may have multiple paths of rotation or translation, but the set configuration of each joint is unique.

Wherein, the forward and backward translation refers to the movement close to and away from the guide tube in the horizontal direction, the left and right rotation refers to the clockwise or counterclockwise rotation in the horizontal direction, and the up and down rotation refers to the clockwise or counterclockwise rotation in the vertical direction. rotate. It can be seen from FIG. 14 that for the first rotary joint 1112, the second rotary joint 1113, the third rotary joint 1114, the fourth rotary joint 1121 and the fifth rotary joint 1122, Clockwise or counterclockwise from the zero position corresponds to the angles of rotation θ1, θ2, θ3, θ4 and θ5, respectively, and remains within the limit; for the horizontal movement joint 1111, it can be translated in a straight line in the horizontal direction from the zero position L1mm, and keep within the limit. The angle ranges of θ1, θ2, θ3, θ4 and θ5 can be selected according to the rotation needs of different joints, and the range of L1 can also be selected according to the needs of horizontal movement.

It should be noted that the joints of the adjustment arm and the tool arm are not limited to those shown in FIGS. 12 to 14 , and may include other types and quantities of joints.

Moreover, since the perspective of the operator wearing the augmented reality device will change during the alignment process of the robotic arm and the guide tube (that is, the coordinate position of the augmented reality device will change), and, with According to the positioning of each joint of the mechanical arm, the actual position of the feature point on the mechanical arm may not reach the ideal position (for example, the positioning is too high). Then, the positioning path of the mechanical arm needs to be planned in real time. Wherein, if the change of the relative position between the positioning device and the augmented reality device is known (i.e. the embodiment shown in FIG. 6 ) or the relative position is fixed (i.e. the embodiment shown in FIG. 8 ), with wearing The change of the angle of view of the operator of the augmented reality device can perform real-time position matching according to the feature points on the mechanical arm, and adjust the position of the augmented reality device, the guide tube, and the patient in real time through the positioning device. The coordinate mapping relationship between the operating terminals realizes real-time calibration through the transformation of the coordinate position of the augmented reality device relative to the mechanical arm. Referring to FIG. 15, the positioning device 20 is separated from the augmented reality device 30. When the coordinate position of the augmented reality device 30 changes, that is, it moves from the coordinate system (X54, Y54, Z54) to the coordinate system (X54', Y54', Z54'), then, through the positioning device 20, adjust the relationship between the augmented reality device 30 and the guide tube 12 and the patient operation end 10 again according to the coordinate system (X54', Y54', Z54'). coordinate mapping relationship among them, and then replan the positioning path of the robotic arm.

The alignment method further includes a step S3 of performing virtual-real fusion registration of the set configuration of the robotic arm and the current configuration of the robotic arm, and displaying it in an augmented reality device.

That is to say, the virtual ideal position reached after the alignment of the feature points on the mechanical arm is aligned with the current actual position to perform virtual-real fusion coordinate registration. Since the mechanical arm is set on the patient operating end, and the position of the patient operating end is fixed, the set configuration of the mechanical arm and the current configuration of the mechanical arm after coordinate registration have the same The same origin of coordinates makes it possible to move the feature points on the robotic arm to the coordinates of the ideal position when the augmented reality device is used to guide the robotic arm to align with the guide tube.

The alignment method further includes a step S4 of guiding the robotic arm to align with the guide tube according to the set configuration of the robotic arm displayed in the augmented reality device.

Referring to FIG. 2, the steps of performing virtual-real fusion registration between the set configuration of the robotic arm and the current configuration and guiding the alignment operation of the robotic arm and the guide tube include:

Step S41, judging the alignment sequence of the robotic arms to be aligned in the augmented reality device, and selecting the current robotic arm to be aligned; the step of judging the alignment sequence of the robotic arms to be aligned It includes: according to the relative position between the augmented reality device and each of the robotic arms, preferentially select the robotic arm closest to the augmented reality device to start the alignment operation; according to the distance from the augmented reality device Align each robotic arm in sequence from near to far.

Step S42: In the augmented reality device, the set configuration of the currently-to-be-aligned robotic arm is fused to the currently-to-be-aligned mechanical arm, that is, the virtual set configuration Modeling graphics and data fusion are superimposed on the actual current to-be-aligned robotic arm.

Step S43, displaying the set configuration of the Nth joint of the robotic arm currently to be aligned on the augmented reality device, and the operator guides the alignment operation of the Nth joint through the real-time planned path and prompts, Wherein, N is a positive integer, and the initial N=1.

The steps of performing virtual-real fusion registration and guiding the manipulator to align with the guide tube also include:

Step S44, during the process of guiding the alignment operation of the Nth joint, detect in real time whether the actual position of the aligned feature point on the Nth joint coincides with the ideal position; if not, repeat the step S43 until The actual position after alignment of the Nth joint coincides with the ideal position; if it coincides, the alignment operation of the Nth joint is completed, and the steps S42 to S43 are repeated in a loop, and the N The correction is N+1 until the alignment operations of all joints of the robotic arm currently to be aligned are completed.

In addition, the steps of performing virtual-real fusion registration and guiding the alignment operation between the robotic arm and the guide tube also include:

Wherein, referring to FIG. 16 , taking the patient operating end provided with four mechanical arms (mechanical arm 1, mechanical arm 2, mechanical arm 3, and mechanical arm 4) as an example and each mechanical arm contains 6 joints, according to the mechanical The order of the distance between the arm and the augmented reality device from near to far can be adjusted arm by arm from arm 1 to arm 4; for each arm, alignment adjustment can be performed joint by joint, and the alignment adjustment of the current joint Start the alignment adjustment of the next joint when finished. Each joint on each robotic arm has position coordinates. For example, the position coordinates of the 6 joints on robotic arm 1 are (X101, Y101, Z101) ~ (X106, Y106, Z106), and the position coordinates of 6 joints on robotic arm 2 are The position coordinates of the joints are (X201, Y201, Z201) ~ (X206, Y206, Z206), and the position coordinates of the six joints on the robot arm 3 are (X301, Y301, Z301) ~ (X306, Y306, Z306 ), the position coordinates of the six joints on the mechanical arm 4 are (X401, Y401, Z401) ~ (X406, Y406, Z406), according to the coordinate system (X54, Y54, Z54) of the augmented reality device 30 and each The relative position between the position coordinates of each joint on the three robotic arms realizes joint-by-joint index alignment. The six joints on the mechanical arm can be the horizontal movement joint 1111, the first rotary joint 1112, the second rotary joint 1113, the third rotary joint 1114, the fourth rotary joint 1121 and The fifth rotary joint 1122 can take the above six joints as the first joint to the sixth joint in sequence, and perform alignment adjustment sequentially from the first joint to the sixth joint.

Referring to FIG. 17 , FIG. 17 shows a field of view of the first rotary joint 1112 of the adjustment arm displayed in the augmented reality device during the adjustment process. The first rotary joint 1112 needs to be changed from the actual position 41 (solid line) to ideal position 40 (dashed line).

In addition, when detecting whether the actual position after alignment of the Nth joint coincides with the ideal position, it can be confirmed by calculating the error between the adjusted actual position and the planned ideal position. If the error is within the allowable range of the required specification The inside overlaps. That is to say, the coincidence of the actual position and the ideal position after the alignment of the Nth joint means that the error between the actual position and the ideal position is within the range allowed by the specification.

In summary, the alignment method of the present application enables the accurate position and posture of the aligned mechanical arm to be obtained by analyzing the current relative pose of the patient's operating end and the guide tube; Establishing a virtual three-dimensional model of the current relative pose, planning the set configuration of the robotic arm after the alignment of the robotic arm and the guide tube, and conveniently decomposing the alignment operation to the robotic arm Each joint of each joint; by performing virtual-real fusion registration of the set configuration of the mechanical arm and the current configuration of the mechanical arm, and through the set configuration of the mechanical arm displayed in the augmented reality device, Guide each joint of the mechanical arm to the target position, so that the actual position after positioning coincides with the ideal position, and realize the alignment of the mechanical arm and the guide tube, so that when the mechanical arm is raised and the While improving the alignment accuracy of the guide tube, it can also improve the alignment efficiency and convenience of the mechanical arm and the guide tube.

Referring to FIG. 18 , based on the same inventive concept, an embodiment of the present application provides a surgical robot, including a patient operating end 10 and a controller (not shown), the patient operating end 10 is provided with at least one mechanical arm 11; The controller is communicatively connected with the robotic arm 11 and is configured to execute the alignment method, so as to align the robotic arm 11 with the guide tube.

The surgical robot also includes a doctor control terminal 60, auxiliary equipment 70, a tool cart 80 and an image trolley 90. After the mechanical arm 11 is aligned and connected to the guide tube, the Surgical instruments enter the inside of the target object 13 through the guide tube, and the doctor manipulates the surgical instruments through the doctor control end 60 to perform surgery.

Moreover, the specific setting method of the controller is not limited in the embodiment of the present application. In some embodiments, the controller can be set at the patient operation end 10 as a whole, or at the doctor control end 60 as a whole. , or a part is set at the patient operation end 10, and the other part is set at the doctor control end 60, as long as they can perform corresponding functions.

Based on the same inventive concept, an embodiment of the present application further provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the alignment method of an embodiment of the present application can be implemented.

The computer storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any of the above any suitable combination. More specific examples (a non-exhaustive list) of computer storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory) ), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punched cards with instructions stored thereon Or a raised structure in a groove, and any suitable combination of the above. The computer programs described herein may be downloaded from computer storage media to respective computing/processing devices, or downloaded to external computers or external storage devices over a network, such as the Internet, local area network, wide area network, and/or wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives the computer program from the network and forwards the computer program for storage in a computer storage medium in each computing/processing device.

The above description is only a description of the preferred embodiments of the application, and is not any limitation on the scope of the application. Any changes and modifications made by those of ordinary skill in the technical field of the application based on the above disclosures belong to the protection scope of the claims.

Claims

An alignment method for aligning a robotic arm with a guide tube, comprising:

Obtaining the current relative pose of the patient operating end and the guide tube and establishing a virtual three-dimensional model, the mechanical arm is set on the patient operating end;

According to the virtual three-dimensional model of the current relative pose, a set configuration of the mechanical arm aligned with the guide tube is planned, so as to guide the mechanical arm to perform an alignment operation with the guide tube.
The alignment method according to claim 1, further comprising:

performing virtual-real fusion registration of the set configuration of the robotic arm and the current configuration of the robotic arm, and displaying it in an augmented reality device; and,

According to the set configuration of the mechanical arm displayed in the augmented reality device, guide the mechanical arm to perform an alignment operation with the guide tube.
The alignment method according to claim 2, wherein the step of obtaining the current relative pose of the patient operation end and the guide tube comprises: using a positioning device to align the coordinate system of the patient operation end with the guide tube The coordinate system of the pipe establishes the coordinate mapping relationship.
The alignment method according to claim 3, wherein the positioning device is separated from the augmented reality device, and the coordinate system of the patient operation end and the coordinate system of the guide tube are established by using the positioning device The steps for mapping relationships include:

In the world coordinate system, establish a coordinate mapping relationship between the coordinate system of the positioning device and the coordinate system of the patient operation terminal and the coordinate system of the augmented reality device; and,

In the world coordinate system, a coordinate mapping relationship is established between the coordinate system of the positioning device and the coordinate system of the guide tube and the coordinate system of the augmented reality device, so that the coordinate system of the patient operation end is consistent with the coordinate system of the guide The coordinate system of the pipe establishes the coordinate mapping relationship.
The alignment method according to claim 3, wherein the positioning device is arranged on the augmented reality device, and the coordinate system of the patient's operating end and the coordinate system of the guide tube are established by using the positioning device The steps of the coordinate mapping relationship include:

In the world coordinate system, establish a coordinate mapping relationship between the coordinate system of the augmented reality device and the coordinate system of the patient operation terminal; and,

In the world coordinate system, establish a coordinate mapping relationship between the coordinate system of the augmented reality device and the coordinate system of the guide tube, so that the coordinate system of the patient operation end and the coordinate system of the guide tube establish a coordinate mapping relationship, Wherein, the coordinate system of the positioning device establishes a coordinate mapping relationship with the coordinate system of the augmented reality device through a mechanical position, and establishes a coordinate mapping relationship between the coordinate system of the positioning device and the world coordinate system through a rotation matrix and a translation vector.
The alignment method according to claim 3, wherein the step of planning the set configuration of the mechanical arm after aligning with the guide tube comprises: according to the coordinate system of the patient's operating end and the guide tube Based on the coordinate mapping relationship established by the coordinate system of the tube and the current configuration of the robotic arm, the set configuration of the aligned robotic arm is obtained by planning the movement path of the robotic arm.
The alignment method according to claim 6, wherein the mechanical arm includes a plurality of joints, and by planning the movement path of each joint, each joint of the mechanical arm aligned with the guide tube is obtained. The setting configuration of a joint.
The alignment method according to claim 2, wherein the steps of performing virtual-real fusion registration and guiding the robotic arm to align with the guide tube include:

Step S41, judging the alignment sequence of the robotic arms to be aligned in the augmented reality device, and selecting the current robotic arm to be aligned;

Step S42, merging the set configuration of the robotic arm to be aligned with the robotic arm to be aligned in the augmented reality device;

Step S43 , displaying the set configuration of the Nth joint of the robotic arm to be aligned currently on the augmented reality device, so as to guide the alignment operation of the Nth joint.
The alignment method according to claim 8, wherein the step of judging the alignment sequence of the robotic arms to be aligned comprises: according to the relative position of the augmented reality device and each of the robotic arms, select The robotic arms that are closest to the farthest from the augmented reality device are sequentially used as the currently to-be-aligned robotic arms.
The alignment method according to claim 8, wherein the step of performing virtual-real fusion registration and guiding the robotic arm to align with the guide tube further comprises:

Step S44, during the process of guiding the alignment operation of the Nth joint, detect in real time whether the actual position of the Nth joint after alignment coincides with the ideal position;

If not, repeat step S43 until the actual position of the aligned Nth joint coincides with the ideal position;

If it coincides, the alignment operation of the Nth joint is completed, then the step S42 to the step S43 are repeatedly executed in a loop, and N is corrected to N+1 in repeating the step S43 until the currently to-be-aligned Alignment of all joints of the robotic arm is complete.
The alignment method according to claim 10, wherein the step of performing virtual-real fusion registration and guiding the robotic arm to align with the guide tube further comprises:

Step S45, judging whether the alignment operation of all the mechanical arms to be aligned is completed, if not, repeating the step S41 to the step S44 in a loop until the prompt of all the alignment operations of the mechanical arms to be aligned The alignment operation is complete.
A surgical robot, characterized in that it comprises:

a patient-operated end provided with at least one robotic arm; and,

A controller, connected in communication with the robotic arm, and configured to execute the alignment method according to any one of claims 1-11, so as to achieve alignment of the robotic arm with the guide tube.
A computer storage medium, on which a computer program is stored, wherein the computer program implements the alignment method according to any one of claims 1-11 when executed by a processor.