CN116077155B

CN116077155B - Surgical navigation method based on optical tracking equipment and mechanical arm and related device

Info

Publication number: CN116077155B
Application number: CN202310359467.4A
Authority: CN
Inventors: 谢卫国; 欧阳挺; 张旭
Original assignee: Shenzhen Weide Precision Medical Technology Co ltd
Current assignee: Shenzhen Weide Precision Medical Technology Co ltd
Priority date: 2023-04-06
Filing date: 2023-04-06
Publication date: 2023-06-27
Anticipated expiration: 2043-04-06
Also published as: CN116077155A

Abstract

The embodiment of the application provides an operation navigation method and a related device based on optical tracking equipment and a mechanical arm, wherein the method comprises the following steps: determining a puncture path based on the position of the ultrasonic probe under the condition that the ultrasonic probe is used for scanning to a target point; the target point is determined according to the reconstructed target organ; sending a control instruction to the mechanical arm, wherein the control instruction is used for controlling the mechanical arm to move according to M markers; under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, generating first prompt information, wherein the first prompt information is used for prompting: performing puncturing along the puncture guide rail under the condition that the puncture needle is mounted on the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the position relationship between the puncture needle and the mechanical arm. The method provided by the application can reduce the time spent in the puncturing process and improve the puncturing precision.

Description

Surgical navigation method based on optical tracking equipment and mechanical arm and related device

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to an operation navigation method based on optical tracking equipment and a mechanical arm and a related device.

Background

With the continuous development of computer science and technology, medical technology has made a major breakthrough. Currently, many surgical scenarios require puncturing to obtain focal tissue for diagnosis or treatment, etc. For example, in percutaneous nephrolithotripsy, it is necessary to puncture with kidney stones as target points and then perform lithotripsy to treat kidney stones.

At present, the puncture operation depends on experience of operators such as doctors, the puncture process is long, and the puncture precision is low.

Disclosure of Invention

The embodiment of the application provides an operation navigation method based on optical tracking equipment and a mechanical arm and a related device.

In a first aspect, an embodiment of the present application provides a surgical navigation method based on an optical tracking device and a mechanical arm, which is applied to an electronic device in a surgical navigation system, where the surgical navigation system includes the electronic device, the optical tracking device, an ultrasonic device, and the mechanical arm; the mechanical arm is fixedly connected with the puncture guide, the puncture guide is fixedly connected with the puncture guide rail, and the puncture guide rail is used for clamping the puncture needle; m markers are arranged on the puncture guide, N markers are arranged on the puncture needle, K markers are arranged on an ultrasonic probe of the ultrasonic equipment, M and K are integers greater than or equal to 3, and N is an integer greater than or equal to 1; the optical tracking device positions the puncture guide by tracking the M markers, positions the ultrasonic probe by tracking the K markers, and positions the puncture needle by tracking the N markers; comprising the following steps:

Under the condition that the ultrasonic probe is used for scanning a target point, determining a puncture path based on the position of the ultrasonic probe; the target point is determined according to the reconstructed target organ;

transmitting a control instruction to the mechanical arm, wherein the control instruction is used for controlling the mechanical arm to move according to the M markers;

under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, generating first prompt information, wherein the first prompt information is used for prompting: when the puncture needle is attached to the puncture guide rail, performing puncture along the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm.

With reference to the first aspect, in a possible implementation manner, before the determining, by using the ultrasound probe to scan to a target, a puncture path based on a position of the ultrasound probe, the method further includes:

acquiring a computed tomography (computed tomography, CT) contour of the target organ, the CT contour being derived from CT images acquired by a CT apparatus;

Determining a first direction vector converted from a first reference direction under an ultrasonic coordinate system to a world coordinate system, a second direction vector converted from a second reference direction under the ultrasonic coordinate system to the world coordinate system, and a third direction vector converted from a third reference direction under the ultrasonic coordinate system to the world coordinate system; the first reference direction, the second reference direction, and the third reference direction are common to a space corresponding to the ultrasound apparatus and a space corresponding to the CT contour; the ultrasonic coordinate system is a coordinate system corresponding to the ultrasonic equipment;

determining a fourth direction vector of the first reference direction converted to the world coordinate system in a CT coordinate system, a fifth direction vector of the second reference direction converted to the world coordinate system in the CT coordinate system, and a sixth direction vector of the third reference direction converted to the world coordinate system in the CT coordinate system, wherein the CT coordinate system is a coordinate system corresponding to a CT device;

determining a rotational translation matrix based on a first angle and a first positional offset between the first direction vector and the fourth direction vector, a second angle and a second positional offset between the second direction vector and the fifth direction vector, and a third angle and a third positional offset between the third direction vector and the sixth direction vector;

Transforming the CT contour based on the rotation translation matrix to obtain a transformed CT contour;

acquiring an ultrasonic profile of the target organ, wherein the ultrasonic profile is obtained according to a plurality of ultrasonic images acquired by the ultrasonic probe, each ultrasonic image corresponds to different spatial position information, and the spatial position information is determined according to the K markers;

and registering based on the transformed CT contour and the ultrasonic contour to obtain the reconstructed target organ.

With reference to the first aspect, in a possible implementation manner, in a case that the ultrasonic probe is used to scan a target, determining a puncture path based on a position of the ultrasonic probe includes:

under the condition that the ultrasonic probe is used for scanning the target point, determining a reference point on the ultrasonic probe as a needle insertion point, wherein the reference point is positioned on a line segment where a scanning surface of the ultrasonic probe and the ultrasonic probe intersect;

and generating the puncture path based on the target spot and the needle insertion point.

With reference to the first aspect, in one possible implementation manner, the method further includes:

determining a reference plane, wherein an included angle between the reference plane and the puncture path is larger than or equal to a first threshold value, and the reference plane and the puncture path intersect at a first intersection point;

Displaying the reference plane and the first intersection point;

determining a first projection point of the first end of the puncture guide on the reference plane, and determining a second projection point of the second end of the puncture guide on the reference plane;

when the needle tip position of the virtual surgical needle coincides with the target point and the needle body of the virtual surgical needle coincides with the puncture path, generating first prompt information includes:

and generating the first prompt information when the needle point position of the virtual surgical needle coincides with the target point and the first projection point, the second projection point and the first intersection point are positioned on the same straight line.

displaying the first projection point, the second projection point, and any one or more of the following:

the distance from the target point to the straight line of the puncture guide rail, the distance from the needle inlet point to the straight line of the puncture guide rail, and the included angle between the straight line of the puncture guide rail and the puncture path.

In the process of installing a puncture needle on the puncture guide rail and puncturing along the puncture guide rail, determining the needle point position of the needle point of the puncture needle under an optical coordinate system based on the N markers in real time, wherein the optical coordinate system is a coordinate system corresponding to the optical tracking equipment;

displaying a first distance and a second distance, wherein the first distance is the distance between the needle point position of the puncture needle and the needle inlet point, and the second distance is the distance between the needle point position of the puncture needle and the target point;

and generating second prompt information under the condition that the needle point position of the puncture needle coincides with the target point, wherein the prompt information is used for prompting the puncture needle to reach the target point.

In a second aspect, embodiments of the present application provide a surgical navigation apparatus based on an optical tracking device and a robotic arm, comprising means for performing the method of the first aspect or any possible implementation of the first aspect.

In a third aspect, embodiments of the present application provide a surgical navigation system, the system including an electronic device, an optical tracking device, an ultrasonic device, and a robotic arm; the mechanical arm is fixedly connected with the puncture guide, the puncture guide is fixedly connected with the puncture guide rail, and the puncture guide rail is used for clamping the puncture needle; m markers are arranged on the puncture guide, N markers are arranged on the puncture needle, K markers are arranged on an ultrasonic probe of the ultrasonic equipment, M and K are integers greater than or equal to 3, and N is an integer greater than or equal to 1;

The optical tracking device is used for positioning the puncture guide by tracking the M markers, positioning the puncture needle by tracking the N markers and positioning the ultrasonic probe by tracking the K markers;

the electronic device is used for determining a puncture path based on the position of the ultrasonic probe under the condition that the ultrasonic probe is used for scanning a target point; the target point is determined according to the reconstructed target organ;

the electronic equipment is also used for sending a control instruction to the mechanical arm, and the control instruction is used for controlling the mechanical arm to move according to the M markers;

the mechanical arm is used for moving according to the control instruction;

the electronic device is further configured to generate first prompt information when the needle point position of the virtual surgical needle coincides with the target point, and the needle body direction of the virtual surgical needle coincides with the puncture path, where the first prompt information is used for prompting: when the puncture needle is attached to the puncture guide rail, performing puncture along the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm.

In a fourth aspect, an embodiment of the present application discloses an electronic device, including: a processor and a memory, wherein the memory has stored therein a computer program, the processor invoking the computer program stored in the memory for performing the method as in the first aspect or any of the possible implementations of the first aspect.

In a fifth aspect, the present application also provides another electronic device, including: a processor, a transmitting means, an input means, an output means and a memory for storing computer program code comprising computer instructions which, when executed by the processor, cause the electronic device to perform the method as in the first aspect or any one of the possible implementation manners of the first aspect.

In a sixth aspect, the present embodiments provide a computer readable storage medium having a computer program stored therein, which when run on one or more processors, causes the method as in the first aspect or any one of the possible implementation manners of the first aspect to be performed.

In a seventh aspect, the present embodiments provide a computer program product comprising program instructions which, when executed by a processor, cause the processor to perform a method as in the first aspect or any one of the possible implementations of the first aspect.

In the embodiment of the application, under the condition that an ultrasonic probe is used for scanning a target point, a puncture path is determined based on the position of the ultrasonic probe; the target point is determined according to the reconstructed target organ; sending a control instruction to the mechanical arm, wherein the control instruction is used for controlling the mechanical arm to move according to M markers; under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, generating first prompt information, wherein the first prompt information is used for prompting: performing puncturing along the puncture guide rail under the condition that the puncture needle is mounted on the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm. In the embodiment, compared with a puncture path directly determined on a reconstruction model, a puncture path determined through a scanning process of ultrasonic equipment is more accurate, so that puncture precision is higher; then the electronic equipment controls the mechanical arm to move through the control instruction, and under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, first prompt information is generated to prompt puncture, so that the searching process of the puncture path can be simplified, and the time spent in the puncture process is reduced.

Drawings

FIG. 1 is a schematic diagram of a surgical navigation system provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a surgical navigation method based on an optical tracking device and a robotic arm according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an ultrasound scan using an ultrasound probe provided in an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of another electronic device according to an embodiment of the present application.

Detailed Description

The terminology used in the following embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates to the contrary. It should also be understood that the term "and/or" as used in this application refers to and encompasses any or all possible combinations of one or more of the listed items. The terms first and second and the like in the description, in the claims and in the drawings, are used for distinguishing between different objects and not for describing a particular sequential order.

It should be noted that the terms "first," "second," "third," and "fourth," etc. in the description, claims, and drawings of this application are used for distinguishing between different objects and not for describing a particular sequential order. It should also be understood that in the embodiments of the present application, the numbers before the steps are made for the convenience of understanding and describing the solution, and should not be construed as limiting the order in which the steps are performed.

With the continuous development of computer technology, medical technology has made a major breakthrough. For lesions inside the patient's body, it is often necessary to perform a puncture to obtain lesion tissue for diagnosis or treatment, etc. For example, in percutaneous nephrolithotripsy, it is necessary to puncture a stone as a target point, and then remove the stone after lithotripsy to achieve the therapeutic purpose; also for example, a needle biopsy may be required in the treatment or diagnosis of other diseases of the kidney, such as glomerulonephritis, nephrotic syndrome, etc.

At present, a puncture path in a puncture operation can be determined from a reconstructed three-dimensional model, for example, a target organ can be reconstructed before the puncture operation, then a needle insertion point and a target point are determined from the reconstructed target organ, and then the puncture path is planned. It will be appreciated that although the patient is in a gunpowder state during the puncturing operation, the respiration and heartbeat of the patient still cause changes (such as displacement) in the body of the organ tissue, and if the puncture path planned before the operation is used for puncturing, errors occur, so that the puncturing process is long, and the puncturing precision is low.

Based on the above problems, the embodiments of the present application provide an operation navigation method based on an optical tracking device and a mechanical arm and a related device, through which the duration spent in the puncturing process can be reduced and the puncturing precision can be improved.

The method provided in the embodiment of the present application may be performed by an electronic device in a surgical navigation system, where the electronic device may be any electronic device capable of performing the technical solution disclosed in the embodiment of the present application, and the electronic device may be a tablet computer, a palm computer, a notebook computer, or the like, or may be a terminal device, a server, or a server cluster formed by multiple servers, or the like, which is not limited herein. In the alternative, embodiments of the method may also be implemented by way of a processor executing computer program code.

For easy understanding, the surgical navigation system provided in the embodiments of the present application will be described first. Referring to fig. 1 for exemplary purposes, fig. 1 is a schematic diagram of a surgical navigation system according to an embodiment of the present application. As shown in fig. 1, the surgical navigation system includes an electronic device 101, an optical tracking device 102, an ultrasonic device 103, and a robotic arm 104.

In the present embodiment, the optical tracking device 102 may be understood as a device for measuring spatial position information (such as spatial coordinates) of a marker. Illustratively, the optical tracking device 102 may include a first sensor 1021 and a second sensor 1022. The first sensor 1021 may include a first infrared light emitting diode and a first infrared receiver, and the second sensor 1022 may include a second infrared light emitting diode and a second infrared receiver. When positioning the marker in the tracking range, the first infrared light emitting diode and the second infrared light emitting diode can generate infrared light and irradiate the marker, the reflective coating on the surface of the marker reflects the infrared light back to the first infrared receiver and the second infrared receiver, and the optical tracking device 102 positions the marker by using the infrared light.

In this embodiment, the mechanical arm 104 is fixedly connected with the puncture guide 105, the puncture guide 105 is fixedly connected with the puncture guide rail 106, and the puncture guide rail 106 is used for clamping the puncture needle 107. Illustratively, the robotic arm 104 may include a base 1041, a joint member 1042, and a flange 1043, and the robotic arm 104 may be fixedly coupled to the puncture guide 105 via the flange 1043.

The puncture guide 105 is provided with M markers, the puncture needle 107 is provided with N markers, M is an integer of 3 or more, and N is an integer of 1 or more. It will be appreciated that in three dimensions, 3 points may define a plane, at least one plane may be defined based on the M coordinates corresponding to the M markers, and the spatial location information of each location on the puncture guide 105 under the optical tracking device 102 may be determined in combination with the design of the puncture guide 105. In this embodiment, the puncture needle 107 is used to advance along the puncture guide rail 106 to puncture after the mechanical arm 104 is in place, so that the direction of the puncture needle 107 in the puncture process is determined (may be considered as the direction of the puncture guide rail 106 or the direction of the puncture path), and on this basis, only 1 coordinate is needed to determine the spatial position information of each position on the puncture needle 107 under the optical tracking device 102, so that N is a number greater than or equal to 1. It will also be appreciated that the smaller the number of markers on the needle 107, the lighter the overall weight of the needle 107, and the more convenient the operator can operate.

Illustratively, as shown in fig. 1, the puncture guide 105 may be fixedly connected to a tracer having 4 markers disposed thereon; the puncture needle 107 may be fixedly connected to a structure body on which 4 markers are provided by a locking structure. It will be appreciated that during a puncture procedure, the M markers and the N markers are within the tracking range of the optical tracking device 102, such that the optical tracking device 102 positions the puncture guide 105 by tracking the M markers and the puncture needle 107 by tracking the N markers.

In this embodiment, the ultrasound apparatus 103 may be understood as an apparatus that scans an object to be scanned with an ultrasound beam, then receives a reflected signal of the ultrasound beam, and processes the reflected signal to obtain an image of a target organ in the body of the object to be scanned. Illustratively, the ultrasound device 103 may include an ultrasound probe 1031, a first communication unit, a first processor, and a first power supply. Wherein the first power source may supply power to the ultrasound device 103, the first communication unit may be used for the ultrasound device 103 to communicate with the electronic device 101, and the first processor may be used for processing data acquired by the ultrasound probe 1031 to obtain an ultrasound image.

As shown in fig. 1, an operator such as a doctor may use the ultrasound probe 1031 to move over the body surface of the patient 108 to perform an ultrasound scan of a target organ. The ultrasonic probe 1031 is provided with K markers, where K is an integer greater than or equal to 3. It will be appreciated that in three dimensions, 3 points may define a plane, at least one plane may be defined based on the K coordinates corresponding to the K markers, and the spatial location information of each location on the ultrasound probe 1031 under the optical tracking device 102 may be determined in conjunction with the design of the ultrasound probe 1031. Illustratively, the ultrasonic probe 1031 is fixedly coupled to a structure having 4 markers disposed thereon, as shown in fig. 1, by a locking structure. It will be appreciated that during a puncture procedure, the K markers are within tracking range of the optical tracking device 102, such that the optical tracking device 102 locates the ultrasound probe 1031 by tracking the K markers.

In some embodiments, the electronic device 101 may include a second communication unit, a second processor, and a second power source. Wherein the second power source may power the electronic device 101, the second communication unit is used for the electronic device 101 to communicate with the mechanical arm 104, and/or for the electronic device 101 to communicate with the optical tracking device 102. For example, when the electronic device 101 communicates with the mechanical arm 104 through the second communication unit, a control instruction may be sent to the mechanical arm, where the control instruction is used to control the mechanical arm to move according to the M markers; also illustratively, when the electronic device 101 communicates with the optical tracking device 102 via the second communication unit, the electronic device may determine the position information of the puncture guide 105 by acquiring the coordinates of the M markers, may also acquire the position information of the N marker-determining puncture needles 107, may also acquire the position information of the K marker-determining ultrasonic device 103, and so on.

The second processor is used for determining a puncture path based on the position of the ultrasonic equipment under the condition that the ultrasonic equipment is used for scanning a target point; the control instruction is used for controlling the mechanical arm to move according to the M markers; and the second prompting information is also used for generating the second prompting information when the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, and the second prompting information is used for prompting: performing puncturing along the puncture guide rail under the condition that the puncture needle is mounted on the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm.

The robotic arm 104 may include a third communication unit, a third processor, and a third power source. The third power supply is configured to supply power to the mechanical arm 104, the third communication unit is configured to communicate with the electronic device 101 by using the mechanical arm 104, and the third processor is configured to execute a control instruction sent by the electronic device 101.

Referring to fig. 2 for an exemplary understanding, fig. 2 is a schematic diagram of a surgical navigation method based on an optical tracking device and a mechanical arm according to an embodiment of the present application, where the method may be applied to an electronic device in the surgical navigation system shown in fig. 1, as shown in fig. 2, and the method includes:

201: determining a puncture path based on the position of the ultrasonic probe under the condition that the ultrasonic probe is used for scanning to a target point; the target point is determined from the reconstructed target organ.

It will be appreciated that after the patient is ready, the operator may hold the ultrasound probe in his hand and move it over the patient's body surface to effect an ultrasound scan. In the embodiments of the present application, a target organ may be understood as an organ where a target spot is located, and the organ may be subjected to ultrasound scanning and computed tomography (computed tomography, CT). For example, where the target is a stone, the target organ may be a kidney, spleen, or the like.

For ease of understanding, referring to fig. 3, fig. 3 is a schematic diagram of an ultrasonic scanning using an ultrasonic probe according to an embodiment of the present application. As shown in fig. 3, in the case where the position of the ultrasonic probe 301 is fixed, the scanning range of the ultrasonic probe 301 can be understood as one scanning plane, such as the scanning plane 302 shown in fig. 3. Thus, in order to scan the target organ completely, it is necessary to slowly move the ultrasound probe 301 so that the scan surface emitted at different positions can cover the entire target organ as much as possible.

In this embodiment of the present application, before performing the puncture operation, a three-dimensional reconstruction may be performed on the target organ, for example, a three-dimensional reconstruction may be performed on a CT image of the target organ, and then a target is determined from the reconstructed target organ, where the target may be determined according to a focus, for example, may be a center of the focus, etc., which is not limited in this application. In some embodiments, the reconstruction of the target organ may be performed based on the CT image and the ultrasound image, so the above step 201, before determining the puncture path based on the position of the ultrasound probe, the method shown in fig. 1 further comprises:

(11) A CT contour of the target organ is acquired, the CT contour being derived from CT images acquired by a CT apparatus.

In this step, the CT contour of the target organ can be understood as a three-dimensional contour reconstructed using CT images. For example, a CT scan of the target organ may be performed to obtain a CT image of the target organ, and then a CT contour of the target organ may be segmented from the CT image using a segmentation algorithm.

(12) Determining that a first reference direction under an ultrasonic coordinate system is converted into a first direction vector under a world coordinate system, a second reference direction under the ultrasonic coordinate system is converted into a second direction vector under the world coordinate system, and a third reference direction under the ultrasonic coordinate system is converted into a third direction vector under the world coordinate system; the first reference direction, the second reference direction and the third reference direction are shared by a space corresponding to the ultrasonic equipment and a space corresponding to the CT outline; the ultrasonic coordinate system is a coordinate system corresponding to the ultrasonic equipment.

In this embodiment of the present application, the first reference direction, the second reference direction, and the third reference direction may be collectively referred to as a reference direction. The reference direction is shared in the space corresponding to the ultrasonic device and the space corresponding to the CT contour, which can be understood that the reference direction can determine the direction vector in the space corresponding to the ultrasonic device, or can determine the direction vector in the space corresponding to the CT contour.

For example, the reference direction may be determined from the patient. It will be appreciated that both ultrasound images and CT images are acquired based on the patient, and that the relative relationship between the target organ and the subject is determined even though the time, spatial information, etc. between the acquisition of the ultrasound image and the acquisition of the CT image are different. For example, when acquiring a CT image, the relative relationship between the spatial coordinate information of the CT image and the orientation of the patient is determined; when an ultrasound image is acquired, the relative relationship between the spatial coordinate information of the ultrasound image and the orientation of the patient is also determined. Thus, the reference direction may be a direction determined according to the implementation object, for example, a direction in which the foot of the patient points to the head (abbreviated as a head direction), a direction in which the back is vertically outward (abbreviated as a back vertical direction), a direction in which the body of the patient points to the left of the patient (abbreviated as a left direction), a direction in which the head of the patient points to the foot (abbreviated as a foot direction), a direction in which the body of the patient points to the right of the patient (abbreviated as a right direction), and the like.

The conversion from the ultrasonic coordinate system to the world coordinate system is performed based on the optical tracking device, in this embodiment of the present application, K optical markers are disposed on the ultrasonic device, and a conversion matrix between the K optical markers and the optical tracking device may be obtained by using coordinates of the K optical markers, so that after determining a direction vector of the reference direction in the ultrasonic coordinate system, a direction vector after converting the reference direction from the ultrasonic coordinate system to the world coordinate system may be obtained by using the above conversion matrix.

Taking the first reference direction as the head direction and the second reference direction vector as the back vertical direction as an example, when the ultrasonic equipment is used for scanning the kidney of the patient along the back vertebra direction of the patient, the X direction of the ultrasonic coordinate system is the same as the head direction, and the Y direction of the ultrasonic coordinate system is the same as the back vertical direction. Thus, when the conversion matrix is expressed by the expression (3), a direction vector of the head direction (first reference direction) in the world coordinate system can be obtained

Is->

Direction vector of back vertical direction (second reference direction)>

Is->

. Similarly, in the case where the third reference direction is the left direction, the left direction and the head and back maximum directions satisfy the right rule, and thus, the direction vector +.>

Is that

。

（3）

Alternatively, in the case that the first reference direction, the second reference direction, and the third reference direction are perpendicular to each other, the calculation amount is small, and the efficiency is high.

(13) Determining a fourth direction vector of a first reference direction under the CT coordinate system converted to a world coordinate system, a fifth direction vector of a second reference direction under the CT coordinate system converted to the world coordinate system, and a sixth direction vector of a third reference direction under the CT coordinate system converted to the world coordinate system, wherein the CT coordinate system is a coordinate system corresponding to the CT equipment.

It can be appreciated that the CT apparatus automatically converts to the world coordinate system after capturing the image. Because the patient is positioned specially when the CT image is acquired, for example, the head of the patient enters the CT equipment first, the X direction of the CT image obtained by CT scanning the patient is the left direction, the Y direction of the CT image is the back vertical direction, and the Z direction of the CT image is the head direction. Thereby, a direction vector of the head direction can be obtained

Is (0, 1). Similarly, the direction vector +.>

For (0, 1, 0), the left direction vector +.>

Is (1, 0).

It will be appreciated that the above description of the head direction, the back vertical direction, and the left direction are merely examples, and that other directions may be used in practice to determine the rotation matrix, which is not limited in this application.

(14) The rotational translation matrix is determined based on a first angle and a first position offset between the first and fourth direction vectors, a second angle and a second position offset between the second and fifth direction vectors, and a third angle and a third position offset between the third and sixth direction vectors.

As can be understood from the above steps (12) and (13), in the case where the posture of the patient is not changed, the head direction, the back vertical direction, and the left direction of the patient are fixed in the real world themselves. However, due to the different coordinate systems adopted by the equipment, after the ultrasonic contour and the CT contour are placed under the same world coordinate system, the vectors corresponding to the head direction, the back vertical direction and the left direction of the patient are different, namely the pose of the contour is different.

In the present embodiment, the rotation-translation matrix may be understood as a matrix including rotation components and translation components. In calculating the rotational-translational matrix, the rotational component may be determined first, and for example, one reference direction may be selected for correction, and then the remaining two reference directions may be corrected. For example, in the case where the first reference direction is the head direction, the second reference direction vector is the back vertical direction, and the third reference direction is the left direction, the head direction may be corrected based on the first included angle, and then the rotational component of the rotational-translational matrix may be calculated based on the second included angle and the third included angle.

In the embodiment of the application, the position offset between the direction vectors is used for representing the relative position relationship between the direction vectors. Taking the first reference direction as an example, the first direction vector and the fourth direction vector in the world coordinate system may be in different positions after different transformations, and the first position offset may be understood as a position offset between two points with the same absolute positions on the first direction vector and the fourth direction vector, such as a start point of the direction vector, or an end point of the direction vector, etc. Based on the first positional offset, a translation amount in the first reference direction may be determined, and a complete translation component may be determined in combination with the second positional offset and the third positional offset. It will be appreciated that the ultrasound contours and the CT contours can be identical in pose by transformation of the rotational components, and on the basis of this, the ultrasound contours and the CT contours can be identical in position by transformation of the translational components.

(15) And transforming the CT contour based on the rotation translation matrix to obtain a transformed CT contour.

(16) The method comprises the steps of obtaining an ultrasonic profile of a target organ, wherein the ultrasonic profile is obtained according to a plurality of ultrasonic images acquired by ultrasonic equipment, each ultrasonic image corresponds to different spatial position information, and the spatial position information is determined according to K markers.

In the embodiment of the application, the ultrasonic probe is provided with the K markers, and because the relative position relation between the K markers and the scanning surface of the ultrasonic probe is determined, when the ultrasonic probe is moved to different positions to carry out ultrasonic scanning, the optical tracking device can determine the spatial position information corresponding to different ultrasonic images by tracking the K markers. Based on the spatial position information corresponding to each ultrasonic image and the segmentation algorithm, the ultrasonic profile of the target organ can be reconstructed.

(17) Registering based on the transformed CT contour and the ultrasonic contour to obtain the reconstructed target organ.

It will be appreciated that the ultrasound images are acquired by the ultrasound device and the CT images are acquired by the CT device, and the registration of the two ultrasound contours and the CT contours is performed in the world coordinate system. The CT equipment automatically converts the acquired images into a world coordinate system, and the direction of the spatial position of the CT images converted into the world coordinate system can be arbitrary. The ultrasonic device acquires a two-dimensional ultrasonic image, and the two-dimensional ultrasonic image is converted into a world coordinate system by the identification of the optical tracking device, so that the three-dimensional space position of the converted ultrasonic image is related to the position of the optical tracking device. However, in practical situations, the location of the optical tracking device is generally random, for example, according to the environment of an operating room or according to the habit of a user, which results in that the ultrasound profile in the world coordinate system is also random, so that the ultrasound profile and the CT profile have a large difference in azimuth (such as position and angle), and thus the registration efficiency is low. In the embodiment, the CT contour is transformed through the reference direction, so that the pose difference between the CT contour and the ultrasonic contour is reduced, and the registration efficiency is improved.

The above steps (11) - (17) introduce transforming the target organ based on the reference direction, and registering the transformed CT contour with the ultrasound contour of the target organ to obtain the reconstructed target organ. It can be understood that the reconstructed CT contour includes detailed information of the target organ, the reconstructed ultrasound contour can reflect the real-time state of the target organ, and the target organ obtained by registering the CT contour and the ultrasound contour is more visual and accurate, for example, the internal tissue condition of the target organ can be obtained more accurately, and then the target point determined therefrom is also more accurate. By way of example, the electronic device may include a display screen that may be used to display the reconstructed target organ, and the electronic device may further include an input device, and an operator such as a doctor may select a target based on the target organ displayed on the display screen and display a spatial positional relationship of the target. The input device may be a keyboard, a mouse, a touch screen, etc., which is not limited in this application.

In the embodiment of the present application, the puncture path may be understood as a virtual path including a target point and a needle insertion point. After the target point is determined based on the reconstructed target organ, in the implementation of the application, a complete puncture path is determined based on the position of the ultrasonic probe. For example, a physician may use an ultrasound probe to slowly move over the surface of the patient's skin to scan a target organ, where the ultrasound probe includes a target site on its scan surface, the ultrasound probe may be considered to scan to the target site. At this time, the ultrasonic probe is fixed, the electronic device acquires the coordinates of the K markers recorded by the optical tracking device, the coordinates of the needle insertion point are determined from the ultrasonic probe based on the relative position relationship between the K markers and the ultrasonic probe, and the puncture path can be determined based on the coordinates of the needle insertion point and the coordinates of the target point. The needle insertion point can be selected in various ways only by being on the scanning surface of the ultrasonic probe, and it can be understood that the scanning surface of the ultrasonic probe can scan the target point, and then the needle insertion point determined from the scanning surface of the ultrasonic probe can also puncture the target point.

In some embodiments, step 201 above, where the ultrasound probe is used to scan to the target site, determining the puncture path based on the location of the ultrasound probe includes:

2011: under the condition that the ultrasonic probe is used for scanning to a target point, a reference point on the ultrasonic probe is determined to be a needle insertion point, and the reference point is positioned on a line segment where a scanning surface of the ultrasonic probe and the ultrasonic probe intersect.

2012: and generating a puncture path based on the target point and the needle insertion point.

In this embodiment, the reference point is located on a line segment where the scanning surface of the ultrasonic probe intersects the ultrasonic probe, and for convenience of understanding, the line segment where the scanning surface 302 intersects the ultrasonic probe 301 may be understood as a line segment AC, and the reference point may be any point on the line segment AC, for example, may be a point a, a point B, or a point C, as shown in fig. 3.

In this embodiment, the reference point on the line segment where the scanning surface of the ultrasonic probe intersects with the ultrasonic probe is used as the needle insertion point, the coordinate determining process of the needle insertion point is simple, and the needle insertion point is close to the skin of the patient, so that compared with the setting of the needle insertion point on other spatial positions, the method is more accurate and visual, and the marking is convenient.

202: and sending a control instruction to the mechanical arm, wherein the control instruction is used for controlling the mechanical arm to move according to the M markers.

203: under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, generating first prompt information, wherein the first prompt information is used for prompting: performing puncturing along the puncture guide rail under the condition that the puncture needle is mounted on the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm.

In this embodiment of the application, before the mechanical arm moves to the designated position, the puncture needle is not clamped on the puncture guide rail, and after the mechanical arm moves to the designated position, operators such as doctors install the puncture needle on the puncture guide rail. However, it will be understood that the mechanical arm and the puncture guide rail on the mechanical arm are fixedly connected, the structure of the puncture needle is also fixed, and the electronic device can obtain the simulated surgical needle according to the connection relation between the mechanical arm and the puncture needle and the physical structure of the puncture needle. In the embodiments of the present application, the puncture needle may be understood as a physical needle existing in real space, and the simulated surgical needle may be understood as a non-physical needle obtained by simulating the puncture needle.

The electronics can route planning based on the simulated surgical needle to facilitate the robotic arm to reach the specified location. It can be understood that the electronic device can calculate the needle point position (or can be understood as the needle point coordinate) of the virtual surgical needle, and can determine the needle body direction of the virtual surgical needle, and the electronic device controls the mechanical arm to move through the control instruction, so that the mechanical arm can be considered to reach the designated position under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path.

Illustratively, the manipulator is adjusted to a ready posture according to the position of the patient, a puncture guide is installed at the tail end of the manipulator, and M markers on the puncture guide are in the tracking range of the optical tracking device, which corresponds to the fact that the manipulator is initialized. Then, the electronic equipment calculates a proper motion path so that the puncture needle guide can accurately reach the planned puncture path, and the mechanical arm moves according to the planned motion path through a mechanical arm control command after the motion path is calculated, so that the puncture needle guide is finally in place.

For example, the first prompt information may be a text prompt information, and/or a sound prompt information, etc., where the text prompt information may be displayed on the display screen, and the sound prompt information may be output through the audio output device, which is not limited in this application.

In the embodiment of the application, under the condition that an ultrasonic probe is used for scanning a target point, a puncture path is determined based on the position of the ultrasonic probe; the target point is determined according to the reconstructed target organ; sending a control instruction to the mechanical arm, wherein the control instruction is used for controlling the mechanical arm to move according to M markers; under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, generating first prompt information, wherein the first prompt information is used for prompting: performing puncturing along the puncture guide rail under the condition that the puncture needle is mounted on the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm. In the embodiment, compared with a puncture path directly determined on a reconstruction model, the puncture path determined by the scanning process of the ultrasonic probe is more accurate, so that the puncture precision is higher; then the electronic equipment controls the mechanical arm to move through the control instruction, and under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, first prompt information is generated to prompt puncture, so that the searching process of the puncture path can be simplified, and the time spent in the puncture process is reduced. Moreover, an operator can complete puncturing by pushing the puncture needle along the puncture guide rail, the puncture guide rail can ensure that the needle is not bent in the puncturing process, the operation is simple, and the puncturing time is further saved.

In some embodiments, the method shown in fig. 1 further comprises:

(18) Determining a reference plane, wherein an included angle between the reference plane and the puncture path is larger than or equal to a first threshold value, and the reference plane and the puncture path intersect at a first intersection point;

(19) Displaying the reference plane and the first intersection point;

(20) Determining a first projection point of the first end of the puncture guide rail on a reference plane, and determining a second projection point of the second end of the puncture guide rail on the reference plane;

in the step 203, when the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, first prompt information is generated, including:

(21) And generating first prompt information under the condition that the needle point position of the virtual surgical needle coincides with the target point, and the first projection point, the second projection point and the first intersection point are positioned on the same straight line.

In this embodiment, the angle between the reference plane and the puncture path is understood as the angle between the reference plane and the straight line along which the puncture path is located. In other words, in the case where the puncture path has been determined, the reference plane may be any plane having an angle with respect to a straight line in which the puncture path is located that is greater than or equal to the first threshold value. The angle between the straight line and the plane ranges from 0 ° to 90 °, and the first threshold value may be any value of 85 ° or more and 90 ° or less, for example, and may be specifically set according to practical situations.

It will be appreciated that after the puncture path is determined, there may be a plurality of reference planes that meet the condition, and thus, reference planes at different positions may correspond to different first intersection points; alternatively, the first intersection point may be a target point, and the reference plane may be a plane perpendicular to the puncture path and including the target point. It will be appreciated that while the reference plane and the first intersection point may have different locations, the reference plane and the first intersection point are fixed during the lancing operation, such as the first intersection point being fixed as a target point, the reference plane may be fixed as a plane perpendicular to the lancing path and including the target point.

For example, the electronic device may comprise a display screen for displaying the reference plane and the first intersection point. Alternatively, when the reference plane is displayed, the reference plane may be displayed by displaying the auxiliary line. For example, a plurality of concentric circles can be drawn on the reference plane by taking the first intersection point as the center of a circle, and the spatial sense of the reference plane can be expressed through the display of the plurality of concentric circles.

After determining the reference plane, the electronic device may determine projection points of the two end points of the puncture guide rail on the reference plane, that is, the first projection point and the second projection point. Because the puncture guide rail is used for clamping the puncture needle, when the projection points at the two ends of the puncture guide rail and the first intersection point are positioned on the same straight line, the whole direction of the virtual surgical needle can be considered to coincide with the puncture path.

In some embodiments, the electronic device may further display auxiliary information, where the method further includes:

(22) Displaying the first proxel, the second proxel, any one or more of:

It can be understood that the relative positional relationship between the virtual surgical needle and the puncture path can be intuitively reflected by the display of the auxiliary information. If errors still remain between the mechanical arm and the actual puncture path after the mechanical arm is moved, operators such as doctors and the like can finely adjust the mechanical arm according to auxiliary information, the auxiliary information can enable an operator to feel more direction, and the puncture path can be aligned more rapidly.

In other embodiments, the above method further comprises:

(23) In the process of installing a puncture needle on a puncture guide rail and puncturing along the puncture guide rail, determining the needle point position of the needle point of the puncture needle under an optical coordinate system based on N markers in real time, wherein the optical coordinate system is a coordinate system corresponding to optical tracking equipment;

(24) Displaying a first distance and a second distance, wherein the first distance is the distance between the needle point position of the puncture needle and the needle inlet point, and the second distance is the distance between the needle point position of the puncture needle and the target point;

(25) And generating second prompt information under the condition that the needle point position of the puncture needle coincides with the target point, wherein the prompt information is used for prompting the puncture needle to reach the target point.

It can be understood that after the puncture needle is clamped on the puncture guide rail, the N markers arranged on the puncture needle are in the tracking range of the optical tracking device, and the optical tracking device can determine the needle point position of the needle point of the puncture needle under the optical coordinate system through the coordinates of the N markers and the relative position relationship between the N markers and the needle point of the puncture needle.

In this embodiment, the needle insertion depth of the puncture needle can be reflected in real time by displaying the first distance and the second distance. In the case where the tip position of the puncture needle coincides with the target point, the puncture of the puncture needle can be considered to be completed. The second prompt information may be a text prompt information, and/or a sound prompt information, etc. which may be displayed on the display screen, and the sound prompt information may be output through the audio output device, which is not limited in this application.

The method provided by the embodiment of the present application is described in detail above, and the device provided by the embodiment of the present application is described below.

Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, where the electronic device 40 shown in fig. 4 is applied to a surgical navigation system, and the surgical navigation system includes the electronic device 40, an optical tracking device, an ultrasonic device, and a mechanical arm; the mechanical arm is fixedly connected with the puncture guide, the puncture guide is fixedly connected with the puncture guide rail, and the puncture guide rail is used for clamping the puncture needle; m markers are arranged on the puncture guide, N markers are arranged on the puncture needle, K markers are arranged on an ultrasonic probe of the ultrasonic equipment, M and K are integers larger than or equal to 3, N is an integer larger than or equal to 1, and the optical tracking equipment positions the ultrasonic probe, the mechanical arm and the puncture needle through tracking the markers; the electronic device 40 comprises a determining unit 401, a communication unit 402 and a generating unit 403, optionally the electronic device 40 may further comprise an obtaining unit 404, a rotating unit 405, a registering unit 406 and a display unit 407, wherein the respective units are described as follows:

a determining unit 401, configured to determine a puncture path based on a position of the ultrasonic probe when the ultrasonic probe is used to scan a target point; the target point is determined according to the reconstructed target organ;

A communication unit 402, configured to send a control instruction to the mechanical arm, where the control instruction is configured to control the mechanical arm to move according to the M markers;

a generating unit 403, configured to generate, when the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, first prompt information, where the first prompt information is used to prompt: when the puncture needle is attached to the puncture guide rail, performing puncture along the puncture guide rail; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the positional relationship between the puncture needle and the mechanical arm.

In a possible implementation manner, the electronic device 40 further includes an acquiring unit 404, configured to acquire a CT contour of the target organ, where the CT contour is obtained according to a CT image acquired by the CT device;

a determining unit 401, configured to determine that a first reference direction in an ultrasound coordinate system is converted into a first direction vector in a world coordinate system, a second reference direction in the ultrasound coordinate system is converted into a second direction vector in the world coordinate system, and a third reference direction in the ultrasound coordinate system is converted into a third direction vector in the world coordinate system; the first reference direction, the second reference direction, and the third reference direction are common to a space corresponding to the ultrasound apparatus and a space corresponding to the CT contour; the ultrasonic coordinate system is a coordinate system corresponding to the ultrasonic equipment;

A determining unit 401, configured to determine a fourth direction vector of the first reference direction in the CT coordinate system converted to the world coordinate system, a fifth direction vector of the second reference direction in the CT coordinate system converted to the world coordinate system, and a sixth direction vector of the third reference direction in the CT coordinate system converted to the world coordinate system, where the CT coordinate system is a coordinate system corresponding to the CT apparatus;

a determining unit 401, configured to determine a rotational translation matrix based on a first angle and a first position offset between the first direction vector and the fourth direction vector, a second angle and a second position offset between the second direction vector and the fifth direction vector, and a third angle and a second position offset between the third direction vector and the sixth direction vector;

the electronic device 40 further includes a rotation unit 405, configured to transform the CT profile based on the rotation translation matrix, to obtain a transformed CT profile;

an obtaining unit 404, configured to obtain an ultrasound profile of the target organ, where the ultrasound profile is obtained according to a plurality of ultrasound images acquired by the ultrasound apparatus, and each of the ultrasound images corresponds to different spatial location information, and the spatial location information is determined according to the K markers;

The electronic device 40 further comprises a registration unit 406 for registering based on the transformed CT contour and the ultrasound contour, resulting in the reconstructed target organ.

In a possible implementation manner, the determining unit 401 is further configured to determine, when the ultrasonic device is used to scan the target point, that a reference point on the ultrasonic probe is a needle insertion point, where the reference point is located on a line segment where a scan surface of the ultrasonic probe intersects with the ultrasonic device;

the generating unit 403 is further configured to generate the puncture path based on the target point and the needle insertion point.

In a possible implementation manner, the determining unit 401 is further configured to determine a reference plane, where an angle between the reference plane and the puncture path is greater than or equal to a first threshold, and intersects the puncture path at a first intersection point;

the electronic device 40 further comprises a display unit 407 for displaying the reference plane and the first intersection point;

a determining unit 401, configured to determine a first projection point of the first end of the puncture guide on the reference plane, and determine a second projection point of the second end of the puncture guide on the reference plane;

The generating unit 403 is further configured to generate the first prompt message when the needle tip position of the virtual surgical needle coincides with the target point and the first projection point, the second projection point, and the first intersection point are located on the same straight line.

In a possible implementation manner, the display unit 407 is further configured to display the first projection point, the second projection point, and any one or more of the following:

In a possible implementation manner, the determining unit 401 is further configured to determine, in real time, a needle tip position of the needle tip of the puncture needle under an optical coordinate system, where the optical coordinate system is a coordinate system corresponding to the optical tracking device, based on the N markers during the process of installing the puncture needle on the puncture guide rail and performing puncture along the puncture guide rail;

a display unit 407, configured to display a first distance and a second distance, where the first distance is a distance between a needle tip position of the puncture needle and the needle insertion point, and the second distance is a distance between the needle tip position of the puncture needle and the target point;

The generating unit 403 is further configured to generate, when the needle tip position of the puncture needle coincides with the target point, a second prompting message, where the prompting message is used to prompt the puncture needle to reach the target point.

Referring to fig. 5, fig. 5 is a schematic structural diagram of another electronic device according to an embodiment of the present application. The electronic device 50 as shown in fig. 5 comprises a memory 501, a processor 502. Optionally, the electronic device 50 may also include a communication interface 503 and a bus 504; further optionally, the electronic device 50 may also include a display 505. The memory 501, the processor 502, the communication interface 503, and the display 505 are connected to each other by a bus 504.

The memory 501 is used to provide a storage space, and data such as an operating system and a computer program may be stored in the storage space. Memory 501 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or portable read-only memory (compact disc read-only memory, CD-ROM).

The processor 502 is a module for performing arithmetic operations and logical operations, and may be one or a combination of processing modules such as a central processing unit (central processing unit, CPU), a graphics card processor (graphics processing unit, GPU) or a microprocessor (microprocessor unit, MPU). In addition, the memory 501 stores a computer program, and the processor 502 may call the computer program stored in the memory 501 to execute a corresponding method.

The display 505 is used to implement the display function of the electronic device 50. Illustratively, the display 505 may be used to display the reference plane (and associated auxiliary lines), the first projection point, the second projection point, the puncture path, and the like.

In this embodiment, when the electronic device 50 shown in fig. 5 performs the above method, the processor 502 may control the display function of the display screen 505 and may also control the data communication function of the communication interface 503.

Illustratively, in some embodiments, the processor 502 is configured to determine the puncture path based on the location of the ultrasound device if the ultrasound device is used to scan to a target site;

the processor 502 is further configured to control sending a control instruction to the mechanical arm;

The processor 502 is further configured to generate first prompt information when the needle tip position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path.

In another embodiment, the processor 502 may be used to implement the functions of the determining unit 401, the generating unit 403, the obtaining unit 404, the rotating unit 405, and the registering unit 406 in the electronic device 40. The display 505 may be controlled by the processor 502 for implementing the functions of the display unit 407 in the electronic device 40. Alternatively, the data acquired by the acquiring unit 404 in the electronic device 40 may also be acquired through the communication interface 503.

The present application also provides a computer readable storage medium having computer code stored therein, which when run on a computer causes the computer to perform the method of the above-described embodiments.

The present application also provides a computer program product comprising computer code or a computer program which, when run on a computer, causes the method in the above embodiments to be performed.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. The surgical navigation method based on the optical tracking device and the mechanical arm is characterized in that the method is applied to electronic equipment in a surgical navigation system, and the surgical navigation system comprises the electronic equipment, the optical tracking device, the ultrasonic equipment and the mechanical arm; the mechanical arm is fixedly connected with the puncture guide, the puncture guide is fixedly connected with the puncture guide rail, and the puncture guide rail is used for clamping the puncture needle; m markers are arranged on the puncture guide, N markers are arranged on the puncture needle, K markers are arranged on an ultrasonic probe of the ultrasonic equipment, M and K are integers greater than or equal to 3, and N is an integer greater than or equal to 1; the optical tracking device positions the puncture guide by tracking the M markers, positions the ultrasonic probe by tracking the K markers, and positions the puncture needle by tracking the N markers; the method comprises the following steps:

acquiring a CT profile of a target organ by computer tomography, wherein the CT profile is obtained according to a CT image acquired by CT equipment;

determining that a first reference direction under an ultrasonic coordinate system is converted into a first direction vector under a world coordinate system, a second reference direction under the ultrasonic coordinate system is converted into a second direction vector under the world coordinate system, and a third reference direction under the ultrasonic coordinate system is converted into a third direction vector under the world coordinate system; the first reference direction, the second reference direction and the third reference direction are common to a space corresponding to the ultrasound device and a space corresponding to the CT profile; the ultrasonic coordinate system is a coordinate system corresponding to the ultrasonic equipment;

Determining a fourth direction vector of the first reference direction under a CT coordinate system converted to the world coordinate system, a fifth direction vector of the second reference direction under the CT coordinate system converted to the world coordinate system and a sixth direction vector of the third reference direction under the CT coordinate system converted to the world coordinate system, wherein the CT coordinate system is a coordinate system corresponding to the CT equipment;

determining a rotational translation matrix based on a first angle and a first position offset between the first direction vector and the fourth direction vector, a second angle and a second position offset between the second direction vector and the fifth direction vector, and a third angle and a third position offset between the third direction vector and the sixth direction vector;

acquiring an ultrasonic profile of the target organ, wherein the ultrasonic profile is obtained according to a plurality of ultrasonic images acquired by the ultrasonic equipment, each ultrasonic image corresponds to different spatial position information, and the spatial position information is determined according to the K markers;

Registering based on the transformed CT contour and the ultrasonic contour to obtain a reconstructed target organ;

determining a puncture path based on the position of the ultrasonic probe under the condition that the ultrasonic probe is used for scanning to a target point; the target point is determined according to the reconstructed target organ;

sending a control instruction to the mechanical arm, wherein the control instruction is used for controlling the mechanical arm to move according to the M markers;

under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, generating first prompt information, wherein the first prompt information is used for prompting: performing piercing along the piercing guide with the piercing needle mounted to the piercing guide; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the position relationship between the puncture needle and the mechanical arm.

2. The method of claim 1, wherein the determining a penetration path based on the location of the ultrasound probe if the ultrasound probe is used to scan to a target site comprises:

3. The method according to claim 2, wherein the method further comprises:

displaying the reference plane and the first intersection point;

determining a first projection point of a first end of the puncture guide on the reference plane, and determining a second projection point of a second end of the puncture guide on the reference plane;

under the condition that the needle point position of the virtual surgical needle coincides with the target point and the needle body of the virtual surgical needle coincides with the puncture path, generating first prompt information comprises:

and generating the first prompt information under the condition that the needle point position of the virtual surgical needle coincides with the target point and the first projection point, the second projection point and the first intersection point are positioned on the same straight line.

4. A method according to claim 3, characterized in that the method further comprises:

displaying the first proxel, the second proxel, any one or more of:

5. The method according to claim 3 or 4, characterized in that the method further comprises:

in the process of installing the puncture needle on the puncture guide rail and puncturing along the puncture guide rail, determining the needle point position of the needle point of the puncture needle under an optical coordinate system based on the N markers in real time, wherein the optical coordinate system is a coordinate system corresponding to the optical tracking equipment;

and generating second prompt information under the condition that the needle point position of the puncture needle coincides with the target point, wherein the second prompt information is used for prompting the puncture needle to reach the target point.

6. A surgical navigation system, wherein the system comprises an electronic device, an optical tracking device, an ultrasonic device and a mechanical arm; the mechanical arm is fixedly connected with the puncture guide, the puncture guide is fixedly connected with the puncture guide rail, and the puncture guide rail is used for clamping the puncture needle; m markers are arranged on the puncture guide, N markers are arranged on the puncture needle, K markers are arranged on an ultrasonic probe of the ultrasonic equipment, M and K are integers greater than or equal to 3, and N is an integer greater than or equal to 1;

the optical tracking device is used for positioning the puncture guide by tracking the M markers, positioning the ultrasonic probe by tracking the K markers and positioning the puncture needle by tracking the N markers;

the electronic equipment is used for acquiring a CT profile of a target organ through computer tomography, and the CT profile is obtained according to a CT image acquired by the CT equipment;

the electronic equipment is used for determining that a first reference direction under an ultrasonic coordinate system is converted into a first direction vector under a world coordinate system, a second reference direction under the ultrasonic coordinate system is converted into a second direction vector under the world coordinate system and a third reference direction under the ultrasonic coordinate system is converted into a third direction vector under the world coordinate system; the first reference direction, the second reference direction and the third reference direction are common to a space corresponding to the ultrasound device and a space corresponding to the CT profile; the ultrasonic coordinate system is a coordinate system corresponding to the ultrasonic equipment;

The electronic device is configured to determine that the first reference direction in a CT coordinate system is converted to a fourth direction vector in the world coordinate system, the second reference direction in the CT coordinate system is converted to a fifth direction vector in the world coordinate system, and the third reference direction in the CT coordinate system is converted to a sixth direction vector in the world coordinate system, where the CT coordinate system is a coordinate system corresponding to the CT device;

the electronic device is configured to determine a rotational translation matrix based on a first angle and a first positional offset between the first direction vector and the fourth direction vector, a second angle and a second positional offset between the second direction vector and the fifth direction vector, and a third angle and a third positional offset between the third direction vector and the sixth direction vector;

the electronic equipment is used for transforming the CT contour based on the rotation translation matrix to obtain a transformed CT contour;

the electronic equipment is used for acquiring an ultrasonic profile of the target organ, the ultrasonic profile is obtained according to a plurality of ultrasonic images acquired by the ultrasonic equipment, each ultrasonic image corresponds to different spatial position information, and the spatial position information is determined according to the K markers;

The electronic equipment is used for registering based on the transformed CT contour and the ultrasonic contour to obtain a reconstructed target organ;

the electronic equipment is used for determining a puncture path based on the position of the ultrasonic probe under the condition that the ultrasonic probe is used for scanning a target point; the target point is determined according to the reconstructed target organ;

the electronic equipment is further used for sending a control instruction to the mechanical arm, and the control instruction is used for controlling the mechanical arm to move according to the M markers;

the mechanical arm is used for moving according to the control instruction;

the electronic equipment is further used for generating first prompt information when the needle point position of the virtual surgical needle coincides with the target point and the needle body direction of the virtual surgical needle coincides with the puncture path, and the first prompt information is used for prompting: performing piercing along the piercing guide with the piercing needle mounted to the piercing guide; the virtual surgical needle is reconstructed based on the physical structure of the puncture needle and the position relationship between the puncture needle and the mechanical arm.

7. Surgical navigation device based on an optical tracking apparatus and a robotic arm, characterized by comprising means for performing the method according to any of claims 1-5.

8. An electronic device, comprising: a processor and a memory, wherein the memory has stored therein a computer program, the processor invoking the computer program stored in the memory for performing the method of any of claims 1-5.

9. A computer readable storage medium, in which a computer program is stored which, when run on one or more processors, causes the method of any one of claims 1-5 to be performed.