CN115500940A

CN115500940A - Positioning display method of surgical needle and related device

Info

Publication number: CN115500940A
Application number: CN202211001558.2A
Authority: CN
Inventors: 谢卫国; 张子奇; 严春凡; 张旭
Original assignee: Shenzhen Weide Precision Medical Technology Co ltd
Current assignee: Shenzhen Weide Precision Medical Technology Co ltd
Priority date: 2022-08-19
Filing date: 2022-08-19
Publication date: 2022-12-23

Abstract

The embodiment of the application provides a positioning display method and a related device of a surgical needle, wherein the method comprises the following steps: acquiring a puncture path, wherein the puncture path is a virtual path from a surgical needle to a target point; determining a reference plane, wherein an included angle between the reference plane and the puncture path is greater than or equal to a first threshold value, and the reference plane and the puncture path are intersected at a first intersection point; displaying the reference plane and the first intersection point; determining a first projection point of a first reference point on the reference plane, and determining a second projection point of a second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle; and displaying the first projection point and the second projection point. The time spent in the alignment process of the surgical needle to coincide with the puncture path can be reduced by the present application.

Description

Positioning display method of surgical needle and related device

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a positioning display method of a surgical needle and a related device.

Background

With the continuous development of computer science and technology, the medical technology also meets the major breakthrough. For example, before the percutaneous puncture operation, the doctor can determine the puncture path of the surgical needle according to the specific situation of the implemented object, and the puncture path can be understood as a virtual path. In the operation, the optical tracking device can track the operation needle in real time and display the operation needle on the display device, and a doctor can use the operation needle and the puncture path displayed on the display device as reference, so that the operation needle and the puncture path in the operation space are overlapped for effective puncture.

However, since the puncture path is only a virtual path displayed by the display device, it does not exist in a real space; further, since the actual space is three-dimensional, it is difficult for an operator such as a doctor to grasp the adjustment direction and distance of the surgical needle, and a large amount of time is required for the alignment process of overlapping the surgical needle with the puncture path.

Disclosure of Invention

The embodiment of the application provides a positioning display method and a related device of a surgical needle, and the time spent in an alignment process of coinciding the surgical needle with a puncture path can be reduced, and the puncture efficiency is improved.

In a first aspect, an embodiment of the present application provides a method for displaying a position of a surgical needle, including:

acquiring a puncture path, wherein the puncture path is a virtual path from a surgical needle to a target point;

determining a reference plane, wherein an included angle between the reference plane and the puncture path is greater than or equal to a first threshold value, and the reference plane and the puncture path intersect at a first intersection point;

displaying the reference plane and the first intersection point;

determining a first projection point of a first reference point on the reference plane, and determining a second projection point of a second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle;

and displaying the first projection point and the second projection point.

According to the method provided by the embodiment of the application, the virtual puncture path of the surgical needle is obtained, the reference plane with the included angle larger than or equal to the first threshold value is determined according to the puncture path, and the reference plane and the first intersection point between the reference plane and the puncture path are displayed; when the surgical needle is displayed, a first projection point of a first reference point on the reference plane is displayed, and a second projection point of a second reference point on the reference plane is displayed. Since the final target of the surgical needle is to be aligned with the puncture path, the operator can know the pose of the surgical needle, such as the inclination angle of the surgical needle, more clearly and intuitively by displaying the reference plane, the first intersection point and the projection point of the reference point on the reference plane, and if the distance between the first projection point and the second projection point is too far, the inclination angle of the surgical needle can be considered to be too large; the operator can know the position distance between the surgical needle and the puncture path more clearly and intuitively, for example, the surgical needle is at the right side of the puncture path. Therefore, the operator can feel directional in the process of moving the surgical needle by the mode, the operator can consider that the surgical needle is approximately aligned with the puncture path by using the first intersection point as a target point and enabling the first projection point and the second projection point to coincide with the first intersection point as far as possible, and the alignment of the puncture path can be completed by finely adjusting the position of the surgical needle subsequently, so that the operation difficulty of the user is greatly reduced, and the time spent in the alignment process of the surgical needle and the puncture path in coincidence is further reduced.

In particular, when the puncture path is perpendicular to the reference plane, the surgical needle is moved such that the first projected point, the second projected point, and the first intersection point coincide with each other, and it is considered that the surgical needle is aligned with the puncture path.

With reference to the first aspect, in a possible implementation manner, before determining a first projection point of a first reference point on the reference plane and determining a second projection point of a second reference point on the reference plane, the method further includes:

determining a projection direction vector of a reference direction vector on the reference plane, wherein the reference direction vector is a fixed direction determined according to the real space of the surgical needle;

and displaying the first auxiliary line corresponding to the projection direction vector.

With reference to the first aspect, in one possible implementation manner, the determining a reference plane, displaying the reference plane and the first intersection, and displaying a first auxiliary line corresponding to the projection direction vector includes:

taking a coordinate plane of a space coordinate system where the puncture path is located as a first plane, wherein the first plane comprises a first candidate point and a first candidate line;

determining an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is used for rotating the first plane to an included angle with the puncture path which is larger than or equal to the first threshold value, and the position of the first candidate point on the puncture path; the rotation angle is an included angle between a direction vector of the direction of the first candidate line and the projection direction vector;

rotating the first plane based on the rotation angle on the coordinate plane to obtain a second plane;

transforming the second plane based on the orientation transformation matrix to obtain the reference plane, the first intersection point, and the first auxiliary line; the first auxiliary line is obtained by rotating and translating based on the first candidate line; the first intersection point is obtained based on the first candidate point in a translation mode;

and displaying the reference plane, the first intersection, and the first auxiliary line.

With reference to the first aspect, in a possible implementation manner, the surgical needle is clamped by a clamp, at least 3 markers are fixedly arranged on the clamp, and spatial coordinates of the first reference point and the second reference point are determined by acquiring coordinates of the at least 3 markers through an optical tracking device; the reference direction vector may include any one or more of:

a direction vector of the target point to the optical tracking device, a direction vector of the target point to a foot direction of the execution object, and a direction vector of the target point to a head direction of the execution object.

With reference to the first aspect, in a possible implementation manner, the first intersection point is the target point.

With reference to the first aspect, in a possible implementation manner, the method further includes:

and displaying second auxiliary lines, wherein the second auxiliary lines comprise at least one concentric auxiliary line taking the first intersection point as the center of a circle and a plurality of angle auxiliary lines on the concentric auxiliary line.

With reference to the first aspect, in a possible implementation manner, the first reference point is a needle tail of the surgical needle, and the second reference point is a needle tip of the surgical needle, or a rotation point for controlling the surgical needle.

In a second aspect, an embodiment of the present application provides a positioning display device for a surgical needle, including:

the acquiring unit is used for acquiring a puncture path, wherein the puncture path is a virtual path for the surgical needle to puncture to a target point;

a determination unit, configured to determine a reference plane, where an included angle between the reference plane and the puncture path is greater than or equal to a first threshold, and the reference plane intersects the puncture path at a first intersection point;

a display unit for displaying the reference plane and the first intersection point;

the determining unit is further configured to determine a first projection point of a first reference point on the reference plane, and determine a second projection point of a second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle;

the display unit is further configured to display the first projection point and the second projection point.

In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a display screen, where the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions, so that the method in the first aspect or any one of the possible implementation manners of the first aspect is performed.

In a fourth aspect, an embodiment of the present application provides a chip, including a logic circuit and an interface, where the logic circuit is coupled to the interface; the interface is for inputting and/or outputting code instructions, and the logic circuit is for executing the code instructions to cause the method of the first aspect or any possible implementation manner of the first aspect to be performed.

In a fifth aspect, an embodiment of the present application discloses a computer program product, which includes program instructions that, when executed by a processor, cause the method in the first aspect or any possible implementation manner of the first aspect to be performed.

In a sixth aspect, embodiments of the present application provide a computer-readable storage medium, where a computer program is stored, and when the computer program runs on a processor, the computer program causes the method in the first aspect or any possible implementation manner of the first aspect to be performed. Illustratively, the computer program product may be a software installation package.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments or the background art of the present application, the drawings used in the embodiments or the background art of the present application will be briefly described below.

FIG. 1 is a schematic view of a robotic arm based surgical scene provided in an embodiment of the present application;

FIG. 2 is a diagrammatic view of a surgical needle provided in accordance with an embodiment of the present application;

FIG. 3 is a schematic view of a sight provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a scene with a sight orientation determined according to a field of view orientation of an optical tracking device according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a relationship between a vector and a 90 ° line and a 0 ° line provided in an embodiment of the present application;

FIG. 6a is a schematic view of a scenario of finding a puncture path according to a sight according to an embodiment of the present application;

FIG. 6b is a schematic diagram of determining coordinates of a reference point on a surgical needle provided by an embodiment of the present application;

FIG. 7 is a schematic diagram illustrating a result of finding a puncture path according to an aiming device provided by an embodiment of the application;

fig. 8 is a schematic flowchart of a method for displaying the positioning of a surgical needle according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of another electronic device provided in an embodiment of the present application.

Detailed Description

The terminology used in the following embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the specification of the present application and the appended claims, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the listed items. The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.

At present, with the continuous development of science and technology, the application of mechanical arms in the operation is more and more extensive. In the present embodiment, the robotic arm may be understood as a device for securing a surgical needle in space. For the convenience of understanding, a doctor is described as an example of percutaneous renal surgery on a patient, please refer to fig. 1, and fig. 1 is a schematic view of a surgical scene based on a mechanical arm according to an embodiment of the present application.

The surgical scene shown in fig. 1 includes a patient 101, a robotic arm 103, a surgical needle 104, an optical tracking device 105, and a display device 106. Wherein there is a communicative connection between the display device 106 and the optical tracking device 105, the surgical needle 104 is held by a holder, the holder including a marker thereon, the marker being within a tracking range of the optical tracking device 105.

As shown in fig. 1, the mechanical arm 103 and the surgical needle 104 are connected together through a physical structure, and the connection between the mechanical arm 103 and the surgical needle 104 may be a fixed connection, such as welding, for example; or may be an active connection.

Attached to the surgical needle 104 is a marker, the marker surface layer including a light reflective coating. In the embodiment of the present application, the marker is located in the tracking range of the optical tracking device, and the optical tracking device 105 can determine the spatial position information of the marker through the reflective coating on the surface of the marker, thereby realizing real-time tracking of the marker. It is understood that, because the structure of the surgical needle 104 is fixed, the relative relationship between other positions in the surgical needle 104 and the marker is fixed, and therefore, the spatial position information of other positions on the surgical needle, such as the spatial position information of the needle tip of the surgical needle 104, can be obtained based on the spatial position information of the marker and the relative position relationship with the marker.

To facilitate understanding of the specific structure of the surgical needle 104, reference may be made to fig. 2 by way of example, and fig. 2 is a schematic view of a surgical needle provided by an embodiment of the present application. As shown in fig. 2, the surgical needle includes a needle body 200 and a needle handle 201, and a holder 202 is connected with the needle handle, specifically, a threaded connection capable of adjusting tightness is provided. Gripper 202 illustratively includes 4 markers thereon, namely marker 203, marker 204, marker 205, and marker 206. It is understood that the number of the labeling substance may be adjusted in practice as long as the number of the labeling substance is 3 or more.

In the embodiment of the present application, the surgical needle 104 is connected to the mechanical arm 103, and the surgeon can move the position of the surgical needle 104 by adjusting the mechanical arm 103. Also, the optical tracking device 105 may track the surgical needle 104 in real time and then display the position of the surgical needle 104 in real space in real time through the display device 106.

During operation, the patient 101 lies on the operating table, and the doctor can plan the puncture path 102 of the surgical needle 104 in advance according to the lesion position of the patient 101. It will be appreciated that in real space the surgical needle 104 is positioned by the optical tracking device 105, i.e. the spatial position of the surgical needle 104 is based on the world coordinate system corresponding to the optical tracking device 105. Therefore, the puncture path 102 planned by the doctor should be a puncture path in the world coordinate system, so that the condition that the surgical needle can "overlap" the puncture path 102 is satisfied.

Illustratively, a patient's kidney may be scanned by an ultrasound device that acquires ultrasound images, whose ultrasound probe may include markers for localization. Similar to the surgical needle 104, since the position of the marker on the ultrasonic probe is fixed, the spatial position information of the lesion may be determined by the relative positional relationship between the scanning range of the ultrasonic apparatus and the ultrasonic probe and the spatial position relationship of the marker, and then the puncture path may be determined based on the spatial position information of the lesion.

In this way, the spatial position of the puncture path and the spatial position of the surgical needle are both obtained based on the tracking by the optical tracking device 105, and therefore, spatial unification of the puncture path 102 and the surgical needle 104 is achieved.

In the embodiment of the present application, the puncture path 102 is not a path existing in the real space, but should be understood as a target path of the real surgical needle 104. As shown in fig. 1, the puncture path 102 includes a target point 1021 and a needle access point 1022, where the target point 1021 may be understood as a lesion or other target location. During the operation, the surgeon adjusts the direction of the real surgical needle 104 to be parallel to the direction of the puncture path 102 as much as possible through the mechanical arm 103, points the surgical needle 104 to the target point 1021 as much as possible, and then slowly advances the surgical needle 104 to puncture.

It will be appreciated that the puncture path 102 may be determined in combination with various information, such as the size, location, distance from the skin, and relationship to surrounding organs of the target point 1021. It will be appreciated that after the puncture path 102 is determined, the location of the needle access point 1022 at which the surgical needle 104 punctures from the skin of the patient 101 can be obtained.

It should also be appreciated that in the event that the body of the patient 101 does not change position, the puncture path displayed by the display device 106 will not change position. However, during the real space movement of the surgical needle 104 to align with the puncture path, the position of the surgical needle displayed by the display device 106 will change constantly.

At present, in the operation process, an operator such as a doctor can move the surgical needle in the real space according to the relative position relationship between the puncture path and the surgical needle displayed on the display device in real time so as to puncture the surgical needle by aiming at the planned puncture path. However, since the puncture path is only a virtual path displayed by the display device, it does not exist in a real space; and the actual space is three-dimensional, it is difficult for the operator to grasp the adjustment direction and distance of the surgical needle, resulting in a large amount of time spent for the alignment process of coinciding the surgical needle with the puncture path.

Based on the above problems, the embodiments of the present application provide a surgical needle positioning method and a related device. It can be understood that the surgical needle positioning method provided by the embodiment of the present application may be performed by a surgical needle positioning apparatus, which may be any apparatus capable of implementing the method provided by the present application, and for example, the surgical needle positioning apparatus may be an electronic device such as a laptop or a desktop computer. It should be understood that the method embodiments of the present application may also be implemented by means of a processor executing computer program code.

In order to visually understand the method provided by the present application, the method provided by the present application will be described below by taking a doctor as an operator and performing percutaneous renal surgery on a patient as an example. Illustratively, the method provided herein comprises the steps of:

1. CT image acquisition preparation

The doctor judges whether the enhanced CT image needs to be acquired according to the specific condition of the patient. If the doctor judges that the enhanced CT image does not need to be acquired, the patient lies on a CT scanning bed and then pushes the CT scanning bed into a CT scanning window to acquire a common CT image. If the doctor judges that the patient needs to acquire the enhanced CT image, the contrast agent is injected into the vein of the patient and then the patient is pushed into a CT scanning window to acquire the enhanced CT image.

It will be appreciated that contrast agents can be visualized through the blood circulation system at locations with abundant vascular supply, and that visualization of these locations can be visually understood as enhancement, and CT images obtained after enhancement and CT scans can yield more information about the kidney, such as kidney stones and occupancy, kidney cysts, etc. It will be appreciated that subsequent registration of the enhanced CT image to the ultrasound image may also allow the physician to obtain more renal information than a normal CT image.

2. CT data acquisition

And (3) pushing the patient into a CT scanning window, and acquiring CT data according to the judgment result in the step (1), wherein the CT data comprises a CT image. After the scanning is finished, taking the CT image data of the kidney of the patient.

It will be appreciated that if an enhanced CT scan is performed on a patient, including a sweep, arterial, venous, and delayed phase scan, generally, the follow-up steps can be performed using the sweep data.

3. CT data processing

The CT image acquired in step 2 is input into the surgical needle positioning device (which may also be referred to as a renal surgical navigation system) provided in the present application. In the renal surgery navigation system, a three-dimensional renal contour is segmented and reconstructed from the CT data acquired in the step 2 by an artificial intelligence algorithm, and is simply referred to as a CT-renal contour for convenience of understanding and description.

Illustratively, the artificial intelligence algorithm may be a deep learning algorithm, a Stochastic Gradient Descent (SGD), a back propagation (backward), a forward propagation (forward), and the like. It is understood that the CT-renal contours described above may include overall renal information, such as the contour of the kidney, the internal substance of the kidney, etc.

4. Ultrasound image acquisition

In the art, let the patient lie prone on the operation table, use ultrasonic equipment to scan the observation along the direction of patient back vertebra to combine the real-time spatial position of the ultrasonic image that optical tracking equipment gathered of location ultrasonic equipment. It can be understood that the breathing frequency of the patient can be reminded to keep slow and stable as much as possible in the ultrasound image acquisition process, and the spatial position fluctuation error caused by breathing is reduced.

5. Ultrasound image processing

In the renal surgery navigation system, a three-dimensional renal contour is segmented and reconstructed according to the plurality of ultrasound images acquired in the step 4 and the corresponding spatial position information through an artificial intelligence algorithm, and the three-dimensional renal contour is simply referred to as an ultrasound-renal contour for convenience of understanding and description.

6. Registration

In this step, the renal surgery navigation system registers the CT-renal contour and the ultrasound-renal contour.

For registration between CT-renal contour and ultrasound-renal contour, an Iterative Closest Point (ICP) algorithm may be exemplarily implemented. Wherein the ICP algorithm can transform sets of points in different coordinate systems into a common coordinate system by minimizing registration errors.

Exemplarily, the registration process may be divided into two stages, namely a coarse registration stage and a fine registration stage, where the coarse registration stage may be understood as a coarser registration stage performed under the condition that the transformation between the source point cloud and the target point cloud is almost unknown, and the purpose of the coarse registration stage is mainly to provide a better initial value of a transformation matrix for the fine registration; fine registration may be understood as further optimization resulting in a more accurate transformation matrix, i.e. the final registration matrix, given an initial transformation matrix.

7. Determining a puncture path

Illustratively, the kidney can be reconstructed three-dimensionally based on CT images of the kidney, and then the doctor can determine the target point and the needle insertion point according to the condition of the patient, for example, the position of the lesion in the patient, the size of the lesion, the distance between the lesion and the skin, and the like. In the present embodiment, the puncture path may be understood as a straight line or a line segment including a target point and a needle insertion point. Typically, the puncture path is a line segment, and one of the endpoints is the target point.

8. Positioning a surgical needle based on an aiming plane

8.1 drawing sighting device

In the embodiment of the present application, the scope may be understood as an auxiliary line (such as a line segment or a curve) for assisting the positioning of the surgical needle, and the scope may be displayed by the display device. In the embodiment of the application, the sighting device can be considered as a virtual sighting device and is not understood as a device in a real space.

It will be appreciated that the scope of the present application will ultimately need to be spatially perpendicular to the puncture path, and that the center of the scope will need to coincide with the target point of the puncture path. However, it is difficult to determine the sight from only one plane and one point in space. Therefore, the collimator can be drawn on the XOY plane first, and then the collimator can be subjected to orientation conversion based on the spatial position information of the puncture path.

In the present embodiment, the aiming surface may be understood as a plane perpendicular to the puncture path. It will be appreciated that there are multiple planes in space that are perpendicular to the path of penetration, wherein the target plane that includes the target point of the path of penetration is understood to be the target point targeting plane. In the embodiment of the application, according to the relationship between the point and the surface, the target point aiming surface includes the target point of the puncture path, and it can be understood that the spatial coordinate corresponding to the target point is on the target point aiming surface.

8.1.1 drawing the sighting device on XOY plane

For ease of understanding, referring to fig. 3, fig. 3 is a schematic view of a sight according to an embodiment of the present disclosure.

Illustratively, as shown in fig. 3, 4 concentric circles can be drawn with the origin of the world coordinate system (0,0,0) as the center and the radii r of 10, 20, 30, and 40, respectively. The solid black circle shown in fig. 3 may be understood as the center of the sight, or may also be referred to as the center of the sight.

Then, long scale lines are drawn. A long reference line is drawn with the origin (0,0,0) as one end of the line segment and (rcos θ, rsin θ, 0) as the other end of the line segment. Wherein. R may be 38, θ may be-30 °,0 °, 30 °, 60 °, 90 °, 120 °, 150 °, 180 °, and 210 °, respectively, as shown in fig. 3.

And then drawing short scale lines. With (r) ₁ cosθ，r ₁ sin θ, 0) is an end point of the line segment, and is represented by (r) ₂ cosθ，r ₂ sin θ, 0) draw a short tick mark for the other end of the line segment. Wherein r is as defined above ₁ Is 38, above r ₂ 42, theta is-30 DEG, -20 DEG, -10 DEG, 0 DEG, 10 DEG, 20 DEG, 30 DEG, 40 DEG, 50 DEG, 60 DEG, 70 DEG, 80 DEG, 90 DEG, 100 DEG, 110 DEG, 120 DEG, 130 DEG, 140 DEG, 150 DEG, 160 DEG, 170 DEG, 180 DEG, 190 DEG, 200 DEG, and 210 deg, respectively.

Finally, angle information can be added beside the long scale lines, and the final effect is shown in fig. 3. For ease of understanding, the sight located in the XOY plane may be referred to simply as the initial sight.

8.1.2 changing the sight to the sighting surface

First, assume three-dimensional space coordinates T (a, b, c) of a target point in a puncture path, and three-dimensional space coordinates of a needle insertion point are E (m, n, T). Therefore, a normal vector of the aiming plane perpendicular to the puncture path can be obtained

Is (m-a, n-b, t-c).

Secondly, as can be seen from step 8.1.1, the sight is drawn in the XOY plane, so that the normal vector of the original sight

Then, according to the normal vector of the aiming surface

And the normal vector of the initial sight

The initial sight is rotated to the targeting surface. The implementation can be realized in the following way exemplarily:

normal vector of the above-mentioned aiming plane

Projection vector projected on XOY plane

Is (m-a, n-b, 0), the projection vector is obtained

And the X-axis vector n _x (1,0,0) angle θ _x Theta of _x Can be obtained by the formula (1):

according to the normal vector of the aiming plane

Finding a vector

And the Z axis vector

Angle of (theta) _z Theta of _z Can be obtained by the formula (2):

since the angle range is (0, π), if the collimation surface needs to be rotated to the plane corresponding to the collimator (i.e., the XOY plane), the first rotation direction is related to the normal vector of the collimation surface

Is related to the y coordinate of (a). Wherein, in the case that the y coordinate n-b is greater than 0, the normal vector

Clockwise rotating the angle theta around the Z axis by taking (0,0,0) as a rotating point _x (ii) a Normal vector in case the y-coordinate n-b is smaller than 0

Counterclockwise rotating angle theta around the Z axis by taking (0,0,0) as a rotating point _x 。

In the normal vector

Rotate by theta clockwise or counterclockwise about (0,0,0) _x Based on (0,0,0), the rotation point is taken as the rotation point to rotate the angle theta around the Y axis clockwise _z At this time, the normal vector

Is parallel to the Z-axis.

Conversely, if it is necessary to rotate the scope to the targeting surface, first, the scope is rotated counterclockwise by an angle θ about the Y-axis with (0,0,0) as the rotation point _z . Then, in the case where the y-coordinate n-b is greater than 0, the normal vector is added

Counterclockwise rotating angle theta around the Z axis by taking (0,0,0) as a rotating point _x In the case where the y-coordinate n-b is less than 0, the normal vector is added

Clockwise rotating the angle theta around the Z axis by taking (0,0,0) as a rotating point _x . After the transformation, the original scope, which was originally located in the XOY plane, is already located at the aiming plane perpendicular to the puncture path.

8.1.3 moving the center of the sight onto the puncture path

In this step, when the puncture path has been determined, the coordinates of any point on the puncture path may be used as the center of the scope. The coordinates of the needle insertion point may be, for example, the coordinates of the half-way point between the needle insertion point and the target point, and the like.

In some embodiments, the coordinates of the target point of the puncture path may be taken as the center of the sight, in which case the sight may be considered to be located at the target point targeting surface. Exemplarily, since the target point coordinate T is (a, b, c), the center of the collimator (0,0,0) can be translated by the distances a, b, and c along the X, Y, Z axis, respectively, based on step 8.1.2, so that the collimator center and the target point coincide.

8.2 determining the Direction of the sighting device from the Direction of View of the optical tracking device

For ease of understanding, referring to fig. 4 by way of example, fig. 4 is a schematic view of a scene with a sight oriented according to a field of view of an optical tracking device according to an embodiment of the present application.

As shown in fig. 4, the needle insertion point on puncture path 401 is point E, which may also be referred to as needle insertion point E; the target point on puncture path 401 is point T, which may also be referred to as target point T. The scope 402 has been moved to the target targeting surface via step 8.1 above, with the center of the scope being the target point T.

As shown in fig. 4, the visual field direction of the optical tracking device 403 is the positive direction of the Z axis, and therefore, the direction vector of the visual field direction of the optical tracking device 403 is (0,0,1). Selecting a unit direction vector in the direction of the sight pointing into the optical tracking device 403

As an intermediate vector, the vector is,

is (0,0, -1).

Then, calculating a vector according to a point-to-space plane formula

Projection vector on target point collimation plane

Finally, the vector is obtained

Is (x) ₀ ，y ₀ ，z ₀ ) Wherein x is ₀ ，y ₀ ，z ₀ Can be represented by formula (3):

calculating the 90-degree direction line, the 0-degree direction line and the vector of the sighting device according to a vector included angle formula

The included angles therebetween are respectively included angles theta ₉₀ Angle theta ₀ 。

Finally, vector quantity

The relationship between the vector and the 90 ° direction line and the 0 ° direction line can be represented by fig. 5, and fig. 5 is a vector provided by the embodiment of the present application

And a 90 DEG line and a 0 DEG line.

As shown in FIGS. 5 (a), (b), (c) and (d), the following are shown

When the angle between the direction line and 0 degree is less than 90 degrees, the sighting device rotates clockwise by theta by taking the puncture path as a rotating shaft ₉₀ Then the 90-degree direction line of the sighting device can be aligned with

Are in the same direction. In that

The included angle between the direction line and the 0 degree direction line is more thanIn the case of 90 °, the scope is rotated counterclockwise by θ with the puncture path as the rotation axis ₉₀ Then the 90-degree direction line of the sighting device can be aligned with

Are in the same direction.

It will be appreciated that the sighting device is rotated to align the 90 deg. direction line with

When the directions of (a) and (b) are the same, the puncture path is rotated as a rotation axis, and the amount of calculation is large in this manner.

Thus, in some embodiments, the angle θ is obtained ₉₀ Then, the initial collimator in the XOY plane is rotated as follows: rotate theta around the Z axis with the origin (0,0,0) as the rotation point ₉₀ Pointing the 90 ° direction of the sight at the optical tracking device; the scope is then rotated to the targeting surface in accordance with step 8.1 above.

It will be appreciated that instead of aligning the 90 deg. line of sight with the optical tracking device, other lines of sight may be aligned with the optical tracking device in a similar manner to that described above, such as a 30 deg. line or a 180 deg. line, etc.

8.3, determining the real-time position of the surgical needle on the sighting device

It will be appreciated that, through steps 8.1 and 8.2 above, the aimer can be determined to the aiming surface, and the center of the aiming surface is a point on the puncture path, such as a needle insertion point, a target point, etc. It is also possible to determine the direction of the sighting surface directly facing the optical tracking device, for example as described above

The sight may be rotated to point in a direction towards the optical tracking device.

In this step, the center of the sighting device is taken as a target point, and the 90-degree direction line of the sighting device points to the optical tracking device. For example, referring to fig. 6a, fig. 6a is a schematic view of a scenario for finding a puncture path according to a sight according to an embodiment of the present application.

As shown in fig. 6a, the 90 direction line of sight 605 is directed to optical tracking device 603, with the center of sight 605 being the target point T on the puncture path. It is to be understood that the sight 605 as shown in fig. 6a does not exist in real space, and fig. 6a depicts the sight for ease of understanding.

Similar to the description relating to fig. 1, the surgical needle 602 is connected to a robotic arm 601, and the marker on the surgical needle 602 is within the tracking range of an optical tracking device 603. The optical tracking device 603 includes a communication link with a display device 604, the display device 604 can display a virtual puncture path and a surgical needle tracked by the optical tracking device 603, and in this embodiment, the display device 604 can also display a sighting device 605.

It is understood that the display device 604 may display the sight 605 at any angle with respect to the sight 605 when displaying the sight 605. For ease of review by the physician, the sight 605 may be displayed in a field of view of the puncture path, with the resulting display shown in FIG. 6 a.

During the operation, the surgeon adjusts the position of the surgical needle 602 by the robot arm 601, and thus the position of the surgical needle 602 is changed in real time. The needle body of the surgical needle 602 can be abstracted into a line segment, and according to the principle that two points determine a line, the three-dimensional space coordinates of two points (referred to as reference points for short) on the surgical needle 602 can be tracked in real time by the optical tracking device 603 to determine the pose of the needle body, for example, the needle tail point and the needle tip point of the surgical needle 602.

However, in the case where the needle end point and the needle tip point of the surgical needle 602 are used as reference points, the needle tip and the needle end point move simultaneously during the fine adjustment, resulting in a large deviation of the fine adjustment. Therefore, to reduce the fine tuning offset, a fixed structure on the robotic arm 601 may be used as a reference point, such as a fine tuning rotation point. In this embodiment, the fine-tuning rotation point is used for supporting or fixing the whole surgical needle, and may be disposed at one-half or one-third of the needle body of the surgical needle.

For example, the fixed surgical needle can be loosened by rotating the fine adjustment rotation point, and after the angle of the surgical needle is adjusted, the fine adjustment rotation point can be rotated and tightened to fix the surgical needle. In this embodiment, the coordinate corresponding to the fine tuning rotation point may be a coordinate of a rotation center point of the structure, and it can be understood that when the fine tuning rotation point is rotated and unscrewed to adjust the angle of the surgical needle, another reference point (such as a needle tip or a needle tail of the surgical needle) on the surgical needle may move, and the fine tuning rotation point may not move (because the rotation center may not move), which may reduce the operation difficulty of an operator aligning the surgical needle to the puncture path.

The fine tuning rotation point and the needle tail point are taken as reference points for example. Exemplarily, in the case that the optical tracking device does not track the surgical needle through the marker, it is assumed that the coordinate system corresponding to the surgical needle is a coordinate system as shown in fig. 6b, wherein the needle tip of the surgical needle is located at the origin of the coordinate system, the initial coordinate of the fine tuning rotation point for controlling the needle body is (L, 0,0), and the initial coordinate of the needle tail point is (K, 0,0). The optical tracking matrix of the optical tracking device 603 is a matrix T, which can be represented by equation (4):

under the condition that the optical tracking device tracks the surgical needle through the marker, the real-time coordinate R of the fine adjustment rotating point and the real-time coordinate E of the needle tail point can be obtained through calculation according to the initial coordinate of the fine adjustment rotating point, the initial coordinate of the needle tail point and the matrix T, wherein the coordinate R is (R is _x1 *L+T _x ，R _y1 *L+T _y ，R _z1 *L+T _z ) The coordinate E is (R) _x1 *K+T _x ，R _y1 *K+T _y ，R _z1 *K+T _z )。

According to the formula from the point to the space plane, the projection point R of the fine tuning rotation point projected to the plane where the sighting device 605 is located is obtained through calculation ₀ And the projected point E of the needle tail point to the plane of the sight 605 ₀ And then the projected point is displayed on the display device 604. It will be appreciated that the display device 604 displays the projected points in conjunction with the sight 605The angles are the same. Illustratively, as shown at 606 in FIG. 6a, a fine rotation point displayed by the display device 604 may be understood as the fine rotation point, abbreviated as fine rotation point 606;607 may be understood as the needle tail point, abbreviated as needle tail point 607, displayed by the display device 604.

9. Puncture with surgical needle

As can be understood from step 8.3, since the center of the collimator is the target point, in the case that the projection point of the reference points (such as the fine adjustment rotation point and the needle tail point) on the collimator coincides with the center of the collimator, the surgical needle is on the same straight line with the puncture path, and then the surgical needle is advanced along the straight line to perform the operation.

Take the above fine tuning rotation point and the needle tail point as reference points. In the actual operation process, the fine adjustment rotating point can be firstly coincided with the center of the sighting device, and the needle tail point is close to the fine adjustment rotating point as much as possible, so that fine adjustment can be conveniently carried out. Then, the mechanical arm support is fixed for fine adjustment (the fine adjustment rotating point is fixed at the moment, but the needle tail point can be changed in position) until the fine adjustment rotating point, the needle tail point and the center point of the sighting device are superposed, and the puncture path alignment can be considered to be completed.

Referring to fig. 7, fig. 7 is a schematic diagram illustrating a result of finding a puncture path according to an aiming device according to an embodiment of the present application. In the case of three-point coincidence, the needle body of the surgical needle is already coincident with the puncture path, as shown in fig. 7.

The method provided by the present application is described above by taking a doctor as an operator and performing percutaneous renal surgery on a patient as an example, and the method provided by the embodiments of the present application is described below in conjunction with fig. 8 as a whole. Exemplarily, please refer to fig. 8, fig. 8 is a schematic flowchart of a method for displaying a positioning of a surgical needle according to an embodiment of the present application, where the method includes:

801: and acquiring a puncture path, wherein the puncture path is a virtual path from the surgical needle to a target point.

In the embodiment of the present application, the target point may be understood as a point to be punctured by a surgical needle inside the body of the subject. The implementation object may be any object requiring a surgical needle to perform puncture, and may be, for example, a human, such as a patient, a volunteer, or the like; or may be an animal. The target point can be a focus point in the body of the implementation object, or can also be called a focus target point, a target point and the like; or may be an artificially set point in a test experiment for compliance (such as an animal experiment).

In a possible implementation manner, the positioning and displaying device of the surgical needle can acquire a CT image of a region of interest (such as a lesion region) of the implementation object, and then perform three-dimensional reconstruction on the region of interest, and determine the puncture path according to the position of the target point in the implementation object and the situation around the target point. In another possible implementation manner, the positioning display device of the surgical needle may include an input device, and the path information input through the input device may be used as the puncture path, and the input device may be, for example, a keyboard, a mouse, a touch screen, or the like.

It is understood that the puncture path includes the above-mentioned target point, and may be represented by a line segment in an actual scene, for example, a dotted line portion displayed by the display device in fig. 7, which may be understood as a planned puncture path. For other introduction of this step, reference may also be made to the above description of fig. 1, and the related description of step 7.

802: and determining a reference plane, wherein an included angle between the reference plane and the puncture path is greater than or equal to a first threshold value, and the reference plane and the puncture path intersect at a first intersection point.

In this step, the included angle between the reference plane and the puncture path may be understood as an included angle between the reference plane and a straight line where the puncture path is located. It is understood that the included angle between the straight line and the plane is in the range of 0 to 90 °, and the first threshold may be any value greater than or equal to 85 ° and less than or equal to 90 ° as an example, and may be set according to the actual situation.

Generally, the closer the angle is to 90, the better the penetration guide for the operator. Theoretically, when the reference plane is perpendicular to the puncture path, the positioning display effect of the surgical needle is the best, the auxiliary effect on finding the puncture path is the greatest, and the surgical needle can be directly aligned with the puncture path. In this step, in the case where the reference plane is perpendicular to the puncture path, the reference plane may be understood as the aiming plane in the foregoing embodiment.

It will be appreciated that in the case where the angle is not equal to 90, the operator may perform a first alignment of the puncture path guided by the first reference plane, the first projection point and the second projection point, and then perform a fine adjustment of the surgical needle to perform a second alignment of the puncture path.

In this step, the first intersection point is an intersection point between the puncture path and the reference plane, and for example, the first intersection point may be a needle insertion point on the puncture path, and may be another point between the needle insertion point and a target point, such as a half point, and the specific description may also refer to the description of step 8.1.3.

In some embodiments, the first intersection point is the target point. It can be understood that the surgical needle finally needs to puncture with the puncture path and puncture to the target point, and therefore, the target point is displayed as the first intersection point, which can better assist the operator such as a doctor to find the puncture path.

In this step, for example, the positioning and displaying device of the surgical needle may determine a direction vector corresponding to the puncture path through any two points on the puncture path, and then obtain the reference plane according to the direction vector.

803: and displaying the reference plane and the first intersection point.

In general, the puncture path is considered to be definite, as it does not change after planning before a certain operation or experiment. Although there may be a plurality of planes having an angle greater than or equal to the first threshold with respect to the puncture path, in the embodiment of the present application, the reference plane is understood to be a fixed plane, for example, the reference plane may be a plane perpendicular to the puncture path, or may be a plane having an angle of 89 ° with respect to the puncture path. Therefore, after the puncture path is determined, the reference plane and the first intersection point are also determined, and the reference plane and the first intersection point are not changed in the subsequent operation process or experiment process.

Illustratively, as shown in FIG. 6b, the reference plane may be the plane of the sight 605, the first intersection point may be the target point T, and the spatial location of the reference plane and the first intersection point may not change after the puncture path has been determined. However, there may be a plurality of display modes when the positioning display device of the surgical needle displays the reference plane and the first intersection. For example, the display can be performed at any fixed position on the display screen, such as the upper left, the upper right, or the lower left corner of the display screen; alternatively, the positioning device of the surgical needle may also receive a user operation to change the display position, such as moving from the upper left to the upper right. It will be appreciated that the movement of the reference plane and the first intersection by the user operation is to be understood as a movement of the display position, which is a change of the display effect, while the spatial position of the reference plane and the first intersection is not changed.

Illustratively, as shown in FIG. 6a, the sight 605 and the target point T may be displayed at the upper right of the display device 604, the sight 605 and the target point T may be displayed at the lower right of the display device 604, or the sight 605 and the target point T may be displayed by moving to the middle of the display device 604 in response to a user operation. It will be appreciated that the position of the sight 605 and target point T in real space (e.g. the scene shown in figure 6 a) does not change, regardless of the position of the display.

In this step, displaying the reference plane may be understood as displaying a part of the reference plane, for example, a certain region including the first intersection, and may not be understood as displaying a plane in a mathematical sense. For example, the positioning display device of the surgical needle may display a circular region centered on the first intersection point on the reference plane, or a square region, a rectangular region, or other irregular regions centered on the first intersection point.

In the present step, the first step is carried out, the reference plane may be displayed including an auxiliary line. In some embodiments, the method shown in fig. 8 further comprises: and displaying a second auxiliary line, wherein the second auxiliary line comprises at least one concentric auxiliary line taking the first intersection point as a center of a circle and a plurality of angle auxiliary lines on the concentric auxiliary line.

In this embodiment, the display of the second auxiliary line can better assist the operator in controlling the position of the surgical needle, and further reduce the time taken for the alignment process of overlapping the surgical needle with the puncture path. Exemplarily, the reference plane including the second auxiliary line in the present embodiment may be understood as the sight shown in fig. 3.

804: determining a first projection point of a first reference point on the reference plane, and determining a second projection point of a second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle.

In an embodiment of the present application, the first reference point and the second reference point are determined according to a structure of the surgical needle. A line is determined based on the two non-coincident points, and the position of the surgical needle can be determined from the two reference points (i.e., the first reference point and the second reference point) on the surgical needle. Illustratively, the first reference point is a needle tail of the surgical needle, and the second reference point is a needle tip of the surgical needle; or, the first reference point is a needle tail of the surgical needle, and the second reference point is a rotation point for controlling the surgical needle. The rotation point may be understood as the fine-tuning rotation point in the foregoing embodiment, and refer to the description of step 803.

In this step, the surgical needle is clamped by a holder, at least 3 markers are fixedly disposed on the holder, and the spatial coordinates of the first reference point and the second reference point are determined by acquiring the coordinates of the at least 3 markers through an optical tracking device, which may be referred to the related description of fig. 1 and 2. That is, the coordinates of the first reference point and the second reference point can be determined by tracking the marker by the optical tracking device and the relative position relationship between the first reference point and the second reference point and the marker, and then the first projection point and the second projection point can be calculated according to the point-to-space plane formula, which can be referred to the description of step 8.3.

805: and displaying the first projection point and the second projection point.

It will be appreciated that the surgical needle is actually present in real space and the surgical needle needs to be moved continuously to align with the puncture path, and therefore the position of the surgical needle is changed, and the first projection point and the second projection point are also changed.

Also by way of example in fig. 6a, the reference plane may be the plane in which the sight 605 lies, the first intersection point may be the target point T, and the display device 604 will not display the

proxels

606 and 607 in the case where the surgical needle is not within the tracking range of the optical tracking device 603. In the case where the surgical needle is in the tracking range of the optical tracking device 603, the display device 604 displays the projected point of the reference point tracked by the optical tracking device 603.

In the embodiment of the present application, the positioning display device of the surgical needle may perform a differentiated display when displaying the first intersection point, the first projection point, and the second projection point. For example, the dots can be distinguished by different colors, different sizes and different shapes, and can be labeled.

According to the method provided by the embodiment of the application, the virtual puncture path of the surgical needle is obtained, the reference plane with the included angle larger than or equal to the first threshold value is determined according to the puncture path, and the reference plane and the first intersection point between the reference plane and the puncture path are displayed; when the surgical needle is displayed, a first projection point of a first reference point on the reference plane is displayed, and a second projection point of a second reference point on the reference plane is displayed. Since the final target of the surgical needle is to align with the puncture path, the operator can know the pose of the surgical needle, such as the inclination angle of the surgical needle, more clearly and intuitively by displaying the reference plane, the first intersection point and the projection point of the reference point on the reference plane, and if the distance between the first projection point and the second projection point is too far, the inclination angle of the surgical needle can be considered to be too large; the operator can know the position distance between the surgical needle and the puncture path more clearly and intuitively, for example, the surgical needle is at the right side of the puncture path. Therefore, the operator can feel directional in the process of moving the surgical needle by the mode, the operator can think that the surgical needle is approximately aligned with the puncture path by using the first intersection point as a target point and enabling the first projection point and the second projection point to coincide with the first intersection point as much as possible, and the alignment of the puncture path can be completed by finely adjusting the position of the surgical needle subsequently, so that the operation difficulty of the user is greatly reduced, and the time spent in the alignment process of the surgical needle and the puncture path in coincidence is further reduced.

In addition to displaying the reference plane and the first intersection, auxiliary lines may be displayed according to a reference direction. In some embodiments, the method of fig. 8, before step 804, further includes:

806: and determining a projection direction vector of a reference direction vector on the reference plane, wherein the reference direction vector is a fixed direction determined according to the real space of the surgical needle.

807: and displaying the first auxiliary line corresponding to the projection direction vector.

In this embodiment of the application, a direction corresponding to the reference direction vector may be referred to as a reference direction, and the reference direction is a fixed direction determined by a real space of the surgical needle, where the real space of the surgical needle may be understood as a real space where the surgical needle is located when being used, such as an operating room space and an experimental space of an animal experiment. The reference direction is fixed and unchanged in the process that the surgical needle aligns to the puncture path, and the reference direction in the real space is displayed, so that an operator can feel more directional in the process of moving the surgical needle and can more easily master the moving direction of the surgical needle, the time spent in the aligning process of the surgical needle and the puncture path in coincidence is reduced, and the puncture efficiency is improved.

In this embodiment, the reference direction may be a direction in which the target point points to the optical tracking device (first direction for short), a direction in which the target point points to a foot of the implementation object (second direction for short), and a direction in which the target point points to a head of the implementation object (third direction for short).

It will be appreciated that, typically, during operation, the subject is not subject to positional displacement, such as being previously anesthetized; the optical tracking device does not move, and thus, the first direction, the second direction, and the third direction may be considered to be fixed. If the reference direction is the second direction, the operator can move the surgical needle with the foot of the implementation object as a reference, and the optical tracking device can be placed near the foot for better tracking effect of the surgical needle; similarly, if the reference direction is the first direction or the third direction.

In this embodiment, there are various ways for determining the direction vector of the reference direction, such as by a surgical needle. For example, after the surgical needle is placed in parallel with the surgical object and the optical tracking device tracks the marker on the surgical needle, the coordinates of the needle tip and the needle tail may be determined by an optical tracking matrix (e.g., formula (4)), and the second direction or the third direction may be determined based on the coordinates of the needle tip and the needle tail. It should be understood that other reference directions set may be analogized.

It will be appreciated that since the surgical needle is generally directed towards the optical tracking device for better tracking effect of the surgical needle, the error of determining the direction vector of the third direction is minimized, and the effect and user experience of the first direction as the reference direction are better. When the reference direction is the first direction, the specific determination process may refer to the description of step 802.

It will be appreciated that after the reference direction vector is determined, the projection direction vector can be calculated according to a point-to-space plane formula. It will also be appreciated that the reference plane is determined, as is the reference direction, and therefore the projection direction vector and the corresponding first auxiliary line are also determined, i.e. the display effect of the first auxiliary line can be changed, but the spatial position of the projection vector corresponding to the first auxiliary line is not changed.

In this embodiment, the display order of step 807 and step 803 is not limited, and the reference plane, the first intersection, and the first auxiliary line may be displayed together, or the reference plane, the first intersection, and the first auxiliary line may be displayed first. When the first auxiliary line is displayed, a line segment may be displayed, specific information of the reference direction may be marked, and other manners (such as color, thickness, shape, and the like) may be used to distinguish the first auxiliary line from other auxiliary lines (such as the second auxiliary line) in other embodiments of the present application, which is not limited in this application.

Alternatively, the reference direction may be one or more. For example, the auxiliary lines corresponding to the first direction may be displayed, or the auxiliary lines corresponding to the first direction and the second direction may be displayed together.

It is understood that, in order to simplify the calculation step, the positioning display device of the surgical needle may first create an initial point and an initial auxiliary line on the coordinate plane, and then perform a rotational translation on the coordinate plane to convert the coordinate plane into the reference plane, where the initial point is converted into the first intersection point, and the initial auxiliary line is converted into the first auxiliary line and/or the second auxiliary line. The coordinate plane may be an XOY plane, an XOZ plane, and a YOZ plane.

Therefore, in some embodiments, the determining a reference plane, displaying the reference plane and the first intersection, and displaying the first auxiliary line corresponding to the projection direction vector includes:

(a) Taking a coordinate plane of a space coordinate system where the puncture path is located as a first plane, wherein the first plane comprises a first candidate point and a first candidate line;

in this embodiment, the spatial coordinate system where the puncture path is located may be understood as a coordinate system corresponding to the optical tracking device, that is, a world coordinate system. Illustratively, an XOY plane may be used as the first plane, and one point in XOY may be used as the first candidate point, such as the origin. The first candidate line may be understood as a line segment, for example, a line segment having the first intersection as an end point, or a straight line in which the first candidate line is located includes the first intersection.

(b) Determining an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is used for rotating the first plane to an included angle with the puncture path which is larger than or equal to the first threshold value, and the position of the first candidate point on the puncture path; the rotation angle is an included angle between a direction vector of the direction of the first candidate line and the projection direction vector;

in this step, the orientation transformation matrix may include a rotation component and a translation component, for example, as described in step 8.1.2, the rotation component may be determined based on an included angle between a normal vector of the first plane and a vector corresponding to the puncture path, and the translation component may be obtained according to a distance between the first candidate point and a second candidate point on the puncture path, so as to obtain the orientation transformation matrix.

The present application is not particularly limited with respect to specific values of the above-described rotational component and translational component. For example, for the rotation component, if the first plane is transformed based on the orientation transformation matrix, the first plane may be rotated to have an angle with the puncture path greater than or equal to the first threshold, for example, based on the rotation component, the first plane may be rotated to a position perpendicular to the puncture path, or the first plane may be rotated to a position having an angle of 89 ° with the puncture path, and it is understood that the rotation components corresponding to different angles are different. For example, as for the translation component, as long as the first plane is transformed based on the orientation transformation matrix, the position of the first candidate point on the puncture path may be obtained, for example, the first candidate point may be transformed to a target point on the puncture path, or may be transformed to another point on the puncture path, and it is understood that the translation components moved to different points on the puncture path are different.

In this step, after the orientation transformation matrix is determined, the first plane may be transformed based on the orientation transformation matrix to obtain a transformed first plane, and then the rotation angle may be determined. Wherein the transformed first plane includes a transformed first candidate line, and the transformed first candidate line is obtained based on the first candidate line rotational translation, and the collimator of the XOY plane is transformed to the collimation plane, similar to the step 8.1.2 in the previous embodiment.

In this step, an included angle between the projection direction vector of the reference direction vector on the transformed first plane and the direction vector of the transformed first candidate line may be calculated to obtain the rotation angle. For example, in the case where the first plane is a collimator and the first candidate line is a 90 ° line in the collimator, and the reference direction is the first direction (i.e. the direction pointing to the optical tracking device), the details of this step can also be found in the description of step 8.2 above.

(c) And rotating the first plane based on the rotation angle on the coordinate plane to obtain a second plane.

It is understood that after the orientation transformation matrix and the rotation angle have been determined, the first plane may be transformed to the collimation plane based on the orientation transformation matrix to obtain a transformed first plane; then, the first plane after the conversion is rotated based on the rotation angle with the puncture path as an axis.

However, since the puncture path is one path in space, there is often a certain angle with each coordinate axis, and the calculation amount of rotation with the puncture path as the axis is large. Therefore, in the present embodiment, the first plane is rotated on the coordinate plane, and it can be understood that the rotation axis used when the coordinate plane is rotated is a coordinate axis, and the calculation amount is small. For example, when the coordinate plane is an XOY plane, the rotation is performed with reference to the Z axis.

(d) Transforming the second plane based on the orientation transformation matrix to obtain the reference plane, the first intersection point, and the first auxiliary line; the first auxiliary line is obtained by rotating and translating based on the first candidate line; the first intersection point is obtained based on the translation of the first candidate point;

it is understood that the reference plane may be obtained by transforming the second plane based on the orientation transformation matrix, wherein the first auxiliary line in the reference plane may be obtained by translating the first auxiliary line through the rotation angle and the rotation of the orientation transformation matrix, and the first intersection point in the reference plane may be obtained by translating a translation component in the orientation transformation matrix.

(e) And displaying the reference plane, the first intersection, and the first auxiliary line.

It is to be understood that the first plane may include a second candidate line in addition to the first candidate line and the first candidate point. The second candidate line is first rotated by the rotation angle and then transformed by the orientation transformation matrix to obtain the second auxiliary line in the embodiment of the present application, which may be specifically referred to the descriptions of steps 8.1, 8.2, and 8.3.

The method of the embodiments of the present application is explained in detail above, and the apparatus of the embodiments of the present application is provided below.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 90 is used for executing the positioning display method of the surgical needle provided by the above embodiment. It is understood that any device capable of implementing the methods provided herein is within the scope of the present application. The electronic device 90 may be a notebook computer, a desktop computer, or the like, for example, and the embodiments of the present application are not limited thereto.

As shown in fig. 9, the electronic device 90 includes an obtaining unit 901, a determining unit 902, and a displaying unit 903, and optionally, may further include a transforming unit 904. Wherein the description of each unit is as follows:

an obtaining unit 901, configured to obtain a puncture path, where the puncture path is a virtual path that a surgical needle punctures to a target point;

a determining unit 902, configured to determine a reference plane, where an included angle between the reference plane and the puncture path is greater than or equal to a first threshold, and the reference plane intersects with the puncture path at a first intersection point;

a display unit 903 for displaying the reference plane and the first intersection;

a determining unit 902, further configured to determine a first projection point of a first reference point on the reference plane, and determine a second projection point of a second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle;

a display unit 903, configured to display the first projection point and the second projection point.

Optionally, the determining unit 902 is further configured to determine a projection direction vector of a reference direction vector on the reference plane, where the reference direction vector is a fixed direction determined according to a real space of the surgical needle;

the display unit 903 is further configured to display the first auxiliary line corresponding to the projection direction vector.

Optionally, the determining unit 902 is specifically configured to use a coordinate plane of a spatial coordinate system where the puncture path is located as a first plane, where the first plane includes a first candidate point and a first candidate line;

a determining unit 902, specifically configured to determine an orientation transformation matrix and a rotation angle, where the orientation transformation matrix is used to rotate the first plane to an angle with the puncture path that is greater than or equal to the first threshold, and a position of the first candidate point on the puncture path; the rotation angle is an included angle between a direction vector of the direction of the first candidate line and the projection direction vector;

the apparatus further comprises a transformation unit 904, configured to rotate the first plane based on the rotation angle on the coordinate plane to obtain a second plane;

a transforming unit 904, configured to transform the second plane based on the orientation transformation matrix to obtain the reference plane, the first intersection point, and the first auxiliary line; the first auxiliary line is obtained based on the rotation and translation of the first candidate line; the first intersection point is obtained based on the first candidate point in a translation mode;

the display unit 903 is further configured to display the reference plane, the first intersection point, and the first auxiliary line.

Optionally, the surgical needle is clamped by a holder, at least 3 markers are fixedly arranged on the holder, and the spatial coordinates of the first reference point and the second reference point are determined by acquiring the coordinates of the at least 3 markers by an optical tracking device; the reference direction vector comprises any one or more of:

the direction vector of the target point pointing to the direction of the optical tracking device, the direction vector of the target point pointing to the direction of the foot of the implementation object, and the direction vector of the target point pointing to the direction of the head of the implementation object.

Optionally, the first intersection point is the target point.

Optionally, the display unit 903 is further configured to display a second auxiliary line, where the second auxiliary line includes at least one concentric auxiliary line centered around the first intersection point, and a plurality of angle auxiliary lines on the concentric auxiliary line.

Referring to fig. 10, fig. 10 is a schematic structural diagram of another electronic device according to an embodiment of the present disclosure. The electronic device 100 may be used to implement the above-described positioning display method for the surgical needle.

As shown in fig. 10, the electronic device 100 includes a memory 1001 and a processor 1002. Optionally, the electronic device 100 may further include a communication interface 1003 and a bus 1004; further optionally, the electronic device 100 may further include a display screen 1005. The memory 1001, the processor 1002, the communication interface 1003, and the display screen 1005 are communicatively connected to each other via the bus 1004.

The memory 1001 is used to provide a storage space, and data such as an operating system and a computer program may be stored in the storage space. The memory 1001 includes, but is not limited to, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM).

The processor 1002 is a module for performing arithmetic operations and logical operations, and may be one or a combination of plural kinds of processing modules such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a microprocessor unit (MPU), or the like. In addition, a computer program is stored in the memory 1001, and the processor 1002 may call the computer program stored in the memory 1001 to execute a corresponding method.

The display screen 1005 is used to implement the display function of the electronic device 100. Illustratively, the display screen 1005 may be used to display the collimator, the first projection point, the second projection point, the puncture path, and the like. In some embodiments, the display screen 1005 may implement the functions of the display device 106 and the display device 604 in the previous embodiments.

In this embodiment of the application, when the electronic device 100 shown in fig. 10 executes the above positioning display method for a surgical needle, the processor 1002 may control a display function of the display screen 1005 and may also control a data communication function of the communication interface 1003.

Illustratively, in some embodiments, a processor 1002 for obtaining a puncture path;

a processor 1002 for determining a reference plane, determining a first projected point of a first reference point on the reference plane, and determining a second projected point of a second reference point on the reference plane;

and the processor 1002 is configured to control the display screen 1005 to display the reference plane, the first intersection point, the first projection point and the second projection point.

In another embodiment, the processor 1002 may be configured to implement the functions of the acquiring unit 901, the determining unit 902, and the transforming unit 904 in the electronic device 90; the display screen 1005 may be controlled by the processor 1002 for performing the functions of the display unit 903 in the electronic device 90. Alternatively, the data acquired by the acquisition unit 901 in the electronic device 90 may also be acquired through the communication interface 1003.

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program runs on a processor, the method in the embodiment of the present application may be implemented.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the following claims.

Claims

1. A method for displaying the position of a surgical needle, the method comprising:

displaying the reference plane and the first intersection point;

and displaying the first projection point and the second projection point.

2. The method of claim 1, wherein prior to determining a first projected point of a first reference point on the reference plane and determining a second projected point of a second reference point on the reference plane, the method further comprises:

determining a projection direction vector of a reference direction vector on the reference plane, the reference direction vector being a fixed direction determined from a real space of the surgical needle;

and displaying a first auxiliary line corresponding to the projection direction vector.

3. The method of claim 2, wherein the determining a reference plane, displaying the reference plane and the first intersection, and displaying a first auxiliary line corresponding to the projection direction vector comprises:

determining an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is used for rotating the first plane to an included angle between the first plane and the puncture path, and the included angle is larger than or equal to the first threshold, and the position of the first candidate point on the puncture path is determined; the rotation angle is an included angle between a direction vector of the direction of the first candidate line and the projection direction vector;

transforming the second plane based on the orientation transformation matrix to obtain the reference plane, the first intersection point and the first auxiliary line; the first auxiliary line is obtained based on rotation and translation of the first candidate line; the first intersection point is obtained based on the translation of the first candidate point;

displaying the reference plane, the first intersection point, and the first auxiliary line.

4. The method according to claim 2 or 3, wherein the surgical needle is held by a holder on which at least 3 markers are fixedly arranged, the spatial coordinates of the first reference point and the second reference point being determined by acquiring the coordinates of the at least 3 markers by means of an optical tracking device; the reference direction vector comprises any one or more of:

5. The method of any one of claims 1-4, wherein the first intersection point is the target point.

6. The method according to any one of claims 1-5, further comprising:

and displaying a second auxiliary line, wherein the second auxiliary line comprises at least one concentric auxiliary line taking the first intersection point as a circle center, and a plurality of angle auxiliary lines on the concentric auxiliary line.

7. The method of any one of claims 1-6, wherein the first reference point is a needle tail of the surgical needle and the second reference point is a needle tip of the surgical needle, or a rotation point for controlling the surgical needle.

8. A device for displaying the position of a surgical needle, the device comprising:

the determining unit is used for determining a reference plane, wherein an included angle between the reference plane and the puncture path is larger than or equal to a first threshold value, and the reference plane and the puncture path intersect at a first intersection point;

a display unit for displaying the reference plane and the first intersection;

9. An electronic device, comprising a processor, a memory, and a display screen, the memory for storing a computer program comprising program instructions, the processor configured to invoke the program instructions such that the method of any of claims 1-7 is performed.

10. A computer-readable storage medium, characterized in that it stores a computer program comprising program instructions which, when executed by a processor, cause the method according to any one of claims 1-7 to be performed.