CN109893247B - Navigation and display method and device for operation path of surgical instrument - Google Patents
Navigation and display method and device for operation path of surgical instrument Download PDFInfo
- Publication number
- CN109893247B CN109893247B CN201910288110.5A CN201910288110A CN109893247B CN 109893247 B CN109893247 B CN 109893247B CN 201910288110 A CN201910288110 A CN 201910288110A CN 109893247 B CN109893247 B CN 109893247B
- Authority
- CN
- China
- Prior art keywords
- point
- camera
- navigation path
- surgical instrument
- navigation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the invention provides a navigation and display method and device for an operation path of a surgical instrument. The method comprises the following steps: acquiring sampling points of the operation path of the surgical instrument according to the medical image information; generating a surgical instrument navigation path according to the sampling point; and calculating the position of a rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information, and performing real-time guidance according to the navigation information. The embodiment of the invention can guide the access operation of the surgical instrument in real time, and visually display the navigation picture at the optimal visual angle to obtain clearer and more accurate navigation information.
Description
Technical Field
The embodiment of the invention relates to the technical field of image-guided surgery navigation, in particular to a navigation and display method and device for an operation path of a surgical instrument.
Background
The image-guided surgery navigation technology provides a very visual navigation information for doctors based on various medical image information and augmented reality technology. Wherein, the medical image information refers to Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), Digital Subtraction Angiography (DSA), and 2D \3D endoscopic images; the provided navigation information specifically comprises the position of the key tissues (tumor, nerve, blood vessel, bone and face) of the patient, the relative position relationship between the endoscope and the surgical instruments and the key tissues of the patient, augmented reality-based endoscope extension images, the operation path and the like. The image-guided surgical navigation system greatly expands the limited visual range of the traditional endoscope and realizes the accurate navigation (sub-millimeter level) of surgical instruments.
However, currently, an efficient and intuitive rendering display method is still lacking for navigation information such as the access progress of a surgical instrument, the relative position relationship between the surgical instrument and a patient key tissue, and the deviation between the surgical instrument and a surgical path, and the navigation efficiency is also affected by the angle and definition of a rendered image, so how to render images of the surgical instrument and the navigation path at an optimal view angle to visually display the navigation information becomes a problem to be solved urgently.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a navigation and display method and device for an operation path of a surgical instrument.
The embodiment of the invention provides a navigation and display method for an operation path of a surgical instrument, which comprises the following steps:
acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
generating a surgical instrument navigation path according to the sampling point;
and calculating the position of a rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information, and performing real-time guidance according to the navigation information.
The embodiment of the invention provides a navigation and display device for an operation path of a surgical instrument, which comprises:
the acquisition unit is used for acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
the generating unit is used for generating a surgical instrument navigation path according to the sampling point;
and the navigation display unit is used for calculating the position of the rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information and performing real-time guidance according to the navigation information.
The embodiment of the invention also provides electronic equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the navigation and display method of the operation path of the surgical instrument is realized.
The embodiment of the invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for navigating and displaying the operation path of the surgical instrument is realized.
According to the navigation and display method and device for the operation path of the surgical instrument, the navigation path of the surgical instrument is generated, the access operation of the surgical instrument is guided in real time, and the position of the rendering camera is changed in real time according to the relative position relation between the tip position of the surgical instrument and the navigation path, so that a navigation picture is visually displayed at the optimal visual angle, and clearer and more accurate navigation information is obtained.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a flowchart illustrating a method for navigating and displaying an operation path of a surgical instrument according to an embodiment of the present invention;
FIG. 2 is a schematic illustration of calculating a rendered camera position according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for adjusting a navigation path according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a navigation and display device for operation path of a surgical instrument according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an apparatus for adjusting a navigation path according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart illustrating a method for navigating and displaying an operation path of a surgical instrument according to an embodiment of the present invention.
As shown in fig. 1, the navigation and display method for the operation path of the surgical instrument provided by the embodiment of the present invention specifically includes the following steps:
s11, acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
specifically, a number of path sampling points are obtained from patient medical image information.
The medical image information specifically includes: electron Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), Digital Subtraction Angiography (DSA), and 2D \3D endoscopic images, among others.
S12, generating a surgical instrument navigation path according to the sampling point;
specifically, a three-dimensional tubular model is fitted to a series of sampling points by adopting an interpolation algorithm and is used as a navigation path of the surgical instrument.
S13, calculating the position of a rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information, and performing real-time guidance according to the navigation information;
in particular, the navigation path is used to guide an approach operation of the surgical instrument. And changing the position of the rendering camera in real time according to the relative position relation between the tip position of the current surgical instrument and the navigation path so as to obtain an optimal navigation picture (the surgical navigation path is always positioned at the lower right part in the rendering picture). The navigation picture is a picture rendered by rendering the visual angle of the camera, the camera position with the best visual angle can render to obtain an ideal navigation picture, the access progress of the current surgical instrument and the deviation information of the surgical instrument and the surgical path can be displayed in real time, and the position of the surgical instrument can be conveniently adjusted in real time.
The navigation information displayed specifically includes: the position of the key tissues of a patient (tumor, nerve, blood vessel, bone and face), the relative position relationship between the endoscope and surgical instruments and the key tissues of the patient, augmented reality-based endoscope extension images, operation paths and the like.
According to the navigation and display method for the operation path of the surgical instrument, the navigation path of the surgical instrument is generated, the access operation of the surgical instrument is guided in real time, and the position of the rendering camera is changed in real time according to the relative position relation between the tip position of the surgical instrument and the navigation path, so that a navigation picture is visually displayed at the optimal visual angle, and clearer and more accurate navigation information is obtained.
On the basis of the foregoing embodiment, S12 specifically includes:
performing cardinal number spline interpolation fitting on the three-dimensional space coordinate information of the sampling points to obtain an interpolated point set;
and generating a corresponding smooth curved hollow tubular three-dimensional model according to the point set after interpolation.
Specifically, three-dimensional space coordinates of a plurality of path sampling points are obtained according to medical image information of a patient, cardinal number spline interpolation fitting is carried out on the three-dimensional space coordinate information of the path sampling points, and a point set after interpolation is obtained. And generating a corresponding smooth curved hollow tubular three-dimensional model, namely a surgical navigation path, by using the interpolated point set. The parameters required to be set in the process include: interpolation rate, radius of the hollow tubular three-dimensional model and initial color.
It should be noted that the three-dimensional space coordinates of the sampling point are coordinates in the entire three-dimensional space coordinate system, and the entire three-dimensional space includes the entire critical tissue, the surgical operation path, and the position of the surgical instrument.
On the basis of the foregoing embodiment, the step S13 of calculating the position of the rendering camera according to the relative position relationship between the tip position of the current surgical instrument and the navigation path specifically includes:
vertically mapping the tip position of the current surgical instrument to the navigation path to obtain a mapping point, and acquiring a point which is closest to the mapping point in the point set;
specifically, the position P of the tip of the current surgical instrument is vertically mapped to a surgical navigation path to obtain a mapping point P ', and a point closest to the mapping point P' in the interpolated point set is obtained
According to the point
The position of the camera and the rendering parameters are calculated. The parameters of the rendering camera include an upward direction, a focal position, a focal length, a front-back cut-off display plane position, and a rendering method of the camera.
Calculating a camera position vector and a position vector of a camera focus according to a point which is closest to the mapping point in the point set and establishing a camera coordinate system; the method comprises the following specific steps:
when the nth point in the point set
When the mapping point is nearest, the position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
fig. 2 shows a schematic illustration of the calculation of the rendering camera position.
As shown in figure 2 of the drawings, in which,
the camera-to-path distance parameter is r for the position vectors of the nth point, the (n +1) th point and the (n + 5) th point in the point set after the cardinal number spline interpolation.
And obtaining a focal length size parameter according to the position vector of the rendering camera and the position vector of the focal point of the rendering camera.
The unit vectors of the three coordinate axes of the camera coordinate system are:
the rendering coordinate system is the whole three-dimensional space coordinate system mentioned in the above embodiments, and the whole critical tissues, the operation path and the positions of the surgical instruments are included in the rendering coordinate system.
On the basis of the above embodiment, the step S13 of performing real-time guidance according to the relative position relationship between the current tip position of the surgical instrument and the navigation path specifically includes:
generating a colored semitransparent wafer by taking a mapping point of the tip position of the current surgical instrument on the navigation path as a circle center, and generating a series of transparent rings which are equally spaced and vertical to the center line of the navigation path in real time on the navigation path before the mapping point so as to indicate the access progress of the current surgical instrument;
specifically, the position P of the tip of the surgical instrument during the operation is vertically mapped to the navigation path, so as to obtain a mapping point P 'and a vertical distance D between P and the surgical path, wherein the navigation path before the mapping point P' is the path that the surgical instrument has traveled.
Generating a semitransparent wafer with bright color by taking the mapping point P' as the center of a circle, wherein the parameters required to be set comprise: disc radius, disc color, disc opacity.
And generating a series of transparent circular rings which are equally spaced and are perpendicular to the central line of the path on the navigation path before the mapping point P', and indicating the access progress of the current surgical instrument. The setting of the parameters includes: the radius of the inner circle and the outer circle of the circular ring, the opacity of the circular ring, the color of the circular ring and the interval between the circular rings.
And setting the color of the navigation path according to the vertical distance between the current tip position of the surgical instrument and the navigation path so as to guide in real time.
Specifically, the color of the navigation path varies with the vertical distance D to indicate real-time adjustment of the surgical instrument.
On the basis of the above embodiment, the setting the color of the navigation path according to the perpendicular distance between the current tip position of the surgical instrument and the navigation path for real-time guidance includes:
when the vertical distance is larger than or equal to a specified threshold value, setting the color of the navigation path to be red to warn the path deviation;
when the vertical distance is less than the specified threshold, the color of the navigation path is set to green to indicate that the path is safe.
Specifically, when the vertical distance D exceeds the upper threshold, the path is pure red to play a warning role, when the vertical distance D is smaller than the lower threshold, the path is green to indicate safety, and when the vertical distance D is within the upper and lower thresholds, the path is colored according to the distance-color mapping map. The parameter that needs to be set for this step is the distance-color mapping map.
After the surgical instrument navigation path is generated, the embodiment of the invention also provides a method for adjusting the navigation path so as to obtain a qualified navigation path.
Fig. 3 is a flowchart illustrating a method for adjusting a navigation path according to an embodiment of the present invention.
As shown in fig. 3, the method specifically includes the following steps:
s21, sequentially traversing all points in the point set from the first point of the point set after interpolation, and performing simulated roaming of the navigation path;
specifically, after the surgical instrument navigation path is generated, the navigation is simulated in the navigation path, and the camera position is updated with the navigation path timing in steps (e.g., 100 ms/step). If there are N points in the point set S, the roaming process needs to start from the first point (N is 0), sequentially traverse all the points in the point set S, and roam to the nth point with a roaming progress I of N/(N-5) × 100%.
S22, calculating the position of the rendering camera according to the current roaming position, and rendering pictures to display navigation information;
specifically, the process of calculating the position of the rendering camera from the current roaming position (nth point in the point set) in the roaming process is the same as the embodiment of calculating the camera position in S13 described above.
And S23, after the roaming is finished, adjusting the navigation path according to the roaming effect.
Specifically, it is determined whether the roaming progress I is already equal to 100%. If not, stepping is performed (n ═ n + 1). If the roaming rate reaches 100%, judging whether the current operation navigation path is qualified according to the roaming effect after the roaming is finished. If the current surgical navigation path is unqualified, the sampling point is reselected, the navigation path is adjusted, and if the current surgical navigation path is qualified, the finally qualified navigation path is used as the operation path of the surgical instrument.
It should be noted that the navigation path acquired in the embodiment of real-time guidance and display may be generated directly through the sampling point, instead of being adjusted through a roaming operation, or may be an operation navigation path that is determined to be qualified and is obtained by adjusting the navigation path through a roaming operation.
On the basis of the above embodiment, the calculating the position of the rendering camera according to the current roaming position includes:
the current roaming position is the nth point in the point set
The position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
fig. 4 is a schematic structural diagram of a navigation and display device for a surgical instrument operation path according to an embodiment of the present invention.
As shown in fig. 4, the navigation and display device for operation path of surgical instrument provided by the embodiment of the present invention includes: an acquisition unit 11, a generation unit 12 and a navigation display unit 13, wherein:
the acquisition unit 11 is used for acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
specifically, a number of path sampling points are obtained from patient medical image information.
The medical image information specifically includes: electron Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), Digital Subtraction Angiography (DSA), and 2D \3D endoscopic images, among others.
The generating unit 12 is configured to generate a surgical instrument navigation path according to the sampling point;
specifically, a three-dimensional tubular model is fitted to a series of sampling points by adopting an interpolation algorithm and is used as a navigation path of the surgical instrument.
The navigation display unit 13 is configured to calculate a position of the rendering camera according to a relative position relationship between a current tip position of the surgical instrument and the navigation path, generate a rendering image to display navigation information, and perform real-time guidance according to the navigation information.
In particular, the navigation path is used to guide an approach operation of the surgical instrument. And changing the position of the rendering camera in real time according to the relative position relation between the tip position of the current surgical instrument and the navigation path so as to obtain an optimal navigation picture (the surgical navigation path is always positioned at the lower right part in the rendering picture). The navigation picture is a picture rendered by rendering the visual angle of the camera, the camera position with the best visual angle can render to obtain an ideal navigation picture, the access progress of the current surgical instrument and the deviation information of the surgical instrument and the surgical path can be displayed in real time, and the position of the surgical instrument can be conveniently adjusted in real time.
The navigation information displayed specifically includes: the position of the key tissues of a patient (tumor, nerve, blood vessel, bone and face), the relative position relationship between the endoscope and surgical instruments and the key tissues of the patient, augmented reality-based endoscope extension images, operation paths and the like.
The navigation and display device for the operation path of the surgical instrument, provided by the embodiment of the invention, can be used for generating the navigation path of the surgical instrument, guiding the access operation of the surgical instrument in real time, and changing the position of the rendering camera in real time according to the relative position relation between the tip position of the surgical instrument and the navigation path, so that a navigation picture is visually displayed at an optimal visual angle, and clearer and more accurate navigation information is obtained.
On the basis of the above embodiment, the generating unit 12 includes: a fitting module and a generating module, wherein:
the fitting module is used for carrying out radix spline interpolation fitting on the three-dimensional space coordinate information of the sampling points to obtain an interpolated point set;
and the generating module is used for generating a corresponding smooth curved hollow tubular three-dimensional model according to the point set after interpolation.
Specifically, three-dimensional space coordinates of a plurality of path sampling points are obtained according to medical image information of a patient, cardinal number spline interpolation fitting is carried out on the three-dimensional space coordinate information of the path sampling points, and a point set after interpolation is obtained. And generating a corresponding smooth curved hollow tubular three-dimensional model, namely a surgical navigation path, by using the interpolated point set. The parameters required to be set in the process include: interpolation rate, radius of the hollow tubular three-dimensional model and initial color.
It should be noted that the three-dimensional space coordinates of the sampling point are coordinates in the entire three-dimensional space coordinate system, and the entire three-dimensional space includes the entire critical tissue, the surgical operation path, and the position of the surgical instrument.
On the basis of the above embodiment, the navigation display unit 13 includes: an acquisition module and a calculation module, wherein:
the acquisition module is used for vertically mapping the tip position of the current surgical instrument to the navigation path to obtain a mapping point and acquiring a point which is closest to the mapping point in the point set;
specifically, the current hand will beThe position P of the tip of the surgical instrument is vertically mapped to a surgical navigation path to obtain a mapping point P ', and a point which is closest to the mapping point P' in the point set after interpolation is obtained
According to the point
The position of the camera and the rendering parameters are calculated. The parameters of the rendering camera include an upward direction, a focal position, a focal length, a front-back cut-off display plane position, and a rendering method of the camera.
The calculation module is used for calculating a camera position vector and a position vector of a camera focus according to the point which is closest to the mapping point in the point set and establishing a camera coordinate system; the method specifically comprises the following steps:
when the nth point in the point set
When the mapping point is nearest, the position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
as camera coordinatesThe unit vector in the upward direction in the system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
with reference to figure 2 of the drawings,
the camera-to-path distance parameter is r for the position vectors of the nth point, the (n +1) th point and the (n + 5) th point in the point set after the cardinal number spline interpolation.
And obtaining a focal length size parameter according to the position vector of the rendering camera and the position vector of the focal point of the rendering camera.
The unit vectors of the three coordinate axes of the camera coordinate system are:
the rendering coordinate system is the whole three-dimensional space coordinate system mentioned in the above embodiments, and the whole critical tissues, the operation path and the positions of the surgical instruments are included in the rendering coordinate system.
On the basis of the above embodiment, the navigation display unit 13 further includes: a generation module and a guidance module, wherein:
the generation module is used for generating a colored semitransparent wafer by taking a mapping point of the tip position of the current surgical instrument on the navigation path as a circle center, and generating a series of transparent rings which are at equal intervals and perpendicular to the center line of the navigation path in real time on the navigation path in front of the mapping point so as to indicate the access progress of the current surgical instrument;
specifically, the position P of the tip of the surgical instrument during the operation is vertically mapped to the navigation path, so as to obtain a mapping point P 'and a vertical distance D between P and the surgical path, wherein the navigation path before the mapping point P' is the path that the surgical instrument has traveled.
Generating a semitransparent wafer with bright color by taking the mapping point P' as the center of a circle, wherein the parameters required to be set comprise: disc radius, disc color, disc opacity.
And generating a series of transparent circular rings which are equally spaced and are perpendicular to the central line of the path on the navigation path before the mapping point P', and indicating the access progress of the current surgical instrument. The setting of the parameters includes: the radius of the inner circle and the outer circle of the circular ring, the opacity of the circular ring, the color of the circular ring and the interval between the circular rings.
The guiding module is used for setting the color of the navigation path according to the vertical distance between the tip position of the current surgical instrument and the navigation path so as to guide in real time.
Specifically, the color of the navigation path varies with the vertical distance D to indicate real-time adjustment of the surgical instrument.
On the basis of the above embodiment, the guidance module is specifically configured to set the color of the navigation path to red to alert a path deviation when the vertical distance is greater than or equal to a specified threshold; when the vertical distance is less than the specified threshold, the color of the navigation path is set to green to indicate that the path is safe.
Specifically, when the vertical distance D exceeds the upper threshold, the path is pure red to play a warning role, when the vertical distance D is smaller than the lower threshold, the path is green to indicate safety, and when the vertical distance D is within the upper and lower thresholds, the path is colored according to the distance-color mapping map. The parameter that needs to be set for this step is the distance-color mapping map.
Fig. 5 is a schematic structural diagram illustrating an apparatus for adjusting a navigation path according to an embodiment of the present invention.
As shown in fig. 5, an apparatus for adjusting a navigation path according to an embodiment of the present invention includes: a roaming unit 21, a calculating unit 22 and an adjusting unit 23, wherein:
the roaming unit 21 is configured to sequentially traverse all points in the point set from a first point of the interpolated point set, and perform simulated roaming of the navigation path;
specifically, after the surgical instrument navigation path is generated, the navigation is simulated in the navigation path, and the camera position is updated with the navigation path timing in steps (e.g., 100 ms/step). If there are N points in the point set S, the roaming process needs to start from the first point (N is 0), sequentially traverse all the points in the point set S, and roam to the nth point with a roaming progress I of N/(N-5) × 100%.
The calculating unit 22 is configured to calculate a position of the rendering camera according to the current roaming position, and is configured to render a picture to display navigation information;
specifically, the process of calculating the position of the rendering camera from the current roaming position (nth point in the set of points) in the roaming process is the same as the above-described embodiment of calculating the camera position.
The adjusting unit 23 is configured to adjust the navigation path according to the roaming effect after the roaming is completed.
Specifically, it is determined whether the roaming progress I is already equal to 100%. If not, stepping is performed (n ═ n + 1). If the roaming rate reaches 100%, judging whether the current operation navigation path is qualified according to the roaming effect after the roaming is finished. If the current surgical navigation path is unqualified, the sampling point is reselected, the navigation path is adjusted, and if the current surgical navigation path is qualified, the finally qualified navigation path is used as the operation path of the surgical instrument.
It should be noted that the navigation path acquired in the embodiment of the navigation display unit may be generated directly through the sampling point, and is obtained by adjusting the navigation path without performing a roaming operation, or may be a qualified navigation path obtained by adjusting the navigation path through a roaming operation.
On the basis of the above embodiment, the calculating unit 22 is used for the nth point in the point set as the current roaming position
The position of the camera is towardsMeasured as
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
the above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
An example is as follows:
fig. 6 illustrates a physical structure diagram of a server, and as shown in fig. 6, the server may include: a processor (processor)31, a communication Interface (communication Interface)32, a memory (memory)33 and a communication bus 34, wherein the processor 31, the communication Interface 32 and the memory 33 are communicated with each other via the communication bus 34. The processor 31 may call logic instructions in the memory 33 to perform the following method: acquiring sampling points of the operation path of the surgical instrument according to the medical image information; generating a surgical instrument navigation path according to the sampling point; and performing real-time guidance according to the relative position relation between the tip position of the current surgical instrument and the navigation path, and calculating the position of a rendering camera for rendering a picture to display navigation information.
In addition, the logic instructions in the memory 23 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Claims (3)
1. A navigation and display device for a surgical instrument operation path, the device comprising:
the acquisition unit is used for acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
the generating unit is used for generating a surgical instrument navigation path according to the sampling point;
the navigation display unit is used for calculating the position of the rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information and performing real-time guidance according to the navigation information;
the generation unit includes: a fitting module and a first generating module, wherein:
the fitting module is used for carrying out radix spline interpolation fitting on the three-dimensional space coordinate information of the sampling points to obtain an interpolated point set;
the first generation module is used for generating a corresponding smooth curved hollow tubular three-dimensional model according to the point set after interpolation;
the navigation display unit includes: an acquisition module and a calculation module, wherein:
the acquisition module is used for vertically mapping the tip position of the current surgical instrument to the navigation path to obtain a mapping point and acquiring a point which is closest to the mapping point in the point set;
the calculation module is used for calculating a camera position vector and a position vector of a camera focus according to the point which is closest to the mapping point in the point set and establishing a camera coordinate system; wherein the calculating a camera position vector, a position vector of a camera focus and establishing a camera coordinate system comprises:
when the nth point in the point set
When the mapping point is nearest, the position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
horizontal plane in camera coordinate systemThe unit vectors of the two coordinate axes are respectively:
and
the navigation display unit further includes: a second generation module and a guidance module, wherein:
the second generation module is used for generating a colored semitransparent wafer by taking a mapping point of the tip position of the current surgical instrument on the navigation path as a circle center, and generating a series of transparent rings which are at equal intervals and perpendicular to the center line of the navigation path in real time on the navigation path in front of the mapping point so as to indicate the access progress of the current surgical instrument;
the guiding module is used for setting the color of the navigation path according to the vertical distance between the current tip position of the surgical instrument and the navigation path so as to guide in real time;
the guiding module is specifically used for setting the color of the navigation path to be red when the vertical distance is greater than or equal to a specified threshold value so as to warn the deviation of the path; when the vertical distance is less than the specified threshold, setting the color of the navigation path to green to indicate that the path is safe;
the device further comprises: roaming unit, computational element and adjustment unit, wherein:
the roaming unit is used for sequentially traversing all points in the point set from the first point of the point set after interpolation to perform simulated roaming of the navigation path;
the computing unit is used for computing the position of the rendering camera according to the current roaming position and rendering a picture to display navigation information;
the adjusting unit is used for adjusting the navigation path according to the roaming effect after the roaming is finished;
the computing unit is specifically used for the nth point in the point set of the current roaming position
The position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
2. an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the following method for navigating and displaying an operative path of a surgical instrument:
acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
generating a surgical instrument navigation path according to the sampling point;
calculating the position of a rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information, and performing real-time guidance according to the navigation information;
wherein the generating a surgical instrument navigation path from the sampling points comprises:
performing cardinal number spline interpolation fitting on the three-dimensional space coordinate information of the sampling points to obtain an interpolated point set;
generating a corresponding smooth curved hollow tubular three-dimensional model according to the interpolated point set;
the calculating the position of the rendering camera according to the relative position relationship of the tip position of the current surgical instrument and the navigation path comprises:
vertically mapping the tip position of the current surgical instrument to the navigation path to obtain a mapping point, and acquiring a point which is closest to the mapping point in the point set;
calculating a camera position vector and a position vector of a camera focus according to a point which is closest to the mapping point in the point set and establishing a camera coordinate system;
the calculating a camera position vector, a position vector of a camera focus and establishing a camera coordinate system comprises:
when the nth point in the point set
When the mapping point is nearest, the position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
the real-time guidance according to the navigation information comprises:
generating a colored semitransparent wafer by taking a mapping point of the tip position of the current surgical instrument on the navigation path as a circle center, and generating a series of transparent rings which are equally spaced and vertical to the center line of the navigation path in real time on the navigation path before the mapping point so as to indicate the access progress of the current surgical instrument;
setting the color of the navigation path according to the vertical distance between the current tip position of the surgical instrument and the navigation path so as to guide in real time;
the setting the color of the navigation path according to the perpendicular distance between the current tip position of the surgical instrument and the navigation path for real-time guidance comprises:
when the vertical distance is larger than or equal to a specified threshold value, setting the color of the navigation path to be red to warn the path deviation;
when the vertical distance is less than the specified threshold, setting the color of the navigation path to green to indicate that the path is safe;
after the generating of the surgical instrument navigation path, the method further comprises the step of adjusting the navigation path:
sequentially traversing all points in the point set from the first point of the point set after interpolation, and performing simulated roaming of a navigation path;
calculating the position of a rendering camera according to the current roaming position, and rendering a picture to display navigation information;
after the roaming is finished, the navigation path is adjusted according to the roaming effect;
the calculating the position of the rendering camera according to the current roaming position comprises:
the current roaming position is the nth point in the point set
The position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
3. a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the following steps of a method for navigating and displaying a surgical instrument operational path:
acquiring sampling points of the operation path of the surgical instrument according to the medical image information;
generating a surgical instrument navigation path according to the sampling point;
calculating the position of a rendering camera according to the relative position relation between the tip position of the current surgical instrument and the navigation path, generating a rendering picture to display navigation information, and performing real-time guidance according to the navigation information;
wherein the generating a surgical instrument navigation path from the sampling points comprises:
performing cardinal number spline interpolation fitting on the three-dimensional space coordinate information of the sampling points to obtain an interpolated point set;
generating a corresponding smooth curved hollow tubular three-dimensional model according to the interpolated point set;
the calculating the position of the rendering camera according to the relative position relationship of the tip position of the current surgical instrument and the navigation path comprises:
vertically mapping the tip position of the current surgical instrument to the navigation path to obtain a mapping point, and acquiring a point which is closest to the mapping point in the point set;
calculating a camera position vector and a position vector of a camera focus according to a point which is closest to the mapping point in the point set and establishing a camera coordinate system;
the calculating a camera position vector, a position vector of a camera focus and establishing a camera coordinate system comprises:
when the nth point in the point set
When the mapping point is nearest, the position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
the real-time guidance according to the navigation information comprises:
generating a colored semitransparent wafer by taking a mapping point of the tip position of the current surgical instrument on the navigation path as a circle center, and generating a series of transparent rings which are equally spaced and vertical to the center line of the navigation path in real time on the navigation path before the mapping point so as to indicate the access progress of the current surgical instrument;
setting the color of the navigation path according to the vertical distance between the current tip position of the surgical instrument and the navigation path so as to guide in real time;
the setting the color of the navigation path according to the perpendicular distance between the current tip position of the surgical instrument and the navigation path for real-time guidance comprises:
when the vertical distance is larger than or equal to a specified threshold value, setting the color of the navigation path to be red to warn the path deviation;
when the vertical distance is less than the specified threshold, setting the color of the navigation path to green to indicate that the path is safe;
after the generating of the surgical instrument navigation path, the method further comprises the step of adjusting the navigation path:
sequentially traversing all points in the point set from the first point of the point set after interpolation, and performing simulated roaming of a navigation path;
calculating the position of a rendering camera according to the current roaming position, and rendering a picture to display navigation information;
after the roaming is finished, the navigation path is adjusted according to the roaming effect;
the calculating the position of the rendering camera according to the current roaming position comprises:
the current roaming position is the nth point in the point set
The position vector of the camera is
Wherein the content of the first and second substances,
the n +1 point in the point set is defined, and r is the distance between the camera and the navigation path;
the position vector of the focal point of the camera is
Wherein the content of the first and second substances,
the (n + 5) th point in the set of points,
is a unit vector in the upward direction in the camera coordinate system,
the unit vectors of two coordinate axes of the horizontal plane in the camera coordinate system are respectively:
and
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910288110.5A CN109893247B (en) | 2019-04-11 | 2019-04-11 | Navigation and display method and device for operation path of surgical instrument |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910288110.5A CN109893247B (en) | 2019-04-11 | 2019-04-11 | Navigation and display method and device for operation path of surgical instrument |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109893247A CN109893247A (en) | 2019-06-18 |
| CN109893247B true CN109893247B (en) | 2021-01-01 |
Family
ID=66955672
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910288110.5A Active CN109893247B (en) | 2019-04-11 | 2019-04-11 | Navigation and display method and device for operation path of surgical instrument |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109893247B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110833453A (en) * | 2019-12-03 | 2020-02-25 | 山东大学齐鲁医院 | Tumor puncture real-time simulation device and method |
| CN112037314A (en) * | 2020-08-31 | 2020-12-04 | 北京市商汤科技开发有限公司 | Image display method, image display device, display equipment and computer readable storage medium |
| CN113456219B (en) * | 2021-06-15 | 2022-11-29 | 艾瑞迈迪医疗科技（北京）有限公司 | Liver cancer radio frequency ablation operation path planning method and device based on CT image |
| CN116919599B (en) * | 2023-09-19 | 2024-01-09 | 中南大学 | Haptic visual operation navigation system based on augmented reality |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101564289A (en) * | 2009-06-03 | 2009-10-28 | 南京航空航天大学 | Method for real-time error correction of neurosurgery navigation puncture path based on near infrared spectrum |
| US9603668B2 (en) * | 2014-07-02 | 2017-03-28 | Covidien Lp | Dynamic 3D lung map view for tool navigation inside the lung |
| CN105411679B (en) * | 2015-11-23 | 2017-07-14 | 中国科学院深圳先进技术研究院 | One kind punctures path planning method and device for correcting |
| CN108577965A (en) * | 2018-03-19 | 2018-09-28 | 艾瑞迈迪医疗科技（北京）有限公司 | A kind of courses of action method and device for planning |
| CN109124764B (en) * | 2018-09-29 | 2020-07-14 | 上海联影医疗科技有限公司 | Surgical guide device and surgical system |
-
2019
- 2019-04-11 CN CN201910288110.5A patent/CN109893247B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109893247A (en) | 2019-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109893247B (en) | Navigation and display method and device for operation path of surgical instrument | |
| RU2740259C2 (en) | Ultrasonic imaging sensor positioning | |
| US9842401B2 (en) | Segmentation apparatus for interactively segmenting blood vessels in angiographic image data | |
| JP5129480B2 (en) | System for performing three-dimensional reconstruction of tubular organ and method for operating blood vessel imaging device | |
| JP2011212301A (en) | Projection image generation apparatus and method, and program | |
| CN107019522B (en) | X-ray device and data carrier for providing image support to an operator | |
| JP6313024B2 (en) | Method and system for automatically determining a localizer within a scout image | |
| US9830737B2 (en) | Virtual endoscopic image generation device, method, and medium containing program | |
| US9808145B2 (en) | Virtual endoscopic image generation device, method, and medium containing program | |
| JPWO2013014868A1 (en) | Cutting simulation apparatus and cutting simulation program | |
| US10799100B2 (en) | Image processing device, method, and program | |
| JP2004057411A (en) | Method for preparing visible image for medical use | |
| KR101988531B1 (en) | Navigation system for liver disease using augmented reality technology and method for organ image display | |
| JP2013202312A (en) | Surgery support device and surgery support program | |
| US20170287211A1 (en) | Medical image processing apparatus, medical image processing method, and medical image processing system | |
| JPH09131339A (en) | Three-dimensional image processing device | |
| US10438368B2 (en) | Apparatus, method, and system for calculating diameters of three-dimensional medical imaging subject | |
| JP2001250133A (en) | Image display device | |
| EP3547255B1 (en) | Locating an opening of a body cavity | |
| CN110313991B (en) | Static virtual camera positioning | |
| CN115131304A (en) | Pulmonary nodule boundary sphere generation method, device and equipment | |
| CN117677358A (en) | Augmented reality system and method for stereoscopic projection and cross-referencing of intra-operative field X-ray fluoroscopy and C-arm computed tomography imaging | |
| WO2016131717A1 (en) | Corrected flow field estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |