CN117017488A - Puncture arm path planning method comprising non-autonomous motion compensation - Google Patents
Puncture arm path planning method comprising non-autonomous motion compensation Download PDFInfo
- Publication number
- CN117017488A CN117017488A CN202311302349.6A CN202311302349A CN117017488A CN 117017488 A CN117017488 A CN 117017488A CN 202311302349 A CN202311302349 A CN 202311302349A CN 117017488 A CN117017488 A CN 117017488A
- Authority
- CN
- China
- Prior art keywords
- puncture
- path
- focus image
- planning
- arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000004927 fusion Effects 0.000 claims abstract description 13
- 230000008859 change Effects 0.000 claims abstract description 12
- 238000013528 artificial neural network Methods 0.000 claims description 34
- 238000005457 optimization Methods 0.000 claims description 19
- 230000000007 visual effect Effects 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 7
- 230000035515 penetration Effects 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 6
- 208000012661 Dyskinesia Diseases 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 230000017311 musculoskeletal movement, spinal reflex action Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 4
- 102100033620 Calponin-1 Human genes 0.000 claims description 3
- 102100033591 Calponin-2 Human genes 0.000 claims description 3
- 101000945318 Homo sapiens Calponin-1 Proteins 0.000 claims description 3
- 101000945403 Homo sapiens Calponin-2 Proteins 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000011217 control strategy Methods 0.000 claims description 3
- 238000013016 damping Methods 0.000 claims description 3
- 238000013136 deep learning model Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 9
- 230000004888 barrier function Effects 0.000 description 12
- 230000000694 effects Effects 0.000 description 4
- 206010028980 Neoplasm Diseases 0.000 description 3
- 230000003902 lesion Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 208000015592 Involuntary movements Diseases 0.000 description 1
- 206010028851 Necrosis Diseases 0.000 description 1
- 238000002679 ablation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000000338 in vitro Methods 0.000 description 1
- 238000001727 in vivo Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/107—Visualisation of planned trajectories or target regions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The application discloses a puncture arm path planning method comprising non-autonomous motion compensation, which comprises the following steps: acquiring a first puncture focus image; according to the first puncture focus image, a first puncture path is obtained through a first path planning; acquiring a second puncture focus image and first puncture force data; obtaining a second puncture path through second path planning according to the second puncture focus image and the first puncture force data; and obtaining a third puncture path through path fusion according to the first puncture path and the second puncture path. The application realizes unification of path planning on vision and force sense, ensures compensation to take obstacle avoidance and force feedback into consideration, realizes that the path planning in operation keeps the minimum correction amount on the basis of the path planning before operation when compensating non-autonomous movement, keeps puncture stability, avoids the damage degree caused by path change, and can furthest inherit the advantages of the path planning before operation.
Description
Technical Field
The application relates to the technical field of puncture robots, in particular to a puncture arm path planning method comprising non-autonomous motion compensation.
Background
The existing robot system for minimally invasive ablation operation comprises an ultrasonic arm and two sets of pose separation mechanical arms, wherein ultrasonic means are carried out in vitro or in vivo through the ultrasonic arm to see a focus, the front ends of the two sets of pose separation mechanical arms are used for ablating needles to penetrate into a human body and penetrate into the focus, electrodes enter solid tumor tissues to the maximum extent and uniformly, anchor-shaped thin electrode wires are stretched out from the front ends of the ablating electrode needles and inserted into the tumor tissues, heat is generated by the ion oscillation friction of tissue cells of a lesion area through radio frequency output, and the tumor tissue lesion tissues are killed through the heated temperature to cause coagulation necrosis.
In the prior art, when physiological motion compensation is performed, a local dynamic programming method is mostly adopted to obtain a dynamic and locally available puncture path, and the local dynamic path is directly used for puncturing, so that global programming is separated, dynamic randomness is realized, the puncture path is usually caused to vary greatly, the puncture stability is not ensured, and therefore, effective compensation on physiological motion is not available, and the path programming effect is poor.
Disclosure of Invention
The application aims to provide a puncture arm path planning method comprising non-autonomous motion compensation, which aims to solve the technical problem that the path planning effect is poor due to the fact that effective compensation for physiological motion is lacking in the prior art.
In order to solve the technical problems, the application specifically provides the following technical scheme:
a puncture arm path planning method including non-autonomous motion compensation, comprising the steps of:
performing a first path planning on an action path of a puncture arm according to a first puncture focus image to obtain a first puncture path, wherein the first path planning is a puncture path global planning of static compensation non-self-movement before puncture operation;
acquiring a second puncture focus image and first puncture force data, wherein the second puncture focus image corresponds to focus image information in puncture, and the first puncture force data corresponds to force feedback attribute of puncture tissue to a puncture arm in puncture, and the focus image information in puncture comprises real-time information of non-autonomous movement of a puncture object;
performing a second path planning on a real-time action path of the puncture arm according to the second puncture focus image and the first puncture force data to obtain a second puncture path, wherein the second path planning is puncture path locality planning for compensating non-autonomous movement in real time in a puncture operation;
carrying out path fusion on the first puncture path and the second puncture path to obtain a third puncture path, wherein the path fusion is the unification of puncture path local planning and puncture path global planning so as to maintain puncture stability;
the focus image information is at least one of CT image information, ultrasonic image information and MRI image information.
As a preferred aspect of the present application, the first path planning includes:
marking puncture body sites and focus sites in the first puncture focus image;
calibrating the puncture site and the focus site as a first path starting point and a first path ending point respectively;
based on a first puncture focus image, a first path starting point and a first path ending point, obtaining a first puncture path for guiding a puncture arm to travel from a puncture body point to a focus point through a fast expansion random tree algorithm or an improved fast expansion random tree algorithm;
wherein the penetration site and the focus site are a path start point and a path end point of the first penetration path, respectively.
As a preferred embodiment of the present application, the second path planning includes:
performing real-time change detection of involuntary movement on the second puncture focus image, wherein,
when the second puncture focus image generates real-time variation of non-autonomous movement, then
Marking a local path in the first puncture path, which is positioned in the second puncture focus image;
calibrating a path starting point and a path node in the local path as a second path starting point and a second path ending point respectively;
obtaining a second puncture path which guides the puncture arm to travel from the second path starting point to the second path ending point through a heuristic path planning algorithm according to the second puncture focus image, the second puncture force data, the second path starting point and the second path ending point;
and when the real-time variation of the non-autonomous motion does not occur in the second puncture focus image, taking a local path in the second puncture focus image in the first puncture path as a second puncture path for guiding the puncture arm to travel from the starting point of the second path to the end point of the second path.
As a preferred embodiment of the present application, the detecting the real-time variation of the involuntary movement of the second puncture focus image includes:
respectively extracting rigid deformation characteristics in the second puncture focus image and the first puncture focus image;
comparing the similarity of the rigid deformation characteristic of the second puncture focus image with the rigid deformation characteristic of the first puncture focus image, wherein,
when the similarity of the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image is lower than a similarity threshold, the second puncture focus image is marked as the real-time variation of non-autonomous movement;
and when the similarity of the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image is higher than or equal to a similarity threshold value, calibrating the second puncture focus image as real-time variation without non-autonomous movement.
As a preferred embodiment of the present application, the obtaining the second puncture path according to the second puncture focus image, the second puncture force data, the second path starting point and the second path ending point through a heuristic path planning algorithm includes:
according to the second puncture force data, an impedance model is established through a force sense admittance control strategy, a force sense planning model of a second puncture path is obtained, an output item of the force sense planning model corresponds to the second puncture path associated with a force feedback attribute of puncture tissues to a puncture arm, and a model expression of the force sense planning model is as follows:the method comprises the steps of carrying out a first treatment on the surface of the Wherein p is a path node coordinate value corresponding to the second puncture force data, ++>For the first derivative of p>The second derivative of p, M is an inertia coefficient, fe is second puncture force data, K is a rigidity coefficient, and B is a damping coefficient; obtaining a visual planning model of a second puncture path through a rapid expansion random tree algorithm or an improved rapid expansion random tree algorithm according to the second puncture focus image, wherein an output item of the visual planning model corresponds to the second puncture path associated with real-time variation of non-autonomous movement;
taking the mean square error between the output item of the visual planning model and the output item of the force sense planning model as a first optimization target, and taking the mean square error between the output item of the visual planning model and a local path in a second puncture focus image in a first puncture path as a second optimization target;
taking the second puncture focus image as a solving space;
and carrying out minimized solution on the first optimization target and the second optimization target in a solution space to obtain a locally unified second puncture path in force sense and vision.
As a preferred embodiment of the present application, the path fusion includes:
and replacing a local path in the second puncture focus image in the first puncture path by the second puncture path, and taking the replaced first puncture path as a third puncture path.
As a preferable scheme of the application, the method further comprises the steps of convoluting and learning the second puncture path through a neural network to obtain a rapid planning model of the second puncture path;
the construction of the rapid planning model comprises the following steps:
intercepting a local image corresponding to the second puncture focus image from the first puncture focus image to obtain a third puncture focus image;
taking the third puncture focus image as an input item of the first neural network, and taking a local path in the second puncture focus image in the first puncture path as an output item of the first neural network;
taking the second puncture focus image as an input item of a second neural network, and taking a second puncture path as an output item of the second neural network;
taking the mean square error between the output term of the first neural network and the output term of the second neural network as a loss function;
learning and training the first neural network and the second neural network based on the loss function to obtain a rapid planning model of the second puncture path;
the model expression of the rapid planning model is as follows:；/>；
in the method, in the process of the application,lossMSE is a mean square error operator, which is a loss functionS1,S2) Is thatS1 andS2, S1 is the local path in the second puncture focus image in the first puncture path, S2 is the second puncture path, G2 is the second puncture focus image, G3 is the third puncture focus image,CNN1 is a first neural network,CNN2 is a second neural network. As a preferable mode of the application, the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image are extracted by a segmentation-registration deep learning model.
As a preferable scheme of the application, the second puncture force data, the second puncture focus image and the first puncture focus image achieve the unification of the global space coordinates of the first puncture path and the second puncture path through a Cartesian global space coordinate system.
As a preferable scheme of the application, the first puncture focus image is obtained by linearly weighting a plurality of sets of focus image information before puncture operation, and the second puncture focus image is obtained by linearly weighting a plurality of sets of focus image information in puncture operation.
Compared with the prior art, the application has the following beneficial effects:
according to the application, non-autonomous motion compensation is carried out on the preoperative path planning through the path planning in operation, the compensation is established on the image information containing the non-autonomous motion and the force feedback attribute, so that the unification of the path planning in vision and force sense is achieved, the compensation is ensured to be compatible with obstacle avoidance and force feedback, and the minimum correction quantity of the preoperative path planning on the basis of the preoperative path planning is realized when the non-autonomous motion is compensated, the puncture stability is kept, the damage degree caused by path change is avoided, and the advantage of preoperative path planning can be inherited to the greatest extent.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It will be apparent to those of ordinary skill in the art that the drawings in the following description are exemplary only and that other implementations can be obtained from the extensions of the drawings provided without inventive effort.
Fig. 1 is a flowchart of a puncture arm path planning method according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In puncture path planning, effective compensation for physiological motion is lacking, resulting in poor path planning accuracy. Therefore, the puncture arm path planning method comprising the non-autonomous motion compensation is provided, the non-autonomous motion compensation is realized, the path planning in the operation keeps the minimum correction amount on the basis of the path planning before the operation, the puncture stability is kept, the damage degree caused by path change is avoided, and the advantage of the path planning before the operation can be inherited to the greatest extent.
As shown in fig. 1, the present application provides a puncture arm path planning method including non-autonomous motion compensation, which includes the following steps:
performing a first path planning on an action path of a puncture arm according to a first puncture focus image to obtain a first puncture path, wherein the first path planning is a puncture path global planning of static compensation non-self-movement before puncture operation;
acquiring a second puncture focus image and first puncture force data, wherein the second puncture focus image corresponds to focus image information in puncture, and the first puncture force data corresponds to force feedback attribute of puncture tissue to a puncture arm in puncture, and the focus image information in puncture comprises real-time information of non-autonomous movement of a puncture object;
performing a second path planning on a real-time action path of the puncture arm according to the second puncture focus image and the first puncture force data to obtain a second puncture path, wherein the second path planning is puncture path locality planning for compensating non-autonomous movement in real time in a puncture operation;
carrying out path fusion on the first puncture path and the second puncture path to obtain a third puncture path, wherein the path fusion is the unification of puncture path local planning and puncture path global planning so as to maintain puncture stability;
the focus image information is at least one of CT image information, ultrasonic image information and MRI image information.
In order to compensate path planning deviation caused by non-autonomous movement of a puncture object in operation, the puncture path planning is performed in operation and fused to a puncture path planned before operation, so that path planning adjustment is performed according to the non-autonomous movement of the puncture object, and the accuracy of path planning is ensured.
When the non-autonomous motion compensation is performed, the application utilizes the focus image information in the puncture operation, wherein the focus image information in the puncture operation contains the real-time information of the non-autonomous motion of the puncture object, and independently utilizes the focus image information in the puncture operation to perform path planning, thereby obtaining a local planning path for realizing the vision obstacle avoidance, namely realizing the compensation of the vision barrier layer facing the non-autonomous motion.
When the non-autonomous motion compensation is performed, the real-time information of the non-autonomous motion of the puncture object is indirectly contained in the force feedback attribute of the puncture tissue to the puncture arm in the puncture operation, and the path planning is performed on the force feedback attribute of the puncture arm by independently using the puncture tissue in the puncture operation, so that a local planning path for realizing the force feedback obstacle avoidance is obtained, namely the compensation of the force sense obstacle avoidance layer to the non-autonomous motion is realized.
According to the application, when the vision barrier layer faces the compensation of the non-autonomous motion and the force sense barrier layer faces the compensation of the non-autonomous motion, the planned paths of the non-autonomous motion compensation of the two layers are fused and unified, so that the fusion of local features and local features during the non-autonomous motion compensation is realized, the uniformity of local planning is achieved, the finally obtained second puncture path is enabled to give consideration to the force sense barrier and the vision barrier, the barrier is avoided by multiple layers, and the compensation effect of the non-autonomous motion of the second puncture path is improved.
After the unification of the local planning of the non-autonomous motion compensation is realized, the unification of the local planning path (the second puncture path) and the preoperative global planning path (the first puncture path) is fused, so that the unification of the local to global fusion is realized, the least correction amount of the intraoperative path planning on the basis of the preoperative path planning is kept, the puncture stability is kept, the damage degree caused by the path change is avoided, and the advantage of the preoperative path planning can be inherited to the greatest extent.
In order to ensure the planning calculation efficiency and the searching speed, the application adopts a fast expanding random tree algorithm to plan the first puncture path, and in order to ensure that the first puncture path obtained by global planning has the advantages of shortest distance and minimum damage, the application utilizes an improved fast expanding random tree algorithm to plan, and the method comprises the following specific steps:
a first path plan, comprising:
marking puncture body sites and focus sites in the first puncture focus image;
marking penetration body sites and focus sites as a first path starting point and a first path ending point respectively;
based on the first puncture focus image, the first path starting point and the first path ending point, a first puncture path for guiding the puncture arm to travel from the puncture body point to the focus point is obtained through a fast expansion random tree algorithm or an improved fast expansion random tree algorithm;
wherein the penetration site and the focus site are a path start point and a path end point of the first penetration path, respectively.
The application relates to non-autonomous movement fixation information or static information contained in preoperative focus information, when the non-autonomous movement information fluctuates in operation, non-autonomous movement compensation is needed, a path is locally planned, otherwise, a first puncture path is used, the non-autonomous movement information fluctuates in operation, non-autonomous movement compensation is carried out, and the process of the locally planned path is as follows:
a second path plan, comprising:
performing real-time change detection of involuntary movement on the second puncture focus image, wherein,
when the second puncture focus image generates real-time variation of non-autonomous movement, then
Marking a local path in the first puncture path, which is positioned in the second puncture focus image;
calibrating a path starting point and a path node in the local path as a second path starting point and a second path ending point respectively;
obtaining a second puncture path which guides the puncture arm to travel from the second path starting point to the second path ending point through a heuristic path planning algorithm according to the second puncture focus image, the second puncture force data, the second path starting point and the second path ending point;
and when the real-time variation of the non-autonomous motion does not occur in the second puncture focus image, taking a local path in the second puncture focus image in the first puncture path as a second puncture path for guiding the puncture arm to travel from the starting point of the second path to the end point of the second path.
Performing real-time change detection of involuntary movements on the second puncture focus image, comprising:
respectively extracting rigid deformation characteristics in the second puncture focus image and the first puncture focus image;
comparing the similarity of the rigid deformation characteristic of the second puncture focus image with the rigid deformation characteristic of the first puncture focus image, wherein,
when the similarity of the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image is lower than a similarity threshold, the second puncture focus image is marked as the real-time variation of non-autonomous movement;
and when the similarity of the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image is higher than or equal to a similarity threshold value, calibrating the second puncture focus image as real-time variation without non-autonomous movement.
According to the application, when the vision barrier layer faces the compensation of non-autonomous motion and the force sense barrier layer faces the compensation of non-autonomous motion, the planned paths of the non-autonomous motion compensation of the two layers are fused and unified, so that the fusion of local features and local features during the non-autonomous motion compensation is realized, the uniformity of local planning is achieved, the finally obtained second puncture path is enabled to give consideration to the force sense barrier and the vision barrier, the barrier is avoided by multiple layers, and the compensation effect of the non-autonomous motion of the second puncture path is improved, and the method comprises the following specific steps:
obtaining a second puncture path through a heuristic path planning algorithm according to the second puncture focus image, the second puncture force data, the second path starting point and the second path ending point, wherein the method comprises the following steps:
according to the second puncture force data, an impedance model is established through a force sense admittance control strategy, a force sense planning model of a second puncture path is obtained, an output item of the force sense planning model corresponds to the second puncture path associated with a force feedback attribute of puncture tissues to a puncture arm, and a model expression of the force sense planning model is as follows:the method comprises the steps of carrying out a first treatment on the surface of the Wherein p is a path node coordinate value corresponding to the second puncture force data, ++>For the first derivative of p>The second derivative of p, M is an inertia coefficient, fe is second puncture force data, K is a rigidity coefficient, and B is a damping coefficient;
when the non-autonomous motion compensation is performed, the real-time information of the non-autonomous motion of the puncture object is indirectly contained in the force feedback attribute of the puncture tissue to the puncture arm in the puncture operation, and the path planning is performed on the force feedback attribute of the puncture arm by independently using the puncture tissue in the puncture operation, so that a local planning path for realizing the force feedback obstacle avoidance is obtained, namely the compensation of the force sense obstacle avoidance layer to the non-autonomous motion is realized.
Obtaining a visual planning model of the second puncture path through a rapid expansion random tree algorithm or an improved rapid expansion random tree algorithm according to the second puncture focus image, wherein an output item of the visual planning model corresponds to the second puncture path associated with real-time variation of non-autonomous movement;
when the non-autonomous motion compensation is performed, the application utilizes the focus image information in the puncture operation, wherein the focus image information in the puncture operation contains the real-time information of the non-autonomous motion of the puncture object, and independently utilizes the focus image information in the puncture operation to perform path planning, thereby obtaining a local planning path for realizing the vision obstacle avoidance, namely realizing the compensation of the vision barrier layer facing the non-autonomous motion.
Taking the mean square error between the output item of the visual planning model and the output item of the force sense planning model as a first optimization target, and taking the mean square error between the output item of the visual planning model and a local path in the second puncture focus image in the first puncture path as a second optimization target;
taking the second puncture focus image as a solving space;
and carrying out minimized solution on the first optimization target and the second optimization target in a solution space to obtain a locally unified second puncture path in force sense and vision.
Path fusion, comprising:
and replacing a local path in the second puncture focus image in the first puncture path by the second puncture path, and taking the replaced first puncture path as a third puncture path.
According to the application, the first optimization target is set as the mean square error between the output item of the vision planning model and the output item of the force sense planning model, namely, the first optimization target measures the difference between the output item of the vision planning model and the output item of the force sense planning model, the first optimization target is minimized, so that the difference between the second puncture path obtained by the vision planning model and the second puncture path obtained by the force sense planning model is minimized, the second puncture path obtained by the solution is optimized finally, local unification is achieved at the visual obstacle avoidance layer and the force feedback obstacle avoidance layer, and the robustness of the second puncture path is increased.
According to the application, the second optimization target is set as the mean square error between the output item of the vision planning model and the local path in the second puncture focus image in the first puncture path, namely, the second optimization target measures the difference between the output item of the vision planning model and the local path in the second puncture focus image in the first puncture path, the second optimization target is minimized, so that the difference between the second puncture path obtained by the vision planning model and the local path in the second puncture focus image in the first puncture path is minimized, the second puncture path obtained by finally optimizing and solving is transferred from the local path in the second puncture focus image to the second puncture path in the first puncture path to travel, the minimum mechanical correction amount is generated, the puncture stability is kept, the damage degree caused by path change is avoided, and the advantage of preoperative path planning (the shortest distance and the minimum damage) can be inherited to the greatest extent.
In order to further enhance the planning efficiency of the second puncture path, the application constructs a rapid planning model by utilizing a neural network, so that the planned second puncture path integrates the advantages of locally unifying at the visual obstacle avoidance layer and the force feedback obstacle avoidance layer, increasing the robustness of the second puncture path, generating the minimum mechanical correction, keeping the puncture stability and avoiding the damage degree caused by path change, and the application comprises the following specific steps:
performing convolution learning on the second puncture path through a neural network to obtain a rapid planning model of the second puncture path;
the construction of the rapid planning model comprises the following steps:
intercepting a local image corresponding to the second puncture focus image from the first puncture focus image to obtain a third puncture focus image;
taking the third puncture focus image as an input item of the first neural network, and taking a local path in the second puncture focus image in the first puncture path as an output item of the first neural network;
taking the second puncture focus image as an input item of a second neural network, and taking a second puncture path as an output item of the second neural network;
taking the mean square error between the output term of the first neural network and the output term of the second neural network as a loss function;
learning and training the first neural network and the second neural network based on the loss function to obtain a rapid planning model of the second puncture path;
model expression of a fast planning modelThe formula is:；/>；
in the method, in the process of the application,lossMSE is a mean square error operator, which is a loss functionS1,S2) Is thatS1 andS2, S1 is the local path in the second puncture focus image in the first puncture path, S2 is the second puncture path, G2 is the second puncture focus image, G3 is the third puncture focus image,CNN1 is a first neural network,CNN2 is a second neural network.
The dual-network comparison structure of the twin network is utilized, the difference degree of the second puncture path and the first puncture path is used as a damage function, the measured second puncture path is locally close to the first puncture path, and the training sample is derived from the second puncture path obtained in the optimization process, so that the rapid planning model is integrated with the advantages of locally unifying the visual obstacle avoidance layer and the force feedback obstacle avoidance layer, the robustness of the second puncture path is improved, the minimum mechanical correction quantity is generated, the puncture stability is kept, and the damage degree caused by path change is avoided.
The rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image are extracted by a segmentation-registration deep learning model.
The second puncture force data, the second puncture focus image and the first puncture focus image are unified with the global space coordinates of the first puncture path and the second puncture path through a Cartesian global space coordinate system.
The first puncture focus image is obtained by linearly weighting a plurality of groups of focus image information before puncture operation, and the second puncture focus image is obtained by linearly weighting a plurality of groups of focus image information in puncture operation.
According to the application, non-autonomous motion compensation is carried out on the preoperative path planning through the path planning in operation, the compensation is established on the image information containing the non-autonomous motion and the force feedback attribute, so that the unification of the path planning in vision and force sense is achieved, the compensation is ensured to be compatible with obstacle avoidance and force feedback, and the minimum correction quantity of the preoperative path planning on the basis of the preoperative path planning is realized when the non-autonomous motion is compensated, the puncture stability is kept, the damage degree caused by path change is avoided, and the advantage of preoperative path planning can be inherited to the greatest extent.
The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application, the scope of which is defined by the claims. Various modifications and equivalent arrangements of this application will occur to those skilled in the art, and are intended to be within the spirit and scope of the application.
Claims (10)
1. A method of planning a path of a puncture arm comprising non-autonomous motion compensation, comprising the steps of:
acquiring a first puncture focus image, wherein the first puncture focus image corresponds to focus image information before puncture operation, and the focus image information before puncture operation comprises stop motion information of non-autonomous movement of a puncture object;
performing a first path planning on an action path of a puncture arm according to a first puncture focus image to obtain a first puncture path, wherein the first path planning is a puncture path global planning of static compensation non-self-movement before puncture operation;
acquiring a second puncture focus image and first puncture force data, wherein the second puncture focus image corresponds to focus image information in puncture, and the first puncture force data corresponds to force feedback attribute of puncture tissue to a puncture arm in puncture, and the focus image information in puncture comprises real-time information of non-autonomous movement of a puncture object;
performing a second path planning on a real-time action path of the puncture arm according to the second puncture focus image and the first puncture force data to obtain a second puncture path, wherein the second path planning is puncture path locality planning for compensating non-autonomous movement in real time in a puncture operation;
carrying out path fusion on the first puncture path and the second puncture path to obtain a third puncture path, wherein the path fusion is the unification of puncture path local planning and puncture path global planning so as to maintain puncture stability;
the focus image information is at least one of CT image information, ultrasonic image information and MRI image information.
2. A puncture arm path planning method comprising non-autonomous motion compensation according to claim 1, characterized in that: the first path plan includes:
marking puncture body sites and focus sites in the first puncture focus image;
calibrating the puncture site and the focus site as a first path starting point and a first path ending point respectively;
based on a first puncture focus image, a first path starting point and a first path ending point, obtaining a first puncture path for guiding a puncture arm to travel from a puncture body point to a focus point through a fast expansion random tree algorithm or an improved fast expansion random tree algorithm;
wherein the penetration site and the focus site are a path start point and a path end point of the first penetration path, respectively.
3. A puncture arm path planning method comprising non-autonomous motion compensation according to claim 2, characterized in that: the second path planning includes:
performing real-time change detection of involuntary movement on the second puncture focus image, wherein,
when the second puncture focus image generates real-time variation of non-autonomous movement, marking a local path in the second puncture focus image in the first puncture path;
calibrating a path starting point and a path node in the local path as a second path starting point and a second path ending point respectively;
obtaining a second puncture path which guides the puncture arm to travel from the second path starting point to the second path ending point through a heuristic path planning algorithm according to the second puncture focus image, the second puncture force data, the second path starting point and the second path ending point;
and when the real-time variation of the non-autonomous motion does not occur in the second puncture focus image, taking a local path in the second puncture focus image in the first puncture path as a second puncture path for guiding the puncture arm to travel from the starting point of the second path to the end point of the second path.
4. A puncture arm path planning method comprising non-autonomous motion compensation as set out in claim 3, wherein: the real-time variation detection of the non-autonomous movement is performed on the second puncture focus image, which comprises the following steps:
respectively extracting rigid deformation characteristics in the second puncture focus image and the first puncture focus image;
comparing the similarity of the rigid deformation characteristic of the second puncture focus image with the rigid deformation characteristic of the first puncture focus image, wherein,
when the similarity of the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image is lower than a similarity threshold, the second puncture focus image is marked as the real-time variation of non-autonomous movement;
and when the similarity of the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image is higher than or equal to a similarity threshold value, calibrating the second puncture focus image as real-time variation without non-autonomous movement.
5. A puncture arm path planning method comprising non-autonomous motion compensation as set out in claim 4, wherein: the obtaining the second puncture path through a heuristic path planning algorithm according to the second puncture focus image, the second puncture force data, the second path starting point and the second path ending point comprises the following steps:
according to the second puncture force data, an impedance model is established through a force sense admittance control strategy to obtainA force sense planning model of a second puncture path, an output term of the force sense planning model corresponding to the second puncture path associated with a force feedback attribute of a puncture tissue to a puncture arm, a model expression of the force sense planning model being:the method comprises the steps of carrying out a first treatment on the surface of the Wherein p is a path node coordinate value corresponding to the second puncture force data, ++>For the first derivative of p>The second derivative of p, M is an inertia coefficient, fe is second puncture force data, K is a rigidity coefficient, and B is a damping coefficient;
obtaining a visual planning model of a second puncture path through a rapid expansion random tree algorithm or an improved rapid expansion random tree algorithm according to the second puncture focus image, wherein an output item of the visual planning model corresponds to the second puncture path associated with real-time variation of non-autonomous movement;
taking the mean square error between the output item of the visual planning model and the output item of the force sense planning model as a first optimization target, and taking the mean square error between the output item of the visual planning model and a local path in a second puncture focus image in a first puncture path as a second optimization target;
taking the second puncture focus image as a solving space;
and carrying out minimized solution on the first optimization target and the second optimization target in a solution space to obtain a locally unified second puncture path in force sense and vision.
6. A puncture arm path planning method comprising non-autonomous motion compensation as set forth in claim 5, wherein: the path fusion comprises:
and replacing a local path in the second puncture focus image in the first puncture path by the second puncture path, and taking the replaced first puncture path as a third puncture path.
7. A puncture arm path planning method comprising non-autonomous motion compensation as set out in claim 6, wherein: the method further comprises the steps of performing convolution learning on the second puncture path through a neural network to obtain a rapid planning model of the second puncture path;
the construction of the rapid planning model comprises the following steps:
intercepting a local image corresponding to the second puncture focus image from the first puncture focus image to obtain a third puncture focus image;
taking the third puncture focus image as an input item of the first neural network, and taking a local path in the second puncture focus image in the first puncture path as an output item of the first neural network;
taking the second puncture focus image as an input item of a second neural network, and taking a second puncture path as an output item of the second neural network;
taking the mean square error between the output term of the first neural network and the output term of the second neural network as a loss function;
learning and training the first neural network and the second neural network based on the loss function to obtain a rapid planning model of the second puncture path;
the model expression of the rapid planning model is as follows:；/>the method comprises the steps of carrying out a first treatment on the surface of the In the method, in the process of the application,lossMSE is a mean square error operator, which is a loss functionS1,S2) Is thatS1 andS2, S1 is the local path in the second puncture focus image in the first puncture path, S2 is the second puncture path, G2 is the second puncture focus image, G3 is the third puncture focus image,CNN1 is a first neural network,CNN2 is a second neural network.
8. A puncture arm path planning method comprising non-autonomous motion compensation as set out in claim 4, wherein: the rigid deformation characteristic of the second puncture focus image and the rigid deformation characteristic of the first puncture focus image are extracted by a segmentation-registration deep learning model.
9. A puncture arm path planning method comprising non-autonomous motion compensation according to claim 1, characterized in that: the second puncture force data, the second puncture focus image and the first puncture focus image are unified with the global space coordinates of the first puncture path and the second puncture path through a Cartesian global space coordinate system.
10. A puncture arm path planning method comprising non-autonomous motion compensation according to claim 1, characterized in that: the first puncture focus image is obtained by linearly weighting a plurality of groups of focus image information before puncture operation, and the second puncture focus image is obtained by linearly weighting a plurality of groups of focus image information in puncture operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311302349.6A CN117017488B (en) | 2023-10-10 | 2023-10-10 | Puncture arm path planning method comprising non-autonomous motion compensation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311302349.6A CN117017488B (en) | 2023-10-10 | 2023-10-10 | Puncture arm path planning method comprising non-autonomous motion compensation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117017488A true CN117017488A (en) | 2023-11-10 |
| CN117017488B CN117017488B (en) | 2024-01-09 |
Family
ID=88628573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311302349.6A Active CN117017488B (en) | 2023-10-10 | 2023-10-10 | Puncture arm path planning method comprising non-autonomous motion compensation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117017488B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104248471A (en) * | 2013-06-27 | 2014-12-31 | 中国科学院沈阳自动化研究所 | Robot-assisted oblique-tip flexible needle puncture system and method |
| CN106725762A (en) * | 2016-12-30 | 2017-05-31 | 西安中科微光影像技术有限公司 | A kind of automatic puncturing method |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| US20170245759A1 (en) * | 2016-02-25 | 2017-08-31 | Samsung Electronics Co., Ltd. | Image-analysis for assessing heart failure |
| CN109405828A (en) * | 2018-07-30 | 2019-03-01 | 浙江工业大学 | Mobile robot global optimum path planning method based on LTL-A* algorithm |
| CN110338907A (en) * | 2018-04-05 | 2019-10-18 | 云南师范大学 | A kind of haptic navigation system for medical image guidance operation |
| CN113679473A (en) * | 2021-08-23 | 2021-11-23 | 北京航空航天大学 | Human-computer cooperative force feedback ventricular puncture robot device |
| US20210390716A1 (en) * | 2020-06-11 | 2021-12-16 | GE Precision Healthcare LLC | Image registration method and model training method thereof |
| WO2022192959A1 (en) * | 2021-03-18 | 2022-09-22 | Lions Eye Institute Limited | System and method for non-invasive determination of intracranial pressure |
| CN115826586A (en) * | 2023-02-14 | 2023-03-21 | 泉州装备制造研究所 | Path planning method and system fusing global algorithm and local algorithm |
| CN116330347A (en) * | 2023-02-24 | 2023-06-27 | 上海莱陆科技有限公司 | Full-automatic robot testing method based on visual analysis |
| CN116465425A (en) * | 2023-05-10 | 2023-07-21 | 合肥亿图网络科技有限公司 | Heuristic path planning method for local optimization and bidirectional calculation |
-
2023
- 2023-10-10 CN CN202311302349.6A patent/CN117017488B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104248471A (en) * | 2013-06-27 | 2014-12-31 | 中国科学院沈阳自动化研究所 | Robot-assisted oblique-tip flexible needle puncture system and method |
| US20170245759A1 (en) * | 2016-02-25 | 2017-08-31 | Samsung Electronics Co., Ltd. | Image-analysis for assessing heart failure |
| CN106781941A (en) * | 2016-11-24 | 2017-05-31 | 北京理工大学 | A kind of method and its system for simulating microtrauma puncture operation |
| CN106725762A (en) * | 2016-12-30 | 2017-05-31 | 西安中科微光影像技术有限公司 | A kind of automatic puncturing method |
| CN110338907A (en) * | 2018-04-05 | 2019-10-18 | 云南师范大学 | A kind of haptic navigation system for medical image guidance operation |
| CN109405828A (en) * | 2018-07-30 | 2019-03-01 | 浙江工业大学 | Mobile robot global optimum path planning method based on LTL-A* algorithm |
| US20210390716A1 (en) * | 2020-06-11 | 2021-12-16 | GE Precision Healthcare LLC | Image registration method and model training method thereof |
| WO2022192959A1 (en) * | 2021-03-18 | 2022-09-22 | Lions Eye Institute Limited | System and method for non-invasive determination of intracranial pressure |
| CN113679473A (en) * | 2021-08-23 | 2021-11-23 | 北京航空航天大学 | Human-computer cooperative force feedback ventricular puncture robot device |
| CN115826586A (en) * | 2023-02-14 | 2023-03-21 | 泉州装备制造研究所 | Path planning method and system fusing global algorithm and local algorithm |
| CN116330347A (en) * | 2023-02-24 | 2023-06-27 | 上海莱陆科技有限公司 | Full-automatic robot testing method based on visual analysis |
| CN116465425A (en) * | 2023-05-10 | 2023-07-21 | 合肥亿图网络科技有限公司 | Heuristic path planning method for local optimization and bidirectional calculation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117017488B (en) | 2024-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11571258B2 (en) | Path planning method with artificial potential field based on obstacle classification and medical system for steering flexible needle | |
| US11523749B2 (en) | Systems and methods for tracking an intrabody catheter | |
| US11064930B2 (en) | Automatic tracking and adjustment of the view angle during catheter ablation treatment | |
| US20210085387A1 (en) | Guiding cardiac ablation using machine learning (ml) | |
| CN101558993B (en) | Draw the device of abnormal electrical activity in live subject heart | |
| US8942786B2 (en) | Tracking using field mapping | |
| CN101156774B (en) | Mapping of complex fractionated atrial electrogram | |
| US20190374283A1 (en) | Medical path navigation method, planning method and system | |
| CN103747756A (en) | Integrated ablation and mapping system | |
| CN108324245A (en) | The efficiency for repeating to melt is promoted by merging current and previous scaling graph | |
| Winkle et al. | Ablation of atypical atrial flutters using ultra high density-activation sequence mapping | |
| CN111870344B (en) | Preoperative navigation method, system and terminal equipment | |
| CN106510845A (en) | Medical treatment path navigation method and system | |
| CN117017488B (en) | Puncture arm path planning method comprising non-autonomous motion compensation | |
| Xiong et al. | Path planning for flexible needle insertion system based on improved rapidly-exploring random tree algorithm | |
| CN113413214A (en) | Surgical robot manpower feedback method and device based on mixed reality guidance | |
| US20230404676A1 (en) | Visualizing a quality index indicative of ablation stability at ablation site | |
| US20230042941A1 (en) | Graphical user interface template for reducing setup time of electrophysiological procedures | |
| RU2779871C2 (en) | Guided heart ablation using machine learning (ml) | |
| JP2019034150A (en) | System and method of managing ecg data for user defined map | |
| US20240130780A1 (en) | Position tracking for pulsed field ablation | |
| Jiang et al. | The potential field-based trajectory planning for needle insertion in a soft-tissue model | |
| US20230404677A1 (en) | Applying ablation signals to both sides of tissue | |
| Jiang et al. | Correction Control of Needle Deflection Based on Kernel Extreme Learning Machine | |
| KR102160198B1 (en) | Apparatus and method for generating three dimensional voltage difference atrium model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |