CN112022294B - Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance - Google Patents

Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance Download PDF

Info

Publication number
CN112022294B
CN112022294B CN202010853764.0A CN202010853764A CN112022294B CN 112022294 B CN112022294 B CN 112022294B CN 202010853764 A CN202010853764 A CN 202010853764A CN 112022294 B CN112022294 B CN 112022294B
Authority
CN
China
Prior art keywords
puncture
boundary
blood vessel
track
trajectory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010853764.0A
Other languages
Chinese (zh)
Other versions
CN112022294A (en
Inventor
齐鹏
陈禹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongji University
Original Assignee
Tongji University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongji University filed Critical Tongji University
Priority to CN202010853764.0A priority Critical patent/CN112022294B/en
Publication of CN112022294A publication Critical patent/CN112022294A/en
Application granted granted Critical
Publication of CN112022294B publication Critical patent/CN112022294B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/34—Trocars; Puncturing needles
    • A61B17/3403—Needle locating or guiding means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/34—Trocars; Puncturing needles
    • A61B17/3403—Needle locating or guiding means
    • A61B2017/3413—Needle locating or guiding means guided by ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/107—Visualisation of planned trajectories or target regions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2063—Acoustic tracking systems, e.g. using ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Robotics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Pathology (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

The invention relates to a venipuncture robot puncture track planning method based on ultrasonic image guidance, wherein the venipuncture robot controls a puncture piece to puncture a blood vessel, and the method comprises the following steps: 1) acquiring a blood vessel ultrasonic image of a certain part of an object to be punctured, and acquiring a blood vessel segmentation map from the blood vessel ultrasonic image; 2) according to a preset first puncture track of a puncture piece in a blood vessel, a puncture track boundary is defined; 3) dividing a safety region from the blood vessel segmentation graph to obtain a blood vessel boundary; 4) matching the puncture track boundary with the blood vessel boundary, and if the blood vessel boundary completely contains the puncture track boundary, generating a second puncture track according to the matching position and the first puncture track; otherwise, the part of the object to be punctured is replaced, and the method is executed again. Compared with the prior art, the invention plans the track, can realize automatic puncture, has high automation degree and ensures the safety of the puncture track.

Description

Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance
Technical Field
The invention relates to the field of operation of a venipuncture robot, in particular to a method for planning an operation track of the venipuncture robot based on ultrasonic image guidance.
Background
The flow route of the existing ultrasound-assisted upper limb venipuncture technology is as follows: the vein is ultrasonically probed, puncture is carried out, the track of the puncture needle is observed in real time, and the puncture is completed, basically, medical staff manually completes the puncture under the assistance of ultrasonic equipment, most related inventions are ultrasonic guided venipuncture, and the degree of automation is low.
Patent CN111339828A discloses a vein imaging identification method based on infrared image and ultrasonic doppler combination, which includes: obtaining a vein image of a part to be identified of a patient, and preprocessing the vein image. The method helps medical staff to better identify veins through vein development, but does not plan the veins and puncture needle tracks, is not oriented to a vein puncture robot, and needs a doctor to manually control the puncture needle.
Patent CN111134727A provides a puncture guiding system for vein and artery identification based on a neural network, which can display the relative positions of a puncture needle and a blood vessel, but does not involve the identification and segmentation of the subdivision structure of the vein, nor the planning of the track of the puncture needle, and is liable to cause accidents such as the puncture needle touching the venous valve and scratching the blood vessel wall.
Disclosure of Invention
The invention aims to overcome the defect that a venipuncture robot in the prior art does not relate to puncture trajectory planning, and provides an operation trajectory planning method of the venipuncture robot based on ultrasonic image guidance.
The purpose of the invention can be realized by the following technical scheme:
a venipuncture robot puncture trajectory planning method based on ultrasonic image guidance is disclosed, wherein the venipuncture robot controls a puncture piece to perform vascular puncture, and the method comprises the following steps:
a blood vessel segmentation step: acquiring a blood vessel ultrasonic image of a certain part of an object to be punctured, and acquiring a blood vessel segmentation map from the blood vessel ultrasonic image;
a puncture track boundary defining step: according to a preset first puncture track of the puncture piece in the blood vessel, the puncture track limit of the puncture piece in the blood vessel is defined;
a blood vessel boundary defining step: dividing a safety region from the blood vessel segmentation graph to obtain a blood vessel boundary;
boundary matching and puncturing: matching the puncture track limit with the blood vessel limit, and if the blood vessel limit completely contains the puncture track limit, generating a second puncture track of the venipuncture robot according to the matching position of the puncture track limit and the first puncture track; otherwise, the part of the object to be punctured is replaced, and the method is executed again.
Further, in the blood vessel segmentation step, a U-Net frame is adopted to obtain the blood vessel segmentation map from the blood vessel ultrasonic image.
Further, in the step of defining the puncture track boundary, the defining of the puncture track boundary specifically includes expanding the first puncture track outwards to obtain the puncture track boundary.
Further, the extension range of the tail of the first puncture track is larger than that of other parts.
Further, in the step of defining the puncture track boundary, the puncture track boundary is represented by a rectangle, and a calculation expression of the height and the width of the puncture track boundary is as follows:
h＝αdv
w＝βh＝αβdv
where h is the height of the puncture trajectory boundary, α is a first scaling factor, dvAnd the diameter of the blood vessel cavity in the blood vessel segmentation map, w is the width of the puncture track boundary, and beta is a second scaling factor.
Further, in the blood vessel delineation step, the safety region is narrowed according to an image recognition error.
Further, in the blood vessel demarcation step, the safety zone is narrowed according to image recognition errors and the motion and deformation of veins.
Further, the reduction calculation expression of the safety area is as follows:
Figure BDA0002645679950000021
Figure BDA0002645679950000022
in the formula i0For the first coordinate point on the vessel boundary corresponding to the safety region before shrinking, q is the first coordinate point in the safety region after shrinking0Corresponding coordinate point, i1To narrow a second coordinate point, i, on the vessel boundary corresponding to the previous safety zone2To narrow down a third coordinate point, x, on the vessel boundary corresponding to the previous safety zonei0Is i0Abscissa of (a), yi0Is i0Ordinate of (a), xi1Is i1Abscissa of (a), yi1Is i1Ordinate of (a), xi2Is i2Abscissa of (a), yi2Is i2The ordinate of (c).
Further, in the boundary matching and puncturing step, the matching of the puncturing trajectory boundary with the blood vessel boundary specifically includes the following steps:
s401: obtaining a blood vessel boundary diagram according to the blood vessel boundary delimiting step, and converting the blood vessel boundary diagram into a blood vessel boundary bitmap;
s402: dividing an ROI (region of interest) region in the blood vessel boundary bitmap, and removing an unsafe region;
s403: and taking the puncture track boundary as a sliding window, sliding in the ROI area, and searching the sliding window position of the puncture track boundary completely surrounded by the ROI area as a matching position.
Further, in the boundary matching and puncturing step, a second puncturing track of the venipuncture robot is generated according to the matching position of the puncturing track boundary and the first puncturing track, specifically,
establishing a first space Cartesian coordinate system containing a punctured object part corresponding to the blood vessel ultrasonic image, and acquiring coordinates of the puncturing track boundary at each vertex of the first space Cartesian coordinate system and coordinates of a first puncturing track endpoint according to the matching position of the puncturing track boundary; and constructing a space curve, fitting the space curve with the first puncture track, and generating a second puncture track of the venipuncture robot.
Compared with the prior art, the invention has the following advantages:
(1) according to the invention, the puncture track limit of the puncture needle in the blood vessel and the blood vessel limit of the blood vessel are judged, the puncture track limit and the blood vessel limit are matched, and the matching position completely surrounding the puncture track limit in the blood vessel limit is selected, so that the puncture track planning of the venipuncture robot is carried out, the automatic puncture can be realized, the automation degree is high, and the safety of the puncture track is ensured;
(2) the invention utilizes the ultrasonic image, is convenient for segmenting the subdivision structure of the blood vessel and demarcating the perforable area of the puncture needle in the blood vessel;
(3) the invention adopts the U-Net frame to segment the blood vessel ultrasonic image, and the segmentation under the U-Net frame is insensitive to noise, has less parameters and high speed, and is suitable for the segmentation of the ultrasonic image;
(4) the invention considers the possible error of the puncture needle ultrasonic image identification, the puncture track is expanded outwards for a certain distance, and the error of the needle point is larger than that of other parts of the puncture needle, so that the expansion of the tail end of the puncture track is more, and the obtained puncture track boundary is safer and more reliable;
(5) according to the invention, different safety margin requirements are considered, image recognition errors and possible motion and deformation of veins are considered, and the safety region is divided, so that the obtained blood vessel boundary is safer and more reliable.
Drawings
FIG. 1 is an overall flow chart of the present invention;
FIG. 2 is a diagram illustrating a method for implementing step S1 according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a method for implementing step S2 according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a method for implementing step S3 according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating a method for implementing step S4 according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of the puncture needle trajectory location coordinates in step S6 according to the embodiment of the present invention;
fig. 7 is a schematic diagram of the motion trace of the puncture needle in step S6 according to the embodiment of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
Example 1
As shown in fig. 1, the embodiment provides a method for planning a puncture track of a venipuncture robot based on ultrasound image guidance, which is directed at an automatic venipuncture robot to help the robot to plan a puncture track, so that a mechanical arm operates a puncture needle according to a predetermined track to realize automatic puncture, and the venipuncture robot controls a puncture piece to perform blood vessel puncture.
The method comprises the steps of firstly, segmenting an ultrasonic image blood vessel by adopting a U-Net network framework; then, defining a puncture track boundary and a blood vessel boundary according to the segmentation result; then, matching the boundary and judging the puncture feasibility; finally, puncturing with a preset track is carried out, and the method specifically comprises the following steps:
blood vessel segmentation step S1: obtaining a blood vessel ultrasonic image of a certain part of an object to be punctured, and obtaining a blood vessel segmentation map from the blood vessel ultrasonic image. Obtaining a vessel segmentation map requires segmenting the vessel and the internal structure of the vessel, and the structures to be considered include: vessel lumen, vessel wall, venous sinus, etc.
In this embodiment, a U-Net frame is used to obtain a vessel segmentation map from a vessel ultrasound image.
Puncture trajectory delimiting step S2: according to a preset first puncture track of the puncture member in the blood vessel, the puncture track limit of the puncture member in the blood vessel is defined. Based on the blood vessel segmentation graph, the puncture track of the preset action needs to be subjected to boundary fine adjustment by considering the thickness of the blood vessel wall and the width of the blood vessel cavity.
The defining of the puncture track boundary specifically includes extending the first puncture track outwards to obtain the puncture track boundary. The extension range of the tail part of the first puncture track is larger than that of other parts.
The puncture track boundary is represented by a rectangle, and the calculation expression of the height and the width of the puncture track boundary is as follows:
h＝αdv
w＝βh＝αβdv
where h is the height of the puncture trajectory boundary, α is a first scaling factor, dvIs the vessel lumen diameter in the vessel segmentation map, w is the width of the puncture trajectory boundary, and β is the second scaling factor.
Blood vessel delimiting step S3: dividing a safety region from the blood vessel segmentation graph to obtain a blood vessel boundary;
and reducing the safety region according to the image recognition error and the motion and deformation of the vein.
The reduction calculation expression of the safety area is as follows:
Figure BDA0002645679950000051
Figure BDA0002645679950000052
in the formula i0For the first coordinate point on the vessel boundary corresponding to the safety region before shrinking, q is the first coordinate point in the safety region after shrinking0Corresponding coordinate point, i1To narrow a second coordinate point, i, on the vessel boundary corresponding to the previous safety zone2To narrow down a third coordinate point, x, on the vessel boundary corresponding to the previous safety zonei0Is i0Abscissa of (a), yi0Is i0Ordinate of (a), xi1Is i1Abscissa of (a), yi1Is i1Ordinate of (a), xi2Is i2Abscissa of (a), yi2Is i2The ordinate of (c).
The boundary matching and puncturing step comprises the following steps:
limit matching step S4: and matching the puncture track limit with the blood vessel limit, if the blood vessel limit completely contains the puncture track limit, executing step S6, otherwise executing step S5.
Matching the puncture trajectory boundary with the vessel boundary specifically comprises the following steps:
s401: obtaining a blood vessel boundary diagram according to the blood vessel boundary delimiting step, and converting the blood vessel boundary diagram into a blood vessel boundary bitmap;
s402: dividing ROI regions in a blood vessel boundary bitmap, and removing non-safety regions;
s403: and taking the puncture track boundary as a sliding window, sliding in the ROI area, and searching the sliding window position of the puncture track boundary completely surrounded by the ROI area as a matching position.
Part replacement step S5: the boundary matching fails, which indicates that the puncture condition is poor and safe puncture is difficult, the part of the object to be punctured is replaced, and the step returns to step S1;
the puncturing step S6 is executed: and generating a second puncture track of the venipuncture robot according to the matching position of the puncture track boundary and the first puncture track. And placing the puncture area at the matching position with the highest safety margin, and then transmitting the control parameters corresponding to the second puncture track to a mechanical arm control program to control the puncture needle to puncture according to a preset track.
The second puncture track is generated specifically by establishing a first space Cartesian coordinate system containing a punctured object part corresponding to the blood vessel ultrasonic image, and acquiring coordinates of the puncture track boundary at each vertex of the first space Cartesian coordinate system and coordinates of a first puncture track endpoint according to the matching position of the puncture track boundary; and constructing a space curve, and fitting the space curve with the first puncture track by adopting a sine function to generate a second puncture track of the venipuncture robot.
Taking forearm vein recognition as an example, the detailed process implemented is as follows:
1. blood vessel segmentation step
As shown in fig. 2, the vessel segmentation step includes the following sub-steps:
s101: the ultrasonic probe is close to the skin and images the vein, the original blood vessel ultrasonic image is input into the U-Net frame, and the step S102 is carried out;
s102: the U-Net frame is used for segmenting the blood vessel ultrasonic image, and the segmentation under the U-Net frame is insensitive to noise, has few parameters and high speed and is suitable for the segmentation of the ultrasonic image;
s103: and outputting a vessel segmentation map by U-Net.
2. Defining a puncture track boundary
As shown in fig. 3, in the present embodiment, the puncture device is a puncture needle, the puncture trajectory defining step is based on the predetermined movement of the puncture needle in the blood vessel, and the predetermined movement of the puncture needle according to the present invention is: s2 a-inserting the needle and puncturing the blood vessel wall, wherein the needle inserting angle is 30 degrees; s2 b-inserting the needle along a straight line to the central line of the blood vessel cavity; s2 c-continuing to insert the needle and gradually adjusting the angle to 15 degrees; and (4) stopping needle insertion after the angle of S2 d-is adjusted to 15 degrees, and starting injection or blood drawing.
Because the puncture needle ultrasonic image identifies possible errors, the puncture needle track is expanded outwards for a certain distance, because the error of the needle point is larger than that of other parts of the puncture needle, the tail end of the track is expanded more, the expanded contour is surrounded by a rectangle, the width w and the height h of the rectangle are obviously related to the dimension of the blood vessel cavity, and the relationship is regarded as a linear relationship:
h＝αdv
w＝βh＝αβdv
where h is the height of the puncture trajectory boundary, α is a first scaling factor, dvAnd obtaining the diameter of the blood vessel cavity in the blood vessel segmentation map through an ultrasonic image, wherein w is the width of the puncture track boundary, and beta is a second scaling factor.
3. Blood vessel delineation step
As shown in fig. 4, the blood vessel boundary defining step divides the blood vessel boundary into 3 types according to the safety: s3a shows a primary limit, only the area defined by the image is taken as a limit, and the safety is poor; s3b is a secondary boundary, and the safety region is reduced by considering the image recognition error; s3c shows a three-level boundary, with minimum safety region and maximum safety margin, taking into account image recognition errors and possible motion and deformation of veins. The shaded portion in figure 3 delimits the "safe zone", i.e. the zone in which the puncture needle can be moved. The definition of two-level and three-level limits and the method for reducing the limit equidistance and the point i on the first-level limit0And two adjacent points i1、i2And, corresponding to the point q on the new boundary, reducing the distance L:
Figure BDA0002645679950000061
Figure BDA0002645679950000071
in the formula i0For the first coordinate point on the vessel boundary corresponding to the safety region before shrinking, q is the first coordinate point in the safety region after shrinking0Corresponding coordinate point, i1For safety before reductionSecond coordinate point, i, on the vessel boundary corresponding to the region2To narrow down a third coordinate point, x, on the vessel boundary corresponding to the previous safety zonei0Is i0Abscissa of (a), yi0Is i0Ordinate of (a), xi1Is i1Abscissa of (a), yi1Is i1Ordinate of (a), xi2Is i2Abscissa of (a), yi2Is i2The ordinate of (c).
4. Boundary matching step
As shown in fig. 5, the boundary matching step includes the following sub-steps:
step S401: a vessel boundary map is generated. Generating a bitmap on the basis of the original image, wherein the value of a bit image pixel value is as follows:
Figure BDA0002645679950000072
step S402: the ROI is divided, the calculation amount of the subsequent steps is reduced, and the part without the safe region is cut off.
Step S403: and taking the puncture track boundary as a sliding window, sliding in the ROI area, and searching for the position of the sliding window which can be completely surrounded by the safety area. And if the corresponding sliding window position can be found, entering the step 6, otherwise, entering the step 5.
5. Performing a puncturing procedure
If a suitable sliding window position is found, the sliding window is fixed and the location coordinates of the puncture track are determined, as shown in fig. 6.
The center of the contact of the ultrasonic probe and the skin is selected as a space Cartesian coordinate origin. Determining the pixel coordinates of four vertexes of the puncture track boundary as (u) according to image recognition1，v1)、(u2，v2)、(u3，v3) And (u)4，v4) The coordinate of the starting point of the puncture track is (u)1，v1) Considering the error of the needle tip recognition, the coordinate of the end point of the puncture trajectory is (u)5，v5)：
u5＝u4-Δu
v5＝v4-Av
Where Δ u and Δ v are adjustment values selected based on ultrasound image errors.
The space coordinate span corresponding to each pixel is dy and dz, and the pixel coordinate of the space coordinate system origin in the image is (u)0And 0), then the spatial coordinates of the four vertexes of the puncture track boundary and the puncture track end point are:
(yi，zi)＝(-dy(u0-ui)，dzvi)，i＝1，2，3，4，5
fitting of the puncture needle trajectory may be quadratic function fitting, trigonometric function fitting, or the like, and in this embodiment, sinusoidal function fitting is employed, and an analytical formula of a spatial curve is represented by v ═ a sin ω u, and the fitting is carried out (u ═ a sin ω u)1，v1) And (u)5，v5) And (6) fitting. After fitting, the trajectory is input into the robot control program, and the robot motion process is shown in fig. 7.
The embodiment also provides a venipuncture robot puncture trajectory planning device based on ultrasound image guidance, which includes a memory and a processor, where the memory stores a computer program, and the processor invokes the computer program to execute the steps of the method for planning a venipuncture robot puncture trajectory based on ultrasound image guidance.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims (10)

1. A puncture track planning device of a venipuncture robot based on ultrasound image guidance comprises a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of a puncture track planning method of the venipuncture robot based on ultrasound image guidance;
the venipuncture robot controls a puncture piece to puncture a blood vessel, and the method for planning the puncture track of the venipuncture robot based on ultrasonic image guidance comprises the following steps:
a blood vessel segmentation step: acquiring a blood vessel ultrasonic image of a certain part of an object to be punctured, and acquiring a blood vessel segmentation map from the blood vessel ultrasonic image;
a puncture track boundary defining step: according to a preset first puncture track of the puncture piece in the blood vessel, the puncture track limit of the puncture piece in the blood vessel is defined;
a blood vessel boundary defining step: dividing a safety region from the blood vessel segmentation graph to obtain a blood vessel boundary;
boundary matching and puncturing: matching the puncture track limit with the blood vessel limit, and if the blood vessel limit completely contains the puncture track limit, generating a second puncture track of the venipuncture robot according to the matching position of the puncture track limit and the first puncture track; otherwise, the part of the object to be punctured is replaced, and the method is executed again.
2. The ultrasound-image-guided venipuncture robot puncture trajectory planning device according to claim 1, wherein in the vessel segmentation step, the vessel segmentation map is obtained from the vessel ultrasound image by using a U-Net framework.
3. The ultrasound-image-guided venipuncture robot puncture trajectory planning apparatus according to claim 1, wherein in the puncture trajectory boundary defining step, the puncture trajectory boundary is defined by extending the first puncture trajectory outward to obtain a puncture trajectory boundary.
4. The ultrasound-image-guided-based venipuncture robot puncture trajectory planning device of claim 3, wherein the extension range of the tail of the first puncture trajectory is larger than that of other parts.
5. The ultrasound-image-guided venipuncture robot puncture trajectory planning device according to claim 1, wherein in the puncture trajectory boundary defining step, the puncture trajectory boundary is represented by a rectangle, and the calculation expression of the height and width of the puncture trajectory boundary is as follows:
h＝αdv
w＝βh＝αβdv
where h is the height of the puncture trajectory boundary, α is a first scaling factor, dvAnd the diameter of the blood vessel cavity in the blood vessel segmentation map, w is the width of the puncture track boundary, and beta is a second scaling factor.
6. The apparatus for planning a puncture trajectory of a venipuncture robot based on ultrasound image guidance as claimed in claim 1, wherein in the step of demarcating a blood vessel, the safety area is narrowed down according to an image recognition error.
7. The apparatus for planning a puncture trajectory of a venipuncture robot based on ultrasound image guidance as claimed in claim 1, wherein in the step of demarcating a blood vessel, the safety area is narrowed according to an image recognition error and a motion and a deformation of a vein.
8. The ultrasound-image-guided venipuncture robot puncture trajectory planning device according to claim 6 or 7, wherein the reduction calculation expression of the safety region is as follows:
Figure FDA0003238941710000021
in the formula i0For the first coordinate point on the vessel boundary corresponding to the safety region before shrinking, q is the first coordinate point in the safety region after shrinking0Corresponding coordinate point, i1For narrowing the blood corresponding to the pre-safe areaSecond coordinate point on the tube boundary, i2To narrow down a third coordinate point, x, on the vessel boundary corresponding to the previous safety zonei0Is i0Abscissa of (a), yi0Is i0Ordinate of (a), xi1Is i1Abscissa of (a), yi1Is i1Ordinate of (a), xi2Is i2Abscissa of (a), yi2Is i2The ordinate of (c).
9. The ultrasound-image-guided venipuncture robot puncture trajectory planning device of claim 1, wherein in the boundary matching and puncturing step, the matching of the puncture trajectory boundary with the blood vessel boundary specifically comprises the following steps:
s401: obtaining a blood vessel boundary diagram according to the blood vessel boundary delimiting step, and converting the blood vessel boundary diagram into a blood vessel boundary bitmap;
s402: dividing an ROI (region of interest) region in the blood vessel boundary bitmap, and removing an unsafe region;
s403: and taking the puncture track boundary as a sliding window, sliding in the ROI area, and searching the sliding window position of the puncture track boundary completely surrounded by the ROI area as a matching position.
10. The ultrasound-image-guided venipuncture robot puncture trajectory planning apparatus according to claim 1, wherein in the boundary matching and puncturing step, a second puncture trajectory of the venipuncture robot is generated according to the matching position of the puncture trajectory boundary and the first puncture trajectory, specifically,
establishing a first space Cartesian coordinate system containing a punctured object part corresponding to the blood vessel ultrasonic image, and acquiring coordinates of the puncturing track boundary at each vertex of the first space Cartesian coordinate system and coordinates of a first puncturing track endpoint according to the matching position of the puncturing track boundary; and constructing a space curve, fitting the space curve with the first puncture track, and generating a second puncture track of the venipuncture robot.
CN202010853764.0A 2020-08-24 2020-08-24 Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance Active CN112022294B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010853764.0A CN112022294B (en) 2020-08-24 2020-08-24 Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010853764.0A CN112022294B (en) 2020-08-24 2020-08-24 Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance

Publications (2)

Publication Number Publication Date
CN112022294A CN112022294A (en) 2020-12-04
CN112022294B true CN112022294B (en) 2022-02-18

Family

ID=73580575

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010853764.0A Active CN112022294B (en) 2020-08-24 2020-08-24 Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance

Country Status (1)

Country Link
CN (1) CN112022294B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112959327B (en) * 2021-03-31 2022-07-29 上海电气集团股份有限公司 Robot motion control method, system, electronic device, and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110267609A (en) * 2017-01-30 2019-09-20 费森尤斯医疗护理德国有限责任公司 System and method for automatic intubation
CN110537960A (en) * 2018-05-29 2019-12-06 上海联影医疗科技有限公司 Puncture path determination method, storage device and robot-assisted surgery system

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5416900B2 (en) * 2007-11-22 2014-02-12 株式会社東芝 Ultrasonic diagnostic apparatus and puncture support control program
CN106725762A (en) * 2016-12-30 2017-05-31 西安中科微光影像技术有限公司 A kind of automatic puncturing method
EP3565469B1 (en) * 2017-01-05 2022-08-24 Koninklijke Philips N.V. Visualizing a course of a vasculature structure with an occlusion
CN110575253B (en) * 2019-09-27 2020-11-10 西安交通大学医学院第一附属医院 Robot-assisted puncture system based on positioning plate and puncture track acquisition method
CN111513849B (en) * 2020-04-30 2022-04-19 京东方科技集团股份有限公司 Surgical system for puncture, control method and control device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110267609A (en) * 2017-01-30 2019-09-20 费森尤斯医疗护理德国有限责任公司 System and method for automatic intubation
CN110537960A (en) * 2018-05-29 2019-12-06 上海联影医疗科技有限公司 Puncture path determination method, storage device and robot-assisted surgery system

Also Published As

Publication number Publication date
CN112022294A (en) 2020-12-04

Similar Documents

Publication Publication Date Title
US11937963B2 (en) Vascular selection from images
US8355550B2 (en) Methods and apparatus for virtual coronary mapping
Sakai et al. Computer analysis and classification of photographs of human faces
US9092848B2 (en) Methods for automatic segmentation and temporal tracking
US11986330B2 (en) Method and system for generating object 3D point cloud in medical imaging system
CN112022294B (en) Operation trajectory planning method of venipuncture robot based on ultrasonic image guidance
CN109732589A (en) A kind of robot manipulating task track acquisition methods based on line laser sensor
KR101900679B1 (en) Method for 3d coronary registration based on vessel feature, recording medium and device for performing the method
Hadjerci et al. Assistive system based on nerve detection and needle navigation in ultrasound images for regional anesthesia
CN111612778A (en) Preoperative CTA and intraoperative X-ray coronary artery registration method
CN107392847A (en) A kind of fingerprint image joining method based on minutiae point and range image
WO2023169578A1 (en) Image processing method, system and apparatus for interventional surgery
JPH01231183A (en) Linearity deciding device in image processor
CN116807577B (en) Full-automatic venipuncture equipment and full-automatic venipuncture method
Pang et al. On automated tongue image segmentation in Chinese medicine
CN112535519A (en) Puncture robot control method based on real-time ultrasonic image
Yoshizaki et al. Mutual assistance between speech and vision for human-robot interface
CN116013289A (en) Acoustic control management system based on artificial intelligence
CN111968097B (en) Blood vessel puncture image processing method and blood vessel puncture robot
CN112168349A (en) Needle tracking and path display method of venipuncture robot
CN109919935B (en) Head and neck blood vessel segmentation method and device
Zhao et al. Vessel site selection for autonomous cannulation under NIR image guidance
CN109447956A (en) A kind of blood vessel relative width calculation method and system
Bowden et al. Anatomical registration and segmentation by warping template finite element models
CN113256564B (en) Catheter parameter extraction method and device in medical image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant