WO2023193175A1 - 一种基于超声图像的穿刺针实时检测方法及装置 - Google Patents

一种基于超声图像的穿刺针实时检测方法及装置 Download PDF

Info

Publication number
WO2023193175A1
WO2023193175A1 PCT/CN2022/085513 CN2022085513W WO2023193175A1 WO 2023193175 A1 WO2023193175 A1 WO 2023193175A1 CN 2022085513 W CN2022085513 W CN 2022085513W WO 2023193175 A1 WO2023193175 A1 WO 2023193175A1
Authority
WO
WIPO (PCT)
Prior art keywords
puncture needle
image
real
puncture
straight line
Prior art date
Application number
PCT/CN2022/085513
Other languages
English (en)
French (fr)
Inventor
余韩梅
钱程
周寿军
赵福星
陈静涛
蒲尧
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2022/085513 priority Critical patent/WO2023193175A1/zh
Publication of WO2023193175A1 publication Critical patent/WO2023193175A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling

Definitions

  • the present application relates to the field of image processing, and specifically to a method and device for real-time detection of puncture needles based on ultrasound images.
  • Percutaneous puncture surgery is a very common surgical procedure in the clinical diagnosis and treatment of tumors.
  • Conventional and cutting-edge tumor diagnosis and treatment methods such as the gold standard for tumor diagnosis - percutaneous biopsy and one of the three pillar technologies of tumor treatment - interventional therapy (such as radioactive particle therapy, cryoablation, radiofrequency ablation, electrical ablation, magnetic heating therapy, etc.), it is necessary to use percutaneous puncture surgery to accurately place biopsy needles, particles or treatment devices into predetermined target areas.
  • the accuracy of puncture directly determines the effect of diagnosis and treatment.
  • the biggest advantage of percutaneous puncture surgery guided by ultrasound images is that it can provide real-time image guidance and can dynamically monitor blood flow or fluid conditions in the cavity through Doppler imaging, so it has been vigorously promoted in clinical practice.
  • Chinese patent CN113362294A discloses a puncture needle identification method, system and equipment containing puncture needle ultrasonic blood vessel images. It uses the U-Net network in deep learning and combines noise reduction and filtering to identify the area of interest (that is, the area where the puncture needle may exist), and extract the puncture needle position from the above area, but this method does not mention Computing speed and real-time effects.
  • Cosmas et al. proposed a technical framework for automatically detecting puncture needles in two-dimensional ultrasound images using convolutional neural networks, including a fully convolutional network (FCN) and a fast region-based convolutional neural network (R-CNN). .
  • FCN fully convolutional network
  • R-CNN fast region-based convolutional neural network
  • the FCN network is used to extract candidate areas (i.e. areas where puncture needles may exist), and then feeds the candidate areas back to the R-CNN network for more refined puncture needle detection.
  • candidate areas i.e. areas where puncture needles may exist
  • the trajectory and needle tip position of the puncture needle are estimated.
  • the puncture needle part of ultrasound images often exhibits discontinuities and strong artifacts. It is difficult for the above method to avoid detection inaccuracies caused by these two points.
  • Yi Lee et al. proposed a tracking segmentation model based on concurrent space and "squeeze and excitation" channel (scSE) for puncture needle detection and trajectory prediction in ultrasound images.
  • This method uses a lightweight deep learning architecture LinkNet as the baseline segmentation network and integrates the scSE module to learn spatial information to better predict the puncture needle trajectory.
  • the model uses ultrasound images of renal biopsy specimens from 8 patients as a network training data set, uses a boundary following algorithm to obtain the puncture needle outline, and uses Green's formula to calculate the area occupied by the outline. Finally, the puncture needle trajectory is extrapolated through the diagonal of the minimal bounding box of the contour. However, the error of this method exceeds 10°, and the accuracy needs to be improved.
  • the current existing methods for real-time identification of puncture needles use multiple networks or methods for joint detection, and the steps are complex, resulting in slow detection speed and difficulty in meeting the needs of real-time applications.
  • Embodiments of the present application provide a method and device for real-time detection of puncture needles based on ultrasound images, to at least solve the existing technical problem of low accuracy in puncture needle detection and positioning.
  • a real-time detection method of puncture needles based on ultrasound images including the following steps:
  • the technical solution adopted by the embodiment of the present application also includes: intercepting the area containing the puncture needle from the acquired internal image of the punctured organ, including:
  • the area containing the puncture needle is intercepted from the acquired image frame as the area of interest.
  • the technical solution adopted by the embodiment of this application also includes: using the trained UNet++ network to segment the intercepted image, and the identification position of the puncture needle is obtained including:
  • NestedUNet is configured as a classic network of multi-feature fusion, integrating features at different levels, allowing a deep network with a huge amount of parameters to significantly reduce the amount of parameters within an acceptable accuracy range.
  • the technical solutions adopted in the embodiments of this application also include: post-processing the segmented images using the method of removing small connected domains, and it is concluded that the precise positioning area of the puncture needle includes:
  • the technical solution adopted by the embodiment of this application also includes: performing straight line fitting on the pixel points in the precise positioning area of the puncture needle, and using the fitting result as the straight line where the puncture needle is located, including:
  • x i,j represents the abscissa i of the pixel point of the segmented image at the position (i, j)
  • y i,j represents the ordinate j of the pixel point of the segmented image at the position (i, j)
  • is the mean of all abscissas is the mean of all ordinates.
  • the technical solution adopted by the embodiment of the present application also includes: before intercepting the area containing the puncture needle from the acquired internal image of the punctured organ, the method further includes:
  • An ultrasound imaging instrument is used to obtain images of the interior of the punctured organ.
  • the technical solution adopted by the embodiment of the present application also includes: after intercepting the area containing the puncture needle from the acquired internal image of the punctured organ, the method further includes:
  • the fitting result is used as the image of the straight line where the puncture needle is located and is displayed in real time in the original ultrasound image.
  • the technical solution adopted by the embodiment of the present application also includes: after using the fitting result as the image of the straight line where the puncture needle is located and displaying it in real time in the original ultrasound image, the method also includes:
  • the detection of the current frame is completed. If the display ends, the operation will stop; if the display continues, return to the step to intercept the area containing the puncture needle from the acquired internal image of the punctured organ to start the puncture needle detection work of the next frame.
  • a device for real-time detection of puncture needles based on ultrasound images including:
  • An image interception unit used to intercept the area containing the puncture needle from the acquired internal image of the punctured organ
  • the image segmentation unit is used to segment the intercepted image using the trained UNet++ network to obtain the identification position of the puncture needle;
  • the image post-processing unit is used to post-process the segmented images using the small connected domain removal method to obtain the precise positioning area of the puncture needle;
  • the pixel fitting unit is used to perform straight line fitting on the pixel points in the precise positioning area of the puncture needle, and use the fitting result as the straight line where the puncture needle is located.
  • a storage medium stores a program file capable of implementing any of the above-mentioned real-time detection methods of puncture needles based on ultrasound images.
  • a processor is provided, and the processor is configured to run a program, wherein when the program is run, any one of the above real-time detection methods for puncture needles based on ultrasound images is executed.
  • the real-time puncture needle detection method and device based on ultrasound images in the embodiments of the present application intercepts the area containing the puncture needle from the acquired internal image of the punctured organ; uses the trained UNet++ network to segment the intercepted image, and obtains Identify the location of the puncture needle; use the small connected domain removal method to post-process the segmented image to obtain the puncture needle's precise positioning area; perform straight line fitting on the pixels in the puncture needle's precise positioning area, and use the fitting result as the puncture needle The straight line.
  • This application uses UNet++ to detect puncture needles, and uses the connected domain method to post-process the network output results, which improves the puncture needle detection accuracy and computing speed. It has been experimentally verified that the puncture needle real-time detection function has a computing frequency of up to 6Hz. This application improves the puncture needle detection accuracy and calculation speed by improving the neural network and simplifying the post-processing method.
  • Figure 1 is a flow chart of the real-time puncture needle detection method based on ultrasound images in this application;
  • Figure 2 is a post-processing process diagram of puncture needle detection results in the real-time puncture needle detection method based on ultrasound images in this application;
  • Figure 3 is a schematic diagram of real-time detection of puncture needles in the real-time detection method of puncture needles based on ultrasound images in this application.
  • a method for real-time detection of puncture needles based on ultrasound images includes the following steps:
  • the real-time puncture needle detection method based on ultrasound images in the embodiment of the present application intercepts the area containing the puncture needle from the acquired internal image of the punctured organ; uses the trained UNet++ network to segment the intercepted image to obtain the puncture needle identification position; use the small connected domain removal method to post-process the segmented image to obtain the puncture needle precise positioning area; perform straight line fitting on the pixels in the puncture needle precise positioning area, and use the fitting result as the straight line where the puncture needle is located .
  • This application uses UNet++ to detect puncture needles, and uses the connected domain method to post-process the network output results, which improves the puncture needle detection accuracy and computing speed. It has been experimentally verified that the puncture needle real-time detection function has a computing frequency of up to 6Hz. This application improves the puncture needle detection accuracy and calculation speed by improving the neural network and simplifying the post-processing method.
  • the area containing the puncture needle captured from the internal image of the punctured organ includes:
  • the area containing the puncture needle is intercepted from the acquired image frame as the area of interest.
  • the trained UNet++ network is used to segment the intercepted image, and the identification positions of the puncture needle include:
  • NestedUNet is configured as a classic network of multi-feature fusion, integrating features at different levels, allowing deep networks with huge parameters to greatly reduce the number of parameters within an acceptable accuracy range.
  • the method of removing small connected domains is used to post-process the segmented images, and it is concluded that the precise positioning area of the puncture needle includes:
  • straight line fitting is performed on the pixels in the precise positioning area of the puncture needle, and the fitting result is used as the straight line where the puncture needle is located, including:
  • x i,j represents the abscissa i of the pixel point of the segmented image at the position (i, j)
  • y i,j represents the ordinate j of the pixel point of the segmented image at the position (i, j)
  • is the mean of all abscissas is the mean of all ordinates.
  • the method Before intercepting the area containing the puncture needle from the acquired internal image of the punctured organ, the method further includes:
  • An ultrasound imaging instrument is used to obtain images of the interior of the punctured organ.
  • the method after intercepting the area containing the puncture needle from the acquired internal image of the punctured organ, the method also includes:
  • the fitting result is used as the image of the straight line where the puncture needle is located and is displayed in real time in the original ultrasound image.
  • the method after using the fitting result as the image of the straight line where the puncture needle is located and displaying it in real time in the original ultrasound image, the method also includes:
  • the detection of the current frame is completed. If the display ends, the operation will stop; if the display continues, return to the step to intercept the area containing the puncture needle from the acquired internal image of the punctured organ to start the puncture needle detection work of the next frame.
  • this application proposes a real-time detection method of puncture needles based on ultrasound images, which can be used for real-time detection and positioning of puncture needles in ultrasound images during surgery, providing doctors with a visualization of the location and direction of the puncture needle. , thereby helping doctors improve puncture accuracy.
  • step (vi) The detection of the current frame is completed. If the display ends, the operation will stop; if the display continues, return to step (i) to start the puncture needle detection work of the next frame.
  • Deep learning is widely used in medical image segmentation, among which UNet is one of the most frequently used and stable network structures.
  • UNet is one of the most frequently used and stable network structures.
  • AttentionUNet introduces the attention mechanism into UNet.
  • an attention module is used to re-adjust the output features of the encoder.
  • NestedUNet is a classic network for multi-feature fusion. It integrates features at different levels, allowing deep networks with huge parameters to greatly reduce the number of parameters within an acceptable accuracy range. This application compares the performance of attention mechanism and multi-scale feature fusion on this task, and finds that the latter is more suitable for this detection task.
  • This application uses the dice coefficient matrix, accuracy, and sensitivity to measure the performance of each model. This application further calculates the angle root mean square error (AE, unit: degrees) and distance root mean square error (DE, unit: centimeters) to evaluate the accuracy of the predicted needle trajectory.
  • AE angle root mean square error
  • DE distance root mean square error
  • this application collected 1230 ultrasound images, of which 984 were used as training data and 246 were used as test data. Table 1 shows a comparison of the test results of the three methods. The results show that NestedUNet has the best detection accuracy, and the angle and distance deviation between the detection results and the correct values is also the smallest.
  • test results need to be post-processed to extract possible puncture needle areas.
  • puncture needles are often relatively large connected areas, so each connected area is traversed and smaller connected areas are deleted based on the threshold to obtain accurate segmentation results of puncture needles.
  • Figure 2 for a diagram of the post-processing process of puncture needle detection results.
  • Figure 2(a) is a diagram of the neural network segmentation results
  • Figure 2(b) is a diagram of the processing results of removing small-area connected domains.
  • This application needs to display the direction of the puncture needle in real time, so the detected results are linearly fitted to obtain the puncture trajectory of the puncture needle to determine whether it deviates from the target.
  • the puncture needle existing area obtained in the previous step extract the points with pixel values greater than 0 in the accurately segmented image, and assign their coordinates to x and y for straight line fitting.
  • x i,j represents the abscissa i of the pixel point of the segmented image at the position (i, j)
  • y i,j represents the ordinate j of the pixel point of the segmented image at the position (i, j)
  • is the mean of all abscissas is the mean of all ordinates.
  • the key points and the points to be protected in this application method are at least:
  • This application uses UNet++ to detect puncture needles, and uses the connected domain method to post-process the network output results, which improves the puncture needle detection accuracy and computing speed. It has been experimentally verified that the puncture needle real-time detection function has a computing frequency of up to 6Hz.
  • This application improves the puncture needle detection accuracy and calculation speed by improving the neural network and simplifying the post-processing method.
  • a device for real-time detection of puncture needles based on ultrasound images including:
  • An image interception unit used to intercept the area containing the puncture needle from the acquired internal image of the punctured organ
  • the image segmentation unit is used to segment the intercepted image using the trained UNet++ network to obtain the identification position of the puncture needle;
  • the image post-processing unit is used to post-process the segmented images using the small connected domain removal method to obtain the precise positioning area of the puncture needle;
  • the pixel fitting unit is used to perform straight line fitting on the pixel points in the precise positioning area of the puncture needle, and use the fitting result as the straight line where the puncture needle is located.
  • the real-time puncture needle detection device based on ultrasound images in the embodiment of the present application intercepts the area containing the puncture needle from the acquired internal image of the punctured organ; uses the trained UNet++ network to segment the intercepted image to obtain the puncture needle identification position; use the small connected domain removal method to post-process the segmented image to obtain the puncture needle precise positioning area; perform straight line fitting on the pixels in the puncture needle precise positioning area, and use the fitting result as the straight line where the puncture needle is located .
  • This application uses UNet++ to detect puncture needles, and uses the connected domain method to post-process the network output results, which improves the puncture needle detection accuracy and computing speed. It has been experimentally verified that the puncture needle real-time detection function has a computing frequency of up to 6Hz. This application improves the puncture needle detection accuracy and calculation speed by improving the neural network and simplifying the post-processing method.
  • this application proposes a real-time detection device for puncture needles based on ultrasound images, which can be used for real-time detection and positioning of puncture needles in ultrasound images during surgery, providing doctors with a visualization of the location and direction of the puncture needle. , thereby helping doctors improve puncture accuracy.
  • Image interception unit intercept the area containing the puncture needle from the acquired image frame as the region of interest (ROI), with a size of 512 ⁇ 512;
  • Image segmentation unit Use the trained UNet++ network to segment the image to obtain the identification position of the puncture needle;
  • Image post-processing unit Use the small connected domain removal method to post-process the segmented image to obtain the precise positioning area of the puncture needle;
  • Pixel fitting unit Perform straight line fitting on the pixel points in the area, and use the fitting result as the straight line where the puncture needle is located, and display it in real time in the original ultrasound image;
  • step (vi) The detection of the current frame is completed. If the display ends, the operation will stop; if the display continues, return to step (i) to start the puncture needle detection work of the next frame.
  • Image segmentation unit rough segmentation of puncture needles
  • Deep learning is widely used in medical image segmentation, among which UNet is one of the most frequently used and stable network structures.
  • UNet is one of the most frequently used and stable network structures.
  • AttentionUNet introduces the attention mechanism into UNet.
  • an attention module is used to re-adjust the output features of the encoder.
  • NestedUNet is a classic network of multi-feature fusion. It integrates features at different levels, allowing deep networks with huge parameters to greatly reduce the number of parameters within an acceptable accuracy range. This application compares the performance of attention mechanism and multi-scale feature fusion on this task, and finds that the latter is more suitable for this detection task.
  • This application uses the dice coefficient matrix, accuracy, and sensitivity to measure the performance of each model. This application further calculates the angle root mean square error (AE, unit: degrees) and distance root mean square error (DE, unit: centimeters) to evaluate the accuracy of the predicted needle trajectory.
  • AE angle root mean square error
  • DE distance root mean square error
  • this application collected 1230 ultrasound images, of which 984 were used as training data and 246 were used as test data. Table 1 shows a comparison of the test results of the three methods. The results show that NestedUNet has the best detection accuracy, and the angle and distance deviation between the detection results and the correct values is also the smallest.
  • Image post-processing unit Since there may be a variety of interferences in the detection results, the detection results need to be post-processed to extract possible puncture needle areas. In the detection results, puncture needles are often relatively large connected areas, so each connected area is traversed and smaller connected areas are deleted based on the threshold to obtain accurate segmentation results of puncture needles. See Figure 2 for a diagram of the post-processing process of puncture needle detection results.
  • Figure 2(a) is a diagram of the neural network segmentation results
  • Figure 2(b) is a diagram of the processing results of removing small-area connected domains.
  • Pixel fitting unit This application needs to display the direction of the puncture needle in real time, so the detected results are linearly fitted to obtain the puncture trajectory of the puncture needle to determine whether it deviates from the target. According to the puncture needle existing area obtained in the previous step, extract the points with pixel values greater than 0 in the accurately segmented image, and assign their coordinates to x and y for straight line fitting.
  • x i,j represents the abscissa i of the pixel point of the segmented image at the position (i, j)
  • y i,j represents the ordinate j of the pixel point of the segmented image at the position (i, j)
  • is the mean of all abscissas is the mean of all ordinates.
  • the key points and the points to be protected of the device of this application are at least:
  • This application uses UNet++ to detect puncture needles, and uses the connected domain method to post-process the network output results, which improves the puncture needle detection accuracy and computing speed. It has been experimentally verified that the puncture needle real-time detection function has a computing frequency of up to 6Hz.
  • This application improves the puncture needle detection accuracy and calculation speed by improving the neural network and simplifying the post-processing method.
  • This application uses independent collection of body membrane data for verification.
  • the concept of the method shown in Figure 3 is a schematic diagram of real-time detection of puncture needles. This application has been proven through body film experiments that its detection accuracy can reach 91.9%, and its computing speed can make the real-time image update frequency reach 6.25Hz.
  • a storage medium that stores program files that can implement any of the above-mentioned real-time detection methods of puncture needles based on ultrasound images.
  • a processor is used to run a program, wherein when the program is running, it executes any one of the above-mentioned real-time detection methods of puncture needles based on ultrasound images.
  • the disclosed technical content can be implemented in other ways.
  • the system embodiments described above are only illustrative.
  • the division of units can be a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or integrated into Another system, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the units or modules may be in electrical or other forms.
  • Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the above integrated units can be implemented in the form of hardware or software functional units.
  • Integrated units may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as independent products.
  • the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application.
  • the aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

本申请涉及图像处理领域，具体涉及一种基于超声图像的穿刺针实时检测方法及装置。该方法及装置对获取到的被穿刺器官内部图像截取包含穿刺针的区域；利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。本申请使用UNet++对穿刺针进行检测，并利用连通域方法对网络输出结果进行了后处理，提高了穿刺针检测精度和运算速度,经实验验证穿刺针实时检测功能运算频次可达6Hz。本申请通过改进神经网络以及简化后处理方法，提高了穿刺针检测精度和运算速度。

Description

一种基于超声图像的穿刺针实时检测方法及装置 技术领域
本申请涉及图像处理领域，具体而言，涉及一种基于超声图像的穿刺针实时检测方法及装置。
背景技术
经皮穿刺手术在肿瘤的临床诊断和治疗中是十分常见的术式。常规及前沿的肿瘤诊疗手段，如肿瘤诊断金标准-经皮穿刺活检和肿瘤治疗三大支柱技术之一-介入治疗(如放射性粒子疗法、冷冻消融法、射频消融治疗、电消融法，磁热疗法等)，都需要借助经皮穿刺手术将活检针、粒子或治疗器具精准放入预定靶向区域，穿刺的精度直接决定了诊治的效果。超声影像引导下的经皮穿刺手术，最大优势在于可提供实时图像引导，且可以通过多普勒成像动态监测腔道内血流或液体情况，因而得到临床的大力推广。
超声引导术中，软组织器官受到穿刺针的穿刺力和超声探头压在人体表面的挤压力，因此发生器官形变、软器官的变形,使得穿刺针偏离规划轨迹,使得针尖与穿刺目标的相对位置存在一定的不确定性，是进针不准确的重要原因，这直接导致活检的准确率和介入治疗的效果大大折扣，甚至不能进行后续的介入治疗。不准确的进针同时也会增加患者的创伤及术后并发症的发病几率。因此，对穿刺针的位置及进针方向识别可以帮助医生更好的实时调整穿刺针进针方向，增加命中率。
现有的穿刺针识别技术没有体现实时计算的可能性及对应方法的验证效果。中国专利CN113362294A公开了一种含穿刺针超声血管图像的穿刺针识别 方法、系统和设备。其利用深度学习中的U-Net网络，并结合降噪和滤波等手段识别感兴趣区域(即穿刺针可能存在的区域)，并从上述区域中提取出穿刺针位置，但该方法未提及运算速度及实时效果。Cosmas等人提出了一种利用卷积神经网络自动检测二维超声图像中穿刺针的技术框架，包括一个完全卷积网络(FCN)和一个快速的基于区域的卷积神经网络(R-CNN)。其中FCN网络用于提取候选区域(即穿刺针可能存在的区域)，然后将候选区域反馈给R-CNN网络进行更精细的穿刺针检测。根据针检测结果以及穿刺针在超声图像中亮度度值不变性特征，从而估计穿刺针的轨迹和针尖位置。但超声图像中穿刺针部分经常呈现不连续性且伪影较强，上述方法很难避免这两点所造成的检测不准确性。Yi Lee等人提出了一个基于并发空间和“挤压和激励”通道(scSE)的跟踪分割模型，用于超声图像中穿刺针的检测和轨迹预测。该方法采用了一个轻量级的深度学习架构LinkNet作为基线分割网络，并集成scSE模块学习空间信息，以更好地预测穿刺针轨迹。该模型使用8例患者肾活检标本的超声图像作为网络训练数据集，使用边界跟随算法得到穿刺针轮廓，并使用格林公式计算轮廓所占面积。最后通过轮廓的最小边界框的对角线外推穿刺针轨迹。但该方法的误差超过10°以上，精度有待改善。
综上目前用于穿刺针实时识别的已有方法采用多种网络或方法联合检测，步骤复杂，导致检测速度慢，难以满足实时应用需求。
发明内容
本申请实施例提供了一种基于超声图像的穿刺针实时检测方法及装置，以至少解决现有穿刺针检测和定位精度低的技术问题。
根据本申请的一实施例，提供了一种基于超声图像的穿刺针实时检测方 法，包括以下步骤：
对获取到的被穿刺器官内部图像截取包含穿刺针的区域；
利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；
使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。
本申请实施例采取的技术方案还包括：对获取到的被穿刺器官内部图像截取包含穿刺针的区域包括：
在获取的一帧图像中截取包含穿刺针的区域作为感兴趣区域。
本申请实施例采取的技术方案还包括：利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置包括：
利用训练好的NestedUNet网络对截取后的图像进行分割。
本申请实施例采取的技术方案还包括：NestedUNet被配置为多特征融合的经典网络，整合了不同层次的特征，让参数量巨大的深度网络在可接受的精度范围内大幅度的缩减参数量。
本申请实施例采取的技术方案还包括：使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域包括：
遍历各个连通区域，删除面积小于阈值的连通区域，得到穿刺针精准分割结果。
本申请实施例采取的技术方案还包括：对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线包括：
在穿刺针精准定位区域提取出精确分割图像中像素值大于0的点，将其坐标赋值给x、y，用于直线拟合；直线拟合系数的表达式如下：
Figure PCTCN2022085513-appb-000001
Figure PCTCN2022085513-appb-000002
其中，x i,j表示分割图像在(i,j)位置的像素点的横坐标i，y i,j表示分割图像在(i,j)位置的像素点的纵坐标j，
Figure PCTCN2022085513-appb-000003
为所有横坐标的均值，
Figure PCTCN2022085513-appb-000004
为所有纵坐标的均值。
本申请实施例采取的技术方案还包括：在对获取到的被穿刺器官内部图像截取包含穿刺针的区域之前，方法还包括：
使用超声成像仪器获取被穿刺器官内部图像。
本申请实施例采取的技术方案还包括：在对获取到的被穿刺器官内部图像截取包含穿刺针的区域之后，方法还包括：
以拟合结果作为穿刺针所在直线的图像在超声原图中实时显示。
本申请实施例采取的技术方案还包括：在以拟合结果作为穿刺针所在直线的图像在超声原图中实时显示之后，方法还包括：
当前帧检测完成，若结束显示，则停止运行；若继续显示，则回到步骤对获取到的被穿刺器官内部图像截取包含穿刺针的区域开始下一帧的穿刺针检测工作。
根据本申请的另一实施例，提供了一种基于超声图像的穿刺针实时检测装置，包括：
图像截取单元，用于对获取到的被穿刺器官内部图像截取包含穿刺针的区域；
图像分割单元，用于利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；
图像后处理单元，用于使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
像素点拟合单元，用于对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。
根据本申请的另一实施例，提供一种存储介质，存储介质存储有能够实现上述任意一项基于超声图像的穿刺针实时检测方法的程序文件。
根据本申请的另一实施例，提供一种处理器，处理器用于运行程序，其中，程序运行时执行上述任意一项的基于超声图像的穿刺针实时检测方法。
相对于现有技术，本申请实施例产生的有益效果在于：
本申请实施例中的基于超声图像的穿刺针实时检测方法及装置，对获取到的被穿刺器官内部图像截取包含穿刺针的区域；利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。本申请使用UNet++对穿刺针进行检测，并利用连通域方法对网络输出结果进行了后处理，提高了穿刺针检测精度和运算速度,经实验验证穿刺针实时检测功能运算频次可达6Hz。本申请通过改进神经网络以及简化后处理方法，提高了穿刺针检测精度和运算速度。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部 分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1为本申请基于超声图像的穿刺针实时检测方法的流程图；
图2为本申请基于超声图像的穿刺针实时检测方法中穿刺针检测结果后处理过程图；
图3为本申请基于超声图像的穿刺针实时检测方法中穿刺针实时检测示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
实施例1
根据本申请一实施例，提供了一种基于超声图像的穿刺针实时检测方法，包括以下步骤：
对获取到的被穿刺器官内部图像截取包含穿刺针的区域；
利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；
使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。
本申请实施例中的基于超声图像的穿刺针实时检测方法，对获取到的被穿刺器官内部图像截取包含穿刺针的区域；利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。本申请使用UNet++对穿刺针进行检测，并利用连通域方法对网络输出结果进行了后处理，提高了穿刺针检测精度和运算速度,经实验验证穿刺针实时检测功能运算频次可达6Hz。本申请通过改进神经网络以及简化后处理方法，提高了穿刺针检测精度和运算速度。
其中，对获取到的被穿刺器官内部图像截取包含穿刺针的区域包括：
在获取的一帧图像中截取包含穿刺针的区域作为感兴趣区域。
其中，利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置包括：
利用训练好的NestedUNet网络对截取后的图像进行分割。
其中，NestedUNet被配置为多特征融合的经典网络，整合了不同层次的特征，让参数量巨大的深度网络在可接受的精度范围内大幅度的缩减参数量。
其中，使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域包括：
遍历各个连通区域，删除面积小于阈值的连通区域，得到穿刺针精准分割结果。
其中，对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线包括：
在穿刺针精准定位区域提取出精确分割图像中像素值大于0的点，将其坐标赋值给x、y，用于直线拟合；直线拟合系数的表达式如下：
Figure PCTCN2022085513-appb-000005
Figure PCTCN2022085513-appb-000006
其中，x i,j表示分割图像在(i,j)位置的像素点的横坐标i，y i,j表示分割图像在(i,j)位置的像素点的纵坐标j，
Figure PCTCN2022085513-appb-000007
为所有横坐标的均值，
Figure PCTCN2022085513-appb-000008
为所有纵坐标的均值。
其中，在对获取到的被穿刺器官内部图像截取包含穿刺针的区域之前，方法还包括：
使用超声成像仪器获取被穿刺器官内部图像。
其中，在对获取到的被穿刺器官内部图像截取包含穿刺针的区域之后，方法还包括：
以拟合结果作为穿刺针所在直线的图像在超声原图中实时显示。
其中，在以拟合结果作为穿刺针所在直线的图像在超声原图中实时显示之 后，方法还包括：
当前帧检测完成，若结束显示，则停止运行；若继续显示，则回到步骤对获取到的被穿刺器官内部图像截取包含穿刺针的区域开始下一帧的穿刺针检测工作。
下面以具体实施例，对本申请的基于超声图像的穿刺针实时检测方法进行详细说明：
针对现有技术的缺陷，本申请提出一种基于超声图像的穿刺针实时检测方法，可用于术中对超声图像中的穿刺针进行实时检测和定位，为医生提供穿刺针位置和方向的可视化效果，从而帮助医生提高穿刺准确率。
如图1所示，本申请技术方案的基本内容为：
(ⅰ)使用超声成像仪器获取被穿刺器官内部图像；
(ⅱ)在获取的一帧图像中截取包含穿刺针的区域作为感兴趣区域(ROI)，大小为512×512；
(ⅲ)利用训练好的UNet++网络对图像进行分割，得出穿刺针的识别位置；
(ⅳ)使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
(ⅴ)对该区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线，在超声原图中实时显示；
(ⅵ)当前帧检测完成，若结束显示，则停止运行；若继续显示，则回到步骤(ⅰ)开始下一帧的穿刺针检测工作。
根据流程图1，以下详细说明本申请的技术方案：
1)穿刺针粗分割
深度学习被广泛用于医学图像分割，其中UNet是使用最为频繁且稳定的网络结构之一，现已出现多种改进的UNet网络结构，其中AttentionUNet和NestedUNet是两种比较常见的。其中AttentionUNet在UNet中引入注意力机制，在对网络中编码器识别的每个分辨率上的特征与解码器中对应特征进行拼接之前，使用了一个注意力模块，重新调整了编码器的输出特征。NestedUNet是多特征融合的经典网络，它整合了不同层次的特征，让参数量巨大的深度网络在可接受的精度范围内大幅度的缩减参数量。本申请对比了注意力机制和多尺度特征融合在该任务上的表现，发现后者更适用于该检测任务。
本申请使用dice系数矩阵、精确度和灵敏度来测量每个模型的性能。本申请进一步计算了角度均方根误差(AE，单位：度)和距离均方根误差(DE，单位：厘米)来评估预测的针形轨迹的准确性。为了定量分析本网络的有效性，本申请采集了1230张超声图像，其中984张作为训练数据，246张作为测试数据。表1给出三种方法的测试结果对比，结果显示使用NestedUNet时有最好的检测精度，其检测结果与正确值之间的角度和距离偏差也最小。
网络模型 dice系数 精确度 灵敏度 AE DE
NestedUNet 0.828 0.919 0.759 0.270 0.068
UNet 0.806 0.903 0.732 0.920 0.208
AttentionUNet 0.790 0.892 0.713 0.361 0.259
表1
2)分割结果后处理
因检测结果中可能存在多种干扰，故需对检测结果进行后处理以提取可能的穿刺针区域。在检测结果中穿刺针往往是比较大的连通区域，故遍历各个连通区域，基于阈值删除面积较小的连通区域，得到穿刺针精准分割结果。参见图2为穿刺针检测结果后处理过程图，其中图2(a)为神经网络分割结果图，图2(b)为去除小面积连通域处理结果图。
本申请需实时显示穿刺针方向，所以对检测出的结果进行直线拟合得出穿刺针的穿刺轨迹以确定是否偏离目标。根据上一步得到的穿刺针存在区域，提取出精确分割图像中像素值大于0的点，将其坐标赋值给x、y，用于直线拟合。
Figure PCTCN2022085513-appb-000009
Figure PCTCN2022085513-appb-000010
其中，x i,j表示分割图像在(i,j)位置的像素点的横坐标i，y i,j表示分割图像在(i,j)位置的像素点的纵坐标j，
Figure PCTCN2022085513-appb-000011
为所有横坐标的均值，
Figure PCTCN2022085513-appb-000012
为所有纵坐标的均值。
本申请方法的关键点和欲保护点至少在于：
本申请使用UNet++对穿刺针进行检测，并利用连通域方法对网络输出结果进行了后处理，提高了穿刺针检测精度和运算速度,经实验验证穿刺针实时检测功能运算频次可达6Hz。
本申请通过改进神经网络以及简化后处理方法，提高了穿刺针检测精度和运算速度。
本申请采用自主采集体膜数据进行验证，其方法概念图3所示为穿刺针实时检测示意图。本申请经过体膜实验证明，其检测精度可达91.9％，运算速度 可使得图像实时更新频率达到6.25Hz。
实施例2
根据本申请的另一实施例，提供了一种基于超声图像的穿刺针实时检测装置，包括：
图像截取单元，用于对获取到的被穿刺器官内部图像截取包含穿刺针的区域；
图像分割单元，用于利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；
图像后处理单元，用于使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
像素点拟合单元，用于对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。
本申请实施例中的基于超声图像的穿刺针实时检测装置，对获取到的被穿刺器官内部图像截取包含穿刺针的区域；利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。本申请使用UNet++对穿刺针进行检测，并利用连通域方法对网络输出结果进行了后处理，提高了穿刺针检测精度和运算速度,经实验验证穿刺针实时检测功能运算频次可达6Hz。本申请通过改进神经网络以及简化后处理方法，提高了穿刺针检测精度和运算速度。
下面以具体实施例，对本申请的基于超声图像的穿刺针实时检测装置进行详细说明：
针对现有技术的缺陷，本申请提出一种基于超声图像的穿刺针实时检测装置，可用于术中对超声图像中的穿刺针进行实时检测和定位，为医生提供穿刺针位置和方向的可视化效果，从而帮助医生提高穿刺准确率。
如图1所示，本申请技术方案的基本内容为：
(ⅰ)使用超声成像仪器获取被穿刺器官内部图像；
(ⅱ)图像截取单元：在获取的一帧图像中截取包含穿刺针的区域作为感兴趣区域(ROI)，大小为512×512；
(ⅲ)图像分割单元：利用训练好的UNet++网络对图像进行分割，得出穿刺针的识别位置；
(ⅳ)图像后处理单元：使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
(ⅴ)像素点拟合单元：对该区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线，在超声原图中实时显示；
(ⅵ)当前帧检测完成，若结束显示，则停止运行；若继续显示，则回到步骤(ⅰ)开始下一帧的穿刺针检测工作。
根据流程图1，以下详细说明本申请的技术方案：
1)图像分割单元：穿刺针粗分割
深度学习被广泛用于医学图像分割，其中UNet是使用最为频繁且稳定的网络结构之一，现已出现多种改进的UNet网络结构，其中AttentionUNet和NestedUNet是两种比较常见的。其中AttentionUNet在UNet中引入注意力机制，在对网络中编码器识别的每个分辨率上的特征与解码器中对应特征进行拼接之前，使用了一个注意力模块，重新调整了编码器的输出特征。NestedUNet是多特征融合的经典网络，它整合了不同层次的特征，让参数量巨大的深度网 络在可接受的精度范围内大幅度的缩减参数量。本申请对比了注意力机制和多尺度特征融合在该任务上的表现，发现后者更适用于该检测任务。
本申请使用dice系数矩阵、精确度和灵敏度来测量每个模型的性能。本申请进一步计算了角度均方根误差(AE，单位：度)和距离均方根误差(DE，单位：厘米)来评估预测的针形轨迹的准确性。为了定量分析本网络的有效性，本申请采集了1230张超声图像，其中984张作为训练数据，246张作为测试数据。表1给出三种方法的测试结果对比，结果显示使用NestedUNet时有最好的检测精度，其检测结果与正确值之间的角度和距离偏差也最小。
网络模型 dice系数 精确度 灵敏度 AE DE
NestedUNet 0.828 0.919 0.759 0.270 0.068
UNet 0.806 0.903 0.732 0.920 0.208
AttentionUNet 0.790 0.892 0.713 0.361 0.259
表1
2)分割结果后处理
图像后处理单元：因检测结果中可能存在多种干扰，故需对检测结果进行后处理以提取可能的穿刺针区域。在检测结果中穿刺针往往是比较大的连通区域，故遍历各个连通区域，基于阈值删除面积较小的连通区域，得到穿刺针精准分割结果。参见图2为穿刺针检测结果后处理过程图，其中图2(a)为神经网络分割结果图，图2(b)为去除小面积连通域处理结果图。
像素点拟合单元：本申请需实时显示穿刺针方向，所以对检测出的结果进行直线拟合得出穿刺针的穿刺轨迹以确定是否偏离目标。根据上一步得到的穿刺针存在区域，提取出精确分割图像中像素值大于0的点，将其坐标赋值给x、y，用于直线拟合。
Figure PCTCN2022085513-appb-000013
Figure PCTCN2022085513-appb-000014
其中，x i,j表示分割图像在(i,j)位置的像素点的横坐标i，y i,j表示分割图像在(i,j)位置的像素点的纵坐标j，
Figure PCTCN2022085513-appb-000015
为所有横坐标的均值，
Figure PCTCN2022085513-appb-000016
为所有纵坐标的均值。
本申请装置的关键点和欲保护点至少在于：
本申请使用UNet++对穿刺针进行检测，并利用连通域方法对网络输出结果进行了后处理，提高了穿刺针检测精度和运算速度,经实验验证穿刺针实时检测功能运算频次可达6Hz。
本申请通过改进神经网络以及简化后处理方法，提高了穿刺针检测精度和运算速度。
本申请采用自主采集体膜数据进行验证，其方法概念图3所示为穿刺针实时检测示意图。本申请经过体膜实验证明，其检测精度可达91.9％，运算速度可使得图像实时更新频率达到6.25Hz。
实施例3
一种存储介质，存储介质存储有能够实现上述任意一项基于超声图像的穿刺针实时检测方法的程序文件。
实施例4
一种处理器，处理器用于运行程序，其中，程序运行时执行上述任意一项的基于超声图像的穿刺针实时检测方法。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
在本申请的上述实施例中，对各个实施例的描述都各有侧重，某个实施例 中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的系统实施例仅仅是示意性的，例如单元的划分，可以为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM， Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅是本申请的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (10)

  1. 一种基于超声图像的穿刺针实时检测方法，其特征在于，包括以下步骤：
    对获取到的被穿刺器官内部图像截取包含穿刺针的区域；
    利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；
    使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
    对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。
  2. 根据权利要求1所述的基于超声图像的穿刺针实时检测方法，其特征在于，所述对获取到的被穿刺器官内部图像截取包含穿刺针的区域包括：
    在获取的一帧图像中截取包含穿刺针的区域作为感兴趣区域。
  3. 根据权利要求1所述的基于超声图像的穿刺针实时检测方法，其特征在于，所述利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置包括：
    利用训练好的NestedUNet网络对截取后的图像进行分割。
  4. 根据权利要求3所述的基于超声图像的穿刺针实时检测方法，其特征在于，所述NestedUNet被配置为多特征融合的经典网络，整合了不同层次的特征，让参数量巨大的深度网络在可接受的精度范围内大幅度的缩减参数量。
  5. 根据权利要求1所述的基于超声图像的穿刺针实时检测方法，其特征在于，所述使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域包括：
    遍历各个连通区域，删除面积小于阈值的连通区域，得到穿刺针精准分割 结果。
  6. 根据权利要求1所述的基于超声图像的穿刺针实时检测方法，其特征在于，所述对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线包括：
    在穿刺针精准定位区域提取出精确分割图像中像素值大于0的点，将其坐标赋值给x、y，用于直线拟合；直线拟合系数的表达式如下：
    Figure PCTCN2022085513-appb-100001
    Figure PCTCN2022085513-appb-100002
    其中，x i,j表示分割图像在(i,j)位置的像素点的横坐标i，y i,j表示分割图像在(i,j)位置的像素点的纵坐标j，
    Figure PCTCN2022085513-appb-100003
    为所有横坐标的均值，
    Figure PCTCN2022085513-appb-100004
    为所有纵坐标的均值。
  7. 根据权利要求1所述的基于超声图像的穿刺针实时检测方法，其特征在于，在所述对获取到的被穿刺器官内部图像截取包含穿刺针的区域之前，所述方法还包括：
    使用超声成像仪器获取被穿刺器官内部图像。
  8. 根据权利要求1所述的基于超声图像的穿刺针实时检测方法，其特征在于，在所述对获取到的被穿刺器官内部图像截取包含穿刺针的区域之后，所述方法还包括：
    以拟合结果作为穿刺针所在直线的图像在超声原图中实时显示。
  9. 根据权利要求8所述的基于超声图像的穿刺针实时检测方法，其特征在于，在所述以拟合结果作为穿刺针所在直线的图像在超声原图中实时显示之后，所述方法还包括：
    当前帧检测完成，若结束显示，则停止运行；若继续显示，则回到步骤对 获取到的被穿刺器官内部图像截取包含穿刺针的区域开始下一帧的穿刺针检测工作。
  10. 一种基于超声图像的穿刺针实时检测装置，其特征在于，包括：
    图像截取单元，用于对获取到的被穿刺器官内部图像截取包含穿刺针的区域；
    图像分割单元，用于利用训练好的UNet++网络对截取后的图像进行分割，得出穿刺针的识别位置；
    图像后处理单元，用于使用去除小连通域法对分割的图像进行后处理，得出穿刺针精准定位区域；
    像素点拟合单元，用于对穿刺针精准定位区域的像素点进行直线拟合，并以拟合结果作为穿刺针所在直线。
PCT/CN2022/085513 2022-04-07 2022-04-07 一种基于超声图像的穿刺针实时检测方法及装置 WO2023193175A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/085513 WO2023193175A1 (zh) 2022-04-07 2022-04-07 一种基于超声图像的穿刺针实时检测方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/085513 WO2023193175A1 (zh) 2022-04-07 2022-04-07 一种基于超声图像的穿刺针实时检测方法及装置

Publications (1)

Publication Number Publication Date
WO2023193175A1 true WO2023193175A1 (zh) 2023-10-12

Family

ID=88243777

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/085513 WO2023193175A1 (zh) 2022-04-07 2022-04-07 一种基于超声图像的穿刺针实时检测方法及装置

Country Status (1)

Country Link
WO (1) WO2023193175A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117557486A (zh) * 2024-01-11 2024-02-13 深圳爱递医药科技有限公司 基于超声影像的神经麻醉穿刺辅助定位方法
CN118415727A (zh) * 2024-07-05 2024-08-02 杭州唯精医疗机器人有限公司 穿刺针进针控制方法、控制器和手术机器人

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140323854A1 (en) * 2013-04-25 2014-10-30 Konica Minolta, Inc. Ultrasound diagnostic imaging apparatus and ultrasound image display method
CN110569854A (zh) * 2019-09-12 2019-12-13 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN111368825A (zh) * 2020-02-25 2020-07-03 华南理工大学 一种基于语义分割的指针定位方法
CN112785617A (zh) * 2021-02-23 2021-05-11 青岛科技大学 残差UNet直肠癌肿瘤磁共振图像自动分割方法
CN113362294A (zh) * 2021-05-27 2021-09-07 同济大学 含穿刺针超声血管图像的穿刺针识别方法、系统和设备
CN113409320A (zh) * 2021-05-18 2021-09-17 珠海横乐医学科技有限公司 基于多注意力的肝脏血管分割方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140323854A1 (en) * 2013-04-25 2014-10-30 Konica Minolta, Inc. Ultrasound diagnostic imaging apparatus and ultrasound image display method
CN110569854A (zh) * 2019-09-12 2019-12-13 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN111368825A (zh) * 2020-02-25 2020-07-03 华南理工大学 一种基于语义分割的指针定位方法
CN112785617A (zh) * 2021-02-23 2021-05-11 青岛科技大学 残差UNet直肠癌肿瘤磁共振图像自动分割方法
CN113409320A (zh) * 2021-05-18 2021-09-17 珠海横乐医学科技有限公司 基于多注意力的肝脏血管分割方法及系统
CN113362294A (zh) * 2021-05-27 2021-09-07 同济大学 含穿刺针超声血管图像的穿刺针识别方法、系统和设备

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117557486A (zh) * 2024-01-11 2024-02-13 深圳爱递医药科技有限公司 基于超声影像的神经麻醉穿刺辅助定位方法
CN117557486B (zh) * 2024-01-11 2024-04-19 深圳爱递医药科技有限公司 基于超声影像的神经麻醉穿刺辅助定位方法
CN118415727A (zh) * 2024-07-05 2024-08-02 杭州唯精医疗机器人有限公司 穿刺针进针控制方法、控制器和手术机器人

Similar Documents

Publication Publication Date Title
WO2023193175A1 (zh) 一种基于超声图像的穿刺针实时检测方法及装置
Chu et al. Ultrasonic thyroid nodule detection method based on U-Net network
Mwikirize et al. Convolution neural networks for real-time needle detection and localization in 2D ultrasound
CN113781439B (zh) 超声视频病灶分割方法及装置
WO2017092182A1 (zh) 一种在超声影像中自动识别肝脏肿瘤类型的方法
WO2021208739A1 (zh) 眼底彩照图像血管评估方法、装置、计算机设备和介质
WO2020244098A1 (zh) 一种x射线ct图像中金属针的检测定位方法
CN103371870A (zh) 一种基于多模影像的外科手术导航系统
CN108198174B (zh) 一种心血管ivoct与ivus自动配准方法与装置
CN105654490A (zh) 一种基于超声弹性图像的病变区域提取方法及装置
US11426142B2 (en) Computer vision systems and methods for real-time localization of needles in ultrasound images
Chen et al. Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process
WO2023072146A1 (zh) 一种经腔道的超声自动检查系统、控制方法、计算机可读存储介质及电子设备
CN111047580B (zh) 用于超声分子影像研究的声像图定量分析及综合定量分析方法
Wijata et al. Unbiased validation of the algorithms for automatic needle localization in ultrasound-guided breast biopsies
Hu et al. Deep learning-based automatic endometrium segmentation and thickness measurement for 2D transvaginal ultrasound
CN113689424B (zh) 可自动识别图像特征的超声检查系统及识别方法
Al-Battal et al. Object detection and tracking in ultrasound scans using an optical flow and semantic segmentation framework based on convolutional neural networks
KR20190059440A (ko) 유사환자 자동검색을 통한 진단 보조 시스템 및 방법
Tang et al. A real-time needle tracking algorithm with first-frame linear structure removing in 2D ultrasound-guided prostate therapy
CN110533667A (zh) 基于图像金字塔融合的肺部肿瘤ct影像3d分割方法
Jeffree et al. Cervical Segmentation in Ultrasound Images using Level-set Algorithm
CN114820650A (zh) 一种基于超声图像的穿刺针实时检测方法及装置
Chen et al. 2-D ultrasound strain images for breast cancer diagnosis using nonrigid subregion registration
Pahl et al. Cervix Segmentation in Ultrasound Data Based on Numerical Technique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22936122

Country of ref document: EP

Kind code of ref document: A1