CN117788495A - Lower limb artery CTA blood vessel and plaque segmentation method based on deep learning - Google Patents
Lower limb artery CTA blood vessel and plaque segmentation method based on deep learning Download PDFInfo
- Publication number
- CN117788495A CN117788495A CN202311643056.4A CN202311643056A CN117788495A CN 117788495 A CN117788495 A CN 117788495A CN 202311643056 A CN202311643056 A CN 202311643056A CN 117788495 A CN117788495 A CN 117788495A
- Authority
- CN
- China
- Prior art keywords
- plaque
- cta
- image
- lower limb
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000003141 lower extremity Anatomy 0.000 title claims abstract description 62
- 230000011218 segmentation Effects 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 42
- 210000001367 artery Anatomy 0.000 title claims abstract description 41
- 210000004204 blood vessel Anatomy 0.000 title claims abstract description 32
- 238000013135 deep learning Methods 0.000 title claims abstract description 30
- 238000001514 detection method Methods 0.000 claims abstract description 38
- 238000010606 normalization Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000013136 deep learning model Methods 0.000 claims abstract description 6
- 230000002792 vascular Effects 0.000 claims abstract description 4
- 238000010586 diagram Methods 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 15
- 238000005070 sampling Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 10
- 238000013527 convolutional neural network Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 6
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 2
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 238000012795 verification Methods 0.000 claims description 2
- 238000010968 computed tomography angiography Methods 0.000 description 58
- 208000000575 Arteriosclerosis Obliterans Diseases 0.000 description 8
- 208000030831 Peripheral arterial occlusive disease Diseases 0.000 description 8
- 238000003745 diagnosis Methods 0.000 description 5
- 208000004434 Calcinosis Diseases 0.000 description 4
- 238000002583 angiography Methods 0.000 description 4
- 230000002308 calcification Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 210000001105 femoral artery Anatomy 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000002560 therapeutic procedure Methods 0.000 description 2
- 208000031104 Arterial Occlusive disease Diseases 0.000 description 1
- 206010060965 Arterial stenosis Diseases 0.000 description 1
- 201000001320 Atherosclerosis Diseases 0.000 description 1
- 208000031481 Pathologic Constriction Diseases 0.000 description 1
- 206010057469 Vascular stenosis Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 208000021328 arterial occlusion Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013152 interventional procedure Methods 0.000 description 1
- 230000007654 ischemic lesion Effects 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000036262 stenosis Effects 0.000 description 1
- 208000037804 stenosis Diseases 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a lower limb artery CTA blood vessel and plaque segmentation method based on deep learning, which comprises the following steps: pretreatment of lower limb artery CTA images: acquiring a lower limb artery CTA image, adjusting the CTA image to a standard size, and carrying out normalization processing on a CTA image data value; plaque detection of lower limb artery CTA images: constructing a deep learning model with a plaque detection network and a segmentation network, the plaque detection network identifying plaque from the CTA image and locating a local CTA image containing a plaque region; segmentation of lower limb arterial vessel wall and plaque: and (3) performing external expansion on the local CTA image, then cutting, adjusting the size of the cut local CTA image, normalizing, and inputting the local CTA image into a segmentation network to obtain a CTA image for segmenting the blood vessel wall and the plaque. The plaque positioning method is used for positioning the plaque based on the detection network, the plaque and the vascular wall are segmented through deep learning, manual participation of doctors is not needed, the burden of the doctors is reduced, the efficiency is higher, the objectivity and the accuracy are higher, and the consistency and the repeatability of the segmentation result are better.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a lower limb artery CTA blood vessel and plaque segmentation method based on deep learning.
Background
Lower limb arteriosclerosis obliterans (ASO) are a disease that causes arterial stenosis and occlusion due to peripheral atherosclerosis, and further causes ischemic lesions of Lower limbs, and are clinically common and frequently occurring diseases. There are a number of imaging means currently used for ASO examinations, including color doppler ultrasound, computed tomography angiography imaging (Computed Tomography angiography, CTA), magnetic resonance angiography (Magnetic Resonance Angiography, MRA), digital subtraction angiography (Digital Subtraction Angiography, DSA), etc.
DSA is a gold standard for clinical diagnosis of ASO, but this examination is a invasive examination, and is prone to complications, and the examination cost is high, and is not suitable as a conventional examination method. CTA has higher detection rate for ASO detection and diagnosis. The CTA has the advantages of no wound, simple operation, low cost, short time consumption, low risk and the like, can perform omnibearing rotation according to the actual condition of a patient, fully exposes the information of the lower limb artery, comprehensively scans the related information of the lower limb artery, acquires complete lower limb images, provides basis for subsequent treatment, and becomes a preferred examination mode of ASO.
Currently, ASO treatment protocols are based on interventional therapy. Before ASO interventional therapy, a clinician needs to formulate a treatment scheme according to lower limb CTA images, and select a treatment method such as rotary cutting, balloon or stent expansion according to plaque types displayed on the images and the positions of blood vessels. Typically, the clinician needs to determine the location and type of plaque on the CTA image and determine the stenosis of the blood vessel at the plaque site in order to determine the interventional procedure. Clinically, plaque and vessel walls are manually delineated in CTA images by a physician with high experience to calculate the extent of vascular stenosis.
However, there are a number of disadvantages to this manual sketching approach: (1) The lower limb artery image scanned by CTA has a large number of slice images, so that manual film reading can consume a large amount of time and energy, and a great burden is added to doctors; (2) Depending on clinical experience of doctors, the method has strong subjectivity, and consistency and repeatability of the segmentation result cannot be ensured. Compared with manual sketching of the plaque, the plaque automatic segmentation speed is higher through a computer, and the consistency and the repeatability of the segmentation result can be ensured while the accuracy is ensured.
Machine learning is a method that relies on computer means to learn and extract rules from data and then use these rules to predict unknown data. The deep learning is a branch of machine learning, and the deep learning learns and extracts the features capable of effectively completing the task through an iterative algorithm, so that the dependence on manual feature design and selection is greatly reduced, and more stable and efficient diagnosis is realized. Many studies have proposed semi-or fully-automatic calcification segmentation methods. However, they are mostly directed to coronary calcifications, only a few are applied to superficial femoral artery (Superficial Femoral Artery, SFA) calcification. There is currently no application of the automated calcification segmentation method to intact lower limbs, especially the below knee arteries.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art, and provides a lower limb artery CTA blood vessel and plaque segmentation method based on deep learning.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a lower limb artery CTA blood vessel and plaque segmentation method based on deep learning comprises the following steps:
s1, preprocessing a lower limb artery CTA image: acquiring a lower limb artery CTA image, adjusting the CTA image to a standard size, and carrying out normalization processing on a CTA image data value;
s2, plaque detection of a lower limb artery CTA image: constructing a deep learning model with a plaque detection network and a segmentation network, the plaque detection network identifying plaque from the CTA image and locating a local CTA image containing a plaque region;
s3, dividing the vascular wall and plaque of the lower limb artery: and (3) performing external expansion on the local CTA image, then cutting, adjusting the size of the cut local CTA image, normalizing, and inputting the local CTA image into a segmentation network to obtain a CTA image for segmenting the blood vessel wall and the plaque.
Further, the preprocessing of the lower limb artery CTA image further comprises:
the imaging doctor uses ITK-SNAP software to delineate blood vessels and plaques on the CTA image, and the blood vessels and plaques are used as gold standards for dividing the blood vessels and the plaques after verification; in the sketching process, CTA coronary surface, cross section and sagittal surface images of a patient are required to be mutually referred and supplemented, so that the range of arterial plaque of the lower limb is judged, and finally, the sketching of blood vessels and plaque gold standard is finished on the coronary surface images.
Further, normalizing the CTA image data value includes:
selecting a maximum and minimum normalization method to normalize the gray value of the CTA image, wherein the normalization method has the formula:
wherein X is the original image, X max Is the maximum value of pixel values in X min Is the minimum value of the pixel values in X, and X' is the normalized image.
Further, plaque detection of lower limb artery CTA images, including:
the method comprises the steps of selecting a deep-learning convolutional neural network to execute a plaque detection task, dividing the plaque detection network into a coding stage and a decoding stage, wherein the coding stage is responsible for extracting image features, and the decoding stage is used for expanding and reconstructing a feature map;
the coding stage comprises a convolution module and four downsampling modules, wherein the convolution module comprises a convolution layer, a group normalization layer and a correction linear unit layer, and the downsampling module comprises a pooling layer, a convolution layer, a GN layer and a Relu layer;
the decoding stage restores the feature map reduced in the encoding stage to the original dimension through an up-sampling process and outputs a network result, and the decoding stage comprises a convolution layer and four up-sampling modules, wherein the up-sampling modules comprise a deconvolution layer, a convolution layer, a GN layer and a Relu layer.
Further, the pooling layer is used for reducing the dimension of the feature map and the network parameters and reducing the length and the width of the input image to half of the original length and the width; the deconvolution layer is used to expand the received signature by a factor of two both in length and width.
Further, a connection layer is added after each deconvolution layer, and the connection layer superimposes the high-resolution feature map output by the deconvolution layer corresponding to the up-sampling module in the encoding stage with the low-resolution feature map output by the up-sampling module.
Further, using the Focal Loss as a Loss function of the model, the goal of the Focal Loss is to reduce the weight on samples that are easy to classify, make the model more prone to focus on those samples that are difficult to classify during the training process, and speed up the model convergence, the Loss function is:
wherein p is t Representing probability values, parametersAnd gamma is used for coordination and control of weights.
Further, the segmentation of the arterial vessel wall of the lower limb and plaque comprises:
after the detection and the positioning of the plaque area are completed, a plurality of rectangular frames containing the plaque with different sizes are obtained, 4 pixel points are respectively and externally expanded from left to right and from top to bottom of the obtained rectangular frame, and the area containing the plaque is cut out on an original image based on the externally expanded rectangular frame; to unify the image size fed into the segmentation network, the cropped image is first resized to 64×64; and normalizing the obtained small images for unifying data distribution, and finally inputting the small images into a segmentation network.
Further, the split network has the following differences from the plaque detection network:
(1) To achieve more complex multitasking partitioning, a LeakyReLU is used as the activation function;
(2) Because the input image is cut, the required calculation resources are reduced, and a batch normalization layer is adopted for normalization;
(3) The self-attention module is added at the jump connection part, and the multi-head self-attention module in the transducer is adopted, so that the attention mechanism formula is as follows:
inputting the low-resolution characteristic diagram output by the downsampling module into a multi-head self-attention module;
(4) The output result selects Softmax as the activation function.
Further, the multi-head self-attention module comprises the following steps:
cutting the length and the width of the low-resolution characteristic diagram into a plurality of 16 multiplied by 16 small diagrams, if the length or the width of the characteristic diagram is smaller than 16 or cannot be divided by 16, zero padding is carried out on the characteristic diagram, the position corresponding to zero padding is recorded, the size of the output characteristic diagram is B multiplied by N multiplied by Em, and B represents the number of samples processed by a model at one time; n represents the number of image blocks, n= (W/16) × (H/16), W, H represents the width and height of the input image, respectively; em represents the vector length of encoding the input image block into one low-dimensional vector;
encoding the output of the previous layer through a linear layer and changing the size of the output to obtain Q, K, V in the attention mechanism formula, wherein the shapes are b×nh×n×hd, and NH represents how many different spaces the input is mapped to, hd=em/NH;
q, K, V is calculated according to an attention mechanism formula to obtain an output result of B multiplied by N multiplied by Em;
and removing the zero-filling position, so that the final output result is consistent with the size of the input low-resolution characteristic diagram.
Compared with the prior art, the method and the device have the advantages that based on the deep learning image processing technology and CTA, a deep learning model is built, plaque detection is carried out on the lower limb CTA image, and the lower limb arterial vessel and plaque are segmented based on the detection result, so that automatic segmentation of the complete lower limb arterial plaque and plaque region vessel wall is realized, the burden of doctors is reduced, and the diagnosis and treatment efficiency is improved.
Drawings
Fig. 1 is a flow chart of a lower limb arterial vessel and plaque wall segmentation method based on deep learning.
Fig. 2 is a flowchart of a lower limb arterial vessel and plaque wall segmentation method based on deep learning according to an embodiment.
Fig. 3 is a diagram of a patch detection network structure according to an embodiment.
Fig. 4 is a diagram showing a plaque and blood vessel segmentation network configuration according to an embodiment.
Detailed Description
The lower limb artery CTA blood vessel and plaque segmentation method based on deep learning of the invention is further described below with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1 and 2, the invention discloses a lower limb artery CTA vessel and plaque segmentation method based on deep learning, which comprises the following steps:
s1, preprocessing a lower limb artery CTA image: acquiring a lower limb artery CTA image, adjusting the CTA image to a standard size, and carrying out normalization processing on a CTA image data value;
s2, plaque detection of a lower limb artery CTA image: constructing a deep learning model with a plaque detection network and a segmentation network, the plaque detection network identifying plaque from the CTA image and locating a local CTA image containing a plaque region;
s3, dividing the vascular wall and plaque of the lower limb artery: and (3) performing external expansion on the local CTA image, then cutting, adjusting the size of the cut local CTA image, normalizing, and inputting the local CTA image into a segmentation network to obtain a CTA image for segmenting the blood vessel wall and the plaque.
In order to better utilize image information provided by CTA, lighten doctor burden and improve diagnosis efficiency, the invention carries out plaque detection on a lower limb CTA image based on a deep learning image processing technology and CTA, and carries out segmentation on lower limb arterial blood vessels and plaque based on detection results, thereby realizing automatic segmentation of complete lower limb arterial plaque and plaque region blood vessel walls.
Specifically, in step S1, the imaging physician uses ITK-SNAP software to delineate blood vessels and plaque on the CTA image and verifies the blood vessels and plaque as a golden standard for blood vessel and plaque segmentation. In the sketching process, CTA coronary surface, cross section and sagittal surface images of a patient are required to be mutually referred and supplemented, so that the range of arterial plaque of the lower limb is judged, and finally, the sketching of blood vessels and plaque gold standard is finished on the coronary surface images.
To accommodate the input of CNN (Convolutional Neural Networks, convolutional neural network), it is necessary to adjust the CTA images acquired with different matrix sizes to the same size. The image matrix size of all patient data is counted and a more suitable matrix size is selected as the standard size. If the image size is larger than the standard size, adopting an edge clipping strategy to process, and if the image size is smaller than the standard size, carrying out a surrounding zero filling strategy to process. All CTA image sizes were resized to standard sizes using Matlab functions (im size subroutine, matlab, natick, MA, USA).
Since the span of data values of the original CTA image is large, this may reduce the convergence speed of CNN, increasing the complexity of feature learning. By normalizing the image data values, they can be scaled to a smaller interval, which not only helps to accelerate the gradient descent process, but also may help to improve segmentation accuracy. The invention selects the maximum and minimum normalization method to normalize the gray value in the image, and the formula of the normalization method is as follows:
wherein X is the original image, X max Is the maximum value of pixel values in X min Is the minimum value of the pixel values in X, and X' is the normalized image.
Specifically, in step S2, in view of the relatively large size of the CTA image processed by the present invention, but the proportion of the images occupied by the plaque and the blood vessel wall is relatively small, if the plaque and the blood vessel wall are directly segmented into the whole image, the segmentation performance is easily affected by other irrelevant background information, and is reduced. Therefore, the present invention first requires the construction of a detection network to automatically identify plaque from CTA images and locate smaller rectangular frames containing plaque areas. By the aid of the method, background information can be greatly reduced, and interference of redundant information on a segmentation result is reduced.
Referring to fig. 3, the present invention selects CNN (Convolutional Neural Networks, convolutional neural network) in deep learning to perform plaque detection task, and the network of the present invention constructs a classical U-net segmentation network based on and optimizes the same. The structure of the network is divided into two phases, an encoding phase and a decoding phase. The encoding stage is responsible for extracting image features, and the decoding stage is used for expanding and reconstructing feature images so as to obtain segmentation results.
In the encoding phase, the encoding phase includes one convolution module and four downsampling modules. Each convolution module includes a convolution layer, a group normalization (Group Normalization, GN) layer, and a modified linear element (Rectify Linear Unit, relu) layer. Each downsampling module includes a pooling layer, a convolution layer, a GN layer, and a Relu layer.
The convolution layer is responsible for extracting basic features of the CTA image and arranging and combining the features to obtain deep features with more abstract semantic information. Limited by memory resources, large batch training networks cannot be used. Thus, in the case of a small batch (2 is used in this example), the present invention selects a GN layer that is insensitive to the size of the batch in order to avoid using BN (Batch Normalization ) to cause degradation in network performance.
The GN layer is a deep learning normalization method, can effectively accelerate convergence, and improves stability of network performance. The pooling layer is used for reducing the dimension of the feature map and the network parameters and reducing the length and the width of the input image to half of the original length and the width. The Relu layer increases the nonlinear relation among all layers of the network, is beneficial to accelerating convergence, and enables the model to be better suitable for complex task scenes.
The decoding stage is to restore the feature map reduced in the encoding stage to the original size and output the network result. The decoding stage includes 1 convolutional layer and 4 upsampling modules. The upsampling module includes 1 deconvolution layer, 1 convolution layer, one GN layer, and 1 Relu layer, each deconvolution layer being followed by a connection layer. The deconvolution layer functions to double the received signature both in length and width. However, during upsampling, bilinear interpolation will result in the loss of much detail information after the image is enlarged, further reducing the resolution of the image.
Referring to fig. 4, a connection layer is added after each deconvolution layer, and the connection layer superimposes a high resolution feature map output by a convolution layer corresponding to an up-sampling module in the encoding stage with a low resolution feature map output by the up-sampling module. The connection layer can integrate global context information and feature fusion, so that a better segmentation effect is achieved. The present invention uses Focal loss as the loss function of the model as shown in equation (2). It is a result of improvement over the standard cross entropy loss function. The design goal of Focal Loss is to reduce the weight on samples that are easy to classify, making the model more prone to focus on those samples that are difficult to classify during the training process, and to increase the model convergence rate. The loss function Focal loss of the model is:
wherein p is t Representing probability values, parametersAnd gamma is used for coordination and control of weights. The invention judges the detection classification performance of the network by using the accuracy rate, and the higher the accuracy rate is, the more the network automatically detects correct results.
Specifically, in step S3, after the detection and positioning of the plaque area are completed, a number of rectangular frames containing plaques of different sizes are obtained. And (3) expanding the left, right, upper and lower parts of the obtained rectangular frame by 4 pixel points, and cutting out an area containing the plaque on an original image based on the expanded rectangular frame. In order to unify the image sizes fed into the segmentation network, the cropped image is first unified to 64×64. Meanwhile, in order to unify data distribution, the obtained small images are normalized and finally input into a designed segmentation network.
In the embodiment, DSC, F1-Measure, accuracy, sensitivity, specificity, accuracy, false positive rate and false negative rate indexes are adopted to evaluate the performances of the plaque detection network and the segmentation network. The calculation method of DSC, F1-Measure, accuracy, sensitivity, specificity, accuracy, false positive rate and false negative rate indexes is as follows:
DSC＝2TP/(FP+2TP+FN)，F1-Measure＝2TP/(2TP+FP+FN)；
accuracy = TP/(tp+fp), sensitivity = TP/(tp+fn);
specificity = TN/(tn+fp), accuracy accuracy= (tp+tn)/(tp+tn+fp+fn);
pseudo-positive ratio=fp/(tp+fp), pseudo-negative ratio=fn/(tp+fn).
Among them, TP (True Positive) is determined as a Positive sample, and is actually the total number of samples of the Positive sample. TN (True Negative), which is determined as a Negative sample, is in fact the total number of samples of the Negative sample. FP (False Positive) is determined as a Positive sample, but is actually the total number of samples of the negative sample. FN (False Negative) is determined as a Negative sample, but is actually the total number of samples of the positive sample.
Referring to fig. 4, the segmentation network of the present invention makes the following modifications on the basis of the plaque detection network:
(1) To achieve more complex multi-tasking segmentation, the present invention uses a LeakyReLU as the activation function. The output range of the LeakyReLU is wider than that of ReLU, and can provide more abundant information.
(2) As the input image is cropped, the computational resources required are reduced, and a batch normalization layer (Batch Normalization, BN)) is employed for normalization. When the batch size is sufficient, BN can provide relatively stable performance through estimation of the mean and variance of the entire batch data.
(3) The invention adds a self-attention module in the jump connection part. Unlike the direct superposition feature fusion mode adopted in the standard U-net, the invention adopts a multi-head self-attention module in the transducer in order to pay attention to the effective features and reduce the attention to redundant features. The attention mechanism formula is as follows:
the invention inputs the low-resolution characteristic diagram output by the downsampling module into the multi-head self-attention module.
A multi-headed self-attention module comprising the steps of:
firstly, cutting the length and the width of a low-resolution characteristic diagram into a plurality of 16×16 small diagrams, if the length or the width of the characteristic diagram is smaller than 16 or cannot be divided by 16, zero padding is carried out on the characteristic diagram, the position corresponding to zero padding is recorded, the size of the output characteristic diagram is B×N×Em, wherein B represents the number of samples processed by a model at one time; n represents the number of image blocks, n= (W/16) × (H/16), W, H represents the width and height of the input image, respectively; em denotes a vector length for encoding an input image block into one low-dimensional vector.
The output of the previous layer was encoded through a linear layer and the output was varied in size to yield Q, K, V in the attention mechanism formula, all in the shape B x NH x N x Hd, where NH represents how many different spaces the input was mapped into, hd=em/NH.
Q, K, V is calculated according to the attention mechanism formula, and the output result is BXNXEm.
And finally, removing the zero-filling position to enable the final output result to be consistent with the size of the input low-resolution characteristic diagram.
After the low-resolution feature map passes through the self-attention module, the obtained output is overlapped with the corresponding high-resolution feature map in the channel dimension, and the overlapped result is used as the input of the up-sampling module. After the decoding stage, the model outputs a segmentation result consistent with the input size.
(4) The output result selects Softmax as the activation function, the output shape behind the convolution layer is (B, 3, X, Y), X and Y represent the length and width of the output, respectively, where x=y=64; and 3, outputting three-channel results, wherein the three-channel results respectively represent the predicted results of blood vessels, plaques and backgrounds.
In summary, the invention constructs a deep learning model based on the deep learning image processing technology and CTA, carries out plaque detection on the lower limb CTA image, and segments the lower limb arterial vessel and plaque based on the detection result, thereby realizing automatic segmentation of the vessel wall of the complete lower limb arterial plaque and plaque region, reducing the burden of doctors and improving diagnosis and treatment efficiency.
The foregoing description is directed to the preferred embodiments of the present invention, but the embodiments are not intended to limit the scope of the invention, and all equivalent changes or modifications made under the technical spirit of the present invention should be construed to fall within the scope of the present invention.
Claims (10)
1. The lower limb artery CTA blood vessel and plaque segmentation method based on deep learning is characterized by comprising the following steps of:
s1, preprocessing a lower limb artery CTA image: acquiring a lower limb artery CTA image, adjusting the CTA image to a standard size, and carrying out normalization processing on a CTA image data value;
s2, plaque detection of a lower limb artery CTA image: constructing a deep learning model with a plaque detection network and a segmentation network, the plaque detection network identifying plaque from the CTA image and locating a local CTA image containing a plaque region;
s3, dividing the vascular wall and plaque of the lower limb artery: and (3) performing external expansion on the local CTA image, then cutting, adjusting the size of the cut local CTA image, normalizing, and inputting the local CTA image into a segmentation network to obtain a CTA image for segmenting the blood vessel wall and the plaque.
2. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 1, wherein the preprocessing of the lower limb artery CTA image further comprises:
the imaging doctor uses ITK-SNAP software to delineate blood vessels and plaques on the CTA image, and the blood vessels and plaques are used as gold standards for dividing the blood vessels and the plaques after verification; in the sketching process, CTA coronary surface, cross section and sagittal surface images of a patient are required to be mutually referred and supplemented, so that the range of arterial plaque of the lower limb is judged, and finally, the sketching of blood vessels and plaque gold standard is finished on the coronary surface images.
3. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 1, wherein normalizing CTA image data values comprises:
selecting a maximum and minimum normalization method to normalize the gray value of the CTA image, wherein the normalization method has the formula:
wherein X is the original image, X max Is the maximum value of pixel values in X min Is the minimum value of the pixel values in X, and X' is the normalized image.
4. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 1, wherein plaque detection of the lower limb artery CTA image comprises:
the method comprises the steps of selecting a deep-learning convolutional neural network to execute a plaque detection task, dividing the plaque detection network into a coding stage and a decoding stage, wherein the coding stage is responsible for extracting image features, and the decoding stage is used for expanding and reconstructing a feature map;
the coding stage comprises a convolution module and four downsampling modules, wherein the convolution module comprises a convolution layer, a group normalization layer and a correction linear unit layer, and the downsampling module comprises a pooling layer, a convolution layer, a GN layer and a Relu layer;
the decoding stage restores the feature map reduced in the encoding stage to the original dimension through an up-sampling process and outputs a network result, and the decoding stage comprises a convolution layer and four up-sampling modules, wherein the up-sampling modules comprise a deconvolution layer, a convolution layer, a GN layer and a Relu layer.
5. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 4, wherein the pooling layer is used for reducing the dimension of the feature map and the network parameters and reducing the length and the width of the input image to half of the original values; the deconvolution layer is used to expand the received signature by a factor of two both in length and width.
6. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 4, wherein a connection layer is added after each deconvolution layer, and the connection layer superimposes a high-resolution feature map output by a convolution layer corresponding to the up-sampling module in the encoding stage with a low-resolution feature map output by the up-sampling module.
7. The deep learning based lower extremity arterial CTA vessel and plaque segmentation method according to claim 6, wherein Focal Loss is used as a Loss function of the model, the goal of Focal Loss is to reduce the weight of samples easy to classify, make the model more prone to focus on those samples difficult to classify during training, and increase the model convergence rate, the Loss function is:
wherein p is t Representing probability values, parametersAnd gamma is used for coordination and control of weights.
8. The deep learning-based lower limb artery CTA vessel and plaque segmentation method of claim 7, wherein the lower limb artery vessel wall and plaque segmentation method comprises:
after the detection and the positioning of the plaque area are completed, a plurality of rectangular frames containing the plaque with different sizes are obtained, 4 pixel points are respectively and externally expanded from left to right and from top to bottom of the obtained rectangular frame, and the area containing the plaque is cut out on an original image based on the externally expanded rectangular frame; to unify the image size fed into the segmentation network, the cropped image is first resized to 64×64; and normalizing the obtained small images for unifying data distribution, and finally inputting the small images into a segmentation network.
9. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 8, wherein the segmentation network has the following differences from the plaque detection network:
(1) To achieve more complex multitasking partitioning, a LeakyReLU is used as the activation function;
(2) Because the input image is cut, the required calculation resources are reduced, and a batch normalization layer is adopted for normalization;
(3) The self-attention module is added at the jump connection part, and the multi-head self-attention module in the transducer is adopted, so that the attention mechanism formula is as follows:
inputting the low-resolution characteristic diagram output by the downsampling module into a multi-head self-attention module;
(4) The output result selects Softmax as the activation function.
10. The deep learning-based lower limb artery CTA vessel and plaque segmentation method according to claim 9, wherein the multi-head self-attention module comprises the following steps:
cutting the length and the width of the low-resolution characteristic diagram into a plurality of 16 multiplied by 16 small diagrams, if the length or the width of the characteristic diagram is smaller than 16 or cannot be divided by 16, zero padding is carried out on the characteristic diagram, the position corresponding to zero padding is recorded, the size of the output characteristic diagram is B multiplied by N multiplied by Em, and B represents the number of samples processed by a model at one time; n represents the number of image blocks, n= (W/16) × (H/16), W, H represents the width and height of the input image, respectively; em represents the vector length of encoding the input image block into one low-dimensional vector;
encoding the output of the previous layer through a linear layer and changing the size of the output to obtain Q, K, V in the attention mechanism formula, wherein the shapes are b×nh×n×hd, and NH represents how many different spaces the input is mapped to, hd=em/NH;
q, K, V is calculated according to an attention mechanism formula to obtain an output result of B multiplied by N multiplied by Em;
and removing the zero-filling position, so that the final output result is consistent with the size of the input low-resolution characteristic diagram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311643056.4A CN117788495A (en) | 2023-12-04 | 2023-12-04 | Lower limb artery CTA blood vessel and plaque segmentation method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311643056.4A CN117788495A (en) | 2023-12-04 | 2023-12-04 | Lower limb artery CTA blood vessel and plaque segmentation method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117788495A true CN117788495A (en) | 2024-03-29 |
Family
ID=90388234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311643056.4A Pending CN117788495A (en) | 2023-12-04 | 2023-12-04 | Lower limb artery CTA blood vessel and plaque segmentation method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117788495A (en) |
-
2023
- 2023-12-04 CN CN202311643056.4A patent/CN117788495A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112927240B (en) | CT image segmentation method based on improved AU-Net network | |
| Huang et al. | Coronary artery segmentation by deep learning neural networks on computed tomographic coronary angiographic images | |
| CN111354002A (en) | Kidney and kidney tumor segmentation method based on deep neural network | |
| CN112991365B (en) | Coronary artery segmentation method, system and storage medium | |
| CN112785551A (en) | Coronary artery segmentation method based on deep learning | |
| CN112734755A (en) | Lung lobe segmentation method based on 3D full convolution neural network and multitask learning | |
| CN111899165A (en) | Multi-task image reconstruction convolution network model based on functional module | |
| CN116503607B (en) | CT image segmentation method and system based on deep learning | |
| CN117495876B (en) | Coronary artery image segmentation method and system based on deep learning | |
| CN112562058B (en) | Method for quickly establishing intracranial vascular simulation three-dimensional model based on transfer learning | |
| CN115830016B (en) | Medical image registration model training method and equipment | |
| CN113034507A (en) | CCTA image-based coronary artery three-dimensional segmentation method | |
| CN116563533A (en) | Medical image segmentation method and system based on target position priori information | |
| CN113935976A (en) | Method and system for automatically segmenting blood vessels in internal organs by enhancing CT (computed tomography) image | |
| CN116309524A (en) | Method and system for suppressing imaging artifacts of cardiac magnetic resonance movies based on deep learning | |
| CN114565601A (en) | Improved liver CT image segmentation algorithm based on DeepLabV3+ | |
| Lu et al. | Two-stage self-supervised cycle-consistency transformer network for reducing slice gap in MR images | |
| EP4315238A1 (en) | Methods and systems for biomedical image segmentation based on a combination of arterial and portal image information | |
| Cheung | State-of-the-art deep learning method and its explainability for computerized tomography image segmentation | |
| CN111599448B (en) | Multi-view shape constraint system and method for specific coronary artery calcification analysis | |
| CN117635625A (en) | Pancreatic tumor segmentation method based on automatic data enhancement strategy and multi-attention-assisted UNet | |
| CN117523204A (en) | Liver tumor image segmentation method and device oriented to medical scene and readable storage medium | |
| CN116612090A (en) | System and method for determining pulmonary artery embolism index | |
| CN116843697A (en) | Two-stage 3D coronary artery segmentation reconstruction method and system | |
| CN117788495A (en) | Lower limb artery CTA blood vessel and plaque segmentation method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |