CN117788616A - Digital subtraction angiography image generation method based on deep learning - Google Patents

Digital subtraction angiography image generation method based on deep learning Download PDF

Info

Publication number
CN117788616A
CN117788616A CN202311711193.7A CN202311711193A CN117788616A CN 117788616 A CN117788616 A CN 117788616A CN 202311711193 A CN202311711193 A CN 202311711193A CN 117788616 A CN117788616 A CN 117788616A
Authority
CN
China
Prior art keywords
generator
dsa
data
network
discriminator
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311711193.7A
Other languages
Chinese (zh)
Inventor
陈阳
茅伟龙
高雨枫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN202311711193.7A priority Critical patent/CN117788616A/en
Publication of CN117788616A publication Critical patent/CN117788616A/en
Pending legal-status Critical Current

Links

Landscapes

  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to a deep learning method of generating digital subtraction angiography (DSA, digital subtraction angiography) from a single film image from which DSA images are generated without a mask image. The main innovation point of the method is that a residual error dense block (RDB, residual Dense Block) is adopted to directly generate a DSA image from a single image-filling image, and the RDB extracts advanced features through a dense connection layer. To obtain better vascular detail, supervised generation of countermeasure network policies is used during the training phase. The network is composed of a generator network that generates DSA images and a discriminator network that judges whether the input image is a generated DSA image.

Description

Digital subtraction angiography image generation method based on deep learning
Technical Field
The invention relates to a digital subtraction angiography image generation method, in particular to a deep learning-based method for generating a DSA image from a single image, and belongs to the technical field of image generation.
Background
Artificial intelligence (AI, artificial intelligence) has experienced rapid rising and continuing developments since birth. AI began in the 50 s of the last century, but until the last decades AI began to proliferate due to the increase in computing power, the massive accumulation of data, and the improvement of algorithms. AI has made significant breakthroughs in a number of areas such as natural language processing, computer vision and machine learning.
The rise and development of artificial intelligence has faced a key problem of image processing algorithms, namely how to efficiently extract useful features in images, prior to the introduction of deep learning techniques. Conventional approaches rely on manual design features, but there are two limitations to this: firstly, the manual feature extraction is limited by professional experience, and effective features may not be comprehensively extracted; secondly, the artificial feature fusion mode is limited, so that the extracted information cannot be fully utilized. However, the deep learning algorithm can autonomously learn the feature extraction and fusion method through a large number of training processes, which is one of the reasons why deep learning has achieved great achievements in the field of machine vision.
Coronary angiography images (DSA, digital subtraction angiography) are gold standards for clinical analysis of vascular disease. And (3) performing subtraction, enhancement and re-imaging on two frames of X-ray images shot before and after the contrast agent is injected through digital image processing, and eliminating bone and soft tissue images on angiography images to obtain clear pure vascular images. Coronary angiography images provide useful information for cardiovascular diagnosis and surgery. In the challenging task of coronary angiography images, conventional machine vision algorithms extract vessel information by making differences after registering the mask to the mask. Because of the difference of the mask and the interference mask on the vascular structure, the complexity of the human skeleton structure and the like, the image registration of the artificial design features is very difficult.
Disclosure of Invention
The invention aims at the problems existing in the prior art, and provides a deep learning method for generating digital subtraction angiography from a single image, which uses a deep learning method and a proposed network model to perform subtraction generation on a coronary angiography image, so that the dependence on a registration algorithm in a traditional algorithm is avoided, and artifacts in a subtraction result are reduced.
In order to achieve the above object, the technical scheme of the present invention is as follows, a digital subtraction angiography image generation method based on deep learning, the method comprising the following steps:
step 1: a sequence of digital subtraction angiography system images is obtained, data from both the human head and leg portions. All data are from a clinical system with standard injection protocols, with no contrast injected in the first frame of the sequence and with progressive filling of contrast in the subsequent frames;
step 2: the dataset is divided into two parts according to whether the DSA image contains motion artifacts or not according to subjective assessment. The artifact free data, i.e. the data containing the complete vascular structure, is used for training and testing. The data containing artifacts are used only for testing and visual assessment.
Step 3: selecting a certain frame in the sequence as a subtraction frame, namely, the subtraction model aims at obtaining a pixel regression result of a DSA image corresponding to the frame;
step 4: carrying out gray scale normalization on all images in the sequence;
step 5: the data enhancement methods such as turning, translational scaling, rotation and the like are carried out on all images in the sequence;
step 6: writing a generator model based on a depth residual error network, wherein the network comprises a film-filling input end and a DSA image output end;
step 7: writing a depth residual error network-based discriminator model, wherein the network comprises a DSA image input end and a discrimination result output end;
step 8: according to the convolutional neural network written in the step 6, optimizing the output of the generator network by adopting a cross entropy loss function and a mean square error loss function;
step 9: optimizing the output of the discriminator by adopting a cross entropy loss function according to the convolutional neural network written in the step 7;
step 10: according to the steps 8 and 9, the two networks are cross trained until convergence, and then the vascular subtraction network model is obtained.
As an improvement of the invention, the generator model architecture and the discriminant model architecture are detailed:
the model is divided into two networks of a generator and a discriminator; one generator takes the tensor of the subtracted frame data as input, takes the corresponding label of the subtracted frame as output supervision, and is called as a generating network; the discriminator takes real DSA data and DSA data generated by the generator as input, and the data source is output supervision, which is called discriminating network; the generator network generates DSA images and the role of the arbiter is to classify a given input sample, judging whether it is from the generator or real data. The generator and the arbiter compete and cooperate with each other during the training process. The goal of the generator is to generate as realistic DSA subtraction data as possible to fool the arbiter; the goal of the arbiter is to judge the source of the sample as accurately as possible to distinguish between DSA data produced by the generator and real DSA data.
As an improvement of the present invention, the generator architecture details are as follows: generator G contains 5 basic blocks, each of which contains 6 convolutional layers. The convolution kernel size of these layers is 3×3 and the number of filters is 128. In addition to the basic block, there are 4 convolutional layers and 2 deconvolution layers. The convolution kernel size for all these layers is 3 x 3. In these 6 layers, the number of filters is 32, 64, 128, 64, 32, and 1, respectively. The steps of the first layer and the last layer are 1 and the steps of the other layers are 2.
As an improvement of the present invention, the arbiter architecture details are as follows:
the arbiter D is a typical CNN architecture, comprising three convolutional layers and three fully-connected layers, where the first two layers use the prilu as the activation function and the last layer uses the sigmoid function to output probabilities. The convolution kernel size of the convolution layer is 3 x 3, the number of filters is 32, 64, and 128, respectively. The number of nodes in the fully connected layer is 512, 128 and 1, respectively.
As an improvement of the invention, the generator G loss function details:
wherein:
x is the output of the algorithm, Y is the subtraction tag, D is the discriminator, G is the generator, W is wide, and H is high.
As an improvement of the present invention, the said arbiter D loss function details:
wherein:
x is a filling piece, Y is a type of input data of 0 or 1,is the probability predicted by the discriminator, and the value range is 0,1]Between them.
As an improvement of the invention, the generator and discriminant training process details are as follows:
the subtracted frames are input to a generator G, and then the network G is optimized by a loss function L combining the supervised loss and the discriminator loss. The predictions of DSA image and the pair of film-forming images are marked as "negative" labels to confuse the discriminator so that loss of the discriminator can lead the generator to generate more realistic and accurate vessel images. To ensure that the whole network can converge and stabilize, the discriminator D is trained once and then the generator G is optimised twice.
The present invention has the advantage over the prior art that it utilizes a residual dense block (RDB, residual Dense Block) and extracts advanced features through dense connection layers, directly generating DSA images from a single real-time image. In order to obtain better vascular details, the training stage also uses a supervision generation countermeasure network strategy, (1) compared with a general registration-based traditional method, the method does not need to acquire mask images, (2) compared with the general registration-based traditional method, the method is not affected by the mask and interference pattern registration effects, and (3) the method uses the supervision generation countermeasure network strategy in the training stage to obtain better vascular details.
Drawings
FIG. 1 is a schematic diagram of a generator neural network model of the digital subtraction angiography image generation method based on deep learning of the present invention;
FIG. 2 is a schematic diagram of a model of a neural network of a arbiter of the digital subtraction angiography image generation method based on deep learning of the present invention;
FIG. 3 is a schematic diagram of a training flow of the digital subtraction angiography image generation method based on deep learning of the present invention;
FIG. 4 is a schematic diagram of a test flow of the digital subtraction angiography image generation method based on deep learning of the present invention;
FIG. 5 is a schematic representation of a coronary angiography image of the digital subtraction angiography image generation method based on deep learning of the invention;
fig. 6 is a schematic illustration of subtraction result of the digital subtraction angiography image generation method based on deep learning according to the present invention.
The specific embodiment is as follows:
in order to enhance the understanding of the present invention, the present embodiment will be described in detail with reference to the accompanying drawings.
Example 1: referring to fig. 1, a method for generating a digital subtraction angiography image based on deep learning, the method comprising the steps of:
step 1: a sequence of digital subtraction angiography system images is obtained, data from both the human head and leg portions. All data are from a clinical system with standard injection protocols, with no contrast injected in the first frame of the sequence and with progressive filling of contrast in the subsequent frames;
step 2: the dataset is divided into two parts according to whether the DSA image contains motion artifacts or not according to subjective assessment. The artifact free data, i.e. the data containing the complete vascular structure, is used for training and testing. The data containing artifacts are used only for testing and visual assessment.
Step 3: selecting a certain frame in the sequence as a subtraction frame, namely, the subtraction model aims at obtaining a pixel regression result of a DSA image corresponding to the frame;
step 4: carrying out gray scale normalization on all images in the sequence;
step 5: the data enhancement methods such as turning, translational scaling, rotation and the like are carried out on all images in the sequence;
step 6: inputting the film-filling data into a generator model based on a depth residual error network, wherein the network comprises a film-filling input end and a DSA image output end;
step 7: outputting the DSA label and the DSA image in the step 6 to a discriminator model of a depth residual error network respectively, wherein the network comprises a DSA image input end and a discrimination result output end;
step 8: optimizing by adopting a cross entropy loss function and a mean square error loss function according to the output of the convolutional neural network in the step 6 and the label in the step 5;
step 9: according to the output of the convolutional neural network in the step 7, taking a data source as a label, and optimizing the output of the discriminator by adopting a cross entropy loss function;
step 10: according to the steps 8 and 9, the two networks are cross trained until convergence, and then the vascular subtraction network model is obtained.
The generator model architecture and the arbiter model architecture are specifically as follows:
the model is divided into two networks of a generator and a discriminator; one generator takes the tensor of the subtracted frame data as input, takes the corresponding label of the subtracted frame as output supervision, and is called as a generating network; the discriminator takes real DSA data and DSA data generated by the generator as input, and the data source is output supervision, which is called discriminating network; the generator network generates DSA images and the role of the arbiter is to classify a given input sample, judging whether it is from the generator or real data. The generator and the arbiter compete and cooperate with each other during the training process. The goal of the generator is to generate as realistic DSA subtraction data as possible to fool the arbiter; the goal of the arbiter is to judge the source of the sample as accurately as possible to distinguish between DSA data produced by the generator and real DSA data.
The generator architecture details are as follows:
generator G contains 5 basic blocks, each of which contains 6 convolutional layers. The convolution kernel size of these layers is 3×3 and the number of filters is 128. In addition to the basic block, there are 4 convolutional layers and 2 deconvolution layers. The convolution kernel size for all these layers is 3 x 3. In these 6 layers, the number of filters is 32, 64, 128, 64, 32, and 1, respectively. The steps of the first layer and the last layer are 1 and the steps of the other layers are 2.
The details of the arbiter architecture are as follows:
the arbiter D is a typical CNN architecture, comprising three convolutional layers and three fully-connected layers, where the first two layers use the prilu as the activation function and the last layer uses the sigmoid function to output probabilities. The convolution kernel size of the convolution layer is 3
X 3, the number of filters was 32, 64 and 128, respectively. The number of nodes in the fully connected layer is 512, 128 and 1, respectively. The generator G loss function is as follows:
wherein:
x is a missing piece, Y is a label of an input pair, lambda is a weight between loss functions, and the value range is between 0 and 1.
The details of the arbiter D loss function are as follows:
wherein:
x is a filling sheet, Y is a label of an input pair, which takes a value of 0 or 1, andis the probability predicted by the discriminator, and the value range is 0,1]Between them. The generator and arbiter training process details are as follows:
the subtracted frames are input to a generator G, and then the network G is optimized by a loss function L combining the supervised loss and the discriminator loss. The predictions of DSA image and the pair of film-forming images are marked as "negative" labels to confuse the discriminator so that loss of the discriminator can lead the generator to generate more realistic and accurate vessel images. To ensure that the whole network can converge and stabilize, the discriminator D is trained once and then the generator G is optimized twice until convergence.
Specific examples: the invention discloses a digital subtraction angiography image generation method based on deep learning, which comprises a neural network model, a training process and a testing process.
Referring to fig. 1 and 2, the architecture of the neural network model in the method of the present invention is:
generator G contains 5 basic blocks, each of which contains 6 convolutional layers. The convolution kernel size of these layers is 3×3 and the number of filters is 128. In addition to the basic block, there are 4 convolutional layers and 2 deconvolution layers. The convolution kernel size for all these layers is 3 x 3. In these 6 layers, the number of filters is 32, 64, 128, 64, 32, and 1, respectively. The steps of the first layer and the last layer are 1 and the steps of the other layers are 2.
The arbiter D is a typical CNN architecture, comprising three convolutional layers and three fully-connected layers, where the first two layers use the prilu as the activation function and the last layer uses the sigmoid function to output probabilities. The convolution kernel size of the convolution layer is 3 x 3, the number of filters is 32, 64, and 128, respectively. The number of nodes in the fully connected layer is 512, 128 and 1, respectively. .
Referring to fig. 3, the process of training the silhouette generation network in the method of the present invention is as follows:
step 1, acquiring a coronary angiography image sequence,
step 2, selecting a certain frame in the sequence as a segmented frame image;
step 3, carrying out gray scale normalization on all image data, wherein a gray scale normalization formula is as follows:
wherein I (I, j), N (I, j) represent the original image and the normalized imageGray values of (i, j), min, max represent the minimum gray value and the maximum gray value in the original image, respectively;
step 4, randomly selecting a frame of image data to be subtracted, and inputting the frame of image data into a network model;
step 5, optimizing the output of the generator network by adopting a cross entropy loss function and a mean square error loss function, and optimizing the output of the discriminator by adopting the cross entropy loss function;
and 6, judging whether the model is converged, if so, storing the model, and if not, returning to the step 2.
Referring to fig. 4, the process of testing the subtraction result of deep learning in the method of the present invention is as follows:
step 1, obtaining test data and obtaining an image to be subtracted;
step 2, carrying out gray scale normalization on all image data, wherein a gray scale normalization formula is as follows;
wherein I (I, j), N (I, j) represent the original image and the normalized imageGray values of (i, j), min, max represent the minimum gray value and the maximum gray value in the original image, respectively;
step 3, selecting a frame of full image of contrast agent, and inputting the frame of full image into the trained network model;
step 4, obtaining a digital subtraction result;
it should be noted that the above-mentioned embodiments are not intended to limit the scope of the present invention, and equivalent changes or substitutions made on the basis of the above-mentioned technical solutions fall within the scope of the present invention as defined in the claims.

Claims (7)

1. A method of generating a digital subtraction angiography image based on deep learning, the method comprising the steps of:
step 1: obtaining a sequence of images of a digital subtraction angiography system, data from both the head and leg portions of a human, all data from a clinical system with a standard injection protocol, no contrast agent injected in a first frame of the sequence, and a gradual filling of contrast agent in a subsequent frame;
step 2: the dataset is divided into two parts according to subjective assessment, according to whether the DSA image contains motion artifacts, data without artifacts, i.e. data containing complete vascular structures, for training and testing, data containing artifacts only for testing and visual assessment,
step 3: selecting a certain frame in the sequence as a subtraction frame, namely, the subtraction model aims at obtaining a pixel regression result of a DSA image corresponding to the frame;
step 4: carrying out gray scale normalization on all images in the sequence;
step 5: performing a flipping, panning scaling and rotation data enhancement method for all images in the sequence;
step 6: writing a generator model based on a depth residual error network, wherein the network comprises a film-filling input end and a DSA image output end;
step 7: writing a depth residual error network-based discriminator model, wherein the network comprises a DSA image input end and a discrimination result output end;
step 8: according to the convolutional neural network written in the step 6, optimizing the output of the generator network by adopting a cross entropy loss function and a mean square error loss function;
step 9: optimizing the output of the discriminator by adopting a cross entropy loss function according to the convolutional neural network written in the step 7;
step 10: according to the steps 8 and 9, the two networks are cross trained until convergence, and then the vascular subtraction network model is obtained.
2. The method for generating a digital subtraction angiography image based on deep learning according to claim 1, wherein in the step 6: the countermeasure strategies of the generator model architecture and the arbiter model architecture are specifically as follows:
the model is divided into two networks of a generator and a discriminator; one generator takes the tensor of the piece-filling data as input, takes the corresponding label of the piece-filling data as output supervision, and is called as a generating network; the discriminator takes real DSA data and DSA data generated by the generator as input, and the data source is output supervision, which is called discriminating network; the generator network generates DSA images and the role of the arbiter is to classify a given input sample, judging whether it is from a generator or a real label. The generator and the discriminant compete and cooperate with each other in the training process, and the goal of the generator is to generate realistic DSA subtraction data as much as possible so as to deceive the discriminant; the goal of the arbiter is to judge the source of the sample as accurately as possible to distinguish between DSA data produced by the generator and real DSA data.
3. The method for generating a digital subtraction angiography image based on deep learning according to claim 1, wherein in step 7, the generator model architecture is specifically as follows: the generator G contains 5 basic blocks, each of which contains 6 layers of convolution layers with a convolution kernel size of 3 x 3 and a number of filters of 128, and in addition to the basic blocks, there are 4 convolution layers and 2 deconvolution layers, all of which have a convolution kernel size of 3 x 3, and in these 6 layers the number of filters is 32, 64, 128, 64, 32 and 1, respectively, with steps of 1 for the first and last layer and steps of 2 for the other layers.
4. The method for generating a digital subtraction angiography image based on deep learning according to claim 1, wherein in the step 6, the details of the discriminator architecture are as follows:
the arbiter D is a typical CNN architecture, including three convolution layers and three fully connected layers, where the first two layers use the prilu as an activation function, the last layer uses the sigmoid function to output probabilities, the convolution kernel size of the convolution layers is 3×3, the number of filters is 32, 64, and 128, and the number of nodes in the fully connected layers is 512, 128, and 1, respectively.
5. The method for generating a digital subtraction angiography image based on deep learning according to claim 1, wherein in step 8, the generator G loss function details are as follows:
wherein:
y is a label, G is a generator, X is a film, and W and H are the width and height of the image.
6. The method for generating a digital subtraction angiography image based on deep learning according to claim 1, wherein in the step 9, the details of the D loss function of the discriminator are as follows:
wherein:
wherein Y is the label of the input pair, which takes on a value of 0 or 1, andis the probability predicted by the discriminator, and the value range is 0,1]Between them.
7. The method for generating digital subtraction angiography image based on deep learning according to claim 1, wherein in step 10, the generator and arbiter training process is detailed as follows:
the surplus sheet is input to a generator which then generates a loss function L by combining the supervised loss and the discriminator loss G Optimizing by discriminating fraction function L D Optimizing, marking the predictions of DSA image and pair of interference patches as "negative" labels to confuse the discriminator, so that the loss of the discriminator can guide the generator to generate more realistic and accurate vessel images, to ensure that the whole network can converge and stabilize, the discriminator D is trained once, and then the generator G is optimized twice until convergence.
CN202311711193.7A 2023-12-13 2023-12-13 Digital subtraction angiography image generation method based on deep learning Pending CN117788616A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311711193.7A CN117788616A (en) 2023-12-13 2023-12-13 Digital subtraction angiography image generation method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311711193.7A CN117788616A (en) 2023-12-13 2023-12-13 Digital subtraction angiography image generation method based on deep learning

Publications (1)

Publication Number Publication Date
CN117788616A true CN117788616A (en) 2024-03-29

Family

ID=90388260

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311711193.7A Pending CN117788616A (en) 2023-12-13 2023-12-13 Digital subtraction angiography image generation method based on deep learning

Country Status (1)

Country Link
CN (1) CN117788616A (en)

Similar Documents

Publication Publication Date Title
CN110728729B (en) Attention mechanism-based unsupervised CT projection domain data recovery method
Li et al. Accurate retinal vessel segmentation in color fundus images via fully attention-based networks
Zuo et al. R2AU-Net: attention recurrent residual convolutional neural network for multimodal medical image segmentation
CN110675461A (en) CT image recovery method based on unsupervised learning
CN108765392B (en) Digestive tract endoscope lesion detection and identification method based on sliding window
Shi et al. Low-light image enhancement algorithm based on retinex and generative adversarial network
CN112598649B (en) 2D/3D spine CT non-rigid registration method based on generation of countermeasure network
CN114897780B (en) MIP sequence-based mesenteric artery blood vessel reconstruction method
WO2023202265A1 (en) Image processing method and apparatus for artifact removal, and device, product and medium
Popescu et al. Retinal blood vessel segmentation using pix2pix gan
Ni et al. Cycle-interactive generative adversarial network for robust unsupervised low-light enhancement
CN113222975B (en) High-precision retinal vessel segmentation method based on improved U-net
Yu et al. FS-GAN: Fuzzy Self-guided structure retention generative adversarial network for medical image enhancement
CN113781461A (en) Intelligent patient monitoring and sequencing method
CN112541566B (en) Image translation method based on reconstruction loss
Burlin et al. Deep image inpainting
CN116894783A (en) Metal artifact removal method for countermeasure generation network model based on time-varying constraint
CN111814891A (en) Medical image synthesis method, device and storage medium
CN116486465A (en) Image recognition method and system for face structure analysis
KR102476888B1 (en) Artificial diagnostic data processing apparatus and its method in digital pathology images
CN117788616A (en) Digital subtraction angiography image generation method based on deep learning
Liu et al. Vessel segmentation using principal component based threshold algorithm
CN113538451B (en) Method and device for segmenting magnetic resonance image of deep vein thrombosis, electronic equipment and storage medium
CN113223104B (en) Cardiac MR image interpolation method and system based on causal relationship
Genereux et al. An Efficient CNN-BiLSTM-Based Model for Multi-Class Intracranial Hemorrhage Classification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination