CN110503630B - Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model - Google Patents
Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model Download PDFInfo
- Publication number
- CN110503630B CN110503630B CN201910653565.2A CN201910653565A CN110503630B CN 110503630 B CN110503630 B CN 110503630B CN 201910653565 A CN201910653565 A CN 201910653565A CN 110503630 B CN110503630 B CN 110503630B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- image
- loss
- sub
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 206010008111 Cerebral haemorrhage Diseases 0.000 title claims abstract description 34
- 238000013136 deep learning model Methods 0.000 title claims abstract description 12
- 208000032843 Hemorrhage Diseases 0.000 claims abstract description 85
- 230000000740 bleeding effect Effects 0.000 claims abstract description 85
- 238000001514 detection method Methods 0.000 claims abstract description 49
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 21
- 210000004556 brain Anatomy 0.000 claims abstract description 17
- 208000007536 Thrombosis Diseases 0.000 claims abstract description 10
- 230000006835 compression Effects 0.000 claims abstract description 7
- 238000007906 compression Methods 0.000 claims abstract description 7
- 238000010521 absorption reaction Methods 0.000 claims abstract description 5
- 238000009792 diffusion process Methods 0.000 claims abstract description 5
- 230000006870 function Effects 0.000 claims description 55
- 238000011176 pooling Methods 0.000 claims description 36
- 238000009826 distribution Methods 0.000 claims description 30
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 12
- 238000012706 support-vector machine Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 239000008280 blood Substances 0.000 claims description 8
- 210000004369 blood Anatomy 0.000 claims description 8
- 230000036772 blood pressure Effects 0.000 claims description 8
- 230000008485 antagonism Effects 0.000 claims description 6
- 230000002490 cerebral effect Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 230000001575 pathological effect Effects 0.000 claims description 6
- 238000000926 separation method Methods 0.000 claims description 6
- 238000009827 uniform distribution Methods 0.000 claims description 6
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000005096 rolling process Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 claims description 3
- 238000003325 tomography Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 208000031226 Hyperlipidaemia Diseases 0.000 description 1
- 206010020772 Hypertension Diseases 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 206010012601 diabetes mellitus Diseases 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000003625 skull Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Medical Informatics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Primary Health Care (AREA)
- Databases & Information Systems (AREA)
- Epidemiology (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a cerebral hemorrhage classifying, positioning and predicting method based on a three-dimensional deep learning model, which is used for carrying out three-dimensional modeling on a two-dimensional CT image by a surface reconstruction method to obtain a three-dimensional CT image; then, a three-dimensional convolutional neural network is used for extracting features of the three-dimensional CT image, and the extracted features are classified through an SVM classifier, so that whether the CT image contains bleeding points or not is classified and judged; slicing the three-dimensional CT image which is judged to contain the bleeding points by the classifier again, and accurately positioning the bleeding point positions of the two-dimensional CT image after slicing through a target detection network; compression encoding is carried out on physical characteristic information of a patient to serve as a three-dimensional conditional generation condition of an countermeasure network, the generation condition is integrated with random noise, and a three-dimensional CT image is output according to physical indexes of the patient by using a three-dimensional generator in the countermeasure network model, so that the diffusion of blood clots of the brain of the patient with time or the absorption condition of the brain of the patient by a human body is predicted.
Description
Technical Field
The invention belongs to the technical field of deep learning, and particularly relates to a cerebral hemorrhage classifying, positioning and predicting method based on a three-dimensional deep learning model.
Background
Cerebral hemorrhage belongs to cerebrovascular diseases and is mainly caused by hemorrhage caused by rupture of blood vessels in non-exocerebral parenchyma, wherein various reasons for cerebral hemorrhage are included, such as hypertension, hyperlipidemia, diabetes and other cardiovascular diseases, if the discovery and treatment are not timely, about 70% of patients can suffer from sequelae of serious influence on normal life due to incomplete limb functions and the like, and the proportion of the hazard should be paid attention to the people in the whole society.
At present, the method for judging the bleeding point of the human brain usually carries out the skull CT examination on the patient, which is one of the very important and common detection means in the brain at present, but at present, judging whether the human brain bleeds from the CT image and positioning the bleeding point all need very experienced doctors to diagnose, but the illness state is diagnosed by an artificial way, and the illness state misdiagnosis of the patient sometimes happens due to the subjectivity of the human.
Disclosure of Invention
The invention aims to: aiming at the problems, the invention provides a three-dimensional deep learning-based mode, wherein a three-dimensional CT image is obtained by carrying out three-dimensional modeling on a two-dimensional CT image, a three-dimensional convolutional neural network, a target detection network and a three-dimensional condition generation countermeasure network are utilized to rapidly and accurately judge whether a patient suffers from cerebral hemorrhage, and bleeding points can be accurately positioned, and a three-dimensional CT image is generated according to the body index of the patient, so that the disease development of the patient is predicted.
The technical scheme is as follows: in order to achieve the purpose of the invention, the technical scheme adopted by the invention is as follows: a cerebral hemorrhage classifying, locating and predicting method based on a three-dimensional deep learning model comprises the following steps:
the first step: acquiring an original brain two-dimensional CT image set of a patient through CT tomography, wherein the image set is a series of tomographic images obtained by slicing, and modeling a three-dimensional CT image by adopting a surface reconstruction method to obtain the three-dimensional CT image;
and a second step of: building a three-dimensional convolutional neural network, namely a feature extraction model, and extracting features of the three-dimensional CT image; classifying the extracted features by using a support vector machine (Support Vector Machine, SVM) classifier, so as to judge whether bleeding points are contained in the three-dimensional CT image;
and a third step of: building a target detection network; slicing the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step, obtaining a two-dimensional CT image set again, and taking the two-dimensional CT image set as input of a target detection network; optimizing by taking the minimum loss function value of the target detection network as a target, outputting the central position coordinates of the bleeding points and the width and the height of the predicted frame of the bleeding points, and positioning and drawing the position of cerebral hemorrhage in the CT image;
fourth step: taking basic information of a patient and the time of onset as generating conditions, wherein the basic information comprises: age, sex, blood pressure, blood sugar; constructing a three-dimensional condition type generation countermeasure network according to the three-dimensional generator, the three-dimensional discriminator and the three-dimensional classifier; and (3) inputting the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step into the three-dimensional discriminator in the countermeasure network, generating a three-dimensional CT image which is predicted by combining the three-dimensional generator in the countermeasure network with the generation condition, and predicting the diffusion of the cerebral blood clot of the patient with the time or the absorption condition of the cerebral blood clot of the patient by a human body.
Further, the modeling method of the three-dimensional CT image in the first step is as follows:
(1.1) affine transforming each two-dimensional CT image in the two-dimensional CT image set by translation, rotation and scaling, expressed as follows:
P result (y k )＝T×P original (x k )
wherein T is affine transformation matrix, P original For the original two-dimensional CT image, P result Is a two-dimensional CT image after affine transformation; k is the number of pixel points in the image, x k For gray value, y of kth pixel point in affine transformation front image k The gray value of the kth pixel point in the affine transformed image;
(1.2) compensating for loss of CT image quality after affine transformation, wherein the compensation formula is as follows:
wherein S is compensation intensity;
(1.3) according to the two-dimensional CT image contour matching original tomographic image after compensation loss, adopting interpolation method to interpolate between two adjacent two-dimensional CT images after compensation loss, generating missing image between tomographic images so as to meet the requirement of surface reconstruction; and (3) finishing surface reconstruction by adopting a triangular surface patch method, thereby finishing three-dimensional reconstruction of a two-dimensional CT image and obtaining a three-dimensional CT image.
Further, the three-dimensional convolutional neural network comprises a three-dimensional convolutional layer, a three-dimensional pooling layer and a full-connection layer, and the three-dimensional CT image is subjected to feature extraction through the three-dimensional convolutional neural network, and the method comprises the following steps:
(2.1) as the traditional two-dimensional convolution layer only can extract the plane characteristics of a single CT image, the three-dimensional convolution layer can ensure that the space structure information of the three-dimensional CT image is not lost, the three-dimensional characteristics are fully utilized, and the expression capability of the characteristics is enhanced; inputting a three-dimensional CT image into a three-dimensional convolution layer, extracting the characteristics of the three-dimensional CT image by the three-dimensional convolution layer to obtain a three-dimensional characteristic image, wherein the three-dimensional characteristic image represents the whole information of the input three-dimensional CT image, and the three-dimensional convolution layer has the following calculation formula:
wherein ,
and />
An Mth three-dimensional convolution feature block of the L-1 th layer and the I-1 th layer respectively; />
Is the three-dimensional convolution kernel of the L-1 layer; />
Is->
An element in coordinates (x, y, z);
is->
In the process of three-dimensional convolution kernel->
After convolution
The values in coordinates (x-j, y-k, z-l), where (j, k, l) represent the size of the three-dimensional convolution kernel; />
For added bias items, the Activation represents a nonlinear Activation function adopted after convolution;
(2.2) reducing the resolution of the three-dimensional feature map through the three-dimensional pooling layer, and removing redundant information; selecting a maximum pooling layer, and performing size compression on the length, the height and the width of the three-dimensional feature map in a step-size increasing mode to obtain a dimension-reduced three-dimensional feature map;
the convolution kernel size of the maximum pooling layer is J 1 ×K 1 ×L 1 The pooling layer convolution step length is set to S L ×S W ×S H The method comprises the steps of carrying out a first treatment on the surface of the Then:
L af ＝(L be -J 1 )/S L +1
W af ＝(W be -K 1 )/S W +1
H af ＝(H be -L 1 )/S H +1
wherein ,Lbe 、W be 、H be Length, width and height of feature map before pooling, L af 、W af 、H af The length, the width and the height of the feature map after pooling are respectively;
(2.3) cross-stacking N convolution layers and N pooling layers, and performing three-dimensional convolution and three-dimensional pooling processing on the input three-dimensional CT image for N times according to the method in the steps (2.1) - (2.2), so as to output a three-dimensional feature map; wherein, the number of layers of rolling and pooling is selected according to the size of the image;
(2.4) introducing two continuous full-connection layers after the last pooling layer, and integrating the three-dimensional image features output in the step (2.3) through the full-connection layers to obtain advanced feature information; obtaining an advanced three-dimensional feature map; the calculation formula of the full connection layer is as follows:
Dense 1 ＝Activation×(B 1 +W 1 F in )
Dense 2 ＝Activation×(B 2 +W 2 Dense 1 )
reconstructing the feature map subjected to three-dimensional convolution and three-dimensional pooling into a one-dimensional feature vector F in As input to the first fully connected layer; w (W) 1 、W 2 The weight matrixes of the first full-connection layer and the second full-connection layer are respectively; b (B) 1 、B 2 Bias terms of the first and second fully-connected layers, respectively; dense 1 、Dense 2 Representing the outputs of the first and second fully-connected layers, respectively;
(2.5) taking the characteristic information obtained in the step (2.4) as the output of the three-dimensional convolutional neural network, and finishing the characteristic extraction of the three-dimensional convolutional neural network on the three-dimensional CT image; using an SVM classifier to make nonlinear mapping, mapping the characteristic information output by the three-dimensional convolutional neural network to a high-dimensional space by using an inner product kernel function, searching an optimal separation hyperplane, and maximizing a classification boundary, thereby judging whether bleeding points are contained in the three-dimensional CT image; the optimal separation hyperplane means that the classification surface not only correctly separates the front class and the back class, but also maximizes the classification interval.
Further, setting up a target detection network to locate the position of the bleeding point; the method comprises the following steps:
(3.1) slicing the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier, and obtaining a two-dimensional CT image again, wherein the two-dimensional CT image is used as input of a target detection network;
(3.2) dividing a two-dimensional CT image input into a target detection network into S multiplied by S cells, recording each cell as a sub-graph, setting B sliding windows for each sub-graph to detect a target to be detected, namely a bleeding point, and each sliding window is represented by five parameters:
(L X ,L Y ,L W ,L H ,L C )
wherein ,(LX ,L Y ) An abscissa and an ordinate representing the center position of the sliding window; (L) H ,L W ) Representing the length and width of the sliding window; l (L) C In order for the confidence level to be high,an estimated probability, L, indicating whether the current sliding window contains bleeding points and the accuracy of their predictions C The calculation formula of (2) is as follows:
L C ＝P O ×P IOU (1)
wherein ,PO Indicating the probability that the sliding window contains bleeding points, P IOU The overlapping area of the sliding window and the real detection object area is represented, and the unit is a pixel; if the sliding window contains bleeding points, P O =1, otherwise P O =0; for each sub-graph, selecting the largest P in the sliding window contained in the sub-graph IOU Substituting formula (1) to calculate the confidence coefficient of whether the subgraph contains bleeding points;
(3.3) defining a loss function of the target detection network, and optimizing the network by taking the minimum loss function value as an optimization target; judging the confidence coefficient of the bleeding point contained in each sub-graph, judging that the sub-graph contains the bleeding point when the confidence coefficient is higher than a set threshold value, and otherwise, judging that the bleeding point does not exist; selecting a sub-graph with the highest confidence from all sub-graphs containing the bleeding points, and outputting the central position coordinates of the bleeding points in the sub-graph with the highest confidence and the height and the width of the predicted frame of the bleeding points; the size of the threshold value determines the sensitivity of the target detection network to judge the bleeding point; when the loss function is optimized to the minimum, the optimization of the target detection network is finished;
the loss function of the target detection network is defined as follows:
Loss total ＝Loss coord +Loss conf +Loss class (2)
in the Loss total A loss function representing target detection; loss (Low Density) coord Representing the mean square error value of the predicted frame and the actual frame as a loss function of the predicted frame coordinates; loss (Low Density) conf A loss function representing sliding window confidence errors for all subgraphs; loss (Low Density) class Representing a classification error;
and (3.4) inputting the two-dimensional CT image in the step (3.1) into an optimized target detection network, outputting the central position coordinates of the bleeding points and the height and width of the predicted frame of the bleeding points, and further positioning and drawing the positions of the bleeding points.
Further, step (3.3) the Loss function Loss coord 、Loss conf 、Loss class The definition is as follows:
(3.3.1) defining a loss function of predicted bezel coordinates as:
in the formula ,Iij Indicating whether a bleeding point target is contained in the jth sliding window of the ith sub-graph; s is S 2 Representing the number of sub-images to be detected in a brain CT image; b represents the number of sliding windows in each sub-graph; lambda (lambda) coord The weight coefficient of the coordinate error; x is X ij and Yij Respectively representing the abscissa and the ordinate of the central point of the jth sliding window in the ith sub-graph; w (W) ij and Hij To predict the width and length of the frame;
the horizontal and vertical coordinates of the central point of the sliding window of the real bleeding point of the ith sub-graph are respectively represented, and the width and the length of the frame are predicted;
(3.3.2) defining a loss function of sliding window confidence errors for all subgraphs as:
wherein, loss conf Confidence errors caused by the probability of containing bleeding points in the sliding windows of all the subgraphs; lambda (lambda) noobj Is a weight coefficient;
true confidence value indicating that the ith sub-map contains bleeding points, < >>
Representing that the ith sub-graph containsConfidence in the prediction of bleeding points;
(3.3.3) defining the loss function of classification errors due to the sliding window of all sub-graphs as:
in the Loss class Classification errors brought by sliding windows of all subgraphs; class indicates the predicted category, i.e., presence/absence of bleeding points; p is p i (C) The i-th subgraph representing the real image is the probability value of the C-th category;
representing the probability that the ith sub-graph in the predicted image is the C class; by minimizing p i (C) And->
And the mean square error value between the sub-images enables the classification accuracy of the target detection network to be the highest.
Further, the fourth step uses basic information of the patient and the time of onset as generating conditions, wherein the basic information comprises: age, sex, blood pressure, blood sugar; constructing a three-dimensional condition type generation countermeasure network according to the three-dimensional generator, the three-dimensional discriminator and the three-dimensional classifier; inputting the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step into a three-dimensional discriminator in the countermeasure network, generating a three-dimensional CT image which is predicted by combining the three-dimensional generator in the countermeasure network with the generation condition, and predicting the situation that the cerebral blood clot of the patient spreads along with the time or is absorbed by the human body; the method comprises the following steps:
(4.1) compression encoding the time, age, sex, blood pressure and blood sugar of the patient, generating the generation condition of the antagonism network using the encoding result as the three-dimensional condition, and recording the generation condition as G T ；
(4.2) generating new samples obeying the probability distribution of the real data by the three-dimensional generator G according to the input random noise and the generation conditions; the real data refer to a three-dimensional CT image; the random noise is subjected to normal distribution or uniform distribution;
(4.3) respectively inputting the three-dimensional CT image judged by the SVM classifier as cerebral hemorrhage and the new sample generated by the three-dimensional generator G into a three-dimensional discriminator D, wherein the three-dimensional discriminator D judges whether the input sample is the new sample generated by the three-dimensional generator G or the real data sample;
(4.4) training a three-dimensional discriminator D according to the real data sample and the generated sample, so that the discrimination accuracy of the three-dimensional discriminator D on the input data source is maximized, meanwhile, training a three-dimensional generator G by combining the generated condition, minimizing the discrimination accuracy of the three-dimensional discriminator D on the input data source from the real data or the generated data, and enabling the three-dimensional generator G to output a three-dimensional CT image with real pathological characteristics; extracting a feature map of an intermediate hidden layer of the three-dimensional discriminator D, inputting the feature map to the three-dimensional classifier C, and training a loss function of the three-dimensional classifier C to reach a minimum value according to the input feature map, so that an output image of the three-dimensional generator G meets the generation condition; setting training times as N sum ；
(4.5) after the training times are reached, the countermeasure network model reaches Nash equilibrium, and model training is finished; finally, after the random noise fused with the generation conditions passes through the three-dimensional generator G, generating the probability distribution of the new sample equal to the probability distribution of the real data sample; and outputting a three-dimensional CT image which has real pathological characteristics and meets the generation conditions through the three-dimensional generator G, and predicting the diffusion of the blood clot of the brain of the patient or the absorption condition of the brain of the human body along with time according to the predicted three-dimensional CT image output by the three-dimensional generator G by combining the original three-dimensional CT image.
Further, generating an image quality loss function of the countermeasure network by optimizing the three-dimensional condition, so that after random noise fused with the generation condition passes through the three-dimensional generator G, the distribution of the generated new sample is equal to the probability distribution of the real data sample;
the three-dimensional conditional generation antagonism network image quality loss function is defined as follows:
wherein E (-) represents the calculated expected value; p is p data (x) Representing the distribution of real data, x being subject to p data (x) A distributed sampling point; p is p z (z) represents random noise distribution, with normal distribution or uniform distribution, z being compliant with p z (z) a sample point of the distribution; g (z|G) T ) Data generated by the three-dimensional generator G after the random noise and the generation conditions are fused; d (G (z|G) T ) D (x) and D (x) respectively represent the output values of the generated data and the real data through the three-dimensional discriminator D.
Further, the loss function of the three-dimensional classifier C is defined as follows:
where n represents the number of generation conditions, C (G (z|G T ) C (x) and C (x) respectively represent output values of the generated data and the real data through the three-dimensional classifier C; calculate C (G (z|G) T ) And C (x) and production condition G T Mean square error (Mean Square Error, MSE) between.
Further, since the three-dimensional classifier and the three-dimensional classifier both include a part of feature extraction, the amount of calculation of the whole is reduced by sharing the feature extraction model parameters between the three-dimensional classifier and the three-dimensional classifier.
Further, the output dimensions of the target detection network are: sxsx (B x 5+C); wherein s×s is the total number of subgraphs for each image; b is the number of sliding windows used for predicting the target for each sub-graph; each sliding window is represented by 5 parameters, C being the predicted total number of categories.
The beneficial effects are that: compared with the prior art, the technical scheme of the invention has the following beneficial technical effects:
according to the invention, by utilizing a deep learning technology, through carrying out three-dimensional modeling on the two-dimensional CT image, considering the relation on the spatial structure of the CT image, whether the patient suffers from cerebral hemorrhage can be judged more quickly and accurately, the bleeding point can be positioned accurately, and finally, the CT image is generated by utilizing the body index of the patient, so that the development of the patient in the future days can be predicted.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a schematic representation of a three-dimensional convolutional neural network model of the present invention;
FIG. 3 is a schematic diagram of a process for locating a bleeding point by the object detection network of the present invention;
FIG. 4 is a schematic diagram of a three-dimensional conditional generation countermeasure network model of the present invention.
Detailed Description
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
The invention discloses a cerebral hemorrhage classifying, positioning and predicting method based on a three-dimensional deep learning model, which is shown in a figure 1 and comprises the following steps:
the first step: acquiring an original brain two-dimensional CT image set of a patient through CT tomography, wherein the image set is a series of tomographic images obtained by slicing, and modeling a three-dimensional CT image by adopting a surface reconstruction method to obtain the three-dimensional CT image; the method comprises the following steps:
(1.1) affine transforming each two-dimensional CT image in the two-dimensional CT image set by translation, rotation and scaling, expressed as follows:
P result (y k )＝T×P original (x k )
wherein T is affine transformation matrix, P original For the original two-dimensional CT image, P result Is a two-dimensional CT image after affine transformation; k is the number of pixel points in the image, x k For gray value, y of kth pixel point in affine transformation front image k The gray value of the kth pixel point in the affine transformed image;
(1.2) compensating for loss of CT image quality after affine transformation, wherein the compensation formula is as follows:
wherein S is compensation intensity;
(1.3) according to the two-dimensional CT image contour matching original tomographic image after compensation loss, adopting interpolation method to interpolate between two adjacent two-dimensional CT images after compensation loss, generating missing image between tomographic images so as to meet the requirement of surface reconstruction; and (3) finishing surface reconstruction by adopting a triangular surface patch method, thereby finishing three-dimensional reconstruction of a two-dimensional CT image and obtaining a three-dimensional CT image.
And a second step of: building a three-dimensional convolutional neural network, namely a feature extraction model, and extracting features of the three-dimensional CT image; classifying the extracted features by using a support vector machine (Support Vector Machine, SVM) classifier, so as to judge whether bleeding points are contained in the three-dimensional CT image; the feature extraction model is shown in fig. 2; the three-dimensional convolutional neural network comprises a three-dimensional convolutional layer, a three-dimensional pooling layer and a full-connection layer, and the three-dimensional CT image is subjected to feature extraction through the three-dimensional convolutional neural network, and the method comprises the following steps of:
(2.1) as the traditional two-dimensional convolution layer only can extract the plane characteristics of a single CT image, the three-dimensional convolution layer can ensure that the space structure information of the three-dimensional CT image is not lost, the three-dimensional characteristics are fully utilized, and the expression capability of the characteristics is enhanced; inputting a three-dimensional CT image into a three-dimensional convolution layer, extracting the characteristics of the three-dimensional CT image by the three-dimensional convolution layer to obtain a three-dimensional characteristic image, wherein the three-dimensional characteristic image represents the whole information of the input three-dimensional CT image, and the three-dimensional convolution layer has the following calculation formula:
wherein ,
and />
Mth three-dimensional convolution of the I th and L-1 th layers respectively representing the L th layerA feature block; />
Is the three-dimensional convolution kernel of the L-1 layer; />
Is->
An element in coordinates (x, y, z); />
Is->
In the process of three-dimensional convolution kernel->
Post convolution->
The values in coordinates (x-j, y-k, z-l), where (j, k, l) represent the size of the three-dimensional convolution kernel; />
For added bias items, the Activation represents a nonlinear Activation function adopted after convolution;
(2.2) reducing the resolution of the three-dimensional feature map through the three-dimensional pooling layer, and removing redundant information; selecting a maximum pooling layer, and performing size compression on the length, the height and the width of the three-dimensional feature map in a step-size increasing mode to obtain a dimension-reduced three-dimensional feature map;
the convolution kernel size of the maximum pooling layer is J 1 ×K 1 ×L 1 The pooling layer convolution step length is set to S L ×S W ×S H The method comprises the steps of carrying out a first treatment on the surface of the Then:
L af ＝(L be -J 1 )/S L +1
W af ＝(W be -K 1 )/S W +1
H af ＝(H be -L 1 )/S H +1
wherein ,Lbe 、W be 、H be Length, width and height of feature map before pooling, L af 、W af 、H af The length, the width and the height of the feature map after pooling are respectively;
(2.3) cross-stacking N convolution layers and N pooling layers, and performing three-dimensional convolution and three-dimensional pooling processing on the input three-dimensional CT image for N times according to the method in the steps (2.1) - (2.2), so as to output a three-dimensional feature map; wherein, the number of layers of rolling and pooling is selected according to the size of the image;
(2.4) introducing two continuous full-connection layers after the last pooling layer, and integrating the three-dimensional image features output in the step (2.3) through the full-connection layers to obtain advanced feature information; obtaining an advanced three-dimensional feature map; the calculation formula of the full connection layer is as follows:
Dense 1 ＝Activation×(B 1 +W 1 F in )
Dense 2 ＝Activation×(B 2 +W 2 Dense 1 )
reconstructing the feature map subjected to three-dimensional convolution and three-dimensional pooling into a one-dimensional feature vector F in As input to the first fully connected layer; w (W) 1 、W 2 The weight matrixes of the first full-connection layer and the second full-connection layer are respectively; b (B) 1 、B 2 Bias terms of the first and second fully-connected layers, respectively; dense 1 、Dense 2 Representing the outputs of the first and second fully-connected layers, respectively;
(2.5) taking the characteristic information obtained in the step (2.4) as the output of the three-dimensional convolutional neural network, and finishing the characteristic extraction of the three-dimensional convolutional neural network on the three-dimensional CT image; using an SVM classifier to make nonlinear mapping, mapping the characteristic information output by the three-dimensional convolutional neural network to a high-dimensional space by using an inner product kernel function, searching an optimal separation hyperplane, and maximizing a classification boundary, thereby judging whether bleeding points are contained in the three-dimensional CT image; the optimal separation hyperplane means that the classification surface not only correctly separates the front class and the back class, but also maximizes the classification interval.
And a third step of: building a target detection network; slicing the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step, obtaining a two-dimensional CT image set again, and taking the two-dimensional CT image set as input of a target detection network; optimizing by taking the minimum loss function value of the target detection network as a target, outputting the central position coordinates of the bleeding points and the width and the height of the predicted frame of the bleeding points, and positioning and drawing the position of cerebral hemorrhage in the CT image; the effect of the target detection network model is shown in figure 3; the target detection network is built, and the position of the bleeding point is positioned; the method comprises the following steps:
(3.1) slicing the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier, and obtaining a two-dimensional CT image again, wherein the two-dimensional CT image is used as input of a target detection network;
(3.2) dividing a two-dimensional CT image input into a target detection network into S multiplied by S cells, recording each cell as a sub-graph, setting B sliding windows for each sub-graph to detect a target to be detected, namely a bleeding point, and each sliding window is represented by five parameters:
(L X ,L Y ,L W ,L H ,L C )
wherein B is set to 2; (L) X ,L Y ) An abscissa and an ordinate representing the center position of the sliding window; (L) H ,L W ) Representing the length and width of the sliding window; l (L) C For confidence, represent the estimated probability of whether the current sliding window contains bleeding points and the prediction accuracy thereof, L C The calculation formula of (2) is as follows:
L C ＝P O ×P IOU (1)
wherein ,PO Indicating the probability that the sliding window contains bleeding points, P IOU The overlapping area of the sliding window and the real detection object area is represented, and the unit is a pixel; if the sliding window contains bleeding points, P O =1, otherwise P O =0; for each sub-graph, selecting the largest P in the sliding window contained in the sub-graph IOU Substituting formula (1) to calculate the confidence coefficient of whether the subgraph contains bleeding points;
(3.3) defining a loss function of the target detection network, and optimizing the network by taking the minimum loss function value as an optimization target; judging the confidence coefficient of the bleeding point contained in each sub-graph, judging that the sub-graph contains the bleeding point when the confidence coefficient is higher than a set threshold value, and otherwise, judging that the bleeding point does not exist; selecting a sub-graph with the highest confidence from all sub-graphs containing the bleeding points, and outputting the central position coordinates of the bleeding points in the sub-graph with the highest confidence and the height and the width of the predicted frame of the bleeding points; the size of the threshold value determines the sensitivity of the target detection network to judge the bleeding point; when the loss function is optimized to the minimum, the optimization of the target detection network is finished;
the loss function of the target detection network is defined as follows:
Loss total ＝Loss coord +Loss conf +Loss class (2)
in the Loss total A loss function representing target detection; loss (Low Density) coord Representing the mean square error value of the predicted frame and the actual frame as a loss function of the predicted frame coordinates; loss (Low Density) conf A loss function representing sliding window confidence errors for all subgraphs; loss (Low Density) class Representing a classification error;
the Loss function Loss coord 、Loss conf 、Loss class The definition is as follows:
(3.3.1) defining a loss function of predicted bezel coordinates as:
in the formula ,Iij Indicating whether a bleeding point target is contained in the jth sliding window of the ith sub-graph; s is S 2 Representing the number of sub-images to be detected in a brain CT image; b represents the number of sliding windows in each sub-graph; lambda (lambda) coord The weight coefficient of the coordinate error; x is X ij and Yij Respectively represent the jth slide in the ith sub-graphThe abscissa and ordinate of the center point of the movable window; w (W) ij and Hij To predict the width and length of the frame;
the horizontal and vertical coordinates of the central point of the sliding window of the real bleeding point of the ith sub-graph are respectively represented, and the width and the length of the frame are predicted;
(3.3.2) defining a loss function of sliding window confidence errors for all subgraphs as:
wherein, loss conf Confidence errors caused by the probability of containing bleeding points in the sliding windows of all the subgraphs; lambda (lambda) noobj Is a weight coefficient;
true confidence value indicating that the ith sub-map contains bleeding points, < >>
Indicating that the ith sub-map contains predicted confidence for bleeding points;
(3.3.3) defining the loss function of classification errors due to the sliding window of all sub-graphs as:
in the Loss class Classification errors brought by sliding windows of all subgraphs; class represents the predicted category, i.e. with/without bleeding points, the total number of categories is 2; p is p i (C) The i-th subgraph representing the real image is the probability value of the C-th category;
representing the probability that the ith sub-graph in the predicted image is the C class; by minimizing p i (C) And->
The mean square error value between the sub-images enables the classification accuracy of the target detection network to be the highest;
and (3.4) inputting the two-dimensional CT image in the step (3.1) into an optimized target detection network, outputting the central position coordinates of the bleeding points and the height and width of the predicted frame of the bleeding points, and further positioning and drawing the positions of the bleeding points.
The output dimension of the target detection network is: sxsx (B x 5+C); wherein s×s is the total number of subgraphs for each image; b is the number of sliding windows used for predicting the target for each sub-graph; each sliding window is represented by 5 parameters, C being the predicted total number of categories.
Fourth step: taking basic information of a patient and the time of onset as generating conditions, wherein the basic information comprises: age, sex, blood pressure, blood sugar; constructing a three-dimensional condition type generation countermeasure network according to the three-dimensional generator, the three-dimensional discriminator and the three-dimensional classifier; inputting the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step into the three-dimensional discriminator in the countermeasure network, outputting a predicted three-dimensional CT image by combining the generating conditions through the three-dimensional generator in the countermeasure network, and predicting the situation that the cerebral blood clot of the patient spreads along with the time or is absorbed by the human body; the model structure is shown in figure 4; the specific method comprises the following steps:
(4.1) compression encoding the time, age, sex, blood pressure and blood sugar of the patient, generating the generation condition of the antagonism network using the encoding result as the three-dimensional condition, and recording the generation condition as G T ；
(4.2) generating new samples obeying the probability distribution of the real data by the three-dimensional generator G according to the input random noise and the generation conditions; the real data refer to a three-dimensional CT image; the random noise is subjected to normal distribution or uniform distribution;
(4.3) respectively inputting the three-dimensional CT image judged by the SVM classifier as cerebral hemorrhage and the new sample generated by the three-dimensional generator G into a three-dimensional discriminator D, wherein the three-dimensional discriminator D judges whether the input sample is the new sample generated by the three-dimensional generator G or the real data sample;
(4.4) training a three-dimensional discriminator D according to the real data sample and the generated sample, so that the discrimination accuracy of the three-dimensional discriminator D on the input data source is maximized, meanwhile, training a three-dimensional generator G by combining the generated condition, minimizing the discrimination accuracy of the three-dimensional discriminator D on the input data source from the real data or the generated data, and enabling the three-dimensional generator G to output a three-dimensional CT image with real pathological characteristics; extracting a feature map of an intermediate hidden layer of the three-dimensional discriminator D, inputting the feature map to the three-dimensional classifier C, and training a loss function of the three-dimensional classifier C to reach a minimum value according to the input feature map, so that an output image of the three-dimensional generator G meets the generation condition; setting training times as N sum ；
Generating an image quality loss function of the countermeasure network by optimizing the three-dimensional condition, so that after random noise fused with the generation condition passes through the three-dimensional generator G, the distribution of the generated new sample is equal to the probability distribution of the real data sample;
the three-dimensional conditional generation antagonism network image quality loss function is defined as follows:
wherein E (-) represents the calculated expected value; p is p data (x) Representing the distribution of real data, x being subject to p data (x) A distributed sampling point; p is p z (z) represents random noise distribution, with normal distribution or uniform distribution, z being compliant with p z (z) a sample point of the distribution; g (z|G) T ) Data generated by the three-dimensional generator G after the random noise and the generation conditions are fused; d (G (z|G) T ) D (x) and D (x) respectively represent the output values of the generated data and the real data through the three-dimensional discriminator D.
The loss function of the three-dimensional classifier C is defined as follows:
where n represents the number of generation conditions, C (G (z|G T ) C (x) and C (x) respectively represent output values of the generated data and the real data through the three-dimensional classifier C; calculate C (G (z|G) T ) And C (x) and production condition G T Mean square error (Mean Square Error, MSE) between.
The overall calculation amount is reduced by sharing the feature extraction model parameters between the three-dimensional classifier and the three-dimensional discriminator.
(4.5) after the training times are reached, the countermeasure network model reaches Nash equilibrium, and model training is finished; finally, after the random noise fused with the generation conditions passes through the three-dimensional generator G, generating the probability distribution of the new sample equal to the probability distribution of the real data sample; and outputting a three-dimensional CT image which has real pathological characteristics and meets the generation conditions through the three-dimensional generator G, and predicting the diffusion of the blood clot of the brain of the patient or the absorption condition of the brain of the human body along with time according to the predicted three-dimensional CT image output by the three-dimensional generator G by combining the original three-dimensional CT image.
The invention is not limited to the specific embodiments described above, which are intended to be illustrative only and not limiting, as many variations can be made by a person skilled in the art without departing from the spirit of the invention, which fall within the protection of the invention.
Claims (6)
1. A cerebral hemorrhage classifying, positioning and predicting method based on a three-dimensional deep learning model is characterized in that: the method comprises the following steps:
the first step: acquiring an original brain two-dimensional CT image set of a patient through CT tomography, wherein the image set is a series of tomographic images obtained by slicing, and modeling a three-dimensional CT image by adopting a surface reconstruction method to obtain the three-dimensional CT image; the modeling method of the three-dimensional CT image comprises the following steps:
(1.1) affine transforming each two-dimensional CT image in the two-dimensional CT image set by translation, rotation and scaling, expressed as follows:
P result (y k )＝T×P original (x k )
wherein T is affine transformation matrix, P original For the original two-dimensional CT image, P result Is a two-dimensional CT image after affine transformation; k is the number of pixel points in the image, x k For gray value, y of kth pixel point in affine transformation front image k The gray value of the kth pixel point in the affine transformed image;
(1.2) compensating for loss of CT image quality after affine transformation, wherein the compensation formula is as follows:
wherein S is compensation intensity;
(1.3) matching the original tomographic images according to the two-dimensional CT image contours after compensation loss, and interpolating between two adjacent two-dimensional CT images after compensation loss by adopting an interpolation method to generate images missing between the tomographic images;
adopting a triangular surface patch method to finish surface reconstruction, thereby finishing three-dimensional reconstruction of a two-dimensional CT image and obtaining a three-dimensional CT image;
and a second step of: building a three-dimensional convolutional neural network, namely a feature extraction model, wherein the three-dimensional convolutional neural network comprises a three-dimensional convolutional layer, a three-dimensional pooling layer and a full-connection layer, and feature extraction is carried out on a three-dimensional CT image through the three-dimensional convolutional neural network; classifying the extracted features by using a Support Vector Machine (SVM) classifier, and judging whether bleeding points are contained in the three-dimensional CT image, wherein the method comprises the following steps of:
(2.1) inputting the three-dimensional CT image into a three-dimensional convolution layer, extracting the characteristics of the three-dimensional CT image by the three-dimensional convolution layer to obtain a three-dimensional characteristic image, wherein the three-dimensional characteristic image represents the whole information of the input three-dimensional CT image, and the three-dimensional convolution layer has the following calculation formula:
wherein ,
and />
An Mth three-dimensional convolution feature block of the L-1 th layer and the I-1 th layer respectively; />
Is the three-dimensional convolution kernel of the L-1 layer; />
Is->
An element in coordinates (x, y, z);
is->
In the process of three-dimensional convolution kernel->
After convolution
The values in coordinates (x-j, y-k, z-l), where (j, k, l) represent the size of the three-dimensional convolution kernel; />
For added bias items, the Activation represents a nonlinear Activation function adopted after convolution;
(2.2) reducing the resolution of the three-dimensional feature map through the three-dimensional pooling layer, and removing redundant information; selecting a maximum pooling layer, and performing size compression on the length, the height and the width of the three-dimensional feature map in a step-size increasing mode to obtain a dimension-reduced three-dimensional feature map;
the convolution kernel size of the maximum pooling layer is J 1 ×K 1 ×L 1 The pooling layer convolution step length is set to S L ×S W ×S H The method comprises the steps of carrying out a first treatment on the surface of the Then:
L af ＝(L be -J 1 )/S L +1
W af ＝(W be -K 1 )/S W +1
H af ＝(H be -L 1 )/S H +1
wherein ,Lbe 、W be 、H be Length, width and height of feature map before pooling, L af 、W af 、H af The length, the width and the height of the feature map after pooling are respectively;
(2.3) cross-stacking N convolution layers and N pooling layers, and performing three-dimensional convolution and three-dimensional pooling processing on the input three-dimensional CT image for N times according to the method in the steps (2.1) - (2.2), so as to output a three-dimensional feature map; wherein, the number of layers of rolling and pooling is selected according to the size of the image;
(2.4) introducing two continuous full-connection layers after the last pooling layer, and integrating the three-dimensional image features output in the step (2.3) through the full-connection layers to obtain advanced feature information; obtaining an advanced three-dimensional feature map; the calculation formula of the full connection layer is as follows:
Dense 1 ＝Activation×(B 1 +W 1 F in )
Dense 2 ＝Activation×(B 2 +W 2 Dense 1 )
reconstructing the feature map subjected to three-dimensional convolution and three-dimensional pooling into a one-dimensional feature vector F in As input to the first fully connected layer; w (W) 1 、W 2 The weight matrixes of the first full-connection layer and the second full-connection layer are respectively; b (B) 1 、B 2 Bias terms of the first and second fully-connected layers, respectively; dense 1 、Dense 2 Representing the outputs of the first and second fully-connected layers, respectively;
(2.5) taking the characteristic information obtained in the step (2.4) as the output of the three-dimensional convolutional neural network, and finishing the characteristic extraction of the three-dimensional convolutional neural network on the three-dimensional CT image;
using an SVM classifier to make nonlinear mapping, mapping the characteristic information output by the three-dimensional convolutional neural network to a high-dimensional space by using an inner product kernel function, searching an optimal separation hyperplane, and maximizing a classification boundary, thereby judging whether bleeding points are contained in the three-dimensional CT image;
the optimal separation hyperplane means that the classification surface not only correctly separates the front class and the back class, but also maximizes the classification interval;
and a third step of: constructing a target detection network, slicing the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step, obtaining a two-dimensional CT image set again, and taking the two-dimensional CT image set as input of the target detection network;
optimizing by taking the minimum loss function value of the target detection network as a target, outputting the central position coordinates of the bleeding points and the width and the height of the predicted frame of the bleeding points, and positioning and drawing the position of cerebral hemorrhage in the CT image;
(3.1) slicing the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier, and obtaining a two-dimensional CT image again, wherein the two-dimensional CT image is used as input of a target detection network;
(3.2) dividing a two-dimensional CT image input into a target detection network into S multiplied by S cells, recording each cell as a sub-graph, setting B sliding windows for each sub-graph to detect a target to be detected, namely a bleeding point, and each sliding window is represented by five parameters:
(L X ,L Y ,L W ,L H ,L C )
wherein ,(LX ,L Y ) An abscissa and an ordinate representing the center position of the sliding window; (L) H ,L W ) Representing the length and width of the sliding window; l (L) C For confidence, represent the estimated probability of whether the current sliding window contains bleeding points and the prediction accuracy thereof, L C The calculation formula of (2) is as follows:
L C ＝P O ×P IOU (1)
wherein ,PO Indicating the probability that the sliding window contains bleeding points, P IOU The overlapping area of the sliding window and the real detection object area is represented, and the unit is a pixel; if the sliding window contains bleeding points, P O =1, otherwise P O =0; for each sub-graph, selecting the largest P in the sliding window contained in the sub-graph IOU Substituting formula (1) to calculate the confidence coefficient of whether the subgraph contains bleeding points;
(3.3) defining a loss function of the target detection network, and optimizing the network by taking the minimum loss function value as an optimization target; judging the confidence coefficient of the bleeding point contained in each sub-graph, judging that the sub-graph contains the bleeding point when the confidence coefficient is higher than a set threshold value, and otherwise, judging that the bleeding point does not exist;
selecting a sub-graph with the highest confidence from all sub-graphs containing the bleeding points, and outputting the central position coordinates of the bleeding points in the sub-graph with the highest confidence and the height and the width of the predicted frame of the bleeding points;
the size of the threshold value determines the sensitivity of the target detection network to judging the bleeding point; when the loss function is optimized to the minimum, the optimization of the target detection network is finished;
the loss function of the target detection network is defined as follows:
Loss total ＝Loss coord +Loss conf +Loss class (2)
in the Loss total A loss function representing target detection; loss (Low Density) coord Representing the mean square error value of the predicted frame and the actual frame as a loss function of the predicted frame coordinates; loss (Low Density) conf A loss function representing sliding window confidence errors for all subgraphs; loss (Low Density) class Representing a classification error;
the Loss function Loss coord 、Loss conf 、Loss class The definition is as follows:
the loss function defining the predicted bounding box coordinates is:
in the formula ,Iij Indicating whether a bleeding point target is contained in the jth sliding window of the ith sub-graph; s is S 2 Representing the number of sub-images to be detected in a brain CT image; b represents the number of sliding windows in each sub-graph; lambda (lambda) coord The weight coefficient of the coordinate error; x is X ij and Yij Respectively representing the abscissa and the ordinate of the central point of the jth sliding window in the ith sub-graph; w (W) ij and Hij To predict the width and length of the frame;
the horizontal and vertical coordinates of the central point of the sliding window of the real bleeding point of the ith sub-graph are respectively represented, and the width and the length of the frame are predicted;
the loss function defining the sliding window confidence errors for all subgraphs is:
wherein, loss conf Confidence errors caused by the probability of containing bleeding points in the sliding windows of all the subgraphs; lambda (lambda) noobj Is a weight coefficient;
true confidence value indicating that the ith sub-map contains bleeding points, < >>
Indicating that the ith sub-map contains predicted confidence for bleeding points;
the loss function of classification errors brought by the sliding window defining all sub-graphs is:
in the Loss class Classification errors brought by sliding windows of all subgraphs; class indicates the predicted category, i.e., presence/absence of bleeding points; p is p i (C) The i-th subgraph representing the real image is the probability value of the C-th category;
representing the probability that the ith sub-graph in the predicted image is the C class; by minimizing p i (C) And->
The mean square error value between the sub-images enables the classification accuracy of the target detection network to be the highest;
(3.4) inputting the two-dimensional CT image in the step (3.1) into an optimized target detection network, outputting the central position coordinates of the bleeding points and the height and width of the predicted frame of the bleeding points, and further positioning and drawing the positions of the bleeding points;
fourth step: taking basic information of a patient and the time of onset as generating conditions, wherein the basic information comprises: age, sex, blood pressure, blood sugar; constructing a three-dimensional condition type generation countermeasure network according to the three-dimensional generator, the three-dimensional discriminator and the three-dimensional classifier; and (3) inputting the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step into the three-dimensional discriminator in the countermeasure network, outputting a predicted three-dimensional CT image by combining the generating conditions through the three-dimensional generator in the countermeasure network, and predicting the situation that the cerebral blood clot of the patient spreads or is absorbed by the human body along with the time.
2. The method for classifying, locating and predicting cerebral hemorrhage based on three-dimensional deep learning model as claimed in claim 1, wherein the method comprises the following steps: the fourth step takes basic information of a patient and the time of onset as generating conditions, wherein the basic information comprises: age, sex, blood pressure, blood sugar; constructing a three-dimensional condition type generation countermeasure network according to the three-dimensional generator, the three-dimensional discriminator and the three-dimensional classifier; inputting the three-dimensional CT image which is judged to be cerebral hemorrhage by the SVM classifier in the second step into a three-dimensional discriminator in the countermeasure network, generating a three-dimensional CT image which is predicted by combining the three-dimensional generator in the countermeasure network with the generation condition, and predicting the situation that the cerebral blood clot of the patient spreads along with the time or is absorbed by the human body; the method comprises the following steps:
(4.1) compression encoding the time, age, sex, blood pressure and blood sugar of the patient, generating the generation condition of the antagonism network using the encoding result as the three-dimensional condition, and recording the generation condition as G T ；
(4.2) generating new samples obeying the probability distribution of the real data by the three-dimensional generator G according to the input random noise and the generation conditions; the real data refer to a three-dimensional CT image; the random noise is subjected to normal distribution or uniform distribution;
(4.3) respectively inputting the three-dimensional CT image judged by the SVM classifier as cerebral hemorrhage and the new sample generated by the three-dimensional generator G into a three-dimensional discriminator D, wherein the three-dimensional discriminator D judges whether the input sample is the new sample generated by the three-dimensional generator G or the real data sample;
(4.4) training a three-dimensional discriminator D according to the real data sample and the generated sample, so that the discrimination accuracy of the three-dimensional discriminator D on the input data source is maximized, meanwhile, training a three-dimensional generator G by combining the generated condition, minimizing the discrimination accuracy of the three-dimensional discriminator D on the input data source from the real data or the generated data, and enabling the three-dimensional generator G to output a three-dimensional CT image with real pathological characteristics; extracting a feature map of an intermediate hidden layer of the three-dimensional discriminator D, inputting the feature map to the three-dimensional classifier C, and training a loss function of the three-dimensional classifier C to reach a minimum value according to the input feature map, so that an output image of the three-dimensional generator G meets the generation condition; setting training times as N sum ；
(4.5) after the training times are reached, the countermeasure network model reaches Nash equilibrium, and model training is finished; finally, after the random noise fused with the generation conditions passes through the three-dimensional generator G, generating the probability distribution of the new sample equal to the probability distribution of the real data sample; and outputting a three-dimensional CT image which has real pathological characteristics and meets the generation conditions through the three-dimensional generator G, and predicting the diffusion of the blood clot of the brain of the patient or the absorption condition of the brain of the human body along with time according to the predicted three-dimensional CT image output by the three-dimensional generator G by combining the original three-dimensional CT image.
3. The method for classifying, locating and predicting cerebral hemorrhage based on three-dimensional deep learning model as claimed in claim 2, wherein the method comprises the following steps: generating an image quality loss function of the countermeasure network by optimizing the three-dimensional condition, so that after random noise fused with the generation condition passes through the three-dimensional generator G, the distribution of the generated new sample is equal to the probability distribution of the real data sample;
the three-dimensional conditional generation antagonism network image quality loss function is defined as follows:
wherein E (-) represents the calculated expected value; p is p data (x) Representing the distribution of real data, x being subject to p data (x) A distributed sampling point; p is p z (z) represents random noise distribution, with normal distribution or uniform distribution, z being compliant with p z (z) a sample point of the distribution; g (z|G) T ) Data generated by the three-dimensional generator G after the random noise and the generation conditions are fused; d (G (z|G) T ) D (x) and D (x) respectively represent the output values of the generated data and the real data through the three-dimensional discriminator D.
4. A method for classifying, locating and predicting cerebral hemorrhage based on a three-dimensional deep learning model according to claim 3, wherein: the loss function of the three-dimensional classifier C is defined as follows:
where n represents the number of generation conditions, C (G (z|G T ) And C (x) respectively represent the output of the generated data and the real data through the three-dimensional classifier CA value; calculate C (G (z|G) T ) And C (x) and production condition G T Mean square error (Mean Square Error, MSE) between.
5. The three-dimensional deep learning model-based cerebral hemorrhage classifying, locating and predicting method as claimed in claim 4, wherein: and sharing the characteristics of the three-dimensional classifier and the three-dimensional discriminator to extract model parameters.
6. The method for classifying, locating and predicting cerebral hemorrhage based on three-dimensional deep learning model as claimed in claim 1, wherein the method comprises the following steps: the output dimension of the target detection network is: sxsx (B x 5+C); wherein s×s is the total number of subgraphs for each image; b is the number of sliding windows used for predicting the target for each sub-graph; each sliding window is represented by 5 parameters, C being the predicted total number of categories.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910653565.2A CN110503630B (en) | 2019-07-19 | 2019-07-19 | Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910653565.2A CN110503630B (en) | 2019-07-19 | 2019-07-19 | Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110503630A CN110503630A (en) | 2019-11-26 |
| CN110503630B true CN110503630B (en) | 2023-05-09 |
Family
ID=68586684
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910653565.2A Active CN110503630B (en) | 2019-07-19 | 2019-07-19 | Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110503630B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111951950B (en) * | 2019-05-16 | 2023-11-24 | 中国科学院自动化研究所 | Three-dimensional data medical classification system based on deep learning |
| CN111179222B (en) * | 2019-12-09 | 2023-08-22 | 平安科技（深圳）有限公司 | Intelligent cerebral hemorrhage point detection method and device and computer readable storage medium |
| CN110889496B (en) * | 2019-12-11 | 2023-06-06 | 北京工业大学 | Human brain effect connection identification method based on countermeasure generation network |
| CN111340129A (en) * | 2020-03-09 | 2020-06-26 | 南京览笛信息科技有限公司 | Efficient classification method for multi-angle estimation of 3D structure depth network images |
| CN111754520B (en) * | 2020-06-09 | 2023-09-15 | 江苏师范大学 | Deep learning-based cerebral hematoma segmentation method and system |
| CN111932512B (en) * | 2020-08-06 | 2022-08-05 | 吉林大学 | Intracranial hemorrhage detection method applied to CT image based on CNN and NLSTM neural network |
| CN112700859A (en) * | 2020-12-15 | 2021-04-23 | 贵州小宝健康科技有限公司 | Medical diagnosis assisting method and system based on medical images |
| CN113349810B (en) * | 2021-05-27 | 2022-03-01 | 北京安德医智科技有限公司 | Cerebral hemorrhage focus identification and hematoma expansion prediction system and device |
| CN113506634B (en) * | 2021-07-15 | 2024-04-09 | 南京易爱医疗设备有限公司 | Brain Simulation System |
| CN113576508A (en) * | 2021-07-21 | 2021-11-02 | 华中科技大学 | Cerebral hemorrhage auxiliary diagnosis system based on neural network |
| CN113808230A (en) * | 2021-08-26 | 2021-12-17 | 华南理工大学 | Method, system, device and storage medium for improving electrical impedance imaging accuracy |
| CN113723406B (en) * | 2021-09-03 | 2023-07-18 | 乐普（北京）医疗器械股份有限公司 | Method and device for processing support positioning of coronary angiography image |
| CN116071350B (en) * | 2023-03-06 | 2023-07-04 | 同心智医科技(北京)有限公司 | Deep learning-based brain micro-bleeding identification method, device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426808A (en) * | 2014-09-23 | 2016-03-23 | 深圳先进技术研究院 | Intra-brain sagittal line measurement method and system |
| CN106296653A (en) * | 2016-07-25 | 2017-01-04 | 浙江大学 | Brain CT image hemorrhagic areas dividing method based on semi-supervised learning and system |
| CN108564039A (en) * | 2018-04-16 | 2018-09-21 | 北京工业大学 | A kind of epileptic seizure prediction method generating confrontation network based on semi-supervised deep layer |
| CN108717568A (en) * | 2018-05-16 | 2018-10-30 | 陕西师范大学 | A kind of image characteristics extraction and training method based on Three dimensional convolution neural network |
-
2019
- 2019-07-19 CN CN201910653565.2A patent/CN110503630B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426808A (en) * | 2014-09-23 | 2016-03-23 | 深圳先进技术研究院 | Intra-brain sagittal line measurement method and system |
| CN106296653A (en) * | 2016-07-25 | 2017-01-04 | 浙江大学 | Brain CT image hemorrhagic areas dividing method based on semi-supervised learning and system |
| CN108564039A (en) * | 2018-04-16 | 2018-09-21 | 北京工业大学 | A kind of epileptic seizure prediction method generating confrontation network based on semi-supervised deep layer |
| CN108717568A (en) * | 2018-05-16 | 2018-10-30 | 陕西师范大学 | A kind of image characteristics extraction and training method based on Three dimensional convolution neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110503630A (en) | 2019-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110503630B (en) | Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model | |
| CN110070074B (en) | Method for constructing pedestrian detection model | |
| CN106295124B (en) | The method of a variety of image detecting technique comprehensive analysis gene subgraph likelihood probability amounts | |
| CN108288271A (en) | Image detecting system and method based on three-dimensional residual error network | |
| CN105160400A (en) | L21 norm based method for improving convolutional neural network generalization capability | |
| CN111738363B (en) | Alzheimer disease classification method based on improved 3D CNN network | |
| CN109685768A (en) | Lung neoplasm automatic testing method and system based on lung CT sequence | |
| CN107133496B (en) | Gene feature extraction method based on manifold learning and closed-loop deep convolution double-network model | |
| CN110889852A (en) | Liver segmentation method based on residual error-attention deep neural network | |
| CN111862009B (en) | Classifying method of fundus OCT (optical coherence tomography) images and computer readable storage medium | |
| CN111009321A (en) | Application method of machine learning classification model in juvenile autism auxiliary diagnosis | |
| CN112465905A (en) | Characteristic brain region positioning method of magnetic resonance imaging data based on deep learning | |
| CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
| CN113782184A (en) | Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning | |
| CN115375711A (en) | Image segmentation method of global context attention network based on multi-scale fusion | |
| CN115496720A (en) | Gastrointestinal cancer pathological image segmentation method based on ViT mechanism model and related equipment | |
| CN112508884A (en) | Comprehensive detection device and method for cancerous region | |
| CN115496953A (en) | Brain network classification method based on space-time graph convolution | |
| CN116091490A (en) | Lung nodule detection method based on YOLOv4-CA-CBAM-K-means++ -SIOU | |
| CN116825363B (en) | Early lung adenocarcinoma pathological type prediction system based on fusion deep learning network | |
| CN112861881A (en) | Honeycomb lung recognition method based on improved MobileNet model | |
| CN112488996A (en) | Inhomogeneous three-dimensional esophageal cancer energy spectrum CT (computed tomography) weak supervision automatic labeling method and system | |
| CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
| Guo et al. | Pathological Detection of Micro and Fuzzy Gastric Cancer Cells Based on Deep Learning. | |
| CN115862112A (en) | Target detection model for facial image acne curative effect evaluation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |