CN110189299B - Cerebrovascular event automatic identification method and system based on MobileNet - Google Patents
Cerebrovascular event automatic identification method and system based on MobileNet Download PDFInfo
- Publication number
- CN110189299B CN110189299B CN201910324180.1A CN201910324180A CN110189299B CN 110189299 B CN110189299 B CN 110189299B CN 201910324180 A CN201910324180 A CN 201910324180A CN 110189299 B CN110189299 B CN 110189299B
- Authority
- CN
- China
- Prior art keywords
- image data
- carotid artery
- layer
- mobilenet
- carotid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the crossing field of computer technology and medical images, and discloses a method and a system for automatically identifying cerebrovascular events based on MobileNet, wherein the method comprises the following steps: (1) acquiring two-dimensional carotid artery initial ultrasonic image data; (2) segmenting the carotid adventitia to obtain ROI image data; (3) constructing a deep learning network based on the MobileNet, and training; (4) and inputting the ROI image data into a trained deep learning network for testing to obtain a prediction result of whether the cerebrovascular event corresponding to the two-dimensional carotid artery initial ultrasonic image data occurs or not, thereby automatically identifying the cerebrovascular event. The invention adopts a deep learning network based on MobileNet, automatically extracts the ultrasonic carotid artery image characteristics by using a deep learning method, automatically identifies the cerebrovascular events, and can effectively solve the problems of strong subjectivity and redundancy of characteristics of manually defined characteristics.
Description
Technical Field
The invention belongs to the field of intersection of computer technology and medical images, and particularly relates to a method and a system for automatically identifying a cerebrovascular event based on MobileNet, which can automatically identify the cerebrovascular event through a carotid artery ultrasonic image based on the MobileNet.
Background
Cerebrovascular diseases have become the most deadly disease in China, and much and intensive research is carried out on the problem of automatic identification of cerebrovascular events. In medicine, the pathological basis for the occurrence of cerebrovascular events is atherosclerosis, manifested as thickening of the intima-media or plaque formation. Plaque rupture can block blood vessels, leading to cerebral ischemia and hypoxia, and ultimately to cerebrovascular events (e.g., cerebral infarction). The carotid artery is connected with the heart and the brain, has a shallow position surface and a simple structure, and becomes an important window for observing atherosclerotic lesions of a body and evaluating the stability of plaques.
In the past research on the characteristic image of the ultrasonic carotid artery image, the plaque is qualitatively described mainly by researching the vulnerability of the plaque, and the method comprises the following steps: ulcers, intraplaque hemorrhage, lipid rich cores, thin fibrous caps, and the like. In recent years, methods of machine learning have also been applied to this field. The traditional machine learning method is mainly to quantitatively describe the plaque by extracting the texture features of the plaque, and then to classify the plaque by a classifier. The texture features include: gray level co-occurrence matrices, gray level difference matrices, gray level run-length matrices, Law's texture features, fourier spectral analysis, and the like. The classifier includes: AdaBoost classifier, K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and the like.
Because the features need to be manually defined and the features are of various types, feature selection or dimension reduction processing is often required to solve the problem of feature redundancy, and the results are greatly different due to the application of different classifiers. Therefore, the method and the system for automatically extracting the ultrasonic carotid artery image features to automatically identify the cerebrovascular events are of great significance.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention aims to provide a cerebrovascular event automatic identification method and a cerebrovascular event automatic identification system based on MobileNet, wherein the method overall process treatment and the setting mode of each functional module component in a corresponding system device are improved, a deep learning network based on MobileNet is adopted, the ultrasonic carotid artery image characteristics are automatically extracted by using a deep learning method, the cerebrovascular event is automatically identified, and the problems of strong subjectivity of manually defined characteristics and redundancy of the characteristics can be effectively solved; in addition, the method and the corresponding system automatically select the characteristics, and solve the problem that the characteristics need to be manually selected in the traditional method.
To achieve the above object, according to one aspect of the present invention, there is provided a method for automatically identifying cerebrovascular events based on MobileNet, comprising the steps of:
(1) acquiring two-dimensional carotid artery initial ultrasonic image data;
(2) taking an image area of a part surrounded by a carotid adventitia in the two-dimensional carotid artery initial ultrasound image data as a region of interest ROI, and segmenting the carotid adventitia from the two-dimensional carotid artery initial ultrasound image data acquired in the step (1) to obtain ROI image data after segmenting the carotid adventitia;
(3) constructing a deep learning network based on the MobileNet, and training;
wherein the MobileNet-based deep learning network has a structure of separate convolutions; the deep learning network takes ROI image data as input and takes whether cerebrovascular events occur as output;
the training takes the ROI image data of a training set corresponding to the actual occurrence of a cerebrovascular event as a positive sample, and takes the ROI image data of the training set corresponding to the actual non-occurrence of the cerebrovascular event as a negative sample;
(4) and (3) inputting the ROI image data obtained in the step (2) into the trained deep learning network obtained in the step (3) for testing to obtain a prediction result of whether the cerebrovascular event corresponding to the two-dimensional carotid artery initial ultrasonic image data obtained in the step (1) occurs or not, so that the cerebrovascular event is automatically identified.
As a further preferred aspect of the present invention, in the step (1), the two-dimensional carotid artery initial ultrasound image data is a set of two-dimensional ultrasound image data of carotid artery cross sections containing plaque, which respectively correspond to left and right carotid arteries of a patient with plaque in the carotid arteries;
correspondingly, in the step (2), the obtaining of the ROI image data after carotid artery adventitia segmentation specifically means obtaining a set of two ROI image data after carotid artery adventitia segmentation corresponding to the left and right carotid arteries respectively;
in step (3), the training set ROI image data of the positive sample and the training set ROI image data of the negative sample each include a plurality of sets of ROI image data after carotid artery adventitia segmentation, where any set of ROI image data after carotid artery adventitia segmentation including a plaque is specifically two ROI image data after carotid artery adventitia segmentation including a plaque corresponding to the left and right carotid arteries, respectively;
in the step (4), the ROI image data obtained in the step (2) after the segmentation of the carotid artery adventitia is simultaneously input into the trained deep learning network for testing.
As a further preferred aspect of the present invention, in the step (3), the MobileNet-based deep learning network includes 8 layers of detail structures, wherein the layer 1 detail structure and the layer 8 detail structure are convolutional layers; the layer 2-7 detail structures correspond to 3 separate convolution structures;
preferably, the detail structure convolution layer of the layer 1 is a shared convolution layer, and the convolution kernel parameters are obtained by jointly iterating left and right carotid artery ultrasonic image data adopted in the training stage; two operations are arranged behind the output of the detailed structure of the layer 1, one is a channel connection operation, the other is a shuffle operation, and the shuffle operation is a shuffle layer with a channel of 2, which is specifically set by referring to the shuffle idea in shuffle net; any 1 separated convolution structure in the detail structures of the 2 nd to 7 th layers is composed of 1 depth convolution layer and 1 point-by-point convolution layer, wherein a jump connection is arranged between the outputs of the detail structures of the 3 rd layer and the 5 th layer based on the ResNet residual error idea so as to prevent gradient disappearance; a global pooling layer is arranged after the output of the 7 th layer detail structure and is used for performing global pooling operation; the 8 th layer of detail structure convolution layer is an output convolution layer and is used for outputting two types of values representing whether cerebrovascular events occur or not;
preferably, the MobileNet-based deep learning network further uses weight attenuation and BatchNormalization, and uses dropout after the global pooling layer.
As a further preferred embodiment of the present invention, in the step (3), for the training, the learning rate in the training solution parameter is preferably set to 0.0008.
As a further preferable mode of the present invention, in the step (1), the two-dimensional ultrasound image data of any one of the carotid artery cross sections containing the plaque is obtained by scanning the carotid artery cross sections with an ultrasound probe along a vertical direction perpendicular to the carotid artery cross sections, and taking the cross section two-dimensional ultrasound image data where the area of the carotid artery plaque is the largest as the two-dimensional ultrasound image data of the carotid artery cross sections containing the plaque;
preferably, the two-dimensional carotid artery initial ultrasound image data corresponds to the common neck, sinus or bifurcation of the vessels inside and outside the neck.
As a further preferred aspect of the present invention, in the step (2), the dividing of the carotid artery adventitia is performed by a semi-automatic dividing method, which specifically includes the following steps:
(2-1) reading two-dimensional carotid artery initial ultrasonic image data;
(2-2) manually marking at least 8 points at intervals on the carotid adventitia boundary as initial points;
(2-3) interpolating the initial point by using an interpolation filter to obtain a carotid adventitia contour, wherein the contour contains carotid intima-media and plaque; and (4) discarding the image data of the region outside the contour to obtain ROI image data after the carotid adventitia is segmented.
According to another aspect of the present invention, the present invention provides a system for automatic identification of cerebrovascular events based on MobileNet, comprising:
the image acquisition module is used for acquiring two-dimensional carotid artery initial ultrasonic image data;
the image segmentation module is used for taking an image area of a part surrounded by a carotid artery adventitia in the two-dimensional carotid artery initial ultrasonic image data as a region of interest ROI, and segmenting the carotid artery adventitia from the acquired two-dimensional carotid artery initial ultrasonic image data to obtain ROI image data after segmenting the carotid artery adventitia;
the deep learning network based on the MobileNet is used for taking ROI image data as input and taking whether a cerebrovascular event occurs as output; wherein the MobileNet-based deep learning network has a structure of separate convolutions; the MobileNet-based deep learning network is specifically used for testing by taking ROI image data as input to obtain a prediction result of whether a cerebrovascular event corresponding to the two-dimensional carotid artery initial ultrasonic image data occurs or not, and automatically identifying the cerebrovascular event.
As a further preferred aspect of the present invention, in the image acquisition module, the two-dimensional carotid artery initial ultrasound image data is a set of two-dimensional ultrasound image data of carotid artery cross sections containing plaque, which respectively correspond to left and right carotid arteries of a patient with plaque in the carotid artery;
correspondingly, in the image segmentation module, obtaining the ROI image data after segmenting the carotid adventitia specifically means obtaining a group of two ROI image data after segmenting the carotid adventitia corresponding to the left and right carotid arteries respectively;
the MobileNet deep learning network is specifically used for testing by taking the group of two ROI image data after segmenting the carotid adventitia as input at the same time.
As a further preferred embodiment of the present invention, the MobileNet based deep learning network comprises 8 layers of detail structures, wherein the detail structures of layer 1 and layer 8 are convolution layers, and the detail structures of layers 2 to 7 correspond to 3 separate convolution structures;
preferably, the detail structure convolution layer of the layer 1 is a shared convolution layer, and the convolution kernel parameters are obtained by jointly iterating left and right carotid artery ultrasonic image data adopted in the training stage; two operations are provided behind the output of the layer 1 detail structure, one is a channel connection operation, the other is a shuffle operation, and the shuffle operation is a shuffle layer with a channel of 2 specifically set by referring to the shuffle idea in shuffle net; any 1 separated convolution structure in the detail structures of the 2 nd to 7 th layers is composed of 1 depth convolution layer and 1 point-by-point convolution layer, wherein a jump connection is arranged between the outputs of the detail structures of the 3 rd layer and the 5 th layer based on the ResNet residual error idea so as to prevent gradient disappearance; a global pooling layer is arranged after the 7 th layer detail structure is output and is used for performing global pooling operation; the 8 th layer of detail structure convolution layer is an output convolution layer and is used for outputting two types of values representing whether cerebrovascular events occur or not;
preferably, the MobileNet-based deep learning network further uses weight attenuation and BatchNormalization, and uses dropout after the global pooling layer.
As a further preferable aspect of the present invention, in the image acquisition module, the two-dimensional ultrasound image data of any one of the carotid artery cross sections containing the plaque is obtained by scanning the carotid artery cross sections with an ultrasound probe along a vertical direction perpendicular to the carotid artery cross sections, and taking the cross section two-dimensional ultrasound image data at a position where the carotid artery plaque area is maximum as the two-dimensional ultrasound image data of the carotid artery cross sections containing the plaque;
preferably, the two-dimensional carotid artery initial ultrasound image data corresponds to the common neck, sinus or bifurcation of the vessels inside and outside the neck.
Generally speaking, compared with the prior art, the technical scheme of the invention has the following characteristics and advantages that the cerebrovascular event is automatically identified through the network model based on the MobileNet:
(1) the invention utilizes the deep learning network to automatically extract the ultrasonic carotid artery image characteristics to automatically identify the cerebrovascular events, and can effectively solve the problems of strong subjectivity of manually defined characteristics and redundancy of the characteristics; meanwhile, the invention automatically selects the characteristics, and solves the problem that the characteristics need to be manually selected in the traditional method.
(2) Based on the rolling separation idea in the MobileNet and preferably referring to the shuffle idea in the shuffle net, the network structure is designed, the training network is reasonable in design, the solution parameter is reasonable in setting, and a guarantee is provided for accurate training.
The deep learning network is designed based on the idea of separating convolution in the MobileNet and has a structure of separating convolution. The network in the invention comprises 8 layers of detail structures, wherein the detail structures of the 1 st layer and the 8 th layer are common convolutional layers, and the detail structures of the 2 nd to 7 th layers correspond to 3 separate convolutional structures (any 1 separate convolutional structure is composed of 1 depth convolutional layer and 1 point-by-point convolutional layer). According to the invention, the detailed structure of the deep learning network based on the MobileNet is controlled, so that the jump connection can be designed by preferably taking the residual error idea as a reference, and the gradient can be prevented from disappearing. In addition, weight attenuation and BatchNormalization can be preferably used in the network, and dropout is used after the global pooling layer, so that the overfitting problem can be effectively solved.
In addition, the learning rate is preferably set to 0.0008 in the training process, and good model operation results can be obtained.
(3) The carotid artery adventitia is segmented, and the data contained in the adventitia is used for carrying out experiments, so that more information can be obtained compared with the method for segmenting the plaque alone, and particularly when a plurality of plaques exist in one carotid artery ultrasonic image, the problem of subjectivity of manually selecting the plaques is solved. The invention takes ultrasonic image data inside the two-dimensional carotid artery adventitia as ROI image data, and inputs the ROI image data into a deep learning network for training and testing.
(4) According to the invention, the MobileNet deep learning network is applied to the two-dimensional carotid artery ultrasonic image for the automatic identification of the cerebrovascular event for the first time, and based on the method, the risk prediction modeling of the cerebrovascular event can be further carried out, so that clinical early warning is carried out on a patient with high risk of prediction result, an intervention treatment scheme is formulated, and the occurrence of the cerebrovascular event is prevented. In addition, in the prior art, one piece of ultrasonic carotid artery image characteristic data of a patient is generally used as input, but the invention preferably adopts the combination of the ultrasonic carotid artery image characteristic data on the left side and the ultrasonic carotid artery image characteristic data on the right side of a human body to obtain more information, thereby better automatically identifying cerebrovascular events.
Drawings
Fig. 1 is a flow chart of the cerebrovascular event automatic identification method based on MobileNet of the present invention.
Fig. 2(a) and 2(b) are both two-dimensional ultrasound carotid artery image raw data, wherein fig. 2(a) is patient left carotid transection plaque block data, and fig. 2(b) is patient right carotid transection plaque block data.
Fig. 3(a) and 3(b) are ROI images for experiment, i.e. segmentation of the carotid adventitia includes intima-media and plaque data within the posterior-adventitia, wherein fig. 3(a) is ROI data of the left carotid artery cross section of the patient, and fig. 3(b) is ROI data of the right carotid artery cross section of the patient.
Fig. 4 is a diagram of a deep learning network structure employed in the present invention.
FIG. 5 is a ROC curve of the classification result obtained by the deep learning network.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1
The method for automatically identifying cerebrovascular events, as shown in fig. 1, comprises the following steps:
(1) two-dimensional carotid ultrasound image data is acquired. Actual two-dimensional carotid ultrasound images are acquired by the sonographer, with the carotid range preferably being the common neck, sinus, and bifurcation of the vessels inside and outside the neck. The doctor scans the ultrasonic probe perpendicular to the carotid artery cross section, and obtains plaque images of the carotid artery cross section on the left side and the right side of the patient by taking plaque data at the position with the maximum plaque area. Taking this embodiment as an example, the finally acquired plaque image data of this embodiment includes 333 patients, among which 116 patients with cerebrovascular events occur and 217 patients without cerebrovascular events. Fig. 2(a) and 2(b) are two-dimensional ultrasonic carotid artery image raw data, in which fig. 2(a) is patient left carotid transection plaque block data, and fig. 2(b) is patient right carotid transection plaque block data.
(2) The carotid adventitia can be segmented from the acquired two-dimensional carotid ultrasound image data based on the segmentation method in the prior art to obtain the adventitia contour. In this embodiment, the method for segmenting the carotid adventitia is a semi-automatic segmentation in the prior art, and the specific method is as follows:
(2-1) reading two-dimensional carotid artery ultrasound plaque image data;
(2-2) uniformly selecting 8 points at the carotid adventitia boundary interval as initial points;
(2-3) designing an interpolation filter: the filter is a symmetric filter obtained by minimizing the mean square error between the interpolated point and the ideal point. The length of the filter is 81 points and the cut-off frequency is 0.5.
And (2-4) interpolating the initial point by using an interpolation filter to obtain a carotid adventitia contour point, wherein the contour comprises carotid intima-media and plaque data, and pixels outside the contour are set to be 0. FIGS. 3(a) and 3(b) are image data of ROI after segmentation of carotid adventitia, wherein FIG. 3(a) is ROI data of left carotid cross section of patient, and FIG. 3(b) is ROI data of right carotid cross section of patient.
(3) And designing a deep learning network.
The deep learning network is designed according to the following method:
(3-1) using the ultrasonic image data of the left carotid artery and the right carotid artery of the patient as the input of the network, and using whether a cerebrovascular event occurs as the output of the network. Whether a cerebrovascular event has occurred may be identified as 0/1 or other binary indicia, for example, where a 0 may indicate that no cerebrovascular event has occurred and a 1 indicates that a cerebrovascular event has occurred.
And (3-2) designing the structure and model parameters of the deep learning network.
The structure of the deep learning network is designed based on a volume segregation thought in the MobileNet and by referring to a shuffle thought in the shuffle net. The network includes a shared convolutional layer, a shuffle layer, three separate convolutional layers, a global pooling layer, and an output convolutional layer. In separating convolutional layers, a jump connection is made by using the ResNet residual idea to prevent gradient disappearance. To solve the over-fitting problem, weight attenuation and Batchnormalization are used in the network, and dropout is used after the global pooling layer.
The model parameters are set as: the size of the input image is 112 x 112, and the gray value is normalized to be between 0 and 1; the convolution kernel size of the shared convolution layer is 3 x 16, and the step size is 1; the channel of the shuffle layer is set to be 2; the first separate convolutional layer channel is set to 64 with a step size of 2, the second separate convolutional layer channel is set to 64 with a step size of 1, and the third separate convolutional layer channel is set to 128 with a step size of 2. All convolutional layers were followed by relu6 as the excitation layer and BatchNormalization was performed. Fig. 4 is a diagram of a deep learning network architecture.
As shown in fig. 4, the deep learning network is designed based on the idea of separating convolution in MobileNet. The network comprises 8 layers of detail structures, wherein the layer 1 detail structure and the layer 8 detail structure are convolution layers, and the layer 2-7 detail structures correspond to 3 separate convolution structures. The 1 st detail structure convolution layer is a shared convolution layer, and convolution kernel parameters can be obtained by jointly iterating left and right carotid artery ultrasonic image data of a patient with plaque in a carotid artery adopted in a training stage; two operations are carried out after the output of the detailed structure of the layer 1, one is channel connection operation, the other is shuffle operation, and the shuffle operation refers to a shuffle idea in shuffle net and is a shuffle layer with a channel of 2; any 1 separation convolution structure in the detail structures of the 2 nd to 7 th layers is composed of 1 depth convolution layer and 1 point-by-point convolution layer, wherein the output of the 3 rd layer and the 5 th layer is subjected to jump connection by taking residual error as reference so as to prevent gradient disappearance, and global pooling operation is carried out after the output of the 7 th layer; the detail structure convolution layer of layer 8 is an output convolution layer with an output of 2, and represents whether cerebrovascular events occur or not.
(4) And (3) inputting the ultrasonic image data inside the two-dimensional carotid artery adventitia obtained in the step (2) into the deep learning network designed in the step (3) for training and testing to obtain a network model for automatically identifying the cerebrovascular event. The deep learning network training and testing method specifically comprises the following steps:
(4-1) dividing the patients into two groups according to whether the patients have cerebrovascular events, wherein the occurrence event group is recorded as 1 group, and the non-occurrence event group is recorded as 0 group;
(4-2) downsampling the data of 0 groups: the random grouping results in a comparable number of two subsets of data. By sample downsampling, the problem of sample imbalance can be solved, and the finally obtained prediction result has higher sensitivity and robustness.
And (4-3) mixing the two subsets of data in the step (4-2) with the data in the group 1 respectively to obtain two experimental sample data.
(4-4) performing five-fold cross validation grouping on the two experimental sample data respectively, inputting training data into the network, setting parameters of a training solution and initializing network parameters, and updating weights iteratively to obtain the network model for performing automatic identification of cerebrovascular events.
And selecting a binary cross entropy as the loss value in the parameter of the training solution, preferably setting the learning rate to be 0.0008, monitoring the loss value of the verification set, and reducing the learning rate to be 0.1 after 20 times of iteration when the loss value of the verification set is not changed any more. The weight decay penalty weight _ decay is set to 0.01. The trained batch _ size is set to 15 and the tested batch _ size is set to 32. In order to make the result more accurate, training set data needs to be disturbed and network parameters need to be initialized randomly during each training.
Testing the deep learning network, and evaluating the correctness and accuracy of the automatic cerebrovascular event identification method, specifically comprising the following steps:
and selecting a network structure with the loss value as small as possible, and testing by using the test data. Inputting the test data into a network, comparing the obtained predicted tag value with the real tag value of the test data, calculating the number of the data with correct prediction, thereby obtaining the accuracy of each group of test data, and finally calculating the average Accuracy (ACC). Meanwhile, the predicted label value is stored, an ROC curve is drawn together with the real label value, the AUC of the area under the ROC curve is calculated, the larger the AUC value is, the better the classification effect is, the more reasonable the output probability is, and the AUC range is between 0 and 1.
When testing data, due to the limitation of the number of patients, five-fold cross validation can be adopted, namely the data are randomly divided into five folds, one fold of data is used as a test set every time, and the other four fold of data is used as a training set. For each test data, five times of repeated training and testing are carried out, and the obtained average result is used as the result of the test set. Therefore, the independence of the training samples and the test samples can be ensured, and the classification effect is more robust.
Table 1 shows the first set of five-fold cross-validation results for carotid artery cross-sectional ROI image data, table 2 shows the second set of five-fold cross-validation results for carotid artery cross-sectional ROI image data, and table 3 shows the average of the two sets of results.
TABLE 1 first set of five-fold cross-validation results
TABLE 2 second set of five-fold cross-validation results
TABLE 3 average of two sets of results
From table 1 and table 2, the accuracy range of the test set is 82% to 89% when the five-fold cross validation is performed, and the AUC range is 88% to 95%. This is because the grouping is random, the number of training sets and test sets in each group is different, and there is a certain difference in data. Meanwhile, the accuracy is consistent with the result of AUC, and a positive correlation exists, which is consistent with the actual situation, and shows that the result has certain correctness and reliability.
From table 3, the accuracy of the final average result of the test set is about 85.2%, and the AUC value is about 91.5%, indicating that the method has better classification result.
FIG. 5 is a graph of ROC curves and corresponding AUC values for one set of test data in the grouped results. The AUC can be seen to be 91.29%, indicating that the method has high sensitivity and specificity, and good robustness.
Whether the brain event occurs or not is judged currently by acquiring two-dimensional carotid artery initial ultrasonic image data of a patient with plaque in the carotid artery. The ultrasound image used in the embodiment of the present invention is an image of a patient who is examined while being hospitalized, which is based on whether a cerebrovascular event occurs while the patient is hospitalized, that is, the ultrasound image is at the time of hospitalization, and generally, if the patient has no stroke while being hospitalized, the time interval of the subsequent stroke is at least one year or more, and the ultrasound image at the time may have a change, which requires to acquire ultrasound image data again. The brain event in the invention generally refers to the cerebral infarction, including acute cerebral infarction, lacunar infarction and the like; in some cases, although cerebrovascular events do not occur, symptoms such as slurred speech, and black eyes appear, and the cerebrovascular events are classified as not occurring.
Where the invention is not specified in detail, reference is made to the relevant prior art, for example, with respect to the concept of separating convolution in MobileNet, such as Howard a G, Zhu M, Chen B, et al. mobilenes: Efficient connected networks for mobile vision applications [ J ]. arXiv prepropressin: 1704.04861,2017; the shuffle idea can also be referred to the prior art, such as Zhang X, ZHou X, Lin M, et al. Shufflenet: An extreme electronic knowledge relational neural network for mobile devices [ C ]// Proceedings of the IEEE Conference on Computer Vision and Pattern recognition.2018: 6848-; weight attenuation and BatchNormalization can be referenced respectively to Loshchilov I, Hutter F.Fixing weight attenuation in adam [ J ]. arXiv prediction arXiv:1711.05101,2017. and Ioffe S, Szedy C.BatchNormalization: Accelerating network linking by reducing internal covariate shift [ J ]. arXiv prediction arXiv:1502.03167,2015; dropout can refer to Krizhevsky A, Sutskeeper I, Hinton G E.Imagenet classification with default specific neural networks [ C ]// Advances in neural information processing systems.2012: 1097-.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (12)
1. A cerebrovascular event automatic identification method based on MobileNet is characterized by comprising the following steps:
(1) acquiring two-dimensional carotid artery initial ultrasonic image data;
(2) taking an image area of a part surrounded by a carotid adventitia in the two-dimensional carotid artery initial ultrasound image data as a region of interest ROI, and segmenting the carotid adventitia from the two-dimensional carotid artery initial ultrasound image data acquired in the step (1) to obtain ROI image data after segmenting the carotid adventitia;
(3) constructing a deep learning network based on the MobileNet, and training;
wherein the MobileNet-based deep learning network has a structure of separate convolutions; the deep learning network takes ROI image data as input and takes whether cerebrovascular events occur as output;
the training takes the ROI image data of a training set corresponding to the actual occurrence of a cerebrovascular event as a positive sample, and takes the ROI image data of the training set corresponding to the actual non-occurrence of the cerebrovascular event as a negative sample;
(4) inputting the ROI image data obtained in the step (2) into the trained deep learning network obtained in the step (3) for testing to obtain a prediction result of whether a cerebrovascular event corresponding to the two-dimensional carotid artery initial ultrasound image data obtained in the step (1) occurs or not, so that the cerebrovascular event is automatically identified;
in the step (1), the two-dimensional carotid artery initial ultrasound image data is a group of two-dimensional ultrasound image data of carotid artery cross sections containing plaques, which respectively correspond to left and right carotid arteries of a patient with the carotid arteries containing plaques;
correspondingly, in the step (2), the obtaining of the ROI image data after carotid artery adventitia segmentation specifically means obtaining a set of two ROI image data after carotid artery adventitia segmentation corresponding to the left and right carotid arteries respectively;
in step (3), the training set ROI image data of the positive sample and the training set ROI image data of the negative sample each include a plurality of sets of ROI image data after carotid artery adventitia segmentation, where any set of ROI image data after carotid artery adventitia segmentation including a plaque is specifically two ROI image data after carotid artery adventitia segmentation including a plaque corresponding to the left and right carotid arteries, respectively;
in the step (4), the ROI image data obtained in the step (2) after the segmentation of the carotid artery adventitia is simultaneously input into the trained deep learning network for testing.
2. The method for automatically identifying cerebrovascular events based on MobileNet as claimed in claim 1, wherein in said step (3), said MobileNet based deep learning network comprises 8 layers of detail structures, wherein the layer 1 detail structure and the layer 8 detail structure are convolution layers, and the layer 2-7 detail structures correspond to 3 separate convolution structures;
moreover, the 1 st detail structure convolution layer is a shared convolution layer, and convolution kernel parameters are obtained by common iteration of left and right carotid artery ultrasonic image data adopted in a training stage; two operations are arranged behind the output of the detailed structure of the layer 1, one is a channel connection operation, the other is a shuffle operation, and the shuffle operation is a shuffle layer with a channel of 2, which is specifically set by referring to the shuffle idea in shuffle net; any 1 separated convolution structure in the detail structures of the 2 nd to 7 th layers is composed of 1 depth convolution layer and 1 point-by-point convolution layer, wherein a jump connection is arranged between the outputs of the detail structures of the 3 rd layer and the 5 th layer based on the ResNet residual error idea so as to prevent gradient disappearance; a global pooling layer is arranged after the 7 th layer detail structure is output and is used for performing global pooling operation; the level 8 detail structure convolution layer is an output convolution layer for outputting two types of values representing whether cerebrovascular events occur.
3. The method of claim 2, wherein the MobileNet based cerebrovascular event automatic identification method further uses weight attenuation and BatchNormalization, and uses dropout after the global pooling layer.
4. The MobileNet-based cerebrovascular event automatic identification method according to claim 1, wherein in the step (3), the learning rate in the training solution parameters is set to 0.0008 for the training.
5. The method according to claim 1, wherein in step (1), the two-dimensional ultrasound image data of any one of the carotid artery cross sections containing the plaque is obtained by scanning an ultrasound probe along a vertical direction perpendicular to the carotid artery cross section, and the two-dimensional ultrasound image data of the cross section where the area of the carotid artery plaque is the largest is taken as the two-dimensional ultrasound image data of the carotid artery cross section containing the plaque.
6. The method of claim 5 wherein said two-dimensional carotid artery initial ultrasound image data corresponds to the common neck, sinus or bifurcation of the vessels inside and outside the neck.
7. The method for automatically identifying cerebrovascular events based on MobileNet as claimed in claim 1, wherein in the step (2), the carotid artery adventitia is segmented by a semi-automatic segmentation method, which comprises the following steps:
(2-1) reading two-dimensional carotid artery initial ultrasonic image data;
(2-2) manually marking at least 8 points at intervals on the carotid adventitia boundary as initial points;
(2-3) interpolating the initial point by using an interpolation filter to obtain a carotid adventitia contour, wherein the contour contains carotid intima-media and plaque; and (4) discarding the image data of the region outside the contour to obtain ROI image data after the carotid adventitia is segmented.
8. An automatic identification system for cerebrovascular events based on MobileNet, which is characterized by comprising:
the image acquisition module is used for acquiring two-dimensional carotid artery initial ultrasonic image data;
the image segmentation module is used for taking an image area of a part surrounded by a carotid artery adventitia in the two-dimensional carotid artery initial ultrasonic image data as a region of interest ROI, and segmenting the carotid artery adventitia from the acquired two-dimensional carotid artery initial ultrasonic image data to obtain ROI image data after segmenting the carotid artery adventitia;
the deep learning network based on the MobileNet is used for taking ROI image data as input and taking whether a cerebrovascular event occurs as output; wherein the MobileNet-based deep learning network has a structure of separate convolutions; the MobileNet-based deep learning network is specifically used for testing by taking ROI image data as input to obtain a prediction result of whether a cerebrovascular event corresponding to the two-dimensional carotid artery initial ultrasonic image data occurs or not, and automatically identifying the cerebrovascular event;
in the image acquisition module, the two-dimensional carotid artery initial ultrasound image data is a group of two-dimensional ultrasound image data of carotid artery cross sections containing plaques, which respectively correspond to the left and right carotid arteries of a patient with plaques in the carotid arteries;
correspondingly, in the image segmentation module, obtaining the ROI image data after segmenting the carotid artery adventitia specifically means obtaining a group of two ROI image data after segmenting the carotid artery adventitia corresponding to the left and right carotid arteries respectively;
the MobileNet deep learning network is specifically used for testing by taking the group of two ROI image data after segmenting the carotid adventitia as input at the same time.
9. The MobileNet-based cerebrovascular event automatic recognition system according to claim 8, wherein said MobileNet-based deep learning network comprises 8 layers of detail structures, wherein the layer 1 detail structure and the layer 8 detail structure are convolution layers, and the layer 2-7 detail structures correspond to 3 separate convolution structures;
moreover, the 1 st detail structure convolution layer is a shared convolution layer, and convolution kernel parameters are obtained by common iteration of left and right carotid artery ultrasonic image data adopted in a training stage; two operations are provided behind the output of the layer 1 detail structure, one is a channel connection operation, the other is a shuffle operation, and the shuffle operation is a shuffle layer with a channel of 2 specifically set by referring to the shuffle idea in shuffle net; any 1 separated convolution structure in the detail structures of the 2 nd to 7 th layers is composed of 1 depth convolution layer and 1 point-by-point convolution layer, wherein a jump connection is arranged between the outputs of the detail structures of the 3 rd layer and the 5 th layer based on the ResNet residual error idea so as to prevent gradient disappearance; a global pooling layer is arranged after the output of the 7 th layer detail structure and is used for performing global pooling operation; the level 8 detail structure convolution layer is an output convolution layer for outputting two types of values representing whether cerebrovascular events occur.
10. The MobileNet-based cerebrovascular event automatic recognition system of claim 9, wherein the MobileNet-based deep learning network further uses weight attenuation and BatchNormalization, and uses dropout after the global pooling layer.
11. The MobileNet-based cerebrovascular event automatic identification system according to claim 8, wherein in the image acquisition module, any one of the two-dimensional ultrasound image data of carotid artery cross section containing plaque is obtained by scanning the ultrasound probe on each carotid artery cross section along the vertical direction perpendicular to the carotid artery cross section, and the two-dimensional ultrasound image data of the cross section where the area of carotid artery plaque is the largest is taken as the two-dimensional ultrasound image data of the carotid artery cross section containing plaque.
12. The MobileNet-based cerebrovascular event automatic identification system as claimed in claim 11, wherein said two-dimensional carotid artery initial ultrasound image data corresponds to common neck, sinus or a bifurcation of vessels inside and outside the neck.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910324180.1A CN110189299B (en) | 2019-04-22 | 2019-04-22 | Cerebrovascular event automatic identification method and system based on MobileNet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910324180.1A CN110189299B (en) | 2019-04-22 | 2019-04-22 | Cerebrovascular event automatic identification method and system based on MobileNet |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110189299A CN110189299A (en) | 2019-08-30 |
| CN110189299B true CN110189299B (en) | 2020-12-08 |
Family
ID=67714849
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910324180.1A Active CN110189299B (en) | 2019-04-22 | 2019-04-22 | Cerebrovascular event automatic identification method and system based on MobileNet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110189299B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111724365B (en) * | 2020-06-16 | 2021-11-09 | 中国科学院自动化研究所 | Interventional instrument detection method, system and device for endovascular aneurysm repair operation |
| CN113838028A (en) * | 2021-09-24 | 2021-12-24 | 无锡祥生医疗科技股份有限公司 | Carotid artery ultrasonic automatic Doppler method, ultrasonic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102800088A (en) * | 2012-06-28 | 2012-11-28 | 华中科技大学 | Automatic dividing method of ultrasound carotid artery plaque |
| CN103996194A (en) * | 2014-05-23 | 2014-08-20 | 华中科技大学 | Automatic intima-media membrane partitioning method based on ultrasound carotid artery image |
| CN108388833A (en) * | 2018-01-15 | 2018-08-10 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device and equipment |
| CN109259784A (en) * | 2018-08-27 | 2019-01-25 | 上海铱硙医疗科技有限公司 | AI prediction technique, device, equipment and the storage medium of cerebral infarction |
-
2019
- 2019-04-22 CN CN201910324180.1A patent/CN110189299B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102800088A (en) * | 2012-06-28 | 2012-11-28 | 华中科技大学 | Automatic dividing method of ultrasound carotid artery plaque |
| CN103996194A (en) * | 2014-05-23 | 2014-08-20 | 华中科技大学 | Automatic intima-media membrane partitioning method based on ultrasound carotid artery image |
| CN108388833A (en) * | 2018-01-15 | 2018-08-10 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device and equipment |
| CN109259784A (en) * | 2018-08-27 | 2019-01-25 | 上海铱硙医疗科技有限公司 | AI prediction technique, device, equipment and the storage medium of cerebral infarction |
Non-Patent Citations (4)
| Title |
|---|
| MobileNets: efficient convolutional networks for mobile vision applications;Howard A G;《arXiv preprint arXiv:1704.04861》;20170414;论文全文 * |
| 一种基于深度学习的颈动脉斑块超声图像识别方法;赵媛;《中国医疗器械信息》;20171231;论文全文 * |
| 基于卷积神经网络的颈动脉斑块超声图像特征识别;孙夏;《中国医疗器械信息》;20161231;论文全文 * |
| 基于深度置信网络的脑血管病风险预警研究;杨星;《中国优秀硕士学位论文全文数据库 医药卫生科技辑》;20160715(第7期);论文全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110189299A (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886273B (en) | CMR image segmentation and classification system | |
| CN109190690B (en) | Method for detecting and identifying cerebral microhemorrhage points based on SWI image of machine learning | |
| CN110236543B (en) | Alzheimer disease multi-classification diagnosis system based on deep learning | |
| Zhang et al. | Automated semantic segmentation of red blood cells for sickle cell disease | |
| Ilhan et al. | A fully automated hybrid human sperm detection and classification system based on mobile-net and the performance comparison with conventional methods | |
| CN112150428A (en) | Medical image segmentation method based on deep learning | |
| CN111951288B (en) | Skin cancer lesion segmentation method based on deep learning | |
| CN110889852A (en) | Liver segmentation method based on residual error-attention deep neural network | |
| CN109635846A (en) | A kind of multiclass medical image judgment method and system | |
| WO2022127500A1 (en) | Multiple neural networks-based mri image segmentation method and apparatus, and device | |
| CN112465905A (en) | Characteristic brain region positioning method of magnetic resonance imaging data based on deep learning | |
| CN112884788B (en) | Cup optic disk segmentation method and imaging method based on rich context network | |
| CN112446891A (en) | Medical image segmentation method based on U-Net network brain glioma | |
| CN114093501B (en) | Intelligent auxiliary analysis method for child movement epilepsy based on synchronous video and electroencephalogram | |
| Rachmatullah et al. | Convolutional neural network for semantic segmentation of fetal echocardiography based on four-chamber view | |
| WO2021212715A1 (en) | Schizophrenia classification and identification method, operation control apparatus, and medical equipment | |
| CN110189299B (en) | Cerebrovascular event automatic identification method and system based on MobileNet | |
| CN109191422B (en) | System and method for detecting early ischemic stroke based on conventional CT image | |
| CN115147600A (en) | GBM multi-mode MR image segmentation method based on classifier weight converter | |
| CN113786185B (en) | Static brain network feature extraction method and system based on convolutional neural network | |
| CN110827963A (en) | Semantic segmentation method for pathological image and electronic equipment | |
| CN112861881A (en) | Honeycomb lung recognition method based on improved MobileNet model | |
| Kassim et al. | Sensitivity of cross-trained deep cnns for retinal vessel extraction | |
| Xu et al. | A Multi-scale Attention-based Convolutional Network for Identification of Alzheimer's Disease based on Hippocampal Subfields | |
| CN114723937A (en) | Method and system for classifying blood vessel surrounding gaps based on nuclear magnetic resonance image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |