CN111079901A - Acute stroke lesion segmentation method based on small sample learning - Google Patents
Acute stroke lesion segmentation method based on small sample learning Download PDFInfo
- Publication number
- CN111079901A CN111079901A CN201911315831.7A CN201911315831A CN111079901A CN 111079901 A CN111079901 A CN 111079901A CN 201911315831 A CN201911315831 A CN 201911315831A CN 111079901 A CN111079901 A CN 111079901A
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolutional neural
- image
- network
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 34
- 208000006011 Stroke Diseases 0.000 title claims abstract description 28
- 230000003902 lesion Effects 0.000 title claims abstract description 27
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 43
- 238000012549 training Methods 0.000 claims abstract description 17
- 238000005259 measurement Methods 0.000 claims abstract description 8
- 238000010586 diagram Methods 0.000 claims abstract description 7
- 230000000694 effects Effects 0.000 claims abstract description 6
- 238000012360 testing method Methods 0.000 claims abstract description 6
- 238000005481 NMR spectroscopy Methods 0.000 claims 4
- 238000003709 image segmentation Methods 0.000 claims 1
- 238000003759 clinical diagnosis Methods 0.000 abstract description 3
- 238000002372 labelling Methods 0.000 abstract description 2
- 238000013528 artificial neural network Methods 0.000 description 9
- 238000003745 diagnosis Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 208000032382 Ischaemic stroke Diseases 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 238000013399 early diagnosis Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000001154 acute effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an acute stroke lesion segmentation method based on small sample learning. The method comprises the following steps: training a convolutional neural network by using a data sample with an image-level label, wherein the classification accuracy of the image is used as a measurement index; constructing a new convolutional neural network by using the trained convolutional neural network, and constructing an end-to-end convolutional neural network by using a characteristic diagram obtained from an input image by using the trained network; fixing the trained convolutional layer parameters, training a newly constructed convolutional neural network by using a small number of data samples of pixel-level labels, and taking the segmentation precision of the image as a measurement index; after training is finished, the segmentation effect of the network is verified on the test set of the pixel-level labels. The invention only uses a small amount of data samples of pixel level labels and data samples of image level labels, which greatly reduces the cost of labeling data, enhances the engineering operability to a certain extent and assists doctors in clinical diagnosis of patients with acute stroke.
Description
Technical Field
The invention relates to an acute stroke lesion segmentation method based on small sample learning and provided for clinical diagnosis of acute stroke patients.
Background
Acute ischemic stroke is an acute cerebrovascular disease seriously threatening human health, according to the latest epidemiological research, the standardized morbidity, annual morbidity and mortality of the stroke in China are 1114.8/10 ten thousand, 246.8/10 ten thousand and 114.8/10 ten thousand respectively, and the high morbidity and disability rate of the stroke cause heavy economic burden to the government. The early diagnosis and treatment of the acute ischemic stroke can obviously improve the prognosis of a patient, the magnetic resonance examination is crucial to the early diagnosis, and quantitative information such as the size, the signal and the position of a focus has great guiding significance on a subsequent treatment scheme, so that the image information can be quickly and accurately analyzed, and the diagnosis and treatment work of a clinician can be facilitated. However, nowadays, the contradiction between supply and demand of medical image analysis in China is continuously increased, the annual data growth rate is 63%, the doctor growth rate is only 2%, low-age doctors lack diagnosis experience, the regional development has heterogeneity, missed diagnosis is still difficult to avoid under double audits, and the overall image analysis level is low. Thus, there is a need to provide automated analytical methods for rapid localization and quantitative assessment of acute stroke lesions. Currently, there are many methods for segmenting lesions or organs based on deep learning, and these methods utilize the multi-scale image semantic features extracted by the convolutional neural network for segmentation processing. They require a large number of pixel-level labeled data samples, which increases the difficulty and cost of data acquisition.
In 2017, a neural network structure proposed by Chen obtains a segmentation precision of 0.67 on single mode weighted diffusion imaging (DWI); in order to fully utilize the context information of MRI, Zhang proposed a 3D-based neural network structure in 2018; in 2019, Liu proposes a residual convolutional neural network structure to segment acute stroke lesions in multi-modal MRI. These segmentation algorithms for acute stroke lesions all require a large number of data samples labeled at pixel level, which increases the collection cost of data samples, so that the algorithm research for lesion segmentation using a small number of data samples labeled at pixel level will be a development trend. In 2019, Zhao proposed a semi-supervised learning method to segment acute stroke lesions, which utilized prior knowledge that acute stroke lesions present high signals on DWI and low signals on Apparent Diffusion Coefficient (ADC). Although this method achieves a segmentation accuracy of 0.639 using only 15 pixel-level labeled data, a lack of a priori knowledge is available in some medical images. The more effective method is that under the condition of not needing prior knowledge, the segmentation of the acute stroke lesion can be carried out by utilizing a small amount of data samples of the pixel level labels. The small sample learning method can provide a thought for solving the problem.
Small sample learning was originally used for classification tasks, and typical representatives are: matching Networks proposed by Vinyals (Matching Networks), meta-learning (meta-learning) proposed by Ravi, and prototype Networks (Prototypical Networks) proposed by Jake. In subsequent studies, some researchers use small sample learning to perform semantic segmentation of images, and the first work is proposed by Shaban, which includes a branch supporting path and a path for query, which are composed of a neural network developed by Visual Geometry Group (VGG) at oxford university, however, this method is not a Full Connected Network (FCN), which results in that spatial information of images cannot be fully utilized. Then, Rakelly converts the supported ways and the inquired ways into a full convolution structure, and proposes the co-FCN. However, these methods all deal with natural images and, in order to prevent the over-fitting problem under small sample learning, they often use the pre-training parameters of classical neural networks. Due to the obvious difference between the medical image and the natural image, the medical image cannot achieve a good convergence effect under the study of a small sample by using the pre-training parameters of the related network.
At present, the acute stroke lesion segmentation method based on small sample learning is still deficient, and most of the methods are lesion segmentation in a supervision mode based on deep learning, so that a method which can effectively segment the acute stroke lesion by only a few data samples of pixel level labels and data samples of image level labels needs to be researched.
Disclosure of Invention
The invention provides an acute stroke lesion segmentation method based on small sample learning, which can achieve better segmentation accuracy by using a small number of data samples with pixel level labels and data samples with image level labels, and tries to approach or even exceed a segmentation result which can be obtained by a model of the data samples with a large number of pixel level labels. The method greatly reduces the cost of marking data, enhances the engineering operability to a certain extent, and assists doctors in clinical diagnosis of patients with acute stroke. The technical scheme adopted by the method comprises the following steps:
step 1: training a convolutional neural network by using a data sample with an image-level label, wherein the classification accuracy of an image is used as a measurement index of the network;
step 2: and (2) constructing a new convolutional neural network by using the convolutional neural network structure trained in the step (1) and parameters thereof, specifically, connecting the trained network with a new convolutional layer, and forming an end-to-end convolutional neural network by using a characteristic diagram obtained from an input image by using the trained network.
And step 3: fixing the parameters of the convolutional layer trained in the step 2, training a newly constructed convolutional neural network by using a small number of data samples of pixel-level labels, and taking the segmentation precision of the image as a measurement index of the network;
and 4, step 4: verifying the segmentation effect of the network obtained by training in the step 3 on the test set of the pixel-level label, and outputting a segmentation result;
further, when the result obtained in the step 1 is measured, the invention provides that a threshold value is set according to the data of the verification set, the output classification prediction value is larger than or equal to the threshold value, namely the prediction is correct, otherwise, the prediction is wrong.
Compared with the traditional acute stroke lesion segmentation method, the acute stroke lesion segmentation method aiming at small sample learning provided by the invention has the following advantages:
(1) aiming at the segmentation method of the acute stroke lesion, the invention provides a convolutional neural network trained by using the data sample of the image-level label, which can reduce the cost of labeling data and reduce the time for training the neural network.
(2) The invention provides a method for retraining a neural network constructed by a trained neural network under weak supervision and a new convolutional layer by using a small number of data samples of pixel-level labels, which can avoid the occurrence of network overfitting and effectively utilize multi-scale semantic information extracted by the convolutional layer on an image to form an end-to-end segmentation network.
Drawings
Fig. 1 is a schematic diagram of segmentation of a small sample learned acute stroke lesion according to the present invention.
FIG. 2 is a schematic diagram of a convolutional neural network connection.
Detailed Description
The method of the present invention is described in detail with reference to the accompanying drawings and examples.
A schematic diagram of segmentation of an acute stroke lesion based on small sample learning is shown in fig. 1. The method comprises the following general flow: firstly, data samples of image-level labels, including DWI images and ADC images, are subjected to channel fusion to generate data samples of two channels, and the convolutional neural network 1 is trained. And then, carrying out truncation processing on the trained convolutional neural network 1 to obtain a convolutional neural network 2, and copying the parameters of the network 1 to the corresponding network 2. And connecting the network 2 with a new convolutional neural network 3, training the newly constructed convolutional neural network by using a data sample of a pixel-level label, namely a support set image, and verifying the performance of the model on a test set, namely an inquiry set, of the pixel-level label after training is finished.
Step 1: training a convolutional neural network by using a data sample with an image-level label, wherein the classification accuracy of an image is used as a measurement index of the network;
the structure of the convolutional neural network 1 used in the invention refers to the structural form of the classical neural network VGG16, and the image-level label is calibrated by a professional physician, so that the accuracy of the data sample is ensured. The data samples input to the network 1 include DWI images and ADC images, which are normalized separately for the purpose of training the neural network, and then channel fusion, and the normalization process can be written as:
wherein xiIs the ith sample and μ and σ are the mean and standard deviation, respectively.
When the result obtained in the step 1 is measured, the invention provides that a threshold value is set according to the data of the verification set, the output classification prediction value is larger than or equal to the threshold value, namely the prediction is correct, otherwise, the prediction is wrong.
Step 2: and (2) constructing a new convolutional neural network by using the convolutional neural network structure trained in the step (1) and parameters thereof, specifically, connecting the trained network with a new convolutional layer, and forming an end-to-end convolutional neural network by using a characteristic diagram obtained from an input image by using the trained network.
The invention provides a method for constructing an end-to-end segmented network, which comprises the steps of carrying out truncation processing on a trained convolutional neural network 1 to obtain a convolutional neural network 2, copying parameters of the network 1 to the network 2, and connecting the convolutional neural network 2 with a convolutional neural network 3 to form the end-to-end segmented network, wherein the specific connection mode of the convolutional neural network is shown in figure 2.
And step 3: fixing the parameters of the convolutional layer trained in the step 2, retraining a newly constructed convolutional neural network by using a small number of data samples of pixel-level labels, and taking the segmentation precision of the image as a measurement index of the network;
the invention proposes to fix parameters in the convolutional neural network 2, and trains a newly constructed convolutional neural network by using data samples of pixel level labels, namely, image training of a support set, wherein the support set comprises a DWI image, an ADC image and corresponding pixel level labels. After training is completed, the model and parameters are saved.
And 4, step 4: verifying the segmentation effect of the network obtained by training in the step 3 on the test set of the pixel-level label, and outputting a segmentation result;
the invention provides a method for verifying the effect of a model on a query set of pixel-level labels and outputting segmentation results, wherein the query set comprises DWI images, ADC images and corresponding pixel-level labels, and a formula for measuring segmentation precision is as follows:
wherein G and P respectively represent an acute stroke lesion label and a predicted lesion, and | represents a segmentation region of the lesion.
Claims (5)
1. The acute stroke lesion segmentation method based on small sample learning is characterized in that a convolutional neural network is used for extracting semantic features of a nuclear magnetic resonance image of an acute stroke patient and used for acute stroke lesion segmentation, and comprises the following steps:
1) training a convolutional neural network by using a nuclear magnetic resonance image sample with an image-level label, wherein the classification accuracy of the image is used as a measurement index of the network;
2) constructing a new convolutional neural network by using the structure of the convolutional neural network trained in the step 1) and parameters thereof, specifically, connecting the trained network with a new convolutional layer, and constructing an end-to-end convolutional neural network by using a characteristic diagram obtained from an input image by using the trained network;
3) fixing parameters of the trained convolutional layer in the step 2), retraining a newly constructed convolutional neural network by using a small number of nuclear magnetic resonance image samples of pixel-level labels, and taking the segmentation precision of the image as a measurement index of the network;
4) verifying 3) on the test set of the pixel-level label to obtain the segmentation effect of the network, and outputting the segmentation result.
2. The method as claimed in claim 1, wherein a convolutional neural network is trained from the image-level labeled nmr image samples, the convolutional neural network is set as 10 convolutional layers, and finally a probability value is outputted, and the classification accuracy of the image is used as a measure of the network.
3. The method of claim 1, wherein the trained convolutional neural network is connected to a new convolutional layer, the new convolutional layer is set to 2 layers, and the end-to-end convolutional neural network is constructed by using a feature map obtained from the input image by the trained network.
4. The method as claimed in claim 1, wherein a small number of pixel-level labeled nuclear magnetic resonance image samples are set to 5, and a newly constructed convolutional neural network is retrained, during which the network parameters before the new convolutional layer are fixed, and the image segmentation accuracy is used as a measure of the network.
5. The method for acute stroke lesion segmentation based on small sample learning as claimed in claim 1, wherein the validity of the segmentation method is verified on a test set of pixel-level labels, and the overlapping rate of the segmentation result and the labels is used as a measure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911315831.7A CN111079901A (en) | 2019-12-19 | 2019-12-19 | Acute stroke lesion segmentation method based on small sample learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911315831.7A CN111079901A (en) | 2019-12-19 | 2019-12-19 | Acute stroke lesion segmentation method based on small sample learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111079901A true CN111079901A (en) | 2020-04-28 |
Family
ID=70315669
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911315831.7A Pending CN111079901A (en) | 2019-12-19 | 2019-12-19 | Acute stroke lesion segmentation method based on small sample learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111079901A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112686850A (en) * | 2020-12-24 | 2021-04-20 | 上海体素信息科技有限公司 | Method and system for few-sample segmentation of CT image based on spatial position and prototype network |
| CN113128513A (en) * | 2021-04-28 | 2021-07-16 | 西安微电子技术研究所 | Small sample training method based on target segmentation |
| CN113222061A (en) * | 2021-05-31 | 2021-08-06 | 北京理工大学 | MRI image classification method based on two-way small sample learning |
| CN113269784A (en) * | 2021-05-07 | 2021-08-17 | 上海大学 | Foreground segmentation method for small samples |
| CN113284126A (en) * | 2021-06-10 | 2021-08-20 | 安徽省立医院(中国科学技术大学附属第一医院) | Method for predicting hydrocephalus shunt operation curative effect by artificial neural network image analysis |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107203999A (en) * | 2017-04-28 | 2017-09-26 | 北京航空航天大学 | A kind of skin lens image automatic division method based on full convolutional neural networks |
| CN107862695A (en) * | 2017-12-06 | 2018-03-30 | 电子科技大学 | A kind of modified image segmentation training method based on full convolutional neural networks |
-
2019
- 2019-12-19 CN CN201911315831.7A patent/CN111079901A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107203999A (en) * | 2017-04-28 | 2017-09-26 | 北京航空航天大学 | A kind of skin lens image automatic division method based on full convolutional neural networks |
| CN107862695A (en) * | 2017-12-06 | 2018-03-30 | 电子科技大学 | A kind of modified image segmentation training method based on full convolutional neural networks |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112686850A (en) * | 2020-12-24 | 2021-04-20 | 上海体素信息科技有限公司 | Method and system for few-sample segmentation of CT image based on spatial position and prototype network |
| CN113128513A (en) * | 2021-04-28 | 2021-07-16 | 西安微电子技术研究所 | Small sample training method based on target segmentation |
| CN113128513B (en) * | 2021-04-28 | 2023-07-07 | 西安微电子技术研究所 | Small sample training method based on target segmentation |
| CN113269784A (en) * | 2021-05-07 | 2021-08-17 | 上海大学 | Foreground segmentation method for small samples |
| CN113269784B (en) * | 2021-05-07 | 2024-01-30 | 上海大学 | Foreground segmentation method for small samples |
| CN113222061A (en) * | 2021-05-31 | 2021-08-06 | 北京理工大学 | MRI image classification method based on two-way small sample learning |
| CN113222061B (en) * | 2021-05-31 | 2022-12-09 | 北京理工大学 | MRI image classification method based on two-way small sample learning |
| CN113284126A (en) * | 2021-06-10 | 2021-08-20 | 安徽省立医院(中国科学技术大学附属第一医院) | Method for predicting hydrocephalus shunt operation curative effect by artificial neural network image analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111079901A (en) | Acute stroke lesion segmentation method based on small sample learning | |
| CN109670510B (en) | Deep learning-based gastroscope biopsy pathological data screening system | |
| CN108573490B (en) | Intelligent film reading system for tumor image data | |
| CN109035263B (en) | Automatic brain tumor image segmentation method based on convolutional neural network | |
| CN110060774B (en) | Thyroid nodule identification method based on generative confrontation network | |
| CN108464840B (en) | Automatic detection method and system for breast lumps | |
| CN111028206A (en) | Prostate cancer automatic detection and classification system based on deep learning | |
| CN110390674B (en) | Image processing method, device, storage medium, equipment and system | |
| CN112766377B (en) | Left ventricle magnetic resonance image intelligent classification method, device, equipment and medium | |
| CN111862136A (en) | Multi-modal nuclear magnetic image ischemic stroke lesion segmentation method based on convolutional neural network | |
| CN113516210A (en) | Lung adenocarcinoma squamous carcinoma diagnosis model training method and device based on PET/CT | |
| CN114782307A (en) | Enhanced CT image colorectal cancer staging auxiliary diagnosis system based on deep learning | |
| Bano et al. | AutoFB: Automating fetal biometry estimation from standard ultrasound planes | |
| CN115019405A (en) | Multi-modal fusion-based tumor classification method and system | |
| CN114494215A (en) | Transformer-based thyroid nodule detection method | |
| CN114078137A (en) | Colposcope image screening method and device based on deep learning and electronic equipment | |
| CN112200810A (en) | Multi-modal automated ventricular segmentation system and method of use thereof | |
| CN116977338A (en) | Chromosome case-level abnormality prompting system based on visual semantic association | |
| CN116759076A (en) | Unsupervised disease diagnosis method and system based on medical image | |
| Sun et al. | Liver tumor segmentation and subsequent risk prediction based on Deeplabv3+ | |
| CN111340807A (en) | Nidus positioning core data extraction method, system, electronic equipment and storage medium | |
| CN114926486B (en) | Thyroid ultrasound image intelligent segmentation method based on multi-level improvement | |
| CN112686912B (en) | Acute stroke lesion segmentation method based on gradual learning and mixed samples | |
| CN116091446A (en) | Method, system, medium and equipment for detecting abnormality of esophageal endoscope image | |
| CN115409812A (en) | CT image automatic classification method based on fusion time attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200428 |