CN111340807A - Method, system, electronic device and storage medium for extracting core data of lesion location - Google Patents

Abstract

The invention discloses a focus positioning core data extraction method, a system, electronic equipment and a storage medium, for any image in a medical image data set, calculating and fusing the information entropy, the contrast value and the acceptance score value of the image, and calculating the core degree of the image; and (4) arranging all images in the medical image data set in a descending order according to the core degree, and extracting images with the core degree ranking k before as core data. Optimizing the information entropy by using the previous batch of core data and the medical data without pathology; and repeating the process to extract the core data with proper quantity. The invention continuously extracts the core data and simultaneously continuously optimizes the extraction mechanism, so that the extraction performance of the invention is continuously improved. Experiments prove that the method has high practicability, can reduce a large amount of data labeling burden, can train to obtain an excellent focus positioning model, can effectively assist doctors in diagnosis, and reduces misdiagnosis rate.

Description

Method, system, electronic device and storage medium for extracting core data of lesion location

technical field

The invention relates to the field of smart medical treatment, in particular to a method, system, electronic device and storage medium for extracting core data of focus location based on active learning.

Background technique

In recent years, artificial intelligence has become increasingly mature in theory and technology, bringing many conveniences to human daily life. Among them, the development of intelligent medical care is very rapid. For example, the algorithm based on deep learning proposed by Google can explain the signs of diabetic retinopathy; Ni uses deep learning to achieve high accuracy in abdominal organ segmentation and more. These deep learning technologies can assist doctors in identifying diseases, greatly reducing the burden on doctors and helping doctors make more accurate diagnoses. These studies reflect the effectiveness of deep learning technology in medical image analysis, but most of the current research on intelligent disease diagnosis focuses on disease identification, and in the medical field, lesion location information can help doctors make better diagnoses , and for the treatment of most diseases, obtaining the location information of the lesion part is indispensable. At present, the main method of locating the lesion is to rely on the doctor's judgment, which not only greatly increases the workload of the doctor, but also makes mistakes in judging the location of the lesion when the doctor is fatigued, which delays treatment. Therefore, with the help of deep learning-based target detection technology to complete the localization of lesions, intelligent medical treatment can be more accurate and comprehensive, so as to better assist doctors in their work.

The essence of deep learning is that the deep network automatically extracts features from a large amount of data. Data is the guarantee of network learning. The quality and quantity of data affect the performance of the network. And the development of deep learning in image recognition and target detection in the medical field largely relies on fully supervised learning methods, which require a large amount of strongly labeled data to learn. However, in the era of big data medical treatment, although there is a large amount of medical image data, such as X-ray films, CT images, etc., these data will contain some relatively inferior images with low resolution and high noise. In addition to the quality problem, another important problem is that most of these images are not labeled. Although most medical images may have disease labels, there are almost no labeling of lesion locations, so the development of target detection for medical images in deep learning is limited. . And target detection is an indispensable part of smart medical assistance technology, so how to solve this obstacle in the medical field target detection is very important. The most direct way to solve the above two problems is to label all unlabeled data and train the network. However, lesion labeling requires professional-oriented medical knowledge and skills, which is time-consuming, labor-intensive and expensive. Therefore, in the face of massive medical image data without lesion labeling, this method is difficult to implement, and there may be images of poor quality that affect the network. training. It can be seen that the use of deep learning technology to train target detection models for lesion localization requires a large amount of labeled data. The quality of existing data varies, and most of them are not labeled. Professionals are required to select and label data, which increases the workload of doctors, is time-consuming, labor-intensive, and expensive.

Aiming at the situation in deep learning that strong annotation is not obtained due to the high cost of annotation, two mainstream weakly supervised learning solutions, semi-supervised learning and active learning, are proposed. Semi-supervised learning focuses on learning with easy-to-use annotations. It does automatic or semi-automatic labeling with computers without the involvement of human experts. Although this method reduces the labeling cost, its labeling results rely too much on the model trained from the initial labeled data, so the accuracy of the labeling results cannot be guaranteed. Active learning focuses on reducing the number of samples that need to be labeled. Active learning selects the most valuable unlabeled data to be labeled by experts through query functions, and trains the target model through fewer labeled core data sets. Experts participate in the annotation to solve the above problem of relying too much on the benchmark model, the effect is more stable, and it is more suitable for the medical field. However, most of the current active learning research is oriented to image classification, and only a few researches are oriented to target detection, but the initial data set is required to contain some data with labeled information, and it is not used in the medical field. In a word, the existing active learning methods for object detection all require that the initial data set contains some data with annotated information, and need to interact with experts many times, which is not suitable for the actual situation in the medical field. At present, there is no suitable method for the medical field, which can reduce the training data while maintaining the accuracy of the model, realize the localization of lesions, and assist doctors in diagnosis.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a method, system, electronic device and storage medium for extracting core data of lesion location, which can extract core data from medical image data without any lesion labeling information, and solve the problem of intelligent In medical treatment, it is difficult to locate the lesions due to the large amount of unlabeled data and the different quality.

In order to solve the above-mentioned technical problems, the technical solution adopted in the present invention is: a method for extracting core data of lesion location, comprising the following steps:

S1. For any image in the medical image dataset, calculate and fuse the information entropy, contrast value, and inception score value of the image to calculate the core degree of the image;

S2. Arrange all the images in the medical image dataset in descending order according to the core degree, and extract the top k images of the core degree as the core data.

S3, using the last batch of core data and non-pathological medical data to optimize the information entropy;

S4. Repeat steps S1 to S3 to extract an appropriate amount of core data.

：The method of the invention does not require a large amount of lesion labeling data, solves the problems in the prior art that the data quality is different, the lesion labeling data is scarce, the labeling is troublesome, and the cost is expensive, and the core data can be extracted from the medical image data without any lesion labeling information initially. . Considering the importance of images under different evaluation indicators and retaining the characteristics of numerical values, the present invention designs a mean value fusion algorithm, and calculates the mean value for each evaluation index, that is, the core degree of image i in the medical image data set is calculated by the following formula

:

；

;

分别表示图像i的信息熵值、对比度值以及inception score值；i的取值范围为1,2,…,r；

分别表示图像j的信息熵值、对比度值以及inception score值；j的取值范围为1,2,…,r。Among them, r represents the number of images under the same evaluation index;

Represents the information entropy value, contrast value and inception score value of image i respectively; the value range of i is 1, 2, ..., r;

Represent the information entropy value, contrast value and inception score value of image j, respectively; the value range of j is 1, 2, ..., r.

The information entropy optimization process of the present invention is as follows: using the last batch of core data and non-pathological medical data to fine-tune the classification model, that is, fixing the parameter weights of the front layer of the classification model, using the extracted last batch of core data and non-pathological medical data The data training adjusts the weight of the last layer of the classification model, and replaces the original classification model with the fine-tuned classification model, thereby optimizing the information entropy.

，计算公式如下：In step S1, the calculation process of the information entropy value of the image i includes: learning and calculating the image i by using a classification model with pre-trained weights to obtain the information entropy of the image i, and obtain the information entropy of the image i.

,Calculated as follows:

；

;

表示分类模型预测图像i是类别c的置信度。N is the number of output categories of the classification model, i is the ith image of the positive sample set in the unlabeled dataset,

Indicates the confidence that the classification model predicts that image i is of class c.

In step S1, before calculating the contrast value of the image i, GLCM (Gray Level Co-occurrence Matrix) transformation is performed on the image i. The contrast based on the gray level co-occurrence matrix can more accurately represent the texture features of the image. The deeper the texture groove, the greater the contrast and the clearer the image, which is more conducive to the model learning features.

In step S1, the calculation process of the inception score value of the image i includes: cutting the image i to obtain n×n sub-blocks; calculating the inception score of each sub-block, synthesizing the inception scores of all the sub-blocks of the image i, and obtaining the image i's inception score. Accurately calculate the inception score of the image.

Corresponding to the above method, the present invention also provides a core data extraction system for focus location based on active learning, including:

The information entropy calculation module is used to calculate the information entropy of all images in the medical image dataset;

The contrast value calculation module is used to calculate the contrast value of all images in the medical image dataset;

The inception score value calculation module is used to calculate the inceptionscore value of all images in the medical image dataset;

The fusion module is used to calculate the core degree of each image according to the information entropy, contrast value and inception score value of each image;

The sorting module is used to sort all the images in the medical image dataset in descending order according to the core degree, and extract the top k images of the core degree as a batch of core data.

The optimization loop module is used to optimize the information entropy calculation module, and extract core data in a loop until an appropriate amount of core data is extracted.

The information entropy calculation module learns and calculates the image i by using a classification model with pre-trained weights to obtain the information entropy of the image i. The contrast value calculation module of the present invention includes:

A transformation unit, used to perform GLCM transformation on the images in the medical image dataset;

The calculation unit is used to calculate the contrast value of the transformed image.

The inception score value calculation module of the present invention includes:

a cutting unit, configured to perform cutting processing on the images in the medical image data set, and cut each image into n×n sub-blocks;

The inception score calculation unit is used to calculate the inception score of each sub-block;

The synthesis unit is used to synthesize the inception score of n × n sub-blocks to obtain the inception score of the image.

The optimized cycle module of the present invention includes:

The optimization unit is used to fine-tune the classification model in the information entropy calculation module using migration learning, that is, to fix the parameter weights of the previous layers of the network, and only use these data to train and adjust the weights of the last layers. And the fine-tuned model replaces the original model in the selection module.

The loop unit performs the operations of the information entropy calculation module, the contrast value calculation module, the inception score value calculation module, the fusion module, the sorting module and the optimization unit in the optimization loop module in a loop, until an appropriate amount of core data is extracted.

As an inventive concept, the present invention also provides an electronic device for extracting core data of lesion location, which includes a processor; the processor is used for executing the above method.

Preferably, in order to facilitate data acquisition, the electronic device of the present invention further includes a data acquisition module for acquiring medical images and transmitting the lesion images to the processor.

As an inventive concept, the present invention also provides a computer storage medium storing a program; the program is used to execute the above method.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention solves the problems that the model training of lesion location in the current stage of intelligent medical care relies on a large amount of lesion labeling data, and the data quality is different and the lesion labeling data is scarce, which is troublesome and expensive to label. Extracting core data from medical image data can reduce the burden of labeling and the number of interactions for doctors, and use the core data extracted by the present invention to train a target detection model for effectively locating lesions, so as to assist doctors in diagnosis, reduce the work pressure of doctors, and promote The development of target detection in smart medicine;

2. The present invention continuously optimizes its extraction mechanism while continuously extracting core data, so that the extraction performance of the present invention is continuously improved. Experiments have proved that the present invention has high practicability, can reduce the burden of labeling a large amount of data, and effectively assist doctors in diagnosis.

Description of drawings

Fig. 1 is the flow chart of the method of the present invention;

Fig. 2 is the system structure diagram of the present invention;

Figure 3 is a schematic diagram of the network structure of Inception v3 in the selection module of the present invention; wherein, (a) module one in Inception v3; (b) module two in Inception v3; (c) module three in Inception v3; (d) Inception v3 model training the network structure;

4 is a schematic structural diagram of a contrast value calculation module module according to an embodiment of the present invention;

5 is a schematic structural diagram of an inception score value calculation module according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an optimized circulation module according to an embodiment of the present invention.

Detailed ways

The invention adopts the idea of active learning and designs a core data extraction method, which can extract core data from medical image data without any lesion labeling information, which can be used for the training of the detection model of lesion localization target, and can solve the problem of the large amount of data in intelligent medical care. The development of lesion localization is hindered by the uneven quality of labeling.

The development of lesion localization in smart medical care relies on deep learning-based target detection technology, which requires a large amount of labeled data. In medical big data, the quality of medical images varies, and some have quality problems such as noise. In addition, most image data have no lesion location labeling, and labeling requires medical knowledge, which is time-consuming and labor-intensive, and the acquisition cost is too high. The present invention will extract the core data from the data without lesion annotation through the designed selection module (ie, the extraction system of the present invention). In order to accurately extract the core data, the selection module of the present invention includes three image evaluation indicators to evaluate the image. The image is calculated by considering the quality of the image itself and selecting image quality evaluation indicators such as contrast based on the gray level co-occurrence matrix (GLCM). Considering the model training to calculate the image information entropy, the training of the target detection model and the image data set itself are considered, and the Inceptionscore is introduced. It is a metric often used to evaluate the images generated by GANs to evaluate the clarity and variety of the generated images. Generally, a group of images is discriminated, and the larger the value, the clearer and more diverse the image. The present invention designs an effective fusion algorithm for fusion to obtain image coreness. Considering the importance of images under different evaluation indicators and retaining the characteristics of numerical values, the present invention designs a mean value fusion algorithm, calculates the mean value for each evaluation index, and uses the mean value to process the original data , and finally fuse to get the image core degree, the formula is as follows:

；

;

分别表示图像i的信息熵值，对比度值以及inception score的值；i的取值范围为1,2,…,r；

分别表示图像j的信息熵值、对比度值以及inception score值；j的取值范围为1,2,…,r。where r represents the number of images under the same evaluation index,

Represent the information entropy value, contrast value and inception score value of image i respectively; the value range of i is 1, 2, ..., r;

Represent the information entropy value, contrast value and inception score value of image j, respectively; the value range of j is 1, 2, ..., r.

The method of the present invention mainly includes the following three stages: the first stage extracts core data from the unlabeled data set through the selection module; the second stage optimizes the selection module through the extracted core data; the third stage repeats the above two steps to obtain An appropriate amount of core data is finally handed over to human experts to label specific pathological locations.

As shown in Figure 1, the present invention extracts core data according to the core degree, and the specific steps of the above-mentioned process are as follows:

Step 1: First perform GLCM transformation on the image and calculate its contrast value. Its value reflects the sharpness of the image and the depth of the texture grooves. The deeper the texture groove, the greater its contrast and the sharper the image.

Step 2: Learn and calculate the information entropy of the image by using a classification model with pre-trained weights, such as Inception v3 with ImageNet pre-trained weights.

Step 3: Here we cut the image to get n×n blocks (for example, n=5). Calculate the inception score for each block of the image, and finally synthesize the inception scores of all blocks of the image to obtain the inception score of the image.

Step 4: Use the designed fusion algorithm to fuse the values of the three groups of image evaluation indicators to obtain a core degree of the image that is representative of comprehensive evaluation.

Step 5: Rank the images in descending order by their coreness, and extract the top k images with coreness as a batch of core data.

In order to make the extraction of core data more accurate, the present invention adopts batch extraction, and uses the extracted core data to continuously optimize the selection module (that is, optimize the information entropy calculated in the second step, that is, optimize the information entropy calculation module), and the steps are as follows:

Step 1: The core data of the previous batch and the normal images without pathology in the unlabeled pool are used to fine-tune the Inception v3 with pre-weight in the selection module (for example, iterate 1000 rounds), that is, maintain the previous layer of the network. The initial weights are unchanged, and only these data are used to train and adjust the weights of the last layer.

Step 2: Replace the fine-tuned model with the original model in the selection module.

Repeat the above two stages until the right amount of core data is picked. Finally, it is handed over to doctors and experts, who will label these core medical images with specific lesion location information. These core data can be used for the training of the target detection model for locating lesions, and an excellent lesion localization model is obtained.

The architecture of the present invention is shown in Figure 2, which is mainly composed of four parts: (1) Unlabeled dataset pool. There are a large number of low-cost and easy-to-obtain pathological images without pathological location annotations in the unlabeled dataset, including a small number of normal medical images without pathology. (2) Select the module. The selection module includes an information entropy calculation module, a contrast value calculation module, an inceptionscore value calculation module, a fusion module and a sorting module. (3) Expert annotation. Doctors and experts annotate the location of the lesions on the core data set finally selected. (4) Iteratively updated (annotated) core dataset pool. Finally, the entire core data set is annotated by experts to obtain an annotated core data set.

As can be seen from FIG. 2, the extraction system of the embodiment of the present invention includes the following modules:

The information entropy calculation module is used to calculate the information entropy of all images in the medical image dataset;

The contrast value calculation module is used to calculate the contrast value based on the gray level co-occurrence matrix of all the images in the medical image dataset;

The inception score value calculation module is used to calculate the inceptionscore value of all images in the medical image dataset;

The fusion module is used to calculate the core degree of each image according to the information entropy, contrast value and inception score value of each image;

The sorting module is used to sort all the images in the medical image dataset in descending order according to the core degree, and extract the top k images of the core degree as a batch of core data.

The optimization loop module is used to optimize the information entropy module, and loop through the process of extracting a batch of core data until an appropriate amount of core data sets are extracted;

The above information entropy calculation module learns and calculates the image i by using a classification model with pre-trained weights to obtain the information entropy of the image i.

As shown in Figure 3, the classification model in the information entropy calculation module is very important, and Inception v3 has good performance in the classification field. Therefore, the present invention uses Inception v3 to learn images and calculate information entropy. The Inception v3 model has 46 layers, and the input image passes through the convolution layer (Conv), the pooling layer (Pool), and the fully connected layer (FC) to obtain the confidence of image classification. In addition, there are 3 types of Inception modules in the Inception v3 model, namely: inception modules with two consecutive 3×3 convolution kernels; inception modules that decompose n×n convolutions into consecutive n×1 and 1×n convolution kernels Module; an inception module that decomposes an n×n convolution into side-by-side n×1 and 1×n convolution kernels.

As shown in Figure 4, the contrast value calculation module includes:

A transformation unit, used to perform GLCM transformation on the images in the medical image dataset;

The calculation unit is used to calculate the contrast value of the transformed image.

As shown in Figure 5, the inception score value calculation module includes:

a cutting unit, configured to perform cutting processing on the images in the medical image data set, and cut each image into n×n sub-blocks;

The inception score calculation unit is used to calculate the inception score of each sub-block;

The synthesis unit is used to synthesize the inception score of n × n sub-blocks to obtain the inception score of the image.

As shown in Figure 6, the optimization loop module includes:

The optimization unit uses the last batch of core data and non-pathological medical data extracted by the sorting module to fine-tune the classification model in the information entropy calculation module, that is, to fix the parameter weights of the previous layers of the classification model, and use the extracted last batch of core data and No pathological medical data training to adjust the weight of the last layer, and use the fine-tuned classification model to replace the original classification model;

The loop unit performs the operations of the information entropy calculation module, the contrast value calculation module, the inception score value calculation module, the fusion module, the sorting module and the optimization loop module in a loop until an appropriate amount of core data is extracted.

Claims (10)

1. A focus positioning core data extraction method is characterized by comprising the following steps:

s1, calculating and fusing the information entropy of any image in the medical image data set, the contrast value based on the gray level co-occurrence matrix and the acceptance score value, and calculating the core degree of the image;

preferably, the coring level of image i in the medical image dataset is calculated using the following equation

：

；

Wherein r represents the number of images under the same evaluation index;

respectively representing the information entropy value, the contrast value and the acceptance score value of the image i; the value range of i is 1,2, …, r;

respectively representing the information entropy value, the contrast value and the acceptance score value of the image j;

s2, arranging all images in the medical image data set according to the core degree in a descending order, and extracting images with the core degree of k before ranking as a batch of core data;

s3, optimizing the information entropy by using the previous batch of core data and medical data without pathology;

and S4, repeating the steps S1-S3, and extracting a proper amount of core data.

2. The lesion localization core data extraction method according to claim 1, wherein in step S1, the information entropy calculation process of the image i includes: learning and calculating the image i by using a classification model with pre-training weight to obtain the information entropy of the image i

The calculation formula is as follows:

；

n is the output category number of the classification model, i represents the ith image of the positive sample set in the label-free data set,

representing the confidence that the classification model predicted image i is of the class c;

preferably, the classification model is finely adjusted by using the previous batch of core data and the non-pathological medical data, that is, the parameter weight of the front layer of the classification model is fixed, the extracted previous batch of core data and the non-pathological medical data are used for training and adjusting the weight of the last layer of the classification model, and the finely adjusted classification model is used for replacing the original classification model, so that the information entropy is optimized.

3. The lesion localization core data extraction method according to claim 1, wherein in step S1, before calculating the contrast value of the image i, the image i is subjected to gray level co-occurrence matrix transformation.

4. The method for extracting lesion localization core data according to claim 1, wherein in step S1, the process of calculating the value of the concept score of the image i comprises the steps of cutting the image i to obtain n × n sub-blocks, calculating the concept score of each sub-block, and synthesizing the concept scores of all the sub-blocks of the image i to obtain the concept score of the image i.

5. A lesion localization core data extraction system, comprising:

the information entropy calculation module is used for calculating the information entropy of all images in the medical image data set;

the contrast value calculation module is used for calculating the contrast values of all images in the medical image data set based on the gray level co-occurrence matrix;

an inception score value calculation module for calculating the inception score values of all images in the medical image data set;

the fusion module is used for calculating the core degree of each image according to the information entropy, the contrast value and the acceptance score value of each image;

the sorting module is used for sorting all images in the medical image data set in a descending order according to the core degree and extracting images with the core degree of k before ranking as a batch of core data;

the optimization circulation module is used for optimizing the information entropy calculation module and circularly extracting the core data until a proper amount of core data is extracted;

the information entropy calculation module learns and calculates the image i by using a classification model with training weight to obtain the information entropy of the image i.

6. The lesion localization core data extraction system of claim 5, wherein the contrast value calculation module comprises:

the conversion unit is used for carrying out gray level co-occurrence matrix conversion on the images in the medical image data set;

and the calculating unit is used for calculating the contrast value of the converted image.

7. The lesion localization core data extraction system of claim 5, wherein the inceptoscore value calculation module comprises:

a cutting unit configured to perform cutting processing on the images in the medical image dataset, and cut each image into n × n subblocks;

an acceptance score calculating unit for calculating an acceptance score of each sub-block;

and the synthesis unit is used for synthesizing the acceptance score of the n × n sub-blocks to obtain the acceptance score of the image.

8. The lesion localization core data extraction system of claim 5, wherein the optimization loop module comprises:

the optimization unit is used for finely adjusting the classification model in the information entropy calculation module by using the last batch of core data and the non-pathological medical data extracted by the sequencing module, namely fixing the parameter weight of the front layer of the classification model, training and adjusting the weight of the last layer by using the last batch of extracted core data and the non-pathological medical data, and replacing the original classification model by using the finely adjusted classification model;

the loop unit is used for circularly executing the operations of the information entropy calculation module, the contrast value calculation module, the acceptance score value calculation module, the fusion module, the sequencing module and the optimization loop module until the core data with the proper quantity is extracted.

9. An electronic device for extracting lesion localization core data, comprising a processor; the processor is used for executing the method of one of claims 1 to 5; preferably, the system further comprises a data acquisition module for acquiring medical images and transmitting the lesion images to the processor.

10. A computer storage medium characterized by storing a program; the program is used for executing the method of one of claims 1 to 5.

Images