CN112733959A - Lung image classification method, classification network determination method and device - Google Patents
Lung image classification method, classification network determination method and device Download PDFInfo
- Publication number
- CN112733959A CN112733959A CN202110090885.9A CN202110090885A CN112733959A CN 112733959 A CN112733959 A CN 112733959A CN 202110090885 A CN202110090885 A CN 202110090885A CN 112733959 A CN112733959 A CN 112733959A
- Authority
- CN
- China
- Prior art keywords
- pictures
- image
- data set
- network
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Abstract
A method, an apparatus and a storage medium for determining a lung image classification network, and a lung image classification method, an apparatus and a storage medium are disclosed. The method for determining the lung image classification network comprises the steps of obtaining a lung image picture with a label and establishing a lung image data set; pre-training the complete DenseNet according to the pictures and the labels in the data set, and determining a first part of network; extracting image features of all pictures in the data set according to the first partial network, eliminating co-linearity of the image features, and determining the weight of each image feature; training a first linear classifier according to the image features and corresponding weights of the pictures in the data set, and determining a second partial network; and combining the first partial network and the second partial network to determine the lung image classification network.
Description
Technical Field
The present disclosure relates to, but not limited to, the field of image processing technologies, and in particular, to a method, an apparatus, and a storage medium for determining a lung image classification network, and a lung image classification method, an apparatus, and a storage medium.
Background
Timely and accurate diagnosis of pneumonia is crucial to treatment of patients, severe pneumonia not only increases treatment difficulty, but also may cause permanent damage to patients. The CT image diagnosis of pneumonia has the advantages of high accuracy, high speed, capability of judging the influence of pneumonia on the lung and the like, and is widely researched by researchers. Although the computer-aided diagnosis technology combining deep learning and computer vision has been developed greatly in recent years, the diagnosis accuracy is also improved, and in some fields, the accuracy of the computer-aided diagnosis technology reaches or even exceeds the accuracy of manual diagnosis, but the computer-aided diagnosis technology still faces many problems in the application field:
the interpretability is insufficient, doctors are required to make correct diagnosis in the medical field, reasoning for making the diagnosis is also required to be correct, and clear and reasonable reasoning processes cannot be given except for regions which are concerned;
the mobility is poor, even though the deep learning algorithm is better in performance, the model trained on the images generated by different equipment of the same type of medical images does not have good mobility, namely the model trained on the images generated by a batch of equipment is good in performance on a data set of the model, but the model does not well perform on the images generated by the same other equipment, which means that the model possibly does not utilize the most reasonable characteristics;
the robustness is poor, in the deep learning field, the neural network is easily attacked by resisting samples, the classification result can be changed by adding a disturbance which cannot be detected by human on an input picture, and the characteristic is very dangerous for the application in the medical field.
In view of the above problems, new solutions are to be proposed.
Disclosure of Invention
The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
The embodiment of the disclosure provides a method, a device and a storage medium for determining a lung image classification network, which can effectively improve the expression of a model, improve the robustness of the model and endow the model with certain interpretability. The determined classification network/model may be used offline and may be applied to a computer-aided diagnosis system.
The embodiment of the disclosure also provides a method, a device and a storage medium for classifying lung images, which utilize the determined classification network (model) to classify the lung images and help to perform auxiliary diagnosis.
The disclosed embodiment provides a method for determining a lung image classification network, which includes,
acquiring a lung image picture with a label, and establishing a lung image data set; the label includes: a positive swatch label and a negative swatch label;
pre-training the complete DenseNet according to the pictures and the labels in the data set, and determining the pre-trained DenseNet as a first part of network after removing the full connection layer;
extracting image features of all pictures in the data set according to the first partial network, eliminating co-linearity of the image features, and determining the weight of each image feature;
training a first linear classifier according to the image features and the corresponding weights of the pictures in the data set, and determining the trained first linear classifier as a second partial network;
and combining the first partial network and the second partial network to determine the lung image classification network.
In some exemplary embodiments, before pre-training the complete densnet network according to the pictures and labels in the data set, the method further comprises:
preprocessing pictures in the data set, wherein the preprocessing includes at least one of the following steps:
according to a preset size, cutting the pictures in the data set;
turning the pictures with preset proportion in the data set in a preset direction;
and carrying out normalization processing on the pictures in the data set.
In some exemplary embodiments, the pre-training of the complete DenseNet network according to the pictures and labels in the data set comprises:
and training the complete DenseNet by using a random gradient descent algorithm according to the pictures and the labels in the data set until the weight of the DenseNet meets a preset convergence condition.
In some exemplary embodiments, the training of the complete DenseNet network using the stochastic gradient descent algorithm includes:
training the complete DenseNet by using a random gradient descent algorithm and adopting a Focal local Loss function;
wherein the Focal local Loss function is:
Loss＝-α*(1-lp)γ*loglp
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the complete densnet network judges the picture entered into the complete densnet network as a positive sample.
In some exemplary embodiments, the extracting, according to the first partial network, image features of each picture in the data set, eliminating co-linearity of the image features, and determining a weight of each image feature includes:
respectively extracting the image characteristics of each picture in the data set according to the first partial network;
respectively determining the corresponding pseudo features of the pictures according to the image features of the pictures;
training a second linear classifier according to the image features and the pseudo features of the pictures in the data set;
respectively determining the positive sample probability and the negative sample probability of each picture according to the trained second linear classifier;
determining the weight of each image characteristic according to the positive sample probability and the negative sample probability of each picture;
wherein the weight of each image feature is equal to the ratio of the probability of the positive sample to the probability of the negative sample of the home picture.
In some exemplary embodiments, the image features of the picture comprise at least one dimension of features;
the determining the pseudo features corresponding to the pictures respectively according to the image features of the pictures comprises:
the following operations are respectively performed for each picture: respectively generating a pseudo feature for each dimension feature of the picture to obtain a group of pseudo features corresponding to the picture; the pseudo feature is a feature with the same dimensionality corresponding to one randomly extracted picture in other pictures which are not self in the data set;
training a second linear classifier based on image features and pseudo features of pictures in the dataset, comprising:
and training the second linear classifier by using a random gradient descent algorithm by using a Focal local Loss function and using the image characteristics of the pictures in the data set as negative samples and the pseudo characteristics corresponding to the pictures as positive samples.
In some exemplary embodiments, training the first linear classifier according to the image features and the corresponding weights of the pictures in the data set includes:
training the first linear classifier by using a Focal local Loss function and a random gradient descent algorithm;
when calculating the value of the Loss function of each picture, multiplying the value Loss of the Loss function by the corresponding weight w to obtain a new Loss function:
Lossre＝w*Loss
and performing back propagation and parameter updating on the first linear classifier by using the obtained new loss function, and determining the weight of the first linear classifier.
The embodiment of the present disclosure provides a method for classifying lung images, including,
acquiring a lung image picture to be classified;
classifying the lung image picture to be classified according to the lung image classification network determined by any one of the above lung image classification network determination methods.
The disclosed embodiment provides a further electronic device, which includes a memory and a processor, where the memory stores a computer program for performing lung image classification network determination or lung image classification, and the processor is configured to read and execute the computer program for performing lung image classification network determination or lung image classification to execute any one of the above methods for determining a lung image classification network or any one of the above methods for classifying a lung image.
The present disclosure provides a storage medium, in which a computer program is stored, where the computer program is configured to execute any one of the above methods for determining a pulmonary image classification network or any one of the above methods for classifying a pulmonary image when running.
Other aspects will be apparent upon reading and understanding the attached drawings and detailed description.
Drawings
FIG. 1 is a flowchart illustrating a method for classifying lung images according to an embodiment of the present disclosure;
FIG. 2 is an algorithm flow of an algorithm for eliminating collinearity in an embodiment of the present disclosure;
FIG. 3 is a pseudo code diagram of an algorithm for eliminating collinearity in an embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating a method for determining a lung image classification network according to another embodiment of the present disclosure;
fig. 5 is a flowchart illustrating a method for classifying lung images according to another embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The basis of the traditional deep learning model which can be applied to the medical image field is that collected medical image training data and test data obey the same distribution, however, the assumption is difficult to satisfy in the medical image field, and image data generated by different equipment obey different distributions, so that the deep learning model cannot have good performance. The difference of the distribution also limits the mobility of the deep learning model, so that the application range of the model is greatly limited. In order to solve this problem, many methods have been proposed, such as transfer learning, in which the distribution of test data is modeled before testing, and information about the distribution of test data is added to the training process in the training stage, so that the model can obtain better performance on the test data, but the information in the test set is often not easily obtained, so the transfer learning method is limited.
The application of deep learning to the pneumonia image classification task also has the specific problem of network structure design, and different schemes need to be designed aiming at the specific conditions of a data set in the specific problems of jump and connection selection, network depth and width selection, convolution kernel size and the like. Pneumonia image classification also has the problem of unbalanced data distribution, and often needs to be weighted among classes or special loss functions are used for ensuring the performance of the model. Specifically, the following problems need to be considered:
(1) the pneumonia image is larger than a common image in size, higher in resolution and deeper in layer number, and a network with more downsampling to fuse image features is more suitable for the features; the data size of the pneumonia image is small, the fitting capability of the network can be improved by deepening the network, and the problem of overfitting may occur on a data set. In order to balance the problems of large image size and small data size, a neural network with a proper depth needs to be selected, and the balance between the fitting capability and the feature fusion capability is ensured.
(2) The pneumonia images have the problem of unbalanced category, and patients with clinical manifestations possibly judged as pneumonia are recommended by doctors to take the images, so that more positive samples are obtained than negative samples, and the data distribution is completely different from that of actual encountered data. The deep learning model can perform well on data sets with roughly equivalent sample quantities of different types, so that the ideal effect cannot be achieved by directly applying the traditional deep learning model.
In the experiment, the network used in the embodiment of the disclosure tests the number of layers and the number of channels of the network, and combines the performances on a training set and a verification set to select the proper width and depth.
The manual pneumonia diagnosis through the image has the above limitations, and the treatment of assisting doctors through the computer-aided diagnosis is a good solution, and the classification task of researching the pneumonia image is helpful for improving the performance of the computer-aided diagnosis technology. The scheme of the embodiment of the disclosure researches the characteristics of medical images of pneumonia, designs a deep learning model in a targeted manner, and researches an experiment for automatically identifying and classifying pneumonia by a computer by combining the application of causal inference in the field of machine learning.
The following step numbers do not limit a specific execution order, and the execution order of some steps can be adjusted according to specific embodiments. The "first partial network", "second partial network", "first linear classifier", and "second linear classifier" referred to in the following description are used to distinguish the networks or linear classifiers referred to in the different steps, and do not limit the priority, execution order, or other attributes. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the solutions provided by the present disclosure. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that these specific details need not be employed to practice the present disclosure, and that the specific details may be employed in a manner known to those of ordinary skill in the art.
The scheme provided by the embodiment of the disclosure can effectively improve the expression of the model, improve the robustness of the model and endow the model with certain interpretability. The model can be used off-line and can be applied to a computer-aided diagnosis system.
Research shows that the co-linearity among data is one of the reasons for poor model performance and weak generalization capability, and the scheme disclosed by the embodiment of the invention utilizes a data weighting method from the perspective of reducing the co-linearity, so that the performance and the generalization capability of the model are improved, and the requirements of computer-aided diagnosis technology are better met.
Example one
An embodiment of the disclosure provides a method for classifying lung images, a process of which is shown in fig. 1, and the method includes:
5756 lung image pictures with labels are obtained from the Internet, and a lung image data set is established; wherein, including normal lung image picture, also include pneumonia patient's lung image picture, the label that corresponds promptly includes: a positive swatch label and a negative swatch label.
In some exemplary embodiments, a preset number of labeled lung image pictures may be obtained through other approaches to establish the lung image data set as the sample picture. And are not limited to the specific pathways and amounts described above.
randomly cutting the lung image into pictures with 224 x 224 pixels to ensure that the length and the width of the images are consistent; horizontally turning the image with the probability of 0.5 in the training process, and enhancing data; and normalizing the images in the lung image data set to enable the images in the data set to be independently and uniformly distributed.
In some exemplary embodiments, the cutting is performed in a preset size and position; or randomly cutting according to a preset size. The preset size is not limited to the above-mentioned exemplary sizes. The preset position is not limited to a specific position and may be set according to the sample picture condition.
In some exemplary embodiments, the data enhancement comprises: turning the preset direction of a preset proportion picture in all sample pictures; the preset direction includes a horizontal direction or a vertical direction.
pre-training the complete DenseNet network by using images (pictures) and labels in a data set, training to obtain the weight of the DenseNet network by using a random gradient descent algorithm and adopting a Focal local Loss function until the weight is converged;
wherein the loss function
Loss＝-α*(1-lp)γ*log lp (1)
Wherein alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the complete densnet network judges the picture entered into the complete densnet network as a positive sample.
removing the last full connection layer of the pretrained DenseNet obtained in the step 3 to be used as a first partial network, so that each picture passes through the first partial network to obtain the characteristics of the low-dimensional lung image picture;
and (4) weighting the image characteristics obtained in the step (4), and eliminating the collinearity of the data distribution through weighting.
The weight calculation process is shown in fig. 2, and includes:
(a) generating a pseudo feature: generating a pseudo feature for each dimension of image features obtained in the step 4, wherein the pseudo feature is randomly extracted from the features of the non-self samples in the features in the data set, generating a corresponding pseudo feature for each real image feature, and generating a corresponding group of pseudo features for a group of real image features of each image; in some exemplary embodiments, for example, each image has N-dimensional image features, and a dummy feature is formed for each dimension, where the N dummy features form a set of dummy features for the image.
(b) Training a linear classifier: training a linear classifier (marked as a second linear classifier) by using a random gradient descent algorithm, taking the image features of the real picture obtained in the step (4) as negative samples by adopting a Focal local Loss function, and taking the pseudo features generated according to the image features of the real picture in the step (a) as positive samples, so that the linear classifier (the second linear classifier) can correctly classify the two types of samples;
wherein the loss function
Loss＝-α*(1-lp)γ*loglp (2)
Wherein alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the linear classifier (second linear classifier) determines the sample to be a positive sample.
(c) Calculating sample weight: for the image feature of each real sample, the linear classifier (second linear classifier) outputs the ratio of the probability that the sample is a positive sample and a negative sample as the weight of the sample:
where x denotes the sample input to the linear classifier (second linear classifier) and w is the weight corresponding to sample x.
In some exemplary embodiments, the process (algorithm) of step 5 is illustrated in pseudo-code in FIG. 3.
and (4) training a linear classifier (marked as a first linear classifier) by using the image features obtained in the step (4) and the image feature weights obtained in the step (5), and taking the linear classifier (the first linear classifier) as a second partial network. And combining the first part network and the second part network to determine a classification network capable of classifying the lung image. And classifying the new lung image according to the classification network.
Wherein, training the linear classifier (the first linear classifier) to use a random gradient descent algorithm, adopting a Focal local Loss function, and multiplying the value of the Loss function by the sample weight when calculating the value of the Loss function of each sample to obtain a new Loss function:
Lossre＝w*Loss (3)
wherein w is the weight corresponding to the image feature of the sample (picture) calculated in step 5.
Loss is the value of the Loss function, and is calculated according to the following formula:
Loss＝-α*(1-lp)γ*log lp (4)
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the first linear classifier determines the sample to be a positive sample.
And then carrying out back propagation and parameter updating on the first linear classifier to obtain the weight of the linear classifier (the first linear classifier).
The classification network, which is a combination of the first partial network and the second partial network, can classify the new image. The classification network is also referred to as a classification model.
In some exemplary embodiments, the classification network obtained by combining the first partial network and the second partial network includes: directly connecting the first part of network with the second part of network to obtain the classified network; or, the output of the first part network is used as the input of the second part network to obtain the classification network.
The disclosed embodiments also provide relevant experimental data as follows:
experimental setup:
in the experiment, the labeled lung image data mentioned in the first embodiment is used, the data is divided into different parts, all the parts are not overlapped, the proportion of positive and negative samples of the different parts is different, and the samples of the different parts are distributed differently. The training of the classification network (model) of the first embodiment of the disclosure, the training of the common neural network model DenseNet and the training of the DenseNet which has used the Focal local are respectively performed on the same part of data, and other part of samples of the data set are used for verification and test, and the average accuracy and the variance of the accuracy of the test under different distributions and positive and negative sample ratios are calculated.
The experimental environment was the Linux 18.04 system, Python 3.8 environment, Pytorch version 1.4.
Experimental results and analysis:
the classification network (model) of the first embodiment of the disclosure, the trained neural network model DenseNet and the DenseNet using the Focal local are respectively trained and determined by using the same part of data, the weights of the networks are respectively obtained, other part of samples of the data set are used for verification and test, and the average accuracy and the variance of the accuracy of the test under different distributions and positive and negative sample ratios are calculated. The performance of different models on the same test set was compared. The results of the experiment are reported in table 1.
TABLE 1 test accuracy comparison table
As can be seen from table 1, the classification network (model) determined by the scheme of the first embodiment of the disclosure is improved compared with the average value of the accuracy of the common model, the variance of the accuracy is reduced, and the classification effect on the lung images is better. The data are relatively similar in performance on different distributed data, and the mobility is relatively good.
Example two
The embodiment of the present disclosure further provides a method for determining a lung image classification network, as shown in fig. 4, including:
step 41, acquiring a lung image picture with a label, and establishing a lung image data set; the label includes: a positive swatch label and a negative swatch label;
step 42, pre-training the complete DenseNet network according to the pictures and the labels in the data set, and determining the pre-trained DenseNet network as a first partial network after removing the full connection layer;
step 43, extracting image features of each picture in the data set according to the first partial network, eliminating co-linearity of the image features, and determining the weight of each image feature;
step 44, training a first linear classifier according to the image features and the corresponding weights of the pictures in the data set, and determining the trained first linear classifier as a second partial network;
and step 45, combining the first partial network and the second partial network to determine the lung image classification network.
In some exemplary embodiments, before pre-training the complete densnet network according to the pictures and labels in the data set, the method further comprises:
preprocessing pictures in the data set, wherein the preprocessing includes at least one of the following steps:
according to a preset size, cutting the pictures in the data set;
turning the pictures with preset proportion in the data set in a preset direction;
and carrying out normalization processing on the pictures in the data set.
In some exemplary embodiments, the flipping of the preset direction of the picture with the preset proportion in the data set includes:
turning the pictures with preset proportion in the data set in the horizontal direction;
or, turning over the pictures with preset proportion in the data set in the vertical direction.
In some exemplary embodiments, the pre-training of the complete DenseNet network according to the pictures and labels in the data set comprises:
and training the complete DenseNet by using a random gradient descent algorithm according to the pictures and the labels in the data set until the weight of the DenseNet meets a preset convergence condition.
In some exemplary embodiments, the training of the complete DenseNet network using the stochastic gradient descent algorithm includes:
training the complete DenseNet by using a random gradient descent algorithm and adopting a Focal local Loss function;
wherein the Focal local Loss function is:
Loss＝-α*(1-lp)γ*log lp
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs a diagram of a complete DenseNet network judging inputs to the complete DenseNet networkProbability of a tile being a positive sample.
In some exemplary embodiments, the extracting, according to the first partial network, image features of each picture in the data set, eliminating co-linearity of the image features, and determining a weight of each image feature includes:
respectively extracting the image characteristics of each picture in the data set according to the first partial network;
respectively determining the corresponding pseudo features of the pictures according to the image features of the pictures;
training a second linear classifier according to the image features and the pseudo features of the pictures in the data set;
respectively determining the positive sample probability and the negative sample probability of each picture according to the trained second linear classifier;
determining the weight of each image characteristic according to the positive sample probability and the negative sample probability of each picture;
wherein the weight of each image feature is equal to the ratio of the probability of the positive sample to the probability of the negative sample of the home picture.
In some exemplary embodiments, the image features of the picture comprise at least one dimension of features;
the determining the pseudo features corresponding to the pictures respectively according to the image features of the pictures comprises:
the following operations are respectively performed for each picture: respectively generating a pseudo feature for each dimension feature of the picture to obtain a group of pseudo features corresponding to the picture; the pseudo feature is a feature with the same dimensionality corresponding to one randomly extracted picture in other pictures which are not self in the data set;
training a second linear classifier based on image features and pseudo features of pictures in the dataset, comprising:
and training the second linear classifier by using a random gradient descent algorithm by using a Focal local Loss function and using the image characteristics of the pictures in the data set as negative samples and the pseudo characteristics corresponding to the pictures as positive samples.
In some exemplary embodiments, training the first linear classifier according to the image features and the corresponding weights of the pictures in the data set includes:
training the first linear classifier by using a Focal local Loss function and a random gradient descent algorithm;
when calculating the value of the loss function of each picture, multiplying the value of the loss function by the corresponding weight to obtain a new loss function:
Lossre＝w*Loss
wherein w is the corresponding weight, and Loss is the value of the Loss function;
loss is calculated according to the following formula:
Loss＝-α*(1-lp)γ*log lp
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the first linear classifier determines the sample to be a positive sample.
And performing back propagation and parameter updating on the first linear classifier by using the obtained new loss function, and determining the weight of the first linear classifier.
EXAMPLE III
An embodiment of the present disclosure further provides a method for classifying lung images, as shown in fig. 5, including:
step 51, acquiring a lung image picture to be classified;
step 52, classifying the lung image picture to be classified according to the lung image classification network determined by the method according to any one of the embodiments.
Example four
The embodiment of the present disclosure further provides a device for determining a lung image classification network, including:
the sample acquisition module is used for acquiring a lung image picture with a label and establishing a lung image data set; the label includes: a positive swatch label and a negative swatch label;
the first partial network determining module is configured to pre-train the complete DenseNet according to the pictures and the labels in the data set, and determine the pre-trained DenseNet as a first partial network after removing the full connection layer;
the weight determining module is arranged for extracting the image characteristics of each picture in the data set according to the first part of network, eliminating co-linearity of the image characteristics and determining the weight of each image characteristic;
the second partial network determining module is used for training the first linear classifier according to the image characteristics and the corresponding weight of the pictures in the data set and determining the trained first linear classifier as a second partial network;
a classification network determining module configured to combine the first partial network and the second partial network to determine the lung image classification network.
In some exemplary embodiments, the apparatus further comprises a pre-processing module;
the preprocessing module is configured to preprocess the pictures in the data set before the complete DenseNet network is pre-trained according to the pictures and the labels in the data set, and the preprocessing module at least includes one of the following:
according to a preset size, cutting the pictures in the data set;
turning the pictures with preset proportion in the data set in a preset direction;
and carrying out normalization processing on the pictures in the data set.
In some exemplary embodiments, the flipping of the preset direction of the picture with the preset proportion in the data set includes:
turning the pictures with preset proportion in the data set in the horizontal direction;
or, turning over the pictures with preset proportion in the data set in the vertical direction.
In some exemplary embodiments, the first partial network determination module is further configured to:
and training the complete DenseNet by using a random gradient descent algorithm according to the pictures and the labels in the data set until the weight of the DenseNet meets a preset convergence condition.
In some exemplary embodiments, the first partial network determination module is further configured to:
training the complete DenseNet by using a random gradient descent algorithm and adopting a Focal local Loss function;
wherein the Focal local Loss function is:
Loss＝-α*(1-lp)γ*log lp
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the complete densnet network judges the picture entered into the complete densnet network as a positive sample.
In some exemplary embodiments, the weight determining module is further configured to:
respectively extracting the image characteristics of each picture in the data set according to the first partial network;
respectively determining the corresponding pseudo features of the pictures according to the image features of the pictures;
training a second linear classifier according to the image features and the pseudo features of the pictures in the data set;
respectively determining the positive sample probability and the negative sample probability of each picture according to the trained second linear classifier;
determining the weight of each image characteristic according to the positive sample probability and the negative sample probability of each picture;
wherein the weight of each image feature is equal to the ratio of the probability of the positive sample to the probability of the negative sample of the home picture.
In some exemplary embodiments, the image features of the picture comprise at least one dimension of features;
the weight determining module respectively determines the pseudo features corresponding to the pictures according to the image features of the pictures, and the weight determining module comprises the following steps:
the following operations are respectively performed for each picture: respectively generating a pseudo feature for each dimension feature of the picture to obtain a group of pseudo features corresponding to the picture; the pseudo feature is a feature with the same dimensionality corresponding to one randomly extracted picture in other pictures which are not self in the data set;
training a second linear classifier based on image features and pseudo features of pictures in the dataset, comprising:
and training the second linear classifier by using a random gradient descent algorithm by using a Focal local Loss function and using the image characteristics of the pictures in the data set as negative samples and the pseudo characteristics corresponding to the pictures as positive samples.
In some exemplary embodiments, the second partial network determination module is further configured to:
training the first linear classifier by using a Focal local Loss function and a random gradient descent algorithm;
when calculating the value of the loss function of each picture, multiplying the value of the loss function by the corresponding weight to obtain a new loss function:
Lossre＝w*Loss
wherein w is the corresponding weight, and Loss is the value of the Loss function;
loss is calculated according to the following formula:
Loss＝-α*(1-lp)γ*log lp
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the first linear classifier determines the sample to be a positive sample.
The second partial network determination module is further configured to: and performing back propagation and parameter updating on the first linear classifier by using the obtained new loss function, and determining the weight of the first linear classifier.
EXAMPLE five
The embodiment of the present disclosure further provides a lung image classification device, including:
the image acquisition module is used for acquiring lung image pictures to be classified;
the classification module is configured to classify the lung image picture to be classified according to the lung image classification network determined by the method according to any one of the embodiments.
The disclosed embodiment also provides an electronic device, which includes a memory and a processor, where the memory stores a computer program for determining a pulmonary image classification network, and the processor is configured to read and execute the computer program for determining a pulmonary image classification network to perform any one of the above methods for determining a pulmonary image classification network.
The disclosed embodiment also provides an electronic device, which includes a memory and a processor, where the memory stores a computer program for lung image classification, and the processor is configured to read and execute the computer program for lung image classification to perform any one of the above methods for lung image classification.
The present disclosure also provides a storage medium, in which a computer program is stored, where the computer program is configured to execute any one of the above methods for determining a lung image classification network when the computer program runs.
The present disclosure also provides a storage medium having a computer program stored therein, where the computer program is configured to execute any one of the above methods for classifying lung images when the computer program is executed.
The lung image classification network (model) determined by the scheme provided by the present disclosure based on sample weighting to remove the collinearity has the following advantages:
(1) the model has stronger fitting capability, good classification performance and high precision;
(2) the generalization capability of the model is strong, the model can have better performance on data with different distributions, and the data distribution and the training data are different in practical medical clinical application;
(3) the model can run end to end, and is suitable for real-time classification auxiliary diagnosis and treatment under the clinical condition.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Claims (10)
1. A method for determining a lung image classification network includes,
acquiring a lung image picture with a label, and establishing a lung image data set; the label includes: a positive swatch label and a negative swatch label;
pre-training the complete DenseNet according to the pictures and the labels in the data set, and determining the pre-trained DenseNet as a first part of network after removing the full connection layer;
extracting image features of all pictures in the data set according to the first partial network, eliminating co-linearity of the image features, and determining the weight of each image feature;
training a first linear classifier according to the image features and the corresponding weights of the pictures in the data set, and determining the trained first linear classifier as a second partial network;
and combining the first partial network and the second partial network to determine the lung image classification network.
2. The method of claim 1,
before pre-training the complete densnet network according to the pictures and the labels in the data set, the method further comprises:
preprocessing pictures in the data set, wherein the preprocessing includes at least one of the following steps:
according to a preset size, cutting the pictures in the data set;
turning the pictures with preset proportion in the data set in a preset direction;
and carrying out normalization processing on the pictures in the data set.
3. The method according to claim 1 or 2,
the pre-training of the complete DenseNet network according to the pictures and the labels in the data set comprises:
and training the complete DenseNet by using a random gradient descent algorithm according to the pictures and the labels in the data set until the weight of the DenseNet meets a preset convergence condition.
4. The method of claim 3,
the training of the complete DenseNet network by using the stochastic gradient descent algorithm comprises the following steps:
training the complete DenseNet by using a random gradient descent algorithm and adopting a Focal local Loss function;
wherein the focallloss function is:
Loss＝-α*(1-lp)γ*loglp
wherein, alpha and gamma are hyper-parameters of Focal local, lpIs the probability that the complete densnet network judges the picture entered into the complete densnet network as a positive sample.
5. The method according to claim 1 or 2,
the extracting image features of each picture in the data set according to the first partial network, eliminating co-linearity of the image features, and determining the weight of each image feature includes:
respectively extracting the image characteristics of each picture in the data set according to the first partial network;
respectively determining the corresponding pseudo features of the pictures according to the image features of the pictures;
training a second linear classifier according to the image features and the pseudo features of the pictures in the data set;
respectively determining the positive sample probability and the negative sample probability of each picture according to the trained second linear classifier;
determining the weight of each image characteristic according to the positive sample probability and the negative sample probability of each picture;
wherein the weight of each image feature is equal to the ratio of the probability of the positive sample to the probability of the negative sample of the home picture.
6. The method of claim 5,
the image features of the picture comprise at least one dimension of features;
the determining the pseudo features corresponding to the pictures respectively according to the image features of the pictures comprises:
the following operations are respectively performed for each picture: respectively generating a pseudo feature for each dimension feature of the picture to obtain a group of pseudo features corresponding to the picture; the pseudo feature is a feature with the same dimensionality corresponding to one randomly extracted picture in other pictures which are not self in the data set;
training a second linear classifier based on image features and pseudo features of pictures in the dataset, comprising:
and training the second linear classifier by using a random gradient descent algorithm by using a Focal local Loss function and using the image characteristics of the pictures in the data set as negative samples and the pseudo characteristics corresponding to the pictures as positive samples.
7. The method according to claim 1 or 2,
the training of the first linear classifier according to the image features and the corresponding weights of the pictures in the data set comprises:
training the first linear classifier by using a FocalLoss loss function and a random gradient descent algorithm;
when calculating the value of the Loss function of each picture, multiplying the value Loss of the Loss function by the corresponding weight w to obtain a new Loss function:
Lossre＝w*Loss
and performing back propagation and parameter updating on the first linear classifier by using the obtained new loss function, and determining the weight of the first linear classifier.
8. A method for classifying lung images, comprising,
acquiring a lung image picture to be classified;
the pulmonary image classification network determined according to the method of any one of claims 1 to 7, classifying the pulmonary image picture to be classified.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program for performing lung image classification network determination or lung image classification, and the processor is configured to read and execute the computer program for performing lung image classification network determination or lung image classification to perform the method of any one of claims 1 to 7 or 8.
10. A storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the method of any of claims 1 to 7 or 8 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110090885.9A CN112733959A (en) | 2021-01-22 | 2021-01-22 | Lung image classification method, classification network determination method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110090885.9A CN112733959A (en) | 2021-01-22 | 2021-01-22 | Lung image classification method, classification network determination method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112733959A true CN112733959A (en) | 2021-04-30 |
Family
ID=75593747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110090885.9A Pending CN112733959A (en) | 2021-01-22 | 2021-01-22 | Lung image classification method, classification network determination method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112733959A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117496275A (en) * | 2023-12-29 | 2024-02-02 | 深圳市软盟技术服务有限公司 | Class learning-based depth image classification network training method, electronic equipment and storage medium |
-
2021
- 2021-01-22 CN CN202110090885.9A patent/CN112733959A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117496275A (en) * | 2023-12-29 | 2024-02-02 | 深圳市软盟技术服务有限公司 | Class learning-based depth image classification network training method, electronic equipment and storage medium |
| CN117496275B (en) * | 2023-12-29 | 2024-04-02 | 深圳市软盟技术服务有限公司 | Class learning-based depth image classification network training method, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111507335B (en) | Method and device for automatically labeling training images used for deep learning network | |
| US11348249B2 (en) | Training method for image semantic segmentation model and server | |
| Gorriz et al. | Cost-effective active learning for melanoma segmentation | |
| JP6710135B2 (en) | Cell image automatic analysis method and system | |
| EP3690714A1 (en) | Method for acquiring sample images for inspecting label among auto-labeled images to be used for learning of neural network and sample image acquiring device using the same | |
| CN109344851B (en) | Image classification display method and device, analysis instrument and storage medium | |
| Liu et al. | Co-correcting: noise-tolerant medical image classification via mutual label correction | |
| CN111738269A (en) | Model training method, image processing device, model training apparatus, and storage medium | |
| CN111967464A (en) | Weak supervision target positioning method based on deep learning | |
| CN114330499A (en) | Method, device, equipment, storage medium and program product for training classification model | |
| CN113610842B (en) | OCT image retina detachment and splitting automatic segmentation method based on CAS-Net | |
| CN111127400A (en) | Method and device for detecting breast lesions | |
| Gautam et al. | Demonstrating the risk of imbalanced datasets in chest x-ray image-based diagnostics by prototypical relevance propagation | |
| CN112991281B (en) | Visual detection method, system, electronic equipment and medium | |
| Jung et al. | Weakly supervised thoracic disease localization via disease masks | |
| CN112733959A (en) | Lung image classification method, classification network determination method and device | |
| CN113762151A (en) | Fault data processing method and system and fault prediction method | |
| CN117576079A (en) | Industrial product surface abnormality detection method, device and system | |
| CN111652277A (en) | False positive filtering method, electronic device and computer readable storage medium | |
| Khan et al. | Ddnet: Diabetic retinopathy detection system using skip connection-based upgraded feature block | |
| CN114445649A (en) | Method for detecting RGB-D single image shadow by multi-scale super-pixel fusion | |
| Pei et al. | FGO-Net: Feature and Gaussian Optimization Network for visual saliency prediction | |
| US20220366671A1 (en) | Method and system for identifying anomalies in x-rays | |
| CN111612021B (en) | Error sample identification method, device and terminal | |
| CN114764948A (en) | Living body detection method, living body detection device, living body detection equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |