CN116824212A - Fundus photo classification method based on small sample learning - Google Patents
Fundus photo classification method based on small sample learning Download PDFInfo
- Publication number
- CN116824212A CN116824212A CN202310529974.8A CN202310529974A CN116824212A CN 116824212 A CN116824212 A CN 116824212A CN 202310529974 A CN202310529974 A CN 202310529974A CN 116824212 A CN116824212 A CN 116824212A
- Authority
- CN
- China
- Prior art keywords
- fundus
- picture
- data
- picture data
- small sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000012549 training Methods 0.000 claims abstract description 20
- 238000004364 calculation method Methods 0.000 claims description 20
- 238000010606 normalization Methods 0.000 claims description 16
- 239000013598 vector Substances 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 210000005252 bulbus oculi Anatomy 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 6
- 230000003247 decreasing effect Effects 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 230000001965 increasing effect Effects 0.000 claims description 3
- 230000008030 elimination Effects 0.000 claims 1
- 238000003379 elimination reaction Methods 0.000 claims 1
- 230000000087 stabilizing effect Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 description 6
- 238000010801 machine learning Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 210000001367 artery Anatomy 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 210000001736 capillary Anatomy 0.000 description 1
- 210000003161 choroid Anatomy 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 210000001508 eye Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000001328 optic nerve Anatomy 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a fundus photo classification method based on small sample learning, and belongs to the field of photo classification. The method comprises the following steps: model training stage and actual application stage; the model training stage comprises reading a plurality of fundus picture data and labels, and performing standardized processing and enhancement processing; training and constructing a network model based on small sample learning according to the processed fundus picture data and the relation weighting label; the practical application stage comprises the steps of reading unclassified fundus picture data and carrying out standardized processing; and inputting the processed unclassified fundus picture data into the network model based on small sample learning, and outputting a classification result by the model. The standardized processing unifies the format of fundus image data, so that the data distribution is more reasonable. The invention also uses the relation weighting label to improve the traditional one-hot label, so that the distance between labels can be better embodied, and the data can be balanced.
Description
Technical Field
The invention belongs to the field of picture classification, and particularly relates to a fundus photo classification method based on small sample learning.
Background
Vision is the information acceptor that people know the world and most important to acquire knowledge. The eyeball is taken as an organ carrier of vision, including optic disc (optic nerve), blood vessel, retina tissue, choroid and the like, is the only part of the whole body, and can directly and intensively observe arteries, veins and capillaries by naked eyes, and the characteristics can often reflect various properties of the human body, such as age, sex or the existence of certain diseases and the like. Such information is often difficult to obtain directly from human observation, without professionals, and must be identified and categorized by computer technology.
To date, computer technology development based on machine learning and artificial intelligence has been put into the work of picture classification by a large number of researchers. Including classification of natural scenes, classification of handwriting fonts, classification of animals, and the like. Among them, classification of fundus photographs is relatively small, and there are still problems in classifying and judging age, sex, and the like from fundus images: first, fundus datasets face a large challenge in both quantity and quality. Conventional deep learning algorithms typically require a large amount of data support, but the fundus data sets such as DDR, APTOS disclosed on the network are far from being of sufficient scale. In a single data set, the sources of fundus pictures are also five-in-eight, and the colors, definition, contrast, brightness, size, eyeball integrity and the like are different from each other, so that the quality levels are uneven. Second, there is a serious data imbalance phenomenon in the fundus data set, including a quantitative imbalance and a difficulty imbalance. Typically, the fundus data of the highest proportion will occupy about half of the total data volume, even higher, while the data of the least proportion may be only 1/20 to 1/30. The unbalance can cause the machine learning model to ignore the feature learning of few categories in the training process, and pay more attention to the samples of a plurality of categories, so that the problems of poor model effect and high accuracy are finally caused. At the same time, some types of data may be encountered which are difficult to distinguish while being small in quantity. Third, current research into the task of classifying bottom-of-eye images almost ignores the correlation between their labels. Typically we use one-hot coding to represent the label of the input data and use this as a parameter to input into the loss function for loss calculation, updating the model. Different from common classification tasks, such as fundus pictures of different categories divided by age groups, certain and deducible relations exist between the fundus pictures, for example, the distance between the fundus of teenagers and the fundus of young people is necessarily smaller than the distance between the fundus of teenagers and the fundus of elderly people, and if the distance between the categories is still determined simply and roughly by using a one-hot coding mode, the method is not reasonable enough. The three problems described above result in machine learning with a large room for improvement in the performance of fundus picture classification tasks.
Based on the technical problems, the invention provides a fundus photo classification method based on small sample learning, which is used for assisting fundus photo classification and expanding fundus data sets with different classification standards.
Disclosure of Invention
The invention aims to at least solve one of the technical problems in the prior art and provides a fundus photo classification method based on small sample learning.
Comprising two stages: model training stage and actual application stage;
the model training stage comprises the steps of reading a plurality of fundus picture data and labels thereof, and carrying out standardization processing and enhancement processing;
training and constructing a network model based on small sample learning according to the processed fundus picture data and the relation weighting label;
the practical application stage comprises the steps of reading unclassified fundus picture data and carrying out standardized processing;
and inputting the processed unclassified fundus picture data into the network model based on small sample learning, and taking the category with the largest value in the vector output by the model as the classification result of the fundus picture.
The optional model training stage includes reading a plurality of fundus image data and labels thereof and performing normalization processing and enhancement processing, including:
reading a photo file path, and opening corresponding fundus picture data through an opencv-python library;
the fundus picture is standardized, which comprises the following steps:
cutting the fundus picture into 512 x 512 pixels by using a circular cutting method, wherein the background is pure black, and the center is a circular fundus format;
calculating the average value and standard deviation of the pixel values of the fundus picture after removing the background area by using a normalization method for removing the background;
enhancing the picture, including:
and carrying out random up-and-down overturn, random left-and-right overturn and random rotation treatment on the fundus picture.
The optional circular clipping method comprises the following steps: primary cutting and secondary cutting;
the specific relation of the preliminary cutting is as follows:
P′＝Cut G (Mask(P G ,tol),P o )；
wherein P is O Representing the original fundus picture, P G Representing P o The converted gray level picture, tol represents a threshold value super parameter, mask represents dividing a fundus region and a background region according to the threshold value tol, and obtaining a corresponding Mask, cut G Representing the original picture P according to the mask o Cutting the redundant background of the fundus region to obtain a picture P' which is primarily cut, so that the edge of the fundus region becomes the edge of the picture;
the specific relation of the secondary cutting is as follows:
where l represents the length of the longer side in P', and Resize represents scaling the picture to a specified size;Cut C and (3) taking l/2 as a radius to cut out a circular area as a final fundus area, setting the pixel value of the rest part as 0 as a background area, and finally scaling the image into 512 x 512 pixels to obtain the fundus image P which finally meets the standard.
The optional normalization method for eliminating the background comprises the following steps:
distinguishing an eyeball area and a background area of the fundus picture;
calculating the mean value and standard deviation of pixel values of the eyeball area;
the specific relation is as follows:
wherein, sigma P＞0 P represents a pixel value greater than 0 in the accumulated picture P, count (P>0) The number of pixels greater than 0 in the statistical picture P is represented, and mean and std represent the mean and standard deviation of the picture rejection background respectively.
Optionally, training and constructing a network model based on small sample learning according to the processed fundus image data and the relation weighted label, including:
dividing the processed fundus image data into a query set and a support set according to the image real label;
inputting the query set and the support set together into a network based on small sample learning;
outputting a prediction result of fundus picture data by a network;
calculating the cross entropy loss according to the relation weighted label and the prediction result of the fundus picture data, and reversely transmitting and updating the model parameters;
and (3) circulating the processes of loss calculation and parameter updating until the loss tends to be stable, and not reducing any more, so as to obtain the network model based on the small sample learning.
The optional query set and the support set are divided into:
extracting K pictures from the processed fundus picture data in each category as a support set, wherein the specific relation is as follows:
wherein S represents a support set, C represents different category numbers, K represents the number of pictures extracted from each category, and x i And y i Respectively representing the processed fundus picture data and the real label;
after the support set is obtained, the rest fundus image data after processing are all used as the query set, and the specific relation is as follows:
wherein Q represents a query set, M represents the total data amount of fundus pictures, and x j And y j The fundus picture data and the real label after the remaining processing are respectively represented, and C and K have the same meaning as in the above-described relational expression.
Optionally, the small sample learning-based network includes: the device comprises a feature extraction module and a distance measurement module;
the characteristic extraction module calculates and extracts characteristic graphs of the query set and the support set by adopting a U-Net3 network, and the specific relation is as follows:
F i ＝extractor(x i ),(i＝1,2,…,C)；
wherein F is i Representing fundus picture data x i The feature map obtained through the U-Net3 network, and the extrator represents a feature extraction module;
the distance measurement module comprises:
splicing the feature images extracted in the query set and the feature images extracted in the support set in the channel dimension, wherein the specific relation is as follows:
F ij ＝F i ⊕F j ,(i＝1,2,…,C)；
wherein F is i And F j Respectively representing the feature graphs extracted from the i-type support set and the query set, and the function of the feature graphs represents that the two feature graphs are spliced in the channel dimension, F ij Representing the spliced characteristic diagram;
inputting the spliced feature map F into a ResNet18 network, calculating score vectors of the support set and the query set, and selecting the type with the largest score as a prediction result of fundus picture data, wherein the specific relation is as follows:
R ij ＝Distance(F ij ),(i＝1,2,…,C)；
wherein R is ij Representing the scores of the query set over class i, distance represents the Distance metric network.
Optionally, the calculating the cross entropy loss according to the relation weighted label and the prediction result of the fundus image data, and back-propagating and updating the model parameters includes:
the relationship weighted label can be selected in 3 weighted modes, and the specific relationship formula is as follows:
wherein i represents a real label, j represents a predictive label, N represents a category number, weight A ij ，weightB ij ，wrightC ij Under the condition that the real label is i and the predicted label is j, 3 relation weights of the corresponding positions of the label vectors are respectively adopted; the values of the three are 1 at the correct label position, and the positions are the followingThe values of the error label and the correct label are respectively increased, stabilized and decreased at the increasing speed and the decreasing speed, when the label distance reaches the maximum, namely the labels are respectively 0 and N-1, the weight value is 0 and the minimum is reached;
the specific relation of the cross entropy loss calculation is as follows:
wherein i represents the category to which the input data actually belongs, out i Direct output of the representation model at the corresponding class position, y i Representing the relationship weight at the corresponding position, determined by the relationship-weighted label, softmax (out i ) Representing the prediction probability of the corresponding type obtained through calculation of a softmax function;
and after the loss value is calculated, the model parameters are updated by back propagation.
Optionally, the practical application stage includes reading unclassified fundus picture data and performing standardization processing, including:
after the unclassified fundus photo data is read, only the normalization processing is performed, no enhancement operation is performed, and then the unclassified fundus photo data is input into the network model based on small sample learning.
Optionally, inputting the processed unclassified fundus picture data into the network model based on small sample learning, and taking the class with the largest value in the vector output by the model as the classification result of the fundus picture, including:
predicting the unclassified fundus picture data by adopting the network model based on small sample learning;
the class with the largest value in the vectors output by the model is used as the classifying result of the fundus picture;
and carrying out no loss calculation on the classification result and updating the model parameters.
The invention provides a fundus photo classification method based on small sample learning. The invention can unify the format of the fundus picture as much as possible, clearly distinguish the eyeball area from the background area of the picture, and can well avoid the interference of extreme pixel value (equal to 0) of the black background area on the calculation of the mean value and the standard deviation of the fundus area, so that the obtained mean value and standard deviation can realize more reasonable data distribution. The invention also uses the relation weighted label to improve the traditional one-hot label, can lead the distance between different labels to be more reasonably reflected, realize better loss calculation and realize certain balance for severely unbalanced data.
Drawings
FIG. 1 is a flow chart diagram of a fundus photo classification method based on small sample learning according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating the normalization effect according to an embodiment of the present invention;
FIG. 3 is a diagram showing the normalization effect according to an embodiment of the present invention
FIG. 4 is a schematic diagram of a network structure for small sample learning according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a network structure for small sample learning according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a comparison of 3 relationship weighted labels with one-hot labels according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the drawings and detailed description for the purpose of better understanding of the technical solution of the present invention to those skilled in the art. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present invention belong to the protection scope of the present invention.
Unless specifically stated otherwise, technical or scientific terms used herein should be defined in the general sense as understood by one of ordinary skill in the art to which this invention belongs. The use of "including" or "comprising" and the like in the present invention is not intended to limit the shape, number, step, action, operation, component, original and/or group thereof referred to, nor exclude the presence or addition of one or more other different shapes, numbers, steps, actions, operations, components, original and/or group thereof. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or order of the indicated features.
As shown in fig. 1, the present invention provides a fundus photo classification method S200 based on small sample learning, which specifically includes steps S210 to S270:
s210, reading a plurality of fundus photo data and labels thereof.
Specifically, in this embodiment, not only fundus photo data but also a tag corresponding to fundus photo needs to be read, a list composed of a training data file path and training data tag data may be transferred, and a corresponding picture is opened through an opencv-python (cv 2) library, and the read data is transmitted to a subsequent task.
The fundus photo label of this embodiment includes different ages, respectively 0 to 18, 19 to 49, 50 to 79, and 80 and above.
S220, standardization processing.
Specifically, a circular clipping method is used for clipping fundus pictures into 512 x 512 pixel sizes, the background is pure black (the pixel value is 0), and the center is in a circular fundus format; then, calculating the average value and standard deviation of the pixel values of the fundus picture after removing the background area by using a normalization method for removing the background;
it should be noted that this operation can achieve standardization of most of the pictures in the dataset, however, due to the fact that the background pixel values of different pictures are different, some of the pictures are relatively close to pure black, i.e. the pixel values are smaller than 10, and some of the pictures are gray, i.e. the pixel values are larger than 30, and meanwhile, due to the illumination problem during shooting, the fundus color is relatively close to the pixel values of the background color. The factors can not be completed at one time, and a plurality of threshold values tol are needed to be selected for cutting different data, so that the standardization of all data can be finally realized. Otherwise, a clipping error condition will occur.
S230, enhancing processing.
Specifically, the fundus picture is subjected to random up-and-down overturn, random left-and-right overturn and random rotation treatment.
S240, training and constructing a network model based on small sample learning according to the processed fundus photo data and the relation weighted labels.
Specifically, dividing the processed fundus image data into a query set and a support set according to the image real label; inputting the query set and the support set into a network based on small sample learning; outputting a classification result of fundus picture data by a network; calculating the cross entropy loss according to the classification result of the relation weighted label and the fundus picture data, and reversely transmitting and updating the model parameters; and finally, the process of calculating the loss and updating the parameters is circulated until the loss tends to be stable and does not decline any more, so that a network model based on small sample learning is obtained.
S250, unclassified fundus photo data are read.
Specifically, the corresponding picture is opened through the opencv-python (cv 2) library, and the read data is transmitted to the subsequent task.
In the actual application process, after fundus photo data is read, only standardized processing is performed, data enhancement processing is not performed, and then an original image is input into a network model based on small sample learning to perform result prediction.
S260, standardization processing. The processing is the same as S220.
S270, inputting the processed fundus photo data to be screened into the network model based on small sample learning, and outputting a classification result by the model.
In the actual application process, the prediction result is not subjected to loss calculation, and the model parameters are not updated.
It should be noted that, for the standardized processing of the fundus image, the format of the fundus image can be unified as much as possible, the eyeball area and the background area of the image can be clearly distinguished, and the interference of the extreme pixel value (equal to 0) of the black background area on the calculation of the mean value and the standard deviation of the fundus area can be well avoided, and the obtained mean value and standard deviation can realize more reasonable data distribution.
Furthermore, the invention improves the traditional one-hot label by using the relation weighted label, can more reasonably embody the distance between different labels, realizes better loss calculation, and can realize certain balance for severely unbalanced data.
Specifically, the network modeling process is as follows:
first, a plurality of fundus picture data and a label thereof are read and subjected to normalization processing and enhancement processing. The effect of the normalization process is schematically shown in fig. 2.
Specifically, the photo file path is read, and the corresponding fundus picture data is opened by using the opencv-python (cv 2) library.
The circular cropping method comprises primary cropping and secondary cropping, wherein the fundus picture is cropped to 512 x 512 pixels, the background is pure black (the pixel value is 0), and the center is in a circular fundus format.
The specific relation of the preliminary clipping is as follows:
P′＝Cut G (Mask(P G ,tol),P O )；
wherein P is O Representing the original fundus picture, P G Representing P O The converted gray level picture, tol represents a threshold value super parameter, mask represents dividing a fundus region and a background region according to the threshold value tol, and obtaining a corresponding Mask, cut G Representing the original picture P according to the mask O And (3) cutting the redundant background of the fundus region to obtain a picture P' which is primarily cut, so that the edge of the fundus region becomes the edge of the picture.
The specific relation of the secondary clipping is as follows:
where l represents the length of the longer side in P', and Resize represents scaling the picture to a specified size; cut (Cut) C And (3) taking l/2 as a radius to cut out a circular area as a final fundus area, setting the pixel value of the rest part as 0 as a background area, and finally scaling the image into 512 x 512 pixels to obtain the fundus image P which finally meets the standard.
And calculating the average value and standard deviation of the pixel values of the fundus picture after removing the background area by using a normalization method for removing the background.
The specific relation is as follows:
wherein, sigma P＞0 P represents a pixel value greater than 0 in the accumulated picture P, count (P>0) Representing the number of pixels greater than 0 in the statistical picture P, mean and strd represent the mean and standard deviation of the picture ignoring background, respectively.
And (5) performing random up-and-down overturn, random left-and-right overturn and random rotation treatment on the fundus picture.
Secondly, training and constructing a network model based on small sample learning according to the processed fundus picture data and the relation weighting labels.
Specifically, the processed fundus image data is divided into a query set and a support set according to the image real label.
Extracting K pictures from the processed fundus picture data in each category as a support set, wherein the specific relation is as follows:
wherein S represents a support set, C represents different category numbers, K represents the number of pictures extracted from each category, and x i And y i Respectively representing the processed fundus picture data and the real label;
after the support set is obtained, the rest fundus image data after processing are all used as the query set, and the specific relation is as follows:
wherein Q represents a query set, M represents the total data amount of fundus pictures, and x j And y j The fundus picture data and the real label after the remaining processing are respectively represented, and C and K have the same meaning as in the above-described relational expression.
The query set and the support set are input into a network based on small sample learning, and a prediction result of fundus image data is output by the network. The network based on the small sample learning comprises a feature extraction module and a distance measurement module, and the structure schematic diagram of the network is shown in fig. 3.
Specifically, the feature extraction module calculates and extracts feature graphs of the query set and the support set by adopting a U-Net3 network, and the specific relation is as follows:
F i ＝extractor(x i ),(i＝1,2,…,C)；
wherein F is i Representing fundus picture data x i The feature map obtained through the U-Net3 network, and the extrator represents a feature extraction module;
the distance measurement module splices the feature images extracted in the query set and the feature images extracted in the support set in the channel dimension, and the specific relation is as follows:
F ij ＝F i ⊕F j ,(i＝1,2,…,C)；
wherein F is i And F j Respectively representing the feature graphs extracted from the i-type support set and the query set, and the function of the feature graphs represents that the two feature graphs are spliced in the channel dimension, i ij Representing the spliced characteristic diagram;
inputting the spliced feature map F into a ResNet18 network, calculating score vectors of the support set and the query set, and selecting the type with the largest score as a prediction result of fundus picture data, wherein the specific relation is as follows:
R ij ＝Distance(F ij ),(i＝1,2,…,C)；
wherein R is ij Representing the scores of the query set over class i, distance represents the Distance metric network.
Thirdly, calculating cross entropy loss according to the classification result of the relation weighted label and the fundus picture data, and reversely transmitting and updating the model parameters; and (3) circulating the process of calculating the loss and updating the parameters until the loss tends to be stable, and not reducing any more, so as to obtain the network model based on small sample learning.
Specifically, the relational weighted tags use weight C ij To calculate, the specific relation is as follows:
wherein i represents a real label, j represents a predictive label, N represents a category number, weight C ij Under the condition that the real label is i and the predicted label is j, the relation weight of the corresponding position of the label vector is calculated; the values of the labels at the positions of the correct labels are 1, the distance d (d=abs (i-j)) between the correct labels is reduced at a decreasing speed along with the increase of the distance d between the correct labels, and when the distance d between the correct labels reaches the maximum, namely the labels are respectively 0 and N-1, the weight value is 0, and the distance d is the minimum;
the specific relation for cross entropy loss calculation is as follows:
wherein i represents the category to which the input data actually belongs, out i Direct output of the representation model at the corresponding class position, y i Representing the relationship weight at the corresponding position, determined by the relationship-weighted label, softmax (out i ) Representing the prediction probability of the corresponding type obtained through calculation of a softmax function;
and after the loss value is calculated, the model parameters are updated by back propagation.
In the training phase, each round of loss calculation is used to update the model, and in this embodiment, the model is stored once every certain number of rounds, and the one with the best effect is finally selected as the final prediction model. In the actual application stage, the model only predicts the result and does not need to update itself, so that calculation loss is not needed. Of course, if the later input data gradually increases, the model is expected to be further updated, training can be continued on the basis of the existing prediction model, and the model with better effect can be reselected.
Fourth, unclassified fundus image data is read and standardized.
After unclassified fundus photo data is read, only the normalization processing is performed, no enhancement operation is performed, and then the unclassified fundus photo data is input into the network model based on small sample learning. The standardized processing method is the same as the first step.
Fifthly, the processed unclassified fundus image data is input into a network model based on small sample learning, and a classification result is output by the model.
Specifically, a network model based on small sample learning is adopted to predict the unclassified fundus picture data, and the category with the highest model output probability is used as a classification result of the fundus picture;
the fundus photo classification method for small sample-based learning will be further described below with reference to specific embodiments:
the present example identifies categories of fundus pictures of different age groups, including the steps of:
s1, reading a plurality of fundus picture data and labels thereof, and performing standardization processing and enhancement processing;
s2, dividing the processed fundus image data into a query set and a support set according to the image real label;
s3, inputting the query set and the support set into a network based on small sample learning, and outputting a prediction result of fundus picture data by the network;
s4, calculating cross entropy loss according to the relation weighted labels and the prediction result of the fundus picture data, and reversely transmitting and updating model parameters;
s5, circulating the process of calculating the loss and updating the parameters until the loss tends to be stable and does not drop any more, and obtaining the network model based on the small sample learning;
s6, reading unclassified fundus picture data and performing standardization processing;
s7, inputting the processed unclassified fundus image data into a network model based on small sample learning, and outputting a prediction result by the model.
The invention provides a fundus photo classification method based on small sample learning, which has the following beneficial effects:
firstly, the invention can unify the format of the fundus picture as much as possible, clearly distinguish the eyeball area from the background area of the picture, and can well avoid the interference of the extreme pixel value (equal to 0) of the black background area on the calculation of the mean value and the standard deviation of the fundus area, so that the obtained mean value and standard deviation can realize more reasonable data distribution.
The traditional one-hot label is improved by using the relation weighted label, so that the distances among different labels can be more reasonably reflected, better loss calculation is realized, and certain balance can be realized on severely unbalanced data.
Thirdly, the network based on small sample learning provided by the invention can use a small amount of fundus image data to predict input data, has high model performance and generalization capability, and can reduce the influence caused by insufficient data.
It is to be understood that the above embodiments are merely illustrative of the application of the principles of the present invention, but not in limitation thereof. Various modifications and improvements may be made by those skilled in the art without departing from the spirit and substance of the invention, and are also considered to be within the scope of the invention.
Claims (10)
1. The fundus photo classification method based on small sample learning is characterized by comprising two stages: model training stage and actual application stage;
the model training stage comprises the steps of reading a plurality of fundus picture data and labels thereof, and carrying out standardization processing and enhancement processing;
training and constructing a network model based on small sample learning according to the processed fundus picture data and the relation weighting label;
the practical application stage comprises the steps of reading unclassified fundus picture data and carrying out standardized processing;
and inputting the processed unclassified fundus picture data into the network model based on small sample learning, and taking the category with the largest value in the vector output by the model as the classification result of the fundus picture.
2. The method according to claim 1, wherein the model training phase includes reading a plurality of fundus picture data and labels thereof and performing normalization processing and enhancement processing, including:
reading a photo file path, and opening corresponding fundus picture data through an opencv-python cv2 library;
the fundus picture is standardized, which comprises the following steps:
cutting the fundus picture into 512 x 512 pixels by using a circular cutting method, wherein the background is pure black, and the center is a circular fundus format;
calculating the average value and standard deviation of the pixel values of the fundus picture after removing the background area by using a normalization method for removing the background;
enhancing the picture, including:
and carrying out random up-and-down overturn, random left-and-right overturn and random rotation treatment on the fundus picture.
3. The method of claim 2, wherein the circular clipping method comprises: primary cutting and secondary cutting;
the specific relation of the preliminary cutting is as follows:
P′＝Cut G (Mask(P G ,tol),P o )；
wherein P is o Representing the original fundus picture, P G Representing P O The converted gray level picture, tol represents a threshold value super parameter, mask represents dividing a fundus region and a background region according to the threshold value tol, and obtaining a corresponding Mask, cut G Representing the original picture P according to the mask O Cutting the redundant background of the fundus region to obtain a picture P' which is primarily cut, so that the edge of the fundus region becomes the edge of the picture;
the specific relation of the secondary cutting is as follows:
where l represents the length of the longer side in P', and Resize represents scaling the picture to a specified size; cut (Cut) C And (3) taking l/2 as a radius to cut out a circular area as a final fundus area, setting the pixel value of the rest part as 0 as a background area, and finally scaling the image into 512 x 512 pixels to obtain the fundus image P which finally meets the standard.
4. The method of claim 2, wherein the background culling normalization method comprises:
distinguishing an eyeball area and a background area of the fundus picture;
calculating the mean value and standard deviation of pixel values of the eyeball area;
the specific relation is as follows:
wherein, sigma P＞0 P represents a pixel value greater than 0 in the accumulated picture P, count (P>0) Representation systemAnd counting the number of pixels larger than 0 in the picture P, wherein mean and std respectively represent the average value and standard deviation of the picture elimination background.
5. The method of claim 1, wherein training and constructing a small sample learning-based network model from the processed fundus picture data and the relationship weighted labels comprises:
dividing the processed fundus image data into a query set and a support set according to the image real label;
inputting the query set and the support set together into a network based on small sample learning;
outputting a prediction result of fundus picture data by a network;
calculating the cross entropy loss according to the relation weighted label and the prediction result of the fundus picture data, and reversely transmitting and updating the model parameters;
and (3) circulating the processes of loss calculation and parameter updating until the loss tends to be stable, and not reducing any more, so as to obtain the network model based on the small sample learning.
6. The method of claim 5, wherein the query set and the support set are partitioned in the following manner:
extracting K pictures from the processed fundus picture data in each category as a support set, wherein the specific relation is as follows:
wherein S represents a support set, C represents different category numbers, K represents the number of pictures extracted from each category, and x i And y i Respectively representing the processed fundus picture data and the real label;
after the support set is obtained, the rest fundus image data after processing are all used as the query set, and the specific relation is as follows:
wherein Q represents a query set, M represents the total data amount of fundus pictures, and x j And y j The fundus picture data and the real label after the remaining processing are respectively represented, and C and K have the same meaning as in the above-described relational expression.
7. The method of claim 5, wherein the small sample learning-based network comprises: the device comprises a feature extraction module and a distance measurement module;
the characteristic extraction module calculates and extracts characteristic graphs of the query set and the support set by adopting a U-Net3 network, and the specific relation is as follows:
F i ＝extractor(x i ),(i＝1,2,…,C)；
wherein F is i Representing fundus picture data x i The feature map obtained through the U-Net3 network, and the extrator represents a feature extraction module;
the distance measurement module comprises:
splicing the feature images extracted in the query set and the feature images extracted in the support set in the channel dimension, wherein the specific relation is as follows:
wherein F is i And F j Representing the feature graphs extracted from the class i support set and the query set respectively,representing the concatenation of two feature maps in the channel dimension, F ij Representing the spliced characteristic diagram;
inputting the spliced feature map F into a ResNet18 network, calculating score vectors of the support set and the query set, and selecting the type with the largest score as a prediction classification result of fundus picture data, wherein the specific relation is as follows:
R ij ＝Distance(F ij ),(i＝1,2,…,C)；
wherein R is ij Representing the scores of the query set over class i, distance represents the Distance metric network.
8. The method of claim 5, wherein calculating the cross entropy loss from the relationship weighted labels and the predicted outcome of the fundus picture data and back-propagating updates the model parameters comprises:
the relationship weighted label can be selected in 3 weighted modes, and the specific relationship formula is as follows:
wherein i represents a real label, j represents a predicted label, N represents a category number, weight _ij ，weightB ij ，wrightC ij Under the condition that the real label is i and the predicted label is j, 3 relation weights of the corresponding positions of the label vectors are respectively adopted; the values of the three are 1 at the positions of the correct labels, and along with the increase of the distance d (d=abs (i-j)) between the incorrect labels and the correct labels, the values of the three are respectively reduced at the increasing, stabilizing and decreasing speeds, and when the label distance reaches the maximum, namely the labels are respectively 0 and N-1, the weight value is 0 and the minimum value is reached;
the specific relation of the cross entropy loss calculation is as follows:
wherein i represents the category to which the input data actually belongs, out i Direct output of the representation model at the corresponding class position, y i Representing the relationship weight at the corresponding position, determined by the relationship-weighted label, softmax (out i ) Representing the prediction probability of the corresponding type obtained through calculation of a softmax function;
and after the loss value is calculated, the model parameters are updated by back propagation.
9. The method according to claim 1, wherein the actual application stage includes reading unclassified fundus picture data and performing normalization processing, including:
after the unclassified fundus photo data is read, only the normalization processing is performed, no enhancement operation is performed, and then the unclassified fundus photo data is input into the network model based on small sample learning.
10. The method according to claim 1, wherein inputting the processed unclassified fundus picture data into the small sample learning-based network model, taking a class with the largest value among vectors output by the model as a classification result of the fundus picture, comprises:
predicting the unclassified fundus picture data of claim 9 using the small sample learning-based network model of claim 5;
the class with the largest value in the vectors output by the model is used as the classifying result of the fundus picture;
and carrying out no loss calculation on the classification result and updating the model parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310529974.8A CN116824212A (en) | 2023-05-11 | 2023-05-11 | Fundus photo classification method based on small sample learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310529974.8A CN116824212A (en) | 2023-05-11 | 2023-05-11 | Fundus photo classification method based on small sample learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116824212A true CN116824212A (en) | 2023-09-29 |
Family
ID=88128338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310529974.8A Pending CN116824212A (en) | 2023-05-11 | 2023-05-11 | Fundus photo classification method based on small sample learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116824212A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557840A (en) * | 2023-11-10 | 2024-02-13 | 中国矿业大学 | Fundus lesion grading method based on small sample learning |
-
2023
- 2023-05-11 CN CN202310529974.8A patent/CN116824212A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557840A (en) * | 2023-11-10 | 2024-02-13 | 中国矿业大学 | Fundus lesion grading method based on small sample learning |
| CN117557840B (en) * | 2023-11-10 | 2024-05-24 | 中国矿业大学 | Fundus lesion grading method based on small sample learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3674968B1 (en) | Image classification method, server and computer readable storage medium | |
| CN113344849B (en) | Microemulsion head detection system based on YOLOv5 | |
| CN103136504B (en) | Face identification method and device | |
| CN109344851B (en) | Image classification display method and device, analysis instrument and storage medium | |
| CN114821189B (en) | Focus image classification and identification method based on fundus image | |
| CN111368672A (en) | Construction method and device for genetic disease facial recognition model | |
| CN113111979B (en) | Model training method, image detection method and detection device | |
| CN112927209B (en) | CNN-based significance detection system and method | |
| CN115661943A (en) | Fall detection method based on lightweight attitude assessment network | |
| CN112836625A (en) | Face living body detection method and device and electronic equipment | |
| CN116824212A (en) | Fundus photo classification method based on small sample learning | |
| CN111863241B (en) | Fundus imaging classification system based on integrated deep learning | |
| CN114330499A (en) | Method, device, equipment, storage medium and program product for training classification model | |
| CN113240655A (en) | Method, storage medium and device for automatically detecting type of fundus image | |
| CN114677730A (en) | Living body detection method, living body detection device, electronic apparatus, and storage medium | |
| CN115147640A (en) | Brain tumor image classification method based on improved capsule network | |
| CN111462093A (en) | Method for classifying diseases based on fundus images | |
| CN117611599B (en) | Blood vessel segmentation method and system integrating centre line diagram and contrast enhancement network | |
| CN115187982B (en) | Algae detection method and device and terminal equipment | |
| CN111968107A (en) | Uncertainty-based retinopathy of prematurity plus lesion classification system | |
| CN111598144A (en) | Training method and device of image recognition model | |
| CN109977754A (en) | A kind of detection method and system of household register photo quality | |
| CN112862761B (en) | Brain tumor MRI image segmentation method and system based on deep neural network | |
| CN112668668B (en) | Postoperative medical image evaluation method and device, computer equipment and storage medium | |
| CN117576381B (en) | Target detection training method, electronic device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |