CN109670489B - Weak supervision type early senile macular degeneration classification method based on multi-instance learning - Google Patents
Weak supervision type early senile macular degeneration classification method based on multi-instance learning Download PDFInfo
- Publication number
- CN109670489B CN109670489B CN201910120667.8A CN201910120667A CN109670489B CN 109670489 B CN109670489 B CN 109670489B CN 201910120667 A CN201910120667 A CN 201910120667A CN 109670489 B CN109670489 B CN 109670489B
- Authority
- CN
- China
- Prior art keywords
- image
- instance learning
- method based
- neural network
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 206010064930 age-related macular degeneration Diseases 0.000 title claims abstract description 17
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 12
- 230000002087 whitening effect Effects 0.000 claims abstract description 8
- 238000013528 artificial neural network Methods 0.000 claims abstract description 7
- 239000011521 glass Substances 0.000 claims abstract description 5
- 230000006870 function Effects 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 8
- 208000002780 macular degeneration Diseases 0.000 claims description 8
- 230000003902 lesion Effects 0.000 claims description 7
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 208000000260 Warts Diseases 0.000 claims description 4
- 239000012528 membrane Substances 0.000 claims description 4
- 201000010153 skin papilloma Diseases 0.000 claims description 4
- 238000012549 training Methods 0.000 abstract description 8
- 230000004927 fusion Effects 0.000 description 6
- 238000012216 screening Methods 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 238000002372 labelling Methods 0.000 description 4
- 208000035719 Maculopathy Diseases 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 201000010099 disease Diseases 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 208000022873 Ocular disease Diseases 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005713 exacerbation Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000004393 visual impairment Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/197—Matching; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention discloses a weak supervision type early senile macular degeneration classification method based on multi-instance learning, which comprises the following steps: step one, collecting fundus images; cutting the image, extracting a green channel, sharpening and whitening; step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method; taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and the predicted result of the position of the verrucosis of the glass film is obtained after the image passes through each convolutional layer; at the output end of the network, the three prediction results are weighted and fused to be used as the final output result; the algorithm designs a special weak supervision training method based on multi-instance learning to train and detect the classifier, and only needs to know whether the fundus image has drusen or not, but can train the classifier without knowing the specific focus position; the algorithm can effectively save the cost of marking training data and improve the efficiency while ensuring the precision.
Description
Technical Field
The invention relates to a disease detection and classification method, in particular to an early senile macular degeneration classification method.
Background
Senile maculopathy is one of the common ocular diseases, mainly occurring in the elderly population older than 55 years. Senile macular degeneration can be clinically divided into three stages, namely early stage, medium stage and late stage.
Once the patient enters the advanced stage, vision loss symptoms can occur over weeks to months. In order to effectively suppress the exacerbation of senile macular degeneration, timely early-stage senile macular degeneration screening is particularly important. And the early senile macular degeneration does not affect the vision of the patient and is not easy to be found by the patient. Fundus examination is widely used clinically as a means of early screening. But relying on manual screening by a physician and marking of lesion locations can take a significant amount of time. Therefore, in order to improve screening efficiency, it is of great importance to develop an automatic early maculopathy screening system.
Clinically, the primary condition of early age-related maculopathy is the occurrence of drusen in the macular area. Yellow-white spots appear in the macular region on the eye fundus, and according to the characteristics of the disease, the current mainstream early maculopathy classification method comprises the steps of preprocessing an image, extracting the characteristics, and finally designing a classifier for classification. In the classifier training stage, an intensive supervision method is often adopted, namely, eye bottom images which contain enough drusen focus are required to be collected and the positions of the eye bottom images are marked. However, precisely labeled fundus images are very costly and time consuming to acquire in reality.
In the prior art, the study on a fundus image macular degeneration segmentation method based on a supervision descriptor is available, the supervision descriptor learning is combined with the image bottom layer characteristics through the segmentation method based on the supervision descriptor learning, the generalized low-rank matrix approximation method is used for reducing the dimension of data, and then a manifold regularization term is constructed by combining data labels, so that the image characteristics are extracted, and a certain segmentation effect can be obtained by combining an SVM classifier. However, in the process of extracting the features, the method involves iterative optimization solution of the matrix, is huge in calculated amount, is very time-consuming in practical application, depends on label information of image pixels, and has more defects in practical application.
Disclosure of Invention
The invention aims to solve the technical problem of providing a weak supervision type early senile macular degeneration classification method based on multi-instance learning, which can effectively save the cost of marking training data and improve the efficiency while ensuring the accuracy.
In order to solve the technical problems, the technical scheme of the invention is as follows: a weak supervision early senile macular degeneration classification method based on multi-instance learning comprises the following steps:
step one, collecting fundus images;
preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening;
step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method;
taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; and at the output end of the network, the three prediction results are weighted and fused to be used as the final output result.
In the second step, cutting the fixed size in the center of the image, cutting the image into 512 x 512 pixels, ensuring that the sample has uniform size, and extracting the green channel for detection; then sharpening the image by using a Laplace template; and finally, performing whitening treatment on the image to ensure that the mean value of all pixels is 0 and the variance is 1.
As a preferred technical solution, in the third step, the convolution kernel size of each convolution layer is 3*3, the step length is 1, and the channel numbers are 32, 64, and 128, respectively.
As a preferred technical solution, in the third step, a Generalized Mean (GM) function is used to convert the prediction result at the pixel level into the prediction result at the image level, and the prediction loss of each convolution layer is calculated, where the GM function is defined as:
wherein X is i For the i-th input picture,for the pixel prediction result output by the classifier, < >>R is a super parameter, which is the prediction result of the ith input picture, and 4 is taken here;
the loss of each convolution layer is as follows:
wherein Y is i And I is an indication function, which is the real result of the ith input picture.
As a preferable technical scheme, area constraint is increased for the focus position prediction of each convolution layer, wherein the specific constraint mode is that if the sum of pixel areas of positive examples predicted is larger than a constraint value and the picture is predicted to contain drusen, corresponding weight is punished; the area constraint is defined as:
wherein v is i A for predicted lesion area i For a given area constraint.
As a preferred technical solution, after predicting and increasing the area constraint for the lesion position of each layer of convolution layer, the total loss of each layer of convolution layer is:
L side (θ,ω)=l mil (θ,ω)+ηgl ac (θ,ω)
where η is a given weight value.
As a preferred technical solution, the fusion loss of weighted fusion of three prediction results is:
the total loss of the convolutional neural network is:
due to the adoption of the technical scheme, the weak supervision type early senile macular degeneration classification method based on multi-instance learning comprises the following steps of: step one, collecting fundus images; preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening; step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method; taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; at the output end of the network, the three prediction results are weighted and fused to be used as the final output result; the algorithm designs a special weak supervision training method based on multi-instance learning to train and detect the classifier, and only needs to know whether the fundus image has drusen or not, but can train the classifier without knowing the specific focus position; the algorithm can effectively save the cost of marking training data and improve the efficiency while ensuring the precision.
Drawings
Fig. 1 is a schematic diagram of an embodiment of the present invention.
Detailed Description
The invention is further elucidated below in connection with fig. 1 and an embodiment. In the following detailed description, certain exemplary embodiments of the present invention are described by way of illustration only. It is needless to say that the person skilled in the art realizes that the described embodiments may be modified in various different ways without departing from the spirit and scope of the invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive in scope.
A weak supervision early senile macular degeneration classification method based on multi-instance learning comprises the following steps:
step one, collecting fundus images;
preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening;
step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method;
taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; and at the output end of the network, the three prediction results are weighted and fused to be used as the final output result.
In the second step, firstly, cutting the fixed size in the center of the image, cutting the picture into a picture with the size of 512 multiplied by 512 pixels, ensuring that the sample has the uniform size, and secondly, extracting a green channel for detection; then sharpening the image by using a Laplace template; and finally, performing whitening treatment on the image to ensure that the mean value of all pixels is 0 and the variance is 1.
In the third step, the convolution kernel size of each convolution layer is 3*3, the step length is 1, and the channel numbers are 32, 64 and 128 respectively. Converting the predicted result at the pixel level into the predicted result at the image level using a Generalized Mean (GM) function, calculating the predicted loss for each layer of convolutional layers, the GM function being defined as:
wherein X is i For the i-th input picture,for the pixel prediction result output by the classifier, < >>R is a super parameter, which is the prediction result of the ith input picture, and 4 is taken here;
the loss of each convolution layer is as follows:
wherein Y is i And I is an indication function, which is the real result of the ith input picture.
In order to overcome the characteristic that a positive sample is excessively predicted by a multi-instance learning method, area constraint is increased for focus position prediction of each layer of convolution layer, wherein the specific constraint mode is that if the sum of pixel areas of positive instances is predicted to be larger than a constraint value and the picture is predicted to contain drusen, corresponding weight is punished; the area constraint is defined as:
wherein v is i A for predicted lesion area i For a given area constraint.
After the area constraint is increased for the focus position prediction of each layer of convolution layer, the total loss of each layer of convolution layer is as follows:
L side (θ,ω)=l mil (θ,ω)+ηgl ac (θ,ω)
where η is a given weight value.
As a preferred technical solution, the fusion loss of weighted fusion of three prediction results is:
the total loss of the convolutional neural network is:
training the neural network using samples containing only image tags, the weight update uses a random gradient descent method.
In the focus detection stage, the test image is input into a trained neural network to realize image classification, so as to obtain a predicted result of whether the image contains drusen or not. And simultaneously, taking the fusion result of pixel prediction of the three convolution layers as the prediction result of the focus position.
Compared with the existing method, the method has the following difference:
(1) traditional drusen detection and lesion marking algorithms are strong supervisory methods that require the acquisition of a large number of sample data with pixel-level labeling. The training process of the algorithm adopts a weak supervision method, only the labeling information of the image level is required to be collected, and the labeling of the pixel level is not required, so that the cost of labeling sample data can be greatly saved.
(2) Similar multi-instance learning methods have weaker feature extraction capability, and the method innovatively uses a multi-scale fusion mode and a method adopting area constraint to enhance the feature extraction capability of a network, so that the position of drusen can be better detected.
(3) The segmentation method based on supervised descriptor learning has strong feature extraction capability, but the algorithm involves a large number of matrix optimization iterative operations, has huge calculation amount, depends on image pixel label information, and is difficult to solve the problems of high cost and long time consumption; compared with the prior art, the method and the device can greatly save marking time and cost by marking information at an image level, realize classification of senile macular lesions and detect focus positions.
In the technical scheme, the detection network of drusen can also adopt a residual network as a main frame, so that the depth of the network is increased. The detection and segmentation network of drusen can also adopt a mode of combining multi-instance learning and attention network.
The foregoing has shown and described the basic principles, main features and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (4)
1. A weak supervision early age-related maculopathy classification method based on multi-instance learning, which is characterized by comprising the following steps:
step one, collecting fundus images;
preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening;
step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method;
taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; at the output end of the network, the three prediction results are weighted and fused to be used as the final output result;
in the third step, the predicted result at the pixel level is converted into the predicted result at the image level by using a Generalized Mean (GM) function, and the predicted loss of each convolution layer is calculated, where the GM function is defined as:
wherein X is i For the i-th input picture,for the pixel prediction result output by the classifier, < >>R is a super parameter, which is the prediction result of the ith input picture, and 4 is taken here;
the loss of each convolution layer is as follows:
2. The weak supervision early age-related macular degeneration classification method based on multi-instance learning as set forth in claim 1, wherein in the second step, a fixed size is cut out in the center of the image, the picture is cut out into a 512×512 pixel size picture, the sample is ensured to have a uniform size, and then the green channel is extracted for detection; then sharpening the image by using a Laplace template; and finally, performing whitening treatment on the image to ensure that the mean value of all pixels is 0 and the variance is 1.
3. The method for classifying early senile macular degeneration based on multi-instance learning of claim 1, wherein in the third step, the convolution kernel size of each convolution layer is 3*3, the step size is 1, and the channel number is 32, 64, 128, respectively.
4. The weak supervision early age-related macular degeneration classification method based on multi-instance learning of claim 1, wherein the area constraint is increased for the focus position prediction of each convolution layer in such a way that if the sum of the pixel areas predicted as positive instances is larger than the constraint value and the picture is predicted as containing drusen, the corresponding weight is penalized; the area constraint is defined as:
wherein v is i A for predicted lesion area i For a given area constraint.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910120667.8A CN109670489B (en) | 2019-02-18 | 2019-02-18 | Weak supervision type early senile macular degeneration classification method based on multi-instance learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910120667.8A CN109670489B (en) | 2019-02-18 | 2019-02-18 | Weak supervision type early senile macular degeneration classification method based on multi-instance learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109670489A CN109670489A (en) | 2019-04-23 |
CN109670489B true CN109670489B (en) | 2023-06-27 |
Family
ID=66152377
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910120667.8A Active CN109670489B (en) | 2019-02-18 | 2019-02-18 | Weak supervision type early senile macular degeneration classification method based on multi-instance learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109670489B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110689081B (en) * | 2019-09-30 | 2020-08-21 | 中国科学院大学 | Weak supervision target classification and positioning method based on bifurcation learning |
CN111179227B (en) * | 2019-12-16 | 2022-04-05 | 西北工业大学 | Mammary gland ultrasonic image quality evaluation method based on auxiliary diagnosis and subjective aesthetics |
CN111986150B (en) * | 2020-07-17 | 2024-02-09 | 万达信息股份有限公司 | The method comprises the following steps of: digital number pathological image Interactive annotation refining method |
CN113822252B (en) * | 2021-11-24 | 2022-04-22 | 杭州迪英加科技有限公司 | Pathological image cell robust detection method under microscope |
CN117576052A (en) * | 2023-11-27 | 2024-02-20 | 中国矿业大学 | Method for diagnosing focus of mammary gland X-ray photography |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106530305B (en) * | 2016-09-23 | 2019-09-13 | 北京市商汤科技开发有限公司 | Semantic segmentation model training and image partition method and device calculate equipment |
CN107437092B (en) * | 2017-06-28 | 2019-11-15 | 苏州比格威医疗科技有限公司 | The classification method of retina OCT image based on Three dimensional convolution neural network |
CN107862695A (en) * | 2017-12-06 | 2018-03-30 | 电子科技大学 | A kind of modified image segmentation training method based on full convolutional neural networks |
CN108062753B (en) * | 2017-12-29 | 2020-04-17 | 重庆理工大学 | Unsupervised domain self-adaptive brain tumor semantic segmentation method based on deep counterstudy |
US10176405B1 (en) * | 2018-06-18 | 2019-01-08 | Inception Institute Of Artificial Intelligence | Vehicle re-identification techniques using neural networks for image analysis, viewpoint-aware pattern recognition, and generation of multi- view vehicle representations |
CN109035252B (en) * | 2018-06-29 | 2019-09-24 | 山东财经大学 | A kind of super-pixel method towards medical image segmentation |
-
2019
- 2019-02-18 CN CN201910120667.8A patent/CN109670489B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN109670489A (en) | 2019-04-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109670489B (en) | Weak supervision type early senile macular degeneration classification method based on multi-instance learning | |
CN109886273B (en) | CMR image segmentation and classification system | |
JP6999812B2 (en) | Bone age evaluation and height prediction model establishment method, its system and its prediction method | |
CN109325942B (en) | Fundus image structure segmentation method based on full convolution neural network | |
WO2021196632A1 (en) | Intelligent analysis system and method for panoramic digital pathological image | |
CN109886946B (en) | Deep learning-based early senile maculopathy weakening supervision and classification method | |
CN112488234B (en) | End-to-end histopathology image classification method based on attention pooling | |
CN111882560B (en) | Lung parenchyma CT image segmentation method based on weighted full convolution neural network | |
CN110245657B (en) | Pathological image similarity detection method and detection device | |
CN111931811B (en) | Calculation method based on super-pixel image similarity | |
CN112184657A (en) | Pulmonary nodule automatic detection method, device and computer system | |
TWI684997B (en) | Establishing method of bone age assessment and height prediction model, bone age assessment and height prediction system, and bone age assessment and height prediction method | |
WO2023045231A1 (en) | Method and apparatus for facial nerve segmentation by decoupling and divide-and-conquer | |
Bai et al. | Automatic segmentation of cervical region in colposcopic images using K-means | |
CN112862830A (en) | Multi-modal image segmentation method, system, terminal and readable storage medium | |
CN110766670A (en) | Mammary gland molybdenum target image tumor localization algorithm based on deep convolutional neural network | |
CN106803248A (en) | Fuzzy license plate image blur evaluation method | |
CN112233061A (en) | Deep learning-based skin basal cell carcinoma and Babylonia disease identification method | |
Yang et al. | Towards unbiased COVID-19 lesion localisation and segmentation via weakly supervised learning | |
CN111476794A (en) | UNET-based cervical pathological tissue segmentation method | |
CN115526834A (en) | Immunofluorescence image detection method and device, equipment and storage medium | |
CN116883341A (en) | Liver tumor CT image automatic segmentation method based on deep learning | |
CN115206495A (en) | Renal cancer pathological image analysis method and system based on CoAtNet deep learning and intelligent microscopic device | |
CN114140437A (en) | Fundus hard exudate segmentation method based on deep learning | |
CN110826629A (en) | Otoscope image auxiliary diagnosis method based on fine-grained classification |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |