CN109670489B - Weak supervision type early senile macular degeneration classification method based on multi-instance learning - Google Patents

Weak supervision type early senile macular degeneration classification method based on multi-instance learning Download PDF

Info

Publication number
CN109670489B
CN109670489B CN201910120667.8A CN201910120667A CN109670489B CN 109670489 B CN109670489 B CN 109670489B CN 201910120667 A CN201910120667 A CN 201910120667A CN 109670489 B CN109670489 B CN 109670489B
Authority
CN
China
Prior art keywords
image
instance learning
method based
neural network
result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910120667.8A
Other languages
Chinese (zh)
Other versions
CN109670489A (en
Inventor
曹桂平
谢新林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Original Assignee
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shiyuan Electronics Thecnology Co Ltd filed Critical Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority to CN201910120667.8A priority Critical patent/CN109670489B/en
Publication of CN109670489A publication Critical patent/CN109670489A/en
Application granted granted Critical
Publication of CN109670489B publication Critical patent/CN109670489B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/197—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/32—Normalisation of the pattern dimensions
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Pathology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

The invention discloses a weak supervision type early senile macular degeneration classification method based on multi-instance learning, which comprises the following steps: step one, collecting fundus images; cutting the image, extracting a green channel, sharpening and whitening; step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method; taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and the predicted result of the position of the verrucosis of the glass film is obtained after the image passes through each convolutional layer; at the output end of the network, the three prediction results are weighted and fused to be used as the final output result; the algorithm designs a special weak supervision training method based on multi-instance learning to train and detect the classifier, and only needs to know whether the fundus image has drusen or not, but can train the classifier without knowing the specific focus position; the algorithm can effectively save the cost of marking training data and improve the efficiency while ensuring the precision.

Description

Weak supervision type early senile macular degeneration classification method based on multi-instance learning
Technical Field
The invention relates to a disease detection and classification method, in particular to an early senile macular degeneration classification method.
Background
Senile maculopathy is one of the common ocular diseases, mainly occurring in the elderly population older than 55 years. Senile macular degeneration can be clinically divided into three stages, namely early stage, medium stage and late stage.
Once the patient enters the advanced stage, vision loss symptoms can occur over weeks to months. In order to effectively suppress the exacerbation of senile macular degeneration, timely early-stage senile macular degeneration screening is particularly important. And the early senile macular degeneration does not affect the vision of the patient and is not easy to be found by the patient. Fundus examination is widely used clinically as a means of early screening. But relying on manual screening by a physician and marking of lesion locations can take a significant amount of time. Therefore, in order to improve screening efficiency, it is of great importance to develop an automatic early maculopathy screening system.
Clinically, the primary condition of early age-related maculopathy is the occurrence of drusen in the macular area. Yellow-white spots appear in the macular region on the eye fundus, and according to the characteristics of the disease, the current mainstream early maculopathy classification method comprises the steps of preprocessing an image, extracting the characteristics, and finally designing a classifier for classification. In the classifier training stage, an intensive supervision method is often adopted, namely, eye bottom images which contain enough drusen focus are required to be collected and the positions of the eye bottom images are marked. However, precisely labeled fundus images are very costly and time consuming to acquire in reality.
In the prior art, the study on a fundus image macular degeneration segmentation method based on a supervision descriptor is available, the supervision descriptor learning is combined with the image bottom layer characteristics through the segmentation method based on the supervision descriptor learning, the generalized low-rank matrix approximation method is used for reducing the dimension of data, and then a manifold regularization term is constructed by combining data labels, so that the image characteristics are extracted, and a certain segmentation effect can be obtained by combining an SVM classifier. However, in the process of extracting the features, the method involves iterative optimization solution of the matrix, is huge in calculated amount, is very time-consuming in practical application, depends on label information of image pixels, and has more defects in practical application.
Disclosure of Invention
The invention aims to solve the technical problem of providing a weak supervision type early senile macular degeneration classification method based on multi-instance learning, which can effectively save the cost of marking training data and improve the efficiency while ensuring the accuracy.
In order to solve the technical problems, the technical scheme of the invention is as follows: a weak supervision early senile macular degeneration classification method based on multi-instance learning comprises the following steps:
step one, collecting fundus images;
preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening;
step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method;
taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; and at the output end of the network, the three prediction results are weighted and fused to be used as the final output result.
In the second step, cutting the fixed size in the center of the image, cutting the image into 512 x 512 pixels, ensuring that the sample has uniform size, and extracting the green channel for detection; then sharpening the image by using a Laplace template; and finally, performing whitening treatment on the image to ensure that the mean value of all pixels is 0 and the variance is 1.
As a preferred technical solution, in the third step, the convolution kernel size of each convolution layer is 3*3, the step length is 1, and the channel numbers are 32, 64, and 128, respectively.
As a preferred technical solution, in the third step, a Generalized Mean (GM) function is used to convert the prediction result at the pixel level into the prediction result at the image level, and the prediction loss of each convolution layer is calculated, where the GM function is defined as:
Figure BDA0001971780230000021
wherein X is i For the i-th input picture,
Figure BDA0001971780230000022
for the pixel prediction result output by the classifier, < >>
Figure BDA0001971780230000023
R is a super parameter, which is the prediction result of the ith input picture, and 4 is taken here;
the loss of each convolution layer is as follows:
Figure BDA0001971780230000024
wherein Y is i And I is an indication function, which is the real result of the ith input picture.
As a preferable technical scheme, area constraint is increased for the focus position prediction of each convolution layer, wherein the specific constraint mode is that if the sum of pixel areas of positive examples predicted is larger than a constraint value and the picture is predicted to contain drusen, corresponding weight is punished; the area constraint is defined as:
Figure BDA0001971780230000031
wherein v is i A for predicted lesion area i For a given area constraint.
As a preferred technical solution, after predicting and increasing the area constraint for the lesion position of each layer of convolution layer, the total loss of each layer of convolution layer is:
L side (θ,ω)＝l mil (θ,ω)+ηgl ac (θ,ω)
where η is a given weight value.
As a preferred technical solution, the fusion loss of weighted fusion of three prediction results is:
Figure BDA0001971780230000032
the total loss of the convolutional neural network is:
Figure BDA0001971780230000033
due to the adoption of the technical scheme, the weak supervision type early senile macular degeneration classification method based on multi-instance learning comprises the following steps of: step one, collecting fundus images; preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening; step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method; taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; at the output end of the network, the three prediction results are weighted and fused to be used as the final output result; the algorithm designs a special weak supervision training method based on multi-instance learning to train and detect the classifier, and only needs to know whether the fundus image has drusen or not, but can train the classifier without knowing the specific focus position; the algorithm can effectively save the cost of marking training data and improve the efficiency while ensuring the precision.
Drawings
Fig. 1 is a schematic diagram of an embodiment of the present invention.
Detailed Description
The invention is further elucidated below in connection with fig. 1 and an embodiment. In the following detailed description, certain exemplary embodiments of the present invention are described by way of illustration only. It is needless to say that the person skilled in the art realizes that the described embodiments may be modified in various different ways without departing from the spirit and scope of the invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive in scope.
A weak supervision early senile macular degeneration classification method based on multi-instance learning comprises the following steps:
step one, collecting fundus images;
preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening;
step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method;
taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; and at the output end of the network, the three prediction results are weighted and fused to be used as the final output result.
In the second step, firstly, cutting the fixed size in the center of the image, cutting the picture into a picture with the size of 512 multiplied by 512 pixels, ensuring that the sample has the uniform size, and secondly, extracting a green channel for detection; then sharpening the image by using a Laplace template; and finally, performing whitening treatment on the image to ensure that the mean value of all pixels is 0 and the variance is 1.
In the third step, the convolution kernel size of each convolution layer is 3*3, the step length is 1, and the channel numbers are 32, 64 and 128 respectively. Converting the predicted result at the pixel level into the predicted result at the image level using a Generalized Mean (GM) function, calculating the predicted loss for each layer of convolutional layers, the GM function being defined as:
Figure BDA0001971780230000041
wherein X is i For the i-th input picture,
Figure BDA0001971780230000042
for the pixel prediction result output by the classifier, < >>
Figure BDA0001971780230000043
R is a super parameter, which is the prediction result of the ith input picture, and 4 is taken here;
the loss of each convolution layer is as follows:
Figure BDA0001971780230000044
wherein Y is i And I is an indication function, which is the real result of the ith input picture.
In order to overcome the characteristic that a positive sample is excessively predicted by a multi-instance learning method, area constraint is increased for focus position prediction of each layer of convolution layer, wherein the specific constraint mode is that if the sum of pixel areas of positive instances is predicted to be larger than a constraint value and the picture is predicted to contain drusen, corresponding weight is punished; the area constraint is defined as:
Figure BDA0001971780230000051
wherein v is i A for predicted lesion area i For a given area constraint.
After the area constraint is increased for the focus position prediction of each layer of convolution layer, the total loss of each layer of convolution layer is as follows:
L side (θ,ω)＝l mil (θ,ω)+ηgl ac (θ,ω)
where η is a given weight value.
As a preferred technical solution, the fusion loss of weighted fusion of three prediction results is:
Figure BDA0001971780230000052
the total loss of the convolutional neural network is:
Figure BDA0001971780230000053
training the neural network using samples containing only image tags, the weight update uses a random gradient descent method.
In the focus detection stage, the test image is input into a trained neural network to realize image classification, so as to obtain a predicted result of whether the image contains drusen or not. And simultaneously, taking the fusion result of pixel prediction of the three convolution layers as the prediction result of the focus position.
Compared with the existing method, the method has the following difference:
(1) traditional drusen detection and lesion marking algorithms are strong supervisory methods that require the acquisition of a large number of sample data with pixel-level labeling. The training process of the algorithm adopts a weak supervision method, only the labeling information of the image level is required to be collected, and the labeling of the pixel level is not required, so that the cost of labeling sample data can be greatly saved.
(2) Similar multi-instance learning methods have weaker feature extraction capability, and the method innovatively uses a multi-scale fusion mode and a method adopting area constraint to enhance the feature extraction capability of a network, so that the position of drusen can be better detected.
(3) The segmentation method based on supervised descriptor learning has strong feature extraction capability, but the algorithm involves a large number of matrix optimization iterative operations, has huge calculation amount, depends on image pixel label information, and is difficult to solve the problems of high cost and long time consumption; compared with the prior art, the method and the device can greatly save marking time and cost by marking information at an image level, realize classification of senile macular lesions and detect focus positions.
In the technical scheme, the detection network of drusen can also adopt a residual network as a main frame, so that the depth of the network is increased. The detection and segmentation network of drusen can also adopt a mode of combining multi-instance learning and attention network.
The foregoing has shown and described the basic principles, main features and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (4)

1. A weak supervision early age-related maculopathy classification method based on multi-instance learning, which is characterized by comprising the following steps:
step one, collecting fundus images;
preprocessing an image, cutting the image, extracting a green channel, sharpening and whitening;
step three, detecting drusen and dividing focus by using a convolution neural network and multi-instance learning combination method;
taking a convolutional neural network as a main frame, wherein the convolutional neural network comprises three convolutional layers, and a prediction result of a glass membrane wart disease position is obtained after an image passes through each convolutional layer; at the output end of the network, the three prediction results are weighted and fused to be used as the final output result;
in the third step, the predicted result at the pixel level is converted into the predicted result at the image level by using a Generalized Mean (GM) function, and the predicted loss of each convolution layer is calculated, where the GM function is defined as:
Figure QLYQS_1
wherein X is i For the i-th input picture,
Figure QLYQS_2
for the pixel prediction result output by the classifier, < >>
Figure QLYQS_3
R is a super parameter, which is the prediction result of the ith input picture, and 4 is taken here;
the loss of each convolution layer is as follows:
Figure QLYQS_4
wherein Y is i For the real result of the ith input picture, I is an indication function, t is the sequence number of the convolution layer, and t=1 or 2 or 3;
Figure QLYQS_5
and outputting a result of the ith picture in the corresponding convolution layer t.
2. The weak supervision early age-related macular degeneration classification method based on multi-instance learning as set forth in claim 1, wherein in the second step, a fixed size is cut out in the center of the image, the picture is cut out into a 512×512 pixel size picture, the sample is ensured to have a uniform size, and then the green channel is extracted for detection; then sharpening the image by using a Laplace template; and finally, performing whitening treatment on the image to ensure that the mean value of all pixels is 0 and the variance is 1.
3. The method for classifying early senile macular degeneration based on multi-instance learning of claim 1, wherein in the third step, the convolution kernel size of each convolution layer is 3*3, the step size is 1, and the channel number is 32, 64, 128, respectively.
4. The weak supervision early age-related macular degeneration classification method based on multi-instance learning of claim 1, wherein the area constraint is increased for the focus position prediction of each convolution layer in such a way that if the sum of the pixel areas predicted as positive instances is larger than the constraint value and the picture is predicted as containing drusen, the corresponding weight is penalized; the area constraint is defined as:
Figure QLYQS_6
wherein v is i A for predicted lesion area i For a given area constraint.
CN201910120667.8A 2019-02-18 2019-02-18 Weak supervision type early senile macular degeneration classification method based on multi-instance learning Active CN109670489B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910120667.8A CN109670489B (en) 2019-02-18 2019-02-18 Weak supervision type early senile macular degeneration classification method based on multi-instance learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910120667.8A CN109670489B (en) 2019-02-18 2019-02-18 Weak supervision type early senile macular degeneration classification method based on multi-instance learning

Publications (2)

Publication Number Publication Date
CN109670489A CN109670489A (en) 2019-04-23
CN109670489B true CN109670489B (en) 2023-06-27

Family

ID=66152377

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910120667.8A Active CN109670489B (en) 2019-02-18 2019-02-18 Weak supervision type early senile macular degeneration classification method based on multi-instance learning

Country Status (1)

Country Link
CN (1) CN109670489B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110689081B (en) * 2019-09-30 2020-08-21 中国科学院大学 Weak supervision target classification and positioning method based on bifurcation learning
CN111179227B (en) * 2019-12-16 2022-04-05 西北工业大学 Mammary gland ultrasonic image quality evaluation method based on auxiliary diagnosis and subjective aesthetics
CN111986150B (en) * 2020-07-17 2024-02-09 万达信息股份有限公司 The method comprises the following steps of: digital number pathological image Interactive annotation refining method
CN113822252B (en) * 2021-11-24 2022-04-22 杭州迪英加科技有限公司 Pathological image cell robust detection method under microscope
CN117576052A (en) * 2023-11-27 2024-02-20 中国矿业大学 Method for diagnosing focus of mammary gland X-ray photography

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530305B (en) * 2016-09-23 2019-09-13 北京市商汤科技开发有限公司 Semantic segmentation model training and image partition method and device calculate equipment
CN107437092B (en) * 2017-06-28 2019-11-15 苏州比格威医疗科技有限公司 The classification method of retina OCT image based on Three dimensional convolution neural network
CN107862695A (en) * 2017-12-06 2018-03-30 电子科技大学 A kind of modified image segmentation training method based on full convolutional neural networks
CN108062753B (en) * 2017-12-29 2020-04-17 重庆理工大学 Unsupervised domain self-adaptive brain tumor semantic segmentation method based on deep counterstudy
US10176405B1 (en) * 2018-06-18 2019-01-08 Inception Institute Of Artificial Intelligence Vehicle re-identification techniques using neural networks for image analysis, viewpoint-aware pattern recognition, and generation of multi- view vehicle representations
CN109035252B (en) * 2018-06-29 2019-09-24 山东财经大学 A kind of super-pixel method towards medical image segmentation

Also Published As

Publication number Publication date
CN109670489A (en) 2019-04-23

Similar Documents

Publication Publication Date Title
CN109670489B (en) Weak supervision type early senile macular degeneration classification method based on multi-instance learning
CN109886273B (en) CMR image segmentation and classification system
JP6999812B2 (en) Bone age evaluation and height prediction model establishment method, its system and its prediction method
CN109325942B (en) Fundus image structure segmentation method based on full convolution neural network
WO2021196632A1 (en) Intelligent analysis system and method for panoramic digital pathological image
CN109886946B (en) Deep learning-based early senile maculopathy weakening supervision and classification method
CN112488234B (en) End-to-end histopathology image classification method based on attention pooling
CN111882560B (en) Lung parenchyma CT image segmentation method based on weighted full convolution neural network
CN110245657B (en) Pathological image similarity detection method and detection device
CN111931811B (en) Calculation method based on super-pixel image similarity
CN112184657A (en) Pulmonary nodule automatic detection method, device and computer system
TWI684997B (en) Establishing method of bone age assessment and height prediction model, bone age assessment and height prediction system, and bone age assessment and height prediction method
WO2023045231A1 (en) Method and apparatus for facial nerve segmentation by decoupling and divide-and-conquer
Bai et al. Automatic segmentation of cervical region in colposcopic images using K-means
CN112862830A (en) Multi-modal image segmentation method, system, terminal and readable storage medium
CN110766670A (en) Mammary gland molybdenum target image tumor localization algorithm based on deep convolutional neural network
CN106803248A (en) Fuzzy license plate image blur evaluation method
CN112233061A (en) Deep learning-based skin basal cell carcinoma and Babylonia disease identification method
Yang et al. Towards unbiased COVID-19 lesion localisation and segmentation via weakly supervised learning
CN111476794A (en) UNET-based cervical pathological tissue segmentation method
CN115526834A (en) Immunofluorescence image detection method and device, equipment and storage medium
CN116883341A (en) Liver tumor CT image automatic segmentation method based on deep learning
CN115206495A (en) Renal cancer pathological image analysis method and system based on CoAtNet deep learning and intelligent microscopic device
CN114140437A (en) Fundus hard exudate segmentation method based on deep learning
CN110826629A (en) Otoscope image auxiliary diagnosis method based on fine-grained classification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant