CN109658466B - Disease retina optical coherence tomography image simulation generation method - Google Patents

Disease retina optical coherence tomography image simulation generation method Download PDF

Info

Publication number
CN109658466B
CN109658466B CN201811500608.5A CN201811500608A CN109658466B CN 109658466 B CN109658466 B CN 109658466B CN 201811500608 A CN201811500608 A CN 201811500608A CN 109658466 B CN109658466 B CN 109658466B
Authority
CN
China
Prior art keywords
image
disease
generator
oct
discriminator
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811500608.5A
Other languages
Chinese (zh)
Other versions
CN109658466A (en
Inventor
陈新建
查雪玮
石霏
朱伟芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou University
Original Assignee
Suzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou University filed Critical Suzhou University
Priority to CN201811500608.5A priority Critical patent/CN109658466B/en
Publication of CN109658466A publication Critical patent/CN109658466A/en
Application granted granted Critical
Publication of CN109658466B publication Critical patent/CN109658466B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

The invention provides a method for generating a retina OCT disease image with feasibility and effectiveness, wherein the generated OCT image can be used for expanding a training data set of an OCT disease image classification algorithm; the method is based on conditional generation of an antagonistic network cGAN, a network structure is composed of a generator and a discriminator, and a normal retina OCT image is converted into an OCT disease image which can be used for supplementing a classification model training set by combining a generated antagonistic loss function cGAN loss with a novel structure similarity loss function SSIM loss.

Description

Disease retina optical coherence tomography image simulation generation method
Technical Field
The invention relates to a simulation generation method of optical coherence tomography images of disease retinas, belonging to the technical field of simulation generation of retinal images.
Background
Optical Coherence Tomography (OCT) can display tomographic images of fundus tissues, and is an effective ophthalmic disease image diagnosis technique. With the rapid development of artificial intelligence and the deep combination of the medical industry, the OCT ophthalmic disease image automatic classification algorithm with high accuracy and large calculation amount can assist doctors to find diseases and improve the working efficiency of hospitals. At present, an OCT ophthalmologic disease image automatic classification algorithm is based on a large amount of ophthalmologic disease image data, and a composed data set is used for training and testing a classification model. However, the process of acquiring OCT ophthalmic disease data is complicated, and although the number of normal retinal OCT images is sufficient, the number of disease images is far from that, and the amount of image data of different kinds of diseases is also greatly different. The training model is greatly influenced by the data sets with less data volume and larger quantity difference, so that an image for training the classification model can be generated by using a retina OCT disease influence simulation generation algorithm, the data volume of the training set is expanded, the unbalance of different disease data quantities is made up, and the accuracy of the automatic classification algorithm is improved.
The current image simulation generation algorithm has the following defects: (1) Most algorithms are used for generating non-medical images such as human faces or living scenes, the algorithms for generating the OCT images of the retina are few, the difference of the OCT images of part of different diseases is not obvious enough, and difficulty is added to the realization of the generation algorithm; (2) Most OCT image simulation generation methods are based on traditional algorithms, and few algorithms are used for combining medical image generation and deep learning with the characteristic of integrated extraction features.
Pigment epithelium layer detachment (PED) is an early symptom of central serous choroidopathy and exudative age-related macular degeneration, patients with severe Drusen (Drusen) can affect vision, macular Hole (MH) is not high in prevalence rate, but macular tissues of patients are damaged, and central vision is greatly weakened. The three ophthalmic diseases are diseases which are clinically concerned by doctors, and an algorithm for OCT image simulation generation of the three ophthalmic diseases does not exist so far.
Disclosure of Invention
The purpose is as follows: in order to overcome the defects in the prior art, the invention provides a simulation generation method of optical coherence tomography images of disease retinas.
The technical scheme is as follows: in order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a simulation generation method of optical coherence tomography image of disease retina comprises the following steps,
(1) Establishing a training data set: OCT image x of normal retina i And OCT image y of retina with disease i Form a series of paired images { (x) i ,y i ) Forming a training data set;
(2) Establishing a network structure of the model: the network structure is based on cGAN and comprises two neural networks of a generator G and a discriminator D; sending the paired data sets established in the step (1) to a generator for training, converting a normal OCT image into a disease OCT image by the generator, and inputting the generated image and an original image to a discriminator; the other set of inputs of the discriminator is the paired original image and real image; finally, the discriminator maps the input picture into a scalar which represents the probability that the input picture is a real picture, and the network of the generator is optimized after the scalar is reversely transmitted to the generator;
(3) Composition of the loss function: the loss function comprises two parts, wherein one part is a cGAN antagonistic loss function corresponding to cGAN, and the other part is a structural similarity loss function proposed aiming at the image of the ophthalmic disease; combining the two functions to continuously train the model established in the step (2).
Preferably, when the normal retinal OCT image is stitched with the retinal OCT image with the disease, the left side is the normal OCT image and the right side is the disease OCT image.
Preferably, the generator and the discriminator have the same target pair, the training process is as follows, the generator makes the generated image more real as possible, and the discriminator is used for judging whether the image is true or false; along with the lapse of time, the generator and the arbiter mutually game, continuously fight against, two networks have reached a dynamic equilibrium finally: the image generated by the generator is close to a real image, and the discriminator can better identify the real image and the false image; and finally obtaining a trained generator model cGAN:
Figure BDA0001898046880000021
preferably, the formula of the cGAN oppositional loss function is:
L cGAN (G,D)＝E x,y [log D(x,y)]+E x,z [log(1-D(x,G(x,z))]，
wherein G (x, z) is the generator output image and D (x, y) is the discriminator discrimination result.
Preferably, the function of the structural similarity loss function SSIM loss is to compare the structural similarity between the generated image and the target image, so that the generated image is as close to the target image as possible, and the formula is as follows:
Figure BDA0001898046880000022
wherein the content of the first and second substances,
Figure BDA0001898046880000023
Figure BDA0001898046880000031
Figure BDA0001898046880000032
Figure BDA0001898046880000033
s denotes the generated image G (x, z), y denotes the real image, μ s Is the average value of s,. Mu. y Is the average value of the y and,
Figure BDA0001898046880000034
is the variance of s and is,
Figure BDA0001898046880000035
is the variance of y, σ sy Is the covariance of s and y; r and C represent the height and width of the image, C 1 ＝0.0001,c 2 =0.0009 is a constant for maintaining stability.
Preferably, the cGAN antagonistic loss function in combination with the structural similarity loss function proposed for ophthalmic disease images forms a final loss function formula as follows:
Figure BDA0001898046880000036
λ ranges from 90 to 110.
Has the advantages that: the invention provides an automatic generation method of a retina OCT disease image with feasibility and effectiveness, which utilizes an improved deep learning model cGAN to generate retina OCT images with various pathological changes, and the generated images can be used as a training data set of a retina disease image classification model.
Drawings
FIG. 1 is a block diagram of a network architecture of generators and discriminators in accordance with the present invention;
FIG. 2 is an example of an OCT image of the retina of PED disease generated by the present invention;
FIG. 3 is an example of a Drusen disease retinal OCT image generated by the present invention;
fig. 4 is an example of MH disease retinal OCT images generated by the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
The invention provides a simulation generation method of optical coherence tomography images of disease retinas, which mainly comprises the following three aspects: establishing a training data set, establishing a network structure of a model, and forming a loss function.
(1) Establishing a training data set:
three types of OCT image data sets are respectively established by taking three ophthalmic diseases as examples, each type of data set splices a normal retina OCT image and a disease OCT image with the same size, the normal OCT image is arranged on the left side, the disease OCT image is arranged on the right side, the size of the images is 512 multiplied by 885, the size of the spliced training images is 1024 multiplied by 885, and the number of the images of each disease in the training set is about 1000.
(2) Network structure of the model:
based on cGAN, an image { x, z } formed by adding random noise z to an original image x is learned, and mapping to a target image y is represented by G: { x, z } → y.
The network structure comprises a generator part and a discriminator part, and a training data set is composed of a series of paired images { (x) i ,y i ) Composition of, wherein x i Indicating normal retinal OCT image, y i Representing OCT images of the retina with disease, these established pairs of training data sets will be sent to the generator for training. The generator G is used for generating normal OCT image x i Conversion into a disease OCT image y i At the same time, the discriminator is allowed to judge it as true as much as possible, and finally, the generated image is combined with the original image x i Are input to the discriminator together. The discriminator is used to receive the real image or the image generated by the generator G, and its main role is to discriminate the "true" image from the "false" image as much as possible. The discriminator maps the input picture into a scalar quantity which isRepresenting the probability that the input image is a real image, the closer to 1, the greater the likelihood that the input image is a real image.
These two components constitute a supervised learning mode, as shown in fig. 1, and the final objective function is as follows:
L cGAN (G,D)＝E x,y [log D(x,y)]+E x,z [log(1-D(x,G(x,z))]，
wherein, G (x, z) is the output image of the generator, and D (x, y) and D (x, G (x, z)) are the discrimination results of the discriminator. The generator and the discriminator have opposite and uniform targets, the training process is an optimization process as follows, and a trained generator model cGAN is finally obtained:
Figure BDA0001898046880000041
in the whole process, the generator makes the generated image more real as much as possible, and the function of the discriminator is to judge whether the image is true or false. Along with the time, the generator and the discriminator game mutually and continuously resist, and finally the two networks reach a dynamic balance: the image generated by the generator is close to the distribution of a real image, and the discriminator can better identify the truth of the image.
(3) Composition of the loss function:
in order to better apply cGAN to the retina OCT image, the invention designs a loss function SSIM loss specially used for the OCT image. The function of SSIM loss is to compare the structural similarity between the generated image and the target image so that the generated image is as close as possible to the target image, and the formula of SSIM loss is as follows:
Figure BDA0001898046880000051
wherein the content of the first and second substances,
Figure BDA0001898046880000052
Figure BDA0001898046880000053
Figure BDA0001898046880000054
Figure BDA0001898046880000055
s denotes the generated image G (x, z), y denotes the real image, μ s Is the average value of s,. Mu. y Is the average value of the y and,
Figure BDA0001898046880000056
is the variance of s and is the sum of the variance,
Figure BDA0001898046880000057
is the variance of y, σ sy Is the covariance of s and y. R and C represent the height and width of the image, C 1 ＝0.0001,c 2 =0.0009 is a constant for maintaining stability.
The two loss functions cGAN loss and SSIM loss are combined, and the final loss function formula is as follows:
Figure BDA0001898046880000059
the lambda ranges from 90 to 110.
(4) The experimental results are as follows: the generated OCT influence quality is evaluated by adopting a subjective method and an objective method, and compared with the method before improvement.
Subjective evaluation: four technical experts in the field are invited, 60 retinal OCT images are displayed for each person, wherein 30 images are generated by the generator, and 30 real images are generated by the generator; the experiment stipulates that the expert needs to judge the authenticity of the seen image within 3 seconds, and the final experimental result is shown in table 1, in which the data represents the percentage of the generated image marked as a real image.
TABLE 1 before and after the improvement of the method, the probability of marking the generated image as a real image is subjectively evaluated
Figure BDA0001898046880000058
Objective evaluation: namely, a mature deep learning classification network is used to verify the quality of generated OCT images, and examples of the generated OCT images of the three diseases of the retina are shown in figures 2-4.
Firstly, a three-classification model is trained by using real images, and the OCT images of diseases generated by the method provided by the invention are used for testing, so as to observe whether the generated images can be accurately classified, wherein the number of training sets is 300 for each type of diseases, and the number of images of testing sets is 100 for each type of diseases. The classification accuracy is shown in table 2, and the accuracy is high.
TABLE 2 before and after the method improvement, the classification model trained by real images is used to test the accuracy of the generated image classification
Figure BDA0001898046880000061
Then, a three-classification model is trained by using the disease OCT image generated by the invention, and a real image is used for testing whether the model can classify the image well, wherein the training set and the testing set respectively comprise 300 and 100 diseases of each type. The classification accuracy is shown in table 3, and the accuracy is high.
TABLE 3 test accuracy of true image classification using classification models trained with generated images before and after improvement of the method
Figure BDA0001898046880000062
Up to this point, a method for automatically generating an OCT image of the retina has been implemented and verified. The invention provides a loss function aiming at the retina OCT disease image based on cGAN and corresponding improvement, and the generated OCT image can be used as a training data set of an automatic classification algorithm of the ophthalmology disease image, so that the problems of unbalanced training data set and the like in the classification process are greatly improved.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention, and such modifications and adaptations are intended to be within the scope of the invention.

Claims (5)

1. A simulation generation method of optical coherence tomography images of disease retinas is characterized in that: comprises the following steps of (a) carrying out,
(1) Establishing a training data set: OCT image x of normal retina i And OCT image y of retina with disease i Form a series of paired images { (x) i ,y i ) Forming a training data set;
(2) Establishing a network structure of the model: the network structure is based on cGAN and comprises two neural networks of a generator G and a discriminator D; the paired data sets established in the step (1) are sent to a generator for training, the generator converts a normal OCT image into a disease OCT image, and the generated image and an original image are input to a discriminator; the other set of inputs of the discriminator is the paired original image and real image; finally, the discriminator maps the input picture into a scalar which represents the probability that the input picture is a real picture, and the network of the generator is optimized after the scalar is reversely transmitted to the generator;
(3) Composition of the loss function: the loss function comprises two parts, wherein one part is a cGAN antagonistic loss function corresponding to cGAN, and the other part is a structural similarity loss function proposed aiming at the image of the ophthalmic disease; combining the two functions to continuously train the model established in the step (2);
the structural similarity loss function SSIMloss has the following formula:
Figure FDA0003910295630000011
wherein the content of the first and second substances,
Figure FDA0003910295630000012
Figure FDA0003910295630000013
Figure FDA0003910295630000014
Figure FDA0003910295630000015
s denotes the generated image G (x, z), y denotes the real image, μ s Is the average value of s, μ y Is the average value of y and is,
Figure FDA0003910295630000016
is the variance of s and is the sum of the variance,
Figure FDA0003910295630000017
is the variance of y, σ sy Is the covariance of s and y; r and C represent the height and width of the image, C 1 ＝0.0001,c 2 =0.0009 is a constant for maintaining stability.
2. The method for generating an optical coherence tomography image of a disease according to claim 1, wherein: when the normal retina OCT image is spliced with the retina OCT image with the disease, the left side is the normal OCT image, and the right side is the disease OCT image.
3. The method for simulating generation of the optical coherence tomography image of the disease retina according to claim 1, wherein: the generator and the discriminator have the same target opposites, the training process is as follows, the generator makes the generated image more real, and the discriminator is used for judging whether the image is true or false; and (3) as time goes on, the generator and the discriminator game with each other, continuously resist, finally the two networks reach a dynamic balance, and finally a trained generator model cGAN is obtained:
Figure FDA0003910295630000021
4. the method for simulating generation of the optical coherence tomography image of the disease retina according to claim 1, wherein: the formula of the cGAN confrontation loss function is:
L cGAN (G,D)＝E x,y [log D(x,y)]+E x,z [log(1-D(x,G(x,z))]，
wherein G (x, z) is the generator output image and D (x, y) is the discriminator discrimination result.
5. The method for simulating generation of the optical coherence tomography image of the disease retina according to claim 1, wherein: the final loss function formula formed by the combination of the cGAN antagonistic loss function and the structural similarity loss function proposed for the ophthalmic disease image is as follows:
Figure FDA0003910295630000022
the lambda ranges from 90 to 110.
CN201811500608.5A 2018-12-10 2018-12-10 Disease retina optical coherence tomography image simulation generation method Active CN109658466B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811500608.5A CN109658466B (en) 2018-12-10 2018-12-10 Disease retina optical coherence tomography image simulation generation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811500608.5A CN109658466B (en) 2018-12-10 2018-12-10 Disease retina optical coherence tomography image simulation generation method

Publications (2)

Publication Number Publication Date
CN109658466A CN109658466A (en) 2019-04-19
CN109658466B true CN109658466B (en) 2023-02-24

Family

ID=66113461

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811500608.5A Active CN109658466B (en) 2018-12-10 2018-12-10 Disease retina optical coherence tomography image simulation generation method

Country Status (1)

Country Link
CN (1) CN109658466B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110415182B (en) * 2019-06-18 2024-03-05 平安科技（深圳）有限公司 Fundus OCT image enhancement method, device, equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108495110A (en) * 2018-01-19 2018-09-04 天津大学 A kind of virtual visual point image generating method fighting network based on production

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10115194B2 (en) * 2015-04-06 2018-10-30 IDx, LLC Systems and methods for feature detection in retinal images

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108495110A (en) * 2018-01-19 2018-09-04 天津大学 A kind of virtual visual point image generating method fighting network based on production

Also Published As

Publication number Publication date
CN109658466A (en) 2019-04-19

Similar Documents

Publication Publication Date Title
Cheng et al. Sparse dissimilarity-constrained coding for glaucoma screening
US7992999B2 (en) Automated assessment of optic nerve head with spectral domain optical coherence tomography
CN102567734B (en) Specific value based retina thin blood vessel segmentation method
CN110010219A (en) Optical coherence tomography image retinopathy intelligent checking system and detection method
KR102198395B1 (en) Method and System for Early Diagnosis of Glaucoma and Displaying suspicious Area
CN112101424B (en) Method, device and equipment for generating retinopathy identification model
CN107862678A (en) A kind of eye fundus image reference-free quality evaluation method
CN110570421A (en) multitask fundus image classification method and apparatus
JP2019192215A (en) 3d quantitative analysis of retinal layers with deep learning
US11062444B2 (en) Artificial intelligence cataract analysis system
Khan et al. A new approach of cup to disk ratio based glaucoma detection using fundus images
CN113243887B (en) Intelligent diagnosis and treatment instrument for macular degeneration of old people
CN109658466B (en) Disease retina optical coherence tomography image simulation generation method
CN106446805B (en) A kind of eyeground shine in optic cup dividing method and system
Shaik et al. Glaucoma identification based on segmentation and fusion techniques
Chen et al. Automated diagnosis of age‐related macular degeneration using multi‐modal vertical plane feature fusion via deep learning
US20220122730A1 (en) Using artificial intelligence and biometric data for serial screening exams for medical conditions
CN117237711A (en) Bimodal fundus image classification method based on countermeasure learning
Purwanithami et al. Hemorrhage diabetic retinopathy detection based on fundus image using neural network and FCM segmentation
Sarangpure CUP and DISC OPTIC Segmentation Using Optimized Superpixel Classification for Glaucoma Screening
CN114048687A (en) Myopia and high myopia prediction model and application thereof
Singh et al. Performance analysis of machine learning techniques for glaucoma detection based on textural and intensity features
Karnowski et al. Retina image analysis and ocular telehealth: the Oak Ridge National Laboratory-Hamilton Eye Institute case study
Kuppusamy An artificial intelligence formulation and the investigation of glaucoma in color fundus images by using BAT algorithm
Pathan et al. Automated detection of pathological and non-pathological myopia using retinal features and dynamic ensemble of classifiers

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant