CN113362258A - Method, device, equipment and medium for denoising fundus color photograph image of cataract patient - Google Patents
Method, device, equipment and medium for denoising fundus color photograph image of cataract patient Download PDFInfo
- Publication number
- CN113362258A CN113362258A CN202110771823.4A CN202110771823A CN113362258A CN 113362258 A CN113362258 A CN 113362258A CN 202110771823 A CN202110771823 A CN 202110771823A CN 113362258 A CN113362258 A CN 113362258A
- Authority
- CN
- China
- Prior art keywords
- image
- denoising
- noise
- loss
- fundus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 208000002177 Cataract Diseases 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000005457 optimization Methods 0.000 claims abstract description 25
- 230000006870 function Effects 0.000 claims description 35
- 238000004088 simulation Methods 0.000 claims description 14
- 210000004220 fundus oculi Anatomy 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 3
- 239000002609 medium Substances 0.000 description 20
- 238000012549 training Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 235000002911 Salvia sclarea Nutrition 0.000 description 4
- 244000182022 Salvia sclarea Species 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 210000000695 crystalline len Anatomy 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000001356 surgical procedure Methods 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000012120 mounting media Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002980 postoperative effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Abstract
The embodiment of the invention discloses a method, a device, equipment and a medium for denoising an eye fundus color image of a cataract patient. The method comprises the following steps: acquiring a clear fundus image of a normal person and a real noise fundus image of a cataract patient; adding noise to the clear fundus image to obtain a simulated noise fundus image; constructing a generator and denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image; constructing a first discriminator, respectively inputting the combination of the simulated noise fundus image, the clear fundus image and the simulated de-noising image to obtain the loss of the first discriminator, and optimizing the generator according to the loss; denoising the real noise fundus image by using the optimized generator to obtain a real denoised image; constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain the loss of the second discriminator, and optimizing the generator according to the loss; and denoising the real noise fundus image according to the generator after the optimization again to obtain a final denoising result. The method realizes effective denoising of the eye fundus image of the cataract patient.
Description
Technical Field
The embodiment of the invention relates to the technical field of medical detection, in particular to a method, a device, equipment and a medium for denoising fundus color images of cataract patients.
Background
Cataracts are caused by a decrease in lens clarity and are the major cause of blindness worldwide. Because no effective medicine exists in the current clinical treatment of cataract, the operation is the only effective treatment means. However, cataract surgery can cause damage to the eye, and in the presence of other fundus diseases, both the risk of surgery and post-operative recovery can be affected. For clinical cases of cataract with fundus lesions, a targeted surgical plan needs to be made according to the condition of a patient to evaluate surgical risks and predict surgical effects. However, due to the opacity of the crystalline lens of the cataract patient, noise interference can be generated in the image shooting process, so that clear fundus color photographs are difficult to shoot, the evaluation accuracy rate of doctors on the health condition of the fundus of the patient is interfered, and the application of an artificial intelligent diagnosis algorithm to the fundus color photographs of the cataract patient is restricted. Therefore, the lower quality of the fundus color-photograph image of the cataract patient leads to heavy workload and higher misdiagnosis rate of doctors in fundus examination of the cataract patient, and the treatment effect of the cataract operation is seriously influenced.
At present, with the rapid development of computer vision technology, a plurality of effective image denoising algorithms are proposed to enhance the image quality, but such algorithms are based on abundant training data. Data pairs consisting of noise images and clear images with the same content in a natural scene can be obtained easily, so that an image denoising model can be trained effectively. However, for cataract patients, only a noise fundus image can be acquired before an operation, and a corresponding clear fundus image is lacked, so fundus image data for training an image denoising model cannot be obtained, the existing algorithm cannot be well applied, the accuracy of the denoising result of the fundus color image of the cataract patient is low at present, and valuable preoperative reference cannot be provided.
Disclosure of Invention
The embodiment of the invention provides a cataract patient fundus color photograph image denoising method, a device, equipment and a medium, which are used for reducing the dependence of a training image denoising model on clear cataract fundus image data, so that the denoising and image quality enhancement of the cataract patient fundus image are realized under the condition of lacking real denoising training data.
In a first aspect, an embodiment of the present invention provides a method for denoising a fundus color image of a cataract patient, where the method includes:
acquiring a clear fundus image of a normal person and a real noise fundus image of a cataract patient;
adding noise to the clear fundus image to obtain a simulated noise fundus image;
constructing a generator, and denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image;
constructing a first discriminator, and respectively inputting the combination of the clear fundus image and the simulated noise fundus image and the combination of the simulated de-noising image and the simulated noise fundus image into the first discriminator to obtain a first loss of the first discriminator so as to optimize the generator according to the first loss;
denoising the real noise fundus image by using the optimized generator to obtain a real denoised image;
constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain a second loss of the second discriminator, and optimizing the generator again according to the second loss;
and denoising the real noise fundus image according to the generator after the optimization again to obtain a final denoising result.
In a second aspect, an embodiment of the present invention further provides a device for denoising a fundus color image of a cataract patient, where the device includes:
the image acquisition module is used for acquiring a clear fundus image of a normal person and a real noise fundus image of a cataract patient;
the noise simulation module is used for adding noise to the clear fundus image to obtain a simulated noise fundus image;
the simulation denoising module is used for constructing a generator and denoising the simulation noise fundus image by using the generator to obtain a simulation denoising image;
a first optimization module, configured to construct a first discriminator, and input a combination of the clear fundus image and the simulated noise fundus image, and a combination of the simulated noise fundus image and the simulated noise fundus image to the first discriminator, respectively, to obtain a first loss of the first discriminator, so as to optimize the generator according to the first loss;
the real denoising module is used for denoising the real noise fundus image by using the optimized generator to obtain a real denoising image;
the second optimization module is used for constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain a second loss of the second discriminator, and optimizing the generator again according to the second loss;
and the final denoising module is used for denoising the real noise fundus image according to the generator after the optimization again so as to obtain a final denoising result.
In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method for denoising the fundus color image of the cataract patient provided by any embodiment of the invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the method for denoising a fundus color image of a cataract patient according to any embodiment of the present invention.
The embodiment of the invention provides a cataract patient fundus color photograph image denoising method, which comprises the steps of firstly obtaining a clear fundus image of a normal person and a real noise fundus image of a cataract patient, then adding noise into the clear fundus image to obtain a simulated noise fundus image, then constructing a generator, denoising the simulated noise fundus image by using the generator to obtain a simulated denoising image, then constructing a first discriminator to optimize the generator according to a first loss obtained by the clear fundus image, the simulated noise fundus image and the simulated denoising image of the first discriminator, so that the generator can better remove noise in the simulated noise fundus image, then denoising the real noise fundus image by using the optimized generator to obtain a real denoising image, and constructing a second discriminator to generate the generator according to a second loss obtained by the real denoising image and the simulated denoising image of the second discriminator And optimizing the image, and finally denoising the real noise fundus image according to the generator optimized again to obtain a final denoising result. The method for denoising the fundus color-image of the cataract patient provided by the embodiment of the invention realizes that the image denoising model is guided to be optimized under the condition that the clear fundus image of the real cataract patient does not exist, thereby realizing the denoising and image quality enhancement of the fundus image of the cataract patient and reducing the dependence of the training denoising model on the clear fundus image data of the cataract.
Drawings
Fig. 1 is a flowchart of a method for denoising a fundus color image of a cataract patient according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a cataract patient fundus color-photograph image denoising device according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Example one
Fig. 1 is a flowchart of a method for denoising a fundus color-image of a cataract patient according to an embodiment of the present invention. The embodiment can be applied to the situation that the denoising is performed only according to the real noise fundus image of the cataract patient before the operation to better establish the operation scheme, and the method can be executed by the cataract patient fundus color photograph image denoising device provided by the embodiment of the invention, the device can be realized by hardware and/or software, and can be generally integrated in computer equipment. As shown in fig. 1, the method specifically comprises the following steps:
and S11, acquiring a clear fundus image of a normal person and a real noise fundus image of the cataract patient.
Specifically, the method provided in this embodiment can enable the output of the real noise fundus image in the denoising model to learn the good performance of the output of the denoising model of the simulated noise fundus image obtained by adding noise to the clear fundus image through the domain adaptation, and then the clear fundus image of a normal person, that is, the clear fundus image of a person who does not suffer from cataract, can be used as source domain data in the domain adaptation, and a certain amount of clear fundus image samples can be stored in advance to be used in each application method, while the real noise fundus image of a cataract patient can be used as target domain data in the domain adaptation, and when a certain cataract patient needs to be preoperatively examined, the real noise fundus image of the patient can be acquired to perform subsequent steps.
And S12, adding noise to the clear fundus image to obtain a simulated noise fundus image.
Specifically, the real noise fundus image of the cataract patient can be simulated by adding noise into the clear fundus image, so that the denoising model obtained by training the simulated noise fundus image can be better suitable for processing the real noise fundus image. However, the simulated noise distribution in the simulated noise fundus image is not limited to be completely consistent with the real noise distribution in the real noise fundus image, and only certain similarity is required.
And S13, constructing a generator, and denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image.
Specifically, a generator, which may be a shared weight generator between the simulated noise fundus image and the true noise fundus image, may be constructed based on the image translation framework in the depth learning. After the construction of the generator is completed, the simulated noise fundus image can be firstly used for training, and then the generator can be firstly used for denoising the simulated noise fundus image to obtain a corresponding denoised simulated image.
S14, constructing a first discriminator, and respectively inputting the combination of the clear fundus image and the simulated noise fundus image and the combination of the simulated denoising image and the simulated noise fundus image into the first discriminator to obtain a first loss of the first discriminator so as to optimize the generator according to the first loss.
After the construction is completed, the clear fundus image, the simulated noise fundus image, the simulated denoising image and the simulated noise fundus image can be combined and input into the first discriminator, so that a first loss corresponding to the first discriminator can be obtained, the generator can be optimized according to the first loss, and the optimized generator can better remove cataract noise in the simulated noise fundus image.
Optionally, the first loss is:
LGAN＝E[logD1(s′,s)]+E[log(1-D1(s′,G(s′)))]
wherein L isGANRepresenting said first loss, s 'representing said simulated noise fundus image, s representing said clear fundus image, G (s') representing said simulated de-noised image, D1(s', s) represents the output of the first discriminator with the combination of the clear fundus image and the simulated noise fundus image as input, D1(s ', G (s')) represents the output of the first discriminator with the combination of the simulated denoised image and the simulated noise fundus image as input, and log () represents the logarithm, E [ deg. ]]Representing a mathematical expectation.
Further optionally, the optimizing the generator according to the first loss includes: determining a first loss function from the first loss to optimize the generator according to the first loss function. Specifically, a first loss function used for optimizing the generator may be defined according to the obtained first loss, so that the generator may be optimized according to a target value of the first loss function, and each time the optimization is completed, the generator may be reused to obtain a new simulated denoised image to obtain a new first loss, so as to iterate the optimization process, where the smaller the value of the first loss function is, the better the result is.
Further optionally, the first loss function is:
LG1＝LGAN+λLl1
wherein L isG1Representing said first loss function, LGANRepresents the first loss, and λ represents Ll1Weight of (1), Ll1Represents the L1 norm loss, and Ll1＝E[‖s-G(s′)‖1]S represents the clear fundus image, s' represents the clear fundus imageA simulated noise fundus image, G (s') representing the simulated de-noised image, | | | | | | luminance1Represents a 1-norm, E]Representing a mathematical expectation.
And S15, denoising the real noise fundus image by using the optimized generator to obtain a real denoised image.
Specifically, after the first optimization of the generator is completed, the real cataract data can be further introduced into the optimization process, so as to realize the denoising of the real noise fundus image. Specifically, the real noise fundus image of the target domain can be input into the optimized generator in the same way as the simulated noise fundus image of the source domain, so as to obtain a corresponding denoised real image.
S16, constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain a second loss of the second discriminator, and optimizing the generator again according to the second loss.
After the construction is completed, the real denoising image and the simulated denoising image can be respectively input into the second discriminator, so that a second loss corresponding to the second discriminator, namely, a difference between the source domain data and the target domain data can be obtained, and the generator can be optimized again according to the second loss, so that the generator can be guided to output denoising results in similar distribution, namely, better denoising results.
Optionally, the second loss is:
LD＝E[logD2(G(s′))]+E[log(1-D2(G(t)))]
wherein L isDRepresenting the second loss, s 'representing the simulated noise fundus image, G (s') representing the simulated de-noised image, t representing the real noise fundus image, G (t) representing the real de-noised image, D2(G (s')) represents the output of the second discriminator with the simulated denoised image as input, D2(G (t)) represents the true denoised image as inputThe output of the second discriminator, log () represents the logarithm, E [, ]]Representing a mathematical expectation.
Further optionally, the optimizing the generator again according to the second loss includes: and determining a second loss function according to the second loss so as to optimize the generator again according to the second loss function. Specifically, a second loss function used for optimizing the generator may be defined according to the obtained second loss, so that the generator may be optimized again according to a target value of the second loss function, and each time the optimization is completed, the generator may be reused to obtain a new simulated denoised image and a new real denoised image to obtain a new second loss, thereby iterating the optimization process, where the smaller the value of the second loss function is, the better the result is.
Further optionally, the second loss function is:
LG2＝LGAN+λ1Ll1+λ2LD
wherein L isG2Representing said second loss function, LGANRepresents said first loss, Ll1Represents the L1 norm loss, LDRepresents said second loss, λ1Represents Ll1Weight of (a), λ2Represents LDThe weight of (c).
And S17, denoising the real noise fundus image according to the generator after the optimization again to obtain a final denoising result.
Specifically, after the whole optimization process of the generator is completed, the generator can be used for denoising the real noise fundus image of the cataract patient needing preoperative examination, so that a final better denoising result is obtained.
The technical scheme provided by the embodiment of the invention comprises the steps of firstly obtaining a clear fundus image of a normal person and a real noise fundus image of a cataract patient, then adding noise into the clear fundus image to obtain a simulated noise fundus image, then constructing a generator, denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image, then constructing a first discriminator to optimize the generator according to a first loss obtained by the clear fundus image, the simulated noise fundus image and the simulated denoised image of the first discriminator so as to enable the generator to better remove the noise in the simulated noise fundus image, then denoising the real noise fundus image by using the optimized generator to obtain a real denoised image, constructing a second discriminator to optimize the generator again according to a second loss obtained by the real denoised image and the simulated denoised image of the second discriminator, and finally, denoising the real noise fundus image according to the generator optimized again to obtain a final denoising result. The method realizes that the image denoising model is guided to be optimized under the condition that the clear eye fundus image of the real cataract patient does not exist, thereby realizing the denoising and the image quality enhancement of the eye fundus image of the cataract patient and reducing the dependence of the training denoising model on the clear eye fundus image data of the cataract.
Example two
Fig. 2 is a schematic structural diagram of a cataract patient fundus color image denoising device according to a second embodiment of the present invention, which may be implemented by hardware and/or software, and may be generally integrated in a computer device, for executing the cataract patient fundus color image denoising method according to any embodiment of the present invention. As shown in fig. 2, the apparatus includes:
the image acquisition module 21 is used for acquiring clear fundus images of normal people and real noise fundus images of cataract patients;
a noise simulation module 22 for adding noise to the clear fundus image to obtain a simulated noise fundus image;
the simulation denoising module 23 is configured to construct a generator, and denoise the simulation noise fundus image by using the generator to obtain a simulation denoised image;
a first optimization module 24, configured to construct a first discriminator, and input a combination of the clear fundus image and the simulated noise fundus image, and a combination of the simulated noise fundus image and the simulated noise fundus image to the first discriminator, respectively, to obtain a first loss of the first discriminator, so as to optimize the generator according to the first loss;
a real denoising module 25, configured to denoise the real noise fundus image by using the optimized generator to obtain a real denoised image;
a second optimization module 26, configured to construct a second discriminator, and input the real denoised image and the simulated denoised image into the second discriminator to obtain a second loss of the second discriminator, so as to optimize the generator again according to the second loss;
and the final denoising module 27 is configured to denoise the real noise fundus image according to the generator after the optimization again, so as to obtain a final denoising result.
The technical scheme provided by the embodiment of the invention comprises the steps of firstly obtaining a clear fundus image of a normal person and a real noise fundus image of a cataract patient, then adding noise into the clear fundus image to obtain a simulated noise fundus image, then constructing a generator, denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image, then constructing a first discriminator to optimize the generator according to a first loss obtained by the clear fundus image, the simulated noise fundus image and the simulated denoised image of the first discriminator so as to enable the generator to better remove the noise in the simulated noise fundus image, then denoising the real noise fundus image by using the optimized generator to obtain a real denoised image, constructing a second discriminator to optimize the generator again according to a second loss obtained by the real denoised image and the simulated denoised image of the second discriminator, and finally, denoising the real noise fundus image according to the generator optimized again to obtain a final denoising result. The method realizes that the image denoising model is guided to be optimized under the condition that the clear eye fundus image of the real cataract patient does not exist, thereby realizing the denoising and the image quality enhancement of the eye fundus image of the cataract patient and reducing the dependence of the training denoising model on the clear eye fundus image data of the cataract.
On the basis of the above technical solution, optionally, the first loss is:
LGAN＝E[logD1(s′,s)]+E[log(1-D1(s′,G(s′)))]
wherein L isGANRepresenting said first loss, s 'representing said simulated noise fundus image, s representing said clear fundus image, G (s') representing said simulated de-noised image, D1(s', s) represents the output of the first discriminator with the combination of the clear fundus image and the simulated noise fundus image as input, D1(s ', G (s')) represents the output of the first discriminator with the combination of the simulated denoised image and the simulated noise fundus image as input, and log () represents the logarithm, E [ deg. ]]Representing a mathematical expectation.
On the basis of the above technical solution, optionally, the first optimization module 24 is specifically configured to:
determining a first loss function from the first loss to optimize the generator according to the first loss function.
On the basis of the above technical solution, optionally, the first loss function is:
LG1＝LGAN+λLl1
wherein L isG1Representing said first loss function, LGANRepresents the first loss, and λ represents Ll1Weight of (1), Ll1Represents the L1 norm loss, and Ll1＝E[‖s-G(s′)‖1]S represents the clear fundus image, s 'represents the simulated noise fundus image, G (s') represents the simulated de-noised image, | | | | survival1Represents a 1-norm, E]Representing a mathematical expectation.
On the basis of the above technical solution, optionally, the second loss is:
LD＝E[logD2(G(s′))]+E[log(1-D2(G(t)))]
wherein L isDRepresenting said second loss, s 'representing said simulated noise fundus image, G (s') representing said simulated noise fundus imageA noisy image, t representing the true noisy fundus image, G (t) representing the true denoised image, D2(G (s')) represents the output of the second discriminator with the simulated denoised image as input, D2(G (t)) represents the output of the second discriminator with the true denoised image as input, and log () represents the logarithm, E [ deg. ]]Representing a mathematical expectation.
On the basis of the above technical solution, optionally, the second optimization module 26 is specifically configured to:
and determining a second loss function according to the second loss so as to optimize the generator again according to the second loss function.
On the basis of the above technical solution, optionally, the second loss function is:
LG2＝LGAN+λ1Ll1+λ2LD
wherein L isG2Representing said second loss function, LGANRepresents said first loss, Ll1Represents the L1 norm loss, LDRepresents said second loss, λ1Represents Ll1Weight of (a), λ2Represents LDThe weight of (c).
The device for denoising the fundus color-image of the cataract patient provided by the embodiment of the invention can execute the method for denoising the fundus color-image of the cataract patient provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
It should be noted that, in the embodiment of the device for denoising a fundus color image of a cataract patient, the included units and modules are only divided according to functional logic, but are not limited to the above division as long as corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a computer device provided in the third embodiment of the present invention, and shows a block diagram of an exemplary computer device suitable for implementing the embodiment of the present invention. The computer device shown in fig. 3 is only an example, and should not bring any limitation to the function and the scope of use of the embodiments of the present invention. As shown in fig. 3, the computer apparatus includes a processor 31, a memory 32, an input device 33, and an output device 34; the number of the processors 31 in the computer device may be one or more, one processor 31 is taken as an example in fig. 3, the processor 31, the memory 32, the input device 33 and the output device 34 in the computer device may be connected by a bus or in other ways, and the connection by the bus is taken as an example in fig. 3.
The memory 32 is a computer readable storage medium, and can be used for storing software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the denoising method for fundus color images of cataract patients in the embodiment of the present invention (for example, the image acquisition module 21, the noise simulation module 22, the simulation denoising module 23, the first optimization module 24, the real denoising module 25, the second optimization module 26, and the final denoising module 27 in the denoising device for fundus color images of cataract patients). The processor 31 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 32, so as to implement the above-mentioned method for denoising fundus color images of cataract patients.
The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 32 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 32 may further include memory located remotely from the processor 31, which may be connected to a computer device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 33 can be used to acquire a clear fundus image of a normal person and a real noise fundus image of a cataract patient, and to generate key signal inputs and the like relating to user settings and function control of the computer apparatus. The output device 34 includes a display screen and the like, and can be used to display the final denoising result to the user.
Example four
The fourth embodiment of the present invention further provides a storage medium containing computer executable instructions, which when executed by a computer processor, is configured to perform a method for denoising a fundus oculi color photograph image of a cataract patient, the method including:
acquiring a clear fundus image of a normal person and a real noise fundus image of a cataract patient;
adding noise to the clear fundus image to obtain a simulated noise fundus image;
constructing a generator, and denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image;
constructing a first discriminator, and respectively inputting the combination of the clear fundus image and the simulated noise fundus image and the combination of the simulated de-noising image and the simulated noise fundus image into the first discriminator to obtain a first loss of the first discriminator so as to optimize the generator according to the first loss;
denoising the real noise fundus image by using the optimized generator to obtain a real denoised image;
constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain a second loss of the second discriminator, and optimizing the generator again according to the second loss;
and denoising the real noise fundus image according to the generator after the optimization again to obtain a final denoising result.
The storage medium may be any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the computer system in which the program is executed, or may be located in a different second computer system connected to the computer system through a network (such as the internet). The second computer system may provide the program instructions to the computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations, such as in different computer systems that are connected by a network. The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.
Of course, the embodiment of the present invention provides a storage medium containing computer executable instructions, and the computer executable instructions are not limited to the method operations described above, and can also perform related operations in the method for denoising a fundus color image of a cataract patient provided by any embodiment of the present invention.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A cataract patient eyeground color photograph image denoising method is characterized by comprising the following steps:
acquiring a clear fundus image of a normal person and a real noise fundus image of a cataract patient;
adding noise to the clear fundus image to obtain a simulated noise fundus image;
constructing a generator, and denoising the simulated noise fundus image by using the generator to obtain a simulated denoised image;
constructing a first discriminator, and respectively inputting the combination of the clear fundus image and the simulated noise fundus image and the combination of the simulated de-noising image and the simulated noise fundus image into the first discriminator to obtain a first loss of the first discriminator so as to optimize the generator according to the first loss;
denoising the real noise fundus image by using the optimized generator to obtain a real denoised image;
constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain a second loss of the second discriminator, and optimizing the generator again according to the second loss;
and denoising the real noise fundus image according to the generator after the optimization again to obtain a final denoising result.
2. The method for denoising a fundus oculi color-photographed image of a cataract patient according to claim 1, wherein the first loss is:
LGAN＝E[logD1(s′，s)]+E[log(1-D1(s′，G(s′)))]
wherein L isGANRepresenting said first loss, s 'representing said simulated noise fundus image, s representing said clear fundus image, G (s') representing said simulated de-noised image, D1(s', s) represents the output of the first discriminator with the combination of the clear fundus image and the simulated noise fundus image as input, D1(s ', G (s')) represents the output of the first discriminator with the combination of the simulated denoised image and the simulated noise fundus image as input, and log () represents the logarithm, E [ deg. ]]Representing a mathematical expectation.
3. The method for denoising fundus oculi color photographic image of cataract patient according to claim 1, wherein said optimizing the generator according to the first loss comprises:
determining a first loss function from the first loss to optimize the generator according to the first loss function.
4. The method for denoising a fundus oculi color-photographed image of a cataract patient according to claim 3, wherein the first loss function is:
LG1＝LGAN+λLl1
wherein L isG1Representing said first loss function, LGANRepresents the first loss, and λ represents Ll1Weight of (1), Ll1Represents the L1 norm loss, and Ll1＝E[‖s-G(s′)‖1]S represents the clear fundus image, s 'represents the simulated noise fundus image, G (s') represents the simulated de-noised image, | | | | survival1Represents a 1-norm, E]Representing a mathematical expectation.
5. The method for denoising a fundus oculi color-photographed image of a cataract patient according to claim 1, wherein the second loss is:
LD＝E[logD2(G(s′))]+E[log(1-D2(G(t)))]
wherein L isDRepresenting the second loss, s 'representing the simulated noise fundus image, G (s') representing the simulated de-noised image, t representing the real noise fundus image, G (t) representing the real de-noised image, D2(G (s')) represents the output of the second discriminator with the simulated denoised image as input, D2(G (t)) represents the output of the second discriminator with the true denoised image as input, and log () represents the logarithm, E [ deg. ]]Representing a mathematical expectation.
6. The method for denoising fundus oculi color photographic image of cataract patient according to claim 1, wherein said re-optimizing the generator according to the second loss comprises:
and determining a second loss function according to the second loss so as to optimize the generator again according to the second loss function.
7. The method for denoising a fundus oculi color-photographed image of a cataract patient according to claim 6, wherein the second loss function is:
LG2＝LGAN+λ1Ll1+λ2LD
wherein L isG2Representing said second loss function, LGANRepresents said first loss, Ll1Represents the L1 norm loss, LDRepresents said second loss, λ1Represents Ll1Weight of (a), λ2Represents LDThe weight of (c).
8. The utility model provides a cataract patient eye ground color photograph image denoising device which characterized in that includes:
the image acquisition module is used for acquiring a clear fundus image of a normal person and a real noise fundus image of a cataract patient;
the noise simulation module is used for adding noise to the clear fundus image to obtain a simulated noise fundus image;
the simulation denoising module is used for constructing a generator and denoising the simulation noise fundus image by using the generator to obtain a simulation denoising image;
a first optimization module, configured to construct a first discriminator, and input a combination of the clear fundus image and the simulated noise fundus image, and a combination of the simulated noise fundus image and the simulated noise fundus image to the first discriminator, respectively, to obtain a first loss of the first discriminator, so as to optimize the generator according to the first loss;
the real denoising module is used for denoising the real noise fundus image by using the optimized generator to obtain a real denoising image;
the second optimization module is used for constructing a second discriminator, inputting the real de-noised image and the simulated de-noised image into the second discriminator to obtain a second loss of the second discriminator, and optimizing the generator again according to the second loss;
and the final denoising module is used for denoising the real noise fundus image according to the generator after the optimization again so as to obtain a final denoising result.
9. A computer device, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the cataract patient fundus color image denoising method of any of claims 1-7.
10. A computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the method for denoising a fundus color image of a cataract patient according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110771823.4A CN113362258B (en) | 2021-07-08 | 2021-07-08 | Method, device, equipment and medium for denoising fundus color photograph image of cataract patient |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110771823.4A CN113362258B (en) | 2021-07-08 | 2021-07-08 | Method, device, equipment and medium for denoising fundus color photograph image of cataract patient |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113362258A true CN113362258A (en) | 2021-09-07 |
| CN113362258B CN113362258B (en) | 2022-10-21 |
Family
ID=77538905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110771823.4A Active CN113362258B (en) | 2021-07-08 | 2021-07-08 | Method, device, equipment and medium for denoising fundus color photograph image of cataract patient |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113362258B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115660985A (en) * | 2022-10-25 | 2023-01-31 | 中山大学中山眼科中心 | Cataract fundus image repairing method and repairing model training method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019024568A1 (en) * | 2017-08-02 | 2019-02-07 | 上海市第六人民医院 | Ocular fundus image processing method and apparatus, computer device, and storage medium |
| WO2019032481A1 (en) * | 2017-08-08 | 2019-02-14 | Siemens Aktiengesellschaft | Synthetic depth image generation from cad data using generative adversarial neural networks for enhancement |
| CN110390650A (en) * | 2019-07-23 | 2019-10-29 | 中南大学 | OCT image denoising method based on intensive connection and generation confrontation network |
| CN110428385A (en) * | 2019-08-09 | 2019-11-08 | 南京工业大学 | A kind of SD-OCT denoising method based on unsupervised confrontation neural network |
| CN111899185A (en) * | 2020-06-18 | 2020-11-06 | 深圳先进技术研究院 | Training method and device of image noise reduction model, electronic equipment and storage medium |
| CN112700390A (en) * | 2021-01-14 | 2021-04-23 | 汕头大学 | Cataract OCT image repairing method and system based on machine learning |
-
2021
- 2021-07-08 CN CN202110771823.4A patent/CN113362258B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019024568A1 (en) * | 2017-08-02 | 2019-02-07 | 上海市第六人民医院 | Ocular fundus image processing method and apparatus, computer device, and storage medium |
| WO2019032481A1 (en) * | 2017-08-08 | 2019-02-14 | Siemens Aktiengesellschaft | Synthetic depth image generation from cad data using generative adversarial neural networks for enhancement |
| CN110390650A (en) * | 2019-07-23 | 2019-10-29 | 中南大学 | OCT image denoising method based on intensive connection and generation confrontation network |
| CN110428385A (en) * | 2019-08-09 | 2019-11-08 | 南京工业大学 | A kind of SD-OCT denoising method based on unsupervised confrontation neural network |
| CN111899185A (en) * | 2020-06-18 | 2020-11-06 | 深圳先进技术研究院 | Training method and device of image noise reduction model, electronic equipment and storage medium |
| CN112700390A (en) * | 2021-01-14 | 2021-04-23 | 汕头大学 | Cataract OCT image repairing method and system based on machine learning |
Non-Patent Citations (2)
| Title |
|---|
| YUHAO LUO ET AL: "Dehaze of Cataractous Retinal Images Using an Unpaired Generative Adversarial Network", 《 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS》 * |
| 胡衍等: "基于深度学习OCT辅助诊断湿性年龄相关性黄斑变性算法的应用", 《中华实验眼科杂志》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115660985A (en) * | 2022-10-25 | 2023-01-31 | 中山大学中山眼科中心 | Cataract fundus image repairing method and repairing model training method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113362258B (en) | 2022-10-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111340819B (en) | Image segmentation method, device and storage medium | |
| Palma-Amestoy et al. | A perceptually inspired variational framework for color enhancement | |
| CN107563434B (en) | Brain MRI image classification method and device based on three-dimensional convolutional neural network | |
| CN108537801A (en) | Based on the retinal angiomatous image partition method for generating confrontation network | |
| CN111008943B (en) | Low-dose DR image noise reduction method and system | |
| CN112700390B (en) | Cataract OCT image repairing method and system based on machine learning | |
| Solís-Pérez et al. | Blood vessel detection based on fractional Hessian matrix with non-singular Mittag–Leffler Gaussian kernel | |
| CN109685814A (en) | Cholecystolithiasis ultrasound image full-automatic partition method based on MSPCNN | |
| WO2021190656A1 (en) | Method and apparatus for localizing center of macula in fundus image, server, and storage medium | |
| CN113362258B (en) | Method, device, equipment and medium for denoising fundus color photograph image of cataract patient | |
| CN114170118A (en) | Semi-supervised multi-mode nuclear magnetic resonance image synthesis method based on coarse-to-fine learning | |
| CN113822289A (en) | Training method, device and equipment of image noise reduction model and storage medium | |
| Lin et al. | A desmoking algorithm for endoscopic images based on improved U‐Net model | |
| CN116843036A (en) | Personalized federal learning method for Alzheimer disease auxiliary diagnosis | |
| CN110598652A (en) | Fundus data prediction method and device | |
| Mahapatra et al. | An optimal statistical feature‐based transformation function for enhancement of retinal images using adaptive enhanced leader particle swarm optimization | |
| Li et al. | Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement | |
| Li et al. | An FPGA-based edge computing and accelerating platform for fast diabetic retinopathy diagnosis | |
| Gu et al. | Effective methods of diabetic retinopathy detection based on deep convolutional neural networks | |
| CN112419169A (en) | CNN medical CT image denoising method based on noise prior | |
| Paul et al. | Computer-Aided Diagnosis Using Hybrid Technique for Fastened and Accurate Analysis of Tuberculosis Detection with Adaboost and Learning Vector Quantization | |
| Miao et al. | Spinal neoplasm image inpainting with deep convolutional neutral networks | |
| Parvathy et al. | Multimodality medical image fusion based on non-sub-sampled contourlet transform | |
| CN110889859A (en) | U-shaped network for fundus image blood vessel segmentation | |
| CN116883803B (en) | Image fusion method and system for glioma edge acquisition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |