CN112700390B

CN112700390B - Cataract OCT image repairing method and system based on machine learning

Info

Publication number: CN112700390B
Application number: CN202110046611.XA
Authority: CN
Inventors: 杨玮枫; 张叶叶; 刘希望
Original assignee: Shantou University
Current assignee: Shantou University
Priority date: 2021-01-14
Filing date: 2021-01-14
Publication date: 2022-04-26
Anticipated expiration: 2041-01-14
Also published as: CN112700390A

Abstract

The invention discloses a cataract OCT image repairing method and system based on machine learning, which introduces an optical information processing technology, changes the frequency spectrum of an object through an optical spatial filter, performs amplitude, phase or composite filtering on an input image, performs fuzzy processing on the image, realizes the simulation of a cataract OCT fuzzy image, adds a neutral density attenuation sheet to an OCT scanner lens to scan a healthy eyeball, obtains an eyeground OCT fuzzy image to simulate the cataract image, scans an OCT clear image of the same person without the attenuation sheet, restores the cataract retina OCT fuzzy image into a clear image, and the restored image can clearly see ten layers of retina structures; the number of network models is reduced to reduce workload and total training time, and the technology for restoring the blurred image clearly is realized by only using a Pix2Pix model.

Description

Cataract OCT image repairing method and system based on machine learning

Technical Field

The invention relates to the technical field of optical coherence tomography image processing, in particular to a cataract OCT image repairing method and system based on machine learning.

Background

Cataracts are one of the most common causes of visual impairment worldwide, with an estimated 1600 million people worldwide. The retinal imaging of cataract patients with a fundus oculi is a very challenging task because the light scattering caused by the turbid fundus media can seriously affect the imaging quality. The image is blurred, the contrast is low, so that doctors are difficult to evaluate the condition of the eyeground of cataract patients to further carry out effective treatment. Therefore, it is very important and clinically significant to develop a technology for restoring the blurred cataract retina image to be clear. At present, the traditional methods include adaptive histogram equalization based on contrast limitation and a brightness gain matrix method of HSV color space to perform color enhancement on a retinal image, and although the methods are beneficial to the contrast and brightness of the image, the methods have no relation with a degradation mechanism of the retinal image and are not directed at a cataractous retinal image. The development of artificial intelligence in the field of medical detection brings new power to a fully-loaded medical system, (1) a research team of Stanford university proposes a new method for diagnosing the probability of skin cancer according to skin images, and although the method is not the first algorithm capable of automatically identifying lesions, the method is the most stable and reliable method at present with the help of deep learning. (2) Deep learning based classification and target detection algorithms are also widely used in the medical field. Computer-aided automatic diagnosis opens up a new research direction for medical image analysis. The accumulation of the images of the dispersed light sources provides data support for automatic detection of the localization of the cataract focus of the child, so that accurate localization and identification of the cataract focus of the child based on a convolutional neural network computer algorithm become possible. The detection accuracy rate of the method reaches high standards in four categories of normal crystal shape, diseased crystal shape, focus compactness and focus non-compactness. (3) jiang et al proposed an automatic diagnosis system for cataract in children, which extracts the region of interest of crystalline lens by using Canny operator and hough transform, inputs it into CNN to extract high-level features, and classifies it by using SVM and SOFTMAX classifier. (4) ce zheng et al evaluated whether the generation of a countermeasure network would synthesize a realistic Optical Coherence Tomography (OCT) image as an educational image for an expert, and the results were also satisfactory. In conclusion, medical detection based on deep learning achieves good results. The classification, identification, prediction, synthesis and automatic diagnosis of cataract-eye fundus diseases are endless, but there is still a lack of technology for restoring the cataract-eye fundus blurred image to be clear for clinical application. This technique is even more important especially in areas and hospitals where medical facilities are in short supply and resources are scarce. The prior related art is that Yuhao Luo et al propose defogging processing on a cataract eye fundus image based on a deep learning unpaired generation countermeasure network. The processed image has higher structural similarity and fidelity with the image of the same patient after cataract operation. There are many drawbacks. Yuhao Luo et al propose defogging processing of cataract fundus images based on a deep learning unpaired generative countermeasure network. The technology is based on the concept of generating a countermeasure network (GAN), and two neural networks are designed: CataractSimGAN and CataractDehazeNet (see references: Yuhao Luo, Kun Chen, Lei Liu, Jiecheng Liu, Jianbo Mao, Genjie Ke and Mingzhai Sun, "haze of Cataracteus recovery Images using an unaided general adaptive Network", 2168-2194(c) IEEE 2020). The purpose of CataractSimGAN is to synthesize a cataract-like image from a clear retinal image and a cataract image without matching. CataractDehazeNet is a training of synthetic cataract-like images and corresponding clear images by supervised learning. A total of 400 retinal images without cataract and 400 blurred images of cataract patients were collected as training data sets. A test data set consisting of 50 cataract images and a clear image of the same patient after surgery. And the effect of the technology is evaluated by taking the postoperative clear image as a reference. The first step of the prior art: the clear image and the cataract image were combined into a blurred cataract-like image using cataracts simgan. The second step is that: and taking the generated fuzzy sample map and the original clear image as a training set to train the CataractDehazeNet defogging network. The third step: and obtaining an algorithm model for defogging treatment of the cataract eye fundus image based on the deep learning unpaired generation countermeasure network through continuous optimization. The fourth step: the same preoperative and postoperative image of the person is taken to test the model.

The disadvantages of the prior art are as follows:

(1) because the clear cataract images of the same person are not used for paired supervision training, the data set in the prior art adopts the method that the clear cataract images are synthesized into the cataract images as a fuzzy training set, the clear cataract images are used as labels, and the synthesized cataract images are not completely the same as the real cataract images although the cataract images have certain similarity, and are also not paired images before and after the operation of the same person, so that the result is influenced to a greater extent. This is a deficiency of the data set.

(2) The CataractDehazeNet network architecture is the same as the pix2pix network (Image-to-Image transformation with Conditional adaptive Networks), and the paired training data will work better according to the pix2pix model principle. This is a disadvantage in network architecture selection.

(3) The selected image data is a color photograph of the fundus, the main observations being blood vessels and optic discs. The fundus retinal structure is divided into substantially ten layers, which is difficult if the deeper layers of the retina need to be seen clearly.

Disclosure of Invention

The invention aims to provide a cataract OCT image repairing method and system based on machine learning, which are used for solving one or more technical problems in the prior art and providing at least one beneficial selection or creation condition.

The invention introduces an optical information processing technology, changes the frequency spectrum of an object through an optical spatial filter, performs amplitude, phase or composite filtering on an input image, and performs fuzzy processing on the image. The processing of the image by the innovative filter algorithm is conveniently researched by utilizing the computer technology. And the simulation of the cataract OCT fuzzy image is realized.

Retinal imaging of cataract patients with a ophthalmoscope is a very challenging task because light scattering due to turbid fundus media can severely affect the imaging quality. The principle of the neutral density attenuation sheet is that light with different wavelengths is contained in one beam of light, and after the light passes through the attenuation sheet, the different wavelengths are all attenuated according to the same proportion. Therefore, according to the optical principle, the neutral density attenuation sheet is added to the lens of the OCT scanner to scan healthy eyeballs, and the eyeground OCT fuzzy image is obtained to simulate a cataract image. And then scanning the OCT clear image of the same person without the attenuation sheet. And forming a training data set by corresponding the blurred image and the clear image one by one. The invention discloses a specific technical scheme for obtaining a clear image of the fundus of a cataract patient in order to restore a blurred image caused by light scattering caused by turbid fundus media of the cataract patient and apply the blurred image to clinic. And solves the drawbacks of the prior art.

In order to achieve the above object, according to an aspect of the present invention, there is provided a machine learning-based cataract OCT image repairing method, including the steps of:

s100, collecting an original image by using an OCT scanner and judging whether the image quality meets the standard or not; when the signal intensity of the OCT interference signal is greater than the preset signal intensity, the quality of the scanned image meets the standard; setting the value of the preset signal intensity to be 8;

s200, storing the image meeting the standard as a clear sample, adding an optical filter to an OCT scanner to acquire the same data source of the original image to obtain a fuzzy sample image so as to simulate a cataract fuzzy image, and forming a pair of clear and correspondingly fuzzy training data set and a pair of test data set; the test data set is used for checking the training effect of the training network;

wherein, a clear fundus image is obtained before the optical filter is added to the OCT scanner, and the optical filter is added to the OCT scanner to simulate a cataract image and form a training data pair with a clear healthy fundus image;

s300, training the improved Pix2Pix network model by using the obtained training data set, testing the model by using the test data set every time of training, and storing the trained network model; the Pix2Pix network model is a derivative network Pix2Pix network model based on GAN;

s400, forming a verification data set by using preoperative and postoperative images of a plurality of cataract patients, inputting the preoperative images into the trained network model for verifying the trained network model, and obtaining a generation result graph with clear restoration of the fuzzy OCT images of the cataract patients;

and S500, comparing the generated result image with the peak signal-to-noise ratio of the postoperative clear image to obtain a structural similarity and a residual image.

Comparing the generated result image with a postoperative clear image to compare the peak signal-to-noise ratio, the structural similarity and the residual image, wherein the smaller the phase difference is, the better the result is;

further, the thickness information of the generated image and the postoperative image is analyzed by using OCT image layering software.

Further, in S200, the method for obtaining the data source in the method for adding the optical filter to the OCT scanner to acquire the same data source can also be obtained by the following simulation: the OCT images before and after cataract surgery are Fourier transformed to a frequency domain, then the difference of the two images is compared, a clear fundus image of a healthy person is written to become a frequency domain processing algorithm of a cataract blurred image, then the OCT images of the healthy person are transformed to the frequency domain to be processed by the algorithm, and after the processing is finished, Fourier inverse transformation is carried out to generate a cataract disease simulation image as a data set; the specific process comprises the following steps:

let the image size of the pre-and post-operative cataract be M N, the DFT transform of the function pixel (x, y) is:

wherein u-0, 1,2., M-1, v-0, 1,2., N-1; where j is an imaginary unit, and M × N is the pixel size of the image (the number of rows and columns of pixels in the image matrix); u and v are frequency domain independent variables; p (u, v) refers to a frequency domain function of the image after Fourier transform; pixel (x, y) represents a function of image pixel values, representing an image;

the preoperative and postoperative cataract images are pre-collected preoperative and postoperative cataract images of a plurality of patients in cataract surgery;

firstly, transforming an image from a spatial domain to a frequency domain according to the above formula (1);

firstly, moving the low-frequency part of the image to the central position of the spectrogram, and then enabling:

u∈(h_mid+n,h_mid-n),v∈(w_mid+n,w_mid-n),P(u,v)＝0

h_midis the high of image 1/2; w is a_midIs the width of image 1/2; p (u, v) is a frequency domain function;

wherein n is an empirical value obtained after observing the spectrogram, if the severity of the disease is different, the range with different values n is obtained according to the empirical value (150,250), the ranges given by u and v are that k random values are randomly generated in the range, k is still obtained according to the severity of the disease, and the value range of k is (10000,20000), thus completing the computer simulation of randomly filtering part of the low frequency of the cataract image, and similarly, filtering part of the high frequency according to the principle;

scrambling the frequency spectrum in a local range, replacing the frequency domain numerical value with a position, exchanging the value of the (u + nn, v + nn) position with the frequency spectrum value at the (u, v) position to achieve the purpose of scrambling the frequency spectrum, wherein the method for scrambling the frequency spectrum in the local range comprises the following steps: p (u, v) ═ P_new(u+nn,v+nn)；

Wherein nn takes a random value, the value of nn is determined according to the actual simulation condition of a computer, and the value range of nn is (0,128);

reducing the image darkness;

the method for reducing the image darkness comprises the following steps: an image brightness average value is obtained, and the image brightness is set to be 0.8 times the brightness average value.

After the four steps are completed, the high frequency and the low frequency of the frequency spectrum are moved back to the original position from the central position, and the frequency spectrum value P after the exchange is obtained_new(u, v) performing inverse Fourier transform to obtain a simulated cataract OCT image pixel (x, y);

note: x is 0,1,2, M-1, y is 0,1,2, N-1.

Further, in S300, the Pix2Pix network model is a classical model applied to supervised image-to-image translation, and the network model is composed of a generator and a discriminator, and is an important application direction of GAN. Image-to-image translation is a process of obtaining a desired output image based on an input image, and can also be regarded as a mapping between images. The method includes the steps of guiding image generation by adding conditions and characteristic information, learning mapping between an input image and an output image, and optimizing the generated image by using a loss function and an optimization function to obtain a specified image.

Further, in S300, when training the network, firstly inputting a blurred image x and a clear real image Y corresponding to the image, taking x as an input of a generator G to obtain a generated image G (x), merging G (x) and x based on a channel dimension as an input of a discriminator D, then outputting a predicted value by the discriminator D, where the range of the predicted value is [0,1], 0 represents that the probability of judging the image as the clear real image is 0, and 1 represents that the probability of judging the image as the clear real image is 1, and if the output predicted value is closer to 1 in the range of [0,1], the probability value that the generated image is judged as the clear real image by the discriminator is larger, the discriminator judgment error is indicated, and the discriminator parameter is adjusted and optimized according to a loss function, so that the discriminator can distinguish that the generated image is not the clear real image; if the output prediction value is closer to 0 in the range of [0,1], the image generated by the generator does not achieve the effect of a clear real image, and the judgment of the discriminator is not disturbed by false or true, the generator is optimized, so that the generated image is consistent with the clear real image, and finally the discriminator cannot discriminate whether the generated image is true or false; in addition, the clear real image Y and the input image x are combined together based on the channel dimension to be used as the input of the discriminator to obtain a predicted value, the output predicted value at the moment is close to 1 in the range of [0,1], if the output predicted value is close to 0, the discriminator judges the clear real image wrongly, and then the discriminator is optimized.

Further, in S300, the training target of the network discriminator D is that the output probability value is close to 0 in the range of [0,1] when the x-blurred image and the generated image g (x) are input, and the output probability value is close to 1 in the range of [0,1] when the x-blurred image and the clear real image Y are input; the training goal of generator G is to make the probability value of the output of discriminator D close to 1 in the range of [0,1] when G (x) and x are generated as the input of discriminator D.

Further, in S300, the training goal of the network discriminator D is to output a small probability value (for example, 0 at minimum) when the input is not a pair of real images (x and g (x)), and output a large probability value (for example, 1 at maximum) when the input is a pair of real images (x and y). The training goal of generator G is to make the probability value of the output of discriminator D close to 1 in the range of [0,1] when G (x) and x are generated as the input of discriminator D.

Further, in S300, the loss function of the network is an index for evaluating the network model, and may also be regarded as applying a constraint to the network model, the goal of training the network is to make the loss function reach a global minimum, in the untrained neural network, the weights of the neurons receiving the input of the previous layer are initialized randomly, so that the random parameters cannot make the neural network reach the assumed function, a suitable objective function needs to be set to constrain the optimization of the neural network, and the objective function obtains an extremum by combining the optimization process of the gradient descent method, in the process of supervised training, the error between the output obtained by each training step and the real sample is reversely propagated to the neurons at each level, and the weights at each level are modified, so that the final convergence of the neural network reaches the expected effect of the design. The objective function of the improved Pix2Pix network model is as follows:

wherein λ₁And λ₂Weighting factors, λ, for the L1 distance and edge loss, respectively₁Has a value range of (82,118) and lambda₂The value range of (1.1) is (0.8) to ensure the stability and convergence of the optimization process; the purpose of the generator G is to approximate the self-generated image to the real image, i.e. the larger the value of D (G (x)), the better, at this time

Will be smaller, so G in the formula is the smallest, and the purpose of the discriminator D is to discriminate between sharp real image and blurred image, where D (x) should be larger and D (G (x)) should be smaller, where D (x) should be smaller

Will become larger, so D in the formula is taken to be the maximum; in that

In (1), arg is the English abbreviation for argument (i.e., argument);

i.e. the value of x when G takes the minimum value,

i.e. the value of x when D takes the maximum value.

Firstly, in order to solve the problems of clear image restoration and clear edge preservation, edge loss sensitive to edge information is introduced into an objective function:

wherein i, j represent the lateral and longitudinal coordinates of the image in equation (3), i.e., G (x)_i,jIndicating the magnitude of the pixel value, y, of the ith row and jth column of the generated image_i,jRepresenting the size of the pixel value of the ith row and the jth column of the real image; e_x,yRepresents a mathematical expectation;

L_cGAN(G,D)＝E_x,y[logD(x,y)]+E_x[log(1-D(x,G(x)))] (4)；

first term E of formula (4)_x,y[logD(x,y)]The probability value that the finger discriminator judges the real image to be true, the second item E_x[log(1-D(x,G(x))]The finger discriminator judges the probability value of the generated image as a real image; e_x,yRepresents a mathematical expectation; e_xRepresents a mathematical expectation;

L_L1(G)＝E_x,y[||y-G(x)||₁] (5)；

equation (5) to make the generated image closer to the standard, an L1 distance constraint is introduced in the objective function;

wherein y-G (x) refers to the distance between the real image and the generated image at the pixel point<1；||y-G(x)||₁The modulus of each component of the vector;

the formula (2) is composed of a GAN loss function of the formula (4), an L1 distance constraint loss function of the formula (5) and an edge loss of the formula (3), wherein lambda is an empirical parameter, and the front term and the rear term in the formula (2) are equal in magnitude around 100; wherein x is an input image x, namely a fuzzy fundus OCT image processed by the optical filter; y is a real image Y, namely a clear fundus OCT image which is not processed by the optical filter; g is a generator; d (x, y) is a discriminator, abbreviated as D; g (x) is a generation image, namely, the input x is used for generating an image G (x) by a generator G.

An Adam optimization method is adopted in the pix2pix model. Adam, a first-order Optimization algorithm that can replace the traditional Stochastic gradient descent process, can iteratively update neural network weights based on training data, was proposed by Diederik Kingma by OpenAI and Jimmy Ba by Toronto university in the ICLR paper filed 2015 (Adam: A Method for Stochartic Optimization).

The invention also provides a cataract OCT image repairing system based on machine learning, which comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to run in the units of the following system:

the original image acquisition unit is used for acquiring an original image by using the OCT scanner and judging whether the image quality meets the standard or not;

the data set dividing unit is used for storing the image which meets the standard as a clear sample, and adding an optical filter to the OCT scanner to acquire the same data source of the original image to obtain a fuzzy sample image to form a training data set and a test data set which are paired;

the network training unit is used for training the derived network Pix2Pix network model based on the GAN by using the obtained training data set, testing the model once per training and storing the trained optimal model;

the model verification unit is used for forming a verification data set by using the preoperative and postoperative images of the cataract patient to verify the model, and obtaining a generation result graph with clear restoration of the fuzzy OCT image of the cataract patient;

the image comparison unit is used for comparing the generated result image with a postoperative clear image and calculating the peak signal-to-noise ratio, the structural similarity and the residual image of the two images through MATLAB software;

and the image analysis unit is used for analyzing the thickness information of the generated image and the postoperative image by utilizing OCT image layering software.

The invention has the beneficial effects that: the invention provides a cataract OCT image repairing method and system based on machine learning, which restores a clear image of a cataract retina OCT blurred image, and the restored image can clearly see ten layers of retina structures and is applied to clinic; the number of network models is reduced to reduce workload and total training time, and the technology of restoring the blurred image clearly is realized only by using a Pix2Pix model; the problem that data are unpaired is solved, and the problem that the paired training is difficult due to the fact that the amount of data is small is solved by using the blurred image attenuated by the neutral density attenuation sheet and the clear image of the same person as a training set; the model trained by the data source obtained through simulation has strong generalization performance, and the problem that the position where the optical filter is added to acquire the original image is difficult to correspond is solved.

Drawings

The above and other features of the present invention will become more apparent by describing in detail embodiments thereof with reference to the attached drawings in which like reference numerals designate the same or similar elements, it being apparent that the drawings in the following description are merely exemplary of the present invention and other drawings can be obtained by those skilled in the art without inventive effort, wherein:

FIG. 1 is a flow chart of a method for repairing cataract OCT image based on machine learning;

FIG. 2 is a comparison of the use of optical filters to capture an original image as an input image, a real image, and a generated sharp image;

FIG. 3 is a comparison of a simulated image of a frequency domain algorithm as an input image, a real image and a generated sharp image;

fig. 4 is a structural diagram of a cataract OCT image repairing system based on machine learning.

Detailed Description

The conception, the specific structure and the technical effects of the present invention will be clearly and completely described in conjunction with the embodiments and the accompanying drawings to fully understand the objects, the schemes and the effects of the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

Fig. 1 is a flow chart of a method for repairing a machine learning-based OCT image of a cataract according to the present invention, and the method for repairing a machine learning-based OCT image of a cataract according to an embodiment of the present invention is described below with reference to fig. 1.

The invention provides a cataract OCT image repairing method based on machine learning, which specifically comprises the following steps:

s200, storing the image meeting the standard as a clear sample, and adding an optical filter to an OCT scanner to acquire the same data source of the original image to obtain a fuzzy sample image to form a training data set and a test data set in pair;

s300, training the improved Pix2Pix network model by using the obtained training data set, testing the model once per training, and storing the trained network model; the Pix2Pix network model is a derivative network Pix2Pix network model based on GAN;

s400, forming a verification data set by using preoperative and postoperative images of a plurality of cataract patients to verify the trained network model, and obtaining a generation result graph with clear restoration of the fuzzy OCT images of the cataract patients;

s500, comparing the generated result image with the peak signal-to-noise ratio of the postoperative clear image to obtain a structural similarity and a residual image;

the generated result image is compared with a postoperative clear image to compare the peak signal-to-noise ratio, the structural similarity and the residual image are different, and the result is better when the difference is smaller.

Further, in S200, the method of applying the optical filter to the OCT scanner to acquire the same data source is: the method comprises the steps of utilizing a computer technology to perform Fourier transform on OCT images before and after cataract surgery to a frequency domain, then comparing the difference of the two images, making corresponding algorithm design, processing the OCT frequency domain images of healthy people, and then performing Fourier inverse transform to generate a cataract disease simulation image as a data set; the specific process comprises the following steps:

let the DFT transform of the function pixel (x, y) of image size M × N be:

wherein u-0, 1,2., M-1, v-0, 1,2., N-1;

transforming the image from a space domain to a frequency domain and then carrying out algorithm processing on the image:

the low frequency part of the image is first moved to the center of the spectrogram. Then order

i∈(h_mid+n,h_mid-n),j∈(w_mid+n,w_mid-n),P(i,j)＝0

Note: n is an empirical value obtained after observing the spectrogram, the values are different according to different severity of diseases, i and j mean that k random values are randomly generated in the range, and k is still the empirical value according to different severity of diseases, so that the simulation of randomly filtering out partial low frequency of the cataract image by the computer is completed, and similarly, the filtering out partial high frequency is also according to the principle;

disturbing the spectrum in a local range, i.e. P (u, v) ═ P_new(u+nn,v+nn)；

Note: nn taking a random value, and determining the value according to the actual simulation condition of the computer;

reducing the image darkness;

after the four steps are completed, the high frequency and the low frequency of the frequency spectrum are moved back to the original position from the central position to obtain P_new(u, v) performing inverse Fourier transform to obtain a simulated cataract OCT image, such as pixel (x, y);

note: x is 0,1,2, M-1, y is 0,1,2, N-1;

Further, in S300, when training the network, firstly inputting an image x and a real image Y corresponding to the image, taking x as an input of a generator G to obtain a generated image G (x), merging G (x) and x based on a channel dimension as an input of a discriminator D, then outputting a predicted value by the discriminator D, if a judgment value is close to 1, judging that the image is real, indicating that the discriminator judges wrongly, and adjusting and optimizing parameters of the discriminator according to a loss function; if the image is close to 0, judging that the image is false, optimizing the generator, and enabling the generated image to be consistent with the real image to the maximum extent; and merging the real image Y and the input image x based on the channel dimension as the input of the discriminator to obtain a predicted value. The judgment at this time is made to be 1 as much as possible. If not, the arbiter is optimized.

Further, in S300, the loss function of the network is an index for evaluating the network model, and may also be regarded as applying a constraint to the network model, the goal of training the network is to make the loss function reach a global minimum, in the untrained neural network, the weights of the neurons receiving the input of the previous layer are initialized randomly, so that the random parameters cannot make the neural network reach the assumed function, a suitable objective function needs to be set to constrain the optimization of the neural network, and the objective function obtains an extremum by combining the optimization process of the gradient descent method, in the process of supervised training, the error between the output obtained by each training step and the real sample is reversely propagated to the neurons at each level, and the weights at each level are modified, so that the final convergence of the neural network reaches the expected effect of the design. The objective function of the pix2pix model is as follows:

L_cGAN(G,D)＝E_x,y[logD(x,y)]+E_x[log(1-D(x,G(x))] (2)

L_L1(G)＝E_x,y[‖y-G(x)‖₁] (3)

the formula (1) is composed of a GAN loss function of the formula (2) and an L1 loss function of the formula (3), wherein lambda is an empirical parameter, and the front and back orders in the formula (1) are equal in magnitude by taking about 100; wherein x is an input image x, namely a fuzzy fundus OCT image processed by the optical filter; y is a real image Y, namely a clear fundus OCT image which is not processed by the optical filter; g is a generator; d is a discriminator; g (x): an image is generated.

The healthy fundus images are processed by the computer technology, fuzzy cataract OCT images are simulated, and paired data sets of healthy corresponding diseases are formed. As another group of data set of the model is used for training the model, the model is stored after the training is finished, and OCT images before and after cataract surgery are used for verifying the model, so that a good result is obtained. The problem of insufficient data set in the existing research is solved, the algorithm processing method is continuously optimized and perfected, and the currently obtained result is shown in fig. 2 and fig. 3, wherein fig. 2 is a comparison graph of an original image collected by adding a filter as an input image x, a real image Y and a generated clear image G (x); fig. 3 shows a comparison of a simulation image of a frequency domain algorithm as an input image x, a real image Y and a generated clear image g (x).

An embodiment of the present invention provides a machine learning-based cataract OCT image restoration system, and as shown in fig. 4, is a structural diagram of the machine learning-based cataract OCT image restoration system of the present invention, and the machine learning-based cataract OCT image restoration system of the embodiment includes: a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program realizes the steps of the above-mentioned embodiment of the system for repairing cataract OCT image based on machine learning.

The system comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to run in the units of the following system:

The cataract OCT image repairing system based on machine learning can be operated in computing equipment such as a desktop computer, a notebook computer, a palm computer and a cloud server. The system for repairing the cataract OCT image based on machine learning can be operated by comprising but not limited to a processor and a memory. Those skilled in the art will appreciate that the example is merely illustrative of a machine learning based cataract OCT image repair system and does not constitute a limitation of a machine learning based cataract OCT image repair system, and may include more or less components than, or in combination with, certain components, or different components, for example, the machine learning based cataract OCT image repair system may also include input and output devices, network access devices, buses, etc.

The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. The general purpose processor can be a microprocessor or the processor can be any conventional processor or the like, the processor is a control center of the operation system of the cataract OCT image repairing system based on machine learning, and various interfaces and lines are utilized to connect various parts of the operation system of the whole cataract OCT image repairing system based on machine learning.

The memory can be used for storing the computer program and/or module, and the processor can realize various functions of the cataract OCT image repairing system based on machine learning by operating or executing the computer program and/or module stored in the memory and calling the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

Although the present invention has been described in considerable detail and with reference to certain illustrated embodiments, it is not intended to be limited to any such details or embodiments or any particular embodiment, so as to effectively encompass the intended scope of the invention. Furthermore, the foregoing describes the invention in terms of embodiments foreseen by the inventor for which an enabling description was available, notwithstanding that insubstantial modifications of the invention, not presently foreseen, may nonetheless represent equivalent modifications thereto.

Claims

1. A cataract OCT image repairing method based on machine learning is characterized by comprising the following steps:

s100, collecting an original image by using an OCT scanner and judging whether the image quality meets the standard or not; when the signal intensity of the OCT interference signal is greater than the preset signal intensity, the quality of the scanned image meets the standard;

s200, storing the images meeting the standard as clear samples, performing Fourier transform on OCT images before and after cataract surgery to a frequency domain, comparing the difference of the two images, writing out a frequency domain processing algorithm for changing a clear fundus image of a healthy person into a cataract fuzzy image, then performing algorithm processing on the OCT images of the healthy person by transforming the OCT images to the frequency domain, performing inverse Fourier transform after the processing is finished to generate a cataract disease simulation image as a data set, and forming a pair of clear and correspondingly fuzzy training data set and a test data set;

s300, training the improved Pix2Pix network model by using the obtained training data set, testing the model by using the test data set every time of training, and storing the trained network model;

in S200, the OCT images before and after cataract surgery are Fourier transformed to a frequency domain, then the difference of the two images is compared, a frequency domain processing algorithm that a clear fundus image of a healthy person is changed into a cataract blurred image is written out, then the OCT images of the healthy person are transformed to the frequency domain for algorithm processing, and after the processing is finished, Fourier inverse transformation is carried out to generate a cataract disease simulation image as a data set, and the method comprises the following steps:

let the DFT transform of the function pixel (x, y) of image size M × N be:

wherein u-0, 1,2., M-1, v-0, 1,2., N-1; wherein j is an imaginary unit, and M × N is the pixel size of the image; u and v are frequency domain independent variables; p (u, v) refers to a frequency domain function of the image after Fourier transform; pixel (x, y) represents a function of image pixel values, representing an image;

firstly, transforming an image from a spatial domain to a frequency domain according to equation (1);

then, the low-frequency part of the image is moved to the center position of the spectrogram, and then:

u∈(h_mid+n,h_mid-n),v∈(w_mid+n,w_mid-n),P(u,v)＝0

n is an empirical value obtained after observing the spectrogram, the range given by u and v means that k random values are randomly generated in the range, k is still obtained according to different severity degrees of diseases, the computer simulation of randomly filtering part of low frequency of the cataract image is completed, and the filtered part of high frequency can be obtained in the same way;

scrambling the frequency spectrum in a local range, and exchanging the frequency domain value for the position, so as to achieve the purpose of scrambling the frequency spectrum, exchanging the value of the (u + nn, v + nn) position with the frequency spectrum value at the (u, v) position, namely P (u, v) ═ P_new(u+nn,v+nn)；

nn takes a random value, the value of nn is determined according to the actual simulation condition of a computer, and the value range of nn is (0,128);

reducing the image darkness;

the method for reducing the image darkness comprises the following steps: acquiring an image brightness average value, and setting the image brightness to be 0.8 times of the brightness average value;

the high frequency and the low frequency of the frequency spectrum are moved back to the original position from the central position to obtain an exchanged frequency spectrum value P_new(u, v) redo

The inverse transformation of the inner leaves is used for obtaining a simulated cataract OCT image pixel (x, y);

2. the method for repairing a cataract OCT (optical coherence tomography) image based on machine learning as claimed in claim 1, wherein in S300, the Pix2Pix network model is a classical model applied to supervised image-to-image translation, and is composed of a generator and a discriminator, and the image-to-image translation is a process of obtaining a desired output image based on an input image, and can also be regarded as a mapping between images, which is a process of guiding image generation by adding condition and feature information, learning the mapping between the input image and the output image, and optimizing the generated image by using a loss function and an optimization function, so as to obtain a specified image.

3. The method for repairing a cataract OCT image based on machine learning of claim 2, wherein in S300, when training a network, firstly inputting a blurred image x and a clear real image Y corresponding to the image, taking x as the input of a generator G to obtain a generated image G (x), combining G (x) and x based on a channel dimension as the input of a discriminator D, then outputting a predicted value by the discriminator D, wherein the predicted value is in the range of [0,1], 0 represents that the probability of judging the image as the clear real image is 0,1 represents that the probability of judging the image as the clear real image is 1, and if the output predicted value is closer to 1 in the range of [0,1], the probability value of judging the generated image as the clear real image by the discriminator is larger, the discriminator judges wrongly, and the discriminator parameter is adjusted and optimized according to a loss function, the discriminator can distinguish that the generated image is not a clear real image; if the output prediction value is closer to 0 in the range of [0,1], the image generated by the generator does not achieve the effect of a clear real image, and the judgment of the discriminator is not disturbed by false or true, the generator is optimized, the generated image is consistent with the clear real image as much as possible, and finally the discriminator cannot discriminate whether the generated image is true or false; in addition, the clear real image Y and the input image x are combined together based on the channel dimension to be used as the input of the discriminator to obtain a predicted value, the output predicted value at the moment is close to 1 in the range of [0,1], if the output predicted value is close to 0, the discriminator judges the clear real image wrongly, and then the discriminator is optimized.

4. The method for repairing cataract OCT image based on machine learning as claimed in claim 3, wherein in S300, the training goal of the network discriminant D is to output a probability value close to 0 in the range of [0,1] when inputting x blurred image and the generated image G (x), and output a probability value close to 1 in the range of [0,1] when inputting x blurred image and the clear real image Y; the training goal of generator G is to make the probability value of the output of discriminator D close to 1 in the range of [0,1] when G (x) and x are generated as the input of discriminator D.

5. The method for repairing an OCT image of a cataract based on machine learning of claim 4, wherein in S300, the loss function of the network is an index for evaluating the network model, which can also be regarded as applying a constraint to the network model, the goal of training the network is to make the loss function reach a global minimum, in the untrained neural network, the weight of each neuron receiving the input of the previous layer is initialized randomly, such that the random parameter does not make the neural network reach the assumed function, a suitable objective function needs to be set to constrain the optimization of the neural network, and the optimization process combined with the gradient descent method makes the objective function obtain an extreme value, in the process of supervised training, the error between the output obtained by each training step and the real sample is propagated back to each neuron, and each weight is modified, so that the neural network finally converges to achieve the designed expected effect, the objective function of the improved Pix2Pix network model is as follows:

wherein λ₁And λ₂Weighting factors, λ, for the L1 distance and edge loss, respectively₁Has a value range of (82,118) and lambda₂The value range of (1.1) is (0.8) to ensure the stability and convergence of the optimization process; the purpose of the generator G is to make the self-generated image clear, i.e. the larger the value of D (G (x)), the better, at this time

Will be smaller, so G in the formula is the minimum, and the discriminator D is the maximum; in that

In the above, arg is an English abbreviation of argument;

i.e. the value of x when G takes the minimum value,

namely the value of x when the maximum value of D is taken;

wherein i, j represents the coordinates of the horizontal and vertical directions of the image in formula (3), i.e., G (x)_i,jIndicating the magnitude of the pixel value, y, of the ith row and jth column of the generated image_i,jRepresenting the size of the pixel value of the ith row and the jth column of the real image; e_x,yRepresents a mathematical expectation;

L_cGAN(G,D)＝E_x,y[log D(x,y)]+E_x[log(1-D(x,G(x)))] (4)；

the first item in the formula (4) is a probability value that the discriminator judges that the real image is true, and the second item is a probability value that the discriminator judges that the generated image is the real image; e_x,yRepresents a mathematical expectation; e_xRepresents a mathematical expectation;

L_L1(G)＝E_x,y[||y-G(x)||₁] (5)；

equation (2) is composed of the GAN loss function of equation (4), the L1 distance constraint loss function of equation (5), and the edge loss of equation (3), where x is the input image x, i.e., the blurred fundus OCT image after filter processing; y is a real image Y, namely a clear fundus OCT image which is not processed by the optical filter; g is a generator; d (x, y) is a discriminator, abbreviated as D; g (x) is a generated image.

6. A system for machine learning-based OCT image repair of cataracts, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to run in the units of the following system:

the data set dividing unit is used for storing standard-conforming images as clear samples, Fourier transforming OCT images before and after cataract surgery to a frequency domain, comparing the difference of the two images, writing out a frequency domain processing algorithm for changing a clear fundus image of a healthy person into a cataract blurred image, then transforming the OCT images of the healthy person to the frequency domain for algorithm processing, and performing inverse Fourier transformation after the processing is finished to generate a cataract disease simulation image as a data set to form a pair of clear and correspondingly blurred training data set and a test data set;

the image analysis unit is used for analyzing the thickness information of the generated image and the postoperative image by using OCT image layering software;

the method comprises the following steps of Fourier transforming OCT images before and after cataract surgery to a frequency domain, then comparing the difference of the two images, writing out a frequency domain processing algorithm for changing a clear fundus image of a healthy person into a cataract blurred image, then transforming the OCT images of the healthy person to the frequency domain for algorithm processing, and then performing Fourier inverse transformation to generate a cataract disease simulation image as a data set after the processing is finished, wherein the method comprises the following steps:

let the DFT transform of the function pixel (x, y) of image size M × N be:

u∈(h_mid+n,h_mid-n),v∈(w_mid+n,w_mid-n),P(u,v)＝0

reducing the image darkness;

the high frequency and the low frequency of the frequency spectrum are moved back to the original position from the central position to obtain an exchanged frequency spectrum value P_new(u, v) performing inverse Fourier transform to obtain a simulated cataract OCT image pixel (x, y);