CN117314911A - Method, device and storage medium for optimizing eye medical image - Google Patents
Method, device and storage medium for optimizing eye medical image Download PDFInfo
- Publication number
- CN117314911A CN117314911A CN202311606025.1A CN202311606025A CN117314911A CN 117314911 A CN117314911 A CN 117314911A CN 202311606025 A CN202311606025 A CN 202311606025A CN 117314911 A CN117314911 A CN 117314911A
- Authority
- CN
- China
- Prior art keywords
- image
- optimizing
- eye
- medical image
- ophthalmic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012545 processing Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims abstract description 9
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 7
- 210000004175 meibomian gland Anatomy 0.000 claims description 27
- 239000011159 matrix material Substances 0.000 claims description 22
- 238000003708 edge detection Methods 0.000 claims description 13
- 239000000284 extract Substances 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000002401 inhibitory effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 11
- 230000008439 repair process Effects 0.000 abstract description 2
- 230000006872 improvement Effects 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 238000003672 processing method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000009977 dual effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000001629 suppression Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 210000000744 eyelid Anatomy 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000008485 antagonism Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 210000004907 gland Anatomy 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 150000002632 lipids Chemical class 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 208000005494 xerophthalmia Diseases 0.000 description 2
- 208000027691 Conduct disease Diseases 0.000 description 1
- 235000002911 Salvia sclarea Nutrition 0.000 description 1
- 244000182022 Salvia sclarea Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000001704 evaporation Methods 0.000 description 1
- 239000004519 grease Substances 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000028327 secretion Effects 0.000 description 1
- 230000004488 tear evaporation Effects 0.000 description 1
- 238000009736 wetting Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/094—Adversarial learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method, a device and a storage medium for optimizing an eye medical image, which relate to the technical field of image processing and comprise the following steps: acquiring an eye image; detecting an area with a specular reflection phenomenon, and determining a reflection area; generating a mask according to the reflection area; generating an image of the mask coverage area by using a cyclic generation countermeasure network, namely generating an image; reserving the part outside the mask of the eye image, and fusing the part with the generated image to form an optimized image; the loop generation countermeasure network is provided with a double-discriminator module consisting of a local discriminator and a global discriminator, and a plurality of areas of an input image are enhanced to different degrees through the cooperation of the double-discriminator module and a generator. The invention can accurately identify and process the reflective area in the eye medical image, and repair the portion covered by the reflective area into the image which is as close to the reality as possible through the cyclic countermeasure generation network, thereby improving the detection precision of the eye medical image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a device and a storage medium for optimizing an eye medical image.
Background
In the ophthalmic medical program, it is an essential link to conduct disease diagnosis (detection) by acquiring medical images of a target site, for example, in the meibomian gland detection program: the meibomian glands are buried inside the eyelid, and the openings are located at the eyelid margin, so that the meibomian glands secrete various lipid components to form a lipid layer, thereby preventing excessive tear evaporation. Once the number and the morphology of the meibomian glands are abnormal, abnormal secretion of oil in the meibomian glands is often caused, the stability of the tear film is reduced, and finally, the meibomian glands are dysfunctional, which is a main cause of the excessive-evaporation type xerophthalmia, and the xerophthalmia seriously affects the normal life of people. In the prior art, the method for detecting meibomian glands by using medical images mainly comprises the following steps: the morphological structure and the number of the meibomian glands are automatically counted by shooting the meibomian gland images and adopting the traditional image processing technology to segment and extract the meibomian glands from the whole images. Although the detection mode can relatively objectively reflect the condition of the meibomian glands, the operation is quick and simple, and the discomfort of a patient is small; but in the course of implementing the present invention the inventors found that: because the inner surface of the eyelid is covered with the grease layer with protection and wetting functions, the surface reflection is easy to generate in the imaging process of the meibomian gland to form high-brightness spots, and the high-brightness spots are affected by the high-brightness spot areas, so that the traditional image processing method is difficult to extract the meibomian gland quickly and effectively, and the detection error is larger. With the rapid development of deep learning in the field of image processing, various convolutional neural networks have been used to improve the efficiency of image detection. However, most deep learning networks are highly supervised learning, requiring a large number of paired annotation images to form a training sample. In the medical field, it requires that two pictures be taken simultaneously on the same part of the same patient, one with and one without glistening, which is very difficult to achieve, i.e. the application of deep learning in medical image detection and recognition, especially meibomian glands, is very limited. Therefore, how to design an image processing scheme more suitable for an eye medical image, and optimizing the eye medical image to improve the detection accuracy becomes a technical problem to be solved.
Disclosure of Invention
The invention aims to at least solve one of the technical problems in the prior art or related technologies, and discloses a method, a device and a storage medium for optimizing an eye medical image, wherein the specificity of the eye medical image is considered, the detail expressive force of the eye medical image is enhanced, and the detection precision is further improved.
In a first aspect of the invention, a method of optimizing an ophthalmic medical image is disclosed, comprising: acquiring an eye image; detecting an area with a specular reflection phenomenon (namely a reflection phenomenon) in an eye image, and determining a reflection area; generating a mask according to the reflection area; generating an image of the mask coverage area by using a cyclic generation countermeasure network, namely generating an image; reserving the part outside the mask of the eye image, and fusing the part with the generated image to form an optimized image; wherein cyclically generating an input image against the network comprises a plurality of images of an eye region displayed by the eye image; the circular generation countermeasure network is provided with a double-discriminator module consisting of a local discriminator and a global discriminator, the double-discriminator module not only considers the macroscopic similarity in the picture mapping, but also carries out targeted consideration on key details, and guides the generator to carry out different-degree enhancement on a plurality of areas of the input image so as to improve the image quality of the generated image on the global and key details.
In the technical scheme, an antagonism network (CycleGAN), namely a weak supervision model, is generated based on improved circulation, a cross-domain mapping function with reflective and non-reflective images is learned, training of paired images is not needed, and similar images shot by the same patient at different times can be accepted as input. The model is able to capture context and semantic information, so the output image is similar in content and structure to the input content. Considering the characteristics of eye medical images (most of which have reflection phenomena), the CycleGAN needs to enhance a plurality of local areas of an input image differently from other parts, for example, for small bright spots in a whole dark background, a single global image discriminator in the traditional CycleGAN cannot provide required self-adaption, so the invention improves the discriminator structure for adaptively enhancing the local areas, adopts a global-local dual-discriminator structure, and both the local discriminator and the global discriminator are discriminated based on the PatchGAN.
According to the method for optimizing an ophthalmic medical image disclosed by the invention, preferably, the step of determining the reflection area specifically comprises the following steps: the boundary with a strong brightness change is detected by using an edge detection algorithm, a reflection area is determined according to the boundary, and/or an area with brightness higher than a set threshold is determined as a reflection area.
According to the method for optimizing the eye medical image disclosed by the invention, preferably, the step of fusing specifically comprises the following steps: setting a mask portion of the eye image as a monochromatic patch as a first image; setting a non-mask portion of the output image of the loop-generated countermeasure network as a monochromatic patch as a second image; the first image and the second image are combined.
According to the method for optimizing the eye medical image disclosed by the invention, preferably, the local discriminator and the global discriminator implement discrimination tasks based on the PatchGAN algorithm; the local discriminator randomly cuts local blocks from the clear image and the output image of the generator, learns to distinguish the local blocks from the clear image or the image output by the generator, and the clear image is an image without specular reflection phenomenon in the input image; the global discriminator extracts features from the input image, generates an N matrix and compares the N matrix with the output image of the generator, thereby forming a global comparison concept, and considering the difference of global receptive field information. The image is judged on two scales of global and local details, so that the overall similarity of image generation is guided, and the targeted emphasis enhancement is guided on the reflective local, so that the generated image is closer to the real situation on the details.
According to the method for optimizing the eye medical image disclosed by the invention, preferably, an edge detection algorithm is improved based on a Canny edge detection algorithm, and the specific process of edge detection comprises the following steps: performing surface blurring treatment on the eye image to filter noise; calculating gradients by using a Scharr operator; according to the direction of the gradient, comparing adjacent points in the direction, and inhibiting non-maximum values; obtaining a strong edge, a virtual edge and a non-edge according to the double-threshold classification; and the virtual edges connected with the strong edges are edges, the virtual edges not connected with the strong edges are weak edges, and the weak edges are restrained.
According to the method of optimizing an ophthalmic medical image disclosed in the present invention, preferably, the step of surface blurring treatment is expressed as:
wherein M is ij Representing the value of each element in the matrix, P ij Representing the pixel component value, P, corresponding to each element in the matrix 0 A pixel component value representing the center of the matrix, T representing a threshold value,representing the absolute difference between the pixel component value corresponding to the matrix element and the pixel component value corresponding to the center element; if M ij If the value of (2) is less than 0, M will be ij Set to 0.
According to the method of optimizing an ophthalmic medical image disclosed in the present invention, preferably, the input image and the ophthalmic image to be processed are the same ophthalmic image of the patient.
According to the method of optimizing an ocular medical image disclosed in the present invention, preferably, the ocular image is a meibomian gland image.
In a second aspect, the invention discloses an apparatus for optimizing medical images of the eye, comprising: a memory for storing program instructions; a processor for invoking program instructions stored in the memory to implement a method of optimizing an ophthalmic medical image as in any one of the above aspects.
A third aspect of the invention discloses a computer readable storage medium storing program code for implementing a method of optimizing an ophthalmic medical image according to any of the above-mentioned aspects.
The beneficial effects of the invention at least comprise: the improved cyclic countermeasure generation network is adopted to reconstruct the region with the specular reflection phenomenon in the eye medical image, so that clear eye images are obtained, the disease condition is helped to be studied and judged, and the image processing scheme provided by the invention does not need to carry out model training on paired pictures. By using the global-local dual-discriminator structure, the input image has a plurality of local areas (reflective areas) and other areas (other areas except reflective areas) of the image are enhanced to different degrees, more details can be paid attention to, the discrimination capability of the discriminator is improved, the generating capability of the generator is improved along with the improvement of the discriminating capability of the discriminator, and the generator can generate a non-reflective image based on the input of the reflective image, so that the generated image is closer to a real image along with the improvement of the generating capability of the generator, and the restored image is closer to an ideal non-reflective clear image. Under the conditions that the image illumination is uneven and the contrast ratio between gland and non-gland areas is low due to factors such as illumination angles, irregular imaging areas on the surface of the meibomian gland and the like, the global-local dual-discriminant structure is used for carrying out emphasis enhancement on the reflective area different from the whole image, thereby being more beneficial to generating a predicted image which is closer to a real meibomian gland image in detail, greatly reducing the detection difficulty finally and enabling the detection result to be more accurate.
Drawings
Fig. 1 shows a flow diagram of a method of optimizing an ophthalmic medical image according to one embodiment of the invention.
FIG. 2 shows a schematic diagram of a model of a loop generation countermeasure network, according to one embodiment of the invention.
FIG. 3 illustrates a global-local dual arbiter schematic of a loop generation countermeasure network, according to one embodiment of the invention.
Fig. 4 shows a schematic block diagram of an apparatus for optimizing an ophthalmic medical image according to an embodiment of the invention.
Figure 5 shows an unoptimized meibomian gland image.
Figure 6 shows an image of the meibomian glands after processing by a conventional image processing method.
Figure 7 shows an image of the meibomian glands after treatment by the optimization method of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and therefore the present invention is not limited to the specific embodiments disclosed below.
As shown in fig. 1, according to one embodiment of the present invention, the method for optimizing an ophthalmic medical image disclosed in the present invention includes: step S101, obtaining an eye image; step S102, detecting the area with the mirror reflection phenomenon in the eye image, and determining the reflection area; step S103, generating a mask according to the reflection area; step S104, generating an image of the coverage area of the resist network generation mask, namely generating an image, based on the improved cycle; step S105, reserving the part outside the mask of the eye image, and fusing the part with the generated image to form an optimized image; wherein the improved loop generates a plurality of images of the eye region displayed by the input image of the countermeasure network including the eye image; the improved cyclic generation contrast network is provided with a double-discriminator module consisting of a local discriminator and a global discriminator, and a plurality of areas of an input image are enhanced to different degrees through the cooperation of the double-discriminator module and a generator so as to improve the image quality of the generated image.
In this embodiment, the network structure of the conventional loop generation countermeasure network is improved, a dual-arbiter module is used to replace the original network arbiter, the input image of the loop generation countermeasure network is locally and globally authenticated based on the new local arbiter and the global arbiter, and several local areas of the input image are differently enhanced from other parts, so that the image quality of the output image is improved, and more detailed information can be displayed, as shown in fig. 5, 6 and 7: figure 5 is an image of an unoptimized meibomian gland (original image) containing several light reflecting areas (i.e. several bright spots of different sizes in the figure), representative areas being shown in boxes a, b, c. Fig. 6 is a meibomian gland image processed by a conventional image processing method, which has a certain enhancement to an original image, for example, a reflection area in a frame a and a reflection area in a frame b are repaired to a certain extent, but the reflection area in the frame a cannot be completely eliminated, a significant color difference (distortion) exists between a part and surrounding pixels after the elimination in the frame b, and the reflection area in the frame c cannot be enhanced, so that the conventional method shown in fig. 6 has a certain enhancement effect to the original image, but still cannot meet the requirement of disease condition research judgment (detection). Fig. 7 is a meibomian gland image processed by the optimization method of the present invention, which substantially eliminates the reflective areas in the a and c frames, and also has more accurate local restoration of the reflective areas in the b frame, without significant chromatic aberration with surrounding pixels. Fig. 7 shows that the optimization method proposed according to the above embodiment repairs the specular reflection area in the original image to make it as close as possible to the real situation of the meibomian gland, and the whole process does not need to perform model training on paired pictures, which is more suitable for processing medical images than the conventional image processing method. The optimization principle mainly comprises the following steps: by using the global-local dual-discriminant structure, the input image has a plurality of local areas (namely, the reflective areas) and other parts of the image (namely, other areas except for reflective areas) which are enhanced to different degrees, more details can be focused, the generation capacity of the generator can be improved along with the improvement of the discriminant capacity of the discriminant, and the generator can generate a non-reflective image based on the input of the reflective image, so that the generated image is closer to a real image along with the improvement of the generation capacity of the generator, the better light removal can be realized, and the restored image is closer to an ideal non-reflective meibomian gland clear image.
According to the above embodiment, preferably, step S102 specifically includes: the boundary with a strong brightness change is detected by using an edge detection algorithm, a reflection area is determined according to the boundary, and/or an area with brightness higher than a set threshold is determined as a reflection area.
According to the above embodiment, preferably, the step of fusing in step S105 specifically includes: setting a mask portion of the eye image as a single color patch (generally as a black patch) as a first image; setting a non-mask portion of the output image of the loop-generated countermeasure network as a monochromatic patch as a second image; the first image and the second image are combined.
According to the above embodiment, it is preferable that the local arbiter and the global arbiter perform authentication tasks based on the PatchGAN algorithm; the local discriminator randomly cuts local blocks from the clear image and the output image of the generator, learns to distinguish the local blocks from the clear image or the image output by the generator, and the clear image is an image without specular reflection phenomenon in the input image; the global discriminator extracts features from the input image, generates an N matrix and compares the N matrix with the output image of the generator, thereby forming a global comparison concept, and considering the difference of global receptive field information.
According to the above embodiment, preferably, the edge detection algorithm is improved based on the Canny edge detection algorithm, and the specific process of edge detection includes: performing surface blurring treatment on the eye image to filter noise; calculating gradients by using a Scharr operator; according to the direction of the gradient, comparing adjacent points in the direction, and inhibiting non-maximum values; obtaining a strong edge, a virtual edge and a non-edge according to the double-threshold classification; and the virtual edges connected with the strong edges are edges, the virtual edges not connected with the strong edges are weak edges, and the weak edges are restrained.
In this embodiment, unlike the conventional Canny edge detection algorithm, the present invention uses surface blurring filtering noise instead of gaussian filtering kernel, and changes the 3 x 3 Sobel operator to adopt Scharr operator. The reason for this improvement is: both noise and edge signals belong to the high frequency signal. Gaussian blur will blur all high frequency information, whereas surface blur methods involving preserving edge function will not take all neighboring pixels into smooth computation, but rather will make a threshold value, which is only taken into account if the difference in gray level between the center pixel and the other pixels is smaller than this threshold value. Thus, for pixels with a large difference from the center pixel gray scale, it is considered that they contain effective information instead of noise, and therefore are not included in the smoothing process, and it is more appropriate to preserve edge details. According to the invention, non-edge characteristics can be eliminated by adopting non-maximum suppression, and meanwhile, the accuracy of locating the edge is improved; and by adopting hysteresis thresholding, the probability of edge omission can be effectively reduced. From data experience, the sharpening (Scharr) operator performs better than the Sobel operator in terms of continuity and handling of details.
Non-maximum suppression: the non-maximum value suppression is an image processing method, and the principle of the non-maximum value suppression is that local maximum values are found around pixel points, and gray values corresponding to the non-maximum value points are regarded as background pixels. If the gradient values of the neighborhood of pixel points satisfy the locally optimal condition, the pixel points are considered as pixels with edge features. By executing the method, non-edge points can be effectively reduced, and only pixel points with potential edge characteristics are left. The main purpose of this step is to exclude non-edge pixels, preserving candidate image edges.
Hysteresis thresholding: the edge is often subject to interruption due to noise. The hysteresis thresholding method solves this problem, which is set according to two thresholds: a high threshold and a low threshold. In dual thresholding, the high threshold removes most of the noise, but at the same time some useful edge information is lost; while a low threshold value can retain more edge information. A pixel is also marked as an edge if its response value lies between the low and high thresholds and the pixel is in an eight-adjacency relationship with surrounding pixels that have been marked as an edge. Thus, the problem of edge interruption can be effectively solved.
According to the above embodiment, preferably, the step of the surface blurring process is expressed as:
wherein M is ij Representing the value of each element in the matrix, P ij Representing the pixel component value, P, corresponding to each element in the matrix 0 A pixel component value representing the center of the matrix, T representing a threshold value,representing the absolute difference between the pixel component value corresponding to the matrix element and the pixel component value corresponding to the center element; if M ij If the value of (2) is less than 0, M will be ij Set to 0.
According to the above embodiment, it is preferable that the input image and the eye image to be processed are the same eye image of the patient.
According to the above embodiment, preferably, the eye image is a meibomian gland image.
As shown in fig. 2, a model framework diagram of a loop generation countermeasure network is also disclosed according to yet another embodiment of the present invention: comprises a forward mapper G, a backward mapper F, a forward mapper G, a backward mapper G, a forward mapper Y, a backward mapper F, a forward mapper Y, a backward mapper X and corresponding discriminants D Y and D X;
the object of the model is to learn the mapping function between the source domain X (image with glints) and the target domain Y (clear image without glints);
the model accepts unpaired training picture samples X e X and Y e Y,
comprising two mappers: forward mapper G: X-Y and backward mapper F: Y-X,
training data set
Two antagonism discriminators D x and D y are used: d y is to distinguish the set of (non-reflective) target images { y } from the set of (non-reflective) images { G (x) }, D x is to distinguish the set of (reflective) source images { x } from the set of (reflective) images { F (y) }.
As shown in fig. 3, a structure of a global-local dual arbiter for loop generation countermeasure network is also disclosed according to still another embodiment of the present invention. D x and D y are configured as global-local dual arbiter structures, i.e., D x and D y each have a global arbiter and a local arbiter. Both the global arbiter and the local arbiter use PatchGAN to perform authentication tasks. PatchGAN is an image processing scheme that uses several convolution layers to process an input image, ultimately mapping the feature map into an N matrix using a convolution with a channel number of 1. In this matrix, each point represents an evaluation value of a small area in the original image, which is equivalent to evaluation using a single evaluation value. The benefit of outputting an N x N matrix is that each small region can be evaluated independently, which is also the meaning of Patch, compared to outputting only one value. To achieve loss computation, the tags are also set to an nxn format. PatchGAN has the advantage that it uses more "receptive fields" so that more areas can be comprehensively considered. Local arbiter: local blocks are randomly clipped from the output and the true normal light image and learn a global arbiter that distinguishes whether they are true (from the true image) or false (from the enhanced output): features are extracted from an input image by using a convolutional neural network, and then an N matrix is generated for comparison, so that a global comparison concept is formed, and the difference of global receptive field information is considered.
As shown in fig. 4, an apparatus 300 for optimizing an ophthalmic medical image is also disclosed according to yet another embodiment of the present invention, comprising: a memory 301 for storing program instructions; a processor 302 for invoking program instructions stored in memory to implement the method of optimizing an ophthalmic medical image as in the above-described embodiments.
According to yet another embodiment of the present invention, a computer readable storage medium storing program code for implementing the method of optimizing an ophthalmic medical image as in the above embodiment is also disclosed.
All or part of the steps in the various methods of the above embodiments may be performed by controlling related hardware by a program, which may be stored in a readable storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (ErasableProgrammable Read Only Memory, EPROM), one-time programmable Read-Only Memory (One-timeProgrammable Read-Only Memory, OTPROM), electrically erasable rewritable Read-Only Memory (EEPROM), compact disc Read-Only Memory (CD-ROM) or other optical disk Memory, magnetic disk Memory, tape Memory, or any other medium capable of being used for carrying or storing data.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of optimizing an ophthalmic medical image, comprising:
acquiring an eye image;
detecting an area with a specular reflection phenomenon in the eye image, and determining a reflection area;
generating a mask according to the reflection area;
generating an image of the mask coverage area by using a cyclic generation countermeasure network, namely generating an image;
reserving the part outside the mask of the eye image, and fusing the part with the generated image to form an optimized image;
wherein the loop generates a plurality of images of an eye region displayed by the eye image;
the cyclic generation countermeasure network is provided with a double-discriminant module composed of a local discriminant and a global discriminant, and a plurality of areas of an input image are enhanced to different degrees through the cooperation of the double-discriminant module and a generator so as to improve the image quality of the generated image.
2. The method for optimizing an ophthalmic medical image according to claim 1, characterized in that said step of determining a reflection area comprises in particular: detecting a boundary with a strong brightness change by using an edge detection algorithm, determining the reflection area according to the boundary, and/or determining an area with brightness higher than a set threshold value as the reflection area.
3. The method for optimizing an ophthalmic medical image according to claim 1, characterized in that the step of fusing comprises in particular:
setting a mask portion of the eye image as a monochromatic patch as a first image;
setting a non-mask portion of the output image of the loop-generated countermeasure network as the monochromatic patch as a second image;
and combining the first image and the second image.
4. The method of optimizing an ophthalmic medical image of claim 1, wherein the local arbiter and the global arbiter perform an authentication task based on a PatchGAN algorithm; the local discriminator randomly cuts local blocks from the clear image and the output image of the generator, learns to distinguish whether the local blocks are from the clear image or the image output by the generator, and the clear image is an image without specular reflection phenomenon in the input image; the global discriminator extracts features from the input image, generates an N x N matrix and compares the N x N matrix with the output image of the generator, thereby forming a global comparison concept, and taking the difference of global receptive field information into consideration.
5. The method for optimizing an ophthalmic medical image according to claim 2, wherein the edge detection algorithm is modified based on a Canny edge detection algorithm, and the specific process of edge detection comprises:
performing surface blurring processing on the eye image to filter noise;
calculating gradients by using a Scharr operator;
according to the direction of the gradient, comparing adjacent points in the direction, and inhibiting non-maximum values;
obtaining a strong edge, a virtual edge and a non-edge according to the double-threshold classification;
and the virtual edges connected with the strong edges are edges, the virtual edges not connected with the strong edges are weak edges, and the weak edges are restrained.
6. The method of optimizing an ophthalmic medical image of claim 5 wherein the step of surface blurring is represented by:
；
wherein M is ij Representing the value of each element in the matrix, P ij Representing the pixel component value, P, corresponding to each element in the matrix 0 A pixel component value representing the center of the matrix, T representing a threshold value,representing the absolute difference between the pixel component value corresponding to the matrix element and the pixel component value corresponding to the center element; if M ij If the value of (2) is less than 0, M will be ij Set to 0.
7. The method of optimizing an ophthalmic medical image according to any one of claims 1 to 6, characterized in that the input image and the ophthalmic image to be processed are the same ophthalmic image of the patient.
8. The method of optimizing an ophthalmic medical image of any one of claims 1 to 6 wherein the ophthalmic image is a meibomian gland image.
9. An apparatus for optimizing medical images of the eye, comprising:
a memory for storing program instructions;
a processor for invoking the program instructions stored in the memory to implement the method of optimizing an ophthalmic medical image as in any of claims 1-8.
10. A computer readable storage medium, characterized in that it stores a program code for implementing the method of optimizing an ophthalmic medical image according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311606025.1A CN117314911B (en) | 2023-11-29 | 2023-11-29 | Method, device and storage medium for optimizing eye medical image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311606025.1A CN117314911B (en) | 2023-11-29 | 2023-11-29 | Method, device and storage medium for optimizing eye medical image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117314911A true CN117314911A (en) | 2023-12-29 |
| CN117314911B CN117314911B (en) | 2024-03-01 |
Family
ID=89286947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311606025.1A Active CN117314911B (en) | 2023-11-29 | 2023-11-29 | Method, device and storage medium for optimizing eye medical image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117314911B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019148940A (en) * | 2018-02-26 | 2019-09-05 | パナソニック株式会社 | Learning processing method, server device, and reflection detection system |
| CN111145155A (en) * | 2019-12-26 | 2020-05-12 | 上海美沃精密仪器股份有限公司 | Meibomian gland recognition method |
| CN112001404A (en) * | 2020-08-25 | 2020-11-27 | 华中农业大学 | Image generation model and method for self-adaptive global and local double-layer optimization |
| CN115170427A (en) * | 2022-07-18 | 2022-10-11 | 中山火炬职业技术学院 | Image mirror surface highlight removal method based on weak supervised learning |
| CN115660979A (en) * | 2022-10-20 | 2023-01-31 | 桂林理工大学 | Attention mechanism-based double-discriminator image restoration method |
| CN116309435A (en) * | 2023-03-13 | 2023-06-23 | 卓外（上海）医疗电子科技有限公司 | Medical image specular reflection restoration method and system |
| CN117094910A (en) * | 2023-09-05 | 2023-11-21 | 华东师范大学 | Anti-network image deblurring method based on nonlinear inactive network |
-
2023
- 2023-11-29 CN CN202311606025.1A patent/CN117314911B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019148940A (en) * | 2018-02-26 | 2019-09-05 | パナソニック株式会社 | Learning processing method, server device, and reflection detection system |
| CN111145155A (en) * | 2019-12-26 | 2020-05-12 | 上海美沃精密仪器股份有限公司 | Meibomian gland recognition method |
| CN112001404A (en) * | 2020-08-25 | 2020-11-27 | 华中农业大学 | Image generation model and method for self-adaptive global and local double-layer optimization |
| CN115170427A (en) * | 2022-07-18 | 2022-10-11 | 中山火炬职业技术学院 | Image mirror surface highlight removal method based on weak supervised learning |
| CN115660979A (en) * | 2022-10-20 | 2023-01-31 | 桂林理工大学 | Attention mechanism-based double-discriminator image restoration method |
| CN116309435A (en) * | 2023-03-13 | 2023-06-23 | 卓外（上海）医疗电子科技有限公司 | Medical image specular reflection restoration method and system |
| CN117094910A (en) * | 2023-09-05 | 2023-11-21 | 华东师范大学 | Anti-network image deblurring method based on nonlinear inactive network |
Non-Patent Citations (2)
| Title |
|---|
| RAGHUDEEP GADDE ET.AL.: ""Detail Me More: Improving GAN’s photo-realism of complex scenes"", 《2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
| 胡宇航等: ""基于条件生成对抗网络的梯级表面高光去除方法"", 《计算机应用研究》, vol. 39, no. 9 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117314911B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wu et al. | IID-Net: Image inpainting detection network via neural architecture search and attention | |
| RU2691195C1 (en) | Image and attribute quality, image enhancement and identification of features for identification by vessels and individuals, and combining information on eye vessels with information on faces and/or parts of faces for biometric systems | |
| CN110458883B (en) | Medical image processing system, method, device and equipment | |
| Zhang et al. | The application of visual saliency models in objective image quality assessment: A statistical evaluation | |
| JP4755202B2 (en) | Face feature detection method | |
| US11188771B2 (en) | Living-body detection method and apparatus for face, and computer readable medium | |
| CN112417955B (en) | Method and device for processing tour inspection video stream | |
| CN111681198A (en) | Morphological attribute filtering multimode fusion imaging method, system and medium | |
| CN111507932A (en) | High-specificity diabetic retinopathy characteristic detection method and storage equipment | |
| Davy et al. | Reducing anomaly detection in images to detection in noise | |
| CN111192241A (en) | Quality evaluation method and device of face image and computer storage medium | |
| CN113537037A (en) | Pavement disease identification method, system, electronic device and storage medium | |
| CN111259757A (en) | Image-based living body identification method, device and equipment | |
| Queiroz et al. | Endoscopy image restoration: A study of the kernel estimation from specular highlights | |
| CN117314911B (en) | Method, device and storage medium for optimizing eye medical image | |
| Ahmed et al. | Retina based biometric authentication using phase congruency | |
| Zabihi et al. | Vessel extraction of conjunctival images using LBPs and ANFIS | |
| Fosco et al. | Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines | |
| CN111860288B (en) | Face recognition method, device and system and readable storage medium | |
| Mazumder et al. | Deep learning approaches for diabetic retinopathy detection by image classification | |
| Prakash et al. | Three level automatic segmentation of optic disc using LAB color space contours and morphological operation | |
| Jiang et al. | A supervised method for retinal image vessel segmentation by embedded learning and classification | |
| CN117408938A (en) | Real-time tear film rupture time analysis method, device and system | |
| NL2024816B1 (en) | Detection method for detecting an occlusion of an eye region in an image | |
| CN115409765B (en) | Blood vessel extraction method and device based on fundus retina image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |