WO2023087754A1

WO2023087754A1 - Method for repairing optic disc area of fundus image and related product

Info

Publication number: WO2023087754A1
Application number: PCT/CN2022/105896
Authority: WO
Inventors: 陈荡荡; 和超; 张大磊
Original assignee: 北京鹰瞳科技发展股份有限公司; 上海鹰瞳医疗科技有限公司
Priority date: 2021-11-19
Filing date: 2022-07-15
Publication date: 2023-05-25
Also published as: CN113808054A; CN113808054B

Abstract

The present disclosure relates to a method for repairing an optic disc area of a fundus image and a related product. The method comprises: acquiring a first optic disc area image corresponding to a high-exposure fundus image and a second optic disc area image corresponding to a low-exposure fundus image; performing image alignment on the second optic disc area image and the first optic disc area image to generate an aligned optic disc area image aligned with the first optic disc area image; determining a final optic disc area image according to the first optic disc area image and the aligned optic disc area image; and using the final optic disc area image to repair the first optic disc area image. The solution of the present disclosure can not only effectively solve the problem of information loss of an optic disc overexposure area of a fundus image, but also reserve a real color of a fundus image.

Description

Method for inpainting optic disc area of fundus image and related products

Cross References to Related Applications

This disclosure claims the priority of the following Chinese patent application: a Chinese patent application filed on November 19, 2021, with application number 202111373039.4, and the title of the invention is "Method for Repairing Optic Disc Area of Fundus Image and Related Products" .

technical field

The present disclosure generally relates to the field of image processing techniques. More specifically, the present disclosure relates to a method, an apparatus, and a computer-readable storage medium for inpainting an optic disc region of a fundus image.

Background technique

The optic disc (optic disc), which is called the optic disc, can also be called the optic nerve head. The optic disc is located approximately 3 millimeters ("mm") nasally from the macula and is a well-defined, reddish disc-shaped structure approximately 1.5 mm in diameter. When collecting fundus retinal images, the optic disc area is often overexposed due to high reflectivity, and the overexposure of the optic disc will cause loss of information in the optic disc area of the fundus image. The current mainstream image restoration and enhancement technology is High Dynamic Range Imaging ("HDR") technology, which can be used in digital image processing to achieve a larger exposure dynamic range (that is, a larger difference in light and shade).

However, for the images processed by the Image Signal Processing ("ISP") unit, the brightness spatial linear relationship between different exposure images processed by the ISP unit is destroyed, and HDR technology cannot be used for brightness modeling to repair the image. Therefore, how to repair and enhance the image processed by the ISP unit has become an urgent technical problem to be solved.

Contents of the invention

In order to at least partly solve the technical problems mentioned in the background art, the solution of the present disclosure provides a solution for repairing the optic disc region of the fundus image. By using the solution disclosed in the present disclosure, the overexposed area of the video disc can be effectively repaired and enhanced. To this end, the present disclosure provides solutions in the following aspects.

In one aspect, the present disclosure provides a method for repairing an optic disc region of a fundus image, comprising: acquiring a first optic disc region image corresponding to a high-exposure fundus image and a second image corresponding to a low-exposure fundus image image of the optic disc region; image alignment of the second optic disc region image with the first optic disc region image to generate an aligned optic disc region image aligned with the first optic disc region image; according to the first optic disc region image Determining a final optic disc region image with the aligned optic disc region image; and repairing the first optic disc region image by using the final optic disc region image.

In one embodiment, acquiring the first optic disc region image and the second optic disc region image comprises: performing an optic disc detection algorithm on the low-exposure fundus image to determine its corresponding initial optic disc region; and based on the initial The optic disc region performs an image interception operation on the high-exposure fundus image and the low-exposure fundus image respectively, so as to acquire the first optic disc region image and the second optic disc region image.

In another embodiment, wherein determining a final optic disc zone image based on the first optic disc zone image and the aligned optic disc zone image comprises: aligning the optic disc zone image based on the first optic disc zone image and the aligned optic disc zone image performing brightness correction on the region image to obtain a corrected aligned optic disc region image; determining a composite coefficient associated with the final optic disc region image based on the first optic disc region image and the corrected aligned optic disc region image; and based on the Combining coefficients, the first optic disc region image and the corrected aligned optic disc region image determine a final optic disc region image.

In yet another embodiment, performing brightness correction on the aligned optic disc region image based on the first optic disc region image and the aligned optic disc region image, so as to obtain the corrected aligned optic disc region image comprises: converting the optic disc region image and said aligned optic disc region image to HSV color space to determine first luminance data corresponding to said first optic disc region image and second luminance data corresponding to said aligned optic disc region image; Execute a binarization operation on the first brightness data to determine a mask area of the first optic disc region image; perform an edge detection operation according to the mask area of the first optic disc area image to generate an edge of the mask area and performing brightness correction on the aligned optic disc region image based on the first brightness data, the second brightness data and the edge of the mask region, so as to obtain a corrected aligned optic disc region image.

In yet another embodiment, brightness correction is performed on the aligned optic disc region image based on the first brightness data, the second brightness data and the edge of the mask region, so as to obtain a corrected aligned optic disc region image Including: calculating the first brightness average value and the second brightness average value corresponding to the first brightness data and the second brightness data at the edge of the mask area respectively; calculating the first brightness average value and the second brightness average value The difference between the two lightness average values; and aligning the optic disc region image to perform brightness correction according to the sum result of the second brightness data and the difference value, so as to obtain a corrected aligned optic disc region image.

In yet another embodiment, aligning the optic disc region image according to the summation result of the second brightness data and the difference value and performing brightness correction, so as to obtain the corrected aligned optic disc region image comprises: responding to the summation result greater than a preset threshold, using the preset threshold as the corrected second brightness data to perform brightness correction to obtain a corrected image of the optic disc region aligned; And the result is used as the corrected second brightness data to perform brightness correction, so as to obtain the corrected aligned optic disc region image.

In yet another embodiment, wherein determining the composite coefficients associated with the final optic disc region image based on the first optic disc region image and the corrected aligned optic disc region image comprises: based on the first optic disc region image and the corrected aligned optic disc region image The distance between each of the optic disc region images and the center of the mask region and the radius of the mask region determine the synthesis coefficients associated with the final optic disc region image.

In yet another embodiment, wherein determining the final optic disc region image based on the synthesis coefficients, the first optic disc region image and the corrected aligned optic disc region image comprises: using the synthesis coefficients to combine the first optic disc region image and The corrected aligned optic disc region images are composited in RGB color space to generate said final optic disc region image.

In another aspect, the present disclosure also provides a device for repairing the optic disc region of the fundus image, including: a processor; and a memory connected to the processor, where computer program codes are stored in the memory, when the When the computer program code is executed, it causes the processor to execute the aforementioned multiple embodiments.

In yet another aspect, the present disclosure also provides a computer-readable storage medium, on which are stored computer-readable instructions for repairing the optic disc region of the fundus image, when the computer-readable instructions are executed by one or more processors , to implement the aforementioned multiple embodiments.

Through the scheme of the present disclosure, the aligned optic disc region image is obtained by aligning the optic disc region image corresponding to the low-exposure fundus image with the optic disc region image corresponding to the high-exposure fundus image, and then by aligning the optic disc region image corresponding to the high-exposure fundus image The final optic disc area image determined by the optic disc area repairs and enhances the optic disc area image corresponding to the high-exposure fundus image, so as to solve the problem of missing information due to overexposure of the optic disc area image corresponding to the high-exposure fundus image, in order to obtain high-quality Image. Furthermore, the disclosed solution converts the image into HSV color space and RGB color space for processing, which can preserve the true color of the fundus image, and is convenient for subsequent diagnosis and treatment of the fundus.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are shown by way of illustration and not limitation, and the same or corresponding reference numerals indicate the same or corresponding parts wherein:

FIG. 1 is an exemplary flowchart illustrating a method for repairing an optic disc region of a fundus image according to an embodiment of the present disclosure;

2 is an exemplary schematic diagram showing a high-exposure fundus image and a low-exposure fundus image according to an embodiment of the present disclosure;

Fig. 3 is an exemplary schematic diagram showing an initial optic disc area of a low-exposure fundus image and an image of the optic disc area determined after performing image capture according to an embodiment of the present disclosure;

FIG. 4 is an exemplary flowchart illustrating obtaining a corrected aligned optic disc region image according to an embodiment of the present disclosure;

FIG. 5 is another exemplary flowchart illustrating obtaining a corrected aligned optic disc region image according to an embodiment of the present disclosure;

FIG. 6 is an exemplary schematic diagram illustrating determining a composite coefficient according to an embodiment of the present disclosure;

Fig. 7 is an exemplary schematic diagram illustrating a restored and enhanced fundus image according to an embodiment of the present disclosure;

FIG. 8 is a block diagram illustrating an overall exemplary process for repairing the optic disc region of a fundus image according to an embodiment of the present disclosure; and

FIG. 9 is a block diagram illustrating an apparatus for repairing an optic disc region of a fundus image according to an embodiment of the present disclosure.

Detailed ways

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. It should be understood that the embodiments described in this specification are only some of the embodiments provided by the present disclosure to facilitate a clear understanding of the solutions and comply with legal requirements, but not all embodiments of the present disclosure can be implemented. Based on the embodiments disclosed in this specification, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present disclosure.

FIG. 1 is an exemplary flowchart illustrating a method 100 for inpainting an optic disc region of a fundus image according to an embodiment of the present disclosure. As shown in FIG. 1 , at step S102 , a first optic disc region image corresponding to a high-exposure fundus image and a second optic disc region image corresponding to a low-exposure fundus image are acquired. In one embodiment, the aforementioned high-exposure fundus image and low-exposure fundus image can be collected by, for example, an ophthalmic instrument fundus camera. It can be understood that the aforementioned high-exposure fundus image refers to a fundus image with normal overall exposure (that is, brightness) and overexposed optic disc area (for example, as shown in the left figure in FIG. 2 ). The aforementioned low-exposure fundus image refers to a fundus image with a dark overall brightness and a normal (clear) optic disc area (such as shown in the right figure in Figure 2). Based on the collected high-exposure fundus images and low-exposure fundus images, their respective corresponding first optic disc region images and second optic disc region images can be acquired. Specifically, firstly, an optic disc detection algorithm may be performed on the low-exposure fundus image to determine its corresponding initial optic disc area (such as the rectangular box shown in FIG. 3 ). Next, based on the initial optic disc area, an image capture operation may be performed on the high-exposure fundus image and the low-exposure fundus image respectively, so as to acquire the first optic disc area image and the second optic disc area image. The foregoing optic disc detection and image capture will be described in detail later with reference to FIG. 3 .

After obtaining the first optic disc region image and the second optic disc region image, at step S104, the second optic disc region image is image-aligned with the first optic disc region image to generate an aligned optic disc region aligned with the first optic disc region image image. In one embodiment, the aforementioned image alignment operation may be performed using, for example, the OpenCV software library. In particular, the aforementioned image alignment operations are usually implemented using the feature-based image alignment method in OpenCV. In the embodiment of the present disclosure, the feature-based image alignment method refers to extracting the feature points of the second optic disc region image and the first optic disc region image respectively, and then combining the feature points of the second optic disc region image with the features of the first optic disc region image Click to match. Calculate a conversion rule (such as image similarity change matrix) according to the matching result, and then map the second optic disc region image to the first optic disc region image based on the aforementioned conversion rule, and finally obtain an aligned optic disc region image aligned with the first optic disc region image. Alternatively or additionally, image alignment operations can be performed using AlignMTB from the OpenCV software library.

Based on the obtained aligned optic disc region images, the method 100 proceeds to step S106. At this step, a final optic disc region image is determined based on the first optic disc region image and the aligned optic disc region image. In an embodiment, brightness correction may be performed on the aligned optic disc region image first based on the first optic disc region image and the aligned optic disc region image, so as to obtain a corrected aligned optic disc region image. Next, determine the synthesis coefficient associated with the final optic disc area image according to the first optic disc area image and the corrected aligned optic disc area image, and then determine the final optic disc area based on the synthesis coefficient, the first optic disc area image and the corrected aligned optic disc area image image. The aforementioned determination process of the final optic disc region image will be described in detail later with reference to FIGS. 4-5 .

Finally, at step S108, the first optic disc region image is repaired using the final optic disc region image. Specifically, the obtained final optic disc region image (that is, the repaired and enhanced normal optic disc region image) is used to replace (or add to) the first optic disc region image intercepted by the high-exposure fundus image (such as shown in the left figure in Figure 7). shown), so as to obtain the repaired and enhanced fundus image (such as shown in the right figure in Figure 7).

In combination with the above description, it can be seen that the scheme of the present disclosure utilizes the fundus image (i.e. low-exposure fundus image) with darker overall brightness and normal (clear) optic disc area to make the overall exposure (i.e. brightness) normal and the optic disc area overexposed The fundus image (that is, the high-exposure fundus image) is repaired and enhanced to solve the problem of missing information in the optic disc area caused by overexposure of the optic disc area, so as to obtain high-quality fundus images for subsequent diagnosis and treatment of the fundus.

FIG. 2 is an exemplary schematic diagram illustrating a high-exposure fundus image and a low-exposure fundus image according to an embodiment of the present disclosure. The left figure in Figure 2 shows a high-exposure fundus image. It can be seen that the overall brightness of the high-exposure fundus image is normal, but the optic disc area is overexposed (such as the circular area pointed by the arrow in the left figure in Figure 2). The right figure in Figure 2 shows the low-exposure fundus image. It can be seen that the overall brightness of the low-exposure fundus image is dark, while the optic disc area is clear (such as the circular area pointed by the arrow in the right figure in Figure 2). As mentioned above, performing the optic disc detection algorithm on low-exposure fundus images can determine its corresponding initial optic disc area. Further, performing image interception operations on the high-exposure fundus image and the low-exposure fundus image respectively based on the initial optic disc area can obtain the first optic disc area image corresponding to the high-exposure fundus image and the second optic disc area corresponding to the low-exposure fundus image image. In one embodiment, the optic disc detection algorithm can be performed on low-exposure fundus images using deep learning detection algorithms such as the YOLO detection network to determine the initial optic disc area, such as the rectangular box shown in FIG. 3 .

Fig. 3 is an exemplary schematic diagram showing an initial optic disc area of a low-exposure fundus image and an image of the optic disc area determined after performing image capture according to an embodiment of the present disclosure. The upper left figure in Figure 3 shows the high-exposure fundus image, in which the rectangular frame represents the initial optic disc area obtained by performing the optic disc detection algorithm (such as YOLO detection network) on the low-exposure fundus image. Using the initial optic disc area (that is, the rectangular frame) to intercept the high-exposure fundus image, the optic disc area image shown in the upper right figure in Figure 3 can be obtained, that is, the first optic disc area corresponding to the high-exposure fundus image image. Similarly, the lower left image in Figure 3 shows a low-exposure fundus image, and the initial optic disc area (that is, the rectangular frame) is used to intercept the image, and the optic disc area shown in the lower right image in Figure 3 can be obtained The image, that is, the image of the second optic disc region corresponding to the low-exposure fundus image. Further, image alignment of the second optic disc region image and the first optic disc region image is performed to obtain an aligned optic disc region image, and the final optic disc region image can be determined according to the obtained first optic disc region image and the aligned optic disc region image, so that the final optic disc region image can be used Repair the image of the first optic disc region corresponding to the high-exposure fundus image.

According to the foregoing description, it can be known that, first, brightness correction may be performed on the aligned optic disc region image based on the first optic disc region image and the aligned optic disc region image, so as to obtain a corrected aligned optic disc region image. How to obtain the corrected aligned optic disc region image will be described in detail below with reference to FIG. 4 .

FIG. 4 is an exemplary flowchart illustrating obtaining a corrected aligned optic disc region image according to an embodiment of the present disclosure. It should be understood that FIG. 4 is a specific embodiment of the method 100 in FIG. 1 above, so the above descriptions about FIG. 1 are also applicable to FIG. 4 .

As shown in FIG. 4 , at step S402, the first optic disc region image and the aligned optic disc region image are converted to the HSV color space to determine the first lightness data corresponding to the first optic disc region image and the first lightness data corresponding to the aligned optic disc region image The second lightness data of . It can be understood that the first optic disc region image and the aligned optic disc region image are initially located in the RGB color space, so the embodiment of the present disclosure transforms the first optic disc region image and the aligned optic disc region image from the RGB color space to the HSV color space. In an implementation scenario, for example, an OpenCV tool can be used to convert the RGB color space to the HSV color space, thereby determining the first brightness data corresponding to the first optic disc region image (that is, the brightness V in the HSV color space) and the alignment corresponding to the optic disc region image The second lightness data of .

Based on the obtained first brightness data, at step S404, a binarization operation is performed on the first brightness data to determine a mask region of the first optic disc region image. Specifically, an appropriate brightness threshold can be selected according to the first brightness data (for example, the brightness threshold is 245), and then the first brightness data lower than the brightness threshold is set to "0", and the first brightness data higher than the brightness threshold is set to "0". The brightness data is set to "1", thereby realizing binarization for the first brightness data. The image area where the first lightness data is set to "1" is the mask area of the first optic disc area image (for example, the area shown in the circle in Figure 6 similar to the "cloud" shape).

After obtaining the mask area of the first optic disc area image, at step S406, an edge detection operation is performed according to the mask area of the first optic disc area image to generate an edge of the mask area. In one embodiment, an edge detection operator such as a Sobel operator, a Canny operator, or a Laplacian operator may be used to perform edge detection on the mask area, so as to obtain the edge of the mask area. Those skilled in the art can select an appropriate edge detection operator according to the implementation scenario, and the embodiments of the present disclosure make no limitation here.

According to the obtained first brightness data, second brightness data and the edge of the mask area, the step proceeds to step S408. At this step, brightness correction is performed on the aligned optic disc region image based on the first brightness data, the second brightness data and the edge of the mask region, so as to obtain a corrected aligned optic disc region image. How to obtain the corrected aligned optic disc region image based on the first brightness data, the second brightness data and the edge of the mask region will be described in detail below with reference to FIG. 5 .

FIG. 5 is another exemplary flowchart illustrating obtaining a corrected aligned optic disc region image according to an embodiment of the present disclosure. It can be understood that FIG. 5 is a specific implementation manner of step S408 in FIG. 4 above, so the above description about FIG. 4 is also applicable to FIG. 5 .

As shown in FIG. 5 , at step S502 , the first average brightness and the second average brightness corresponding to the first brightness data and the second brightness data at the edge of the mask area are respectively calculated. Based on the first average brightness value and the second average brightness value obtained, at step S504, a difference between the first average brightness value and the second average brightness value is calculated. In an exemplary scenario, assuming that the first lightness average value is recorded as V _C and the second lightness average value is recorded as V _E , the difference between the first lightness average value and the second lightness average value can be recorded as V _C -V _E . Further, at step S506, brightness correction is performed on the optic disc region image according to the summation result of the second brightness data and the difference value, so as to obtain a corrected aligned optic disc region image. It can be understood that the aforementioned summation result refers to the sum of the second brightness data of each pixel in the image of the aligned optic disc area and the aforementioned difference, that is, after adding (increasing) the difference to the second brightness data of each pixel the result of.

In one embodiment, in response to the above-mentioned summation result being greater than a preset threshold, brightness correction is performed using the preset threshold as the corrected second brightness data, so as to obtain a corrected aligned optic disc region image. Taking the preset threshold value of 250 as an example, when the summing result is greater than 250, the preset threshold value of 250 is used as the corrected second brightness data. When the summation result is less than 250, the summation result (the sum of the second brightness data and the difference value) is used as the corrected second brightness data for brightness correction, so as to obtain a corrected aligned optic disc region image. Based on the aforementioned brightness correction operation, the overall brightness of the corrected aligned optic disc region image can be made close to the overall brightness of the first optic disc region image, so that the restored and enhanced fundus image can retain the true color.

After the corrected aligned optic disc region image is obtained, the synthesis coefficient associated with the final optic disc region image can be determined according to the first optic disc region image and the corrected aligned optic disc region image. Specifically, according to the distance between the first optic disc region image and the corrected aligned optic disc region image and the center of the mask region (that is, the mask region obtained in step S404 shown in FIG. 4 ) and the mask region The radius of determines the composite coefficient associated with the final optic disc area. How to determine the combination coefficient will be described in detail below with reference to FIG. 6 .

FIG. 6 is an exemplary diagram illustrating determining a combination coefficient according to an embodiment of the present disclosure. As shown in Figure 6, it is assumed that the area shown in the rectangle is the image of the first optic disc area, and the area shown in the shape of a "cloud" in the inner circle is the mask area, and the solid circle in the mask area represents the center of the mask area point. By processing the mask area, the weight (coefficient) of the image in the first optic disc area is larger when the distance from the center of the mask area is farther away, and the weight (coefficient) of the image in the first optic disc area is larger, and the corrected alignment optic disc is closer to the center of the mask area. The weight (coefficient) of the area image is relatively large, and the sum of the weight (coefficient) of the first optic disc area image and the weight (coefficient) of the corrected aligned optic disc area image is 1, thus determining the final optic disc area image. composite coefficient. Based on the aforementioned processing methods, in an exemplary scenario, the embodiment of the present disclosure expresses the aforementioned combination coefficient, the image distance from the center of the mask area, and the radius of the mask area by the following formula:

f(d)=d^2/R^2 (1)

Among them, R indicates that the radius of the mask area is R, d indicates the distance from the center of the mask area; f(d) indicates the synthesis coefficient.

According to the synthesis coefficient obtained above, the first optic disc region image and the corrected aligned optic disc region image, the final optic disc region image can be determined. In one embodiment, the first optic disc region image and the corrected aligned optic disc region image may be synthesized in RGB color space by using the synthesis coefficients to generate a final optic disc region image. According to the previous knowledge, the first optic disc region image and the corrected aligned optic disc region image are located in the RGB color space. In an implementation scenario, the first optic disc region image and the corrected aligned optic disc region image may be synthesized in the RGB color space based on the following formula:

R(x,y)＝R1(x,y)*f(d)+R2(x,y)*(1-f(d)) (2)

Among them, R(x, y) represents the pixel value (red channel value) corresponding to the pixel at the coordinate position (x, y) in the final optic disc region image, and R1(x, y) represents the pixel value in the first optic disc region image The pixel value (red channel value) of the pixel at the coordinate position (x, y), R2(x, y) represents the pixel at the pixel point at the coordinate position (x, y) in the corrected aligned optic disc region image value (red channel value), f(d) represents the synthesis factor, and d represents the distance from the center of the masked area.

After the final optic disc region image is obtained based on the above formula (2), the final optic disc region image is used to replace the first optic disc region image of the high-exposure fundus image to obtain a repaired and enhanced fundus image, as shown in FIG. 7 .

Fig. 7 is an exemplary schematic diagram showing a restored and enhanced fundus image according to an embodiment of the present disclosure. The upper part of the left image in Figure 7 shows the final area image, and the lower part of the left image in Figure 7 shows the high-exposure fundus image, and the area shown in the rectangular box is the first optic disc region image corresponding to the high-exposure fundus image . In the implementation scenario, replace the area in the rectangular box (that is, the first disc area image) shown in the upper left image in Figure 7 with the final area image shown in the upper left image in Figure 7, and the right image in Figure 7 can be obtained The shown restoration and enhancement of the fundus image realizes the restoration and enhancement of the image of the first optic disc region of the high-exposure fundus image.

Based on the above description, it can be known that the embodiments of the present disclosure repair and enhance the high-exposure fundus image by using the low-exposure fundus image, so as to solve the problem that the optic disc area of the fundus image is overexposed and causes information loss in the optic disc area. Further, the embodiment of the present disclosure can preserve the true color of the fundus image by converting the optic disc region image into the HSV color space for processing, so that the effect of restoration and enhancement is better and the quality of the restoration and enhancement of the fundus image is higher. Using the high-quality fundus image after restoration is beneficial to the subsequent diagnosis and treatment of the fundus.

FIG. 8 is a block diagram illustrating an overall exemplary process for repairing the optic disc region of a fundus image according to an embodiment of the present disclosure. As shown in FIG. 8 , at step S801 and step S802 , a high-exposure fundus image A and a low-exposure fundus image B are collected by, for example, an ophthalmic instrument fundus camera. Based on the collected high-exposure fundus image A and low-exposure fundus image B, first at step S803, the optic disc detection algorithm is executed on the low-exposure fundus image B to determine the initial optic disc area (such as the rectangular box shown in Figure 3) . In one embodiment, the aforementioned video disc detection algorithm may include but not limited to the YOLO detection network algorithm. After determining the initial optic disc area, at step S804 and step S805, based on the aforementioned initial optic disc area, the image interception operation is performed on the high-exposure fundus image A and the low-exposure fundus image B, and then the corresponding image of the high-exposure fundus image A is obtained. The first optic disc region image C and the second optic disc region image D corresponding to the low-exposure fundus image B. Next, at step S806 , the second optic disc region image D is image-aligned with the first optic disc region image C to obtain an aligned optic disc region image E aligned with the first optic disc region image C. The final optic disc area image can be determined by processing the first optic disc area image C and the aligned optic disc area image E. The first optic disc region image C of the high-exposure fundus image A can be restored by using the aforementioned final optic disc region image.

Obtain the first optic disc region image C and align the optic disc region image E according to the above, at step S807 and step S808, convert the first optic disc region image C and the aligned optic disc region image E to the HSV color space respectively, and obtain the first optic disc region image C and the aligned optic disc region image E The first brightness data corresponding to the image C and the second brightness data corresponding to the aligned optic disc region image E. Then at step S809, a binarization operation is performed on the first lightness data to determine the mask area of the first optic disc area image C, and at step S810, an edge detection operation is performed on the mask area using an edge detection operator, to get the edges of the masked area. Based on the first lightness data, the second lightness data and the edge of the mask area obtained above, at step S811, it can be determined according to the first lightness data, the second lightness data and the edge of the mask area to be associated with the final optic disc area image composite coefficient. Regarding the determination of the combination coefficient, reference may be made to the content described in the above-mentioned FIGS. 4-6 , which will not be repeated in this disclosure.

After the composite coefficients are obtained, the flow proceeds to step S812, where the obtained composite coefficients are used to composite the obtained first optic disc region image C and the aligned optic disc region image E to determine a final optic disc region image. According to the foregoing, the first optic disc region image C and the aligned optic disc region image E can be synthesized based on the above formula (2), so as to determine the final optic disc region image. Finally, the first optic disc region image C of the high-exposure fundus image A is repaired and enhanced by using the final optic disc region image to generate a restored and enhanced fundus image. For the aforementioned restoration and enhancement process, reference may be made to the content described in FIG. 7 above, and details will not be repeated here in the present disclosure.

FIG. 9 is a block diagram illustrating an apparatus 900 for repairing an optic disc region of a fundus image according to an embodiment of the present disclosure. It will be appreciated that a device implementing the disclosed solutions may be a single device (such as a computing device) or a multi-function device including various peripheral devices.

As shown in FIG. 9 , the devices of the present disclosure may include a central processing unit or central processing unit (“CPU”) 911, which may be a general-purpose CPU, a dedicated CPU, or other execution units for information processing and program execution. Further, the device 900 may also include a large-capacity memory 912 and a read-only memory ("ROM") 913, wherein the large-capacity memory 912 may be configured to store various types of data, including various fundus images to be repaired, algorithm data, intermediate Results and various programs required to run device 900. The ROM 913 can be configured to store data and instructions required for power-on self-test of the device 900, initialization of each functional module in the system, basic input/output drivers of the system, and booting the operating system.

Optionally, device 900 may also include other hardware platforms or components, such as shown tensor processing unit ("TPU") 914, graphics processing unit ("GPU") 915, field programmable gate array ("FPGA") ) 916 and Machine Learning Unit ("MLU") 917. It can be understood that although various hardware platforms or components are shown in the device 900, these are only exemplary and non-limiting, and those skilled in the art can add or remove corresponding hardware according to actual needs. For example, the device 900 may only include a CPU, a related storage device and an interface device to implement the method for repairing the optic disc region of the fundus image of the present disclosure.

In some embodiments, in order to facilitate data transfer and interaction with external networks, the device 900 of the present disclosure further includes a communication interface 918, so that it can be connected to a local area network/wireless local area network ("LAN/WLAN") 905 through the communication interface 918, In turn, a connection to a local server 906 or to the Internet ("Internet") 907 may be made via the LAN/WLAN. Alternatively or additionally, the device 900 of the present disclosure may also directly connect to the Internet or a cellular network based on a wireless communication technology, such as a 3rd generation (“3G”), 4th generation (“4G”), or 4th generation 5th generation (“5G”) wireless communication technology. In some application scenarios, the device 900 of the present disclosure can also access the server 908 and database 909 of the external network as needed, so as to obtain various known image models, data and modules, and can store various data remotely, for example, with It is used to present various data or instructions such as image alignment, disc detection, edge detection, etc.

The peripheral equipment of the device 900 may include a display device 902 , an input device 903 and a data transmission interface 904 . In one embodiment, the display device 902 may include, for example, one or more speakers and/or one or more visual displays, which are configured to give voice prompts and/or the final result of the process or final result of repairing the optic disc region of the fundus image of the present disclosure or image video display. The input device 903 may include other input buttons or controls, such as a keyboard, mouse, microphone, gesture capture camera, etc., configured to receive input of fundus images and/or user instructions. Data transfer interface 904 may include, for example, serial, parallel, or Universal Serial Bus ("USB"), Small Computer System Interface ("SCSI"), Serial ATA, FireWire ("FireWire"), PCI Express, and HD Multimedia Interface (“HDMI”), etc., configured for data transmission and interaction with other devices or systems. According to the solution of the present disclosure, the data transmission interface 904 can receive the fundus image collected by the fundus camera, and transmit the fundus image or various other types of data or results to the device 900 .

The aforementioned CPU 911, large-capacity memory 912, ROM 913, TPU 914, GPU 915, FPGA 916, MLU 917 and communication interface 918 of the device 900 of the present disclosure can be connected to each other through the bus 919, and realize data interaction with peripheral devices through the bus . In one embodiment, through the bus 919, the CPU 911 can control other hardware components in the device 900 and its peripherals.

The device for repairing the optic disc region of the fundus image that can be used to implement the present disclosure has been described above with reference to FIG. 9 . It should be understood that the device structure or architecture here is only exemplary, and the implementation manner and implementation entity of the present disclosure are not limited thereto, and changes can be made without departing from the spirit of the present disclosure.

According to the above description in conjunction with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present disclosure can also be implemented by software programs. The present disclosure thus also provides a computer program product. The computer program product can be used to implement the method for repairing the optic disc region of the fundus image described in the present disclosure in conjunction with accompanying drawings 1-8.

It should be noted that, while operations of the disclosed methods are depicted in the figures in a particular order, this does not require or imply that these operations must be performed in that particular order, or that all illustrated operations must be performed, to achieve desirable results. Conversely, the steps depicted in the flowcharts may be performed in an altered order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution.

It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims of the present disclosure, when the specification and drawings are used, they are only used to distinguish different objects, and Not intended to describe a particular order. The terms "comprising" and "comprises" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and/or components, but do not exclude one or more other features, integers , steps, operations, elements, components, and/or the presence or addition of collections thereof.

It should also be understood that the terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used in this disclosure and the claims, the singular forms "a", "an" and "the" are intended to include plural referents unless the context clearly dictates otherwise. It should be further understood that the term "and/or" used in the present disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

Although the embodiments of the present disclosure are as above, the content described is only an embodiment adopted for the convenience of understanding the present disclosure, and is not intended to limit the scope and application scenarios of the present disclosure. Anyone skilled in the technical field described in the present disclosure can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the patent protection scope of the present disclosure , must still be subject to the scope defined by the appended claims.

Claims

A method for inpainting an optic disc region of a fundus image, comprising:

acquiring a first optic disc region image corresponding to the high-exposure fundus image and a second optic disc region image corresponding to the low-exposure fundus image;

image-aligning the second optic disc region image with the first optic disc region image to generate an aligned optic disc region image aligned with the first optic disc region image;

determining a final optic disc zone image based on the first optic disc zone image and the aligned optic disc zone image; and

Restoring the first optic disc region image by using the final optic disc region image.
The method of claim 1, wherein acquiring the first optic disc region image and the second optic disc region image comprises:

performing an optic disc detection algorithm on the low-exposure fundus image to determine its corresponding initial optic disc area; and

An image interception operation is performed on the high-exposure fundus image and the low-exposure fundus image based on the initial optic disc area, so as to acquire the first optic disc area image and the second optic disc area image.
The method of claim 1, wherein determining a final optic disc region image based on the first optic disc region image and the aligned optic disc region image comprises:

performing brightness correction on the aligned optic disc region image based on the first optic disc region image and the aligned optic disc region image, to obtain a corrected aligned optic disc region image;

determining synthesis coefficients associated with a final optic disc zone image based on the first optic disc zone image and the corrected aligned optic disc zone image; and

A final optic disc zone image is determined based on the synthesis coefficients, the first optic disc zone image, and the corrected aligned optic disc zone image.
The method according to claim 3, wherein performing brightness correction on the aligned optic disc region image based on the first optic disc region image and the aligned optic disc region image, so as to obtain the corrected aligned optic disc region image comprises:

converting the first optic disc region image and the aligned optic disc region image to HSV color space to determine first luminance data corresponding to the first optic disc region image and second luminance data corresponding to the aligned optic disc region image data;

performing a binarization operation on the first lightness data to determine a mask area of the first optic disc region image;

performing an edge detection operation based on a masked region of the first optic disc region image to generate edges of the masked region; and

Perform brightness correction on the aligned optic disc region image based on the first brightness data, the second brightness data and the edge of the mask region, so as to obtain a corrected aligned optic disc region image.
The method according to claim 4, wherein brightness correction is performed on the image of the aligned optic disc region based on the first brightness data, the second brightness data and the edge of the mask region, so as to obtain a corrected aligned optic disc Area images include:

calculating the first lightness average value and the second lightness average value corresponding to the first lightness data and the second lightness data respectively at the edge of the mask area;

calculating the difference between the first lightness average and the second lightness average; and

Aligning the optic disc region image according to the summation result of the second brightness data and the difference value to perform brightness correction, so as to obtain a corrected aligned optic disc region image.
The method according to claim 5, wherein aligning the optic disc region image according to the summation result of the second brightness data and the difference value and performing brightness correction, so as to obtain the corrected aligned optic disc region image comprises:

In response to the addition result being greater than a preset threshold value, performing brightness correction on the preset threshold value as the corrected second brightness data, so as to obtain a corrected aligned optic disc region image; or

In response to the addition result being less than the preset threshold value, the addition result is used as the second brightness data after correction to perform brightness correction, so as to obtain a corrected image of the optic disc region aligned.
The method of claim 5, wherein determining synthesis coefficients associated with a final optic disc zone image based on the first optic disc zone image and the corrected aligned optic disc zone image comprises:

Determining a synthesis coefficient associated with the final optic disc region image based on a distance between each of the first optic disc region image and the corrected aligned optic disc region image and a center of the mask region and a radius of the mask region .
The method of claim 3, wherein determining a final optic disc region image based on the synthesis coefficients, the first optic disc region image, and the corrected aligned optic disc region image comprises:

Combining the first optic disc region image and the corrected aligned optic disc region image in RGB color space using the composition coefficients to generate the final optic disc region image.
A device for inpainting the optic disc region of a fundus image, comprising:

processor; and

A memory connected to the processor, wherein computer program code is stored in the memory, and when the computer program code is executed, the processor executes the method according to any one of claims 1-8 .
A computer-readable storage medium, on which computer-readable instructions for repairing the optic disc area of the fundus image are stored, and when the computer-readable instructions are executed by one or more processors, the implementation of claims 1-8 any one of the methods described.