WO2023137904A1

WO2023137904A1 - Eye fundus image-based lesion detection method and apparatus, and device and storage medium

Info

Publication number: WO2023137904A1
Application number: PCT/CN2022/090164
Authority: WO
Inventors: 郑喜民; 王天誉; 舒畅; 陈又新
Original assignee: 平安科技（深圳）有限公司
Priority date: 2022-01-21
Filing date: 2022-04-29
Publication date: 2023-07-27
Also published as: CN114494734A

Abstract

The present application relates to the fields of image recognition and digital treatment. Disclosed are an eye fundus image-based lesion detection method and apparatus, and a computer device and a storage medium. The method comprises: acquiring an eye fundus screening image, wherein the eye fundus screening image comprises a scanning image and an angiography image; inputting the scanning image into a first network in a dual-channel network of a deep learning network model, and acquiring a first image feature obtained by the first network, wherein the first image feature comprises an eye fundus curvature and a reflectivity; inputting the angiography image into a second network in the dual-channel network of the deep learning network model, and acquiring a second image feature obtained by the second network, wherein the second image feature comprises a blood vessel density and an eye fundus tissue thickness; fusing the first image feature with the second image feature to obtain a fused feature; and according to the fused feature, matching a macular lesion level corresponding to the image. The present application can improve the accuracy of recognizing a macular lesion level of the eye fundus.

Description

Lesion detection method, device, equipment and storage medium based on fundus image

This application claims the priority of the Chinese patent application with the application number 202210073516.3 and the invention title "Method, device, equipment and storage medium for lesion detection based on fundus image" submitted to the China Patent Office on January 21, 2022, the entire contents of which are incorporated in this application by reference.

technical field

The present application relates to the fields of image recognition and digital medical care, in particular to a fundus image-based lesion detection method, device, computer equipment and storage medium.

Background technique

Age-related macular degeneration (AMD) is an eye disease that seriously affects the vision of the elderly. Currently, AMD is detected through in-depth and time-consuming analysis of fundus images based on color fundus photographs. The inventor realized that the fundus images used in the current research are fundus color images and optical coherence tomography (OCT) images respectively. The fundus features of the two images are independent of each other, and the degree of fundus maculopathy cannot be accurately identified.

technical problem

The main purpose of this application is to provide a lesion detection method, device, computer equipment and storage medium based on fundus images, aiming to solve the problem of low accuracy in identifying the degree of fundus macular lesions at present.

technical solution

In order to achieve the above-mentioned purpose of the invention, this application proposes a lesion detection method based on fundus images, including:

Obtaining a fundus screening image, the fundus screening image including a scan image and an angiographic image;

The scanned image is input to the first network in the dual-channel network of the deep learning network model, and the first image features obtained by the first network are obtained; the first image features include fundus curvature and reflectivity;

The contrast image is input to the second network in the dual-channel network of the deep learning network model, and the second image features obtained by the second network are obtained; the second image features include blood vessel density and fundus tissue thickness;

Fusing the first image feature with the second image feature to obtain a fusion feature;

Match the grade of maculopathy corresponding to the image according to the fusion feature.

The present application also provides a lesion detection device based on fundus images, including:

The fundus image module is used to obtain fundus screening images, and the fundus screening images include scanning images and contrast images;

The first network module is used to input the scanned image into the first network in the dual-channel network of the deep learning network model, and obtain the first image features obtained by the first network; the first image features include fundus curvature and reflectivity;

The second network module is configured to input the contrast image into the second network in the dual-channel network of the deep learning network model, and obtain the second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness;

A feature fusion module, configured to fuse the first image feature with the second image feature to obtain a fusion feature;

A grade matching module, configured to match the macular lesion grade corresponding to the image according to the fusion feature.

The present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, a fundus image-based lesion detection method is implemented, wherein the fundus image-based lesion detection method includes:

The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a method for detecting a lesion based on a fundus image is implemented, wherein the method for detecting a lesion based on a fundus image includes:

Beneficial effect

The present application example provides an image based on optical coherence tomography and angiography of optical coherence tomography, which can improve the precision and accuracy of macular degeneration identification and detection.

Description of drawings

Fig. 1 is a schematic flow chart of an embodiment of the lesion detection method based on fundus images of the present application;

FIG. 2 is a schematic structural diagram of an embodiment of a lesion detection device based on fundus images of the present application;

Fig. 3 is a schematic structural block diagram of an embodiment of the computer equipment of the present application.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

BEST MODE FOR CARRYING OUT THE INVENTION

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In this embodiment of the present application, data related to medical diagnosis can be acquired and processed based on an artificial intelligence deep learning model. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

Referring to Fig. 1, the embodiment of the present application provides a fundus image-based lesion detection method, including steps S10-S50. The detailed description of each step of the fundus image-based lesion detection method is as follows. The fundus image-based lesion detection method can be completed by an application program with corresponding functions, such as the built-in "fundus image macular degeneration detection and identification" function in the application program. Through this function, the application program can identify and determine whether macular degeneration occurs and grade macular degeneration from fundus screening images. The application program can run Running on the terminal device or in the cloud server, therefore, the method for detecting the lesion based on the fundus image can also be understood as being completed by the terminal device or the cloud server running the application program.

S10. Acquire a fundus screening image, where the fundus screening image includes a scan image and a contrast image.

This embodiment is applied to the scene of detecting whether macular degeneration occurs in the fundus and the grade of macular degeneration based on image recognition technology. First, the fundus screening image is obtained. The fundus screening image includes a scanned image and a contrast image. The scan image is an optical coherence tomography image (OCT for short), and the contrast image is an optical coherence tomography angiography image (OCTA for short). The features are different, based on the scanned image and the angiographic image, the fundus macular degeneration can be identified and detected more accurately.

S20. Input the scanned image to the first network in the dual-channel network of the deep learning network model, and acquire the first image features obtained by the first network; the first image features include fundus curvature and reflectivity.

In this embodiment, after the fundus screening image is obtained, and the fundus screening image includes a scan image and a contrast image, in this embodiment, the deep learning network model is used to learn the image features of the scan image and the contrast image to distinguish the macular lesion. The deep learning network is a dual-channel network, including a first network and a second network. The scan image and the contrast image are respectively received through two different networks, wherein the first network can extract features for judging maculopathy in the scan image, and the scan image is input to the first network of the dual-channel network of the deep learning network model. , acquiring the first image features obtained by the first network; the first image features include fundus curvature and reflectance, that is, converting the extracted first image features into fundus parameters, including fundus curvature and reflectance.

S30. Input the contrast image into the second network of the dual-channel network of the deep learning network model, and acquire the second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness.

In this embodiment, after the scanned image is input into the first network of the dual-channel network, the first image features obtained by the first network are obtained; after the first image features include fundus curvature and reflectivity, the contrast image is input to the second network of the dual-channel network of the deep learning network model, and the second image features obtained by the second network are obtained; the second image features include blood vessel density and fundus tissue thickness, wherein the second network can extract features for judging macular degeneration in the contrast image, and convert the extracted second image features into parameters of the fundus, including blood vessel density and fundus tissue thickness.

S40. Fusion the first image feature and the second image feature to obtain a fusion feature.

In this embodiment, after the scanned image is input to the first network of the dual-channel network, the first image features obtained by the first network are obtained; the first image features include fundus curvature and reflectivity, and the contrast image is input to the second network of the dual-channel network, and the second image features obtained by the second network are obtained; after the second image features include blood vessel density and fundus tissue thickness, in order to judge macular degeneration more accurately, the fundus features recognized by two different images are combined, that is, the first image feature and the second image feature are fused to obtain a fusion Features, the fusion features include the values of various fundus parameters for determining macular degeneration, including fundus curvature, reflectivity, blood vessel density, and fundus tissue thickness, etc., and the multiple fundus parameters are obtained by fusing the first image feature and the second image feature, that is, the second image feature also includes fundus curvature and reflectance, and the fundus curvature and reflectance of the first image feature are corrected based on the fundus curvature and reflectance in the second image feature, so as to obtain the fusion feature.

S50. Match the maculopathy grade corresponding to the image according to the fusion feature.

In this embodiment, after the first image feature is fused with the second image feature to obtain the fusion feature, the grade of macular degeneration corresponding to the image is matched according to the fusion feature, the image features representing macular degeneration extracted from the optical coherence tomography image and the optical coherence tomography angiography image are fused, and the corresponding macular degeneration grade is matched according to the fusion feature. In one embodiment, each grade of macular degeneration is preset with a numerical range of each fundus parameter. When the range of the fundus parameters is within the range, the fusion feature is matched with the corresponding grade of macular degeneration, thereby improving the precision and accuracy of macular degeneration identification and detection.

This embodiment provides a method based on optical coherence tomography images and optical coherence tomography angiography images, combined with a deep dual-channel neural network algorithm for fundus macular lesions for identification and graded detection. Firstly, a fundus screening image is obtained. The fundus screening image includes a scan image and a contrast image, and then the scan image is input to the first network in the dual-channel network of the deep learning network model, and the first image features obtained by the first network are acquired; the first image features include fundus curvature and reflectance, and the contrast images are input to the deep learning network. The second network in the dual-channel network of the model obtains the second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness, and the first image features are fused with the second image features to obtain fusion features. The fundus curvature and reflectance in the first image feature are corrected for the fundus curvature and reflectance to obtain a fusion feature; the macular degeneration level corresponding to the image is matched according to the fusion feature, thereby improving the recognition and detection of macular degeneration. Accuracy and accuracy.

In one embodiment, the matching the maculopathy grade corresponding to the image according to the fusion feature includes:

Calculate the loss degree of the fusion feature and the standard feature, and match the maculopathy grade according to the loss degree.

In this embodiment, in the process of matching the macular lesion grade corresponding to the image according to the fusion feature, the fusion feature is compared with the standard feature, and then the loss degree of the fusion feature and the standard feature is calculated, and then the macular lesion grade is matched according to the loss degree, wherein, in the fusion feature, the loss degree of the first image feature and the second image feature corresponding to the standard feature are of the same importance. Then add the first loss degree and the second loss degree to obtain the loss degree of the fusion feature and the standard feature, thereby accurately calculating and matching the features of the two images, and improving the accuracy of macular lesion recognition.

In one embodiment, the fusion of the first image feature and the second image feature to obtain the fusion feature includes:

acquiring the first fundus curvature and first reflectance of the first image feature, and acquiring the second fundus curvature and second reflectance of the second image feature;

Obtaining the first blood vessel density and the first fundus tissue thickness of the first image feature, and acquiring the second blood vessel density and the second fundus tissue thickness of the second image feature;

Obtain the fusion ratio of each feature in the first image feature and the second image feature;

According to the fusion ratio, the first fundus curvature and the first reflectance, the second fundus curvature and the second reflectance, the first blood vessel density and the first fundus tissue thickness, and the second blood vessel density and the second fundus tissue thickness are fused to obtain fusion features.

In this embodiment, in the process of fusing the first image feature with the second image feature to obtain the fused feature, first obtain the first fundus curvature and the first reflectance of the first image feature, obtain the second fundus curvature and the second reflectance of the second image feature, obtain the first blood vessel density and the first fundus tissue thickness of the first image feature, obtain the second blood vessel density and the second fundus tissue thickness of the second image feature, and then obtain the fusion ratio of the first image feature and each feature in the second image feature, that is, the ratio of each feature in the first image feature and the ratio of each feature in the second image feature, The second image feature also includes fundus curvature and reflectivity, and the first image feature also includes blood vessel density and fundus tissue thickness. Then, according to the fusion ratio, the first fundus curvature and the first reflectance, the second fundus curvature and the second reflectance, the first blood vessel density and the first fundus tissue thickness, the second blood vessel density and the second fundus tissue thickness are fused to obtain fusion features, and the recognized fundus features of two different images are combined to improve the accuracy of the fused features.

In one embodiment, after the fundus screening image is acquired, it includes:

Acquiring a first redundant image of the surrounding parts of the fundus of the fundus screening image;

Deleting the first redundant image from the fundus screening image to obtain a fundus screening image to be selected;

Obtaining a fovea area image of the macula from the fundus screening image to be selected;

In the fundus screening image to be selected, the image of the fovea area is randomly flipped to obtain a target fundus screening image.

In this embodiment, after the fundus screening image is obtained, the fundus screening image needs to be enhanced, including the enhancement of the scanned image and the contrast image. Specifically, the first redundant image of the fundus surrounding part of the fundus screening image is obtained, the first redundant image is deleted from the fundus screening image to obtain a fundus screening image to be selected, and then the fovea region image of the macula is obtained from the fundus screening image to be selected, and then the image of the fovea region is randomly flipped in the fundus screening image to be selected. The enhancement is performed to obtain the target fundus screening image, thereby improving the recognition accuracy of the fundus screening image.

In one embodiment, after performing random flipping on the fovea region image in the fundus screening image to be selected to obtain the target fundus screening image, further comprising:

Invoking regional dynamic histogram equalization to process the target fundus screening image to obtain an equalized target fundus screening image;

Invoking a Laplacian filter to enhance the equalized target fundus screening image, so as to enhance the characteristics of fundus curvature, reflectivity, blood vessel density and thickness of each layer of tissue in the target fundus screening image, and obtain the enhanced target fundus screening image.

In this embodiment, the fovea area image is randomly flipped in the fundus screening image to be selected, and after the target fundus screening image is obtained, an image enhancement is performed on the fundus screening graphic. Further, the contrast of the image after the first enhancement may be low, and the image is enhanced for the second time. Specifically, the target fundus screening image is processed by calling regional dynamic histogram equalization to obtain an equalized target fundus screening image, and then the Laplace filter is called to enhance the equalized target fundus screening image. By enhancing the characteristics of the fundus curvature, reflectivity, blood vessel density and thickness of each layer of tissue in the target fundus screening image, the enhanced target fundus screening image is obtained, so as to continue to enhance the key details in the fundus screening image, such as fundus curvature, reflectivity, blood vessel density and the thickness of each layer of tissue, so as to improve the accuracy of fundus screening image recognition.

In one embodiment, before the first network in the two-channel network of the described scanning image input to deep learning network model, also comprise:

Constructing a dual-channel network with two branch networks, the two branch networks comprising a first network and a second network;

Constructing several branches of the first network and several branches of the second network respectively, and configuring a filter function of each branch; information can be transmitted between the branches of the first network and the second network;

The feature information of the scan image or the contrast image is enhanced based on the filter function.

In this embodiment, before the scanned image is input to the first network of the dual-channel network of the deep learning network model, a dual-channel network with two branch networks is constructed. The two branch networks include the first network and the second network, and the features of different images are extracted through two different branch networks, and then several branches of the first network and several branches of the second network are respectively constructed, and a filter function of each branch is configured; information can be transferred between the branches of the first network and the second network, so that the features of the first image and the second image can be better integrated, and then based on the filter function. The feature information of the scanned image or the angiographic image is enhanced, thereby improving the accuracy of image recognition.

In one embodiment, after constructing the several branches of the first network and the several branches of the second network respectively, and configuring the filter function of each branch, it further includes:

Obtaining information channels contained in a filter corresponding to the filter function;

Obtain a first channel parameter of the first information channel and a second channel parameter of the second information channel;

judging whether the first channel parameter and the second channel parameter are constrained to grow centripetally in the parameter hyperspace;

If so, adding the second information channel to the second information channel to prune the first network or the second network, where the second information channel is an information channel after the first information channel.

In this embodiment, after constructing several branches of the first network and several branches of the second network, and configuring the filter function of each branch, obtain the information channels contained in the filter corresponding to the filter function, each filter contains multiple channels for the access and transmission of feature data, and then obtain the first channel parameters of the first information channel and the second channel parameters of the second information channel, wherein the first information channel and the second information channel represent two different channels in the filter, and the first information channel is located in front of the second information channel, and the first information channel transmits the calculated feature data to the second information channel Channel, further, judge whether the first channel parameter and the second channel parameter are constrained to grow centripetally in the parameter hyperspace, and if so, add the second information channel to the second information channel to prune the first network or the second network, wherein the second information channel is an information channel after the first information channel, when multiple filters are constrained to grow centripetally in the parameter hyperspace, although they begin to generate more and more similar information, the information transmitted by the corresponding input channel of the next layer is still fully used, and the channels of the next layer can be fused, thereby reducing memory usage, power consumption and The required floating-point operation improves calculation efficiency and improves the screening efficiency of lesions in fundus images.

Referring to Figure 2, the present application also provides a lesion detection device based on fundus images, including:

The fundus image module 10 is used to obtain fundus screening images, and the fundus screening images include scanned images and contrast images;

The first network module 20 is configured to input the scanned image to the first network in the dual-channel network, and obtain the first image features obtained by the first network; the first image features include fundus curvature and reflectivity;

The second network module 30 is configured to input the contrast image into the second network in the dual-channel network, and obtain the second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness;

A feature fusion module 40, configured to fuse the first image feature with the second image feature to obtain a fusion feature;

A grade matching module 50, configured to match the macular lesion grade corresponding to the image according to the fusion feature.

As mentioned above, it can be understood that each component of the fundus image-based lesion detection device proposed in this application can realize the function of any one of the above-mentioned fundus image-based lesion detection methods.

In one embodiment, after the fundus screening image is acquired, it includes:

Referring to FIG. 3 , an embodiment of the present application also provides a computer device, which may be a mobile terminal, and its internal structure may be as shown in FIG. 3 . The computer equipment includes a processor, a memory, a network interface, and a display device and an input device connected through a system bus. Wherein, the network interface of the computer device is used to communicate with external terminals through a network connection. The input device of the computer equipment is used for receiving user's input. The computer is designed with a processor to provide computing and control capabilities. The memory of the computer device includes storage media. The storage medium stores an operating system, computer programs and databases. The database of the computer device is used to store data. When the computer program is executed by the processor, a lesion detection method based on the fundus image is realized.

The aforementioned processor executes the aforementioned lesion detection method based on a fundus image, comprising: acquiring a fundus screening image, the fundus screening image including a scan image and a contrast image; inputting the scan image to a first network in a dual-channel network, and obtaining first image features obtained by the first network; the first image features include fundus curvature and reflectivity; inputting the contrast image to a second network in the dual-channel network, and obtaining second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness; Perform fusion to obtain fusion features; match the macular lesion grade corresponding to the image according to the fusion features.

The computer equipment provides a method based on optical coherence tomography images and optical coherence tomography angiography images, combined with a deep dual-channel neural network algorithm for fundus macular lesions for identification and grading detection. Firstly, a fundus screening image is obtained. Network, to obtain the second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness, the first image features and the second image features are fused to obtain fusion features, the fusion features include the values of various fundus parameters for determining macular degeneration, including fundus curvature, reflectivity, blood vessel density, and fundus tissue thickness, etc. Correct the fundus curvature and reflectivity of the first image feature to obtain the fusion feature; match the macular degeneration level corresponding to the image according to the fusion feature, thereby improving the accuracy and accuracy of macular degeneration identification and detection.

An embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium may be non-volatile or volatile, and a computer program is stored thereon. When the computer program is executed by the processor, a fundus image-based lesion detection method is implemented, comprising the steps of: acquiring a fundus screening image, the fundus screening image including a scanned image and a contrast image; inputting the scanned image into a first network in a dual-channel network, and obtaining a first image feature obtained by the first network; the first image feature includes fundus curvature and reflectivity; The contrast image is input to the second network in the dual-channel network, and the second image feature obtained by the second network is obtained; the second image feature includes blood vessel density and fundus tissue thickness; the first image feature is fused with the second image feature to obtain a fusion feature; and the macular degeneration grade corresponding to the image is matched according to the fusion feature.

The computer-readable storage medium provides a method for identifying and grading detection of fundus macular lesions based on optical coherence tomography images and optical coherence tomography angiography images, combined with a deep dual-channel neural network algorithm. First, a fundus screening image is obtained, and the fundus screening image includes a scan image and a contrast image, and then the scan image is input to a first network in a dual-channel network to obtain first image features obtained by the first network; the first image features include fundus curvature and reflectance, and the contrast images are input to the dual-channel network. The second network in the second network obtains the second image feature obtained by the second network; the second image feature includes blood vessel density and fundus tissue thickness, and the first image feature is fused with the second image feature to obtain a fusion feature. The fusion feature includes the values of various fundus parameters for determining macular degeneration, including fundus curvature, reflectivity, blood vessel density, and fundus tissue thickness. Correct the fundus curvature and reflectance of the first image feature to obtain the fusion feature; match the macular degeneration grade corresponding to the image according to the fusion feature, thereby improving the accuracy and accuracy of macular degeneration identification and detection.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

Any reference to memory, storage, database or other media provided herein and used in the examples may include non-volatile or volatile memory.

Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM), among others.

Claims

A lesion detection method based on fundus images, including:

Obtaining a fundus screening image, the fundus screening image including a scan image and an angiographic image;

The scanned image is input to the first network in the dual-channel network of the deep learning network model, and the first image features obtained by the first network are obtained; the first image features include fundus curvature and reflectivity;

The contrast image is input to the second network in the dual-channel network of the deep learning network model, and the second image features obtained by the second network are obtained; the second image features include blood vessel density and fundus tissue thickness;

Fusing the first image feature with the second image feature to obtain a fusion feature;

Match the grade of maculopathy corresponding to the image according to the fusion feature.
The lesion detection method based on the fundus image according to claim 1, wherein said matching the macular lesion grade corresponding to the image according to the fusion feature comprises:

Calculate the loss degree of the fusion feature and the standard feature, and match the maculopathy grade according to the loss degree.
The lesion detection method based on fundus images according to claim 1, wherein said fusing said first image features with second image features to obtain fusion features includes:

acquiring the first fundus curvature and first reflectance of the first image feature, and acquiring the second fundus curvature and second reflectance of the second image feature;

Obtaining the first blood vessel density and the first fundus tissue thickness of the first image feature, and acquiring the second blood vessel density and the second fundus tissue thickness of the second image feature;

Obtain the fusion ratio of each feature in the first image feature and the second image feature;

According to the fusion ratio, the first fundus curvature and the first reflectance, the second fundus curvature and the second reflectance, the first blood vessel density and the first fundus tissue thickness, and the second blood vessel density and the second fundus tissue thickness are fused to obtain fusion features.
The lesion detection method based on the fundus image according to claim 1, wherein, after the acquisition of the fundus screening image, comprising:

Acquiring a first redundant image of the surrounding parts of the fundus of the fundus screening image;

Deleting the first redundant image from the fundus screening image to obtain a fundus screening image to be selected;

Obtaining a fovea area image of the macula from the fundus screening image to be selected;

In the fundus screening image to be selected, the image of the fovea area is randomly flipped to obtain a target fundus screening image.
The lesion detection method based on a fundus image according to claim 4, wherein, after randomly flipping the image of the fovea region in the fundus screening image to be selected, after obtaining the target fundus screening image, further comprising:

Invoking regional dynamic histogram equalization to process the target fundus screening image to obtain an equalized target fundus screening image;

Invoking a Laplacian filter to enhance the equalized target fundus screening image, so as to enhance the characteristics of fundus curvature, reflectivity, blood vessel density and thickness of each layer of tissue in the target fundus screening image, and obtain the enhanced target fundus screening image.
The lesion detection method based on fundus image according to claim 1, wherein, before the first network in the dual-channel network of the described scanned image input to the deep learning network model, further comprising:

Constructing a dual-channel network with two branch networks, the two branch networks comprising a first network and a second network;

Constructing several branches of the first network and several branches of the second network respectively, and configuring a filter function of each branch; information can be transmitted between the branches of the first network and the second network;

The feature information of the scan image or the contrast image is enhanced based on the filter function.
The lesion detection method based on fundus images according to claim 6, wherein, after constructing several branches of the first network and several branches of the second network respectively, and configuring the filter function of each branch, further comprising:

Obtaining information channels contained in a filter corresponding to the filter function;

Obtain a first channel parameter of the first information channel and a second channel parameter of the second information channel;

judging whether the first channel parameter and the second channel parameter are constrained to grow centripetally in the parameter hyperspace;

If so, adding the second information channel to the second information channel to prune the first network or the second network, where the second information channel is an information channel after the first information channel.
A lesion detection device based on fundus images, including:

The fundus image module is used to obtain fundus screening images, and the fundus screening images include scanning images and contrast images;

The first network module is used to input the scanned image into the first network in the dual-channel network of the deep learning network model, and obtain the first image features obtained by the first network; the first image features include fundus curvature and reflectivity;

The second network module is configured to input the contrast image into the second network in the dual-channel network of the deep learning network model, and obtain the second image features obtained by the second network; the second image features include blood vessel density and fundus tissue thickness;

A feature fusion module, configured to fuse the first image feature with the second image feature to obtain a fusion feature;

A grade matching module, configured to match the macular lesion grade corresponding to the image according to the fusion feature.
A computer device, comprising a memory and a processor, the memory stores a computer program, wherein, when the processor executes the computer program, a method for detecting a lesion based on a fundus image is realized:

Wherein, the described lesion detection method based on fundus image comprises:

Obtaining a fundus screening image, the fundus screening image including a scan image and an angiographic image;

The scanned image is input to the first network in the dual-channel network of the deep learning network model, and the first image features obtained by the first network are obtained; the first image features include fundus curvature and reflectivity;

The contrast image is input to the second network in the dual-channel network of the deep learning network model, and the second image features obtained by the second network are obtained; the second image features include blood vessel density and fundus tissue thickness;

Fusing the first image feature with the second image feature to obtain a fusion feature;

Match the grade of maculopathy corresponding to the image according to the fusion feature. 10. The computer device according to claim 9, wherein said matching the grade of macular degeneration corresponding to said image according to said fusion feature comprises:

Calculate the loss degree of the fusion feature and the standard feature, and match the maculopathy grade according to the loss degree.
The computer device according to claim 9, wherein said merging the first image feature and the second image feature to obtain the fused feature comprises:

acquiring the first fundus curvature and first reflectance of the first image feature, and acquiring the second fundus curvature and second reflectance of the second image feature;

Obtaining the first blood vessel density and the first fundus tissue thickness of the first image feature, and acquiring the second blood vessel density and the second fundus tissue thickness of the second image feature;

Obtain the fusion ratio of each feature in the first image feature and the second image feature;

According to the fusion ratio, the first fundus curvature and the first reflectance, the second fundus curvature and the second reflectance, the first blood vessel density and the first fundus tissue thickness, and the second blood vessel density and the second fundus tissue thickness are fused to obtain fusion features.
The computer device according to claim 9, wherein, after the acquisition of the fundus screening image, comprising:

Acquiring a first redundant image of the surrounding parts of the fundus of the fundus screening image;

Deleting the first redundant image from the fundus screening image to obtain a fundus screening image to be selected;

Obtaining a fovea area image of the macula from the fundus screening image to be selected;

In the fundus screening image to be selected, the image of the fovea area is randomly flipped to obtain a target fundus screening image.
The computer device according to claim 12, wherein, after randomly flipping the fovea region image in the fundus screening image to be selected to obtain the target fundus screening image, further comprising:

Invoking regional dynamic histogram equalization to process the target fundus screening image to obtain an equalized target fundus screening image;

Invoking a Laplacian filter to enhance the equalized target fundus screening image, so as to enhance the characteristics of fundus curvature, reflectivity, blood vessel density and thickness of each layer of tissue in the target fundus screening image, and obtain the enhanced target fundus screening image.
The computer device according to claim 9, wherein, before the first network in the dual-channel network of the deep learning network model, the scanned image further comprises:

Constructing a dual-channel network with two branch networks, the two branch networks comprising a first network and a second network;

Constructing several branches of the first network and several branches of the second network respectively, and configuring a filter function of each branch; information can be transmitted between the branches of the first network and the second network;

The feature information of the scan image or the contrast image is enhanced based on the filter function.
The computer device according to claim 14, wherein, after constructing several branches of the first network and several branches of the second network respectively, and configuring the filter function of each branch, further comprising:

Obtaining information channels contained in a filter corresponding to the filter function;

Obtain a first channel parameter of the first information channel and a second channel parameter of the second information channel;

judging whether the first channel parameter and the second channel parameter are constrained to grow centripetally in the parameter hyperspace;

If so, adding the second information channel to the second information channel to prune the first network or the second network, where the second information channel is an information channel after the first information channel.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, a fundus image lesion detection method is implemented: acquiring a fundus screening image, and the fundus screening image includes a scanned image and a contrast image;

The scanned image is input to the first network in the dual-channel network of the deep learning network model, and the first image features obtained by the first network are obtained; the first image features include fundus curvature and reflectivity;

The contrast image is input to the second network in the dual-channel network of the deep learning network model, and the second image features obtained by the second network are obtained; the second image features include blood vessel density and fundus tissue thickness;

Fusing the first image feature with the second image feature to obtain a fusion feature;

Match the grade of maculopathy corresponding to the image according to the fusion feature.
The computer-readable storage medium according to claim 16, wherein the matching the macular degeneration grade corresponding to the image according to the fusion feature comprises:

Calculate the loss degree of the fusion feature and the standard feature, and match the maculopathy grade according to the loss degree.
The computer-readable storage medium according to claim 16, wherein said merging the first image feature with the second image feature to obtain the fused feature comprises:

acquiring the first fundus curvature and first reflectance of the first image feature, and acquiring the second fundus curvature and second reflectance of the second image feature;

Obtaining the first blood vessel density and the first fundus tissue thickness of the first image feature, and acquiring the second blood vessel density and the second fundus tissue thickness of the second image feature;

Obtain the fusion ratio of each feature in the first image feature and the second image feature;

According to the fusion ratio, the first fundus curvature and the first reflectance, the second fundus curvature and the second reflectance, the first blood vessel density and the first fundus tissue thickness, and the second blood vessel density and the second fundus tissue thickness are fused to obtain fusion features.
The computer-readable storage medium according to claim 16, wherein, after acquiring the fundus screening image, comprising:

Acquiring a first redundant image of the surrounding parts of the fundus of the fundus screening image;

Deleting the first redundant image from the fundus screening image to obtain a fundus screening image to be selected;

Obtaining a fovea region image of the macula from the fundus screening image to be selected;

In the fundus screening image to be selected, the image of the fovea area is randomly flipped to obtain a target fundus screening image.
The computer-readable storage medium according to claim 16, wherein, after randomly flipping the image of the fovea region in the fundus screening image to be selected, after obtaining the target fundus screening image, further comprising:

Invoking regional dynamic histogram equalization to process the target fundus screening image to obtain an equalized target fundus screening image;

Invoking a Laplacian filter to enhance the equalized target fundus screening image, so as to enhance the characteristics of fundus curvature, reflectivity, blood vessel density and thickness of each layer of tissue in the target fundus screening image, and obtain the enhanced target fundus screening image.