WO2021169128A1 - Method and apparatus for recognizing and quantifying fundus retina vessel, and device and storage medium - Google Patents

Abstract

A method and apparatus for recognizing and quantifying a fundus retina vessel, and a device and a storage medium, applicable to the field of precision medicines. The method comprises: inputting an original fundus image into a pre-trained U-shaped convolutional neural network model for processing, to obtain a target feature map; performing optic disk segmentation based on the target feature map; segmenting the original fundus image to obtain the arteriovenous vessel recognition result; carrying out region-of-interest positioning based on the optic disk segmentation result; extracting a vessel centerline according to the arteriovenous vessel recognition result, detecting key points in the vessel centerline, removing the key points to obtain a plurality of mutually independent vessel segments, and correcting arteriovenous category information on each vessel segment; and obtaining the vessel diameter of each vessel segment after category information correction based on the extracted vessel centerline, and then quantifying arteriovenous vessels in a region of interest. The method facilitates improving the fundus retina artery and vein vessel identification precision, thereby improving the quantization precision.

Description

Method, device, equipment and storage medium for recognizing and quantifying fundus retinal blood vessels

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on February 29, 2020, the application number is 202010134390.7, and the invention title is "Fundus Retinal Vessel Recognition and Quantification Method, Apparatus, Equipment, and Storage Medium", and its entire content Incorporated in this application by reference.

Technical field

This application relates to the field of computer vision technology, and in particular to a method, device, equipment and storage medium for recognizing and quantifying retinal blood vessels in the fundus.

Background technique

Fundus retinal arteriovenous blood vessels have always been a key area of medical research, especially those within the range of 1pd-1.5pd (Papillary Diameter) from the center of the optic disc. The ratio of the diameters of the two or the morphological changes are many. The basis for early diagnosis of systemic and hematological diseases, such as cardiovascular disease, diabetes, and hypertension. Calculating the diameter ratio of the fundus retinal arteriovenous vessels requires precise classification of arteriovenous vessels. The traditional fundus retinal diagnosis relies on the doctor to observe the fundus image and obtain the diagnosis result through his own medical experience. This method takes a long time and The workload is large, and the recognition and classification of arteriovenous vessels is greatly affected by subjectivity. With the development of computer image processing technology, the fundus color photographs are currently used to extract the retinal blood vessels of the fundus. Because the color fundus photographs have the characteristics of uneven brightness, interlaced blood vessels and background colors, and small differences between arteries and veins, they can be used to identify arteries and veins. Classification causes certain difficulties, and the existing research on automatic recognition of arteriovenous blood vessels mainly realizes local arteriovenous blood vessel recognition based on color or partial structure of blood vessels, but the inventor realizes that the recognition accuracy is still limited, which leads to lower quantification accuracy.

Summary of the invention

The embodiments of the present application provide a method, device, equipment, and storage medium for recognizing and quantifying retinal blood vessels in the fundus, which is beneficial to improve the recognition accuracy of retinal arteriovenous blood vessels in the fundus, thereby improving the quantification accuracy.

In the first aspect, the embodiments of the present application provide a method for identifying and quantifying retinal blood vessels in the fundus, and the method includes:

Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.

In a second aspect, an embodiment of the present application provides a device for identifying and quantifying retinal blood vessels in the fundus, which includes:

The feature extraction module is used to input the original fundus image into the pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

A video disc segmentation module, configured to perform video disc segmentation based on the target feature map to obtain a video disc segmentation result;

The blood vessel recognition module is used to segment the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous blood vessel recognition result;

The region positioning module is used to locate the region of interest based on the result of the segmentation of the optic disc, and obtain the region of interest positioning result;

The centerline extraction module is used to extract the centerline of the blood vessel according to the result of the arteriovenous blood vessel identification, use the neighborhood connectivity determination method to detect the key points in the centerline of the blood vessel, remove the key points to obtain multiple independent blood vessel segments, and Correct the arterial and venous category information on each blood vessel segment, and obtain each blood vessel segment with the revised arterial and venous category information;

The diameter ratio calculation module is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information by using the method of boundary detection based on the corrected blood vessel center line of the arteriovenous category information, and calculate the sensation based on the blood vessel diameter The ratio of the diameters of arteries and veins in the region of interest.

In a third aspect, an embodiment of the present application provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored on the memory and running on the processor, and the processor executes the Realize in computer program:

Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.

In the fourth aspect, the embodiments of the present application provide a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. The steps in the method.

In the embodiment of the present application, feature extraction of the original fundus image is performed to obtain target feature maps of multiple scales, and the optic disc segmentation is performed based on the obtained target feature maps to obtain the results of the optic disc segmentation. The pre-trained cascaded segmentation network model is used to analyze the original fundus image. Perform segmentation to obtain the results of arteriovenous and vascular recognition, locate the region of interest based on the results of the optic disc segmentation, extract the centerline of the blood vessel based on the result of the arteriovenous and vascular recognition, and use the neighborhood connectivity determination method to detect the key points in the centerline of the blood vessel and remove the key Point to obtain multiple independent blood vessel segments, and correct the arterial and vein category information on each blood vessel segment. Based on the extracted blood vessel centerline, the boundary detection method is used to obtain the blood vessel diameter of each blood vessel segment after the category information is corrected. According to the obtained blood vessel diameter, the ratio of the diameter of the arterial blood vessel to the venous blood vessel in the region of interest is calculated, thereby improving the recognition accuracy of the fundus retinal arteriovenous blood vessel, thereby improving the quantification accuracy.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Figure 1 is an application architecture diagram provided by an embodiment of the application;

FIG. 2 is a schematic flowchart of a method for identifying and quantifying fundus retinal blood vessels according to an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a video disk segmentation network provided by an embodiment of this application;

Figure 4-a is a schematic diagram of a down-sampling process provided by an embodiment of the application;

Fig. 4-b is an example diagram of a part of a U-shaped convolutional neural network model encoder provided by an embodiment of the application;

Figure 5-a is a schematic diagram of an up-sampling process provided by an embodiment of the application;

Fig. 5-b is an example diagram of a decoder part of a U-shaped convolutional neural network model provided by an embodiment of the application;

FIG. 6 is an example diagram of a video disc segmentation provided by an embodiment of the application; FIG.

FIG. 7 is an example diagram of a cascaded segmentation network model provided by an embodiment of this application;

FIG. 8 is an example diagram of a desired output area A and an actual output area B of a U-shaped segmentation network provided by an embodiment of the application;

FIG. 9 is a schematic diagram of a process of performing arteriovenous and vascular recognition based on original fundus images according to an embodiment of the application;

FIG. 10 is an example diagram of segmenting fundus image blocks according to an embodiment of this application;

FIG. 11-a is an example diagram of a location result of a region of interest provided by an embodiment of the application; FIG.

Fig. 11-b is an example diagram of a blood vessel centerline provided by an embodiment of the application;

FIG. 11-c is an example diagram of detection of key points on the centerline of a blood vessel according to an embodiment of the application; FIG.

FIG. 12 is a schematic structural diagram of a device for identifying and quantifying fundus retinal blood vessels according to an embodiment of the application;

FIG. 13 is a schematic structural diagram of an electronic device provided by an embodiment of this application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The terms "including" and "having" appearing in the specification, claims, and drawings of this application and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment. In addition, the terms "first", "second", "third", etc. are used to distinguish different objects, but not to describe a specific sequence.

First, in conjunction with related drawings, an example of the network system architecture to which the solution of the embodiment of the present application may be applied is introduced. Please refer to FIG. 1. FIG. 1 is an application architecture diagram provided by an embodiment of the application. As shown in FIG. 1, it includes an electronic device, an image acquisition device, and a database. The image acquisition device and the database are respectively connected and communicated with the electronic device through a network. The electronic device includes a processor. The electronic device can be any device that can realize the fundus retinal blood vessel identification and quantification method provided in the embodiments of the present application, such as a supercomputer in a medical research room, a computer in a hospital examination room, a server, etc. The image capture device can be any device that can capture fundus images, such as a color fundus camera, etc. In an application scenario, in the ophthalmological examination room of a hospital, when the image capture device captures the fundus image of the examinee, it will send it to the electronics through the network The device transmits the fundus image, and the electronic device executes the fundus retinal blood vessel identification and quantification method provided in this application to identify and quantify the fundus image, and output the quantization result. The database can be a local database or an external database. The so-called local database refers to the own database of enterprises, hospitals, and research laboratories. The so-called external database refers to the fundus image database commonly used at home and abroad, for example: STARE (structured analysis of the retinal) Database, DRIVE (digital retinal images for vessel extraction) database, etc., in another application scenario, when the staff in the laboratory need to test the fundus retinal blood vessel identification and quantification method provided in this application, they can use the network from the database The fundus image for testing is acquired in the, and the test operation is performed by the electronic device. Of course, when the above-mentioned database is a local database, the image acquisition device can establish a connection with the database and store the captured fundus images in the database.

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of a method for identifying and quantifying fundus retinal blood vessels according to an embodiment of the application. As shown in FIG. 2, it includes steps S21-S27:

S21: Obtain an original fundus image.

In the specific embodiment of the present application, the original fundus image may be a fundus image collected in real time by an image acquisition device anywhere, for example: fundus images of experimental subjects collected in a medical laboratory, fundus images of patients collected in a hospital examination room, etc., of course The original fundus image can also be a fundus image in an open source database such as DRIVE, which is not specifically limited.

S22: Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales.

In the specific embodiment of the present application, FIG. 3 is a schematic structural diagram of a disc segmentation network, where the main part of the disc segmentation network is a U-shaped convolution on the left side (from the input of the original fundus image to the output of the target feature map) The neural network model, in addition to the most basic input layer and output layer, the U-shaped convolutional neural network model includes multiple hidden layers, and the multiple hidden layers are symmetrical, forming the encoder part of the U-shaped convolutional neural network model (below Sampling part) and decoder part (upsampling part).

In an embodiment, the input of the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales includes:

A: Input the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map;

Among them, as shown in Figure 4-a, step A includes:

S41: Perform convolution processing on the original fundus image to extract key features, and obtain a feature map with the same size as the original fundus image;

S42: Perform a maximum pooling operation on the feature map obtained through convolution processing, reduce the size of the feature map layer by layer, and obtain the high-dimensional feature map through alternating processing of several convolutional layers and pooling layers.

In the specific embodiment of the present application, the key feature is a feature with strong characterization ability, for example, a feature with a large pixel value. As shown in Figure 4-b, the original fundus image is input into the encoder part of the U-shaped convolutional neural network model, and first passes the conv layer conv for convolution processing, and the original fundus image is the same size as the original fundus image after the initial convolution operation. The feature map of C1 in Figure 3, and then the feature map C1 goes through the maximum pooling layer Max-pooling for down-sampling and maximum pooling operation, reducing the size of the feature map C1 to get the feature map C2, and the feature map C2 goes through the convolutional layer Conv and the maximum pooling layer Max-pooling are processed to obtain the feature map C3, so after the convolutional layer conv and the maximum pooling layer Max-pooling alternate processing, a low-resolution high-dimensional feature map is obtained, forming a low-dimensional feature map To the high-dimensional feature pyramid. Among them, the convolution processing of the encoder part can use a 3*3 convolution kernel, and the step size can be 2. During the maximum pooling operation, the feature map is reduced in size by a preset multiple. For example, if the downsampling is performed by 2 times , The size of the feature map C1 is 48*48, the feature map C2 is obtained after a maximum pooling operation, then the size of the feature map C2 is 24*24, and the size of the feature map C3 is 12*12 in the same way.

B: Input the high-dimensional feature map into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and output multiple scale target feature maps.

Among them, as shown in Figure 5-a, step B includes:

S51: Perform an up-sampling operation on the high-dimensional feature map, and enlarge the size of the high-dimensional feature map layer by layer;

S52. Combine the low-dimensional features extracted from each network layer in the encoding stage with the high-dimensional features extracted symmetrically in the decoding stage through the jump connection layer, to obtain an initial feature map of each network layer, and the initial feature map of each network layer Different scales;

S53: Output the initial feature map of each network layer through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.

In the specific embodiment of the present application, as shown in Figure 5-b, the decoder part of the U-shaped convolutional neural network model performs an up-sampling operation on the high-dimensional feature map obtained by the down-sampling of the encoder part, and also enlarges the high by a preset multiple. The size of the dimensional feature map, such as the feature map P5 in Figure 3, is enlarged to the size of the feature map P4 after an upsampling operation. If the original size of the feature map P5 is 12*12, the size of the feature map P4 is 24 *24, the specific up-sampling can be performed by commonly used interpolation methods, such as nearest neighbor interpolation, bilinear interpolation, mean interpolation, etc. The specifics are not limited. After each high-dimensional feature map with an enlarged size is obtained, the low-dimensional feature map extracted in the encoding stage of the same network layer and the corresponding high-dimensional feature map are merged. For example, the feature map C2 and the feature map P2 in Figure 3 belong to the same network Layer feature maps extracted symmetrically in the encoding stage and the decoding stage, through the skip-connection layer (skip-connection) to merge C2 and P2 to get the initial feature map of the network layer, because there will be multiple network layer merging operations, so , Will get the initial feature map of multiple scales. At this point, we output the initial feature map of each network layer by adding the output branch of the attention mechanism to obtain multiple scales of target feature maps. Among them, the attention mechanism is the SE module, and the SE module is operated by Squeeze , Excitation operation, Reweight operation complete the recalibration of the initial feature map of multiple scales in the channel dimension, so that the output target feature map has a better effect, and it can solve the problem of unclear optic disc boundaries and different sizes of optic disc areas. problem.

S23: Perform optic disc segmentation based on the target feature map to obtain a optic disc segmentation result.

In the specific embodiment of this application, a two-stage optic disc segmentation network is used for optic disc segmentation, the first stage adopts U-shaped convolutional neural network model for feature extraction, and the second stage performs optic disc segmentation based on the features extracted in the first stage . Specifically, the segmentation of the optic disc based on the output target feature map to obtain the result of the optic disc segmentation includes: fusing the output target feature map to obtain an image to be segmented, and performing candidate frame regression on the image to be segmented Processing to locate the position of the disc from the image to be segmented, and output the bounding box information of the disc, cut out the calibrated disc area image block according to the bounding box information of the disc, and input it into the pre-trained U-shaped segmentation network, after feature extraction and The up-sampling operation outputs the result of video disc segmentation.

As shown in Figure 3, after the U-shaped convolutional neural network model outputs multiple scales of target feature maps, they are fused to obtain a better resolution image to be segmented, and the image to be segmented is characterized by the candidate frame regression module ROIAlign Extraction, traverse each candidate area according to the extracted feature map, and do not quantize the floating-point number boundary, and then divide the candidate area into n*n rectangular units as shown in Figure 6, and do not quantize each unit boundary. Determine the four coordinate positions in each rectangular cell according to the fixed rules, calculate the values of these four positions by bilinear interpolation, and perform the maximum pooling operation to obtain the feature map on the right side of Figure 6, and then go through the Flatten The module performs flattening processing, outputs the bounding box box of the video disc, crops out the image block of the video disc area according to the bounding box box of the video disc, and inputs the cut out image block of the video disc area into the U-shaped segmentation network for video disc segmentation to obtain the final segmented video disc image. The two-stage optic disc segmentation model design helps to eliminate the interference of the fundus highlight noise caused by the shooting environment or poor shooting technology on the optic disc segmentation, and to obtain a higher precision optic disc segmentation result.

S24: Use a pre-trained cascaded segmentation network model to segment the original fundus image to obtain an arteriovenous and blood vessel recognition result.

In the specific embodiment of the present application, the arteriovenous and venous blood vessels are segmented and identified through the cascaded segmentation network model. In order to make the recognition accuracy of arteriovenous blood vessels higher, as shown in FIG. 7, a three-time U-shaped segmentation network cascade method is used here. They are the first U-shaped segmentation network, the second U-shaped segmentation network, and the third U-shaped segmentation network. The structure of each U-shaped segmentation network is shown in the enlarged view on the lower side of Figure 7. The left side is the feature extraction part, and the right The side is the up-sampling part, the left side uses 2*2 maximum pooling, the right side uses 2*2 inverse convolution, and both sides use 3*3 convolution kernels for feature extraction, the entire U-shaped segmentation network There is no fully connected layer, only the necessary convolutional layers are connected. In the entire training process of the cascaded segmentation network model, the loss function DiceLoss of each U-shaped segmentation network needs to be considered. First, calculate the Dice value, using the following formula:

Among them, as shown in Figure 8, Dice represents the degree of overlap between the expected output area A and the actual output area B of each U-shaped segmentation network. Smooth is the smoothing coefficient, which is set to 1 by default; each U-shaped segmentation is calculated according to the Dice value The loss function of the network uses the formula: L _K = DiceLoss = 1-Dice, L _k represents the value of the loss function of each U-shaped segmentation network, and finally, calculates the weighted average _{sum of the loss function L k of each U-shaped segmentation network,} To obtain the loss value of the entire preset cascade segmentation network model, the following formula is used:

Among them, Loss represents the loss value of the entire preset cascade segmentation network model, L _k represents the loss function of each U-shaped segmentation network, K represents the number of U-shaped segmentation networks in the preset cascade segmentation network model, where K is 3. indivual. The loss value of the entire preset cascade segmentation network model is calculated here to guide the optimization training of the preset cascade segmentation network model to obtain more accurate arteriovenous and vascular recognition results. The smaller the Loss value, the dynamic of the cascade segmentation network model. The venous vessel recognition result is more accurate.

In one embodiment, as shown in FIG. 9, the segmentation of the original fundus image using the pre-trained cascaded segmentation network model to obtain the arteriovenous vessel recognition result includes steps S91-S93:

S91, extracting the green channel image of the original fundus image, and performing histogram equalization processing on the green channel image to obtain a green channel image with enhanced contrast;

S92: Cut the contrast-enhanced green channel image into multiple fundus image blocks;

S93: Input the multiple fundus image blocks into a preset cascade segmentation network model for segmentation, and obtain an arteriovenous blood vessel recognition result.

For the acquired original fundus image in RGB format, first select the green channel image with obvious blood vessel structure for histogram equalization to enhance the contrast, and then use the patch operation to cut the contrast-enhanced green channel image into Two fundus image blocks Patch1 and Patch2, and then the two fundus image blocks Patch1 and Patch2 are input into the preset cascade segmentation network model for segmentation. Of course, the number of fundus image blocks cut out in actual operation is much larger, for example, there are Patch3, Patch4, Patch5, etc., each of the cut out fundus image blocks can have overlapping parts. Input the cut fundus image block into the first U-shaped segmentation network for feature extraction and up-sampling to obtain the first output result, use the first output result as the input of the second U-shaped segmentation network, perform feature extraction and up-sampling to obtain the second output result Output the result, take the second output result as the input of the third U-shaped segmentation network, perform feature extraction and up-sampling to obtain the arteriovenous blood vessel recognition result. The arteriovenous blood vessel recognition result includes the classification information of the arteriovenous blood vessel, for example: artery Blood vessels, venous blood vessel labels, etc.

S25: Perform locating the region of interest based on the result of the segmentation of the optic disc, and obtain the region of interest locating result.

In the specific embodiment of this application, the region of interest (ROI), that is, in machine vision and image processing, outlines the area to be processed from the processed image in the form of boxes, circles, ellipses, irregular polygons, etc. , Here specifically refers to the fundus area within the range of 1pd-1.5pd (Papillary Diameter) from the center of the optic disc. Based on the results of optic disc segmentation, ellipse fitting is performed on the optic disc boundary. Specifically, the least squares method can be used for ellipse fitting, and then the center of the optic disc is determined, and the fundus area within the range of 1pd-1.5pd is located with the center of the optic disc as the center. For the region of interest, the location result of the region of interest is obtained as a candidate region for subsequent pipe diameter measurement, as shown in Figure 11-a.

S26: Extract the blood vessel centerline according to the result of the arteriovenous blood vessel identification, use the neighborhood connectivity determination method to detect the key points in the blood vessel centerline, remove the key points to obtain a plurality of independent blood vessel segments, and compare each blood vessel segment The type information of the artery and vein is corrected, and each blood vessel segment after the correction of the type information of the artery and vein is obtained.

In the specific embodiment of this application, firstly, the results of arteriovenous blood vessel recognition are binarized to generate a fundus blood vessel binarized map, and the fundus blood vessel binarized map is input into the U-shaped segmentation network for the extraction of the centerline of the blood vessel, and the refined output is The centerline diagram of the blood vessel is shown in Figure 11-b. Then use the neighborhood connectivity determination method to detect the key points in the centerline of the blood vessel, such as branch points and intersections. The neighborhood connectivity determination method can be based on the 8-neighbor connectivity determination method, and N8(p) is used to represent the pixel. In the 8-neighborhood of p, for the pixel p and pixel q with pixel value x, if q is in the set N8(p), then it is determined that pixel p and pixel q are 8 connected, and it can be determined as shown in Figure 11-c. After the intersection and branch points are detected, the detected intersection and branch points are removed from the center line of the blood vessel to separate each blood vessel segment based on the blood vessel center line. Independence. The connectivity of each blood vessel segment is determined. After the connectivity of each blood vessel segment is determined, the arteriovenous type information on each blood vessel segment is counted, and then the arteriovenous type information on each blood vessel segment is corrected by voting method, and the information is obtained. Each blood vessel segment after the correction of the arteriovenous category information. For example, if a blood vessel segment has a majority of pixels marked with arterial information labels, other pixels marked with vein information are also marked as arterial information to ensure that the pixel category information on the same blood vessel segment is consistent. Compared with the traditional morphological refinement operation, the deep learning-based method for extracting the centerline of blood vessels avoids the manual design of complex thinning rules, while reducing false positive branches in the centerline of the blood vessel, enabling more accurate centerline extraction result.

S27: Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessel in the region of interest is calculated according to the blood vessel diameter. The ratio of the diameter of the vein to the diameter of the vein.

In the specific embodiment of the present application, based on the extraction of the blood vessel centerline in step S26, the method of boundary detection is used to calculate the blood vessel diameter of each blood vessel segment after the correction of the arteriovenous category information in the region of interest. Specifically, the blood vessel center point Is the center of the circle, traverse in a rectangular area with a pixel range of 40*40 to find the boundary point closest to the center line of each vessel segment after the correction of the arterial and vein category information, and the distance between the boundary point and the vessel center point is the radius r , The vessel diameter 2r of each vessel segment after the correction of the arteriovenous category information is obtained.

According to the blood vessel diameter of each blood vessel segment in the region of interest, the medical Parr-Hubbard-knudtson formula is used to calculate CRAE (centralretinalarteryequivalent, the equivalent value of central retinal artery blood vessel diameter) and CRVE (centralretinalveinequivalent, the equivalent value of central retinal vein blood vessel diameter) ), and then use the following formula to calculate the ratio of the diameters of arterial vessels and venous vessels in the region of interest:

Among them, AVR (ArterioletoVenule Ratio) represents the ratio of the diameter of arteries and veins in the region of interest,

A _i and A _j respectively represent the largest arterial vessel diameter and the smallest arterial vessel diameter of the region of interest obtained, and 0.88 is a fixed coefficient;

V _i and V _j respectively represent the maximum and minimum venous diameter of the region of interest obtained, 0.95 is a fixed coefficient, disease prediction and assessment can be made based on the calculated AVR value, which has certain clinical guiding significance.

It can be seen that the specific embodiment of the present application obtains target feature maps of multiple scales by extracting features from the original fundus image, performs optic disc segmentation based on the obtained target feature maps, and obtains the result of optic disc segmentation, using a pre-trained cascaded segmentation network The model segmented the original fundus image to obtain the results of arteriovenous and vascular recognition. Based on the results of the optic disc segmentation, the region of interest was located. The centerline of the blood vessel was extracted based on the result of the arteriovenous and vascular recognition. The neighborhood connectivity determination method was used to detect the key points in the centerline of the blood vessel. Point, remove the key point to obtain multiple independent blood vessel segments, and correct the arterial and vein category information on each blood vessel segment. Based on the extracted blood vessel centerline, use the method of boundary detection to obtain the corrected blood vessel category information Based on the obtained vessel diameter, the ratio of the diameter of the arterial vessel and the venous vessel in the region of interest is calculated based on the obtained vessel diameter, thereby improving the recognition accuracy of the fundus retinal arteriovenous blood vessel, thereby improving the quantification accuracy.

The present application also provides a device for identifying and quantifying fundus retinal blood vessels. The device for identifying and quantifying fundus retinal blood vessels may be a computer program (including program code) running in a terminal. The device for recognizing and quantifying retinal blood vessels in the fundus can execute the method shown in FIG. 2. Please refer to Figure 12, the device includes:

The feature extraction module 1201 is used to input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

The optic disc segmentation module 1202 is configured to perform optic disc segmentation based on the target feature map to obtain a optic disc segmentation result;

The blood vessel recognition module 1203 is configured to use a pre-trained cascade segmentation network model to segment the original fundus image to obtain an arteriovenous blood vessel recognition result;

The region positioning module 1204 is used for locating the region of interest based on the result of the segmentation of the optic disc, and obtaining the region of interest locating result;

The centerline extraction module 1205 is used to extract the centerline of the blood vessel according to the result of the arteriovenous blood vessel identification, use the neighborhood connectivity determination method to detect the key points in the centerline of the blood vessel, and remove the key points to obtain multiple independent blood vessel segments, And revise the arteriovenous type information on each blood vessel segment, and obtain each blood vessel segment after the arteriovenous type information is revised;

The diameter ratio calculation module 1206 is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information by using the method of boundary detection based on the corrected blood vessel center line of the arteriovenous type information, and calculate according to the blood vessel diameter The ratio of the diameters of arteries and veins in the region of interest.

Optionally, the feature extraction module 1201 is specifically used to input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales:

Input the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map;

The high-dimensional feature map is input into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and target feature maps of multiple scales are output.

Optionally, the feature extraction module 1201 performs key feature extraction in the encoder part of the U-shaped convolutional neural network model that inputs the original fundus image to obtain a high-dimensional feature map, which is specifically used for:

Performing convolution processing on the original fundus image to extract key features to obtain a feature map with the same size as the original fundus image;

A maximum pooling operation is performed on the feature map obtained through convolution processing, the size of the feature map is reduced layer by layer, and the high-dimensional feature map is obtained through alternate processing of several convolutional layers and pooling layers.

Optionally, the feature extraction module 1201 specifically uses the high-dimensional feature map to input the high-dimensional feature map into the decoder part of the U-shaped convolutional neural network model to perform an upsampling operation and output target feature maps of multiple scales. At:

Perform an up-sampling operation on the high-dimensional feature map, and enlarge the size of the high-dimensional feature map layer by layer;

The low-dimensional features extracted from each network layer in the encoding stage and the high-dimensional features extracted symmetrically in the decoding stage are merged through the jump connection layer to obtain the initial feature map of each network layer, and the initial feature map of each network layer has a different scale ；

The initial feature map of each network layer is output through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.

Optionally, the video disc segmentation module 1202 is specifically configured to perform video disc segmentation based on the target feature map to obtain a video disc segmentation result:

Fusing the target feature maps to obtain an image to be segmented;

Performing candidate frame regression processing on the image to be segmented, so as to locate the position of the optic disc from the image to be segmented, and output bounding box information of the optic disc;

According to the bounding box information of the video disc, cut out the calibrated video disc area image block and input it into the pre-trained U-shaped segmentation network, and output the segmentation result of the video disc after feature extraction and up-sampling operations.

Optionally, the blood vessel identification module 1203 is specifically used for segmenting the original fundus image by using a pre-trained cascaded segmentation network model to obtain an arteriovenous vessel recognition result:

Extracting the green channel image of the original fundus image, and performing histogram equalization processing on the green channel image to obtain a green channel image with enhanced contrast;

Cutting the contrast-enhanced green channel image into multiple fundus image blocks;

The multiple fundus image blocks are input into a preset cascade segmentation network model for segmentation, and an arteriovenous blood vessel recognition result is obtained.

Optionally, the diameter ratio calculation module 1206 is specifically used to obtain the diameter of each blood vessel segment after the correction of the arteriovenous category information using the method of boundary detection:

Take the blood vessel center point as the center, traverse in a rectangular area with a pixel range of 40*40 to find the boundary point closest to the center line of each blood vessel segment after the correction of the arteriovenous category information, and use the closest boundary point with the The distance of the blood vessel center point is the radius r, and the blood vessel diameter 2r of each blood vessel segment after the correction of the arteriovenous type information is obtained.

Optionally, the diameter ratio calculation module 1206 uses the following formula to calculate the diameter ratio of arterial blood vessels and venous blood vessels in the region of interest:

Among them, AVR represents the ratio of the diameter of arteries and veins in the region of interest, and CRAE represents the equivalent value of the diameter of the central retinal artery.

A _i and A _j respectively represent the maximum arterial vessel diameter and the minimum arterial vessel diameter in the region of interest obtained, 0.88 is a fixed coefficient; CRVE represents the equivalent value of the central retinal vein vessel diameter,

V _i and V _j represent the maximum venous vessel diameter and the minimum venous vessel diameter of the obtained region of interest, respectively, and 0.95 is a fixed coefficient.

It should be noted that each step in the method for identifying and quantifying fundus retinal blood vessels shown in FIG. 2 can be executed by each unit module in the device for identifying and quantifying fundus retinal blood vessels provided by the embodiments of the present application, and can achieve the same or Similar beneficial effects, the fundus retinal blood vessel identification and quantification device provided in the embodiments of the present application can be applied to the scene of fundus retinal blood vessel identification and quantification. Specifically, the above-mentioned fundus retinal blood vessel identification and quantification device can be applied to a server, a computer, or a mobile device. Terminals and other equipment.

According to an embodiment of the present application, the modules of the fundus retinal blood vessel identification and quantification device shown in FIG. 12 can be separately or all combined into one or several additional units to form, or some of the modules can also It is further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit may also be realized by multiple units, or the functions of multiple units may be realized by one unit. In other embodiments of the present application, the facial motion unit recognition device may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by multiple units in cooperation.

According to another embodiment of the present application, a general-purpose computing device such as a computer including a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM) and other processing elements and storage elements can be used Run a computer program (including program code) capable of executing the steps involved in the corresponding method shown in FIG. 2 to construct the fundus retinal blood vessel identification and quantification device as shown in FIG. 12, and to implement the application The method of identifying and quantifying the fundus retinal blood vessels of the cases. The computer program may be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device through the computer-readable recording medium, and run in it.

Based on the foregoing description, please refer to FIG. 13, which is a schematic structural diagram of an electronic device provided by an embodiment of the application. As shown in FIG. 13, it includes: a memory 1301 for storing one or more computer programs; a processor 1302 , Used to call the computer program stored in the memory 1301 to execute the steps in the above-mentioned embodiment of the fundus retinal blood vessel identification and quantification method; the communication interface 1303, used to perform input and output, the communication interface 1303 can be one or more; it is understandable, Each part of the electronic device communicates via a bus connection. Wherein, the processor 1302 is specifically configured to call a computer program to execute the following steps:

Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.

Optionally, the processor 1302 executes the input of the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales, including:

Input the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map;

The high-dimensional feature map is input into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and target feature maps of multiple scales are output.

Optionally, the processor 1302 performs key feature extraction by inputting the original fundus image into the encoder part of the U-shaped convolutional neural network model to obtain a high-dimensional feature map, including:

Performing convolution processing on the original fundus image to extract key features to obtain a feature map with the same size as the original fundus image;

Perform a maximum pooling operation on the feature map obtained through convolution processing, reduce the size of the feature map layer by layer, and obtain the high-dimensional feature map through alternate processing of several convolutional layers and pooling layers;

Optionally, the processor 1302 performs the up-sampling operation by inputting the high-dimensional feature map into the decoder part of the U-shaped convolutional neural network model, and outputting target feature maps of multiple scales, including:

Perform an up-sampling operation on the high-dimensional feature map, and enlarge the size of the high-dimensional feature map layer by layer;

The low-dimensional features extracted from each network layer in the encoding stage and the high-dimensional features extracted symmetrically in the decoding stage are merged through the jump connection layer to obtain the initial feature map of each network layer, and the initial feature map of each network layer has a different scale ；

The initial feature map of each network layer is output through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.

Optionally, the processor 1302 performing the segmentation of the video disc based on the target feature map to obtain a result of the video disc segmentation includes:

Fusing the target feature maps to obtain an image to be segmented;

Performing candidate frame regression processing on the image to be segmented, so as to locate the position of the optic disc from the image to be segmented, and output bounding box information of the optic disc;

According to the bounding box information of the video disc, cut out the calibrated video disc area image block and input it into the pre-trained U-shaped segmentation network, and output the segmentation result of the video disc after feature extraction and up-sampling operations.

Optionally, the processor 1302 executes the segmentation of the original fundus image by using the pre-trained cascaded segmentation network model to obtain the arteriovenous and vascular recognition result, including:

Extracting the green channel image of the original fundus image, and performing histogram equalization processing on the green channel image to obtain a green channel image with enhanced contrast;

Cutting the contrast-enhanced green channel image into multiple fundus image blocks;

The multiple fundus image blocks are input into a preset cascade segmentation network model for segmentation, and an arteriovenous blood vessel recognition result is obtained.

Optionally, the processor 1302 executes the method of using the boundary detection to obtain the vessel diameter of each vessel segment after the correction of the arteriovenous category information, including:

Take the blood vessel center point as the center, traverse in a rectangular area with a pixel range of 40*40 to find the boundary point closest to the center line of each blood vessel segment after the correction of the arteriovenous category information, and use the closest boundary point with the The distance of the blood vessel center point is the radius r, and the blood vessel diameter 2r of each blood vessel segment after the correction of the arteriovenous type information is obtained.

Optionally, the processor 1302 uses the following formula to calculate the ratio of the diameters of arterial vessels and venous vessels in the region of interest:

Among them, AVR represents the ratio of the diameter of arteries and veins in the region of interest, and CRAE represents the equivalent value of the diameter of the central retinal artery.

A _i and A _j respectively represent the maximum arterial vessel diameter and the minimum arterial vessel diameter in the region of interest obtained, 0.88 is a fixed coefficient; CRVE represents the equivalent value of the central retinal vein vessel diameter,

V _i and V _j represent the maximum venous vessel diameter and the minimum venous vessel diameter of the obtained region of interest, respectively, and 0.95 is a fixed coefficient.

Exemplarily, the foregoing electronic device may be a computer, a notebook computer, a tablet computer, a palmtop computer, a server, and other devices. The electronic device may include, but is not limited to, a memory 1301, a processor 1302, and a communication interface 1303. Those skilled in the art can understand that the schematic diagram is only an example of the electronic device, and does not constitute a limitation on the electronic device, and may include more or fewer components than those shown in the figure, or a combination of certain components, or different components.

It should be noted that since the processor 1302 of the electronic device executes the computer program to implement the steps in the above-mentioned fundus retinal blood vessel identification and quantification method, the above-mentioned embodiments of the fundus retinal blood vessel identification and quantification method are all applicable to the electronic device, and All can achieve the same or similar beneficial effects.

The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method for identifying and quantifying fundus retinal blood vessels are implemented:

Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.

Optionally, when the computer program is executed by the processor, it is also used to implement: input the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map; The high-dimensional feature map is input into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and target feature maps of multiple scales are output.

Optionally, when the computer program is executed by the processor, the computer program is also used to implement: performing convolution processing on the original fundus image to extract key features to obtain a feature map with the same size as the original fundus image; The processed feature map is subjected to a maximum pooling operation, the size of the feature map is reduced layer by layer, and the high-dimensional feature map is obtained through alternate processing of several convolutional layers and pooling layers.

Optionally, when the computer program is executed by the processor, the computer program is also used to implement: perform an up-sampling operation on the high-dimensional feature map, and enlarge the size of the high-dimensional feature map layer by layer;

The low-dimensional features extracted from each network layer in the encoding stage and the high-dimensional features extracted symmetrically in the decoding stage are merged through the jump connection layer to obtain the initial feature map of each network layer, and the initial feature map of each network layer has a different scale ；

The initial feature map of each network layer is output through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.

Optionally, when the computer program is executed by the processor, the computer program is also used to realize: fuse the target feature map to obtain the image to be segmented; Locate the position of the disc in the image, and output the bounding box information of the disc; cut out the calibrated disc area image block according to the bounding box information of the disc and input it into the pre-trained U-shaped segmentation network, and output the segmentation result of the disc after feature extraction and up-sampling.

Optionally, when the computer program is executed by the processor, the computer program is also used to implement: extracting the green channel image of the original fundus image, and performing histogram equalization processing on the green channel image to obtain a green channel image with enhanced contrast; The contrast-enhanced green channel image is cut into multiple fundus image blocks; the multiple fundus image blocks are input into a preset cascade segmentation network model for segmentation, and an arteriovenous blood vessel recognition result is obtained.

Optionally, when the computer program is executed by the processor, the computer program is also used to realize: traverse a rectangular area with a pixel range of 40*40 with the center point of the blood vessel as the center, and find each blood vessel whose distance is corrected by the category information of the artery and vein. The boundary point of the segment center line with the closest distance, and the radius r is the distance between the closest boundary point and the blood vessel center point, and the blood vessel diameter 2r of each blood vessel segment after the correction of the arteriovenous category information is obtained.

Optionally, when the computer program is executed by the processor, the following formula is used to calculate the ratio of the diameters of arterial vessels and venous vessels in the region of interest:

Among them, AVR represents the ratio of the diameter of arteries and veins in the region of interest, and CRAE represents the equivalent value of the diameter of the central retinal artery.

A _i and A _j respectively represent the maximum arterial vessel diameter and the minimum arterial vessel diameter in the region of interest obtained, 0.88 is a fixed coefficient; CRVE represents the equivalent value of the central retinal vein vessel diameter,

V _i and V _j represent the maximum venous vessel diameter and the minimum venous vessel diameter of the obtained region of interest, respectively, and 0.95 is a fixed coefficient.

Exemplarily, the computer program in the computer-readable storage medium includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium may be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc.

It should be noted that since the computer program of the computer-readable storage medium is executed by the processor to realize the steps in the above-mentioned fundus retinal blood vessel identification and quantification method, all examples of the above-mentioned fundus retinal blood vessel identification and quantification method are applicable to the computer The storage medium is readable and can achieve the same or similar beneficial effects.

The above-disclosed are only part of the embodiments of this application. Of course, it cannot be used to limit the scope of rights of this application. Those of ordinary skill in the art can understand all or part of the procedures for implementing the above-mentioned embodiments and make them in accordance with the claims of this application. The equivalent change of is still within the scope of this application.

Claims (20)

1. A method for identifying and quantifying retinal blood vessels in the fundus, wherein the method includes:

   Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

   Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

   Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

   Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

   The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

   Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.
2. The method according to claim 1, wherein said inputting the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales comprises:

   Input the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map;

   The high-dimensional feature map is input into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and target feature maps of multiple scales are output.
3. The method according to claim 2, wherein the inputting the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map comprises:

   Performing convolution processing on the original fundus image to extract key features to obtain a feature map with the same size as the original fundus image;

   A maximum pooling operation is performed on the feature map obtained through convolution processing, the size of the feature map is reduced layer by layer, and the high-dimensional feature map is obtained through alternate processing of several convolutional layers and pooling layers.
4. The method according to claim 2, wherein the inputting the high-dimensional feature map into the decoder part of the U-shaped convolutional neural network model to perform an upsampling operation and outputting target feature maps of multiple scales comprises:

   Perform an up-sampling operation on the high-dimensional feature map, and enlarge the size of the high-dimensional feature map layer by layer;

   The low-dimensional features extracted from each network layer in the encoding stage and the high-dimensional features extracted symmetrically in the decoding stage are merged through the jump connection layer to obtain the initial feature map of each network layer, and the initial feature map of each network layer has a different scale ；

   The initial feature map of each network layer is output through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.
5. The method according to any one of claims 1 to 4, wherein the segmentation of the optic disc based on the target feature map to obtain a result of the optic disc segmentation comprises:

   Fusing the target feature maps to obtain an image to be segmented;

   Performing candidate frame regression processing on the image to be segmented, so as to locate the position of the optic disc from the image to be segmented, and output bounding box information of the optic disc;

   According to the bounding box information of the video disc, cut out the calibrated video disc area image block and input it into the pre-trained U-shaped segmentation network, and output the segmentation result of the video disc after feature extraction and up-sampling operations.
6. The method according to any one of claims 1 to 4, wherein the segmenting the original fundus image by using a pre-trained cascaded segmentation network model to obtain an arteriovenous and vascular recognition result comprises:

   Extracting the green channel image of the original fundus image, and performing histogram equalization processing on the green channel image to obtain a green channel image with enhanced contrast;

   Cutting the contrast-enhanced green channel image into multiple fundus image blocks;

   The multiple fundus image blocks are input into a preset cascade segmentation network model for segmentation, and an arteriovenous blood vessel recognition result is obtained.
7. The method according to any one of claims 1 to 4, wherein the obtaining the vessel diameter of each vessel segment after the correction of the arteriovenous category information by the method of boundary detection comprises:

   Take the blood vessel center point as the center, traverse in a rectangular area with a pixel range of 40*40 to find the boundary point closest to the center line of each blood vessel segment after the correction of the arteriovenous category information, and use the closest boundary point with the The distance of the blood vessel center point is the radius r, and the blood vessel diameter 2r of each blood vessel segment after the correction of the arteriovenous type information is obtained.
8. The method according to claim 1, wherein the following formula is used to calculate the ratio of the diameters of arterial vessels and venous vessels in the region of interest:

   

   Among them, AVR represents the ratio of the diameter of arteries and veins in the region of interest, and CRAE represents the equivalent value of the diameter of the central retinal artery.

   

   A _i and A _j respectively represent the maximum arterial vessel diameter and the minimum arterial vessel diameter in the region of interest obtained, 0.88 is a fixed coefficient; CRVE represents the equivalent value of the central retinal vein vessel diameter,

   

   V _i and V _j represent the maximum venous vessel diameter and the minimum venous vessel diameter of the obtained region of interest, respectively, and 0.95 is a fixed coefficient.
9. A device for identifying and quantifying retinal blood vessels in the fundus, wherein the device comprises:

   The feature extraction module is used to input the original fundus image into the pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

   A video disc segmentation module, configured to perform video disc segmentation based on the target feature map to obtain a video disc segmentation result;

   The blood vessel recognition module is used to segment the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous blood vessel recognition result;

   The region positioning module is used to locate the region of interest based on the result of the segmentation of the optic disc, and obtain the region of interest positioning result;

   The centerline extraction module is used to extract the centerline of the blood vessel according to the result of the arteriovenous blood vessel identification, use the neighborhood connectivity determination method to detect the key points in the centerline of the blood vessel, remove the key points to obtain multiple independent blood vessel segments, and Correct the arterial and venous category information on each blood vessel segment, and obtain each blood vessel segment with the revised arterial and venous category information;

   The diameter ratio calculation module is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information by using the method of boundary detection based on the corrected blood vessel center line of the arteriovenous category information, and calculate the sensation based on the blood vessel diameter The ratio of the diameters of arteries and veins in the region of interest.
10. An electronic device, wherein the electronic device includes a processor, a memory, and a computer program that is stored on the memory and can run on the processor, and when the processor executes the computer program:

    Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

    Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

    Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

    Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

    The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

    Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.
11. The electronic device according to claim 10, wherein the processor executes the input of the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales, comprising:

    Input the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map;

    The high-dimensional feature map is input into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and target feature maps of multiple scales are output.
12. The electronic device according to claim 11, wherein the processor executes the input of the original fundus image into the encoder part of the U-shaped convolutional neural network model for key feature extraction to obtain a high-dimensional feature map ,include:

    Performing convolution processing on the original fundus image to extract key features to obtain a feature map with the same size as the original fundus image;

    A maximum pooling operation is performed on the feature map obtained through convolution processing, the size of the feature map is reduced layer by layer, and the high-dimensional feature map is obtained through alternate processing of several convolutional layers and pooling layers.
13. The electronic device according to claim 11, wherein the processor performs an up-sampling operation by inputting the high-dimensional feature map into the decoder part of the U-shaped convolutional neural network model, and outputting multiple scales Target feature map, including:

    Perform an up-sampling operation on the high-dimensional feature map, and enlarge the size of the high-dimensional feature map layer by layer;

    The low-dimensional features extracted from each network layer in the encoding stage and the high-dimensional features extracted symmetrically in the decoding stage are merged through the jump connection layer to obtain the initial feature map of each network layer, and the initial feature map of each network layer has a different scale ；

    The initial feature map of each network layer is output through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.
14. The electronic device according to any one of claims 10 to 13, wherein the execution by the processor to perform the segmentation of the optic disc based on the target feature map to obtain a result of the segmentation of the optic disc comprises:

    Fusing the target feature maps to obtain an image to be segmented;

    Performing candidate frame regression processing on the image to be segmented, so as to locate the position of the optic disc from the image to be segmented, and output bounding box information of the optic disc;

    According to the bounding box information of the video disc, cut out the calibrated video disc area image block and input it into the pre-trained U-shaped segmentation network, and output the segmentation result of the video disc after feature extraction and up-sampling operations.
15. The electronic device according to any one of claims 10 to 13, wherein the processor executes the segmentation of the original fundus image using a pre-trained cascaded segmentation network model to obtain an arteriovenous and vascular recognition result, comprising :

    Extracting the green channel image of the original fundus image, and performing histogram equalization processing on the green channel image to obtain a green channel image with enhanced contrast;

    Cutting the contrast-enhanced green channel image into multiple fundus image blocks;

    The multiple fundus image blocks are input into a preset cascade segmentation network model for segmentation, and an arteriovenous blood vessel recognition result is obtained.
16. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it is used to realize:

    Input the original fundus image into a pre-trained U-shaped convolutional neural network model for processing to obtain target feature maps of multiple scales;

    Performing segmentation of the optic disc based on the target feature map, and obtaining a segmentation result of the optic disc;

    Segmenting the original fundus image by using a pre-trained cascade segmentation network model to obtain an arteriovenous and vascular recognition result;

    Locate the region of interest based on the results of the segmentation of the optic disc, and obtain the region of interest location results;

    The blood vessel centerline is extracted according to the result of arteriovenous blood vessel identification, the neighborhood connectivity determination method is used to detect the key points in the blood vessel centerline, the key points are removed to obtain multiple independent blood vessel segments, and the movement on each blood vessel segment is determined. Revise the vein category information to obtain each blood vessel segment after the correction of the arteriovenous category information;

    Based on the blood vessel center line of each blood vessel segment corrected by the arteriovenous category information, the method of boundary detection is used to obtain the blood vessel diameter of each blood vessel segment corrected by the arteriovenous category information, and the arterial blood vessels and veins in the region of interest are calculated based on the blood vessel diameters. The ratio of the diameters of blood vessels.
17. The computer-readable storage medium according to claim 16, wherein, when the computer program is executed by the processor, it is also used to realize: input the original fundus image into the encoder part of the U-shaped convolutional neural network model to perform The key feature is extracted to obtain a high-dimensional feature map; the high-dimensional feature map is input into the decoder part of the U-shaped convolutional neural network model to perform an up-sampling operation, and a target feature map of multiple scales is output.
18. The computer-readable storage medium according to claim 17, wherein, when the computer program is executed by the processor, it is also used to implement: performing convolution processing on the original fundus image to extract key features, and obtain the difference between the original fundus and the original fundus image. Feature maps with the same image size; perform a maximum pooling operation on the feature maps obtained through convolution processing, reduce the size of the feature maps layer by layer, and obtain the high-dimensional feature maps through alternate processing of several convolutional layers and pooling layers.
19. The computer-readable storage medium according to claim 17, wherein, when the computer program is executed by the processor, it is also used to implement: performing an up-sampling operation on the high-dimensional feature map, and amplifying the high-dimensional feature map layer by layer size;

    The low-dimensional features extracted from each network layer in the encoding stage and the high-dimensional features extracted symmetrically in the decoding stage are merged through the jump connection layer to obtain the initial feature map of each network layer, and the initial feature map of each network layer has a different scale ；

    The initial feature map of each network layer is output through the output branch of each network layer to obtain target feature maps of multiple scales, and an attention mechanism is added to the output branch of each network layer.
20. The computer-readable storage medium according to any one of claims 16 to 19, wherein when the computer program is executed by the processor, it is also used to realize: fuse the target feature map to obtain the image to be segmented; The image to be segmented is subjected to candidate frame regression processing to locate the position of the optic disc from the image to be segmented, and output the bounding box information of the optic disc; according to the bounding box information of the optic disc, cut out the calibrated image block of the optic disc area and input the pre-trained U-shape In the segmentation network, the segmentation results of the video disc are output after feature extraction and up-sampling operations.

Images