WO2021068523A1 - 眼底图像黄斑中心定位方法、装置、电子设备及存储介质 - Google Patents

眼底图像黄斑中心定位方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021068523A1
WO2021068523A1 PCT/CN2020/093338 CN2020093338W WO2021068523A1 WO 2021068523 A1 WO2021068523 A1 WO 2021068523A1 CN 2020093338 W CN2020093338 W CN 2020093338W WO 2021068523 A1 WO2021068523 A1 WO 2021068523A1
Authority
WO
WIPO (PCT)
Prior art keywords
fundus image
center point
area
confidence
preset
Prior art date
Application number
PCT/CN2020/093338
Other languages
English (en)
French (fr)
Inventor
李葛
王瑞
王立龙
唐义君
张萌
高鹏
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Priority to US17/620,733 priority Critical patent/US20220415087A1/en
Priority to JP2021568982A priority patent/JP7242906B2/ja
Publication of WO2021068523A1 publication Critical patent/WO2021068523A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/197—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • This application relates to the technical field of medical image processing based on artificial intelligence, and in particular to a method, device, electronic equipment, and storage medium for positioning the macular center of a fundus image.
  • the macula concentrates a large number of visual function cells. If the lesions in the macular area are not detected and treated in time, the chance of blindness is greatly increased. Therefore, accurate macular center positioning is of great significance to the diagnosis of retinopathy.
  • there is a method for locating the center of the macula based on fundus images which first locates the center of the optic disc and then locates the center of the macula according to the center of the optic disc; or uses a deep neural network target detection model to directly detect the macular area.
  • the inventor realizes that the existing method for positioning the center of the macula based on fundus images completely relies on the positioning of the center of the optic disc. Once the center of the optic disc fails to locate the center of the macula, it cannot be effectively positioned, and the calculation complexity is high, the timeliness is low, and the robustness is poor.
  • the macular center positioning method based on deep neural network does not rely on the positioning of the optic disc center, the macular area is very susceptible to the influence of the diseased area, image quality and atrophy area, which makes the macular area unable to be successfully detected and therefore unable to be effectively located . Therefore, it is necessary to propose a new fundus image macular center positioning solution, which can not completely rely on the positioning of the optic disc center, and can effectively locate the macular center when the macular area is blocked or the fundus image quality is poor.
  • a method for positioning the macular center of a fundus image including:
  • the detection result of the fundus image detection model includes: the optic disc area in the fundus image and the corresponding first detection frame and first confidence, the macula area and the corresponding second detection frame and second Confidence;
  • the second confidence is less than the preset second confidence threshold, use a pre-trained left and right eye recognition model to identify whether the fundus image to be detected is a left eye fundus image or a right eye fundus image;
  • a device for positioning the macular center of a fundus image comprising:
  • the input module is used to input the fundus image to be detected into the pre-trained fundus image detection model
  • the obtaining module is used to obtain the detection result of the fundus image detection model, where the detection result includes: the optic disc area in the fundus image and the corresponding first detection frame and the first confidence, the macula area and the corresponding second Detection frame and second confidence level;
  • a calculation module configured to calculate the coordinates of the center point of the optic disc area according to the first detection frame and calculate the coordinates of the center point of the macula area according to the second detection frame;
  • the comparison module is configured to compare the second confidence level with a preset first confidence level threshold and a preset second confidence level threshold, wherein the preset first confidence level threshold is greater than the preset second confidence level threshold ;
  • a recognition module configured to use a pre-trained left and right eye recognition model to recognize whether the fundus image to be detected is a left eye fundus image or a right eye fundus image when the second confidence is less than the preset second confidence threshold ;
  • the correction module is used for correcting the center point of the macula area by using different correction models for the left eye fundus image and the right eye fundus image.
  • An electronic device including a processor configured to execute computer-readable instructions stored in a memory to implement the following steps:
  • the detection result of the fundus image detection model includes: the optic disc area in the fundus image and the corresponding first detection frame and first confidence, the macula area and the corresponding second detection frame and second Confidence;
  • the second confidence is less than the preset second confidence threshold, use a pre-trained left and right eye recognition model to identify whether the fundus image to be detected is a left eye fundus image or a right eye fundus image;
  • One or more readable storage media storing computer readable instructions
  • the computer readable storage medium storing computer readable instructions
  • the one Or multiple processors perform the following steps:
  • the detection result of the fundus image detection model includes: the optic disc area in the fundus image and the corresponding first detection frame and first confidence, the macula area and the corresponding second detection frame and second Confidence;
  • the second confidence is less than the preset second confidence threshold, use a pre-trained left and right eye recognition model to identify whether the fundus image to be detected is a left eye fundus image or a right eye fundus image;
  • device, electronic equipment and storage medium for positioning the macular center of the fundus image, the detection frame of the optic disc area and the macular area in the fundus image to be detected and the confidence of the corresponding detection frame are output through the fundus image detection model, and then the confidence of the corresponding detection frame is based on the detection frame Calculate the coordinates of the center points of the optic disc area and the macular area, and finally, according to the correspondence between the left and right eyes of the macular area and the optic disc area, use the optic disc center to correct the undetected macular area and the fundus image with lower confidence in the macular area. Even when the macular area is blocked or the quality of the fundus image is poor, the center point of the macular area can be effectively located. It solves the problem of macular region detection failure due to image quality and lesion occlusion in the yellow disc positioning method based on deep learning, and eliminates the dependence of the macular center positioning and the optic disc center positioning in the traditional method.
  • Fig. 1 is a flowchart of a method for positioning the macular center of a fundus image provided by an embodiment of the present application.
  • Fig. 2 is a schematic diagram of the network structure of the Mask RCNN network provided by an embodiment of the present application.
  • Fig. 3 is a structural diagram of a device for positioning the macular center of a fundus image provided by an embodiment of the present application.
  • Fig. 4 is a schematic diagram of an electronic device provided by an embodiment of the present application.
  • the method for positioning the macular center of the fundus image of the present application is applied to one or more electronic devices.
  • the electronic device is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions.
  • Its hardware includes, but is not limited to, a microprocessor and an application specific integrated circuit (ASIC) , Programmable Gate Array (Field-Programmable Gate Array, FPGA), Digital Processor (Digital Signal Processor, DSP), embedded equipment, etc.
  • ASIC application specific integrated circuit
  • FPGA Field-Programmable Gate Array
  • DSP Digital Processor
  • embedded equipment etc.
  • the electronic device may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the electronic device can interact with the user through a keyboard, a mouse, a remote control, a touch panel, or a voice control device.
  • FIG. 1 is a flowchart of a method for locating the macular center of a fundus image according to Embodiment 1 of the present application. According to different needs, the execution order in this flowchart can be changed, and some steps can be omitted.
  • S11 Input the fundus image to be detected into a pre-trained fundus image detection model.
  • the fundus image is an image taken by an eye detection device for diagnosing ocular pathologies.
  • the fundus refers to the tissues in the back of the eyeball, including the retina, optic papilla, macula, and central retinal arteries and veins. Therefore, the fundus image to be tested includes the macula and the surrounding area of the macula, the retina and its surroundings The resulting optic disc area.
  • the fundus image detection model is obtained by training using a sample image of the location of the known characteristic area, the input is a fundus image, and the output is a fundus image marked with the characteristic area.
  • the characteristic area is at least one of the optic disc area and the macular area.
  • the method before S1 (inputting the fundus image to be detected into a pre-trained fundus image detection model), the method further includes:
  • the fundus image detection model is obtained by training the deep neural network with a large amount of sample data, so that after the unlabeled fundus image is input to the fundus image detection model, the fundus image detection model outputs the fundus image marked with one or more characteristic regions .
  • the training process of the fundus image detection model includes:
  • the training of the fundus image detection model ends; otherwise, when the test pass rate is less than the preset pass rate threshold, increase the training set
  • the Mask RCNN network is trained based on the increased number of training sets until the test pass rate is greater than or equal to the preset pass rate threshold.
  • the fundus image may include the fundus image of the left eye or the fundus image of the right eye of healthy persons and patients with eye diseases.
  • the characteristic area in the fundus image may be obtained by recognizing and marking the fundus image manually or by other automatic recognition methods, and the content of the marking may be any one or more of the macula area and the optic disc area.
  • any one or more of the macula area and the optic disc area in the 10 fundus images is marked, and the fundus image marked with one or more characteristic areas is used as the data
  • the data set is then divided into a training set and a test set.
  • the number of fundus images marked with one or more characteristic areas in the training set is greater than the number of fundus images marked with one or more characteristic areas in the test set.
  • the number of fundus images marked with one or more characteristic areas 80% of the fundus images of one or more characteristic regions are used as the training set, and the remaining 30% of the fundus images are used as the test set.
  • the Mask RCNN network is selected as the prototype of the fundus image detection model, the default input parameters are used when the Mask RCNN network is initialized, and the input parameters are continuously adjusted during the training process.
  • the fundus image detection model is generated by training, the fundus images in the test set are used to compare the results.
  • the trained fundus image detection model is verified. If the test pass rate is less than the preset pass rate threshold, for example, the pass rate is less than 98%, increase the number of fundus images involved in training and retrain the Mask RCNN network until the fundus is trained
  • the test pass rate of the image detection model is greater than or equal to the preset pass rate threshold.
  • the Mask RCNN network is an existing technology, and this application will not elaborate on it here.
  • the method further includes:
  • the constructing a sample data set based on the fundus images marked with one or more of the characteristic regions and the categories of the corresponding characteristic regions includes: based on the fundus images marked with one or more of the characteristic regions and the categories of the corresponding characteristic regions Construct a first sample data set, construct a second sample data set based on the fundus image after the flip processing marked with one or more of the characteristic regions and the categories of the corresponding characteristic regions, and combine the first sample data set with The second sample data set serves as the sample data set.
  • the flip processing includes random flip, mirror flip, horizontal flip, and other rotations.
  • the angle of rotation can range from -15 degrees to +15 degrees.
  • the initial data set includes a first data set and a second data set, wherein the first data set is a fundus image taken and marked by an eye detection device, and the second data set is obtained by comparing the first data set to the second data set.
  • a data set is obtained by processing such as flipping and mirroring. In this way, the fundus images in the first data set are expanded to achieve the purpose of supplementing the fundus images. Training the fundus image detection model based on a larger amount of data sets can improve the detection accuracy of the fundus image detection model and enhance the generalization of the fundus image detection model performance.
  • the method further includes:
  • the network structure of the Mask RCNN network includes:
  • Multi-layer backbone network each layer of backbone network adopts MobileNet V2 network;
  • the backbone network of each layer is used to extract the features of the input fundus image, which is a top-down structure, and performs down-sampling processing layer by layer.
  • the backbone network C1 of the first layer performs feature extraction on the input fundus image F0, and outputs a 256*256 feature map F1 to the first Layer 2 backbone network C2;
  • Layer 2 backbone network C2 performs feature extraction on the input feature map F1, and outputs a 128*128 feature map F2 to Layer 3 backbone network C3;
  • Layer 3 backbone network C3 pairs Perform feature extraction on the input feature map F2, and output a 64*64 feature map F3 to the backbone network C4 of the fourth layer; and so on;
  • the size of the feature map output by the backbone network of the previous layer is the backbone network of the next layer 2 times the output feature map.
  • Multi-layer feature pyramid network layer the input of the previous feature pyramid network layer is the sum of the output of the next feature pyramid network layer and the output of the backbone network at the same layer as the previous feature pyramid network layer;
  • the Feature Pyramid Networks (FPN) layer is used to predict the feature map of each layer. It has a bottom-up structure and performs up-sampling processing layer by layer.
  • FPN Feature Pyramid Networks
  • the input of the fifth layer feature pyramid network layer P5 is the input of the backbone network located at the same layer as the fifth layer (equivalent to a 1X1 convolution); the fourth layer feature pyramid network
  • the input of layer P4 is the sum of the output of the fifth layer feature pyramid network layer P5 and the output of the backbone network C4 (the fourth layer backbone network) located on the same layer as the fourth layer;
  • the input of the third layer feature pyramid network layer P3 is The sum of the output of the fourth layer feature pyramid network layer P4 and the output of the backbone network C3 (the third layer backbone network) at the same layer as the third layer; and so on.
  • the attention block layer makes the feature extraction ability of the network more focused on the optic disc area and the macular area while reducing the noise introduced by other areas.
  • This layer performs the RoIAlign operation on the stride corresponding to the feature maps obtained at the four different scales of [P2 P3 P4 P5] and the attention layer to generate RoI, and obtains a fixed-size proposal feature map, which is input to the fully connected layer for target detection and positioning .
  • This layer performs Concat connection to the proposal feature map output by the pooling layer, and then the network is divided into three parts: fully connected prediction class, fully connected prediction rectangular box, and fully connected prediction confidence score.
  • This layer is used to output three values, which are the category of the feature area in the fundus image, the detection frame of the feature area in the fundus image, and the confidence of the detection frame.
  • the characteristic area is any one or more of the optic disc area and the macular area.
  • the backbone network adopts a lightweight Mobile Net V2 network, which reduces the amount of calculation of network parameters and can improve the detection speed of the fundus image detection model.
  • an attention layer is added after the feature pyramid network layer, so that the feature extraction ability of the network is more focused on the optic disc area and the macula area, which further improves the detection speed of the fundus image detection model and improves the detection accuracy of the fundus image detection model.
  • the detection result includes: the optic disc area in the fundus image and the corresponding first detection frame and first confidence, the macula area and the corresponding second detection frame and The second degree of confidence.
  • the output of the fundus image detection model is a fundus image marked with a characteristic area. That is, the output is a fundus image in which the macular area and/or the optic disc area are marked with a detection frame.
  • the confidence is used to indicate the accuracy of the characteristic region detected by the fundus image detection model, and the value range is 0-1.
  • the fundus image detection model when using the trained fundus image detection model to process any fundus image, for most, such as 80% of the fundus images, the fundus image detection model can and accurately output the fundus image marked with the optic disc area. That is, the detection frame can be used to accurately mark the location of the optic disc area, and the obtained confidence is also high.
  • the macular area may be marked in the output fundus image, or the macular area may not be marked. However, because the macular area is prone to disease or occlusion, the macular area detected by the fundus image detection model will have errors, and the corresponding confidence may be high or low.
  • the fundus image output by the fundus image detection model is directly marked with the macular area and the confidence is high, it can be directly used as the result; if the fundus image output by the fundus image detection model is marked with the macular area but the confidence is low, or The output of the fundus image is not marked with the macular area, and the position of the macular area can be corrected according to the position of the optic disc area.
  • the outline of the detection frame is rectangular, and the shape is determined by the sample data during the training process of the fundus image detection model. If the shape of the mark in the sample data is changed during the training process, for example, circular or irregular shapes are used. And so on, the trained fundus image detection model will also mark the corresponding shape, so as to output the contour of the corresponding shape.
  • S13 Calculate the coordinates of the center point of the optic disc area according to the first detection frame and calculate the coordinates of the center point of the macula area according to the second detection frame.
  • the detection frame of the optic disc area and the detection frame of the macula area can be obtained from the fundus image marked with the characteristic area.
  • the coordinates of the center point of the optic disc area can be calculated according to the multiple first coordinates, and the coordinates of the center point of the macula area can be calculated according to the multiple second coordinates.
  • the preset first confidence threshold and the preset second confidence threshold are both preset thresholds used to determine the correctness of the detection frame of the macular region.
  • the preset first confidence threshold is greater than the preset second confidence threshold.
  • the left and right eye recognition model can be pre-trained by offline training.
  • the specific training process is: collecting multiple fundus images, and marking each fundus image to indicate that the fundus image is a left eye fundus image or a right eye fundus image, using the fundus image and the corresponding logo as a data set, and then The data set is divided into training set and test set.
  • the number of training sets is greater than the number of test sets.
  • the deep neural network for example, convolutional neural network, is trained to obtain the left and right eye recognition model, and then the training is tested based on the test set.
  • the test pass rate of a good left and right eye recognition model When the test pass rate is greater than or equal to the preset pass rate threshold, the left and right eye recognition model is output. When the test pass rate is less than the preset pass rate threshold, the training set and The test set is used to train the left and right eye recognition model based on the new training set until the test pass rate is greater than or equal to the preset pass rate threshold.
  • the relative positions of the macula area and the optic disc area are different.
  • the optic disc area In the left eye fundus image, the optic disc area is on the left, the macula area is on the right, and the macula area is on the right of the optic disc area; right eye In the fundus image, the optic disc area is on the right, the macula area is on the left, and the macula area is on the left of the optic disc area.
  • the second confidence level is less than the preset first confidence level threshold, it indicates that the confidence level of the macular area detected by the fundus image detection model is very low, or the macular area is not detected. At this time, further identification of the to-be-detected macular area is required. Whether the fundus image is the left eye fundus image or the right eye fundus image, and then different correction methods are used to correct the detection frame of the macular area, and further correct the center point of the macular area.
  • the correcting the center point of the macula area by using different correction models for the left eye fundus image and the right eye fundus image includes:
  • a first correction model and the center point coordinates of the optic disc area are used to correct the center point of the macula area, and the first correction model is:
  • a second correction model and the center point coordinates of the optic disc area are used to correct the center point of the macula area, and the second correction model is:
  • W represents the width of the fundus image to be detected
  • H represents the height of the fundus image to be detected
  • (x oc , y oc ) is the calculated center point coordinates of the optic disc area
  • h is the The height of the first detection frame corresponding to the optic disc area
  • (x fovea , y fovea ) is the coordinate of the center point of the macular area obtained by correction.
  • the method includes:
  • the third correction model and the coordinates of the center point of the optic disc area are used to compare the macular area The center point is corrected, and the third correction model is:
  • (x fc , y fc ) are the coordinates of the center point of the macular area obtained by the final correction
  • (x dc , y dc ) are the coordinates of the center point of the macular area obtained by calculation
  • (x fovea , y fovea ) is based on The coordinates of the center point of the macular area obtained by the correction of the center point of the optic disc area.
  • the second confidence is between the preset first confidence threshold and the preset second confidence threshold, it indicates that the fundus image detection model has detected the macular area, but the confidence of the detected macular area is not high, and the marked macula The accuracy of the detection frame of the area is not high, and the detection frame of the macular area needs to be corrected, and then the center point of the macular area is corrected.
  • the method includes:
  • the center point coordinates of the macular area calculated according to the second detection frame are used as the final center point coordinates of the macular area.
  • the fundus image detection model detects the macular area and the corresponding confidence of the macular area is higher (greater than the preset first confidence threshold), the macular area detected by the fundus image detection model is considered to be good, and the fundus may not be needed at this time
  • the image detection model detects the optic disc area. Only when the fundus image detection model cannot detect the macular area or the macular area but the corresponding confidence of the macular area is low (less than the preset first confidence threshold), then it is necessary to rely on the detection result of the optic disc area to perform the macular area Fix. Therefore, the use of the above technical solution can reduce the dependence on the video disc area to a certain extent.
  • the method for positioning the macular center of the fundus image obtains a fundus image detection model based on Mask RCNN network training, and outputs the detection frame and corresponding detection of the optic disc area and the macular area in the fundus image to be detected through the fundus image detection model
  • the confidence of the frame is calculated based on the detection frame, and the coordinates of the center point of the optic disc area and the macular area are calculated.
  • the confidence of the undetected macular area and the macular area is compared with the center of the optic disc. Correct the low fundus image.
  • the center point of the macular area can be effectively located. It solves the problem of macular area detection failure due to image quality and lesion occlusion in the yellow disk positioning method based on deep learning. Moreover, the fundus image detection model separately detects the macular area and the optic disc area, so that even when the optic disc area in the fundus image is missing, the center of the macula can still be effectively located, eliminating the dependence of the traditional method on the positioning of the center of the macula and the center of the optic disc. Sex.
  • the Mask RCNN network structure is changed to reduce the amount of network parameter calculations, improve the detection speed of the fundus image detection model, and improve the timeliness of the macular center positioning.
  • an attention layer is added to make The feature extraction capability of the network focuses more on the optic disc area and the macular area, which further improves the detection speed of the fundus image detection model, improves the detection accuracy of the fundus image detection model, and assists in improving the positioning of the center point of the macular area.
  • FIG. 2 is a structural diagram of a device for positioning the macular center of a fundus image provided in the second embodiment of the present application.
  • the fundus image macular center positioning device 30 runs in an electronic device, which can solve the problem of macular region detection failure due to image quality, lesion occlusion, etc. in the yellow disk positioning method based on deep learning, and can eliminate the macular center positioning in the traditional method Dependence on the central positioning of the optic disc.
  • the device 30 for positioning the macular center of the fundus image may include: an input module 301, a training module 302, an acquisition module 303, a calculation module 304, a comparison module 305, an identification module 306, a correction module 307 and a determination module 308.
  • the input module 301 is used to input the fundus image to be detected into the pre-trained fundus image detection model.
  • the fundus image is an image taken by an eye detection device for diagnosing ocular pathologies.
  • the fundus refers to the tissues in the back of the eyeball, including the retina, optic papilla, macula, and central retinal arteries and veins. Therefore, the fundus image to be tested includes the macula and the surrounding area of the macula, the retina and its surroundings The resulting optic disc area.
  • the fundus image detection model is obtained by training using a sample image of the location of the known characteristic area, the input is a fundus image, and the output is a fundus image marked with the characteristic area.
  • the characteristic area is at least one of the optic disc area and the macular area.
  • the fundus image macular center positioning device 30 before the input module 301 inputs the fundus image to be detected into the pre-trained fundus image detection model, the fundus image macular center positioning device 30 further includes:
  • the training module 302 is used to train the fundus image detection model.
  • the fundus image detection model is obtained by training the deep neural network with a large amount of sample data, so that after the unlabeled fundus image is input to the fundus image detection model, the fundus image detection model outputs the fundus image marked with one or more characteristic regions .
  • the training process of the training module 302 for training the fundus image detection model includes:
  • the training of the fundus image detection model ends; otherwise, when the test pass rate is less than the preset pass rate threshold, increase the training set
  • the Mask RCNN network is trained based on the increased number of training sets until the test pass rate is greater than or equal to the preset pass rate threshold.
  • the fundus image may include the fundus image of the left eye or the fundus image of the right eye of healthy persons and patients with eye diseases.
  • the characteristic area in the fundus image may be obtained by recognizing and marking the fundus image manually or by other automatic recognition methods, and the content of the marking may be any one or more of the macula area and the optic disc area.
  • any one or more of the macula area and the optic disc area in the 10 fundus images is marked, and the fundus image marked with one or more characteristic areas is used as the data
  • the data set is then divided into a training set and a test set.
  • the number of fundus images marked with one or more characteristic areas in the training set is greater than the number of fundus images marked with one or more characteristic areas in the test set. For example, the number of fundus images marked with one or more characteristic areas in the test set 80% of the fundus images of one or more characteristic regions are used as the training set, and the remaining 30% of the fundus images are used as the test set.
  • the Mask RCNN network is selected as the prototype of the fundus image detection model, the default input parameters are used when the Mask RCNN network is initialized, and the input parameters are continuously adjusted during the training process.
  • the fundus image detection model is generated by training, the fundus images in the test set are used to compare the results.
  • the trained fundus image detection model is verified. If the test pass rate is less than the preset pass rate threshold, for example, the pass rate is less than 98%, increase the number of fundus images involved in training and retrain the Mask RCNN network until the fundus is trained
  • the test pass rate of the image detection model is greater than or equal to the preset pass rate threshold.
  • the Mask RCNN network is an existing technology, and this application will not elaborate on it here.
  • the method further includes:
  • the constructing a sample data set based on the fundus images marked with one or more of the characteristic regions and the categories of the corresponding characteristic regions includes: based on the fundus images marked with one or more of the characteristic regions and the categories of the corresponding characteristic regions Construct a first sample data set, construct a second sample data set based on the fundus image after the flip processing marked with one or more of the characteristic regions and the categories of the corresponding characteristic regions, and combine the first sample data set with The second sample data set serves as the sample data set.
  • the flip processing includes random flip, mirror flip, horizontal flip, and other rotations.
  • the angle of rotation can range from -15 degrees to +15 degrees.
  • the initial data set includes a first data set and a second data set, wherein the first data set is a fundus image taken and marked by an eye detection device, and the second data set is obtained by comparing the first data set to the second data set.
  • a data set is obtained by processing such as flipping and mirroring. In this way, the fundus images in the first data set are expanded to achieve the purpose of supplementing the fundus images. Training the fundus image detection model based on a larger amount of data sets can improve the detection accuracy of the fundus image detection model and enhance the generalization of the fundus image detection model performance.
  • the fundus image macular center positioning device 30 further includes:
  • the network structure of the Mask RCNN network includes:
  • Multi-layer backbone network each layer of backbone network adopts MobileNet V2 network;
  • the backbone network of each layer is used to extract the features of the input fundus image, which is a top-down structure, and performs down-sampling processing layer by layer.
  • the backbone network C1 of the first layer performs feature extraction on the input fundus image F0, and outputs a 256*256 feature map F1 to the first Layer 2 backbone network C2;
  • Layer 2 backbone network C2 performs feature extraction on the input feature map F1, and outputs a 128*128 feature map F2 to Layer 3 backbone network C3;
  • Layer 3 backbone network C3 pairs Perform feature extraction on the input feature map F2, and output a 64*64 feature map F3 to the backbone network C4 of the fourth layer; and so on;
  • the size of the feature map output by the backbone network of the previous layer is the backbone network of the next layer 2 times the output feature map.
  • Multi-layer feature pyramid network layer the input of the previous feature pyramid network layer is the sum of the output of the next feature pyramid network layer and the output of the backbone network at the same layer as the previous feature pyramid network layer;
  • the Feature Pyramid Networks (FPN) layer is used to predict the feature map of each layer. It has a bottom-up structure and performs up-sampling processing layer by layer.
  • FPN Feature Pyramid Networks
  • the input of the fifth layer feature pyramid network layer P5 is the input of the backbone network located at the same layer as the fifth layer (equivalent to a 1X1 convolution); the fourth layer feature pyramid network
  • the input of layer P4 is the sum of the output of the fifth layer feature pyramid network layer P5 and the output of the backbone network C4 (the fourth layer backbone network) located on the same layer as the fourth layer;
  • the input of the third layer feature pyramid network layer P3 is The sum of the output of the fourth layer feature pyramid network layer P4 and the output of the backbone network C3 (the third layer backbone network) at the same layer as the third layer; and so on.
  • the attention block layer makes the feature extraction ability of the network more focused on the optic disc area and the macular area while reducing the noise introduced by other areas.
  • This layer performs the RoIAlign operation on the stride corresponding to the feature maps obtained at the four different scales of [P2 P3 P4 P5] and the attention layer to generate RoI, and obtains a fixed-size proposal feature map, which is input to the fully connected layer for target detection and positioning .
  • This layer performs Concat connection on the proposal feature map output by the pooling layer, and then the network is divided into three parts: fully connected prediction class, fully connected prediction rectangular box, and fully connected prediction confidence.
  • This layer is used to output three values, which are the category of the feature area in the fundus image, the detection frame of the feature area in the fundus image, and the confidence of the detection frame.
  • the characteristic area is any one or more of the optic disc area and the macular area.
  • the backbone network adopts a lightweight Mobile Net V2 network, which reduces the amount of calculation of network parameters and can improve the detection speed of the fundus image detection model.
  • an attention layer is added after the feature pyramid network layer, so that the feature extraction ability of the network is more focused on the optic disc area and the macula area, which further improves the detection speed of the fundus image detection model and improves the detection accuracy of the fundus image detection model.
  • the obtaining module 303 is configured to obtain the detection result of the fundus image detection model, where the detection result includes: the optic disc area in the fundus image and the corresponding first detection frame and first confidence, the macular area and the corresponding first The second detection frame and the second confidence level.
  • the output of the fundus image detection model is a fundus image marked with a characteristic area. That is, the output is a fundus image in which the macular area and/or the optic disc area are marked with a detection frame.
  • the confidence is used to indicate the accuracy of the characteristic region detected by the fundus image detection model, and the value range is 0-1.
  • the fundus image detection model when using the trained fundus image detection model to process any fundus image, for most, such as 80% of the fundus images, the fundus image detection model can and accurately output the fundus image marked with the optic disc area. That is, the detection frame can be used to accurately mark the location of the optic disc area, and the obtained confidence is also high.
  • the macular area may be marked in the output fundus image, or the macular area may not be marked. However, because the macular area is prone to disease or occlusion, the macular area detected by the fundus image detection model will have errors, and the corresponding confidence may be high or low.
  • the fundus image output by the fundus image detection model is directly marked with the macular area and the confidence is high, it can be directly used as the result; if the fundus image output by the fundus image detection model is marked with the macular area but the confidence is low, or The output of the fundus image is not marked with the macular area, and the position of the macular area can be corrected according to the position of the optic disc area.
  • the outline of the detection frame is rectangular, and the shape is determined by the sample data during the training process of the fundus image detection model. If the shape of the mark in the sample data is changed during the training process, for example, circular or irregular shapes are used. And so on, the trained fundus image detection model will also mark the corresponding shape, so as to output the contour of the corresponding shape.
  • the calculation module 304 is configured to calculate the center point coordinates of the optic disc area according to the first detection frame and calculate the center point coordinates of the macula area according to the second detection frame.
  • the detection frame of the optic disc area and the detection frame of the macula area can be obtained from the fundus image marked with the characteristic area.
  • the coordinates of the center point of the optic disc area can be calculated according to the multiple first coordinates, and the coordinates of the center point of the macula area can be calculated according to the multiple second coordinates.
  • the comparison module 305 is configured to compare the second confidence level with a preset first confidence level threshold and a preset second confidence level threshold.
  • the preset first confidence threshold and the preset second confidence threshold are both preset thresholds used to determine the correctness of the detection frame of the macular region.
  • the preset first confidence threshold is greater than the preset second confidence threshold.
  • the recognition module 306 is configured to recognize whether the fundus image to be detected is the fundus image of the left eye or the fundus of the right eye by using a pre-trained left and right eye recognition model when the second confidence is less than the preset second confidence threshold image.
  • the left and right eye recognition model can be pre-trained by offline training.
  • the specific training process is: collecting multiple fundus images, and marking each fundus image to indicate that the fundus image is a left eye fundus image or a right eye fundus image, using the fundus image and the corresponding logo as a data set, and then The data set is divided into training set and test set.
  • the number of training sets is greater than the number of test sets.
  • the deep neural network for example, convolutional neural network, is trained to obtain the left and right eye recognition model, and then the training is tested based on the test set.
  • the test pass rate of a good left and right eye recognition model When the test pass rate is greater than or equal to the preset pass rate threshold, the left and right eye recognition model is output. When the test pass rate is less than the preset pass rate threshold, the training set and The test set is used to train the left and right eye recognition model based on the new training set until the test pass rate is greater than or equal to the preset pass rate threshold.
  • the relative positions of the macula area and the optic disc area are different.
  • the optic disc area In the left eye fundus image, the optic disc area is on the left, the macula area is on the right, and the macula area is on the right of the optic disc area; right eye In the fundus image, the optic disc area is on the right, the macula area is on the left, and the macula area is on the left of the optic disc area.
  • the second confidence level is less than the preset first confidence level threshold, it indicates that the confidence level of the macular area detected by the fundus image detection model is very low, or the macular area is not detected. At this time, further identification of the to-be-detected macular area is required. Whether the fundus image is the left eye fundus image or the right eye fundus image, and then different correction methods are used to correct the detection frame of the macular area, and further correct the center point of the macular area.
  • the correction module 307 is configured to use different correction models for the left eye fundus image and the right eye fundus image to correct the center point of the macula area.
  • the correcting the center point of the macula area by using different correction models for the left eye fundus image and the right eye fundus image includes:
  • a first correction model and the center point coordinates of the optic disc area are used to correct the center point of the macula area, and the first correction model is:
  • a second correction model and the center point coordinates of the optic disc area are used to correct the center point of the macula area, and the second correction model is:
  • W represents the width of the fundus image to be detected
  • H represents the height of the fundus image to be detected
  • (x oc , y oc ) is the calculated center point coordinates of the optic disc area
  • h is the The height of the first detection frame corresponding to the optic disc area
  • (x fovea , y fovea ) is the coordinate of the center point of the macular area obtained by correction.
  • the correction module 307 is further configured to:
  • the third correction model and the coordinates of the center point of the optic disc area are used to compare the macular area The center point is corrected, and the third correction model is:
  • (x fc , y fc ) are the coordinates of the center point of the macular area obtained by the final correction
  • (x dc , y dc ) are the coordinates of the center point of the macular area obtained by calculation
  • (x fovea , y fovea ) is based on The coordinates of the center point of the macular area obtained by the correction of the center point of the optic disc area.
  • the second confidence is between the preset first confidence threshold and the preset second confidence threshold, it indicates that the fundus image detection model has detected the macular area, but the confidence of the detected macular area is not high, and the marked macula The accuracy of the detection frame of the area is not high, and the detection frame of the macular area needs to be corrected, and then the center point of the macular area is corrected.
  • the fundus image macular center positioning device 30 when the second confidence level is greater than the preset first confidence level threshold, the fundus image macular center positioning device 30 further includes:
  • the determining module 308 is configured to use the center point coordinates of the macula area calculated according to the second detection frame as the final center point coordinates of the macula area.
  • the fundus image detection model detects the macular area and the corresponding confidence of the macular area is higher (greater than the preset first confidence threshold), the macular area detected by the fundus image detection model is considered to be good, and the fundus may not be needed at this time
  • the image detection model detects the optic disc area. Only when the fundus image detection model cannot detect the macular area or the macular area but the corresponding confidence of the macular area is low (less than the preset first confidence threshold), then it is necessary to rely on the detection result of the optic disc area to perform the macular area Fix. Therefore, the use of the above technical solution can reduce the dependence on the video disc area to a certain extent.
  • the fundus image macular center positioning device obtains a fundus image detection model through Mask RCNN network training, and outputs the detection frame and corresponding detection of the optic disc area and the macula area in the fundus image to be detected through the fundus image detection model
  • the confidence of the frame is calculated based on the detection frame, and the coordinates of the center point of the optic disc area and the macular area are calculated.
  • the confidence of the undetected macular area and the macular area is compared with the center of the optic disc. Correct the low fundus image.
  • the center point of the macular area can be effectively located. It solves the problem of macular area detection failure due to image quality and lesion occlusion in the yellow disk positioning method based on deep learning. Moreover, the fundus image detection model separately detects the macular area and the optic disc area, so that even when the optic disc area in the fundus image is missing, the center of the macula can be effectively located, eliminating the dependence of the traditional method on the positioning of the center of the macula and the center of the optic disc. Sex.
  • the Mask RCNN network structure is changed to reduce the amount of network parameter calculations, improve the detection speed of the fundus image detection model, and improve the timeliness of the macular center positioning.
  • an attention layer is added to make The feature extraction capability of the network focuses more on the optic disc area and the macular area, which further improves the detection speed of the fundus image detection model, improves the detection accuracy of the fundus image detection model, and assists in improving the positioning of the center point of the macular area.
  • FIG. 3 is a schematic diagram of an electronic device provided in Embodiment 3 of this application.
  • the electronic device 40 includes a memory 401, a processor 402, a communication bus 403, and a transceiver 404.
  • the memory 401 stores computer-readable instructions that can run on the processor 402, such as a fundus image macular center positioning program. .
  • the processor 402 executes the computer-readable instructions, the steps in the above-mentioned method for positioning the macular center of the fundus image are implemented. That is, the electronic device 40 includes a processor 402, and the processor 402 is configured to execute computer-readable instructions stored in the memory 401 to implement the steps in the above-mentioned embodiment of the method for positioning the macular center of the fundus image.
  • the computer-readable instructions may be divided into one or more modules, and the one or more modules are stored in the memory 401 and executed by the processor 402 to complete the method.
  • the one or more modules may be an instruction segment of a series of computer-readable instructions capable of completing specific functions, and the instruction segment is used to describe the execution process of the computer-readable instruction in the electronic device 40.
  • the computer-readable instructions can be divided into the input module 301, the training module 302, the acquisition module 303, the calculation module 304, the comparison module 305, the recognition module 306, the correction module 307, and the determination module 308 in FIG. See the second embodiment for specific functions.
  • the electronic device 40 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the schematic diagram 4 is only an example of the electronic device 40, and does not constitute a limitation on the electronic device 40. It may include more or less components than those shown in the figure, or combine certain components, or different components.
  • the electronic device 40 may also include input and output devices, network access devices, buses, and so on.
  • the so-called processor 402 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor can be a microprocessor or the processor 402 can also be any conventional processor, etc.
  • the processor 402 is the control center of the electronic device 40, which uses various interfaces and lines to connect the entire electronic device 40. Various parts.
  • the memory 401 may be used to store the computer-readable instructions, and the processor 402 executes or executes the computer-readable instructions stored in the memory 401 and calls data stored in the memory 401 to implement the electronic Various functions of the device 40.
  • the memory 401 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may Data (such as audio data, phone book, etc.) created according to the use of the electronic device 40 and the like are stored.
  • the memory 401 may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) Card, Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • a non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) Card, Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the integrated module of the electronic device 40 is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through computer-readable instructions.
  • the computer-readable instructions can be stored in a computer storage medium. When the computer-readable instructions are executed by the processor, they can implement the steps of the foregoing method embodiments.
  • one or more readable storage media storing computer readable instructions are provided.
  • the computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by one or more processors. When executed, the one or more processors are caused to execute the steps in the above-mentioned embodiment of the method for positioning the macular center of the fundus image.
  • the code of the computer-readable instruction may be in the form of source code, object code, executable file, or some intermediate form, etc.
  • the computer-readable medium may include: any entity or device capable of carrying the code of the computer-readable instruction, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read- Only Memory), Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc.
  • ROM Read- Only Memory
  • RAM Random Access Memory
  • electrical carrier signal telecommunications signal
  • software distribution media etc.
  • the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction.
  • the computer-readable medium Does not include electrical carrier signals and telecommunication signals.
  • the readable storage medium in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.
  • modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware, or in the form of hardware plus software functional modules.
  • the above-mentioned integrated modules implemented in the form of software functional modules may be stored in a computer readable storage medium.
  • the above-mentioned software function module is stored in a storage medium and includes several instructions to make an electronic device (which may be a personal computer, a server, or a network device, etc.) or a processor execute the method described in the various embodiments of the present application. Part of the steps.

Abstract

本申请涉及人工智能技术领域，提供一种眼底图像黄斑中心定位方法、装置、电子设备及存储介质，包括：将待检测的眼底图像输入至眼底图像检测模型中；获取眼底图像检测模型的检测结果，检测结果包括：视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；根据第一检测框计算视盘区域的中心点坐标和根据第二检测框计算黄斑区域的中心点坐标；当第二置信度小于预设第二置信度阈值时，识别待检测的眼底图像为左眼眼底图像还是右眼眼底图像并采用不同的修正模型对黄斑区域的中心点进行修正。解决因图像质量、病变遮挡等产生黄斑区域检测失败的问题，消除了传统方法中黄斑中心定位与视盘中心定位的依赖性。

Description

眼底图像黄斑中心定位方法、装置、电子设备及存储介质
本申请以2019年10月11日提交的申请号为201910964514.1，名称为“眼底图像黄斑中心定位方法、装置、电子设备及存储介质”的中国发明申请为基础，并要求其优先权。
技术领域
本申请涉及基于人工智能的医疗图像处理技术领域，具体涉及一种眼底图像黄斑中心定位方法、装置、电子设备及存储介质。
背景技术
黄斑集中了大量的视觉功能细胞，黄斑区的病变如果没有被及时发现和治疗，失明的几率大大提高，因而，准确的黄斑中心定位对视网膜病变诊断具有重要意义。现有技术中，有基于眼底图像的黄斑中心定位方法，先进行视盘中心定位，然后根据视盘中心进行黄斑中心定位；或者使用深度神经网络的目标检测模型直接对黄斑区域进行检测。
发明人意识到现有基于眼底图像的黄斑中心定位方法，完全依赖于视盘中心的定位，一旦视盘中心定位失败黄斑中心则无法有效定位，且计算复杂度高，时效性低，鲁棒性差。基于深度神经网络的黄斑中心定位方法虽然不依赖于视盘中心的定位，但由于黄斑极易受到病变区域、图像质量及萎缩区域对其遮挡的影响，导致黄斑区域无法被成功检测从而无法进行有效定位。因此，有必要提出一种新的眼底图像黄斑中心定位方案，能够不完全依赖于视盘中心的定位，且能够在黄斑区域被遮挡或眼底图像质量差的情况下对黄斑中心进行有效的定位。
发明内容
鉴于以上内容，有必要提出一种眼底图像黄斑中心定位方法、装置、电子设备及计算机存储介质，能够不完全依赖于视盘中心的定位，且能够在黄斑区域被遮挡或眼底图像质量差的情况下对黄斑中心进行有效的定位。
一种眼底图像黄斑中心定位方法，包括：
将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
一种眼底图像黄斑中心定位装置，包括：
输入模块，用于将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
获取模块，用于获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
计算模块，用于根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
比较模块，用于比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
识别模块，用于当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
修正模块，用于针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
一种电子设备，所述电子设备包括处理器，所述处理器用于执行存储器中存储的计算机可读指令以实现如下步骤：
将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
一个或多个存储有计算机可读指令的可读存储介质，所述计算机可读存储介质存储有计算机可读指令，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如下步骤：
将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
上述眼底图像黄斑中心定位方法、装置、电子设备及存储介质中，通过眼底图像检测模型输出待检测的眼底图像中的视盘区域及黄斑区域的检测框及对应检测框的置信度，再根据检测框计算出视盘区域和黄斑区域中心点的坐标，最后根据左右眼中黄斑区域和视盘区域的对应关系，利用视盘中心对未检出的黄斑区域及黄斑区域置信度较低的眼底图像进行修正。即使在黄斑区域被遮挡或眼底图像质量差的情况下依然可以对黄斑区域中心点进行有效的定位。解决了基于深度学习的黄盘定位方法中因图像质量、病变遮挡等产生黄斑区域检测失败的问题，消除了传统方法中黄斑中心定位与视盘中心定位的依赖性。
附图说明
[根据细则91更正 04.08.2020]　
图1是本申请实施例提供的眼底图像黄斑中心定位方法的流程图。
图2是本申请实施例所提供的Mask RCNN 网络的网络结构的示意图。
[根据细则91更正 04.08.2020]　
图3是本申请实施例提供的眼底图像黄斑中心定位装置的结构图。
[根据细则91更正 04.08.2020]　
图4是本申请实施例提供的电子设备的示意图。
具体实施方式
为了能够更清楚地理解本申请的上述目的、特征和优点，下面结合附图和具体实施例对本申请进行详细描述。需要说明的是，在不冲突的情况下，本申请的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本申请，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请。
优选地，本申请的眼底图像黄斑中心定位方法应用在一个或者多个电子设备中。所述电子设备是一种能够按照事先设定或存储的指令，自动进行数值计算和/或信息处理的设备，其硬件包括但不限于微处理器、专用集成电路(Application Specific Integrated Circuit，ASIC)、可编程门阵列(Field－Programmable Gate Array，FPGA)、数字处理器(Digital Signal Processor，DSP)、嵌入式设备等。
所述电子设备可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。所述电子设备可以与用户通过键盘、鼠标、遥控器、触摸板或声控设备等方式进行人机交互。
实施例一
图1是本申请实施例一提供的眼底图像黄斑中心定位方法的流程图。根据不同的需求，该流程图中的执行顺序可以改变，某些步骤可以省略。
S11，将待检测的眼底图像输入至预先训练好的眼底图像检测模型中。
眼底图像是通过眼部检测设备拍摄的用于诊断眼部病变的图像。眼底是指眼球内后部的组织，包括了视网膜、视乳头、黄斑和视网膜中央动静脉，因而，所述待检测的眼底图像中包括了黄斑及其周围所形成的黄斑区域、视网膜及其周围所形成的视盘区域。
所述眼底图像检测模型是利用已知特征区域所在位置的样本图像进行训练得到的，输入是一张眼底图像，输出的是一张标记了特征区域的眼底图像。其中，所述特征区域为视盘区域、黄斑区域中的至少一种。
在一个可选的实施例中，在S1(将待检测的眼底图像输入至预先训练好的眼底图像检测模型中)之前，所述方法还包括：
训练所述眼底图像检测模型。
通过利用大量的样本数据对深度神经网络进行训练得到眼底图像检测模型，以使未做标记的眼底图像输入眼底图像检测模型后，眼底图像检测模型输出标记了一种或多种特征区域的眼底图像。
根据本申请的一个优选实施例，所述眼底图像检测模型的训练过程包括：
获取多张眼底图像；
对每张眼底图像中的一种或多种特征区域进行标记，所述特征区域为黄斑区域、视盘区域；
基于标记了一种或多种特征区域的眼底图像及对应特征区域的类别构建样本数据集；
将所述样本数据集随机分为第一数量的训练集和第二数量的测试集；
将所述训练集输入至Mask RCNN网络中进行训练，得到眼底图像检测模型；
将所述测试集输入至所述眼底图像检测模型中进行测试，得到测试通过率；
判断所述测试通过率是否大于预设通过率阈值；
当所述测试通过率大于或者等于所述预设通过率阈值时，结束眼底图像检测模型的训练；否则，当所述测试通过率小于所述预设通过率阈值时，增加所述训练集的数量并基于增加数量后的训练集训练Mask RCNN网络直至所述测试通过率大于或者等于所述预设通过率阈值。
其中，眼底图像可以包括健康人员和眼部疾病患者的左眼眼底图像或右眼眼底图像。眼底图像中的特征区域可以是由人工或其他自动识别方法对眼底图像进行识别和标记得到的，标记的内容可以是黄斑区域、视盘区域中的任意一个或多个。
示例性的，假设获取了10万张眼底图像，对这10张眼底图像中的黄斑区域、视盘区域中的任意一个或多个进行标记，将标记了一个或多个特征区域的眼底图像作为数据集，再将数据集划分为训练集和测试集，训练集中标记了一个或多个特征区域的眼底图像的数量大于测试集中标记了一个或多个特征区域的眼底图像的数量，例如将标记了一个或多个特征区域的眼底图像中的80％的眼底图像作为训练集，将剩余的30％的眼底图像作为测试集。选择Mask RCNN网络作为眼底图像检测模型的原型，初始化Mask RCNN网络时采用默认的输入参数，在训练的过程中不断调整输入参数，在训练生成眼底图像检测模型后，利用测试集中的眼底图像对所训练得到的眼底图像检测模型进行验证，如果测试通过率小于预设通过率阈值时，例如通过率小于98％，则增加参与训练的眼底图像的数量并重新训练Mask RCNN网络，直至训练得到的眼底图像检测模型的测试通过率大于或者等于预设通过率阈值即可。
所述Mask RCNN网络为现有技术，本申请在此不做详细阐述。
根据本申请的一个优选实施例，在所述获取多张眼底图像之后，所述方法还包括：
对所述眼底图像进行预定角度的翻转处理；
对翻转处理后的眼底图像标记一种或多种所述特征区域；
所述基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建样本数据集包括：基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建第一样本数据集，基于标记了一种或多种所述特征区域的翻转处理后的眼底图像及对应特征区域的类别构建第二样本数据集，将所述第一样本数据集和所述第二样本数据集作为所述样本数据集。
其中，所述翻转处理包括随机翻转、镜像翻转、水平翻转和其他旋转，例如旋转的角度可以从-15度到+15度。
所述初始数据集包括第一数据集和第二数据集，其中，所述第一数据集是由眼部检测设备拍摄的和标记的眼底图像，所述第二数据集是通过对所述第一数据集进行翻转、镜像等处理得到的。如此，扩充了第一数据集中的眼底图像，达到了补充眼底图像的目的，基于更大量的数据集训练眼底图像检测模型，能够提高眼底图像检测模型的检测精度，提升眼底图像检测模型的泛化性能。
在一个可选的实施例中，为了提高眼底图像检测模型的检测速率，所述方法还包括：
对所述Mask RCNN网络的网络结构进行改进。
其中，改进后的Mask RCNN网络的网络结构如下图2所示。
根据本申请的一个优选实施例，所述Mask RCNN网络的网络结构包括：
1)多层主干网络，每层主干网络采用MobileNet V2网络；
所述每层主干网络用于提取输入的眼底图像的特征，为自上向下的结构，逐层进行下采样处理。
示例性的，参阅上图所示，假设输入一张1024*1024的眼底图像F0，第1层的主干网络C1对输入的眼底图像F0进行特征提取，输出一个256*256的特征图F1至第2层的主干网络C2；第2层的主干网络C2对输入的特征图F1进行特征提取，输出一个128*128的特征图F2至第3层的主干网络C3；第3层的主干网络C3对输入的特征图F2进行特 征提取，输出一个64*64的特征图F3至第4层的主干网络C4；以此类推；上一层的主干网络输出的特征图的大小为下一层的主干网络输出的特征图的2倍。
2)多层特征金字塔网络层，上一层特征金字塔网络层的输入为下一层特征金字塔网络层的输出及与所述上一层特征金字塔网络层位于同一层的主干网络的输出之和；
特征金字塔网络层(Feature Pyramid Networks，FPN)用于对每一层的特征图进行预测，为自下向上的结构，逐层进行上采样处理。
示例性的，参阅上图所示，第5层特征金字塔网络层P5的输入为与第5层位于同一层的主干网络的输入(相当于进行了1X1的卷积)；第4层特征金字塔网络层P4的输入为第5层特征金字塔网络层P5的输出和与第4层位于同一层的主干网络C4(第4层主干网络)的输出之和；第3层特征金字塔网络层P3的输入为第4层特征金字塔网络层P4的输出和与第3层位于同一层的主干网络C3(第3层主干网络)的输出之和；以此类推。
3)注意力层。
注意力模块层(Attention Block)，使得网络的特征提取能力更专注于视盘区域和黄斑区域同时减少其他区域引入的噪声。
4)池化层；
该层分别对[P2 P3 P4 P5]及注意力层四个不同尺度得到的特征图对应的stride进行RoIAlign操作生成RoI，得到固定大小的proposal feature map，输入至全连接层来进行目标检测和定位。
5)全连接层；
[根据细则91更正 04.08.2020]　
该层对池化层输出的proposal feature map进行Concat连接，随即网络分为三部分：全连接预测类别class、全连接预测矩形框box、全连接预测置信度score。
6)输出层。
该层用于输出三个值，分别是眼底图像中特征区域的类别、眼底图像中特征区域的检测框及所述检测框的置信度。特征区域为视盘区域和黄斑区域中的任意一种或多种。
在该可选的实施例中，通过对Mask RCNN网络进行改进，主干网络采用轻量级的Mobile Net V2网络，减少了网络参数的计算量，能够提升眼底图像检测模型的检测速度。此外，在特征金字塔网络层之后加入了注意力层，使得网络的特征提取能力更专注于视盘区域和黄斑区域，进一步提升了眼底图像检测模型的检测速度，提升了眼底图像检测模型的检测精度。
S12，获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度。
所述眼底图像检测模型输出的是一张标记了特征区域的眼底图像。即输出的是一张眼底图像，该眼底图像中用检测框标记出了黄斑区域和/或视盘区域。
所述置信度用以表明所述眼底图像检测模型检测出的特征区域的精确度，取值范围为0-1。
经过实验统计，采用所训练出的眼底图像检测模型处理任意一张眼底图像时，对于大多数，例如80％的眼底图像，眼底图像检测模型均能够且准确的输出标记了视盘区域的眼底图像，即能够用检测框准确的标记出视盘区域所在的位置，得到的置信度也较高。输出的眼底图像中有可能标记了黄斑区域，也有可能没有标记黄斑区域。而由于黄斑区域因容易发生病变或者存在遮挡等，导致通过眼底图像检测模型检测出的黄斑区域会存在误差，其对应的置信度有可能较高，也有可能较低。如果眼底图像检测模型输出的眼底图像直接被标记了黄斑区域且置信度较高，则可以直接将其作为结果；如果眼底图像检测模型输出的眼底图像被标记了黄斑区域但置信度较低，或输出的眼底图像未被标记黄斑区域，可以根据视盘区域的位置修正黄斑区域的位置。
需要说明的是，检测框的轮廓呈矩形，该形状是由眼底图像检测模型训练过程中的样本数据决定的，如果在训练过程中改变样本数据中标记的形状，例如使用圆形、不规则形状等，训练出的眼底图像检测模型也将进行相应形状的标记，从而输出相应形状的轮廓。
S13，根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标。
所述眼底图像检测模型输出标记了特征区域的眼底图像之后，可以从标记了特征区域的眼底图像中获取视盘区域的检测框和黄斑区域的检测框。
以标记了特征区域的眼底图像的左上角为原点，以上边界为X轴，以左边界为Y轴，建立XOY坐标系，然后获取XOY坐标系中视盘区域的检测框的各个顶点的第一坐标，及黄斑区域的检测框的各个顶点的第二坐标。
根据多个第一坐标即可计算得到视盘区域的中心点坐标，根据多个第二坐标即可计算得到黄斑区域的中心点坐标。
示例性的，假设视盘区域的检测框的各个顶点的第一坐标分别为(a，b)、(c，b)、(a，d)、(c，d)，则视盘区域的中心点坐标为
Figure PCTCN2020093338-appb-000001
假设黄斑区域的检测框的各个顶点的第二坐标分别为(s，t)、(m，t)、(s，n)、(m，n)，则视盘区域的中心点坐标为
Figure PCTCN2020093338-appb-000002
S14，比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值。
其中，预设第一置信度阈值和预设第二置信度阈值均为预先设置的用以判定黄斑区域的检测框的正确程度的临界值。所述预设第一置信度阈值大于所述预设第二置信度阈值。
S15，当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像。
其中，左右眼识别模型可以采用线下训练的方式预先训练好。
具体的训练过程为：采集多张眼底图像，并对每张眼底图像进行标识以表明所述眼底图像为左眼眼底图像或右眼眼底图像，将眼底图像及对应的标识作为数据集，然后将数据集划分为训练集和测试集，训练集的数量大于测试集的数量，基于训练集对深度神经网络，例如，卷积神经网络进行训练，得到左右眼识别模型，再基于测试集测试所训练好的左右眼识别模型的测试通过率，当测试通过率大于或等于预设通过率阈值时，输出所述左右眼识别模型，当测试通过率小于预设通过率阈值时，重新划分训练集和测试集并基于新的训练集训练左右眼识别模型直到测试通过率大于或等于预设通过率阈值。
由于左眼眼底图像和右眼眼底图像中，黄斑区域和视盘区域的相对位置有所差异，左眼眼底图像中，视盘区域在左，黄斑区域在右，黄斑区域在视盘区域的右边；右眼眼底图像中，视盘区域在右，黄斑区域在左，黄斑区域在视盘区域的左边。
当所述第二置信度小于预设第一置信度阈值时，表明眼底图像检测模型检测出的黄斑区域置信度非常低，或者没有检测出黄斑区域，此时还需进一步识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像，并进而采用不同的修正方式修正黄斑区域的检测框，并进而修正黄斑区域的中心点。
S16，针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
所述针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正包括：
针对所述左眼眼底图像，采用第一修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第一修正模型为：
Figure PCTCN2020093338-appb-000003
Figure PCTCN2020093338-appb-000004
针对所述右眼眼底图像，采用第二修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第二修正模型为：
Figure PCTCN2020093338-appb-000005
Figure PCTCN2020093338-appb-000006
其中，W表示所述待检测的眼底图像的宽，H表示所述待检测的眼底图像的高，(x oc，y oc)为计算得到的所述视盘区域的中心点坐标，h为所述视盘区域对应的所述第一检测框的高，(x fovea，y fovea)为修正得到的黄斑区域的中心点坐标。
根据本申请的一个优选实施例，所述方法包括：
当所述第二置信度小于所述预设第一置信度阈值且大于所述预设第二置信度阈值时，采用第三修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第三修正模型为：
x fc＝0.5*x dc+0.5*x fovea，
y fc＝0.5*y dc+0.5*y fovea，
其中，(x fc，y fc)是最终修正得到的黄斑区域的中心点坐标，(x dc，y dc)是计算得到的所述黄斑区域的中心点坐标，(x fovea，y fovea)是根据视盘区域中心点坐标修正得到的黄斑区域中心点坐标。
第二置信度介于预设第一置信度阈值和预设第二置信度阈值之间时，表明眼底图像检测模型检测出黄斑区域，但检测出的黄斑区域的置信度不高，标记的黄斑区域的检测框准确度不高，需要修正黄斑区域的检测框，并进而修正黄斑区域的中心点。
根据本申请的一个优选实施例，所述方法包括：
当所述第二置信度大于所述预设第一置信度阈值时，将根据所述第二检测框计算得到的所述黄斑区域的中心点坐标作为最终的黄斑区域中心点坐标。
若眼底图像检测模型检测出黄斑区域且黄斑区域对应的置信度较高(大于预设第一置信度阈值)，认为眼底图像检测模型检测出的黄斑区域是较好的，此时可以不需要眼底图像检测模型检测出视盘区域。仅当眼底图像检测模型不能检测出黄斑区域或者检测出黄斑区域但黄斑区域对应的置信度较低(小于预设第一置信度阈值)，此时才需要依赖视盘区域的检测结果对黄斑区域进行修正。因而，采用上述技术方案，可以在一定程度上减少对视盘区域的依赖。
需要说明的是，上述公式中出现的0.3、07、2.42、0.39及0.5等均是通过大量实验研究确定的。
综上，本申请提供的眼底图像黄斑中心定位方法，通过基于Mask RCNN网络训练得到眼底图像检测模型，通过眼底图像检测模型输出待检测的眼底图像中的视盘区域及黄斑区域的检测框及对应检测框的置信度，再根据检测框计算出视盘区域和黄斑区域中心点的坐标，最后根据左右眼中黄斑区域和视盘区域的对应关系，利用视盘中心对未检出的黄斑区域及黄斑区域置信度较低的眼底图像进行修正。即使在黄斑区域被遮挡或眼底图像质量差的情况下依然可以对黄斑区域中心点进行有效的定位。解决了基于深度学习的黄盘定位方法中因图像质量、病变遮挡等产生黄斑区域检测失败的问题。且，眼底图像检测模型分别对黄斑区域和视盘区域独立检测，使得即使在眼底图像中视盘区域缺失 的情况下，黄斑中心依然可以有效定位，消除了传统方法中黄斑中心定位与视盘中心定位的依赖性。
此外，对Mask RCNN网络结构进行改变，减少了网络参数的计算量，提升了眼底图像检测模型的检测速度，提高了黄斑中心定位的时效性，在特征金字塔网络层之后加入了注意力层，使得网络的特征提取能力更专注于视盘区域和黄斑区域，进一步提升了眼底图像检测模型的检测速度，提升了眼底图像检测模型的检测精度，辅助提升了对黄斑区域中心点的定位。
实施例二
图2是本申请实施例二提供的眼底图像黄斑中心定位装置的结构图。
所述眼底图像黄斑中心定位装置30运行于电子设备中，能够解决基于深度学习的黄盘定位方法中因图像质量、病变遮挡等产生黄斑区域检测失败的问题，且能够消除传统方法中黄斑中心定位与视盘中心定位的依赖性。如图3所示，所述眼底图像黄斑中心定位装置30可以包括：输入模块301、训练模块302、获取模块303、计算模块304、比较模块305、识别模块306、修正模块307及确定模块308。
输入模块301，用于将待检测的眼底图像输入至预先训练好的眼底图像检测模型中。
眼底图像是通过眼部检测设备拍摄的用于诊断眼部病变的图像。眼底是指眼球内后部的组织，包括了视网膜、视乳头、黄斑和视网膜中央动静脉，因而，所述待检测的眼底图像中包括了黄斑及其周围所形成的黄斑区域、视网膜及其周围所形成的视盘区域。
所述眼底图像检测模型是利用已知特征区域所在位置的样本图像进行训练得到的，输入是一张眼底图像，输出的是一张标记了特征区域的眼底图像。其中，所述特征区域为视盘区域、黄斑区域中的至少一种。
在一个可选的实施例中，在输入模块301将待检测的眼底图像输入至预先训练好的眼底图像检测模型中之前，所述眼底图像黄斑中心定位装置30还包括：
训练模块302，用于训练所述眼底图像检测模型。
通过利用大量的样本数据对深度神经网络进行训练得到眼底图像检测模型，以使未做标记的眼底图像输入眼底图像检测模型后，眼底图像检测模型输出标记了一种或多种特征区域的眼底图像。
根据本申请的一个优选实施例，所述训练模块302训练眼底图像检测模型的训练过程包括：
获取多张眼底图像；
对每张眼底图像中的一种或多种特征区域进行标记，所述特征区域为黄斑区域、视盘区域；
基于标记了一种或多种特征区域的眼底图像及对应特征区域的类别构建样本数据集；
将所述样本数据集随机分为第一数量的训练集和第二数量的测试集；
将所述训练集输入至Mask RCNN网络中进行训练，得到眼底图像检测模型；
将所述测试集输入至所述眼底图像检测模型中进行测试，得到测试通过率；
判断所述测试通过率是否大于预设通过率阈值；
当所述测试通过率大于或者等于所述预设通过率阈值时，结束眼底图像检测模型的训练；否则，当所述测试通过率小于所述预设通过率阈值时，增加所述训练集的数量并基于增加数量后的训练集训练Mask RCNN网络直至所述测试通过率大于或者等于所述预设通过率阈值。
其中，眼底图像可以包括健康人员和眼部疾病患者的左眼眼底图像或右眼眼底图像。眼底图像中的特征区域可以是由人工或其他自动识别方法对眼底图像进行识别和标记得到的，标记的内容可以是黄斑区域、视盘区域中的任意一个或多个。
示例性的，假设获取了10万张眼底图像，对这10张眼底图像中的黄斑区域、视盘 区域中的任意一个或多个进行标记，将标记了一个或多个特征区域的眼底图像作为数据集，再将数据集划分为训练集和测试集，训练集中标记了一个或多个特征区域的眼底图像的数量大于测试集中标记了一个或多个特征区域的眼底图像的数量，例如将标记了一个或多个特征区域的眼底图像中的80％的眼底图像作为训练集，将剩余的30％的眼底图像作为测试集。选择Mask RCNN网络作为眼底图像检测模型的原型，初始化Mask RCNN网络时采用默认的输入参数，在训练的过程中不断调整输入参数，在训练生成眼底图像检测模型后，利用测试集中的眼底图像对所训练得到的眼底图像检测模型进行验证，如果测试通过率小于预设通过率阈值时，例如通过率小于98％，则增加参与训练的眼底图像的数量并重新训练Mask RCNN网络，直至训练得到的眼底图像检测模型的测试通过率大于或者等于预设通过率阈值即可。
所述Mask RCNN网络为现有技术，本申请在此不做详细阐述。
根据本申请的一个优选实施例，在所述获取多张眼底图像之后，所述方法还包括：
对所述眼底图像进行预定角度的翻转处理；
对翻转处理后的眼底图像标记一种或多种所述特征区域；
所述基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建样本数据集包括：基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建第一样本数据集，基于标记了一种或多种所述特征区域的翻转处理后的眼底图像及对应特征区域的类别构建第二样本数据集，将所述第一样本数据集和所述第二样本数据集作为所述样本数据集。
其中，所述翻转处理包括随机翻转、镜像翻转、水平翻转和其他旋转，例如旋转的角度可以从-15度到+15度。
所述初始数据集包括第一数据集和第二数据集，其中，所述第一数据集是由眼部检测设备拍摄的和标记的眼底图像，所述第二数据集是通过对所述第一数据集进行翻转、镜像等处理得到的。如此，扩充了第一数据集中的眼底图像，达到了补充眼底图像的目的，基于更大量的数据集训练眼底图像检测模型，能够提高眼底图像检测模型的检测精度，提升眼底图像检测模型的泛化性能。
在一个可选的实施例中，为了提高眼底图像检测模型的检测速率，所述眼底图像黄斑中心定位装置30还包括：
对所述Mask RCNN网络的网络结构进行改进。
其中，改进后的Mask RCNN网络的网络结构如下图2所示。
根据本申请的一个优选实施例，所述Mask RCNN网络的网络结构包括：
1)多层主干网络，每层主干网络采用MobileNet V2网络；
所述每层主干网络用于提取输入的眼底图像的特征，为自上向下的结构，逐层进行下采样处理。
示例性的，参阅上图所示，假设输入一张1024*1024的眼底图像F0，第1层的主干网络C1对输入的眼底图像F0进行特征提取，输出一个256*256的特征图F1至第2层的主干网络C2；第2层的主干网络C2对输入的特征图F1进行特征提取，输出一个128*128的特征图F2至第3层的主干网络C3；第3层的主干网络C3对输入的特征图F2进行特征提取，输出一个64*64的特征图F3至第4层的主干网络C4；以此类推；上一层的主干网络输出的特征图的大小为下一层的主干网络输出的特征图的2倍。
2)多层特征金字塔网络层，上一层特征金字塔网络层的输入为下一层特征金字塔网络层的输出及与所述上一层特征金字塔网络层位于同一层的主干网络的输出之和；
特征金字塔网络层(Feature Pyramid Networks，FPN)用于对每一层的特征图进行预测，为自下向上的结构，逐层进行上采样处理。
示例性的，参阅上图所示，第5层特征金字塔网络层P5的输入为与第5层位于同一层的主干网络的输入(相当于进行了1X1的卷积)；第4层特征金字塔网络层P4的输入 为第5层特征金字塔网络层P5的输出和与第4层位于同一层的主干网络C4(第4层主干网络)的输出之和；第3层特征金字塔网络层P3的输入为第4层特征金字塔网络层P4的输出和与第3层位于同一层的主干网络C3(第3层主干网络)的输出之和；以此类推。
3)注意力层。
注意力模块层(Attention Block)，使得网络的特征提取能力更专注于视盘区域和黄斑区域同时减少其他区域引入的噪声。
4)池化层；
该层分别对[P2 P3 P4 P5]及注意力层四个不同尺度得到的特征图对应的stride进行RoIAlign操作生成RoI，得到固定大小的proposal feature map，输入至全连接层来进行目标检测和定位。
5)全连接层；
该层对池化层输出的proposal feature map进行Concat连接，随即网络分为三部分：全连接预测类别class、全连接预测矩形框box、全连接预测置信度。
6)输出层。
该层用于输出三个值，分别是眼底图像中特征区域的类别、眼底图像中特征区域的检测框及所述检测框的置信度。特征区域为视盘区域和黄斑区域中的任意一种或多种。
在该可选的实施例中，通过对Mask RCNN网络进行改进，主干网络采用轻量级的Mobile Net V2网络，减少了网络参数的计算量，能够提升眼底图像检测模型的检测速度。此外，在特征金字塔网络层之后加入了注意力层，使得网络的特征提取能力更专注于视盘区域和黄斑区域，进一步提升了眼底图像检测模型的检测速度，提升了眼底图像检测模型的检测精度。
获取模块303，用于获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度。
所述眼底图像检测模型输出的是一张标记了特征区域的眼底图像。即输出的是一张眼底图像，该眼底图像中用检测框标记出了黄斑区域和/或视盘区域。
所述置信度用以表明所述眼底图像检测模型检测出的特征区域的精确度，取值范围为0-1。
经过实验统计，采用所训练出的眼底图像检测模型处理任意一张眼底图像时，对于大多数，例如80％的眼底图像，眼底图像检测模型均能够且准确的输出标记了视盘区域的眼底图像，即能够用检测框准确的标记出视盘区域所在的位置，得到的置信度也较高。输出的眼底图像中有可能标记了黄斑区域，也有可能没有标记黄斑区域。而由于黄斑区域因容易发生病变或者存在遮挡等，导致通过眼底图像检测模型检测出的黄斑区域会存在误差，其对应的置信度有可能较高，也有可能较低。如果眼底图像检测模型输出的眼底图像直接被标记了黄斑区域且置信度较高，则可以直接将其作为结果；如果眼底图像检测模型输出的眼底图像被标记了黄斑区域但置信度较低，或输出的眼底图像未被标记黄斑区域，可以根据视盘区域的位置修正黄斑区域的位置。
需要说明的是，检测框的轮廓呈矩形，该形状是由眼底图像检测模型训练过程中的样本数据决定的，如果在训练过程中改变样本数据中标记的形状，例如使用圆形、不规则形状等，训练出的眼底图像检测模型也将进行相应形状的标记，从而输出相应形状的轮廓。
计算模块304，用于根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标。
所述眼底图像检测模型输出标记了特征区域的眼底图像之后，可以从标记了特征区域的眼底图像中获取视盘区域的检测框和黄斑区域的检测框。
以标记了特征区域的眼底图像的左上角为原点，以上边界为X轴，以左边界为Y轴，建立XOY坐标系，然后获取XOY坐标系中视盘区域的检测框的各个顶点的第一坐标，及黄斑区域的检测框的各个顶点的第二坐标。
根据多个第一坐标即可计算得到视盘区域的中心点坐标，根据多个第二坐标即可计算得到黄斑区域的中心点坐标。
示例性的，假设视盘区域的检测框的各个顶点的第一坐标分别为(a，b)、(c，b)、(a，d)、(c，d)，则视盘区域的中心点坐标为
Figure PCTCN2020093338-appb-000007
假设黄斑区域的检测框的各个顶点的第二坐标分别为(s，t)、(m，t)、(s，n)、(m，n)，则视盘区域的中心点坐标为
Figure PCTCN2020093338-appb-000008
比较模块305，用于比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值。
其中，预设第一置信度阈值和预设第二置信度阈值均为预先设置的用以判定黄斑区域的检测框的正确程度的临界值。所述预设第一置信度阈值大于所述预设第二置信度阈值。
识别模块306，用于当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像。
其中，左右眼识别模型可以采用线下训练的方式预先训练好。
具体的训练过程为：采集多张眼底图像，并对每张眼底图像进行标识以表明所述眼底图像为左眼眼底图像或右眼眼底图像，将眼底图像及对应的标识作为数据集，然后将数据集划分为训练集和测试集，训练集的数量大于测试集的数量，基于训练集对深度神经网络，例如，卷积神经网络进行训练，得到左右眼识别模型，再基于测试集测试所训练好的左右眼识别模型的测试通过率，当测试通过率大于或等于预设通过率阈值时，输出所述左右眼识别模型，当测试通过率小于预设通过率阈值时，重新划分训练集和测试集并基于新的训练集训练左右眼识别模型直到测试通过率大于或等于预设通过率阈值。
由于左眼眼底图像和右眼眼底图像中，黄斑区域和视盘区域的相对位置有所差异，左眼眼底图像中，视盘区域在左，黄斑区域在右，黄斑区域在视盘区域的右边；右眼眼底图像中，视盘区域在右，黄斑区域在左，黄斑区域在视盘区域的左边。
当所述第二置信度小于预设第一置信度阈值时，表明眼底图像检测模型检测出的黄斑区域置信度非常低，或者没有检测出黄斑区域，此时还需进一步识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像，并进而采用不同的修正方式修正黄斑区域的检测框，并进而修正黄斑区域的中心点。
修正模块307，用于针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
所述针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正包括：
针对所述左眼眼底图像，采用第一修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第一修正模型为：
Figure PCTCN2020093338-appb-000009
Figure PCTCN2020093338-appb-000010
针对所述右眼眼底图像，采用第二修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第二修正模型为：
Figure PCTCN2020093338-appb-000011
Figure PCTCN2020093338-appb-000012
其中，W表示所述待检测的眼底图像的宽，H表示所述待检测的眼底图像的高，(x oc，y oc)为计算得到的所述视盘区域的中心点坐标，h为所述视盘区域对应的所述第一检测框的高，(x fovea，y fovea)为修正得到的黄斑区域的中心点坐标。
根据本申请的一个优选实施例，所述修正模块307还用于：
当所述第二置信度小于所述预设第一置信度阈值且大于所述预设第二置信度阈值时，采用第三修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第三修正模型为：
x fc＝0.5*x dc+0.5*x fovea，
y fc＝0.5*y dc+0.5*y fovea，
其中，(x fc，y fc)是最终修正得到的黄斑区域的中心点坐标，(x dc，y dc)是计算得到的所述黄斑区域的中心点坐标，(x fovea，y fovea)是根据视盘区域中心点坐标修正得到的黄斑区域中心点坐标。
第二置信度介于预设第一置信度阈值和预设第二置信度阈值之间时，表明眼底图像检测模型检测出黄斑区域，但检测出的黄斑区域的置信度不高，标记的黄斑区域的检测框准确度不高，需要修正黄斑区域的检测框，并进而修正黄斑区域的中心点。
根据本申请的一个优选实施例，当所述第二置信度大于所述预设第一置信度阈值时，所述眼底图像黄斑中心定位装置30还包括：
确定模块308，用于将根据所述第二检测框计算得到的所述黄斑区域的中心点坐标作为最终的黄斑区域中心点坐标。
若眼底图像检测模型检测出黄斑区域且黄斑区域对应的置信度较高(大于预设第一置信度阈值)，认为眼底图像检测模型检测出的黄斑区域是较好的，此时可以不需要眼底图像检测模型检测出视盘区域。仅当眼底图像检测模型不能检测出黄斑区域或者检测出黄斑区域但黄斑区域对应的置信度较低(小于预设第一置信度阈值)，此时才需要依赖视盘区域的检测结果对黄斑区域进行修正。因而，采用上述技术方案，可以在一定程度上减少对视盘区域的依赖。
需要说明的是，上述公式中出现的0.3、07、2.42、0.39及0.5等均是通过大量实验研究确定的。
综上，本申请提供的眼底图像黄斑中心定位装置，通过基于Mask RCNN网络训练得到眼底图像检测模型，通过眼底图像检测模型输出待检测的眼底图像中的视盘区域及黄斑区域的检测框及对应检测框的置信度，再根据检测框计算出视盘区域和黄斑区域中心点的坐标，最后根据左右眼中黄斑区域和视盘区域的对应关系，利用视盘中心对未检出的黄斑区域及黄斑区域置信度较低的眼底图像进行修正。即使在黄斑区域被遮挡或眼底图像质量差的情况下依然可以对黄斑区域中心点进行有效的定位。解决了基于深度学习的黄盘定位方法中因图像质量、病变遮挡等产生黄斑区域检测失败的问题。且，眼底图像检测模型分别对黄斑区域和视盘区域独立检测，使得即使在眼底图像中视盘区域缺失的情况下，黄斑中心依然可以有效定位，消除了传统方法中黄斑中心定位与视盘中心定位的依赖性。
此外，对Mask RCNN网络结构进行改变，减少了网络参数的计算量，提升了眼底图像检测模型的检测速度，提高了黄斑中心定位的时效性，在特征金字塔网络层之后加入了注意力层，使得网络的特征提取能力更专注于视盘区域和黄斑区域，进一步提升了眼底图像检测模型的检测速度，提升了眼底图像检测模型的检测精度，辅助提升了对黄斑 区域中心点的定位。
实施例三
图3为本申请实施例三提供的电子设备的示意图。所述电子设备40包括存储器401、处理器402、通信总线403以及收发器404，所述存储器401中存储有可在所述处理器402上运行的计算机可读指令，例如眼底图像黄斑中心定位程序。所述处理器402执行所述计算机可读指令时实现上述眼底图像黄斑中心定位方法实施例中的步骤。即电子设备40包括处理器402，处理器402用于执行存储器401中存储的计算机可读指令以实现上述眼底图像黄斑中心定位方法实施例中的步骤。
示例性的，所述计算机可读指令可以被分割成一个或多个模块，所述一个或者多个模块被存储在所述存储器401中，并由所述处理器402执行，以完成本方法。所述一个或多个模块可以是能够完成特定功能的一系列计算机可读指令的指令段，该指令段用于描述所述计算机可读指令在所述电子设备40中的执行过程。例如，所述计算机可读指令可以被分割成图3中的输入模块301、训练模块302、获取模块303、计算模块304、比较模块305、识别模块306、修正模块307及确定模块308，各模块具体功能参见实施例二。
所述电子设备40可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。本领域技术人员可以理解，所述示意图4仅仅是电子设备40的示例，并不构成对电子设备40的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述电子设备40还可以包括输入输出设备、网络接入设备、总线等。
所称处理器402可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器402也可以是任何常规的处理器等，所述处理器402是所述电子设备40的控制中心，利用各种接口和线路连接整个电子设备40的各个部分。
所述存储器401可用于存储所述计算机可读指令，所述处理器402通过运行或执行存储在所述存储器401内的计算机可读指令，以及调用存储在存储器401内的数据，实现所述电子设备40的各种功能。所述存储器401可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据电子设备40的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器401可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
所述电子设备40集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于计算机存储介质中，该计算机可读指令在被处理器执行时，可实现上述各个方法实施例的步骤。
在一实施例中，提供一个或多个存储有计算机可读指令的可读存储介质，所述计算机可读存储介质存储有计算机可读指令，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行上述眼底图像黄斑中心定位方法实施例中的步骤。所述计算机可读指令的代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机可读指令的代码的任何实 体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。本实施例中的可读存储介质包括非易失性可读存储介质和易失性可读存储介质。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用硬件加软件功能模块的形式实现。
上述以软件功能模块的形式实现的集成的模块，可以存储在一个计算机可读取存储介质中。上述软件功能模块存储在一个存储介质中，包括若干指令用以使得一台电子设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施例所述方法的部分步骤。
对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附关联图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他模块或步骤，单数不排除复数。系统权利要求中陈述的多个模块或装置也可以由一个模块或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。
最后应说明的是，以上实施例仅用以说明本申请的技术方案而非限制，尽管参照较佳实施例对本申请进行了详细说明，本领域的普通技术人员应当理解，可以对本申请的技术方案进行修改或等同替换，而不脱离本申请技术方案的精神和范围。

Claims (20)

  1. 一种眼底图像黄斑中心定位方法，其中，所述方法包括：
    将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
    获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
    根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
    比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
    当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
    针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
  2. 如权利要求1所述的方法，其中，所述针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正包括：
    针对所述左眼眼底图像，采用第一修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第一修正模型为：
    Figure PCTCN2020093338-appb-100001
    Figure PCTCN2020093338-appb-100002
    针对所述右眼眼底图像，采用第二修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第二修正模型为：
    Figure PCTCN2020093338-appb-100003
    Figure PCTCN2020093338-appb-100004
    其中，W表示所述待检测的眼底图像的宽，H表示所述待检测的眼底图像的高，(x oc，y oc)为计算得到的所述视盘区域的中心点坐标，h为所述视盘区域对应的所述第一检测框的高，(x fovea，y fovea)为修正得到的黄斑区域的中心点坐标。
  3. 如权利要求2所述的方法，其中，所述方法包括：
    当所述第二置信度小于所述预设第一置信度阈值且大于所述预设第二置信度阈值时，采用第三修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第三修正模型为：
    x fc＝0.5*x dc+0.5*x fovea，
    y fc＝0.5*y dc+0.5*y fovea，
    其中，(x fc，y fc)是最终修正得到的黄斑区域的中心点坐标，(x dc，y dc)是计算得到的所述黄斑区域的中心点坐标，(x fovea，y fovea)是根据视盘区域中心点坐标修正得到的黄斑区域中心点坐标。
  4. 如权利要求1所述的方法，其中，所述方法包括：
    当所述第二置信度大于所述预设第一置信度阈值时，将根据所述第二检测框计算得到的所述黄斑区域的中心点坐标作为最终的黄斑区域中心点坐标。
  5. 如权利要求1所述的方法，其中，所述眼底图像检测模型的训练过程包括：
    获取多张眼底图像；
    对每张眼底图像中的一种或多种特征区域进行标记，所述特征区域为黄斑区域、视盘区域；
    基于标记了一种或多种特征区域的眼底图像及对应特征区域的类别构建样本数据集；
    将所述样本数据集随机分为第一数量的训练集和第二数量的测试集；
    将所述训练集输入至Mask RCNN网络中进行训练，得到眼底图像检测模型；
    将所述测试集输入至所述眼底图像检测模型中进行测试，得到测试通过率；
    判断所述测试通过率是否大于预设通过率阈值；
    当所述测试通过率大于或者等于所述预设通过率阈值时，结束眼底图像检测模型的训练；否则，当所述测试通过率小于所述预设通过率阈值时，增加所述训练集的数量并基于增加数量后的训练集训练Mask RCNN网络直至所述测试通过率大于或者等于所述预设通过率阈值。
  6. 如权利要求5所述的方法，其中，在所述获取多张眼底图像之后，所述方法还包括：
    对所述眼底图像进行预定角度的翻转处理；
    对翻转处理后的眼底图像标记一种或多种所述特征区域；
    所述基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建样本数据集包括：基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建第一样本数据集，基于标记了一种或多种所述特征区域的翻转处理后的眼底图像及对应特征区域的类别构建第二样本数据集，将所述第一样本数据集和所述第二样本数据集作为所述样本数据集。
  7. 如权利要求5所述的方法，其中，所述Mask RCNN网络的网络结构包括：
    多层主干网络，每层主干网络采用MobileNet V2网络；
    多层特征金字塔网络层，上一层特征金字塔网络层的输入为下一层特征金字塔网络层的输出及与所述上一层特征金字塔网络层位于同一层的主干网络的输出之和；
    注意力层。
  8. 一种眼底图像黄斑中心定位装置，其中，所述装置包括：
    输入模块，用于将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
    获取模块，用于获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
    计算模块，用于根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
    比较模块，用于比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
    识别模块，用于当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
    修正模块，用于针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
  9. 一种电子设备，其中，所述电子设备包括处理器，所述处理器用于执行存储器中存储的计算机可读指令以实现如下步骤：
    将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
    获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
    根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所 述黄斑区域的中心点坐标；
    比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
    当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
    针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
  10. 如权利要求9所述的电子设备，其中，所述针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正包括：
    针对所述左眼眼底图像，采用第一修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第一修正模型为：
    Figure PCTCN2020093338-appb-100005
    Figure PCTCN2020093338-appb-100006
    针对所述右眼眼底图像，采用第二修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第二修正模型为：
    Figure PCTCN2020093338-appb-100007
    Figure PCTCN2020093338-appb-100008
    其中，W表示所述待检测的眼底图像的宽，H表示所述待检测的眼底图像的高，(x oc，y oc)为计算得到的所述视盘区域的中心点坐标，h为所述视盘区域对应的所述第一检测框的高，(x fovea，y fovea)为修正得到的黄斑区域的中心点坐标。
  11. 如权利要求10所述的电子设备，其中，所述处理器还用于执行存储器中存储的计算机可读指令以实现如下步骤：
    当所述第二置信度小于所述预设第一置信度阈值且大于所述预设第二置信度阈值时，采用第三修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第三修正模型为：
    x fc＝0.5*x dc+0.5*x fovea，
    y fc＝0.5*y dc+0.5*y fovea，
    其中，(x fc，y fc)是最终修正得到的黄斑区域的中心点坐标，(x dc，y dc)是计算得到的所述黄斑区域的中心点坐标，(x fovea，y fovea)是根据视盘区域中心点坐标修正得到的黄斑区域中心点坐标。
  12. 如权利要求9所述的电子设备，其中，所述眼底图像检测模型的训练过程包括：
    获取多张眼底图像；
    对每张眼底图像中的一种或多种特征区域进行标记，所述特征区域为黄斑区域、视盘区域；
    基于标记了一种或多种特征区域的眼底图像及对应特征区域的类别构建样本数据集；
    将所述样本数据集随机分为第一数量的训练集和第二数量的测试集；
    将所述训练集输入至Mask RCNN网络中进行训练，得到眼底图像检测模型；
    将所述测试集输入至所述眼底图像检测模型中进行测试，得到测试通过率；
    判断所述测试通过率是否大于预设通过率阈值；
    当所述测试通过率大于或者等于所述预设通过率阈值时，结束眼底图像检测模型的训练； 否则，当所述测试通过率小于所述预设通过率阈值时，增加所述训练集的数量并基于增加数量后的训练集训练Mask RCNN网络直至所述测试通过率大于或者等于所述预设通过率阈值。
  13. 如权利要求12所述的电子设备，其中，在所述获取多张眼底图像之后，所述处理器还用于执行存储器中存储的计算机可读指令以实现如下步骤：
    对所述眼底图像进行预定角度的翻转处理；
    对翻转处理后的眼底图像标记一种或多种所述特征区域；
    所述基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建样本数据集包括：基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建第一样本数据集，基于标记了一种或多种所述特征区域的翻转处理后的眼底图像及对应特征区域的类别构建第二样本数据集，将所述第一样本数据集和所述第二样本数据集作为所述样本数据集。
  14. 如权利要求12所述的电子设备，其中，所述Mask RCNN网络的网络结构包括：
    多层主干网络，每层主干网络采用MobileNet V2网络；
    多层特征金字塔网络层，上一层特征金字塔网络层的输入为下一层特征金字塔网络层的输出及与所述上一层特征金字塔网络层位于同一层的主干网络的输出之和；
    注意力层。
  15. 一个或多个存储有计算机可读指令的可读存储介质，所述计算机可读存储介质存储有计算机可读指令，其中，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如下步骤：
    将待检测的眼底图像输入至预先训练好的眼底图像检测模型中；
    获取所述眼底图像检测模型的检测结果，其中，所述检测结果包括：眼底图像中的视盘区域及对应的第一检测框和第一置信度、黄斑区域及对应的第二检测框和第二置信度；
    根据所述第一检测框计算所述视盘区域的中心点坐标和根据所述第二检测框计算所述黄斑区域的中心点坐标；
    比较所述第二置信度与预设第一置信度阈值和预设第二置信度阈值，其中，所述预设第一置信度阈值大于所述预设第二置信度阈值；
    当所述第二置信度小于所述预设第二置信度阈值时，采用预先训练的左右眼识别模型识别所述待检测的眼底图像为左眼眼底图像还是右眼眼底图像；
    针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正。
  16. 如权利要求15所述的可读存储介质，其中，所述针对所述左眼眼底图像和所述右眼眼底图像采用不同的修正模型对所述黄斑区域的中心点进行修正包括：
    针对所述左眼眼底图像，采用第一修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第一修正模型为：
    Figure PCTCN2020093338-appb-100009
    Figure PCTCN2020093338-appb-100010
    针对所述右眼眼底图像，采用第二修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第二修正模型为：
    Figure PCTCN2020093338-appb-100011
    Figure PCTCN2020093338-appb-100012
    其中，W表示所述待检测的眼底图像的宽，H表示所述待检测的眼底图像的高，(x oc，y oc)为计算得到的所述视盘区域的中心点坐标，h为所述视盘区域对应的所述第一检测框的高，(x fovea，y fovea)为修正得到的黄斑区域的中心点坐标。
  17. 如权利要求16所述的可读存储介质，其中，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器还执行如下步骤：
    当所述第二置信度小于所述预设第一置信度阈值且大于所述预设第二置信度阈值时，采用第三修正模型和所述视盘区域的中心点坐标对所述黄斑区域的中心点进行修正，所述第三修正模型为：
    x fc＝0.5*x dc+0.5*x fovea，
    y fc＝0.5*y dc+0.5*y fovea，
    其中，(x fc，y fc)是最终修正得到的黄斑区域的中心点坐标，(x dc，y dc)是计算得到的所述黄斑区域的中心点坐标，(x fovea，y fovea)是根据视盘区域中心点坐标修正得到的黄斑区域中心点坐标。
  18. 如权利要求15所述的可读存储介质，其中，所述眼底图像检测模型的训练过程包括：
    获取多张眼底图像；
    对每张眼底图像中的一种或多种特征区域进行标记，所述特征区域为黄斑区域、视盘区域；
    基于标记了一种或多种特征区域的眼底图像及对应特征区域的类别构建样本数据集；
    将所述样本数据集随机分为第一数量的训练集和第二数量的测试集；
    将所述训练集输入至Mask RCNN网络中进行训练，得到眼底图像检测模型；
    将所述测试集输入至所述眼底图像检测模型中进行测试，得到测试通过率；
    判断所述测试通过率是否大于预设通过率阈值；
    当所述测试通过率大于或者等于所述预设通过率阈值时，结束眼底图像检测模型的训练；否则，当所述测试通过率小于所述预设通过率阈值时，增加所述训练集的数量并基于增加数量后的训练集训练Mask RCNN网络直至所述测试通过率大于或者等于所述预设通过率阈值。
  19. 如权利要求18所述的可读存储介质，其中，在所述获取多张眼底图像之后，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器还执行如下步骤：
    对所述眼底图像进行预定角度的翻转处理；
    对翻转处理后的眼底图像标记一种或多种所述特征区域；
    所述基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建样本数据集包括：基于标记了一种或多种所述特征区域的眼底图像及对应特征区域的类别构建第一样本数据集，基于标记了一种或多种所述特征区域的翻转处理后的眼底图像及对应特征区域的类别构建第二样本数据集，将所述第一样本数据集和所述第二样本数据集作为所述样本数据集。
  20. 如权利要求18所述的可读存储介质，其中，所述Mask RCNN网络的网络结构包括：多层主干网络，每层主干网络采用MobileNet V2网络；
    多层特征金字塔网络层，上一层特征金字塔网络层的输入为下一层特征金字塔网络层的输出及与所述上一层特征金字塔网络层位于同一层的主干网络的输出之和；
    注意力层。
PCT/CN2020/093338 2019-10-11 2020-05-29 眼底图像黄斑中心定位方法、装置、电子设备及存储介质 WO2021068523A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/620,733 US20220415087A1 (en) 2019-10-11 2020-05-29 Method, Device, Electronic Equipment and Storage Medium for Positioning Macular Center in Fundus Images
JP2021568982A JP7242906B2 (ja) 2019-10-11 2020-05-29 眼底画像から黄斑中心を位置特定する方法、装置、電子機器及び記憶媒体

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910964514.1A CN111046717B (zh) 2019-10-11 2019-10-11 眼底图像黄斑中心定位方法、装置、电子设备及存储介质
CN201910964514.1 2019-10-11

Publications (1)

Publication Number Publication Date
WO2021068523A1 true WO2021068523A1 (zh) 2021-04-15

Family

ID=70231797

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/093338 WO2021068523A1 (zh) 2019-10-11 2020-05-29 眼底图像黄斑中心定位方法、装置、电子设备及存储介质

Country Status (4)

Country Link
US (1) US20220415087A1 (zh)
JP (1) JP7242906B2 (zh)
CN (1) CN111046717B (zh)
WO (1) WO2021068523A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808164A (zh) * 2021-09-08 2021-12-17 西安电子科技大学 红外视频多目标跟踪方法
CN114998353A (zh) * 2022-08-05 2022-09-02 汕头大学·香港中文大学联合汕头国际眼科中心 一种自动检测玻璃体混浊斑飘动范围的系统
CN116524581A (zh) * 2023-07-05 2023-08-01 南昌虚拟现实研究院股份有限公司 一种人眼图像光斑分类方法、系统、设备及存储介质

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111046717B (zh) * 2019-10-11 2024-01-30 平安科技（深圳）有限公司 眼底图像黄斑中心定位方法、装置、电子设备及存储介质
CN112006649A (zh) * 2020-08-25 2020-12-01 张寅升 一种基于神经网络和自适应形态学约束的黄斑检测方法
CN112150463A (zh) * 2020-10-23 2020-12-29 北京百度网讯科技有限公司 用于确定黄斑中心凹位置的方法及装置
CN112017187B (zh) * 2020-11-02 2021-02-05 平安科技（深圳）有限公司 眼底图像黄斑中心定位方法及装置、服务器、存储介质
CN112785582B (zh) * 2021-01-29 2024-03-22 北京百度网讯科技有限公司 热力图生成模型的训练方法、装置、电子设备和存储介质
CN113297913B (zh) * 2021-04-26 2023-05-26 云南电网有限责任公司信息中心 一种配网现场作业人员穿衣规范的识别方法
CN112991343B (zh) * 2021-04-30 2021-08-13 北京至真互联网技术有限公司 眼底图像黄斑区域的识别检测方法和装置及设备
CN113723406B (zh) * 2021-09-03 2023-07-18 乐普（北京）医疗器械股份有限公司 一种对冠脉造影图像进行支架定位的处理方法和装置
CN113744254A (zh) * 2021-09-08 2021-12-03 中山大学中山眼科中心 一种眼底图像分析方法、系统、存储介质和计算机设备
CN113902743A (zh) * 2021-12-08 2022-01-07 武汉爱眼帮科技有限公司 一种基于云端计算的糖尿病视网膜病变的识别方法及装置
CN115049734A (zh) * 2022-08-12 2022-09-13 摩尔线程智能科技(北京)有限责任公司 在图像中定位目标物的方法和装置
CN115132357B (zh) * 2022-08-30 2022-12-23 深圳大学总医院 基于医学影像图预测目标疾病指标状态的装置
CN116309391B (zh) * 2023-02-20 2023-09-05 依未科技(北京)有限公司 图像处理方法及装置、电子设备及存储介质
CN116823828B (zh) * 2023-08-29 2023-12-08 武汉楚精灵医疗科技有限公司 黄斑变性程度参数确定方法、装置、设备及存储介质
CN117437231B (zh) * 2023-12-21 2024-04-26 依未科技(北京)有限公司 近视眼底结构改变的定位方法及装置、图像处理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150238074A1 (en) * 2014-02-24 2015-08-27 Image Technologies Corporation Autofluorescence imaging of macular pigment: image quality criteria and corrections
CN109377474A (zh) * 2018-09-17 2019-02-22 苏州大学 一种基于改进Faster R-CNN的黄斑定位方法
CN109662686A (zh) * 2019-02-01 2019-04-23 北京致远慧图科技有限公司 一种眼底黄斑定位方法、装置、系统及存储介质
CN109934823A (zh) * 2019-03-25 2019-06-25 天津工业大学 一种基于深度学习的dr眼底图像黄斑水肿分级方法
CN111046717A (zh) * 2019-10-11 2020-04-21 平安科技（深圳）有限公司 眼底图像黄斑中心定位方法、装置、电子设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105243669B (zh) * 2015-10-15 2018-05-04 四川和生视界医药技术开发有限公司 眼底图像自动识别分区方法
US10405739B2 (en) * 2015-10-23 2019-09-10 International Business Machines Corporation Automatically detecting eye type in retinal fundus images
CN107292877B (zh) * 2017-07-05 2020-07-03 北京至真互联网技术有限公司 一种基于眼底图像特征的左右眼识别方法
CN108717696B (zh) * 2018-05-16 2022-04-22 上海鹰瞳医疗科技有限公司 黄斑影像检测方法和设备
CN109635669B (zh) * 2018-11-19 2021-06-29 北京致远慧图科技有限公司 图像分类方法、装置及分类模型的训练方法、装置
CN109886955A (zh) * 2019-03-05 2019-06-14 百度在线网络技术（北京）有限公司 用于处理眼底图像的方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150238074A1 (en) * 2014-02-24 2015-08-27 Image Technologies Corporation Autofluorescence imaging of macular pigment: image quality criteria and corrections
CN109377474A (zh) * 2018-09-17 2019-02-22 苏州大学 一种基于改进Faster R-CNN的黄斑定位方法
CN109662686A (zh) * 2019-02-01 2019-04-23 北京致远慧图科技有限公司 一种眼底黄斑定位方法、装置、系统及存储介质
CN109934823A (zh) * 2019-03-25 2019-06-25 天津工业大学 一种基于深度学习的dr眼底图像黄斑水肿分级方法
CN111046717A (zh) * 2019-10-11 2020-04-21 平安科技（深圳）有限公司 眼底图像黄斑中心定位方法、装置、电子设备及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808164A (zh) * 2021-09-08 2021-12-17 西安电子科技大学 红外视频多目标跟踪方法
CN114998353A (zh) * 2022-08-05 2022-09-02 汕头大学·香港中文大学联合汕头国际眼科中心 一种自动检测玻璃体混浊斑飘动范围的系统
CN116524581A (zh) * 2023-07-05 2023-08-01 南昌虚拟现实研究院股份有限公司 一种人眼图像光斑分类方法、系统、设备及存储介质

Also Published As

Publication number Publication date
JP2022534209A (ja) 2022-07-28
US20220415087A1 (en) 2022-12-29
JP7242906B2 (ja) 2023-03-20
CN111046717B (zh) 2024-01-30
CN111046717A (zh) 2020-04-21

Similar Documents

Publication Publication Date Title
WO2021068523A1 (zh) 眼底图像黄斑中心定位方法、装置、电子设备及存储介质
US11961227B2 (en) Method and device for detecting and locating lesion in medical image, equipment and storage medium
Al-Bander et al. Multiscale sequential convolutional neural networks for simultaneous detection of fovea and optic disc
US20200260944A1 (en) Method and device for recognizing macular region, and computer-readable storage medium
EP3783533A1 (en) Artificial intelligence-based ophthalmic disease diagnostic modeling method, apparatus, and system
WO2021082691A1 (zh) 眼部oct图像病灶区域的分割方法、装置及终端设备
WO2021190656A1 (zh) 眼底图像黄斑中心定位方法及装置、服务器、存储介质
CN111242933B (zh) 视网膜图像动静脉的分类装置、设备及存储介质
Oloumi et al. Parabolic modeling of the major temporal arcade in retinal fundus images
CN110580466A (zh) 婴儿踢被子行为识别方法、装置、计算机设备及存储介质
Surendiran et al. Segmentation of optic disc and cup using modified recurrent neural network
CN111160431B (zh) 一种基于多维特征融合的圆锥角膜识别方法及装置
WO2021159643A1 (zh) 基于眼部oct图像的视杯和视盘定位点检测方法及装置
WO2022127043A1 (zh) 基于卷积神经网络的检测方法、装置、计算机设备及存储介质
CN106446805A (zh) 一种眼底照中视杯的分割方法及系统
CN116030042B (zh) 一种针对医生目诊的诊断装置、方法、设备及存储介质
CN110598652B (zh) 眼底数据预测方法和设备
CN112381012A (zh) 眼部图像中目标区域的识别方法、装置及电子设备
WO2023103609A1 (zh) 用于眼前节octa的眼动追踪方法、装置、设备和存储介质
CN116452571A (zh) 一种基于深度神经网络的图像识别方法
Zheng et al. Evaluation of effectiveness of eye massage therapy via classification of periocular images
Zhu et al. Calculation of ophthalmic diagnostic parameters on a single eye image based on deep neural network
CN109447995B (zh) 一种眼前节图像的分割方法及其相关装置
Qomariah et al. Exudate Segmentation for Diabetic Retinopathy Using Modified FCN-8 and Dice Loss.
CA3132685A1 (en) A neural-network-driven topology for optical coherence tomography segmentation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20874468

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021568982

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20874468

Country of ref document: EP

Kind code of ref document: A1