WO2021114817A1 - 基于神经网络的oct图像病灶检测方法、装置及介质 - Google Patents

基于神经网络的oct图像病灶检测方法、装置及介质 Download PDF

Info

Publication number
WO2021114817A1
WO2021114817A1 PCT/CN2020/117779 CN2020117779W WO2021114817A1 WO 2021114817 A1 WO2021114817 A1 WO 2021114817A1 CN 2020117779 W CN2020117779 W CN 2020117779W WO 2021114817 A1 WO2021114817 A1 WO 2021114817A1
Authority
WO
WIPO (PCT)
Prior art keywords
lesion
frame
score
candidate
oct image
Prior art date
Application number
PCT/CN2020/117779
Other languages
English (en)
French (fr)
Inventor
范栋轶
王立龙
王瑞
王关政
吕传峰
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021114817A1 publication Critical patent/WO2021114817A1/zh
Priority to US17/551,460 priority Critical patent/US20220108449A1/en

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20088—Trinocular vision calculations; trifocal tensor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • This application relates to artificial intelligence, and in particular to a neural network-based OCT image lesion detection method, device, electronic equipment, and computer-readable storage medium.
  • Optical Coherence Tomography is an imaging technique used for imaging examination of fundus diseases. It has the characteristics of high resolution, non-contact, and non-invasiveness. Due to the unique optical characteristics of the eyeball structure, OCT imaging technology is widely used in the field of ophthalmology, especially in the examination of fundus diseases.
  • the inventor realizes that at present, the recognition and detection of lesions in ophthalmic OCT is usually implemented by using a deep convolutional neural network model to extract features in the OCT image and training a classifier.
  • the neural network model used requires a large number of training samples and manual annotations. Under normal circumstances, one eye can scan 20-30 OCT images. Although more training samples can be collected at the image level, the cost of collecting a large number of samples at the eye level is relatively high, which makes model training difficult and affects the passage The accuracy of the detection results of the ophthalmic OCT image lesion recognition obtained by the model.
  • the Chinese invention patent with publication number CN110363226A discloses a method, device and medium for classifying and identifying ophthalmic diseases based on random forest.
  • the OCT image is input into the lesion recognition model to output the probability value corresponding to the identified lesion type, and then all the corresponding to the single eye
  • the probability value of the lesion type of the OCT image is input into the random forest classification model to obtain the probability value of the eye belonging to the disease category, so as to obtain the final disease category result, but some smaller lesions cannot be effectively identified, and there may be missed detection errors. Check and other issues.
  • This application provides a neural network-based OCT image lesion detection method, device, electronic equipment, and computer readable storage medium, the main purpose of which is to improve the accuracy of lesion detection and avoid the problem of missed detection and false detection.
  • the first aspect of the present application is to provide a method for detecting lesions in OCT images based on neural networks, including: acquiring OCT images; inputting the OCT images into a lesion detection network model, and using the lesion detection network model Output the focus frame position, focus frame category score, and focus frame positive score of the OCT image; obtain the focus detection result of the OCT image according to the focus frame location, focus frame category score, and focus frame positive score; wherein, the focus detection The network model includes: a feature extraction network layer for extracting image features of the OCT image; a candidate region extraction network layer for extracting all candidate frames in the OCT image; a feature pooling network layer for extracting all candidate frames The corresponding feature maps are pooled to a fixed size; the category detection branch is used to obtain the position and category score of each candidate frame; the lesion positive score regression branch is used to obtain the positive score of each candidate frame belonging to the lesion.
  • the second aspect of the present application is to provide a neural network-based OCT image lesion detection device, including: an image acquisition module for acquiring OCT images; a lesion detection module for inputting the OCT images
  • the focus detection network model which outputs the focus frame position, focus frame category score, and focus frame positive score of the OCT image through the focus detection network model; the result output module is used to output the focus frame location, focus frame category score, and focus frame according to the focus frame location, focus frame category score, and focus frame positive score.
  • the positive score obtains the lesion detection result of the OCT image;
  • the lesion detection network model includes: a feature extraction network layer for extracting image features of the OCT image; a candidate region extraction network layer for extracting the OCT All candidate frames in the image; feature pooling network layer, used to pool the feature maps corresponding to all candidate frames to a fixed size; category detection branch, used to obtain the position and category score of each candidate frame; regression of the positive score of the lesion Branch is used to get the positive score of each candidate frame belonging to the lesion.
  • a third aspect of the present application is to provide an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected with the at least one processor; wherein the memory stores There are instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the following steps: acquiring an OCT image; inputting the OCT image to lesion detection A network model, which outputs the focus frame position, focus frame category score, and focus frame positive score of the OCT image through the focus detection network model; and obtains the value of the OCT image according to the focus frame location, focus frame category score, and focus frame positive score Lesion detection result; wherein, the lesion detection network model includes: a feature extraction network layer for extracting image features of the OCT image; a candidate region extraction network layer for extracting all candidate frames in the OCT image; feature pool The network layer is used to pool the feature maps corresponding to all candidate frames to a fixed size; the category detection branch is used to obtain the
  • the fourth aspect of the present application is to provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the following steps are implemented: acquiring an OCT image; inputting the OCT image A focus detection network model, which outputs the focus frame position, focus frame category score, and focus frame positive score of the OCT image through the focus detection network model; the OCT is obtained according to the focus frame location, focus frame category score, and focus frame positive score of the OCT image
  • the lesion detection result of the image wherein the lesion detection network model includes: a feature extraction network layer for extracting image features of the OCT image; a candidate region extraction network layer for extracting all candidate frames in the OCT image;
  • the feature pooling network layer is used to pool the feature maps corresponding to all candidate frames to a fixed size;
  • the category detection branch is used to obtain the position and category score of each candidate frame;
  • the positive score regression branch of the lesion is used to obtain each One candidate box belongs to the positive score of the lesion.
  • the embodiment of the application combines artificial intelligence and neural network models to perform lesion detection on OCT images, and adds a regression branch of the lesion positive score to the lesion detection network model, and obtains the positive score of each candidate frame belonging to the lesion through the regression branch of the lesion positive score.
  • the positive severity of the lesion is taken into consideration when obtaining the OCT image lesion detection result.
  • the regression branch of the lesion-positive score regression only regresses the lesion-positive degree score, which can avoid inter-class competition, effectively identify smaller lesions, alleviate the problem of misdetection and missed detection, thereby improving the accuracy of lesion detection;
  • the specific quantified positive severity score of the lesion can be obtained through the regression branch of the lesion positive score, which can be used to judge the urgency of the lesion.
  • FIG. 1 is a schematic flowchart of a method for detecting lesions in an OCT image according to an embodiment of the application.
  • FIG. 2 is a schematic diagram of modules of an OCT image lesion detection device provided by an embodiment of the application.
  • FIG. 3 is a schematic diagram of the internal structure of an electronic device that implements the OCT image lesion detection method provided by an embodiment of the application.
  • the technical solution of the present application can be applied to the fields of artificial intelligence, blockchain and/or big data technology, for example, it can specifically involve neural network technology.
  • the data involved in this application such as scores, lesion detection results, etc., can be stored in a database, or can be stored in a blockchain, which is not limited in this application.
  • This application provides a method for detecting lesions.
  • FIG. 1 it is a schematic flowchart of an OCT image lesion detection method provided by an embodiment of this application.
  • the method can be executed by a device, and the device can be implemented by software and/or hardware.
  • the method for detecting a lesion in an OCT image based on a neural network includes: acquiring an OCT image; inputting the OCT image into a lesion detection network model, and outputting the lesion frame position and lesion of the OCT image through the lesion detection network model The frame category score and the lesion frame positive score; the lesion detection result of the OCT image is obtained according to the lesion frame position, the lesion frame category score, and the lesion frame positive score.
  • the lesion detection network model is a neural network model, including: a feature extraction network layer for extracting image features of the OCT image; a candidate region extraction network layer (using a region generation network RegionProposal Network, RPN), using To extract all the candidate frames in the OCT image; the feature pooling network layer is used to pool the feature maps corresponding to all candidate frames to a fixed size; the category detection branch is used to obtain the position and category score of each candidate frame ; The regression branch of the lesion positive score is used to obtain the positive score of each candidate frame belonging to the lesion to reflect the positive severity of the lesion, improve the accuracy of the lesion detection result, and avoid the missed detection error caused by outputting the lesion detection result based on the category score only Check the problem.
  • RPN region generation network RegionProposal Network
  • the feature extraction network layer includes a feature extraction layer and an attention mechanism layer.
  • the feature extraction layer is used to extract image features.
  • the ResNet101 network is used to extract 5 scales simultaneously in the form of a pyramid.
  • the attention mechanism layer includes a channel attention mechanism layer and a spatial attention mechanism layer.
  • the channel attention mechanism layer is used to weight the extracted image features and the feature channel weights to make the features
  • the features extracted by the extraction network layer pay more attention to the effective feature dimensions of the lesion;
  • the spatial attention mechanism layer is used to weight the extracted image features and feature space weights, so that the feature extraction network layer focuses on learning the foreground information of the lesion when extracting features Not background information.
  • the feature channel weight is obtained by the following method: the global maximum pooling processing and the global average pooling processing with the convolution kernel of a*a are respectively performed on the a*a*n-dimensional features, where n represents the number of channels; The maximum pooling processing result is added to the global average pooling processing result to obtain a characteristic channel weight of 1*1*n.
  • the weight of the feature space is obtained by the following method: a global maximum pooling process and a global average pooling process with a convolution kernel of 1*1 are respectively performed on the a*a*n-dimensional features to obtain two a*a*1
  • the first feature map; the two a*a*1 first feature maps obtained are spliced according to the channel dimensions to obtain the second feature map of a*a*2; the second feature map of a*a*2 is rolled
  • Product operation for example, a 7*7*1 convolution kernel can be used for convolution operation
  • This application adds an attention mechanism layer to the feature extraction network layer.
  • an attention mechanism layer to the feature extraction network layer.
  • the feature pooling network layer further performs the step of performing a cropping process on the feature map corresponding to the extracted candidate frame before performing the uniform pooling process on the feature map corresponding to the candidate frame. Specifically, after the features at the respective scales are clipped by the ROI Align to obtain the corresponding feature maps, the 7*7*256 convolution kernels are pooled to a fixed size.
  • the method further includes: preprocessing the OCT image.
  • the preprocessing includes: performing down-sampling processing on the acquired OCT image, and correcting the size of the image obtained through the down-sampling processing. For example, downsample the image from the original resolution of 1024*640 to 512*320, and add the upper and lower black borders to obtain a 512*512 OCT image as the input image of the model.
  • the lesion detection network model before inputting the OCT image into the lesion detection network model, it further includes a training step for the lesion detection network model.
  • the training step of the lesion detection network model includes: acquiring OCT images, and labeling the acquired OCT images to obtain sample images; for example, taking the lesion as the macula as an example, each OCT scans the macular area
  • the sample image is marked by two or more doctors on the location, type and severity of the lesion frame (including minor and severe levels), and an expert doctor will finally review and confirm each labeling result to obtain the final sample Image labeling to ensure the accuracy and consistency of labeling; this application can achieve better sensitivity and specificity only by labeling on a single 2D OCT image, greatly reducing the amount of labeling required and reducing the workload;
  • the marked sample image is preprocessed; the preprocessed sample image is used to train the lesion detection network model, in which the upper left corner coordinates, length, width, and category label of the lesion frame marked in the sample image are used as the input sample of the model
  • the given value is used for training, and the corresponding enhancement processing (including cropping, scaling, rotation, contrast change, etc.) is
  • this application uses regression fitting for a given score label (0.5 represents mild, 1 represents severe) instead of direct classification, because in actual clinical scenarios, doctors will use grades to judge the severity of different lesions Instead of directly giving a specific continuous score between (0-100), it is difficult to directly output labels with classification for lesions between different severity, so the given classification label value (0.5, 1) is used
  • the linear regression fitting positive score is more reasonable and effective. The closer the output score is to 1, the more serious it is, and the closer to 0, the less serious it is or even a false positive.
  • the step of obtaining the lesion detection result of the OCT image according to the position of the lesion frame, the lesion frame category score, and the lesion frame positive score includes: comparing the lesion frame category score of each candidate frame with the lesion frame positive score. Multiply the final score of the candidate frame; use the lesion frame position and the final score as the lesion detection result of the candidate frame; the final lesion detection result can be used to further assist in the diagnosis of the disease result in the retinal macular area of the fundus And urgency analysis.
  • the method further includes: merging candidate frames, for example, merging candidate frames with larger overlap by non-maximum suppression ; Screen the merged candidate frames, specifically, filter according to the category score of each candidate frame after the merging process, if the category score of the candidate frame is greater than or equal to the preset threshold, it will be regarded as the lesion frame, if If the category score of the candidate frame is less than the preset threshold, it will be removed and shall not be used as a lesion frame.
  • the preset threshold can be set manually, or the threshold can be determined according to the maximum Youden index (that is, the sum of the recall rate and the accuracy rate). The maximum Youden index can be tested when the focus detection network model is trained. The maximum Jordon index of the set is determined.
  • this application While fitting the position and category score of the lesion frame, this application adds a regression branch of the lesion positive score reflecting the positive severity of the lesion to quantify the severity of the lesion, thereby outputting the lesion severity score to obtain more accurate detection results. Avoid the problem of missed detection and misdetection caused by only using category scores to output lesion detection results.
  • FIG. 2 it is a functional block diagram of the lesion detection device of the present application.
  • the OCT image lesion detection apparatus 100 described in this application can be installed in an electronic device.
  • the OCT image lesion detection apparatus based on neural network may include an image acquisition module 101, a lesion detection module 102, and a result output module 103.
  • the module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.
  • each module/unit is as follows: the image acquisition module 101 is used to acquire OCT images; the lesion detection module 102 is used to input the OCT images into the lesion detection network model, and output through the lesion detection network model The focus frame position, focus frame category score, and focus frame positive score of the OCT image; the result output module 103 is configured to obtain the focus detection result of the OCT image according to the focus frame location, focus frame category score, and focus frame positive score.
  • the lesion detection network model includes: a feature extraction network layer for extracting image features of the OCT image; a candidate region extraction network layer for extracting all candidate frames in the OCT image; a feature pooling network layer, It is used to pool the feature maps corresponding to all candidate boxes to a fixed size; the category detection branch is used to obtain the position and category score of each candidate box; the lesion positive score regression branch is used to obtain each candidate box belonging to the lesion Positive score.
  • the feature extraction network layer includes a feature extraction layer and an attention mechanism layer.
  • the feature extraction layer is used to extract image features.
  • the ResNet101 network is used to extract 5 scales simultaneously in the form of a pyramid.
  • the attention mechanism layer includes a channel attention mechanism layer and a spatial attention mechanism layer, and the channel attention mechanism layer is used to weight the extracted image features and feature channel weights to enable feature extraction
  • the features extracted by the network layer pay more attention to the effective feature dimensions of the lesion;
  • the spatial attention mechanism layer is used to weight the extracted image features and feature space weights, so that when the feature extraction network layer extracts features, it focuses on learning the foreground information of the lesion. Not background information.
  • the feature channel weight is obtained by the following method: the global maximum pooling processing and the global average pooling processing with the convolution kernel of a*a are respectively performed on the a*a*n-dimensional features, where n represents the number of channels; The maximum pooling processing result is added to the global average pooling processing result to obtain a characteristic channel weight of 1*1*n.
  • the weight of the feature space is obtained by the following method: a global maximum pooling process and a global average pooling process with a convolution kernel of 1*1 are respectively performed on the a*a*n-dimensional features to obtain two a*a*1
  • the first feature map; the two a*a*1 first feature maps obtained are spliced according to the channel dimensions to obtain the second feature map of a*a*2; the second feature map of a*a*2 is rolled
  • Product operation for example, a 7*7*1 convolution kernel can be used for convolution operation
  • This application adds an attention mechanism layer to the feature extraction network layer.
  • an attention mechanism layer to the feature extraction network layer.
  • the feature pooling network layer further performs the step of performing a cropping process on the feature map corresponding to the extracted candidate frame before performing the uniform pooling process on the feature map corresponding to the candidate frame. Specifically, after the features at the respective scales are clipped by the ROI Align to obtain the corresponding feature maps, the 7*7*256 convolution kernels are pooled to a fixed size.
  • the OCT image lesion detection device further includes: a preprocessing module for preprocessing the OCT image before inputting the OCT image into the lesion detection network model after the OCT image is acquired.
  • the pre-processing module includes: a down-sampling unit, used to perform down-sampling processing on the acquired OCT image, and a correction unit, used to correct the image size obtained through the down-sampling processing.
  • the image is down-sampled from the original resolution of 1024*640 to 512*320, and the upper and lower black edges are added to obtain a 512*512 OCT image as the input image of the model.
  • the OCT image lesion detection device further includes a training module for training the lesion detection network model.
  • the training step of the lesion detection network model includes: acquiring OCT images, and labeling the acquired OCT images to obtain sample images; for example, taking the lesion as the macula as an example, each OCT scans the macular area
  • the sample image is marked by two or more doctors on the location, type and severity of the lesion frame (including minor and severe levels), and an expert doctor will finally review and confirm each labeling result to obtain the final sample Image annotation to ensure the accuracy and consistency of the annotation; preprocessing steps are performed on the annotated sample image; the sample image obtained by preprocessing is used to train the lesion detection network model, where the lesion frame marked in the sample image is The coordinates of the upper left corner and the length, width, and category labels are used as the given values of the model input samples for training.
  • the corresponding enhancement processing (including cropping, scaling, rotation, contrast change, etc.) is performed on the image and annotation to improve the training of the model Generalization ability; the positive score of the lesion box (0.5 represents mild, 1 represents severe) is used as the training label for the regression branch of the lesion positive score.
  • this application uses regression fitting for a given score label (0.5 represents mild, 1 represents severe) instead of direct classification, because in actual clinical scenarios, doctors will use grades to judge the severity of different lesions Instead of directly giving a specific continuous score between (0-100), it is difficult to directly output labels with classification for lesions between different severity levels, so the given classification label value (0.5, 1) is used
  • the linear regression fitting positive score is more reasonable and effective. The closer the output score is to 1, the more serious it is, and the closer to 0, the less serious it is or even a false positive.
  • the result output module obtains the lesion detection result in the following manner, including: multiplying the lesion frame category score of each candidate frame and the lesion frame positive score to obtain the final score of the candidate frame; The position of the lesion frame and the final score are used as the lesion detection result of the candidate frame; the final lesion detection result can be used to further assist in the diagnosis of the disease result and urgency analysis of the retinal macular area of the fundus.
  • the result output module further performs the following processing steps: merging the candidate frames, for example, by non-maximum suppression To merge the candidate frames with larger overlap; filter the merged candidate frames, specifically, filter according to the category score of each candidate frame after the merging process, if the category score of the candidate frame is greater than or equal to the preset threshold, Then it is regarded as the lesion frame, and if the category score of the candidate frame is less than the preset threshold, it will be removed and shall not be regarded as the lesion frame.
  • the preset threshold can be set manually, or the threshold can be determined according to the maximum Youden index (that is, the sum of the recall rate and the accuracy rate). The maximum Youden index can be tested when the focus detection network model is trained. The maximum Jordon index of the set is determined.
  • FIG. 3 it is a schematic structural diagram of an electronic device implementing the OCT image lesion detection method according to the present application.
  • the electronic device 1 may include a processor 10, a memory 11, and a bus, and may also include a computer program stored in the memory 11 and running on the processor 10, such as an OCT image lesion detection program 12.
  • the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, magnetic disk, CD etc.
  • the memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, for example, a mobile hard disk of the electronic device 1.
  • the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital) equipped on the electronic device 1. , SD) card, flash card (Flash Card), etc.
  • the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device.
  • the memory 11 can be used not only to store application software and various data installed in the electronic device 1, such as the code of an OCT image lesion detection program, etc., but also to temporarily store data that has been output or will be output.
  • the processor 10 may be composed of integrated circuits in some embodiments, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more Combinations of central processing unit (CPU), microprocessor, digital processing chip, graphics processor, and various control chips, etc.
  • the processor 10 is the control core (Control Unit) of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and runs or executes programs or modules (such as OCT) stored in the memory 11 Image lesion detection program, etc.), and call data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
  • Control Unit Control Unit
  • the bus may be a peripheral component interconnection standard (peripheral component interconnection standard) component interconnect, PCI for short) bus or extended industry standard structure (extended industry standard architecture, EISA for short) bus, etc.
  • the bus can be divided into address bus, data bus, control bus and so on.
  • the bus is configured to implement connection and communication between the memory 11 and at least one processor 10 and the like.
  • FIG. 3 only shows an electronic device with components. Those skilled in the art can understand that the structure shown in FIG. 2 does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure. Components, or a combination of certain components, or different component arrangements.
  • the electronic device 1 may also include a power source (such as a battery) for supplying power to various components.
  • the power source may be logically connected to the at least one processor 10 through a power management device, thereby controlling power
  • the device implements functions such as charge management, discharge management, and power consumption management.
  • the power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators.
  • the electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
  • the electronic device 1 may also include a network interface.
  • the network interface may include a wired interface and/or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.
  • the electronic device 1 may also include a user interface.
  • the user interface may be a display (Display) and an input unit (such as a keyboard (Keyboard)).
  • the user interface may also be a standard wired interface or a wireless interface.
  • the display may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc.
  • the display can also be appropriately called a display screen or a display unit, which is used to display the information processed in the electronic device 1 and to display a visualized user interface.
  • the OCT image lesion detection program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can be implemented: Obtain an OCT image; Input the OCT image into the lesion A detection network model, and output the focus frame position, focus frame category score, and focus frame positive score of the OCT image through the focus detection network model; obtain the OCT image according to the focus frame location, focus frame category score, and focus frame positive score The results of the lesion detection.
  • the integrated module/unit of the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) .
  • the embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored.
  • a computer program is stored on which a computer program is stored.
  • the computer program is executed by a processor, some or all of the steps of the method in the above-mentioned embodiment are implemented, or when the computer program is executed by the processor.
  • the function of each module/unit of the device in the above-mentioned embodiment is realized, which will not be repeated here.
  • the medium involved in this application such as a computer-readable storage medium, may be non-volatile or volatile.
  • modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

一种基于神经网络的OCT图像病灶检测方法、装置及介质，涉及人工智能，其中，方法包括：获取OCT图像（S1）；将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分（S2）；根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果（S3）；其中，所述病灶检测网络模型包括类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分，以反映病灶阳性严重程度。该方法可以避免发生类间竞争，有效识别较小病灶，缓解误检漏检的问题，从而提高病灶检测的准确率。

Description

基于神经网络的OCT图像病灶检测方法、装置及介质
本申请要求于2020年5月28日提交中国专利局、申请号为202010468697.0，发明名称为“基于神经网络的OCT图像病灶检测方法、装置及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能，尤其涉及一种基于神经网络的OCT图像病灶检测方法、装置、电子设备及计算机可读存储介质。
 背景技术
光相干断层扫描(Optical Coherence tomography，OCT)是一种用于眼底疾病影像检查的成像技术，具有高分辨率、非接触、非创伤性的特点。由于眼球结构具有独特的光学特性，所以OCT成像技术在眼科领域尤其是眼底疾病检查中得到广泛的应用。
发明人意识到，目前，对眼科OCT的病灶识别检测通常采用深度卷积神经网络模型提取OCT图像中的特征并训练分类器来实现，所利用的神经网络模型需要大量的训练样本和人工标注。一般情况下，一只眼睛可以扫描得到20-30张OCT图像，虽然在图像级别能收集到较多的训练样本，但在眼睛级别收集大量样本的成本比较大，使得模型训练困难，从而影响通过模型得到的对眼科OCT图像病灶识别检测结果的准确性。
公开号为CN110363226A的中国发明专利公开了一种基于随机森林的眼科病种分类识别方法、装置及介质，将OCT图像输入病灶识别模型输出对应识别的病灶类型的概率值，再将单眼对应的所有OCT图像的病灶类型的概率值输入随机森林分类模型得到该眼属于病种类别的概率值，从而得到最终的病种类别结果，但对于一些较小病灶无法有效进行识别，有可能出现漏检误检等问题。
 技术问题
本申请提供一种基于神经网络的OCT图像病灶检测方法、装置、电子设备及计算机可读存储介质，其主要目的在于提高病灶检测的准确率，避免产生漏检误检的问题。
 技术解决方案
为了实现上述目的，本申请的第一个方面是提供一种基于神经网络的OCT图像病灶检测方法，包括：获取OCT图像；将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
为了实现上述目的，本申请的第二个方面是提供一种基于神经网络的OCT图像病灶检测装置，包括：图像获取模块，用于获取OCT图像；病灶检测模块，用于将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；结果输出模块，用于根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
为了实现上述目的，本申请的第三个方面是提供一种电子设备，所述电子设备包括：至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行以下步骤：获取OCT图像；将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
为了实现上述目的，本申请的第四个方面是提供一种计算机可读存储介质，存储有计算机程序，所述计算机程序被处理器执行时实现以下步骤：获取OCT图像；将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
 有益效果
本申请实施例结合人工智能和神经网络模型对OCT图像进行病灶检测，并在病灶检测网络模型中增加了病灶阳性评分回归分支，通过病灶阳性评分回归分支得到每个候选框属于病灶的阳性得分，以反映病灶阳性严重程度，从而在获取OCT图像病灶检测结果时，将病灶阳性严重程度考虑在内。一方面，病灶阳性评分回归分支仅对病变阳性程度评分进行回归，可以避免发生类间竞争，有效识别较小病灶，缓解误检漏检的问题，从而提高病灶检测的准确率；另一方面，通过病灶阳性评分回归分支可以得到具体量化的病灶阳性严重程度得分，从而用于病灶的急迫性判断。
 附图说明
图1为本申请一实施例提供的OCT图像病灶检测方法的流程示意图。
图2为本申请一实施例提供的OCT图像病灶检测装置的模块示意图。
图3为本申请一实施例提供的实现OCT图像病灶检测方法的电子设备的内部结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
 本发明的实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请的技术方案可应用于人工智能、区块链和/或大数据技术领域，如可具体涉及神经网络技术。可选的，本申请涉及的数据如评分、病灶检测结果等可存储于数据库中，或者可以存储于区块链中，本申请不做限定。
本申请提供一种病灶检测方法。参照图1所示，为本申请一实施例提供的OCT图像病灶检测方法的流程示意图。该方法可以由一个装置执行，该装置可以由软件和/或硬件实现。
在本实施例中，基于神经网络的OCT图像病灶检测方法包括：获取OCT图像；将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果。
其中，所述病灶检测网络模型是一种神经网络模型，包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层（采用区域生成网络RegionProposal Network，RPN），用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分，以反映病灶阳性严重程度，提高病灶检测结果的准确率，避免仅根据类别得分进行输出病灶检测结果而产生漏检误检问题。
在一个实施例中，所述特征提取网络层包括特征提取层和注意力机制层，其中，所述特征提取层用于提取图像特征，例如，采用ResNet101网络，以金字塔的形式同时提取5个尺度下的高维特征图；所述注意力机制层包括通道注意力机制层和空间注意力机制层，所述通道注意力机制层用于对提取的图像特征与特征通道权重进行加权处理，使特征提取网络层提取的特征更关注于病灶有效特征维度；所述空间注意力机制层用于对提取的图像特征与特征空间权重进行加权处理，使特征提取网络层提取特征时，重点学习病灶前景信息而不是背景信息。
所述特征通道权重通过下述方式获取：对a*a*n维特征分别进行卷积核为a*a的全局最大池化处理和全局平均池化处理，其中，n表示通道数；将全局最大池化处理结果与全局平均池化处理结果相加，得到1*1*n的特征通道权重。
所述特征空间权重通过下述方式获取：对a*a*n维特征分别进行卷积核为1*1的全局最大池化处理和全局平均池化处理，得到两个a*a*1的第一特征图；将得到的两个a*a*1的第一特征图按照通道维度进行拼接得到a*a*2的第二特征图；对a*a*2的第二特征图进行卷积运算（例如，可使用7*7*1的卷积核进行卷积运算），得到a*a*1的特征空间权重。
对于利用ResNet101网络提取的5个尺度下的特征图，例如，128*128*256维特征图、64*64*256维特征图、32*32*256维特征图、16*16*256维特征图、8*8*256维特征图，不同尺度的特征图，计算得到的特征空间权重不同。
本申请在特征提取网络层中增加了注意力机制层，通过在特征提取阶段加入注意力机制，可以有效抑制背景信息带来的干扰，可用于提取更为有效鲁棒的特征用于病灶检测识别，提高病灶检测的准确率。
在一个实施例中，所述特征池化网络层在对候选框对应的特征图进行均池化处理之前，还进行对提取的候选框对应的特征图进行裁剪处理的步骤。具体地，在各自尺度下特征经过ROI Align 裁剪得到对应的特征图之后，再经过7*7*256的卷积核均池化至固定大小。
在一个实施例中，在获取OCT图像之后，将所述OCT图像输入病灶检测网络模型之前，还包括：对OCT图像进行预处理。具体地，所述预处理包括：对获取的OCT图像进行降采样处理，并对经过降采样处理得到的图像尺寸进行修正。例如，对图像从原分辨率1024*640降采样至512*320大小，并加入上下黑边得到512*512的OCT图像，作为模型的输入图像。
在一个实施例中，将所述OCT图像输入病灶检测网络模型之前，还包括对所述病灶检测网络模型的训练步骤。
进一步地，对所述病灶检测网络模型的训练步骤包括：采集OCT图像，并对采集的OCT图像进行标注，得到样本图像；例如，以病灶为黄斑为例进行说明，每张OCT扫描黄斑区域的样本图像由两位或两位以上的医生来标注病灶框的位置、类别和严重程度（包括轻微、严重两级），并最终经过一位专家医生对各个标注结果进行复核确认，得到最终的样本图像标注，以保证标注的准确性和一致性；本申请仅在2D单张OCT图像上进行标注即可达到较好的灵敏度和特异度，对所需的标注量大大减少，减轻工作量；对标注后的样本图像进行预处理步骤；利用预处理得到的样本图像对病灶检测网络模型进行训练，其中，将样本图像中标注的病灶框的左上角坐标以及长宽和类别标签作为模型输入样本的给定值用于训练，同时对图像和标注做对应的增强处理(包括裁剪，缩放，旋转，对比度变化等)，用于提高模型训练的泛化能力；将病灶框的阳性评分(0.5代表轻微，1代表严重) 作为病灶阳性评分回归分支的训练标签。
本申请在病灶阳性评分回归分支，采用对给定评分标签(0.5代表轻微，1代表严重)进行回归拟合而非直接分类，因为在实际临床场景下医生对不同病灶均会采取分级评判严重程度而非直接给出一个具体的介于（0-100）的连续评分，而介于不同严重程度之间的病灶则难以用分类直接输出标签，因此给定分级标签值(0.5,1)进行采用线性回归拟合阳性评分更为合理有效，输出得分越接近1则越严重，越接近0则代表越轻微甚至为假阳。
在一个实施例中，根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果的步骤包括：将每个候选框的病灶框类别得分和病灶框阳性得分进行相乘得到所述候选框的最终得分；将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果；最终的病灶检测结果可以用于进一步辅助诊断眼底视网膜黄斑区域的病种结果和急迫性分析。
进一步地，将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果之前，还包括：对候选框进行合并，例如，通过非极大值抑制来合并重合较大的候选框；对合并得到的候选框进行筛选，具体地，根据经过合并处理后的每个候选框的类别得分进行筛选，若候选框的类别得分大于或等于预设阈值，则将其作为病灶框，若候选框的类别得分小于预设阈值，则将其剔除，不得作为病灶框。其中，所述预设阈值可以是人为设置，也可以根据最大化约登指数(即召回率和准确率之和)确定阈值，最大化约登指数可由在对于病灶检测网络模型进行训练时，测试集的最大化约登指数确定。
本申请在拟合病灶框的位置和类别得分的同时，加入了反应病灶阳性严重程度的病灶阳性评分回归分支，以量化病灶的严重程度，从而输出病灶严重评分用于得到更精准的检测结果，避免仅采用类别得分输出病灶检测结果而产生的漏检误检问题。
相比一般的检测网络仅对每个目标框输出一个类别得分的方式，一方面，当某个病灶的外观特征同时与两种或两种以上的病灶类别形似时，则会导致原有检测网络的类别得分较低而被阈值过滤导致漏检，而本申请通过病灶阳性评分回归分支仅对病变阳性程度评分进行回归，可以避免发生类间竞争，缓解误检漏检的问题。另一方面，当有非常小的组织有轻微异常但无临床意义时，病灶检测网络模型可能会检出并得到较高的类别得分，这种情况也可通过病灶阳性评分回归分支得到具体量化的病灶阳性严重程度得分，从而用于急迫性判断。
如图2所示，是本申请病灶检测装置的功能模块图。
本申请所述OCT图像病灶检测装置100可以安装于电子设备中。根据实现的功能，所述基于神经网络的OCT图像病灶检测装置可以包括图像获取模块101、病灶检测模块102、结果输出模块103。本发所述模块也可以称之为单元，是指一种能够被电子设备处理器所执行，并且能够完成固定功能的一系列计算机程序段，其存储在电子设备的存储器中。
在本实施例中，关于各模块/单元的功能如下：图像获取模块101用于获取OCT图像；病灶检测模块102用于将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；结果输出模块103用于根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果。
其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
在一个实施例中，所述特征提取网络层包括特征提取层和注意力机制层，其中，所述特征提取层用于提取图像特征，例如，采用ResNet101网络，以金字塔的形式同时提取5个尺度下的高维特征；所述注意力机制层包括通道注意力机制层和空间注意力机制层，所述通道注意力机制层用于对提取的图像特征与特征通道权重进行加权处理，使特征提取网络层提取的特征更关注于病灶有效特征维度；所述空间注意力机制层用于对提取的图像特征与特征空间权重进行加权处理，使特征提取网络层提取特征时，重点学习病灶前景信息而不是背景信息。
所述特征通道权重通过下述方式获取：对a*a*n维特征分别进行卷积核为a*a的全局最大池化处理和全局平均池化处理，其中，n表示通道数；将全局最大池化处理结果与全局平均池化处理结果相加，得到1*1*n的特征通道权重。
所述特征空间权重通过下述方式获取：对a*a*n维特征分别进行卷积核为1*1的全局最大池化处理和全局平均池化处理，得到两个a*a*1的第一特征图；将得到的两个a*a*1的第一特征图按照通道维度进行拼接得到a*a*2的第二特征图；对a*a*2的第二特征图进行卷积运算（例如，可使用7*7*1的卷积核进行卷积运算），得到a*a*1的特征空间权重。
本申请在特征提取网络层中增加了注意力机制层，通过在特征提取阶段加入注意力机制，可以有效抑制背景信息带来的干扰，可用于提取更为有效鲁棒的特征用于病灶检测识别，提高病灶检测的准确率。
在一个实施例中，所述特征池化网络层在对候选框对应的特征图进行均池化处理之前，还进行对提取的候选框对应的特征图进行裁剪处理的步骤。具体地，在各自尺度下特征经过ROI Align 裁剪得到对应的特征图之后，再经过7*7*256的卷积核均池化至固定大小。
在一个实施例中，OCT图像病灶检测装置还包括：预处理模块，用于在获取OCT图像之后，将所述OCT图像输入病灶检测网络模型之前，对OCT图像进行预处理。具体地，所述预处理模块包括：降采样单元，用于对获取的OCT图像进行降采样处理，修正单元，用于对经过降采样处理得到的图像尺寸进行修正。例如，对图像从原分辨率1024*640降采样至512*320大小，并加入上下黑边得到512*512的OCT图像，作为模型的输入图像。
在一个实施例中，OCT图像病灶检测装置还包括：训练模块，对所述病灶检测网络模型进行训练。
进一步地，对所述病灶检测网络模型的训练步骤包括：采集OCT图像，并对采集的OCT图像进行标注，得到样本图像；例如，以病灶为黄斑为例进行说明，每张OCT扫描黄斑区域的样本图像由两位或两位以上的医生来标注病灶框的位置、类别和严重程度（包括轻微、严重两级），并最终经过一位专家医生对各个标注结果进行复核确认，得到最终的样本图像标注，以保证标注的准确性和一致性；对标注后的样本图像进行预处理步骤；利用预处理得到的样本图像对病灶检测网络模型进行训练，其中，将样本图像中标注的病灶框的左上角坐标以及长宽和类别标签作为模型输入样本的给定值用于训练，同时对图像和标注做对应的增强处理(包括裁剪，缩放，旋转，对比度变化等)，用于提高模型训练的泛化能力；将病灶框的阳性评分(0.5代表轻微，1代表严重) 作为病灶阳性评分回归分支的训练标签。
本申请在病灶阳性评分回归分支，采用对给定评分标签(0.5代表轻微，1代表严重)进行回归拟合而非直接分类，因为在实际临床场景下医生对不同病灶均会采取分级评判严重程度而非直接给出一个具体的介于（0-100）的连续评分，而介于不同严重程度之间的病灶则难以用分类直接输出标签，因此给定分级标签值(0.5,1)进行采用线性回归拟合阳性评分更为合理有效，输出得分越接近1则越严重，越接近0则代表越轻微甚至为假阳。
在一个实施例中，结果输出模块通过下述方式得到病灶检测结果，包括：将每个候选框的病灶框类别得分和病灶框阳性得分进行相乘得到所述候选框的最终得分；将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果；最终的病灶检测结果可以用于进一步辅助诊断眼底视网膜黄斑区域的病种结果和急迫性分析。
进一步地，将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果之前，所述结果输出模块还进行以下处理步骤：对候选框进行合并，例如，通过非极大值抑制来合并重合较大的候选框；对合并得到的候选框进行筛选，具体地，根据经过合并处理后的每个候选框的类别得分进行筛选，若候选框的类别得分大于或等于预设阈值，则将其作为病灶框，若候选框的类别得分小于预设阈值，则将其剔除，不得作为病灶框。其中，所述预设阈值可以是人为设置，也可以根据最大化约登指数(即召回率和准确率之和)确定阈值，最大化约登指数可由在对于病灶检测网络模型进行训练时，测试集的最大化约登指数确定。
如图3所示，是本申请实现OCT图像病灶检测方法的电子设备的结构示意图。
所述电子设备1可以包括处理器10、存储器11和总线，还可以包括存储在所述存储器11中并可在所述处理器10上运行的计算机程序，如OCT图像病灶检测程序12。
其中，所述存储器11至少包括一种类型的可读存储介质，所述可读存储介质包括闪存、移动硬盘、多媒体卡、卡型存储器（例如：SD或DX存储器等）、磁性存储器、磁盘、光盘等。所述存储器11在一些实施例中可以是电子设备1的内部存储单元，例如该电子设备1的移动硬盘。所述存储器11在另一些实施例中也可以是电子设备1的外部存储设备，例如电子设备1上配备的插接式移动硬盘、智能存储卡（Smart Media Card， SMC）、安全数字（Secure Digital， SD）卡、闪存卡（Flash Card）等。进一步地，所述存储器11还可以既包括电子设备1的内部存储单元也包括外部存储设备。所述存储器11不仅可以用于存储安装于电子设备1的应用软件及各类数据，例如OCT图像病灶检测程序的代码等，还可以用于暂时地存储已经输出或者将要输出的数据。
所述处理器10在一些实施例中可以由集成电路组成，例如可以由单个封装的集成电路所组成，也可以是由多个相同功能或不同功能封装的集成电路所组成，包括一个或者多个中央处理器（Central Processing unit，CPU）、微处理器、数字处理芯片、图形处理器及各种控制芯片的组合等。所述处理器10是所述电子设备的控制核心（Control Unit），利用各种接口和线路连接整个电子设备的各个部件，通过运行或执行存储在所述存储器11内的程序或者模块（例如OCT图像病灶检测程序等），以及调用存储在所述存储器11内的数据，以执行电子设备1的各种功能和处理数据。
所述总线可以是外设部件互连标准（peripheral component interconnect，简称PCI）总线或扩展工业标准结构（extended industry standard architecture，简称EISA）总线等。该总线可以分为地址总线、数据总线、控制总线等。所述总线被设置为实现所述存储器11以及至少一个处理器10等之间的连接通信。
图3仅示出了具有部件的电子设备，本领域技术人员可以理解的是，图2示出的结构并不构成对所述电子设备1的限定，可以包括比图示更少或者更多的部件，或者组合某些部件，或者不同的部件布置。
例如，尽管未示出，所述电子设备1还可以包括给各个部件供电的电源（比如电池），优选地，电源可以通过电源管理装置与所述至少一个处理器10逻辑相连，从而通过电源管理装置实现充电管理、放电管理、以及功耗管理等功能。电源还可以包括一个或一个以上的直流或交流电源、再充电装置、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。所述电子设备1还可以包括多种传感器、蓝牙模块、Wi-Fi模块等，在此不再赘述。
进一步地，所述电子设备1还可以包括网络接口，可选地，所述网络接口可以包括有线接口和/或无线接口（如WI-FI接口、蓝牙接口等），通常用于在该电子设备1与其他电子设备之间建立通信连接。
可选地，该电子设备1还可以包括用户接口，用户接口可以是显示器（Display）、输入单元（比如键盘（Keyboard）），可选地，用户接口还可以是标准的有线接口、无线接口。可选地，在一些实施例中，显示器可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED（Organic Light-Emitting Diode，有机发光二极管）触摸器等。其中，显示器也可以适当的称为显示屏或显示单元，用于显示在电子设备1中处理的信息以及用于显示可视化的用户界面。
应该了解，所述实施例仅为说明之用，在专利申请范围上并不受此结构的限制。
所述电子设备1中的所述存储器11存储的OCT图像病灶检测程序12是多个指令的组合，在所述处理器10中运行时，可以实现：获取OCT图像；将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果。
具体地，所述处理器10对上述指令的具体实现方法可参考图1对应实施例中相关步骤的描述，在此不赘述。
进一步地，所述电子设备1集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器（ROM，Read-Only Memory）。
本申请实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述实施例中方法的部分或全部步骤，或者，计算机程序被处理器执行时实现上述实施例中装置的各模块/单元的功能，这里不再赘述。可选的，本申请涉及的介质如计算机可读存储介质可以是非易失性的，也可以是易失性的。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能模块的形式实现。
对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。
因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附关联图标记视为限制所涉及的权利要求。
此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。系统权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第二等词语用来表示名称，而并不表示任何特定的顺序。
最后应说明的是，以上实施例仅用以说明本申请的技术方案而非限制，尽管参照较佳实施例对本申请进行了详细说明，本领域的普通技术人员应当理解，可以对本申请的技术方案进行修改或等同替换，而不脱离本申请技术方案的精神和范围。

Claims (20)

  1. 一种基于神经网络的OCT图像病灶检测方法，其中，所述方法包括：
    获取OCT图像；
    将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；
    根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；
    其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
  2. 如权利要求1所述的基于神经网络的OCT图像病灶检测方法，其中，所述特征提取网络层包括特征提取层和注意力机制层，
    其中，所述特征提取层用于提取图像特征；
    所述注意力机制层包括通道注意力机制层和空间注意力机制层，所述通道注意力机制层用于对提取的图像特征与特征通道权重进行加权处理；所述空间注意力机制层用于对提取的图像特征与特征空间权重进行加权处理。
  3. 如权利要求2所述的基于神经网络的OCT图像病灶检测方法，其中，所述特征通道权重通过下述方式获取：
    对a*a*n维特征分别进行卷积核为a*a的全局最大池化处理和全局平均池化处理；
    将全局最大池化处理结果与全局平均池化处理结果相加，得到1*1*n的特征通道权重。
  4. 如权利要求2所述的基于神经网络的OCT图像病灶检测方法，其中，所述特征空间权重通过下述方式获取：
    对a*a*n维特征分别进行卷积核为1*1的全局最大池化处理和全局平均池化处理，得到两个a*a*1的第一特征图；
    将得到的两个a*a*1的第一特征图按照通道维度进行拼接得到a*a*2的第二特征图；
    对a*a*2的第二特征图进行卷积运算，得到a*a*1的特征空间权重。
  5. 如权利要求1所述的基于神经网络的OCT图像病灶检测方法，其中，根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果的步骤包括：
    将每个候选框的病灶框类别得分和病灶框阳性得分进行相乘得到所述候选框的最终得分；
    将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果。
  6. 如权利要求5所述的基于神经网络的OCT图像病灶检测方法，其中，将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果之前，还包括：
    对候选框进行合并；
    对合并得到的候选框进行筛选，若候选框的类别得分大于或等于预设阈值，则将其作为病灶框，若候选框的类别得分小于预设阈值，则将其剔除。
  7. 一种基于神经网络的OCT图像病灶检测装置，其中，包括：
    图像获取模块，用于获取OCT图像；
    病灶检测模块，用于将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；
    结果输出模块，用于根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；
    其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
  8. 如权利要求7所述的基于神经网络的OCT图像病灶检测装置，其中，所述特征提取网络层包括特征提取层和注意力机制层，
    其中，所述特征提取层用于提取图像特征；
    所述注意力机制层包括通道注意力机制层和空间注意力机制层，所述通道注意力机制层用于对提取的图像特征与特征通道权重进行加权处理；所述空间注意力机制层用于对提取的图像特征与特征空间权重进行加权处理。
  9. 一种电子设备，其中，所述电子设备包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行以下步骤：
    获取OCT图像；
    将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；
    根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；
    其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
  10. 如权利要求9所述的电子设备，其中，所述特征提取网络层包括特征提取层和注意力机制层，
    其中，所述特征提取层用于提取图像特征；
    所述注意力机制层包括通道注意力机制层和空间注意力机制层，所述通道注意力机制层用于对提取的图像特征与特征通道权重进行加权处理；所述空间注意力机制层用于对提取的图像特征与特征空间权重进行加权处理。
  11. 如权利要求10所述的电子设备，其中，所述特征通道权重通过下述方式获取：
    对a*a*n维特征分别进行卷积核为a*a的全局最大池化处理和全局平均池化处理；
    将全局最大池化处理结果与全局平均池化处理结果相加，得到1*1*n的特征通道权重。
  12. 如权利要求10所述的电子设备，其中，所述特征空间权重通过下述方式获取：
    对a*a*n维特征分别进行卷积核为1*1的全局最大池化处理和全局平均池化处理，得到两个a*a*1的第一特征图；
    将得到的两个a*a*1的第一特征图按照通道维度进行拼接得到a*a*2的第二特征图；
    对a*a*2的第二特征图进行卷积运算，得到a*a*1的特征空间权重。
  13. 如权利要求9所述的电子设备，其中，根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果时，具体执行：
    将每个候选框的病灶框类别得分和病灶框阳性得分进行相乘得到所述候选框的最终得分；
    将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果。
  14. 如权利要求13所述的电子设备，其中，将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果之前，所述指令被所述至少一个处理器执行还使所述至少一个处理器执行以下步骤：
    对候选框进行合并；
    对合并得到的候选框进行筛选，若候选框的类别得分大于或等于预设阈值，则将其作为病灶框，若候选框的类别得分小于预设阈值，则将其剔除。
  15. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现以下步骤：
    获取OCT图像；
    将所述OCT图像输入病灶检测网络模型，通过所述病灶检测网络模型输出所述OCT图像的病灶框位置、病灶框类别得分以及病灶框阳性评分；
    根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果；
    其中，所述病灶检测网络模型包括：特征提取网络层，用于提取所述OCT图像的图像特征；候选区域提取网络层，用于提取所述OCT图像中所有候选框；特征池化网络层，用于将所有候选框对应的特征图均池化至固定大小；类别检测分支，用于得到每个候选框的位置和类别得分；病灶阳性评分回归分支，用于得到每个候选框属于病灶的阳性得分。
  16. 如权利要求15所述的计算机可读存储介质，其中，所述特征提取网络层包括特征提取层和注意力机制层，
    其中，所述特征提取层用于提取图像特征；
    所述注意力机制层包括通道注意力机制层和空间注意力机制层，所述通道注意力机制层用于对提取的图像特征与特征通道权重进行加权处理；所述空间注意力机制层用于对提取的图像特征与特征空间权重进行加权处理。
  17. 如权利要求16所述的计算机可读存储介质，其中，所述特征通道权重通过下述方式获取：
    对a*a*n维特征分别进行卷积核为a*a的全局最大池化处理和全局平均池化处理；
    将全局最大池化处理结果与全局平均池化处理结果相加，得到1*1*n的特征通道权重。
  18. 如权利要求16所述的计算机可读存储介质，其中，所述特征空间权重通过下述方式获取：
    对a*a*n维特征分别进行卷积核为1*1的全局最大池化处理和全局平均池化处理，得到两个a*a*1的第一特征图；
    将得到的两个a*a*1的第一特征图按照通道维度进行拼接得到a*a*2的第二特征图；
    对a*a*2的第二特征图进行卷积运算，得到a*a*1的特征空间权重。
  19. 如权利要求15所述的计算机可读存储介质，其中，根据病灶框位置、病灶框类别得分和病灶框阳性评分得到所述OCT图像的病灶检测结果时，具体实现：
    将每个候选框的病灶框类别得分和病灶框阳性得分进行相乘得到所述候选框的最终得分；
    将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果。
  20. 如权利要求19所述的计算机可读存储介质，其中，将所述病灶框位置和所述最终得分作为所述候选框的病灶检测结果之前，所述计算机程序被处理器执行时还用于实现以下步骤：
    对候选框进行合并；
    对合并得到的候选框进行筛选，若候选框的类别得分大于或等于预设阈值，则将其作为病灶框，若候选框的类别得分小于预设阈值，则将其剔除。
PCT/CN2020/117779 2020-05-28 2020-09-25 基于神经网络的oct图像病灶检测方法、装置及介质 WO2021114817A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/551,460 US20220108449A1 (en) 2020-05-28 2021-12-15 Method and device for neural network-based optical coherence tomography (oct) image lesion detection, and medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010468697.0A CN111667468A (zh) 2020-05-28 2020-05-28 基于神经网络的oct图像病灶检测方法、装置及介质
CN202010468697.0 2020-05-28

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/551,460 Continuation US20220108449A1 (en) 2020-05-28 2021-12-15 Method and device for neural network-based optical coherence tomography (oct) image lesion detection, and medium

Publications (1)

Publication Number Publication Date
WO2021114817A1 true WO2021114817A1 (zh) 2021-06-17

Family

ID=72385152

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/117779 WO2021114817A1 (zh) 2020-05-28 2020-09-25 基于神经网络的oct图像病灶检测方法、装置及介质

Country Status (3)

Country Link
US (1) US20220108449A1 (zh)
CN (1) CN111667468A (zh)
WO (1) WO2021114817A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115960605A (zh) * 2022-12-09 2023-04-14 西南政法大学 一种多色荧光碳点及其应用

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667468A (zh) * 2020-05-28 2020-09-15 平安科技（深圳）有限公司 基于神经网络的oct图像病灶检测方法、装置及介质
CN112435256A (zh) * 2020-12-11 2021-03-02 北京大恒普信医疗技术有限公司 基于图像的cnv活动性病灶检测方法及装置、电子设备
CN112541900B (zh) * 2020-12-15 2024-01-02 平安科技（深圳）有限公司 基于卷积神经网络的检测方法、装置、计算机设备及存储介质
CN112668573B (zh) * 2020-12-25 2022-05-10 平安科技（深圳）有限公司 目标检测定位置信度确定方法、装置、电子设备及存储介质
CN112884729B (zh) * 2021-02-04 2023-08-01 北京邮电大学 基于双模态深度学习的眼底疾病辅助诊断方法和装置
CN113362329B (zh) * 2021-08-11 2021-11-19 北京航空航天大学杭州创新研究院 病灶检测模型的训练方法及识别图像中的病灶的方法
CN117710760B (zh) * 2024-02-06 2024-05-17 广东海洋大学 残差的注意神经网络用于胸部x线病灶检测的方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10139507B2 (en) * 2015-04-24 2018-11-27 Exxonmobil Upstream Research Company Seismic stratigraphic surface classification
CN109948607A (zh) * 2019-02-21 2019-06-28 电子科技大学 基于深度学习反卷积网络的候选边框生成和目标检测方法
CN110110600A (zh) * 2019-04-04 2019-08-09 平安科技（深圳）有限公司 眼部oct图像病灶识别方法、装置及存储介质
CN110163844A (zh) * 2019-04-17 2019-08-23 平安科技（深圳）有限公司 眼底病灶检测方法、装置、计算机设备及存储介质
CN110555856A (zh) * 2019-09-09 2019-12-10 成都智能迭迦科技合伙企业(有限合伙) 一种基于深度神经网络的黄斑水肿病变区域分割方法
CN111667468A (zh) * 2020-05-28 2020-09-15 平安科技（深圳）有限公司 基于神经网络的oct图像病灶检测方法、装置及介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108447046B (zh) * 2018-02-05 2019-07-26 龙马智芯（珠海横琴）科技有限公司 病灶的检测方法和装置、计算机可读存储介质
CN110084210B (zh) * 2019-04-30 2022-03-29 电子科技大学 基于注意力金字塔网络的sar图像多尺度舰船检测方法
CN110175993A (zh) * 2019-05-27 2019-08-27 西安交通大学医学院第一附属医院 一种基于FPN的Faster R-CNN肺结核征象检测系统及方法
CN110599451B (zh) * 2019-08-05 2023-01-20 平安科技（深圳）有限公司 医学图像病灶检测定位方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10139507B2 (en) * 2015-04-24 2018-11-27 Exxonmobil Upstream Research Company Seismic stratigraphic surface classification
CN109948607A (zh) * 2019-02-21 2019-06-28 电子科技大学 基于深度学习反卷积网络的候选边框生成和目标检测方法
CN110110600A (zh) * 2019-04-04 2019-08-09 平安科技（深圳）有限公司 眼部oct图像病灶识别方法、装置及存储介质
CN110163844A (zh) * 2019-04-17 2019-08-23 平安科技（深圳）有限公司 眼底病灶检测方法、装置、计算机设备及存储介质
CN110555856A (zh) * 2019-09-09 2019-12-10 成都智能迭迦科技合伙企业(有限合伙) 一种基于深度神经网络的黄斑水肿病变区域分割方法
CN111667468A (zh) * 2020-05-28 2020-09-15 平安科技（深圳）有限公司 基于神经网络的oct图像病灶检测方法、装置及介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115960605A (zh) * 2022-12-09 2023-04-14 西南政法大学 一种多色荧光碳点及其应用
CN115960605B (zh) * 2022-12-09 2023-10-24 西南政法大学 一种多色荧光碳点及其应用

Also Published As

Publication number Publication date
CN111667468A (zh) 2020-09-15
US20220108449A1 (en) 2022-04-07

Similar Documents

Publication Publication Date Title
WO2021114817A1 (zh) 基于神经网络的oct图像病灶检测方法、装置及介质
US11276497B2 (en) Diagnosis assistance system and control method thereof
US10722181B2 (en) Systems, methods, and computer-readable media for using descriptors to identify when a subject is likely to have a dysmorphic feature
US20220198214A1 (en) Image recognition method and device based on deep convolutional neural network
CN113222064A (zh) 一种图像目标对象实时检测方法、系统、终端及存储介质
WO2020108436A1 (zh) 舌面图像分割装置、方法及计算机存储介质
CN113516639B (zh) 基于全景x光片的口腔异常检测模型的训练方法及装置
CN113361482A (zh) 核性白内障识别方法、装置、电子设备以及存储介质
Tian et al. Learning discriminative representations for fine-grained diabetic retinopathy grading
CN111862034A (zh) 图像检测方法、装置、电子设备及介质
Li et al. A deep-learning-enabled monitoring system for ocular redness assessment
Wang et al. PCRTAM-Net: A novel pre-activated convolution residual and triple attention mechanism network for retinal vessel segmentation
CN115578370B (zh) 一种基于脑影像的代谢区域异常检测方法及装置
Xu et al. DUF-Net: A Retinal Vessel Segmentation Method Integrating Global and Local Features with Skeleton Fitting Assistance.
Dong et al. Deep Learning for Detecting Corneal Pathologies From Anterior Segment Optical Coherence Tomography
Jabbar et al. Liver fibrosis processing, multiclassification, and diagnosis based on hybrid machine learning approaches
Sheraz et al. Two stage-network: Automatic localization of Optic Disc (OD) and classification of glaucoma in fundus images using deep learning techniques
CN116665881A (zh) 基于深度学习的婴儿血管瘤皮损图片分析方法及装置
CN117011161A (zh) 图像缺陷智能检测方法、装置、设备及存储介质
CN113724236A (zh) 基于注意力机制的oct图像检测方法及相关设备
CN115035103A (zh) 眼红图像数据的处理方法及眼红等级分析装置
CN115985472A (zh) 基于神经网络的眼底图像标注方法及系统
Han et al. ARTIFICIAL INTELLIGENCE POWERED DYNAMIC SKIN ANALYSIS APPLICATION

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20899919

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20899919

Country of ref document: EP

Kind code of ref document: A1