WO2021139446A1 - Appareil et procédé de prédiction d'effet curatif anti-facteur de croissance de l'endothélium vasculaire (vegf) - Google Patents

Appareil et procédé de prédiction d'effet curatif anti-facteur de croissance de l'endothélium vasculaire (vegf) Download PDF

Info

Publication number
WO2021139446A1
WO2021139446A1 PCT/CN2020/132473 CN2020132473W WO2021139446A1 WO 2021139446 A1 WO2021139446 A1 WO 2021139446A1 CN 2020132473 W CN2020132473 W CN 2020132473W WO 2021139446 A1 WO2021139446 A1 WO 2021139446A1
Authority
WO
WIPO (PCT)
Prior art keywords
oct images
image
vegf
curative effect
features corresponding
Prior art date
Application number
PCT/CN2020/132473
Other languages
English (en)
Chinese (zh)
Inventor
张潇月
张成奋
吕彬
吕传峰
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021139446A1 publication Critical patent/WO2021139446A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/102—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for optical coherence tomography [OCT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/0016—Operational features thereof
    • A61B3/0025—Operational features thereof characterised by electronic signal processing, e.g. eye models
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • A61B3/1225—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes using coherent radiation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Definitions

  • This application relates to the field of medical science and technology, and in particular to an anti-vascular endothelial growth factor VEGF curative effect prediction device and method.
  • ATD age-related macular degeneration
  • VEGF vascular endothelial growth factor
  • intraocular injection is an effective treatment for wet AMD, but anti-VEGF injection therapy is expensive and has strict indications, and the efficacy of different patients varies. Due to the lack of effective anti-VEGF curative effect predictions, doctors often adopt uniform injection methods for patients, resulting in anti-VEGF injections for some patients who are not applicable. Therefore, effective anti-VEGF curative effect prediction is an urgent need of doctors.
  • Optical coherence tomography is a commonly used device for diagnosing ophthalmic diseases. It uses light reflection technology similar to ultrasound imaging to provide image reference for the detection and treatment of ophthalmic diseases.
  • OCT optical coherence tomography
  • the inventor realized that in the currently commonly used anti-VEGF curative effect prediction method, the lesion area (such as effusion, high reflection point, etc.) is segmented through a segmentation network, and then anti-VEGF curative effect prediction is performed after segmentation.
  • the segmentation network based on deep learning requires a large number of doctors' annotations during the training process, and the accuracy of the annotations and the accuracy of the segmentation network segmentation will affect the curative effect prediction results.
  • a large amount of retinal tissue change information that may improve the accuracy of anti-VEGF curative effect prediction is lost, resulting in low accuracy of anti-VEGF curative effect prediction.
  • the application provides an anti-vascular endothelial growth factor VEGF curative effect prediction device and method, which is beneficial to improve the accuracy of anti-VEGF curative effect prediction.
  • the first aspect of this application provides an anti-vascular endothelial growth factor VEGF curative effect prediction device, which includes: an acquisition module for acquiring multiple OCT images of the optical coherence tomography acquired for the macula; a feature extraction module for evaluating the Perform feature extraction on the multiple OCT images to obtain multiple image features corresponding to the multiple OCT images, wherein the image features corresponding to each OCT image in the multiple OCT images include Multi-scale features; a spatial information fusion module for spatial information fusion of multiple image features corresponding to the multiple OCT images to obtain the features corresponding to the multiple OCT images; a determination module, used to determine the features corresponding to the multiple OCT images The characteristics corresponding to the OCT image determine the predictive results of anti-VEGF curative effect.
  • the second aspect of the present application provides a method for predicting the therapeutic effect of anti-vascular endothelial growth factor VEGF, which includes: acquiring multiple OCT images of optical coherence tomography collected for the macular area; performing feature extraction on the multiple OCT images to obtain the results
  • the multiple image features corresponding to the multiple OCT images wherein the image feature corresponding to each OCT image in the multiple OCT images includes the multi-scale feature in each OCT image; and the multiple OCT images Spatial information fusion is performed on the corresponding multiple image features to obtain the features corresponding to the multiple OCT images; according to the features corresponding to the multiple OCT images, the anti-VEGF curative effect prediction result is determined.
  • the third aspect of the present application provides an electronic device that includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and are
  • the configuration is executed by the processor to realize the following method: acquiring multiple OCT images of optical coherence tomography collected for the macula; performing feature extraction on the multiple OCT images to obtain multiple corresponding to the multiple OCT images Image features, wherein the image feature corresponding to each OCT image in the multiple OCT images includes the multi-scale feature in each OCT image; the multiple image features corresponding to the multiple OCT images are spatially Information fusion is used to obtain the features corresponding to the multiple OCT images; according to the features corresponding to the multiple OCT images, the anti-VEGF curative effect prediction result is determined.
  • the fourth aspect of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following method: Obtain a plurality of optical coherence images collected for the macular region A tomography OCT image; feature extraction is performed on the multiple OCT images to obtain multiple image features corresponding to the multiple OCT images, wherein the image features corresponding to each OCT image in the multiple OCT images include The multi-scale features in each OCT image; spatial information fusion is performed on the multiple image features corresponding to the multiple OCT images to obtain the features corresponding to the multiple OCT images; according to the corresponding features of the multiple OCT images Features to determine the predictive results of anti-VEGF curative effect.
  • the application After acquiring multiple OCT images, the application does not need to be segmented, which reduces the cost of labeling in the segmentation network, and at the same time avoids the accuracy error of anti-VEGF curative effect prediction caused by inaccurate segmentation.
  • the image features corresponding to each extracted OCT image include multi-scale features, which increases the richness and comprehensiveness of the extracted features, thereby improving the accuracy of anti-VEGF curative effect prediction.
  • the spatial information fusion of multiple image features effectively utilizes the spatial characteristics of multiple OCT images, enriches the spatial information of the features, and also improves the accuracy of anti-VEGF curative effect prediction.
  • Fig. 1 is a schematic diagram of a network structure for predicting the curative effect of anti-vascular endothelial growth factor VEGF according to an embodiment of the application.
  • Figure 2 is a schematic flow chart of a method for predicting curative effect of anti-vascular endothelial growth factor VEGF provided by an embodiment of the application.
  • FIG. 3 is a schematic diagram of a feature extraction network provided by an embodiment of this application.
  • Fig. 4 is a schematic flow chart of another method for predicting the efficacy of anti-vascular endothelial growth factor VEGF provided by an embodiment of the application.
  • Fig. 5 is a schematic diagram of an anti-vascular endothelial growth factor VEGF curative effect prediction device provided by an embodiment of the application.
  • FIG. 6 is a schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the application.
  • the device and method for predicting the curative effect of anti-vascular endothelial growth factor VEGF provided in the embodiments of the present application are beneficial to improve the accuracy of anti-VEGF curative effect prediction.
  • At least one refers to one or more, and “multiple” refers to two or more.
  • And/or describes the association relationship of the associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone, where A, B can be singular or plural.
  • the character “/” generally indicates that the associated objects before and after are in an “or” relationship.
  • "The following at least one item (a)” or similar expressions refers to any combination of these items, including any combination of a single item (a) or a plurality of items (a).
  • first, second, third, and fourth in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than describing a specific order, Timing, priority, or importance.
  • first information and the second information are only for distinguishing different information, but do not indicate the difference in content, priority, sending order, or importance of the two types of information.
  • the terms “including” and “having” and any variations of them are intended to cover non-exclusive inclusions.
  • a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.
  • the technical solution of this application can be applied to the fields of artificial intelligence, smart city, digital medical and/or blockchain technology to realize smart medical treatment.
  • the data involved in this application can be stored in a database, or can be stored in a blockchain, such as distributed storage through a blockchain, which is not limited in this application.
  • AMD AMD is a major blinding eye disease, which is an aging change in the structure of the macular area.
  • the main manifestation is that the ability of retinal pigment epithelial cells to phagocytose and digest the outer segmental membrane of the optic cell decreases.
  • the residual disc membranes that have not been completely digested are retained in the basal cell protoplasm, discharged outside the cell, and deposited on Bruch membrane , The formation of drusen.
  • vascular endothelial growth factor vascular endothelial growth factor
  • VEGF vascular endothelial growth factor
  • VPF vascular permeability factor
  • Intraocular injection of anti-vascular endothelial growth factor VEGF is an effective treatment for wet AMD, but anti-VEGF injection therapy is expensive and has strict indications, and the effect is different for different patients.
  • Optical coherence tomography is a commonly used device for diagnosing ophthalmic diseases. It uses light reflection technology similar to ultrasound imaging to provide image reference for the detection and treatment of ophthalmic diseases.
  • FIG. 1 is a schematic diagram of a network structure for predicting the curative effect of anti-vascular endothelial growth factor VEGF according to an embodiment of the application.
  • the network structure in the embodiment of the present application includes a feature extraction network for feature extraction of OCT images, and a long and short-term memory artificial neural network for fusion of spatial information of multiple OCT images, and finally achieves resistance Prediction of the efficacy of VEGF.
  • the multiple OCT images may be twelve-line OCT images, that is, 12 OCT images.
  • the feature extraction network is a residual network structure, and the convolutional layer in the feature extraction network is used to extract Multi-scale features in each OCT image.
  • the anti-VEGF curative effect is predicted, and the anti-VEGF curative effect prediction result is obtained, where the anti-VEGF curative effect prediction result includes vision improvement or vision deterioration.
  • the anti-VEGF curative effect prediction results can provide effective references for doctors' treatment plans.
  • the anti-VEGF curative effect prediction probability is less than the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is vision deterioration, indicating that anti-VEGF injection is not recommended.
  • the anti-VEGF curative effect prediction result is determined to be improved vision, indicating that anti-VEGF injection is recommended.
  • the anti-VEGF curative effect prediction probability is equal to the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is visual acuity deterioration or visual acuity improvement. Specifically, visual acuity deterioration or visual acuity improvement can be determined according to requirements.
  • an anti-VEGF curative effect prediction method provided by an embodiment of the present application may include the following steps.
  • An OCT image obtained by scanning the patient's eyes with an optical coherence tomography scanner OCT, and extract the regions collected for the macular area in the multiple initial OCT images to obtain the multiple images collected for the macular area.
  • each of the multiple initial OCT images includes the patient Retinal tissue information.
  • extract the image parts scanned for the macular area in the multiple initial OCT images where the macular area is located in the center of the retina, and the macular area is the most concentrated part of the central visual cells of the retina of the human eye.
  • preprocessing may be performed on the multiple OCT images, which specifically includes: performing image correction on each OCT image, and performing contrast enhancement processing after correction.
  • the image correction includes image tilt correction and/or image brightness correction.
  • the multiple OCT images must first be preprocessed, including image correction processing and contrast enhancement processing, so as to correct too bright or too dark images, and The slanted image is corrected, and at the same time, the contrast of the image is improved, thereby improving the visual effect of the image.
  • the multiple OCT images may be twelve-line OCT images, that is, 12 OCT images.
  • the multiple OCT images are input to a feature extraction network to obtain multiple image features corresponding to the multiple OCT images.
  • FIG. 3 is a schematic diagram of a feature extraction network provided by an embodiment of this application.
  • the feature extraction network is a residual network structure.
  • the input is directly passed to the output as the initial result through the shortcut connection.
  • the convolutional layer in the feature extraction network is used to extract multi-scale features in each OCT image.
  • the multiple image features corresponding to the multiple OCT images are first extracted through the feature extraction network.
  • the multiple OCT images are first input into the feature extraction network, which is a residual network structure.
  • the residual network is a type of convolutional neural network, which is easy to optimize and can be increased by Depth to improve accuracy, the residual block inside the residual network uses jump connections, which alleviates the problem of gradient disappearance caused by increasing depth in the deep neural network.
  • the convolutional layer in the feature extraction network is used to extract the multi-scale features in each OCT image.
  • the features of different scales reflect different image features
  • the features of the shallower scale reflect the features of the shallower image, for example Edges, etc.
  • deeper-scale features reflect deeper-level image features such as object contours.
  • Feature extraction is performed on multiple OCT images through a feature extraction network that combines residual network structure and multi-scale feature extraction functions.
  • feature extraction of multiple OCT images through the feature extraction network of residual network structure can reduce computational cost.
  • And can also alleviate the problem of gradient disappearance caused by increasing depth in deep neural networks.
  • the multi-scale features in the OCT image can be effectively obtained, which increases the richness and comprehensiveness of the extracted features, thereby making The forecast result is more accurate.
  • spatial information fusion is performed on the multiple image features, thereby enriching the spatial information of the features.
  • spatial information fusion first obtain the time corresponding to each OCT image, and sort the image features corresponding to each OCT image in chronological order according to the time corresponding to each OCT image, so as to obtain the corresponding multiple OCT images Time series of multiple image features.
  • the time series of the multiple image features corresponding to the multiple OCT images are input into a long short-term memory artificial neural network (LSTM) to obtain the features corresponding to the multiple OCT images.
  • LSTM long short-term memory artificial neural network
  • the long and short-term memory artificial neural network is a time cyclic neural network, including forgetting gates, input gates and output gates.
  • the image features extracted from the spatial sequence are input into the long and short-term memory artificial neural network in the form of time series, thereby fusing multiple image features corresponding to multiple OCT images of the same patient to enrich the spatial information of the features , Making the prediction method of the network closer to the process of doctors reading the pictures, so that the prediction results are more accurate.
  • the anti-VEGF curative effect is classified into two categories to obtain the anti-VEGF curative effect prediction result, wherein the anti-VEGF curative effect prediction result includes improved vision or worsened vision.
  • the anti-VEGF curative effect prediction results can provide effective references for doctors' treatment plans.
  • the anti-VEGF curative effect prediction probability is less than the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is vision deterioration, indicating that anti-VEGF injection is not recommended.
  • the anti-VEGF curative effect prediction probability is greater than the preset probability threshold, the anti-VEGF curative effect prediction result is determined to be improved vision, indicating that anti-VEGF injection is recommended.
  • the anti-VEGF curative effect prediction probability is equal to the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is visual acuity deterioration or visual acuity improvement. Specifically, visual acuity deterioration or visual acuity improvement can be determined according to requirements.
  • the anti-vascular endothelial growth factor VEGF curative effect prediction method provided by the embodiments of this application, after obtaining multiple OCT images, segmentation is not required, which reduces the cost of labeling in the segmentation network, and avoids inaccurate segmentation, etc.
  • the accuracy error caused by anti-VEGF curative effect prediction the image features corresponding to each extracted OCT image include multi-scale features, which increases the richness and comprehensiveness of the extracted features, thereby improving the accuracy of anti-VEGF curative effect prediction.
  • the spatial information fusion of multiple image features effectively utilizes the spatial characteristics of multiple OCT images, enriches the spatial information of the features, and also improves the accuracy of anti-VEGF curative effect prediction.
  • the solution of this application can also be applied to the field of smart medical care.
  • the anti-VEGF curative effect prediction method provided in this application is used to determine the anti-VEGF curative effect prediction result. Since the anti-VEGF curative effect prediction method provided by the present application can determine the anti-VEGF curative effect prediction results more accurately, this can provide a more accurate basis for the doctor's treatment plan, thereby improving the doctor's treatment efficiency and accuracy.
  • FIG. 4 is a schematic flow chart of another anti-vascular endothelial growth factor VEGF curative effect prediction method provided by an embodiment of the application.
  • another anti-vascular endothelial growth factor VEGF curative effect prediction method provided by the embodiment of the present application may include the following steps.
  • An OCT image obtained by scanning the patient's eyes with an optical coherence tomography scanner OCT, and extract the regions collected for the macular area in the multiple initial OCT images to obtain the multiple images collected for the macular area.
  • each of the multiple initial OCT images includes the patient Retinal tissue information.
  • extract the image parts scanned for the macular area in the multiple initial OCT images where the macular area is located in the center of the retina, and the macular area is the most concentrated part of the central visual cells of the retina of the human eye.
  • the multiple OCT images may be twelve-line OCT images, that is, 12 OCT images.
  • Preprocessing the multiple OCT images includes: performing image correction on each OCT image, and performing contrast enhancement processing after correction.
  • the image correction includes image tilt correction and/or image brightness correction.
  • the multiple OCT images must first be preprocessed, including image correction processing and contrast enhancement processing, so as to correct too bright or too dark images, and The slanted image is corrected, and at the same time, the contrast of the image is improved, thereby improving the visual effect of the image.
  • V represents the R, G, and B channels of each OCT image.
  • V represents the R, G, and B channels of each OCT image.
  • the R, G, and B channel data of each OCT image subtract the minimum value of all the data of the channel respectively, and then divide by the maximum value of all the data of the channel minus the minimum value, and finally The operation of multiplying by 255 to restore it to the value range of [0, 255].
  • the R, G, and B three-channel data of each OCT image can be more evenly distributed from 0 to 255, which improves the contrast of the image, and achieves the purpose of improving the subjective visual effect of the image and enhancing the details of the image.
  • the feature extraction network has a residual network structure, and the convolutional layer in the feature extraction network is used to extract multi-scale features in each OCT image.
  • the multiple image features corresponding to the multiple OCT images are first extracted through the feature extraction network.
  • the multiple OCT images are first input into the feature extraction network, which is a residual network structure.
  • the residual network is a type of convolutional neural network, which is easy to optimize and can be increased by Depth to improve accuracy, the residual block inside the residual network uses jump connections, which alleviates the problem of gradient disappearance caused by increasing depth in the deep neural network.
  • the convolutional layer in the feature extraction network is used to extract the multi-scale features in each OCT image.
  • the features of different scales reflect different image features
  • the features of the shallower scale reflect the features of the shallower image, for example Edges, etc.
  • deeper-scale features reflect deeper-level image features such as object contours.
  • the multiple OCT images are twelve-line OCT images (that is, 12 OCT images).
  • the 12 OCT images are input into the feature extraction network, and 12 image features corresponding to the 12 OCT images are obtained.
  • Feature extraction is performed on multiple OCT images through a feature extraction network that combines residual network structure and multi-scale feature extraction functions.
  • feature extraction of multiple OCT images through the feature extraction network of residual network structure can reduce computational cost.
  • And can also alleviate the problem of gradient disappearance caused by increasing depth in deep neural networks.
  • the multi-scale features in the OCT image can be effectively obtained, which increases the richness and comprehensiveness of the extracted features, thereby making The forecast result is more accurate.
  • the long and short-term memory artificial neural network is a time cyclic neural network, including forgetting gates, input gates and output gates.
  • the image features extracted from the spatial sequence are input into the long and short-term memory artificial neural network in the form of time series, thereby fusing multiple image features corresponding to multiple OCT images of the same patient to enrich the spatial information of the features , Making the prediction method of the network closer to the process of doctors reading the pictures, so that the prediction results are more accurate.
  • the anti-VEGF curative effect prediction probability is used to determine the anti-VEGF curative effect prediction result.
  • the anti-VEGF curative effect prediction results include vision improvement or vision deterioration, and the anti-VEGF curative effect prediction results can provide an effective reference for the doctor's treatment plan.
  • the anti-VEGF curative effect prediction probability is less than the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is vision deterioration.
  • the anti-VEGF curative effect prediction probability is not less than the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is vision improvement.
  • the feature extraction network that combines the residual network structure and the multi-scale feature extraction function is used to extract the features of multiple OCT images.
  • the residual network structure The feature extraction network performs feature extraction on multiple OCT images, which can reduce the computational cost, and can also alleviate the problem of gradient disappearance caused by increasing depth in the deep neural network.
  • the multi-scale features in the OCT image can be effectively obtained, which increases the richness and comprehensiveness of the extracted features, thereby making The forecast result is more accurate.
  • the image features extracted from the spatial sequence are input into the long and short-term memory artificial neural network in the form of time series, thereby fusing multiple image features corresponding to multiple OCT images of the same patient to enrich the spatial information of the features , Making the prediction method of the network closer to the process of doctors reading the pictures, so that the prediction results are more accurate.
  • an anti-vascular endothelial growth factor VEGF curative effect prediction device provided by an embodiment of the application.
  • an anti-vascular endothelial growth factor VEGF curative effect prediction device provided by an embodiment of the present application may include the following modules.
  • the acquiring module 501 is configured to acquire multiple OCT images of the optical coherence tomography acquired for the macula area.
  • the feature extraction module 502 is configured to perform feature extraction on the multiple OCT images to obtain multiple image features corresponding to the multiple OCT images, where each of the multiple OCT images corresponds to the image feature Including the multi-scale features in each OCT image.
  • the spatial information fusion module 503 is configured to perform spatial information fusion on the multiple image features corresponding to the multiple OCT images to obtain the features corresponding to the multiple OCT images.
  • the determining module 504 is configured to determine the anti-VEGF curative effect prediction result according to the characteristics corresponding to the multiple OCT images.
  • the acquiring module 501 is specifically configured to: acquire multiple initial OCT images obtained by scanning the eye with an optical coherence tomography scanner OCT; and extract the multiple initial OCT images for the macula. To obtain the multiple OCT images collected for the macular area.
  • the device further includes a processing module configured to: perform image correction processing on the multiple OCT images to obtain the corrected multiple OCT images, wherein The image correction processing includes image tilt correction and/or image brightness correction; performing contrast enhancement processing on the corrected multiple OCT images to obtain the multiple OCT images after contrast enhancement.
  • a processing module configured to: perform image correction processing on the multiple OCT images to obtain the corrected multiple OCT images, wherein The image correction processing includes image tilt correction and/or image brightness correction; performing contrast enhancement processing on the corrected multiple OCT images to obtain the multiple OCT images after contrast enhancement.
  • the feature extraction module 502 is specifically configured to: input the multiple OCT images into a feature extraction network to obtain multiple image features corresponding to the multiple OCT images, wherein the feature The extraction network is a residual network structure, and the convolutional layer in the feature extraction network is used to extract the multi-scale features in each OCT image.
  • the spatial information fusion module 503 is specifically configured to: obtain the time corresponding to each OCT image in the multiple OCT images; and determine the time corresponding to each OCT image according to the time corresponding to each OCT image.
  • the time sequence of the multiple image features corresponding to the multiple OCT images; the time sequence of the multiple image features corresponding to the multiple OCT images is input into a long and short-term memory artificial neural network to obtain the features corresponding to the multiple OCT images.
  • the determining module 504 is specifically configured to: perform a two-classification of the anti-VEGF curative effect according to the characteristics corresponding to the multiple OCT images to obtain the anti-VEGF curative effect prediction result, wherein the anti-VEGF curative effect
  • the predictive results of VEGF curative effect include improved or worsened vision.
  • the determining module 504 is specifically configured to: determine the anti-VEGF curative effect prediction probability according to the characteristics corresponding to the multiple OCT images; when the anti-VEGF curative effect prediction probability is less than a preset probability threshold , Determining that the anti-VEGF curative effect prediction result is vision deterioration; when the anti-VEGF curative effect prediction probability is greater than the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is vision improvement; when the anti-VEGF curative effect prediction probability When it is equal to the preset probability threshold, it is determined that the anti-VEGF curative effect prediction result is vision deterioration or vision improvement.
  • anti-vascular endothelial growth factor VEGF curative effect prediction device for the specific implementation of the anti-vascular endothelial growth factor VEGF curative effect prediction device in the embodiments of the present application, please refer to the foregoing embodiments of the anti-vascular endothelial growth factor VEGF curative effect prediction method, which will not be repeated here.
  • FIG. 6 is a schematic structural diagram of an electronic device in a hardware operating environment involved in an embodiment of the application.
  • the electronic device of the hardware operating environment involved in the embodiment of the present application may include: a processor 601, such as a CPU.
  • the memory 602 optionally, the memory may be a high-speed RAM memory, or a stable memory, such as a disk memory.
  • the communication interface 603 is used to implement connection and communication between the processor 601 and the memory 602.
  • FIG. 6 does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or a combination of certain components, or different component arrangements. .
  • the memory 602 may include an operating system, a network communication module, and an anti-vascular endothelial growth factor VEGF curative effect prediction program.
  • the operating system is a program that manages and controls the hardware and software resources of electronic equipment, supports the operation of anti-vascular endothelial growth factor VEGF curative effect prediction programs and other software or programs.
  • the network communication module is used to implement communication between various components in the memory 602 and communication with other hardware and software in the electronic device.
  • the processor 601 is used to execute the anti-vascular endothelial growth factor VEGF curative effect prediction program stored in the memory 602 to implement the following steps: Obtain multiple optical coherence tomography OCT images collected for the macular area Perform feature extraction on the multiple OCT images to obtain multiple image features corresponding to the multiple OCT images, wherein the image feature corresponding to each OCT image in the multiple OCT images includes the each OCT Multi-scale features in the image; spatial information fusion is performed on the multiple image features corresponding to the multiple OCT images to obtain the features corresponding to the multiple OCT images; the anti-VEGF is determined according to the features corresponding to the multiple OCT images Efficacy prediction results.
  • Another embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps: Obtain multiple optical coherence tomography images collected from the macula Instrument OCT image; perform feature extraction on the multiple OCT images to obtain multiple image features corresponding to the multiple OCT images, wherein the image feature corresponding to each OCT image in the multiple OCT images includes the Multi-scale features in each OCT image; spatial information fusion is performed on the multiple image features corresponding to the multiple OCT images to obtain the features corresponding to the multiple OCT images; according to the features corresponding to the multiple OCT images, Determine the predictive results of anti-VEGF efficacy.
  • the storage medium involved in this application such as a computer-readable storage medium, may be non-volatile or volatile.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Theoretical Computer Science (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biophysics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Radiology & Medical Imaging (AREA)
  • Molecular Biology (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Resources & Organizations (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Tourism & Hospitality (AREA)
  • Game Theory and Decision Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Evolutionary Biology (AREA)
  • Development Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Epidemiology (AREA)

Abstract

L'invention concerne un appareil et un procédé de prédiction d'effet curatif anti-facteur de croissance de l'endothélium vasculaire (VEGF). L'appareil comprend : un module d'acquisition (501) pour acquérir une pluralité d'images de tomographie par cohérence optique (OCT) collectées pour une région maculaire; un module d'extraction de caractéristiques (502) destiné à effectuer une extraction de caractéristiques au niveau de la pluralité d'images OCT pour obtenir une pluralité de caractéristiques d'image correspondant à la pluralité d'images OCT, la caractéristique d'image correspondant à chaque image OCT de la pluralité d'images OCT comprenant une caractéristique multiéchelle dans chaque image OCT; un module de fusion d'informations spatiales (503) destiné à effectuer une fusion d'informations spatiales au niveau de la pluralité de caractéristiques d'image correspondant à la pluralité d'images OCT, de façon à obtenir des caractéristiques correspondant à la pluralité d'images OCT; et un module de détermination (504) destiné à déterminer un résultat de prédiction d'effet curatif anti-VEGF en fonction des caractéristiques correspondant à la pluralité d'images OCT. L'appareil est avantageux pour améliorer la précision de la prédiction d'un effet curatif anti-VEGF.
PCT/CN2020/132473 2020-09-30 2020-11-27 Appareil et procédé de prédiction d'effet curatif anti-facteur de croissance de l'endothélium vasculaire (vegf) WO2021139446A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011061099.8 2020-09-30
CN202011061099.8A CN111920375B (zh) 2020-09-30 2020-09-30 一种抗血管内皮生长因子vegf疗效预测装置及方法

Publications (1)

Publication Number Publication Date
WO2021139446A1 true WO2021139446A1 (fr) 2021-07-15

Family

ID=73334769

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/132473 WO2021139446A1 (fr) 2020-09-30 2020-11-27 Appareil et procédé de prédiction d'effet curatif anti-facteur de croissance de l'endothélium vasculaire (vegf)

Country Status (2)

Country Link
CN (1) CN111920375B (fr)
WO (1) WO2021139446A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111920375B (zh) * 2020-09-30 2021-01-15 平安科技（深圳）有限公司 一种抗血管内皮生长因子vegf疗效预测装置及方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106455972A (zh) * 2014-04-28 2017-02-22 西北大学 功能性光学相干断层成像的装置、方法和系统
CN110309862A (zh) * 2019-06-11 2019-10-08 广东省人民医院(广东省医学科学院) 基于集成机器学习的dme预后信息预测系统及其应用方法
CN110472530A (zh) * 2019-07-29 2019-11-19 中山大学 基于小波变换和迁移学习的视网膜oct图像分类方法
US20200027209A1 (en) * 2018-07-18 2020-01-23 Case Western Reserve University Predicting response to anti-vascular endothelial growth factor therapy with computer-extracted morphology and spatial arrangement features of leakage patterns on baseline fluorescein angiography in diabetic macular edema
CN111611877A (zh) * 2020-04-30 2020-09-01 杭州电子科技大学 基于多时空信息融合的抗年龄干扰的人脸识别方法
CN111920375A (zh) * 2020-09-30 2020-11-13 平安科技（深圳）有限公司 一种抗血管内皮生长因子vegf疗效预测装置及方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2442715A4 (fr) * 2009-06-18 2018-01-03 Richard M. Awdeh Systeme et procede pour l'imagerie moleculaire in vivo et le theranostic
CN108229580B (zh) * 2018-01-26 2020-12-11 浙江大学 一种基于注意力机制及特征融合的眼底图中糖网特征分级装置
CN109376767B (zh) * 2018-09-20 2021-07-13 中国科学技术大学 基于深度学习的视网膜oct图像分类方法
CN109727259A (zh) * 2019-01-07 2019-05-07 哈尔滨理工大学 一种基于残差u-net网络的视网膜图像分割算法
CN109858429B (zh) * 2019-01-28 2021-01-19 北京航空航天大学 一种基于卷积神经网络的眼底图像病变程度识别与可视化系统
CN110147715A (zh) * 2019-04-01 2019-08-20 江西比格威医疗科技有限公司 一种视网膜OCT图像Bruch膜开角自动检测方法
CN111184508B (zh) * 2020-01-19 2021-05-18 武汉大学 一种基于联合神经网络的心电信号检测装置及分析方法
CN111329445B (zh) * 2020-02-20 2023-09-15 广东工业大学 基于组卷积残差网络和长短期记忆网络的房颤识别方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106455972A (zh) * 2014-04-28 2017-02-22 西北大学 功能性光学相干断层成像的装置、方法和系统
US20200027209A1 (en) * 2018-07-18 2020-01-23 Case Western Reserve University Predicting response to anti-vascular endothelial growth factor therapy with computer-extracted morphology and spatial arrangement features of leakage patterns on baseline fluorescein angiography in diabetic macular edema
CN110309862A (zh) * 2019-06-11 2019-10-08 广东省人民医院(广东省医学科学院) 基于集成机器学习的dme预后信息预测系统及其应用方法
CN110472530A (zh) * 2019-07-29 2019-11-19 中山大学 基于小波变换和迁移学习的视网膜oct图像分类方法
CN111611877A (zh) * 2020-04-30 2020-09-01 杭州电子科技大学 基于多时空信息融合的抗年龄干扰的人脸识别方法
CN111920375A (zh) * 2020-09-30 2020-11-13 平安科技（深圳）有限公司 一种抗血管内皮生长因子vegf疗效预测装置及方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SU, QIANG ET AL.: "Clinical Efficacy of Anti-VEGF on Exudative Senile Macular Degeneration based on ANGIO-OCT", CHINESE REMEDIES & CLINICS, vol. 20, no. 16, 31 August 2020 (2020-08-31), pages 2683 - 2686, XP055827602, ISSN: 1671-2560 *

Also Published As

Publication number Publication date
CN111920375A (zh) 2020-11-13
CN111920375B (zh) 2021-01-15

Similar Documents

Publication Publication Date Title
US12114929B2 (en) Retinopathy recognition system
Bhaskaranand et al. The value of automated diabetic retinopathy screening with the EyeArt system: a study of more than 100,000 consecutive encounters from people with diabetes
Varadarajan et al. Deep learning for predicting refractive error from retinal fundus images
CN110400289B (zh) 眼底图像识别方法、装置、设备和存储介质
WO2020199593A1 (fr) Appareil et procédé d'apprentissage de modèle de segmentation d'image, appareil et procédé de segmentation d'image, et dispositif et support
US10722180B2 (en) Deep learning-based diagnosis and referral of ophthalmic diseases and disorders
US20220165418A1 (en) Image-based detection of ophthalmic and systemic diseases
Hassan et al. Joint segmentation and quantification of chorioretinal biomarkers in optical coherence tomography scans: A deep learning approach
US20210391056A1 (en) Health big data service method and system based on remote fundus screening
Wainwright et al. Effect of image quality, color, and format on the measurement of retinal vascular fractal dimension
Tsikata et al. Automated brightness and contrast adjustment of color fundus photographs for the grading of age-related macular degeneration
Dong et al. Deep learning-based estimation of axial length and subfoveal choroidal thickness from color fundus photographs
WO2021139446A1 (fr) Appareil et procédé de prédiction d'effet curatif anti-facteur de croissance de l'endothélium vasculaire (vegf)
Giancardo Automated fundus images analysis techniques to screen retinal diseases in diabetic patients
TWI742782B (zh) 視網膜分層厚度檢測模型之建立方法、視網膜分層厚度檢測系統、眼部照護裝置、視網膜分層厚度檢測方法及失智症評估和預測方法
CN116763250A (zh) 眼前节图像的眼压信息提取方法、终端和存储介质
Thanh et al. A real-time classification of glaucoma from retinal fundus images using AI technology
Burke et al. An Open-Source Deep Learning Algorithm for Efficient and Fully Automatic Analysis of the Choroid in Optical Coherence Tomography
Yin et al. A cloud-based system for automatic glaucoma screening
Aswathy Diabetic Retinopathy Detection Using Semantic Segmentation And Optic Disc Localization
Mohan et al. Using artificial intelligence in diabetic retinopathy
Shon et al. Development of a β-variational autoencoder for disentangled latent space representation of anterior segment optical coherence tomography images
Almazroa et al. The appropriateness of digital diabetic retinopathy screening images for a computer-aided glaucoma screening system
Sharma et al. Analysis of eye disease classification by fundus images using different machine/deep/transfer learning techniques
Soysa et al. A mobile base application for cataract and conjunctivitis detection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20911442

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25/05/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20911442

Country of ref document: EP

Kind code of ref document: A1