WO2025212937A1

WO2025212937A1 - Geographic atrophy prognostic prediction based on optical coherence tomography segmentation

Info

Publication number: WO2025212937A1
Application number: PCT/US2025/023024
Authority: WO
Inventors: Miao Zhang; Catherine CUKRAS; Adam Benjamin Pely; Kenta Yoshida
Original assignee: Genentech Inc
Current assignee: Genentech Inc
Priority date: 2024-04-03
Filing date: 2025-04-03
Publication date: 2025-10-09
Anticipated expiration: 2026-10-03

Abstract

A computer-based method and system for predicting future geographic atrophy (GA) lesion growth. An optical coherence tomography (OCT) volume for a retina of a subject associated with a first point in time is received by a process over a network. A segmentation model is used to generate a segmented volume corresponding to the OCT volume, the segmented volume comprising a plurality of 3D segments. For each 3D segment of the plurality of 3D segments, a map output is generated. A set of biomarker values is computed for a corresponding set of biomarker features using the map output. A prediction system is used to generate a prediction output that predicts a progression of a GA lesion in the retina with respect to a future point in time after the first point in time using at least one of the set of biomarker values or the map output.

Description

GEOGRAPHIC ATROPHY PROGNOSTIC PREDICTION BASED ON OPTICAL COHERENCE TOMOGRAPHY SEGMENTATION

Inventors: Miao ZHANG, Catherine CUKRAS, Adam Benjamin PELY, Kenta YOSHIDA

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is related to and claims the benefit of the priority date of U.S. Provisional Patent Application No. 63/574,131 filed April 3, 2024, entitled "GEOGRAPHIC ATROPHY PROGNOSTIC PREDICTION BASED ON OPTICAL COHERENCE TOMOGRAPHY SEGMENTATION," as well as of U.S. Provisional Patent Application No. 63/642,523 filed May 3, 2024, entitled "GEOGRAPHIC ATROPHY PROGNOSTIC PREDICTION BASED ON OPTICAL COHERENCE TOMOGRAPHY SEGMENTATION," as well as of U.S. Provisional Patent Application No. 63/765,516 filed February 28, 2025, entitled "GEOGRAPHIC ATROPHY PROGNOSTIC PREDICTION BASED ON OPTICAL COHERENCE TOMOGRAPHY SEGMENTATION," each of which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

[0002] This description is generally directed toward the prediction of geographic atrophy (GA) progression. More particularly, the present description provides methods and systems for predicting the region of growth (ROG) for geographic atrophy lesion at a future point in time using deep learning and segmentation of optical coherence tomography (OCT) images.

BACKGROUND

[0003] Age-related macular degeneration (AMD) is a leading cause of vision loss in patients 50 years or older. Geographic atrophy (GA) is a late-stage form of AMD. GA is the degeneration of the retina and can hinder daily activities such as, for example, driving, reading, etc. GA is characterized by progressive and irreversible loss of choriocapillaris, retinal pigment epithelium (RPE), and photoreceptors. GA progression varies between patients and currently, no widely accepted treatment for preventing or slowing down the progression of GA exists. Therefore, evaluating GA progression in individual patients may be important to researching GA and developing an effective treatment.

[0004] Optical coherence tomography (OCT) can provide a detailed scan of the retina to help detect retinal disease progression and other ophthalmologic problems much earlier than was possible in the past. To investigate the extent of the deterioration in a retina with, for example, AMD or GA, OCT images (e.g., time domain optical coherence tomography (TD-OCT) or spectral domain optical coherence tomography (SD-OCT) images) of the retina may be obtained and used for identifying features that may be associated with varying degenerative levels of disease (e.g., AMD, GA). SD-OCT is an imaging technique in which light is directed at the retina at various optical frequencies and in which the reflected light is collected to capture two-dimensional or three-dimensional, high-resolution, cross-sectional images of the retina via interferometric signals detected as a function of frequencies. Different features that are captured in the SD-OCT images can be identified via retinal segmentation and used in determining the severity of retinal disease, which may help guide the diagnosis and/ortreatment of the disease. However, currently available techniques used in extracting, understanding, and/or interpreting such features may be plagued with tediousness and/or prone to error. Accordingly, the cumbersome nature of the retinal disease investigation process may be a limiting factor in the diagnosis and/or treatment of the disease. Thus, it may be desirable to have one or more methods and/or systems that recognize and take into account these issues.

SUMMARY

[0005] In one or more embodiments, a computer-based method for predicting future geographic atrophy (GA) lesion growth is provided. An optical coherence tomography (OCT) volume for a retina of a subject associated with a first point in time (e.g., first timepoint) is received. In one or more embodiments, the OCT volume is received by a processor over a network. In one or more embodiments, a segmentation model implemented using the processor is used to generate a segmented volume corresponding to the OCT volume, the segmented volume comprising a plurality of three-dimensional (3D) segments. At least one 3D segment of the plurality of 3D segments represents a retinal element that comprises two or more elements. Each element of the two or more elements is either a layer element or a pathological element. For each 3D segment of the plurality of 3D segments, a map output is generated by the processor. A set of biomarker values is computed by the processor for a corresponding set of biomarker features associated with the retina using the map output. A prediction system implemented using the processor is used to generate a prediction output that predicts a progression of a geographic atrophy (GA) lesion in the retina with respect to a future point in time (e.g., future timepoint) after the first point in time using at least one of the set of biomarker values or the map output. [0006] In one or more embodiments, a system comprises one or more data processors; and a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform any one or more of the methods described herein or a portion thereof.

[0007] In one or more embodiments, a computer-program product tangibly embodied in a non-transitory machine-readable storage medium is provided, including instructions configured to cause one or more data processors to perform any one or more of the methods described herein or a portion thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] For a more complete understanding of the principles disclosed herein, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0009] FIG. 1 is a block diagram of an image analysis system in accordance with various embodiments of the present disclosure.

[0010] FIG. 2 is a block diagram of the segmentation model from FIG. 1 in accordance with various embodiments of the present disclosure.

[0011] FIG. 3 is a block diagram of the prediction system from FIG. 1 in accordance with various embodiments of the present disclosure.

[0012] FIG. 4 is a flow chart illustrating an embodiment of a process for predicting GA growth in accordance with one or more embodiments of the present disclosure.

[0013] FIG. 5 is an example 2D-segment and corresponding model output in accordance with one or more embodiments of the present disclosure.

[0014] FIG. 6 is a diagram illustrating generating a plurality of layer thickness maps in accordance with one or more embodiments of the present disclosure.

[0015] FIG. 7 is a diagram illustrating generating a plurality of layer mean intensity maps in accordance with one or more embodiments of the present disclosure.

[0016] FIG. 8 is a diagram illustrating generating a prediction output in accordance with one or more embodiments of the present disclosure.

[0017] FIG. 9 is a diagram illustrating the use of OCT-defined retinal layer loss rates as a surrogate for FAF-defined GA growth rates in accordance with one or more embodiments of the present disclosure.

[0018] FIG. 10 is a flow chart illustrating an embodiment of a process for training a segmentation model in accordance with one or more embodiments of the present disclosure.

[0019] FIG. 11 is a block diagram of a computer system in accordance with various embodiments of the present disclosure.

[0020] It is to be understood that the figures are not necessarily drawn to scale, nor are the objects in the figures necessarily drawn to scale in relationship to one another. The figures are depictions that are intended to bring clarity and understanding to various embodiments of apparatuses, systems, and methods disclosed herein. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. Moreover, it should be appreciated that the drawings are not intended to limit the scope of the present teachings in any way.

DETAILED DESCRIPTION

I. Overview

[0021] The ability to accurately predict how geographic atrophy (GA) will progress over time may be useful in many different scenarios. As one example, predictions about GA progression may be used to improve patient stratification in clinical trials where the goal is to slow GA progression, thereby allowing for improved assessment of treatment effects. Additionally, in some cases, predictions about GA progression may be used to understand disease pathogenesis via correlation to genotypic or phenotypic signatures. Recognizing and taking into account the importance and utility of a methodology and system that can provide these improvements, the specification describes various embodiments for using deep learning to predict future region of growth of GA lesions from retinal imaging data such as, for example, optical coherence tomography (OCT) volumes.

[0022] A GA lesion can be imaged by various imaging modalities. For instance, OCT images may be used to quantify one or more GA lesion areas. GA growth rate, which is the change in an area and/or volume of a lesion over a duration of time (e.g., over some time period), as measured using OCT images, is widely accepted as an anatomic metric for GA progression in clinical trials.

[0023] Some currently available techniques for evaluating GA progression using an OCT image rely on human graders to first manually identify the portion of an OCT image that is the GA lesion. In some cases, this first step is semi-automated, relying on the human grader to make manual refinements and/or corrections to a software-generated initial outline of the GA area. Then, the identified portion of the OCT image is evaluated to determine the GA lesion area and GA growth rate. These techniques may involve a two-step process that can take more time than is desirable, may be prone to human error, may be less accurate than desired, and/or may produce variable results depending on the knowledge and expertise of one or human graders. Further, these types of techniques may be unable to visually present to a medical professional (e.g., clinician, healthcare provider, etc.) how the OCT image might look in the future (e.g., 3 months, 6 months, 9 months, 1 year, etc. later). In present embodiments, GA growth rate (e.g., annualized growth rate) may be predicted from baseline OCT images. [0024] The inability of existing systems to predict GA growth rate is at least in part due to the limitations associated with existing methods and system for retinal segmentation of OCT images. For example, some segmentation algorithms are limited to identifying individual retinal layer elements or individual retinal pathological elements, and at most, identifying one or more such retinal elements. The embodiments described herein, however, recognize that it may be desirable to perform retinal segmentation in a way that identifies multiple segments, each of which represents a retinal element that itself is a combination of two or more elements (e.g., two or more retinal layer elements, two or more retinal pathological elements, or one or more retinal layer elements in combination with one or more retinal pathological elements).

[0025] Using segmentation to identify combined elements in this way may lead to improved model performance and more accurate predictions about GA growth rate where the individual elements themselves may be more difficult to delineate in the imaging. For example, accurately segmenting an external limiting membrane and the myoid zone individually may be difficult with certain OCT images. However, segmenting the area that represents the combined external limiting membrane and myoid zone may be performed more accurately, which may lead to better performance overall and improved accuracy in predictions generated based on this segmentation. Further, segmentation that specifically identifies selected combinations of elements that are chosen for their prognostic characteristics may reduce the overall computing resources, time, and expense that might be otherwise associated with segmenting out each individual element.

[0026] In one or more embodiments, OCT images are segmented to form a segmented volume that is comprised of a plurality of 3D segments. Each 3D segment of the plurality of 3D segments may represent or otherwise identify a corresponding retinal element or a combination of retinal elements. The segmented volume can be further processed to generate a map output that corresponds to each 3D segment. The map output for a given segment (e.g., RPE, ELM, EZ) may include a thickness map, an intensity projection map, or both.

[0027] The thickness map is a 2D representation of the thickness of the layer(s) and/or element(s) represented by 3D segment (e.g., RPE thickness map, ELM thickness map, EZthickness map, and so on). For example, if 3D segment 300 represents the retinal element that includes ELM and MZ, thickness map 302 provides information about the thickness of the layer that includes ELM and MZ. The thickness map may be, for example, a 2D map in which the value (e.g., pixel intensity value, pixel hue value, or the like) at any x,y position on the 2D map indicates the thickness of the corresponding x,y position of 3D segment 300.

[0028] The intensity projection map be generated based on information also obtained from the original OCT images. For example, an intensity projection map may be a 2D map in which the value at any x,y position on the 2D map indicates the mean intensity (based on the OCT images) of the pixels in the z direction for the corresponding x,y position of 3D segment 300. In some embodiments, intensity projection maps are referred to as layer mean intensity projections.

[0029] The thickness map and/or intensity projection maps may be used to extract numerical features or biomarkers. For example, on a thickness map, an area where the thickness is (or below a certain threshold as indicated by the values of the pixels in the map) indicates a loss area (mm²) or, in other words, a geographic atrophy lesion. Different types of numerical features can be extracted from thickness maps, including, but not limited to: (1) an average thickness of the segment outside of the lesion (or outside the area of loss), which would be an average thickness as indicated by the average of the corresponding pixel values; (2) total average thickness across the thickness map, including the area of loss, which would be a total average thickness as indicated by the average of the corresponding pixel values; (3) a count of the number of geographic atrophy lesions present; and (4) a total length of any lesions that meet the border of the thickness map, which provides an indication of how far beyond the imaging field of view the lesion extends. In addition to the above-described features, other types of features (e.g., shape features, morphology features) may also be extracted based on the thickness maps.

[0030] The intensity projection maps may also be used to extract different types of numerical features that may be used as biomarkers. For example, an average intensity and a total average intensity can be extracted. Average intensity is the average pixel intensity outside of the lesion (s) (or area of loss). Total average intensity is the average pixel intensity including the regions of loss. In addition to the above-described features, other types of features (e.g., shape features, morphology features) may also be extracted based on the intensity projection maps. [0031] The embodiments described herein provide different types of models that may be used to provide information about geographic atrophy progression. For example, in some cases, a statistical model is used. The statistical model receives as input various inputs that are used to predict geographic atrophy progression biomarkers (e.g., growth rate, lesion area). These inputs may include, for example, a total area of loss for all segments (or a selected combination of segments), an area of choroidal hypertransmission (e.g., a layer with certain thickness below Bruch's membrane); a ratio between different area losses (e.g., RPE loss / EZ loss), an average thickness of all segments (or a selected combination of segments), total retinal thickness, a count of lesions for all the layers segmented; a length of lesions reaching the image border, an average intensity of all segments; an average intensity of choroidal hypertransmission intensity image, distance of the lesion to the fovea, or a combination thereof. In one example, the model predicts EZ loss area, area, the ratio of RPE loss and EZ loss, count of RPE lesions, distance of RPE lesion to fovea, average RPE thickness, average EZ+IZ thickness, HRF volume, SDD volume, drusen volume, etc.

[0032] In other cases, a deep learning model may be used to predict geographic atrophy progression biomarkers (e.g., growth rate, lesion area). For example, for each segment of interest (e.g., ELM+EZ; EZ+IZ; RPE), the corresponding thickness map and intensity projections map are input into a submodel (e.g., a fuse convolutional neural network (CNN)). The outputs of these three fuse CNNs may be input into another fuse CNN, with its output being sent into another model for processing to generate predicted biomarkers. This other model may include a resnet architecture and various customized layers and features.

[0033] The correlations between the OCT-defined retinal layer (e.g., segment) loss rates and FAF-defined GA growth rates and lesion areas may be determined. For instance, the model may convert information and/or data associated with OCT imaging into information relevant to a fundus autofluorescence (FAF) imaging. In some embodiments, a segmentation model may quantify an extent of outer retina disruption (ORD) and compare the results to manually graded GA areas of a FAF to identify correlations between retinal loss shown in OCT imaging to GA growth rates determined in FAF images. [0034] The predicted biomarkers for geographic progression (e.g., growth rate, lesion area) are traditionally FAF-derived biomarkers that are well-understood in a clinical or clinical trial environment. Here, predicting geographic progression biomarkers based on the metrics identified for segments of interest (e.g., ELM+MZ; EZ+IZ; and RPE) identified from OCT images, as described above, provides a technical improvement. The features (numerical features, loss features, etc.) derived from the various stages of processing of OCT images described above may be transformed into corresponding values for biomarkers that are well-known in the medical industry (clinical / clinical trial) but that can be understood with respect to an entirely different imaging modality— FAF images. Because OCT images, which are 3D (e.g., comprised of a plurality of 2D OCT slices) can provide a more detailed, cross-sectional view of retinal layers, OCT images can provide more precise measurement of tissue thickness and better resolution and visualization of structural changes within the retina (e.g., loss indicative of a GA lesion), using OCT to predict the GA progression biomarkers provides a technical improvement over using FAF images in various situations. Further, OCT images provide more detailed information about the various retinal layers and elements within the retina.

[0035] Accordingly, a desire exists for methods and systems that improve the speed, efficiency, and accuracy associated with predicting GA lesion growth and that provide a way of visualizing the region of growth for GA lesions for a future point in time. Thus, the embodiments described herein provide methods and systems for predicting the values for biomarkers that indicate GA lesion progression.

II. Example Image Analysis System for Predicting Geographic Atrophy (GA) Progression

[0036] Referring now to the figures, FIG. 1 is a block diagram of an image analysis system 101 in accordance with various embodiments. Image analysis system 101 is used to predict the progression of geographic atrophy (GA) lesions in the retinas of subjects. Image analysis system 101 includes computing platform 102, data storage 104, and display system 106. Computing platform 102 may take various forms. In one or more embodiments, computing platform 102 includes a single computer (or computer system) or multiple computers (e.g., one or more processors where a processor may include one or more processors) in communication with each other. In other examples, computing platform 102 takes the form of a cloud computing platform. [0037] Data storage 104 and display system 106 are each in communication with computing platform 102. In some examples, data storage 104, display system 106, or both may be considered part of or otherwise integrated with computing platform 102. Thus, in some examples, computing platform 102, data storage 104, and display system 106 may be separate components in communication with each other, but in other examples, some combination of these components may be integrated together.

[0038] Image analysis system 101 includes image processor 108, which may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, image processor 108 is implemented in computing platform 102. Image processor 108 receives OCT imaging data 110 for processing. For example, OCT imaging data 110 may be received from OCT imaging system 111, retrieved from data storage 104 or some other type of storage (e.g., cloud storage), or received in some other manner.

[0039] Image analysis system 101 may be in communication with OCT imaging system 111 via network 112. For example, image processor 108 may receive OCT imaging data 110 from OCT imaging system 111 over network 112 (e.g., over a network interface). Network 112 may be implemented using a single network or multiple networks in combination. Network 112 may be implemented using any number of wired communications links, wireless communications links, optical communications links, or combination thereof. For example, in various embodiments, network 112 may include the Internet or one or more intranets, landline networks, wireless networks, and/or other appropriate types of networks. In another example, the network 112 may comprise a wireless telecommunications network (e.g., cellular phone network) adapted to communicate with other communication networks, such as the Internet. In some cases, network 112 includes at least one of a local area network (LAN), a virtual local area network (VLAN), a wide area network (WAN), a public land mobile network (PLMN), the Internet, or another type of network.

[0040] The OCT imaging system 111 and image analysis system 101 may each include one or more electronic processors, electronic memories, and other appropriate electronic components for executing instructions such as program code and/or data stored on one or more computer readable mediums to implement the various applications, data, and steps described herein. For example, such instructions may be stored in one or more computer readable media such as memories or data storage devices (e.g., data storage 104) internal and/or external to various components of image analysis system 101, and/or accessible over network 112.

[0041] Although only one of each of OCT imaging system 111 and the image analysis system 101 is shown, there can be more than one of each in other embodiments. Further, although FIG. 1 shows the OCT imaging system 111 and the image analysis system 101 as two separate components, in some embodiments, the OCT imaging system 111 and the image analysis system 101 may be parts of the same system (e.g., and maintained by the same entity such as a health care provider or clinical trial administrator). In some cases, a portion of image analysis system 101 may be implemented as part of OCT imaging system 111. For example, image analysis system 101 may be configured to run as a module implemented using a processor, microprocessor, or some other hardware component of OCT imaging system 111. In still other embodiments, image analysis system 101 may be implemented within a cloud computing system that can be accessed by or otherwise communicate with OCT imaging system 111. In some cases, at least a portion of (e.g., a module of) image processor 108 is implemented within OCT imaging system 111.

[0042] OCT imaging system 111 may generate OCT imaging data 110 that includes one or more OCT images. For example, OCT imaging data 110 may include any number of three- dimensional, two-dimensional, or one-dimensional OCT images. A two-dimensional OCT image may take the form of, for example, without limitation, an OCT B-scan. A three-dimensional OCT image may be referred to as an OCT volume. An OCT volume may itself be comprised of multiple OCT B-scans.

[0043] In one or more embodiments, the OCT imaging system 111, which may include or be referred to as an OCT scanner or machine, is configured to generate OCT imaging data 110 for the tissue of a patient. For example, OCT imaging system 111 may be used to generate OCT imaging data 110 for the retina of a patient. In some instances, OCT imaging system 111 can be a large tabletop configuration used in clinical settings, a portable or handheld dedicated system, or a "smart" OCT system incorporated into user personal devices such as smartphones. In some cases, the OCT imaging system 111 may include an image denoiser that is configured to remove noise and other artifacts from a raw OCT volume image to generate an OCT volume (e.g., OCT volume 114).

[0044] In one or more embodiments, OCT imaging data 110 includes OCT volume 114 for a retina of a subject. OCT volume 114 may be comprised of a plurality of OCT B-scans 115 of the retina of the subject. The plurality of OCT B-scans 115 may include, for example, without limitation, 10s, 100s, 1000s, 10,000s, or some other number of OCT B-scans. An OCT B-scan may also be referred to as an OCT slice image or a cross-sectional OCT image.

[0045] In some embodiments, the retina is a healthy retina. In other embodiments, the retina is one that has been diagnosed with or is suspected of having a retinal disease. For example, the diagnosis may be one of age-related macular degeneration (AMD), neovascular age-related macular degeneration (nAMD), geographic atrophy, or some other type of retinal disease. In one or more embodiments, the retina captured by OCT volume 114 may be one that has been diagnosed (e.g., by a computer system, program, or human) as having a geographic atrophy (GA) lesion. The GA lesion may be a continuous or discontinuous region of the retina that has suffered degeneration (e.g., chronic progressive degeneration). The GA lesion may include one lesion (e.g., one continuous lesion region) or multiple lesions (e.g., discontinuous lesion region comprised of multiple, separate lesions).

[0046] In one or more embodiments, OCT imaging data 110 includes one or more reference OCT images that are captured for one or more reference points in time (e.g., timepoints). The one or more reference points in time may include, for example, a baseline (e.g., with respect to treatment) point in time, a point in time that is 6 months after the baseline (e.g., after a first treatment), a point in time that is 3 months after a first treatment, a point in time that is 12 months after a first treatment, or some other type of reference point in time. In one or more embodiments, a reference point in time may be a point in time prior to treatment, the same day as a treatment dose (e.g., a first treatment dose), or some other type of baseline or reference point in time.

[0047] In one or more embodiments, OCT volume 114 corresponds to a particular reference point in time and captures a GA lesion. While the example interval of time between reference points in time described above is 6 months, the interval of time may be 1 month, 3 months, 9 months, or some other interval of time measured in days, weeks, months, or years. In some embodiments, the OCT images associated with an eye are spaced by a consistent interval of time (e.g., 6 months).

[0048] In some embodiments the one or more reference points in time includes a baseline point in time that is not relative to a treatment.

[0049] In one or more embodiments, image processor 108 may generate a three-dimensional (3D) image input 118 using OCT imaging data 110. For example, OCT volume 114 may be preprocessed using a set of preprocessing operations to form the 3D image input 118. The set of preprocessing operations may include, for example, without limitation, at least one of a normalization operation, a scaling operation, a resizing operation, a horizontal flipping operation, a vertical flipping operation, a cropping operation, a rotation operation, a noise filtering operation, or some other type of preprocessing operation. A normalization operation may be performed to normalize the coordinates of the coordinate system for OCT volume 114. In some cases, pixel values may be normalized (e.g., normalized to values between 0-1). A scaling operation may include, for example, scaling a coordinate system associated with OCT volume 114. A resizing operation may include changing a size of each of the plurality of OCT B-scans 115. A preprocessing operation of the set of preprocessing operations may be performed on one or more of the plurality of OCT B-scans 115 of the OCT volume 114.

[0050] The image processor 108 may include segmentation model 120 and prediction system 122. Segmentation model 120 receives an image input (e.g., 3D image input 118) and processes this image input (e.g., via retinal segmentation) to generate a segmentation output 124. The segmentation output 124 may include any number of images. For example, segmentation output 124 may take the form of a segmented volume 126 that includes a plurality of segmented 2D images (e.g., 2D slices).

[0051] Segmented volume 126 includes a plurality of 3D segments 128. Each 3D segment of the plurality of 3D segments 128 may include a 3D representation of one or more retinal elements. For example, a 3D segment of the plurality of 3D segments 128 may represent or otherwise identify a corresponding retinal element or a combination of retinal elements. Further details about how segmentation model 120 can be used to process 3D image input 118 to generate segmented volume 126 are discussed below in further detail in Section ILA.

[0052] In one or more embodiments, the segmentation output (e.g., segmented volume 126) of segmentation model 120 is sent as input into prediction system 122 for processing (e.g., transmitted to prediction system 122 and/or received by prediction system 122). For example, prediction system 122 may generate map output 130 based on segmented volume 126. Map output 130 may include, for example, multiple sets of maps, such that a set of maps is generated for each 3D segment of plurality of 3D segments 128. Each individual set of maps may include one or more maps.

[0053] Prediction system 122 may compute a set of biomarker values 132 corresponding to a set of biomarker features for the retina based on map output 130. The set of biomarker features for the retina may include, for example, one or more features that can be quantified. In other words, the set of biomarker features may include one or more numerical features.

[0054] In one or more embodiments, prediction system 122 includes prediction model 134. Prediction model 134 may receive map output 130, set of biomarker values 132, or both as input. Prediction model 134 processes this input to generate a prediction output 136 that predicts how the GA lesion will progress over time (e.g., with respect to a future point in time). For example, prediction output 136 may indicate how the GA lesion will progress at a future point in time after the first point in time at which OCT imaging data 110 was generated, after a baseline point in time, after a time at which the initial diagnosis was made, after or just priorto an initial treatment or a treatment dose.

[0055] In some embodiments, clinical data 142 associated with the imaging data 110 and/or subject associated with the imaging data 110 is also provided to the image analysis system 101 and/or the prediction system 122. In some embodiments, the clinical data 142 includes data relating to the demographics, functional measurements, and manual readings from non-OCT imaging modalities. For example, the clinical data may include included age, sex, smoking status, BCVA, low luminance deficit, presence of SDD, FAF GA area, FAF GA lesion contiguity, FAF GA to central fovea distance, and FAF GA fovea involvement. [0056] Prediction output 136 may include, for example, predicted growth image 138, set of predicted progression features 140, and the like, as discussed further below in Section II. B.

II.A. Example Segmentation Model

[0057] FIG. 2 is a block diagram involving the segmentation model 120 from FIG. 1 in accordance with various embodiments. As discussed above in Section II, image processor 108 in FIG. 1 may generate a three-dimensional (3D) image input 118 using OCT imaging data 110 in FIG. 1. Segmentation model 120 receives 3D image input 118 for processing and performs retinal segmentation on 3D image input 118 to generate segmented volume 126, which corresponds to 3D image input 118. In one or more embodiments, segmentation model 120 includes a semantic segmentation model 200. The semantic segmentation model 200 may be implemented using, for example, without limitation, DeepLabv3 (or a similar type of model). In some embodiments, the segmentation model 120 is or includes the retinal segmentation system 108 as described in International Publication No. WO2023205511A1, which is incorporated by reference herein in its entirety. In other embodiments, the segmentation model 120 generates the segmented volume from OCT imaging data according to one or more OCT segmentation techniques as described in International Publication No. WO2023205511A1.

[0058] As previously discussed, segmented volume 126 may include a plurality of 3D segments 128. For instance, each 3D segment of plurality of 3D segments 128 identifies a retinal element 201 that includes a set of elements 202 in which each element in the set of elements 202 may be either a layer element 204 or pathological element 206. An example of a segmented volume comprising a plurality of 3D segments is shown in FIG. 6 (and is shown as the result of an example model/modeling algorithm referred to as EyeNotate).

[0059] For example, retinal segmentation performed by the segmentation model 120 includes the detection and identification of one or more retinal (e.g., retina-associated) elements (such as, for example, retinal element 201) in a retinal image (such as, for example, 3D image input 118). A retinal element may include one or more elements (such as, for example the set of elements 202), each of which takes the form of either a retinal layer element or a retinal pathological element (such as, for example, layer element 204 or pathological element 206, respectively). Detection and identification of one or more retinal layer elements may be referred to as layer element (or retinal layer element) segmentation. Detection and identification of one or more retinal pathological elements may be referred to as pathological element (or retinal pathological element) segmentation.

[0060] A retinal layer element 204 may be, for example, a retinal layer or a boundary associated with a retinal layer. Examples of retinal layers include, but are not limited to, an internal limiting membrane (ILM) layer, a retinal nerve fiber layer (RNFL), a ganglion cell layer (GCL), an inner plexiform layer (IPL), an inner nuclear layer (INL), an outer plexiform layer (OPL), an outer nuclear layer (ONL), an external limiting membrane (ELM) layer, a photoreceptor layer(s), a retinal pigment epithelial (RPE) layer, a layer of RPE detachment, a Bruch's membrane (BM) layer, a choriocapillaris layer, a choroidal stroma layer, an ellipsoid zone (EZ), and other types of retinal layer. In some cases, a retinal layer may be comprised of one or more layers. As one example, a retinal layer may be an outer plexiform layer-Henle fiber layer (OPL-HFL). A boundary associated with a retinal layer may be, for example, an inner boundary of the retinal layer, an outer boundary of the retinal layer, a boundary associated with a pathological feature of the retinal layer (e.g., an inner or outer boundary of detachment of the retinal layer), or some other type of boundary. For example, a boundary may be an inner boundary of an RPE (IB-RPE) detachment layer, an outer boundary of the RPE (OB-RPE) detachment layer, or another type of boundary.

[0061] A retinal pathological element 206 may include, for example, fluid (e.g., a fluid pocket), cells, solid material, or a combination thereof that evidences a retinal pathology (e.g., disease or condition such as AMD or DME). For example, the presence of certain retinal fluids may be a sign of nAMD or DME. Examples of retinal pathological elements include, but are not limited to, intra retinal fluid (IRF), subretinal fluid (SRF), fluid associated with pigment epithelial detachment (PED), hyperreflective material (HRM), subretinal hyperreflective material (SHRM), intraretinal hyperreflective material (IHRM), hyperreflective foci (HRF), a retinal fluid pocket, drusen, a development of fibrosis, subretinal deposits (SDD), and a disruption. In some cases, a retinal pathological element may be a disruption (e.g., discontinuity, delamination, loss, etc.) of a retinal layer or retinal zone. For example, the disruption may be of the ellipsoid zone, of the ELM, of the RPE, or of another layer or zone. The disruption may represent damage to or loss of cells (e.g., photoreceptors) in the area of the disruption.

[0062] Additionally, a retinal pathological element 206 may include a characteristic or subtype of one of the fluids (e.g., IRF, SRF, fluid associated with PED), materials (e.g., HRM, SHRM, IHRM), lesions (e.g., HRF, SHRM lesions), or disruptions. In particular, examples of retinal pathological elements may include characteristics and/or subtypes of the different types of elements and disruptions described above that can be detected and identified via retinal segmentation. For example, whether a retinal fluid is clear or turbid may be detectable and identifiable characteristic of the retinal fluid. Accordingly, in some examples, a retinal pathological element may be clear IRF, turbid IRF, clear SRF, turbid SRF, some other type of clear retinal fluid, some other type of turbid retinal fluid, or a combination thereof. In some cases, for SHRM, shape characteristics (e.g., tall SHRM, dome-shaped SHRM at the foveal center, flat SHRM near the foveal center, dysmorphic, etc.), boundary characteristics (e.g., ill-defined SHRM, well-defined SHRM), reflectivity (e.g., increased reflectivity or other levels of reflectivity), layering characteristics (e.g., hyperreflective bands in SHRM lesions), and lesion characteristics (e.g., the height, width, and/or area of SHRM lesions) may be examples of retinal pathological elements that may be detected and identified via retinal segmentation.

[0063] In various embodiments, retinal element 201 may include, for example without limitation, one, two, three, four, or some other number of elements selected from the list that includes: ILM, RNFL, GCL, IPL, INL, OPL, HFL (Henre-fiber layer), ONL, ELM, myoid zone (MZ), EZ, interdigitation zone (IZ), RPE, Drusen, BM, HFL+ONL, or Drusen+. HFL+ONL may include any tissues between ONL to the next visible structure (e.g., ELM, EZ, RPE) when certain layers are disrupted. Drusen+ may include, for example, Drusen and Bruch's membrane when Drusen exists. Drusen-i- includes Bruch's membrane when RPE is absent. EZ may be a hyperreflective band between MZ and the outer segments of the photoreceptors. The outer segments of the photoreceptors layers may be a hyporeflective band between EZ and IZ. IZ may be a hyperreflective band representing the contact between the apices of the RPE cells and the outer segments of the photoreceptors (previously called the cone outer segment tips and rod outer segment tips). [0064] In various embodiments, the retinal element 201 may include the combination of two or more layer elements, two or more pathological elements, one or more layer elements with one or more pathological elements, ora combination thereof may be used to improve robustness of segmentation. For example, an HFL+ONL retinal element may be formed that includes any tissues between ONL to the next visible structure (e.g., ELM, RZ, RPE). A Drusen+ retinal element may be formed that includes Drusen and BM when Drusen exists in the retina, and may include BM when RPE is absent.

[0065] In various embodiments, segmentation model 120 may include a machine-learning model, such as a neural network (e.g., convolutional neural network (CNN) or artificial neural network (ANN). The one or more neural networks described in this disclosure may be trained using a training dataset (also referred to herein as "training data"). A machine learning model may be trained using a training dataset that includes example inputs that correlate to example outputs. For example, segmentation model 120 may be trained using a segmentation training dataset 208, where segmentation training dataset 208 may include example OCT volumes that are correlated to example segmentation volumes.

[0066] The training data 208 may include clinical data, historical data (e.g., past inputs and outputs of the neural network that have been approved or correct using, for example, a user), and so on. The training of the neural network may include a forward pass and backpropagation through the neural network, which may update weighting parameters for nodes of the neural network to adjust for errors produced in the forward pass (e.g., misclassified objects). In various embodiments, other types of neural networks and other training processes may be used in accordance with the present disclosure.

[0067] In various embodiments, the training data 208 may include training inputs and correlated training outputs that may be received by the segmentation model 120 to generate and/or train a neural network, such as the models described herein. In several embodiments, correlations may indicate causative or predictive links between data (e.g., inputs and outputs) and may include modeled relationships (e.g., mathematical relationships). For example, a neural network, such as deep learning model, may use correlations to determine an output, such as segmentation output 124 (e.g., segmented volume 126), from an input, such as OCT volume 114. In some embodiments, training data 208 may include compiled data (e.g., OCT imaging data and/or clinical data associated with subjects) and/or historical training data, where historical training data includes previously received inputs and corresponding determined outputs (e.g., historical inputs and outputs that have been fed back into system 101). For example, the neural network may iteratively be updated using previously used inputs and determined outputs such that an updated neural network (e.g., segmentation model 120) may be trained using updated training data.

[0068] In some embodiments, segmentation output 124 (e.g., plurality of 3D segments 128) and/or other data obtained by analyzing received images (e.g., OCT images) using a neural network may be presented (e.g., displayed) to a user, such as to provide the user an opportunity to review the data and provide user input to adjust the data as appropriate. For instance, the user input may be analyzed and fed back to update a training dataset used to train the neural network, such as the training datasets previously discussed herein. In this regard, the user input may be provided in a backward pass through the neural network to update neural network parameters based on the user input. In some aspects, the backward pass may include backpropagation and gradient descent. In some cases, the presence of user input may indicate that the outputs are in error (e.g., not correct to the user). In some cases, the lack of user input may indicate that the user has determined the generated outputs to not be in error (e.g., sufficiently correct for the user). Adjustment of the training dataset (e.g., by removing prior training data, adding new training data, and/or otherwise adjusting existing training data) may allow for improved accuracy.

[0069] In one or more embodiments, the training dataset 208 (e.g., segmentation training dataset) may be received from user input (e.g., a user inputting example inputs and outputs using a user interface of system 101). In other embodiments, the training dataset 208 may be retrieved from a memory component or database (e.g., data storage 104) communicatively connected to computing platform 102. For example, one or more training datasets may include clinical data and/or data stored in data storage 104. II. B. Example Prediction System

[0070] FIG. 3 is a block diagram involving the prediction system 122 from FIG. 1 in accordance with various embodiments. In several embodiments, prediction system 122 receives segmented volume 126 as input for processing. Prediction system 122 processes segmented volume 126 to generate map output 130.

[0071] As previously discussed, map output 130 includes one or more maps for each 3D segment of plurality of 3D segments in segmented volume 126. When segmented volume 126 includes a plurality of 2D segmented images, each 3D segment of plurality of 3D segments includes a plurality of 2D segments, with each 2D segment belonging to a corresponding 2D segmented image of the plurality of 2D segmented images.

[0072] For a given 3D segment (e.g., 3D segment 300) of plurality of 3D segments 128, map output 130 may include, for example, thickness map 302, intensity projection map 304, or both. For instance, in one or more embodiments, thickness map 302 may be generated based on a segmentation mask (e.g., 3D segment 300). Thickness map 302 may be a layer thickness map that represents the thickness of the layer represented by 3D segment (e.g., RPE thickness map, ELM thickness map, EZ thickness map, and so on). For example, if 3D segment 300 represents the retinal element that includes ELM and MZ, thickness map 302 provides information about the thickness of the layer that includes ELM and MZ. The layer thickness map may be, for example, a 2D map in which the value (e.g., pixel intensity value, pixel hue value, or the like) at any x,y position on the 2D map indicates the thickness of the corresponding x,y position of 3D segment 300. The pixel intensity value may be, for example, 0 to 255.

[0073] In one or more embodiments, intensity projection map 304 may be a map that provides information about the layer mean intensity for 3D segment 300. For example, intensity projection map 304 may be, for example, a 2D map in which the value at any x,y position on the 2D map indicates the mean intensity of the pixels in the z direction for the corresponding x,y position of 3D segment 300. In some embodiments, intensity projection maps are referred to as layer mean intensity projections.

[0074] Prediction system 122 may use map output 130 to compute a set of biomarker values 132 corresponding to a set of biomarker features. Generating a biomarker value for a corresponding biomarker feature may include, for example, applying one or more masks (e.g., RPE or ELM masks) to a map (e.g., thickness map 302) in orderto compute an accurate numerical measurement. In some embodiments, the set of biomarker features includes one or more biomarker features for a given retinal element (e.g., as represented by 3D segment 300) and may include one or more biomarker features for each retinal element represented by plurality of 3D segments 128. In one or more embodiments, a biomarker feature (and thus its corresponding value) in the set of biomarker features relates to multiple retinal elements. In various embodiments, the set of biomarkers values 132 may be identified using, for example, a machine learning model (e.g., neural network and/or deep-learning model).

[0075] Set of biomarker values 132 may include, for example, without limitation, values for various features computed using thickness maps, these features including, but not limited to, loss area, average thickness, count of GA lesions, length of lesion reaching border, and other features based on shape or morphology of the retinal elements represented or GA lesion(s). Loss area may be an area where the thickness of a retinal element (e.g., retinal element 201 in FIG. 2) is zero (corresponding to a pixel intensity value of zero) or below a certain pre-defined threshold (e.g., corresponding to a pixel intensity value of 1, 2, 3, 4, 5, 6, 7, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20).

[0076] Average thickness for a retinal element may be computed in, for example, millimeters and may be computed in different ways. For example, average thickness may be evaluated as valid average thickness (e.g., average thickness of the retinal element outside the GA lesion, such as RPE loss lesion or ELM loss lesion), or as total average thickness (e.g., average thickness of the retinal element including the regions of the retinal element covered by the GA lesion). Count of GA lesions may be defined by the number of lesion components with an area larger than a predefined threshold. Length of lesion reaching border may be defined as the total length of lesion at the border of the map (e.g., in units millimeters). For example, if a lesion extends up to the edge of two borders of a thickness map, the length of the lesion at each of these borders may be computed.

[0077] In other embodiments, the set of biomarker values 132 may be generated using intensity projection maps (e.g., intensity projection map 304), the features including, for example, average intensity of a retinal element, and other features based on shape or morphology of the retinal elements or GA lesion(s). In some embodiments, the GA lesion may be represented by a dark space (e.g., a pixel or group of pixels having a minimum intensity value). Average intensity may be evaluated as valid average intensity (e.g., average intensity of pixels associated with the retinal element outside of the GA lesion, such as RPE loss lesion or ELM loss lesion), or as total average intensity (e.g., average intensity of the retinal element including the regions of the retinal element covered by the GA lesion).

[0078] In some embodiments, the set of biomarker values 132 may be generated through postprocessing of intensity projection maps. Postprocessing operations may include, for example, removing image noise or filtering out retinal elements based on size (e.g., filtering based on GA lesion size or filtering based on non-Drusen EZ loss sizing).

[0079] In some embodiments, the set of biomarker values 132 may be used to explore correlations between anatomical/structural and functional aspects of the eye, for example, to confirm the benefit of using the biomarker values to predict future GA growth rate. In one or more embodiments, correlations between values for structural features (e.g., GA lesion-related values, ELM loss, EZ loss, RPE loss, and/or ratio of EZ loss and RPE loss), and acuity measurements (e.g., best corrected visual acuity (BCVA), low luminance visual acuity (LLVA), and/or low luminance deficit (LLD)) may be analyzed. In other embodiments, correlations between values and microperimetry measurements (e.g., scotomatous point(s), macular sensitivity, perilesional sensitivity, and/or response sensitivity) may be analyzed.

[0080] In some embodiments, the prediction system 122 may quantify an extent of outer retina disruption (ORD) and compare the results to manually graded GA areas of a FAF to identify correlations between retinal loss shown in OCT imaging to GA growth rates determined in FAF images. In some embodiments, biomarker values 132 such as for example retinal layer loss rates, strongly correlate with FAF-defined GA growth rates. As such, the prediction system 122 may convert information and/or data associated with OCT imaging into information relevant to a fundus autofluorescence (FAF) imaging.

[0081] In some embodiments, and due to the set of biomarker values 132 correlating with FAF-defined GA growth rates, the set of biomarker values 132 serves as a surrogate clinical endpoint in GA studies. For example, the set of biomarker values 132 may include RPE loss and ELM loss, both of which strongly correlate with FAF-defined GA area, both cross-sectionally and longitudinally. As such, RPE loss and ELM loss may serve as potential clinical endpoints in GA studies.

[0082] Prediction system 122 may send and/or provide (e.g., transmit) at least one of map output 130 (e.g., a portion of or all of map output 130) or set of biomarker values 132 (e.g., a portion of or all of set of biomarker values 132) to prediction model 134.

[0083] Prediction model 134 generates prediction output 136 based on map output 130 (e.g., thickness map 302 and/or intensity projection map 304) and/or set of biomarker values 132. Prediction output 136 may include data and/or information associated with predictions of how a GA lesion will progress over a duration of time. For example, prediction output 136 may indicate the growth of a GA lesion with respect to a future point in time (e.g., timepoint TN). AS previously discussed, prediction output 136 may include predicted growth image 138, set of predicted progression features 140, or both. Prediction output 136 strongly correlates with FAF-derived features— for example, the correlation coefficients of features included in prediction output 136 may be over 0.70, over 0.75, over 0.80, over 0.85, and/or over 0.90 (e.g., different coefficients for different features).

[0084] Predicted growth image 138, which may be a 2D or 3D image, may include, for example, a mask 314 and OCT image background 316. OCT image background 316 may be the image layer under mask 314. OCT image background 316 may be, for example, OCT volume 114, one of OCT B-scans 115, a processed version of OCT volume 114, a processed version of one of OCT B-scans 115, or some other representation of OCT volume 114 or OCT B-scans 115. Mask 314 identifies the portion of the OCT image that corresponds to a predicted growth for the geographic atrophy (GA) lesion(s) (which may be continuous or discontinuous) for a future point in time. For example, mask 314 may be an overlay over OCT imaging data 110 that identifies the area that is predicted to be affected by geographic atrophy at the future point in time (e.g., some number of days, weeks, months, or years, after the reference point in time). In one or more embodiments, mask 314 takes the form of an outline or border that is graphically overlaid over OCT imaging data 110 to identify the predicted region of growth for the GA lesion(s). [0085] In one or more embodiments, mask 314 identifies the entire area of OCT imaging data 110 that is predicted to be affected by one or more GA lesions at the future point in time. In other embodiments, mask 314 identifies the difference between the entire area of OCT imaging data 110 that is predicted to be affected by GA lesion(s) at the future point in time and the area of OCT imaging data 110 affected by GA lesion(s) at the reference point in time. In this manner, the region of growth predicted by prediction model 134 may be the new growth between the reference point in time and the future point in time.

[0086] Set of predicted progression features 140 may include, for example, a predicted growth rate, a future lesion area, or some other type of metric. Other features that indicate GA lesion progression and that can be included in set of predicted progression features 140 include, for example, EZ loss area, the ratio of RPE loss and EZ loss, count of RPE lesions, distance of RPE lesion to fovea, average RPE thickness, average EZ+IZ thickness, HRF volume, SDD volume, Drusen volume, and other such predictions based on retinal elements, at a future point in time.

[0087] In some embodiments, imaging data associated with, for example, OCT volumes, segmentation volumes, map outputs, predicted growth images, and so on may refer to numerical values that represent pixel values (e.g., pixel intensities) of an image, and thus imaging data may include raw and/or processed data used to create corresponding image (e.g., visual representation) shown on a display for a user to view and/or manipulate, such as display system 106 of system 101. In some embodiments, showing the images on display system 106 may include providing annotations, such as highlights, comments, numerical values or measurements, markings, overlaying of images, any combinations thereof, and so on.

[0088] In one or more embodiments, prediction model 134 includes a deep learning model, such as deep learning model 310. In other embodiments, prediction model 134 includes a statistical model, such as statistical model 312.

[0089] Deep learning model 310 may be implemented in any of a number of different ways. In one or more embodiments, deep learning model 310 is implemented using a convolutional neural network (CNN) system that includes one or more neural networks. At least one of these one or more neural networks may itself be a convolutional neural network. In some cases, deep learning model 310 includes multiple subsystems and/or layers, each including one or more neural networks.

[0090] In various embodiments, in order to combine information from both maps for each retinal element, deep learning model 310 may include a plurality of fusion CNNs, also referred to as "fuse CNNs".

[0091] As one example, a fuse CNN may be used to combine thickness map and intensity projection map generated for a 3D segment representing the ELM+MZ of the retina. A second fuse CNN may be used to combine the thickness map and intensity projection generated for the 3D segment representing the EZ+IZ of the retina. A third fuse CNN may be used to combine the thickness map and intensity projection generated for the 3D segment representing the RPE of the retina. A fourth fuse CNN may be used to combine the information combined in each of the first three fuse CNNs. In some embodiments, the combined information from the final fuse CNN may be used as input for a residual neural network (ResNet) backbone to extract features at different resolutions, followed by at least two ID CNN layers each of which also includes a maximum pooling layer.

[0092] Statistical model 312 may receive set of biomarker values 132 as input. Such inputs may include, for example without limitation, loss area for various retinal elements, choroidal hypertransmission area (e.g., a layer with certain thickness below BM), the ratio between different area losses of two or more retinal elements, average thickness of all retinal elements, total retinal thickness, count of lesions for all retinal elements, length of lesions reaching imaging border for all retinal elements, average intensity of all retinal elements, average intensity of retinal element corresponding to choroidal hypertransmission, and distance of a lesion to the fovea of the retina.

[0093] Prediction model 134 may be trained using prediction training data 318. Prediction training data 318 may include example inputs that correlate to example outputs. For example, prediction training data 318 may include example map outputs (e.g., thickness maps and/or intensity projection maps) that are correlated to example prediction outputs (e.g., prediction growth images and/or set of predicted progression features). Prediction training data 318 may be retrieved from a memory and/or database (e.g., data storage 104) and/or received from a user input. Similar to segmentation model 120, prediction model 134 may be iteratively trained and updated.

[0094] As previously mentioned, models described herein may each include a neural network and/or machine learning model. In some embodiments, the neural network may be implemented by computing platform 102. In one or more embodiments, the neural network may include various nodes (e.g., neurons) arranged in multiple layers including an input layer receiving one or more inputs, hidden layers, and an output layer providing one or more outputs. The input(s) may collectively provide a training dataset for use in training the neural network. As understood by one of ordinary skill in the art, any desired number of such features may be provided in various embodiments. The training dataset may include any training datasets discussed within this disclosure (e.g., segmentation training dataset, prediction training dataset, and the like).

[0095] In various embodiments, the neural network may be trained on large amounts of data and may be iteratively trained using updated training datasets (e.g., an updated first training dataset and/or updated second training dataset) until the neural network has trained on enough data such that the neural network can perform predictions, or more accurate predictions, of its own.

III. Example Methodologies for Performing Segmentation and Predicting Geographic Atrophy (GA) Progression

[0096] FIG. 4 is a flow chart illustrating an embodiment of a process for predicting GA growth in accordance with one or more embodiments. In one or more embodiments, process 400 may be implemented using image analysis system 101 described in FIG. 1. One or more steps that are not expressly illustrated in FIG. 4 may be included before, after, in between, or as part of the steps of process 400. In some embodiments, process 400 may begin with step 402. For explanatory purposes, process 400 is primarily described within this disclosure with reference to system 101 and its associated arrangement of components as described in FIGS. 1-3. However, process 400 is not limited to such implementations. Any step, sub-step, sub-process, or block of process 400 may be performed in an order or arrangement different from the embodiments

T1 illustrated in Figure 4; some may be omitted, others may be added, and some may be performed simultaneously as appropriate.

[0097] As illustrated in FIG. 4, the process 400 includes, at step 402, receiving an optical coherence tomography (OCT) volume for a retina of a subject associated with a first point in time. The OCT volume may be, for example, OCT volume 114 in FIG. 1. The plurality of OCT B-scans may be, for example, plurality of OCT B-scans 115 in FIG. 1. In some embodiments, the first point in time is the time at which the OCT imaging data 110 was captured. The retina may be a healthy retina. Alternatively, the retina is one that has been diagnosed with or is suspected of having a retinal disease. For example, the diagnosis may be one of age-related macular degeneration (AMD), neovascular age-related macular degeneration (nAMD), geographic atrophy, or some other type of retinal disease.

[0098] The process 400 further includes, at step 404, generating, via a segmentation model, a segmented volume corresponding to the OCT volume, the segmented volume comprising a plurality of 3D segments. The segmentation model may be, for example, segmentation model 120 in FIG. 1. The segmented volume may be, for example, segmented volume 126 in FIG. 1. The plurality of 3D segments may be plurality of 3D segments 128 in FIG. 1. At least one 3D segment of the plurality of 3D segments represents a retinal element that comprises two or more elements. Each element of the two or more elements is either a layer element or a pathological element. As previously discussed herein, generating a segmented volume and its 3D segments includes generating a plurality of segmented 2D segments.

[0099] FIG. 5 shows an example 2D segment 500 and corresponding model output 505 or segmented 2D segment, in accordance with one or more embodiments of the present disclosure. In this example model output 505, the Drusen class includes BM when drusen exists or RPE is absent.

[0100] Referring back to the FIG. 4, the step 406 of process 400 includes generating, for each 3D segment of the plurality of 3D segments, a map output. In some embodiments, a prediction system such as the prediction system 122 of FIG. 1 generates the plurality of 3D segments. The map output may be, for example, map output 130 in FIG. 1. The map output may include, for example, a thickness map (e.g., thickness map 302 in FIG. 3), an intensity projection map (e.g., intensity projection map 304 in FIG. 3), or both, for each segment of the plurality of 3D segments. [0101] Step 408 of process 400 includes computing a set of biomarker values for a corresponding set of biomarker features associated with the retina using the map output. The set of biomarker values may be, for example, set of biomarker values 132 in FIG. 1. In some embodiments, a prediction system such as the prediction system 122 of FIG. 1 computes the set of biomarker values.

[0102] Step 410 of process 400 includes generating, via a prediction system, a prediction output that predicts a progression of a geographic atrophy (GA) lesion in the retina with respect to a future point in time after the first point in time using at least one of the set of biomarker values or the map output. The prediction output may be, for example, prediction output 136 in FIG. 1 and/or FIG. 3. The prediction system may be, for example, prediction system 122. The prediction system may include a prediction model such as prediction model 134 in FIG. 1 and/or FIG. 3. The prediction model may include, for example, a deep learning model, a statistical model, or both. In some embodiments, the prediction system 122 receives and uses the clinical data 142 associated with the subject at the step 410.

[0103] FIG. 6 is a diagram 600 illustrating generating layer thickness maps (e.g., thickness map 302 of FIG. 3 and/or thickness layers maps 602A-C) in accordance with one or more embodiments of the present disclosure. As illustrated, an OCT volume 604 is used to generate a segmented volume 606, which may be the segmented volume 126 of FIG. 3. As illustrated, the segmented volume 606 includes a plurality of 3D segments such as for example 3D segment 608A comprising ELM and MZ, 3D segment 608B comprising EZ and IZ, and 3D segment 608C comprising RPE. These 3D segments 608A-608C are used by the prediction system 122 to generate the corresponding thickness layer maps 602A-602C. In one or more embodiments, layer thickness maps 602A-C may be used to extract numerical features (e.g., set of biomarker values 132 associated with biomarker features). Set of biomarker values may include, for example, values for various features computed using thickness maps, such as loss area (e.g., area where the thickness is 0 and/or below a certain threshold), average thickness (e.g., valid average thickness (average thickness outside of legion) and/or total average thickness (average thickness including regions covered by lesion), count of GA lesions (e.g., defined by the number of connected components with an area larger than a predetermined threshold), length of one or more lesions reaching an imaging border (e.g., the total length of one or more edges of the lesion abutting the imaging border to provide descriptors of the extent the lesion goes beyond the imaging field of view), and/or the like.

[0104] FIG. 7 is a diagram 700 illustrating generating an intensity projection map (e.g., a layer mean intensity map (MIP) in accordance with one or more embodiments of the present disclosure. As previously discussed herein, one or more intensity projection maps (e.g., intensity projection map 304 of FIG. 3 and/or intensity projection maps 702A-C) may be generated based on a segmented volume (e.g., 3D segment 126 of FIG. 3) and/or the OCT imaging data 110. As illustrated, the OCT volume 604 is used to generate the segmented volume 606. In some embodiments, the segmented volume 606 may be used as a mask over the OCT volume 604 to identify the elements of the OCT volume 604 for which the intensity is being quantified. As illustrated, together the OCT volume 604 and the segmented volume 606 are used to create 3D segment 704A comprising ELM and MZ, 3D segment 704B comprising EZ and IZ, and 3D segment 704C comprising RPE. These 3D segments 704A-704C are used by the prediction system 122 to generate the corresponding layer mean intensity projection maps 702A, 702B, and 702C. Each intensity projection map 702A-702C may be a map that provides information about the layer mean intensity for each of the 3D segments 704A-704C. For example, each of the intensity projection maps 702A-C may be, for example, a 2D map in which the value at any x,y position on the 2D map indicates the mean intensity of the pixels in the z direction for the corresponding x,y position of each of the 3D segments 704A-704C, respectively. In various embodiments, the intensity projection map may be used to extract numerical features (e.g., set of biomarker values 132). Such features may include, but are not limited to, an average intensity, such as a valid average intensity (e.g., average intensity outside of the legion) and/or a total average intensity (e.g., average intensity that includes the regions covered by the lesion), and so on.

[0105] FIG. 8 is a diagram 800 illustrating generating a prediction output, such as the prediction output 136 of FIG. 1, using a deep learning model, such as the deep learning model 310 of FIG. 1. In the example diagram 800, convolutional neural networks are used to determine a GA growth rate in accordance with one or more embodiments of the present disclosure. As previously discussed herein, and in several embodiments, a first fuse CNN 808 may be used to combine thickness map 602A and intensity projection map 702B generated for a 3D segment representing the ELM+MZ of the retina. A second fuse CNN 810 may then be used to combine thickness map 604A and intensity projection map 704A generated for the 3D segment representing the EZ+IZ of the retina. A third fuse CNN 812 may be used to combine the thickness map 606A and intensity projection map 706B generated for the 3D segment representing the RPE of the retina. A fourth fuse CNN 814 may be used to combine the information combined in each of the first three fuse CNNs 808, 810, and 812. In some embodiments, the combined information from the final fuse CNN, such as for example the CNN 814, may be used as input for a final network structure 816 that includes a backbone (e.g., a ResNet backbone) to extract features at different resolutions, one or more pooling layers, one or more CNN layers, and one or more fully connected layers. For example, final network structure 816 may include a ResNet backbone, a max pooling layer (e.g., a max pooling layer for the slides), at least two ID CNN layers each of which also includes a maximum pooling layer, and a fully connected layer. The output of the final network structure 816 includes a biomarker prediction such as, for example, GA lesion growth rate over a period of time (e.g., one year, two years).

[0106] FIG. 9 is a diagram 900 illustrating the use of OCT-defined retinal layer loss rates as surrogates for a FAF-defined GA growth rates in accordance with one or more embodiments of the present disclosure. As previously discussed, an OCT volume (e.g., OCT volume 114 and/or OCT volume 902) may be used to generate a segmented volume (e.g., segmented volume 904). For a given 3D segment of the plurality of 3D segments in the segmented volume 904, map output 906 may include thickness maps such as for example a RPE thickness map 908, an ELM+MZ thickness map 910, and an EZ+IZ thickness map 912, intensity projection map (not illustrated in FIG. 10), or both. In some embodiments, the image analysis system 101 uses the thickness maps 906 and/or the intensity project map(s) to generate the set biomarker values 132, which may include for example RPE loss and ELM loss. As RPE loss and ELM loss strongly correlates to FAF- defined GA area, both cross-sectionally and longitudinally, the biomarker values 132 of RPE loss and ELM loss may serve as clinical endpoint in GA studies in some embodiments. FIG. 9 illustrates a comparison of the map output 906 compared to a FAF image 914, which generally is manually graded before being used as a clinical trial endpoint. As the image analysis system 101 provides automated biomarker values 132 that may be a surrogate for the manually-graded FAF image, the image analysis system 101 enables the transformation of OCT imaging data into corresponding values for biomarkers that are well-known in the medical industry (clinical / clinical trial) but that can be understood with respect to an entirely different imaging modality— FAF images.

[0107] In some embodiments, the example methodologies described herein provide a technical improvement to a technical field of GA prediction, detection, and/or treatment and/or a technical effect. For example, in some embodiments, the prediction outputs 136 described with respect to FIGS. 1-3 are used to identify subjects with a high risk of high GA progression over a set period of time, such as for example 1 year or 2 years. In some embodiments, a high risk of high GA progression is defined as a predicted % increase of GA lesion exceeding a % threshold; a predicted area of GA lesion exceeding a threshold area, and/or a predicted EZ loss related OCT feature parameter exceeding a parameter threshold. In other embodiments, a high risk of high GA progression is defined in other ways. In some embodiments, the image analysis system 101 identifies the subject as having a high risk of high GA progression over the set period of time. In some embodiments and when the subject is identified as having a high risk of high GA progression, the subject is treated with a treatment such as for example pegcetacoplan (Syfovre) and avacincaptad pegol (Izervay). However, in other embodiments, the image analysis system 101 can identify whether a subject has a high risk of high GA progression over a set period of time, which can aid in the design of clinical trials.

[0108] In some embodiments, the process 400 described with respect to FIG. 4 includes a new combination of steps that results in a technical improvement and/or technical effect over conventional systems and methods for predicting GA lesion growth at least in part due to the consistent, objective, and scalable analysis provided by the image analysis system 101 described with respect to FIGS. 1-3. That is, and in some embodiments, employing an automated analysis and prediction using the image analysis system 101 and/or the process 400 allows for consistent, objective and scalable analysis and prediction, overcoming the limitations of manual segmentation and grading, which is both time-consuming and subject to intergrader variability.

[0109] In some embodiments, the process 400 includes a new combination of steps that results in the technical improvement over conventional systems and methods of measuring clinical endpoints in GA studies. In some embodiments, with consistent and objective measurements (e.g., the biomarker values 132) outcomes of clinical trials may be more accurately measured. Using the biomarker values 132 as a surrogate for manually graded FAF images is not limited to the clinical trial setting. Instead, and in some embodiments, a portion of the process 400 and/or the image analysis system 101 may be used to generate biomarker values 132 used for in detection and/or diagnosis activities.

[0110] In some embodiments, the process 400 and/or the image analysis system 101 perform retinal segmentation in a way that identifies multiple segments, each of which represents a retinal element that itself is a combination of two or more elements (e.g., two or more retinal layer elements, two or more retinal pathological elements, or one or more retinal layer elements in combination with one or more retinal pathological elements). Using segmentation to identify combined elements in this way may lead to improved model performance and more accurate predictions about GA growth rate where the individual elements themselves may be more difficult to delineate in the imaging. For example, accurately segmenting an external limiting membrane and the myoid zone individually may be difficult with certain OCT images. However, segmenting the area that represents the combined external limiting membrane and myoid zone may be performed more accurately, which may lead to better performance overall and improved accuracy in predictions generated based on this segmentation. Further, segmentation via the process and/or using the image analysis system 101, which specifically identifies selected combinations of elements that are chosen for their prognostic characteristics, may reduce the overall computing resources, time, and expense that might be otherwise associated with segmenting out each individual element. IV. Example Methodologies of Training Portions of the Example Image Analysis System

[0111] FIG. 10 is a flow chart illustrating an embodiment of a process for training a segmentation model in accordance with one or more embodiments. In one or more embodiments, process 1000 may be implemented using image analysis system 101 described in FIG. 1 and its associated arrangement of components as described in FIG. 3. One or more steps that are not expressly illustrated in FIG. 10 may be included before, after, in between, or as part of the steps of process 1000. In some embodiments, process 1000 may begin with step 1002.

[0112] In some cases, step 1002 includes receiving a training dataset that includes a plurality of optical coherence tomography (OCT) volumes for a plurality of retinas. Each OCT volume of the plurality of OCT volumes includes a plurality of OCT-B scans (e.g., 2D slices of the 3D volume). The training dataset may include OCT images from multiple eyes at different disease stages of retinal diseases such as, for example, AMD and GA.

[0113] The process 1000 further includes, at step 1004, preprocessing the plurality of OCT volumes to form 3D training image input for the segmentation model. Preprocessing operations may include, but are not limited to, scaling, resizing, cropping, horizontal flipping, vertical flipping, adding and/or removing noise, translating, and other such preprocessing operations. In some embodiments, the training image input for the segmentation model is the segmentation training data 208. The segmentation model described in steps 1004 and 1006 may be, for example, segmentation model 120 described with respect to FIG. 1.

[0114] Step 1006 of process 1000 includes training the segmentation model to generate a segmented volume corresponding to the 3D training image input. The segmented volume may be, for example, segmented volume 126 in FIG. 1.

[0115] Other training processes may also be implemented. For example, a method for training a prediction model such as prediction model 134 in FIG. 1 may be implemented in a manner similar to the process 1000 described above. The training data may include different data such as, for example, training thickness maps, training intensity projection maps, training data that includes training values for various biomarker features, or a combination thereof. V. Example Experiments

V.A. Example GA Progression Prediction

[0116] In one example experiment, an example segmentation model, EyeNotate, was used to segment OCT volume scans into retinal layers and drusen. Then, eighteen numerical features including area and number of layer disruptions, average layer thickness, and length of area disruption reaching scan boundaries were derived. For example, the baseline OCT features included ONL loss area, ELM loss area, EZ loss area, border ELM loss, count of ELM lesion, border EZ loss, count of EZ lesion, border of RPE loss, count of RPE lesion, mean RNFL thickness, mean GCL+IPL thickness, mean INL+OPL thickness, mean HFL+ONL thickness, mean ELM+MZ thickness, mean EZ+IZ thickness, and mean RPE thickness. Ten non-OCT tabular features including demographics, functional measurements, and manual readings from non-OCT imaging modalities were also included. For example, the non-OCT tabular features included age, sex, smoking status, BCVA, low luminance deficit, presence of SDD, FAF GA area, FAF GA lesion contiguity, FAF GA to central fovea distance, and FAF GA fovea involvement. The endpoint of this example study was the GA lesion growth rate computed from the slope of a linear fit on all available measurements of GA lesion area within a 2-year time frame.

[0117] Baseline study eyes of patients with bilateral GA (NCT02247479; NCT02247631; and NCT02479386, n=1012) were split into training and validation sets. Patients with missing OCT data or clinical features were removed. Example Lasso and XGBoost models were trained to predict the annualized GA growth rate measured on FAF using 5x3 nested cross-validation (3 inner folds for hyperparameter tuning and 5 predefined outer folds for performance reporting). Feature importance was determined via partial dependent plots (PDP), defined as the ratio of variance of PDP per feature versus the variance of sum of all PDPs. In this example, models using OCT extracted features and clinical features were better able to predict GA progression over models using OCT extracted features and models using clinical features. EZ loss related OCT features, including baseline EZ loss area and ratio of RPE to EZ loss, are top risk factors for future GA growth rate. V.B. Example Correlation of OCT-segmented Measurements to FAF-defined GA area [0118] In one example experiment, an example segmentation model, EyeNotate used with DeepLabv3, was trained on 6,718 annotated OCT B-scans from participants involved in intermediate AMD and GA trials (NCT01790802 and NCT02399072; n=189). The example model was applied to 736 OCT volumes from study eyes of a separate GA study (NCT02479386; n=227, 1-7 visits per patient) to generate segmented volumes (e.g., a segmented volume for each of the 736 OCT volumes).

[0119] These segmented volumes were processed to generate map outputs such as, for example, map output 130 described with respect to FIGS. 1-3. These map outputs included thickness maps (e.g., enface layer thickness maps) and intensity projection maps.

[0120] The outer retinal disruption (ORD) (or loss) per volume (i.e., ELM loss, EZ loss, and RPE loss) was derived from the map outputs, including the en face layer thickness maps, using a system such as prediction system 122 described in FIGS. 1-3 and at least a portion of the process described in FIG. 4. Disruption or loss was identified where layer thickness was identified as zero. Specifically, loss area (in units of mm²) was computed where layer thickness was identified as 0. [0121] Correlations were assessed between OCT-defined retinal layer loss and manually graded FAF-defined GA area at the baseline visit. Correlations between annualized OCT-defined retinal layer loss rates and FAF-defined GA growth rates were calculated for 205 patients with at least two visits. Baseline GA area defined by FAF, and RPE loss, ELM loss, and EZ loss on OCT were assessed as 7.8+/- 3.9, 6.0+/-3.3, 8.5+/- 4.2, and 12.4+/4.2mm² respectively. EZ loss was significantly larger than both ELM and RPE loss (p<0.01). In this example, there was a strong correlation between OCT-defined retinal layer loss and FAF-defined GA area, with correlation coefficients of 0.91 for RPE loss, 0.92 for ELM loss, and 0.76 for EZ loss (all p<0.01). In this example and when evaluating the annualized GA lesion growth rate as defined by OCT and FAF, the correlation remained significant: 0.77 for RPE loss rate, 0.78 for ELM loss rate, and 0.46 for EZ loss rate (all p<0.01). In this example, 91% of regions that developed new RPE loss within a year had EZ loss at baseline. [0122] Accordingly, measurements of RPE and ELM loss on OCT may strongly correlate with FAF-defined GA area, both cross-sectionally and longitudinally. Hence, these OCT layers may server as a potential clinical endpoint in GA studies.

VI. Computer Implemented System

[0123] FIG. 11 is a block diagram of a computer system in accordance with various embodiments. Computer system 1100 may be an example of one implementation for computing platform 102 described above in FIG. 1.

[0124] In one or more examples, computer system 1100 can include a bus 1102 or other communication mechanism for communicating information, and a processor 1104 coupled with bus 1102 for processing information. In various embodiments, computer system 1100 can also include a memory, which can be a random-access memory (RAM) 1106 or other dynamic storage device, coupled to bus 1102 for determining instructions to be executed by processor 1104. Memory also can be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 1104. In various embodiments, computer system 1100 can further include a read only memory (ROM) 1108 or other static storage device coupled to bus 1102 for storing static information and instructions for processor 1104. A storage device 1110, such as a magnetic disk or optical disk, can be provided and coupled to bus 1102 for storing information and instructions.

[0125] In various embodiments, computer system 1100 can be coupled via bus 1102 to a display 1112, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 1114, including alphanumeric and other keys, can be coupled to bus 1102 for communicating information and command selections to processor 1104. Another type of user input device is a cursor control, such as a mouse, a joystick, a trackball, a gesture input device, a gaze-based input device, or cursor direction keys for communicating direction information and command selections to processor 1104 and for controlling cursor movement on display 1112. This input device 1116 typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane. However, it should be understood that input devices 1114 allowing for three-dimensional (e.g., x, y, and z) cursor movement are also contemplated herein.

[0126] Consistent with certain implementations of the present teachings, results can be provided by computer system 1100 in response to processor 1104 executing one or more sequences of one or more instructions contained in RAM 1106. Such instructions can be read into RAM 1106 from another computer-readable medium or computer-readable storage medium, such as storage device 1110. Execution of the sequences of instructions contained in RAM 1106 can cause processor 1104 to perform the processes described herein. Alternatively, hard-wired circuitry can be used in place of or in combination with software instructions to implement the present teachings. Thus, implementations of the present teachings are not limited to any specific combination of hardware circuitry and software.

[0127] The term "computer-readable medium" (e.g., data store, data storage, storage device, data storage device, etc.) or "computer-readable storage medium" as used herein refers to any media that participates in providing instructions to processor 1104 for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Examples of non-volatile media can include, but are not limited to, optical, solid state, magnetic disks, such as storage device 1110. Examples of volatile media can include, but are not limited to, dynamic memory, such as RAM 1106. Examples of transmission media can include, but are not limited to, coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 1102.

[0128] Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other tangible medium from which a computer can read.

[0129] In addition to computer readable medium, instructions or data can be provided as signals on transmission media included in a communications apparatus or system to provide sequences of one or more instructions to processor 1104 of computer system 1100 for execution. For example, a communication apparatus may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the disclosure herein. Representative examples of data communications transmission connections can include, but are not limited to, telephone modem connections, wide area networks (WAN), local area networks (LAN), infrared data connections, NFC connections, optical communications connections, etc.

[0130] It should be appreciated that the methodologies described herein, flow charts, diagrams, and accompanying disclosure can be implemented using computer system 1100 as a standalone device or on a distributed network of shared computer processing resources such as a cloud computing network.

[0131] The methodologies described herein may be implemented by various means depending upon the application. For example, these methodologies may be implemented in hardware, firmware, software, or any combination thereof. For a hardware implementation, the processing unit may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or a combination thereof.

[0132] In various embodiments, the methods of the present teachings may be implemented as firmware and/or a software program and applications written in conventional programming languages such as C, C++, Python, etc. If implemented as firmware and/or software, the embodiments described herein can be implemented on a non-transitory computer-readable medium in which a program is stored for causing a computer to perform the methods described above. It should be understood that the various engines described herein can be provided on a computer system, such as computer system 1100, whereby processor 1104 would execute the analyses and determinations provided by these engines, subject to instructions provided by any one of, or a combination of, the memory components RAM 1106, ROM, 1108, or storage device 1110 and user input provided via input device 1114. VII. Example Definitions and Context

[0133] The disclosure is not limited to these exemplary embodiments and applications or to the manner in which the exemplary embodiments and applications operate or are described herein. Moreover, the figures may show simplified or partial views, and the dimensions of elements in the figures may be exaggerated or otherwise not in proportion.

[0134] Unless otherwise defined, scientific and technical terms used in connection with the present teachings described herein shall have the meanings that are commonly understood by those of ordinary skill in the art. Further, unless otherwise required by context, singular terms shall include pluralities and plural terms shall include the singular. Generally, nomenclatures utilized in connection with, and techniques of, chemistry, biochemistry, molecular biology, pharmacology, and toxicology are described herein are those well-known and commonly used in the art.

[0135] As the terms "on," "attached to," "connected to," "coupled to," or similar words are used herein, one element (e.g., a component, a material, a layer, a substrate, etc.) can be "on," "attached to," "connected to," or "coupled to" another element regardless of whether the one element is directly on, attached to, connected to, or coupled to the other element or there are one or more intervening elements between the one element and the other element. In addition, where reference is made to a list of elements (e.g., elements a, b, c), such reference is intended to include any one of the listed elements by itself, any combination of less than all of the listed elements, and/or a combination of all of the listed elements. Section divisions in the specification are for ease of review only and do not limit any combination of elements discussed.

[0136] The term "subject" may refer to a subject of a clinical trial, a person undergoing treatment, a person undergoing anti-cancer therapies, a person being monitored for remission or recovery, a person undergoing a preventative health analysis (e.g., due to their medical history), or any other person or patient of interest. In various cases, "subject" and "patient" may be used interchangeably herein.

[0137] As used herein, "substantially" means sufficient to work for the intended purpose. The term "substantially" thus allows for minor, insignificant variations from an absolute or perfect state, dimension, measurement, result, or the like such as would be expected by a person of ordinary skill in the field but that do not appreciably affect overall performance. When used with respect to numerical values or parameters or characteristics that can be expressed as numerical values, "substantially" means within ten percent.

[0138] The term "ones" means more than one.

[0139] As used herein, the term "plurality" may be 2, 3, 4, 5, 6, 7, 8, 9, 10, or more.

[0140] As used herein, the term "set of" means one or more. For example, a set of items includes one or more items.

[0141] As used herein, the phrase "at least one of," when used with a list of items, means different combinations of one or more of the listed items may be used and only one of the items in the list may be needed. The item may be a particular object, thing, step, operation, process, or category. In other words, "at least one of" means any combination of items or number of items may be used from the list, but not all of the items in the list may be required. For example, without limitation, "at least one of item A, item B, or item C" means item A; item A and item B; item B; item A, item B, and item C; item B and item C; or item A and C. In some cases, "at least one of item A, item B, or item C" means, but is not limited to, two of item A, one of item B, and ten of item C; four of item B and seven of item C; or some other suitable combination.

[0142] As used herein, a "model" can refer to a system, process, relationship, or set of rules, which is instantiated, stored, and/or executed within a computing environment. The model may include, without limitation, algorithms, parameters, data structures, or executable instructions configured to perform one or more functions when processed by one or more computing devices. [0143] As used herein, "machine learning" may be the practice of using algorithms to parse data, learn from it, and then make a determination or prediction about something in the world. Machine learning uses algorithms that can learn from data without relying on rules-based programming.

[0144] As used herein, an "artificial neural network" or "neural network" (NN) may refer to computational models that mimic an interconnected group of artificial nodes or neurons that processes information based on a connectionist approach to computation. Neural networks, which may also be referred to as neural nets, can employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, e.g., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters. In the various embodiments, a reference to a "neural network" may be a reference to one or more neural networks.

[0145] A neural network may compute digital data in two ways: when it is being trained it is in training mode and when it puts what it has learned into practice it is in inference (or prediction) mode. Neural networks learn through a feedback process (e.g., backpropagation) which allows the network to adjust the weight factors (modifying its behavior) of the individual nodes in the intermediate hidden layers so that the output matches the outputs of the training data. In other words, a neural network learns by being fed training data (learning examples) and eventually learns how to reach the correct output, even when it is presented with a new range or set of inputs. A neural network may include, for example, without limitation, at least one of a Feedforward Neural Network (FNN), a Recurrent Neural Network (RNN), a Modular Neural Network (MNN), a Convolutional Neural Network (CNN), a Residual Neural Network (ResNet), an Ordinary Differential Equations Neural Networks (neural-ODE), or another type of neural network.

[0146] As used herein, a "lesion" may be a region in an organ or tissue that has suffered damage via injury or disease. This region may be a continuous or discontinuous region. For example, as used herein, a lesion may include multiple regions. A geographic atrophy (GA) lesion is a region of the retina that has suffered chronic progressive degeneration. As used herein, a GA lesion may include one lesion (e.g., one continuous lesion region) or multiple lesions (e.g., discontinuous lesion region comprised of multiple, separate lesions).

[0147] As used herein, a "lesion area" may be the total area covered by a lesion, whether that lesion be a continuous region or a discontinuous region.

[0148] As used herein, "longitudinal" may refer to over a period of time. The period of time may be in days, weeks, months, years, or some other measure of time.

[0149] As used herein, a "growth rate" corresponding to a GA lesion may be a longitudinal change in the lesion area of the GA lesion. In other words, the "growth rate" may be a change in lesion area over time. In some cases, this growth rate may be an annualized growth rate. This growth rate may also be referred to as a lesion growth rate or a GA growth rate.

VIII. Recitation of Embodiments

[0150] Embodiment 1: A computer-based method comprising: receiving, by a processor over a network, an optical coherence tomography (OCT) volume for a retina of a subject associated with a first point in time; generating, via a segmentation model implemented using the processor, a segmented volume corresponding to the OCT volume, the segmented volume comprising a plurality of 3D segments, wherein at least one 3D segment of the plurality of 3D segments represents a retinal element that comprises two or more elements, wherein each element of the two or more elements is either a layer element or a pathological element; generating, by the processor, for each 3D segment of the plurality of 3D segments, a map output; computing, by the processor, a set of biomarker values for a corresponding set of biomarker features associated with the retina using the map output; and generating, via a prediction system implemented using the processor, a prediction output that predicts a progression of a geographic atrophy (GA) lesion in the retina with respect to a future point in time after the first point in time using at least one of the set of biomarker values or the map output.

[0151] Embodiment 2: The method of embodiment 1, the OCT volume is received over the network from an OCT imaging system via a network interface and wherein generating, via the segmentation model implemented using the processor, the segmented volume comprises: preprocessing the OCT volume to form a 3D image input for the segmentation model; and generating, via the segmentation model, the segmented volume corresponding to the OCT volume based on the 3D image input.

[0152] Embodiment 3: The method of embodiment 1 or embodiment 2, wherein the map output includes at least one of a thickness map or an intensity projection map for a 3D segment of the plurality of 3D segments.

[0153] Embodiment 4: The method of any one of embodiments 1-3, wherein the map output includes at least one of a thickness map or an intensity projection map that corresponds to a retinal element or a combination of retinal elements represented by the plurality of 3D segments. [0154] Embodiment 5: The method of embodiment 4, wherein the set of biomarker values is computed through postprocessing of the intensity projection map that corresponds to a retinal element or a combination of retinal elements represented by the plurality of 3D segments.

[0155] Embodiment 6: The method of any one of embodiments 1-5, wherein the prediction system comprises at least one of a deep learning model or a statistical model.

[0156] Embodiment 7: The method of any one of embodiments 1-6, wherein the prediction output comprises a predicted growth image that indicates a growth of the GA lesion between the first point in time and the future point in time.

[0157] Embodiment 8: The method of any one of embodiments 1-7, wherein the prediction output includes a predicted growth rate for the GA lesion and wherein the prediction output includes at least one feature that strongly correlates with a corresponding feature that would be derived manually from a fundus autofluorescence (FAF) image of the retina of the subject.

[0158] Embodiment 9: The method of any one of embodiments 1-8, wherein the set of biomarker features includes at least one of loss area, average thickness, a count of GA lesions, a length of lesion reaching border, a shape-based feature, a morphology-based feature.

[0159] Embodiment 10: The method of any one of embodiments 1-9, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes an inner limiting membrane (IML) and a retinal nerve fiber layer (RNFL).

[0160] Embodiment 11: The method of any one of embodiments 1-10, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes a ganglion cell layer (GCL) and an inner plexiform layer (IPL)

[0161] Embodiment 12: The method of any one of embodiments 1-11, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes an inner nuclear layer (INL) and an outer plexiform layer (OPL).

[0162] Embodiment 13: The method of any one of embodiments 1-12, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes an external limiting membrane (EML) and an myoid zone (MZ). [0163] Embodiment 14: The method of any one of embodiments 1-13, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes a Henle fiber layer (HFK) and an outer nuclear layer (ONL).

[0164] Embodiment 15: The method of any one of embodiments 1-14, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes ellipsoid zone (EZ) and an interdigitation zone (IZ).

[0165] Embodiment 16: The method of any one of embodiments 1-15, wherein the retinal element represented by a 3D segment of the plurality of 3D segments is a Drusen+ class that includes Drusen and Bruch's membrane when Drusen exists and includes Bruch's membrane when RPE is absent.

[0166] Embodiment 17: The method of any one of embodiments 1-16, wherein generating, via the prediction system implemented using the processor, the prediction output comprises: inputting a thickness map and an intensity projection map for a first retinal element that includes an external limiting membrane (ELM) and myoid zone (MZ) into a first fuse convolutional neural network; inputting a thickness map and an intensity projection map for a second retinal element that includes an ellipsoid zone (EZ) and interdigitation zone (IZ) into a second fuse convolutional neural network; inputting a thickness map and an intensity projection map for a third retinal element that includes a retinal pigment epithelium layer into a third fuse convolutional neural network; and inputting outputs from the first, second, and third convolutional neural networks into a fourth fuse convolutional neural network.

[0167] Embodiment 18: The method of any one of embodiments 1-17, wherein the segmentation model includes a semantic segmentation model.

[0168] Embodiment 19: The method of any one of embodiments 1-18, wherein the plurality of 3D segments includes a first retinal element that includes two or more elements and a second retinal element that includes two or more elements.

[0169] Embodiment 20: The method of any one of embodiments 1-19, wherein the plurality of 3D segments includes at least three retinal elements, each of the at least three retinal elements including two or more elements. [0170] Embodiment 21: The method of any one of embodiments 1-20, further comprising generating, by the processor, a set of correlations between the computed set of biomarker values and a set of retinal measurements.

[0171] Embodiment 22: A system comprising: one or more data processors; and a non- transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform the method of any one of embodiments 1-21.

[0172] Embodiment 23: A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform the method of any one of embodiments 1-21.

IX. Additional Considerations

[0173] Any headers and/or subheaders between sections and subsections of this document are included solely for the purpose of improving readability and do not imply that features cannot be combined across sections and subsection. Accordingly, sections and subsections do not describe separate embodiments.

[0174] While the present teachings are described in conjunction with various embodiments, it is not intended that the present teachings be limited to such embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be appreciated by those of skill in the art. The present description provides preferred exemplary embodiments, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the present description of the preferred exemplary embodiments will provide those skilled in the art with an enabling description for implementing various embodiments. It is understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims. Thus, such modifications and variations are considered to be within the scope set forth in the appended claims. Further, the terms and expressions which have been employed are used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention claimed.

[0175] In describing the various embodiments, the specification may have presented a method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the various embodiments.

[0176] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform part or all of one or more methods and/or part or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer-program product tangibly embodied in a non-transitory machine- readable storage medium, including instructions configured to cause one or more data processors to perform part or all of one or more methods and/or part or all of one or more processes disclosed herein.

[0177] Specific details are given in the present description to provide an understanding of the embodiments. However, it is understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

Claims

1. A computer-based method comprising: receiving, by a processor over a network, an optical coherence tomography (OCT) volume for a retina of a subject associated with a first point in time; generating, via a segmentation model implemented using the processor, a segmented volume corresponding to the OCT volume, the segmented volume comprising a plurality of 3D segments, wherein at least one 3D segment of the plurality of 3D segments represents a retinal element that comprises two or more elements, wherein each element of the two or more elements is either a layer element or a pathological element; generating, by the processor, for each 3D segment of the plurality of 3D segments, a map output; computing, by the processor, a set of biomarker values for a corresponding set of biomarker features associated with the retina using the map output; and generating, via a prediction system implemented using the processor, a prediction output that predicts a progression of a geographic atrophy (GA) lesion in the retina with respect to a future point in time after the first point in time using at least one of the set of biomarker values or the map output.

2. The method of claim 1, wherein the OCT volume is received over the network from an OCT imaging system via a network interface and wherein generating, via the segmentation model implemented using the processor, the segmented volume comprises: preprocessing the OCT volume to form a 3D image input for the segmentation model; and generating, via the segmentation model, the segmented volume corresponding to the OCT volume based on the 3D image input.

3. The method of claim 1 or claim 2, wherein the map output includes at least one of a thickness map or an intensity projection map for a 3D segment of the plurality of 3D segments.

4. The method of any one of claims 1-3, wherein the map output includes at least one of a thickness map or an intensity projection map that corresponds to a retinal element or a combination of retinal elements represented by the plurality of 3D segments.

5. The method of claim 4, wherein the set of biomarker values is computed through postprocessing of the intensity projection map that corresponds to a retinal element or a combination of retinal elements represented by the plurality of 3D segments.

6. The method of any one of claims 1-5, wherein the prediction system comprises at least one of a deep learning model or a statistical model.

7. The method of any one of claims 1-6, wherein the prediction output comprises a predicted growth image that indicates a growth of the GA lesion between the first point in time and the future point in time.

8. The method of any one of claims 1-7, wherein the prediction output includes a predicted growth rate for the GA lesion and wherein the prediction output includes at least one feature that strongly correlates with a corresponding feature that would be derived manually from a fundus autofluorescence (FAF) image of the retina of the subject.

9. The method of any one of claims 1-8, wherein the set of biomarker features includes at least one of loss area, average thickness, a count of GA lesions, a length of lesion reaching border, a shape-based feature, a morphology-based feature.

10. The method of any one of claims 1-9, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes an inner limiting membrane (IML) and a retinal nerve fiber layer (RNFL).

11. The method of any one of claims 1-10, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes a ganglion cell layer (GCL) and an inner plexiform layer (IPL).

12. The method of any one of claims 1-11, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes an inner nuclear layer (INL) and an outer plexiform layer (OPL).

13. The method of any one of claims 1-12, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes an external limiting membrane (EML) and an myoid zone (MZ).

14. The method of any one of claims 1-13, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes a Henle fiber layer (HFK) and an outer nuclear layer (ONL).

15. The method of any one of claims 1-14, wherein the retinal element represented by a 3D segment of the plurality of 3D segments includes ellipsoid zone (EZ) and an interdigitation zone (IZ).

16. The method of any one of claims 1-5, wherein the retinal element represented by a 3D segment of the plurality of 3D segments is a Drusen+ class that includes Drusen and Bruch's membrane when Drusen exists and includes Bruch's membrane when RPE is absent.

17. The method of any one of claims 1-16, wherein generating, via the prediction system implemented using the processor, the prediction output comprises: inputting a thickness map and an intensity projection map for a first retinal element that includes an external limiting membrane (ELM) and myoid zone (MZ) into a first fuse convolutional neural network; inputting a thickness map and an intensity projection map for a second retinal element that includes an ellipsoid zone (EZ) and interdigitation zone (IZ) into a second fuse convolutional neural network; inputting a thickness map and an intensity projection map for a third retinal element that includes a retinal pigment epithelium layer into a third fuse convolutional neural network; and inputting outputs from the first, second, and third convolutional neural networks into a fourth fuse convolutional neural network.

18. The method of any one of claims 1-17, wherein the segmentation model includes a semantic segmentation model.

19. The method of any one of claims 1-18, wherein the plurality of 3D segments includes a first retinal element that includes two or more elements and a second retinal element that includes two or more elements.

20. The method of any one of claims 1-19, wherein the plurality of 3D segments includes at least three retinal elements, each of the at least three retinal elements including two or more elements.

21. The method of any one of claims 1-20, further comprising generating, by the processor, a set of correlations between the computed set of biomarker values and a set of retinal measurements.

22. A system comprising: one or more data processors; and a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform the method of any one of claims 1-21.

23. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform the method of any one of claims 1-21.