WO2022266651A1 - Precision phenotyping of left ventricular hypertrophy with echocardiographic deep learning - Google Patents

Precision phenotyping of left ventricular hypertrophy with echocardiographic deep learning Download PDF

Info

Publication number
WO2022266651A1
WO2022266651A1 PCT/US2022/072977 US2022072977W WO2022266651A1 WO 2022266651 A1 WO2022266651 A1 WO 2022266651A1 US 2022072977 W US2022072977 W US 2022072977W WO 2022266651 A1 WO2022266651 A1 WO 2022266651A1
Authority
WO
WIPO (PCT)
Prior art keywords
echocardiogram
neural network
network model
cardiac
model
Prior art date
Application number
PCT/US2022/072977
Other languages
French (fr)
Inventor
David Ouyang
Susan Cheng
Grant Duffy
Bryan He
Original Assignee
Cedars-Sinai Medical Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cedars-Sinai Medical Center filed Critical Cedars-Sinai Medical Center
Publication of WO2022266651A1 publication Critical patent/WO2022266651A1/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
    • A61B8/0883—Detecting organic movements or changes, e.g. tumours, cysts, swellings for diagnosis of the heart
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5207—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of raw data to produce diagnostic data, e.g. for generating an image
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5223—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for extracting a diagnostic or physiological parameter from medical diagnostic data

Definitions

  • LSH Left ventricular hypertrophy
  • LVH results from chronic remodeling caused by a broad range of systemic and cardiovascular disease conditions including hypertension, aortic stenosis, hypertrophic cardiomyopathy, and cardiac amyloidosis. Early detection and characterization of LV wall thickness can significantly impact patient care.
  • systems and methods are provided for automatically measuring left ventricular dimensions and also identifying patients with increased wall thickness who could benefit from additional screening for hypertrophic cardiomyopathy and cardiac amyloidosis.
  • a deep learning workflow is provided that automatically quantifies ventricular hypertrophy with precision equal to human experts and predicts etiology of increased wall thickness.
  • the systems and methods described herein perform consistently across multiple cohorts while also delivering results in a fraction of the time required for human assessment.
  • the deep learning workflow can automate wall thickness evaluation while facilitating identification of hypertrophic cardiomyopathy and cardiac amyloidosis.
  • a deep learning model is provided to quantify ventricular hypertrophy with precision equal to human experts, estimate left ventricular mass, and predict etiology of LVH.
  • deep learning models may vary in terms of organization, structure, accuracy, input/output information, etc., and are referred to generally herein as “EchoNet-LVH”. Accordingly, while the term EchoNet-LVH may be used in conjunction with specific approaches herein, this is done by way of example and is in no way intended to limit the invention.
  • EchoNet-LVH model Trained on a plurality of echocardiogram videos, EchoNet-LVH model accurately measures one or more of intraventricular wall thickness, left ventricular diameter, and posterior wall thickness.
  • EchoNet-LVH model classifies cardiac amyloidosis and aortic stenosis from other etiologies of LVH. For example, through external datasets from independent international and domestic healthcare systems, EchoNet-LVH accurately quantified ventricular parameters and detected cardiac amyloidosis and aortic stenosis. EchoNet-LVH model utilizes measurements across multiple heart beats. Accordingly, EchoNet-LVH model can more accurately identify subtle changes in left ventricular (LV) geometry and its causal etiologies. Compared to human experts, EchoNet-LVH is more reproducible than human evaluation and provides improved precision in diagnosis of cardiac hypertrophy.
  • LV left ventricular
  • a method for determining etiology of a heart structural health condition comprises: acquiring one or more echocardiograms; inputting the one or more echocardiograms into an etiology prediction model; and generating an output indication including one or more predicted caused of the heart structural health condition; wherein the etiology prediction model is trained on a plurality of echocardiograms that have been identified to indicate left ventricular hypertrophy.
  • a method for assessing structural condition of a heart of a patient includes: receiving echocardiogram data acquired by an echocardiogram system.
  • the acquired echocardiogram data is used to determine one or more of a heart structural condition and one or more causes of the heart structural condition associated with the acquired echocardiogram data via a trained neural network model.
  • the trained neural-network model receives the echocardiogram data as an input (e.g., for continued training).
  • the trained neural network model also includes a first neural network model for determining the heart structural condition and a second neural network model for predicting one or more causes of the heart structural condition.
  • the first neural network model includes at least one atrous convolutional layer performing an atrous convolutional operation on the echocardiogram data.
  • the first neural network model is a segmentation model configured to identify two or more structural key points on the input echocardiogram data.
  • the first neural network model is configured to determine the heart structural condition based on distances between at least two of the two or more structural key points.
  • the first neural network model may also be trained based on sparse labeling of a training echocardiogram dataset.
  • one or more of the first neural network and the second neural network models are optimized according to a number of skipped frames.
  • the heart structural condition is determined based on a desired scan plane, the desired scan plane based on parasternal long axis.
  • a system is for cardiac structure assessment.
  • the system includes: at least one memory storing a trained neural network model and executable instructions, as well as at least one processor communicably coupled to the at least one memory.
  • the trained neural network model is preferably including at least one atrous convolutional layer.
  • the processor when executing the instructions, is configured to: receive a set of echocardiogram frames of a patient from an echocardiogram system, and process the set of echocardiogram frames via the trained neural network model.
  • These segmented frames may identify a plurality of key structural points on the set of segmented echocardiogram frames.
  • the processor is also configured to obtain as output from the trained neural network model, an indication of a cardiac structural condition based on the set of segmented echocardiogram frames.
  • the indication of the cardiac structural condition and/or segmented echocardiogram frames is displayed on a beat-by-beat basis. This may be achieved via a display portion of a user interface coupled to the at least one processor in some approaches. In some approaches the cardiac health condition may be determined on a beat- by-beat basis.
  • the trained neural network model includes an etiology prediction classification model in some approaches based on a ResNet architecture. Moreover, the plurality of key structural points may be identified via corresponding heat-map representations in some approaches. The plurality of key structural points may also be based on centroids of the corresponding heat-maps. [0018] In some approaches, the trained neural network model is trained based on sparse labeling of a training echocardiogram dataset. A decision may also be made in some approaches regarding one or more causes of the cardiac structural condition via the trained neural network model. The one or more causes of the cardiac structural condition may include hypertension, aortic stenosis, hypertrophic cardiomyopathy, cardiac amyloidosis, etc. and/or combinations thereof.
  • the trained neural network model is trained for amyloid classification based on one or more negative training controls, the one or more negative training controls including images from patients with diagnosed HCM, aortic stenosis, hypertension, and ESRD.
  • FIG.1 is a schematic diagram illustrating an overarching echocardiogram system for LVH assessment and corresponding LVH etiology prediction.
  • FIG.2 is a schematic diagram illustrating an architecture of a convolutional neural network model for LVH assessment and corresponding LVH etiology prediction, which can be implemented in the echocardiogram system of FIG. 1, according to an embodiment of the disclosure
  • FIG.3 is a schematic diagram illustrating an architecture of an etiology prediction module included in the convolutional neural network model of FIG.2, according to an embodiment of the disclosure
  • FIG. 3B is a schematic diagram of a bottleneck module included in the inverted residual and max pooling module of FIG.3A, according to an embodiment of the disclosure
  • FIG. 4A shows correlation of human annotations vs.
  • FIG. 4B shows model variation compared to human variation in annotation. Boxplot represents the median as a thick line, 25th and 75th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the median.
  • FIG. 4D shows correlation of human annotations vs.
  • FIG.4E shows model variation on datasets from three healthcare systems compared to human clinical annotation variation. Boxplot represents the mean as a thick line, 25 th and 75 th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the mean.
  • FIG.5A shows model prediction of key points (in green, purple, blue, and yellow) on individual frame of parasternal long video;
  • FIG. 5B shows frame-by-frame prediction of wall thickness and ventricular dimension and automated detection of systole and diastole allowing for beat-to-beat prediction of ventricular hypertrophy;
  • FIGS.6A – 6C show performance of disease etiology classification.
  • FIG.6B shows representative images for selected cases and controls for each etiology.
  • FIGS. 6D – 6F show Performance of disease etiology classification across two independent domestic institutions. A.
  • FIG. 7A – 7C example Hyperparameter Search of Video-based Classification Model.
  • FIG.8 is a high-level flow chart showing an example method for performing LVH etiology prediction, according to an embodiment of the disclosure
  • FIG. 9 is a high-level flow chart showing an example method for training a convolutional neural network model for etiology prediction, according to an embodiment of the disclosure
  • FIGS. 10A – 10D are echocardiogram and corresponding plots showing beat-by- beat ventricular dimension assessment.
  • FIGS. 11A and 11B show comparison of Model Performance with Human Variation.
  • B. Correlation of clinician reported measurements compared to prior study for studies without significant change (n 23,874 at SHC).
  • FIGS. 12A and 12B show Comparison of Model Performance with Prospective Consensus Annotation of Two Level III Echocardiography Certified Cardiologists.
  • the same reference numbers and any acronyms identify elements or acts with the same or similar structure or functionality for ease of understanding and convenience.
  • echocardiography While abundant, low cost, and without ionizing radiation, echocardiography is the most common form of cardiovascular imaging. However, echocardiography relies on expert interpretation and has measurement variation. Given the labor cost of clinical phenotyping and intra-clinician variability in annotation, previous genetic studies from echocardiography parameters have suggested genetic associations were small, and sometimes were not replicated. [0052]
  • the heart is a dynamic organ capable of remodeling and adaption due to physiologic stress or extra-cardiac perturbation. Both intrinsic cardiac disease as well as systemic insults can result in left ventricular hypertrophy (LVH) and phenotypic mimics that is indistinguishable on routine imaging.
  • LH left ventricular hypertrophy
  • echocardiography relies on expert interpretation and its accuracy is dependent on careful application of measurement techniques.
  • recent work has shown that deep learning applied to medical imaging can identify clinical phenotypes beyond conventional image interpretation and with higher accuracy than human experts.
  • the inventors herein have recognized that echocardiography, when enhanced with artificial intelligence models, can provide additional value in understanding disease states by predicting both the presence of left ventricular hypertrophy in a screening population as well as the potential etiology of diagnosis.
  • a method comprises performing frame- level semantic segmentation of the left ventricular wall thickness from parasternal long axis echocardiogram videos and then performing beat-to-beat evaluation of ventricular hypertrophy. After identifying left ventricular hypertrophy, a three-dimensional convolutional neural network with residual connections is used to predict the etiology of the LVH, including predictions for cardiac amyloidosis and aortic stenosis among a background of other hypertrophic diseases.
  • echocardiography enhanced with artificial intelligence models can provide improved value in understanding disease states (e.g., etiology of diseases).
  • methods and systems are provided for a deep learning approach to detect the presence of left ventricular hypertrophy in a screening population and classify potential etiology of diagnosis.
  • EchoNet-LVH an end-to-end deep learning model is provided for labelling the left ventricle, quantifying ventricular wall thickness, and predicting etiology of LVH.
  • a beat-by-beat assessment of ventricular hypertrophy is performed.
  • a frame-level semantic segmentation of the left ventricular wall thickness for input parasternal long axis echocardiogram video is performed to determine beat-to-beat evaluation of ventricular hypertrophy.
  • a three-dimensional convolutional neural network with residual connections predict the etiology of the LVH, including predictions for cardiac amyloidosis and aortic stenosis among a background of other hypertrophic diseases.
  • a technical advantage of the convolutional network model, such as EchoNet-LVH, described herein is improved LVH assessment and prediction of one or more causes for LVH.
  • a risk of LVH may be determined based on a degree of ventricular wall thickness.
  • EchoNet-LVH is a deep learning system for the quantification of left ventricular hypertrophy on echocardiography and automated prediction of etiology of the hypertrophy. EchoNet-LVH achieves state-of-the-art performance in assessment of ventricular thickness and diameter, within the variance in clinical test-retest assessment, and aids in detection of subtle phenotypes particularly challenging for human readers.
  • EchoNet- LVH Integrating the steps of identifying ventricular hypertrophy and subsequent downstream prediction of etiology, EchoNet- LVH provides a fully automated workflow for disease screening in the most common form of cardiac imaging and can potentially greatly improve access to cardiovascular care. Whether used to triage patients for systemic therapy for cardiac amyloidosis or percutaneous valve replacement for aortic stenosis, EchoNet-LVH facilitate early expedited care end-stage heart disease with ventricular remodeling. [0060] In one example, EchoNet-LVH is tested in both domestic and international datasets not seen during model training, EchoNet-LVH’s performance in assessing ventricular thickness was robustly accurate without additional fine-tuning. These different testing regimens span across continents, clinical practice patterns, and instrumentation for image acquisition.
  • EchoNet-LVH performs these tasks in real time with only one GPU; each prediction is more rapid than human assessment, allowing for real-time screening of cardiovascular disease in the clinic setting. Further, EchoNet-LVH is robust to variation in practice patterns across continents. [0061]
  • the methods and systems herein provide an important improvement in assessment of cardiac structures in echocardiogram videos through deep learning. EchoNet-LVH augments current methods for assessing cardiac form and structure to provide more holistic evaluation of cardiovascular disease. With improved precision to detect ventricular remodeling and cardiac dysfunction, EchoNet-LVH enables earlier detection and treatment of subclinical cardiovascular disease. [0062] In this way, the deep neural network models described herein provide an important step towards phenotyping cardiac health evaluation.
  • an echocardiogram system 100 is shown, in accordance with an exemplary embodiment.
  • the echocardiogram system 100 includes an echocardiogram processing system 102 which is communicatively coupled to an echocardiogram acquisition system 120 and user interface 130.
  • echocardiogram transducers 122 and at least one power supply 124 are included.
  • the transducers 122 include ultrasound transducers, but any one or more desired types of echocardiogram transducers 122 may be used.
  • the power supply 124 preferably provides electrical power to the transducers 122.
  • the system 120 may be a clinical grade echocardiogram system.
  • the system 120 may be configured as a mobile ultrasound system.
  • the power supply 124 and transducers 122 may thereby be able to capture echocardiogram data from a sample (e.g., patient) in proximity to the transducers 122.
  • the echocardiogram data generated by the transducers 122 may further be stored in memory 123 at the echocardiogram acquisition system 120. Medical information, e.g., like the echocardiogram data generated by the transducers 122, is sensitive and is often associated with heightened scrutiny in terms of how it is stored.
  • the process of storing the generated echocardiogram data in memory 123 includes converting the data into a different, and preferably more secure form.
  • generated echocardiogram data is encrypted prior to being stored in memory, thereby transforming the echocardiogram data from the general form it was created in, into a more secure form.
  • the encrypted echocardiogram data may thereby be sent from the echocardiogram acquisition system 120 to the echocardiogram processing system 102 with minimal risk of the data being intercepted and improperly accessed.
  • the encrypted echocardiogram data may be processed further before being sent between systems 120, 102.
  • the encrypted data may be processed by an integrated processing unit (not shown) and subsequently transmitted via wired and/or wireless connections to the echocardiogram processing system 102 for cardiac structure evaluation.
  • systems 120 and 102 may be connected over a network capable of transferring data that has been configured in a particular format.
  • the encrypted echocardiogram data may be divided into packets, each of which are uniquely numbered and associated with a destination address before being sent from the echocardiogram acquisition system 120 to the echocardiogram processing system 102.
  • the echocardiogram data may be divided into transmission control protocol/Internet protocol (TCP/IP) packets by a TCP layer, e.g., as would be appreciated by one skilled in the art after reading the present description.
  • TCP/IP transmission control protocol/Internet protocol
  • the data may be decrypted for further use. Additional processes may also be performed on the received data in some approaches, e.g., such as recombining packets of data.
  • the echocardiogram processing system 102 is disposed at a device (e.g., edge device, server, etc.) communicably coupled to the echocardiogram system 120 via wired and/or wireless connections.
  • the echocardiogram processing system 102 is disposed at a separate device (e.g., a workstation) which can acquire echocardiogram data from the echocardiogram system 120 or from a storage device which stores the echocardiogram data acquired by the echocardiogram acquisition system 120.
  • the echocardiogram processing system 102 may comprise at least one processor 104, and be in communication with a user interface 130.
  • the user interface 130 may include a user input device (not shown), and a display device 132.
  • User input devices that may be used include one or more of a touchscreen, a keyboard, a mouse, a trackpad, a motion sensing camera, etc., or other device configured to enable a user to interact with and manipulate data within the echocardiogram system 100.
  • the at least one processor 104 is preferably configured to execute machine readable instructions which may be stored in non-transitory memory 106.
  • the processor 104 may be single core or multi-core, and the programs executed thereon may be configured for parallel or distributed processing.
  • the processor 104 may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and/or configured for coordinated processing.
  • one or more aspects of the processor 104 may be virtualized and executed by remotely-accessible networked computing devices configured in a cloud computing configuration.
  • the processor 104 may include other electronic components capable of carrying out processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphic board.
  • FPGA field-programmable gate array
  • the processor 104 may include multiple electronic components capable of carrying out processing functions.
  • the processor 104 may include two or more electronic components selected from a list of electronic components including: a central processor, a digital signal processor, a field-programmable gate array, and a graphic board.
  • the processor 104 may be configured as a graphical processing unit (GPU) including parallel computing architecture and parallel processing capabilities.
  • GPU graphical processing unit
  • Non-transitory memory 106 may store a pre-processing module 108, a ventricular dimension assessment module 110, an etiology prediction module 111, and echocardiogram data 112.
  • the ventricular dimension assessment 110 may include a neural network model comprising a plurality convolutional layers.
  • the module 110 may further include instructions for implementing the neural network model to receive an echocardiogram video data of a patient acquired and output a corresponding LVH classification (e.g., presence or absence of LVH).
  • the module 110 may store instructions for implementing a neural network model, such as an exemplary model 210 shown at FIG. 2.
  • the module 110 may include trained and/or untrained neural networks and may further include various data, or metadata pertaining to the one or more neural networks stored therein.
  • Non-transitory memory 106 may further store training module 114, which comprises instructions for training one or more neural network models stored in the module 110.
  • Training module 114 may include instructions that, when executed by processor 104, cause echocardiogram processing system 100 to conduct one or more of the steps of method 900 (FIG. 9) for training one or more neural network models with corresponding training data sets, discussed in more detail below.
  • training module 114 includes instructions for implementing one or more gradient descent algorithms, applying one or more loss functions, and/or training routines, for use in adjusting parameters of one or more neural network models of the module 110.
  • Non-transitory memory 106 also stores an inference module 116 that comprises instructions for validating and testing new data with the trained neural network model.
  • Non- transitory memory 106 further stores echocardiogram data 112.
  • echocardiogram data 112 may include a plurality of training sets, each comprising a plurality of echocardiograms.
  • the non-transitory memory 106 may include components disposed at two or more devices, which may be remotely located and/or configured for coordinated processing. In some embodiments, one or more aspects of the non-transitory memory 106 may include remotely-accessible networked storage devices configured in a cloud computing configuration.
  • Non-transitory memory 106 may further store a model interpretable explanation generator module 118.
  • the model interpretable explanation generator module 118 may include an interpretable model configured to provide one or more visual indications for explaining each individual prediction of the neural network model for LVH and etiology classification.
  • the interpretable model may be based on a model interpretability technique (e.g., Local Interpretable Model-Agnostic Explanations (LIME)) provides data that explains how a model made a determination.
  • LIME Local Interpretable Model-Agnostic Explanations
  • the interpretable model may be a local interpretable model.
  • a global interpretable model may be used.
  • the model interpretable explanation generator via the model interpretable explanation generator, one or more important echocardiogram features having a weightage greater than a threshold weightage and contributing to LVH and corresponding etiology prediction may be identified and indicated. While the above examples describe using the LIME technique for interpreting the neural network model output, other interpretability techniques, such as Shapely Additive Explanations, may be used and are within the scope of the disclosure.
  • Display 132 may include one or more display devices utilizing virtually any type of technology.
  • display device 132 may comprise a computer monitor, and may display unprocessed and processed echocardiogram frames.
  • Display device 132 may be combined with processor 104, non-transitory memory 106, and/or user input device in a shared enclosure, or may be peripheral display devices and may comprise a monitor, touchscreen, projector, or other display device known in the art, which may enable a user to view echocardiogram frames, and/or interact with various data stored in non-transitory memory 106.
  • the display 132 is used to display the indication of a cardiac structural condition and/or segmented echocardiogram frames on a beat-by-beat basis.
  • echocardiogram processing system 100 shown in FIG. 1 is for illustration, not for limitation. Another appropriate image processing system may include more, fewer, or different components.
  • the echocardiogram processing system 100 may be used to train and deploy a neural network model, such as an example neural network model discussed below at FIG.2. Neural Network Architecture.
  • FIG.2 it shows a high-level block diagram of a neural network model 200 that may be implemented for identifying LVH and subsequently, classifying one or more causes (e.g., etiology-based causes) for LVH.
  • the neural network model 200 includes EchoNet-LVH.
  • the neural network model 200 comprises a first neural network model which is depicted in the present approach as being a ventricular dimension assessment model 210, but other types of neural network models which would be apparent to one skilled in the art after reading the present description may be implemented in other approaches.
  • the ventricular dimension assessment model 210 itself may also be configured to classify a cardiac structural condition (e.g., such as LVH) based on one or more heart structural parameters.
  • a cardiac structural condition e.g., such as LVH
  • the “cardiac structural conditions” may be referred to specifically as herein as a LVH condition, but this is done by way of example, and is in no way intended to be limiting.
  • the heart structural parameters may include one or more of the intraventricular septum (IVS), left ventricular internal dimension (LVID), and left ventricular posterior wall (LVPW).
  • the ventricular dimension assessment model 210 includes a weak supervision layer 220 and an atrous convolution layer 230.
  • the ventricular dimension assessment model 210 receives echocardiogram frames that have been captured (e.g., created) by a system having the components associated with actually capturing the frames (e.g., see 120 of FIG. 1 above). These frames are passed through the weak supervision layer 220 which may be used to apply different types of supervision (e.g., labeling functions) to the echocardiogram frames and/or information included therein to form numerous unsupervised labeled data sets.
  • the frames are sent through layer 225 which combines information from either one frame or multiple frames across a video, of which some can be labeled and others are not labeled.
  • the echocardiogram frames are also passed through an atrous convolution layer 230 which may be used to apply one or more dilated convolutions to the data sets, e.g., to increase the field of view which is particularly valuable in situations involving real-time segmentation.
  • the frames are additionally input to layer 235 which produces a heat map or probability function of regions were key points may be that can be averaged to a maximally likely key point. From layer 235, the echocardiogram frames are output with corresponding key points identified in the frames themselves 240.
  • the model 210 may be trained for semantic segmentation of parasternal long axis (PLAX) echocardiogram videos and identification of one or more of IVS, LVID, and LVPW. With atrous convolutions to capture longer range features, full resolution PLAX frames may be used as input images for higher resolution assessment of LVH. Accordingly, the echocardiogram frames having the corresponding key points identified may further be provided to an additional assessment module 250 capable of performing assessments associated with LVH.
  • PDAX parasternal long axis
  • the assessment module 250 may be able to utilize key points identified in the frames to determine a presence and/or type of LVH that may be affecting the heart that was examined to produce the echocardiogram frames (i.e., the heart shown in the echocardiogram frames).
  • An etiology prediction model 290 may also be implemented before an etiology classification is output, e.g., as will be described in further detail below with respect to FIG.3. [0081] Due to the tedious nature of annotation, labels may only be provided for a subset of the frames in a video (e.g., one or two frames of a video) by the weak supervision layer 220.
  • the assessment model 210 may transform these sparse annotations into measurement predictions for every frame of the entire video to allow for beat-to-beat estimation of ventricular wall thickness and dimensions.
  • An example method for performing beat-by-beat ventricular assessment is described by Ouyang, D., He, B., Ghorbani, A. et al. in “Video-based AI for beat- to-beat assessment of cardiac function” published in Nature 580, 252–256 (2020), the content of which is incorporated by reference herein in its entirety.
  • the ventricular dimension assessment model 210 is a segmentation model, which is trained to identify a plurality of key points on the echocardiogram frames and measure one or more ventricular dimensions using the plurality of key points.
  • one or more ventricular dimensions are measured on a beat-by-beat basis.
  • Example video frames showing the key points and corresponding plots are shown in FIGS.10A – 10D.
  • trained neural network models may be trained based on one or more negative controls.
  • these negative controls may include images from patients with different causes of LVH.
  • the one or more negative training controls may include images from patients with diagnosed HCM, aortic stenosis, hypertension, and/or ESRD.
  • the neural network model may be trained for amyloid classification based on one or more negative training controls in some approaches.
  • the model 200 may be stored in a memory of a processing system, e.g., such the echocardiogram processing system 100 at FIG.1.
  • the neural network model 200 may receive as input, echocardiogram data of a patient, and output one or more of an indication of a LVH condition and one or more etiology of the LVH condition.
  • the echocardiogram data may be echocardiogram frames derived from one or more examination procedures performed on the patient, and these frames may be pre-processed prior to passing through the neural network model 200.
  • Pre-processing echocardiogram waveform data may include filtering, for example.
  • filtering may be performed to remove very low quality video signals.
  • the approaches herein are able to desirably reduce the unnecessary consumption of computational resources.
  • the approaches herein are desirably able to reduce the amount of processing power that is dedicated to interpreting subpar data. In fact, if this subpar data were to actually be evaluated and incorporated in the training, accuracy of the overall deep learning model may be sacrificed. Pre- processing is thereby able to further improve accuracy, while also reducing the computational throughput associated with achieving this improved accuracy.
  • the neural network model 200 are able to implement processes that ultimately improve operation of the computing components included therein, e.g., as would be appreciated by one skilled in the art after reading the present description.
  • pre-processing of the input echocardiogram frames may be based on the pre- processing operations performed during training the neural network model.
  • the model 200 may include a second video-based convolutional neural network (CNN) model.
  • the second video-based CNN model may be an etiology prediction model 290 configured to classify etiology of LVH.
  • FIG.3 The details of the etiology model 290 are shown at FIG.3 in accordance with an exemplary approach which are in no way intended to limit the invention.
  • the present etiology model 290 may be implemented in conjunction with features from any other embodiment listed herein, such as those described with reference to the other FIGS., such as FIG. 2.
  • such etiology model 290 and others presented herein may be used in various applications and/or in permutations which may or may not be specifically described in the illustrative embodiments listed herein.
  • the etiology model 290 presented herein may be used in any desired environment.
  • FIG. 3 (and the other FIGS.) may be deemed to include any possible permutation.
  • the etiology prediction model 290 may include layers performing spatiotemporal convolutions to predict etiology of LVH. Integrating spatial as well as temporal information, the model 200 combines LVH assessment with video-based model interpretation of echocardiograms and classify videos based on probability of hypertension, aortic stenosis, hypertrophic cardiomyopathy, or cardiac amyloidosis as etiology of ventricular hypertrophy. For instance, spatial and temporal information may be extracted from a graphical representation 302 of the relevant data. This extracted information may be passed to a first spatial convolution module 304, prior to a number of convolution layers 306, 308, 310, 312.
  • each of these convolution layers 306, 308, 310, 312 may further include a desired number and order of spatial and/or temporal sub-layers 316, 318 respectively.
  • the processed information is provided to a spatiotemporal pool 314 which may be used to collect and organize the information, e.g., based on relationships that have been determined and/or defined.
  • a hyperparameter search may be conducted to identify the optimal base architecture for EchoNet-LVH, e.g., as seen in the graphs depicted in FIGS.7A – 7C.
  • a hyperparameter search for a model architecture e.g., such as R3D, which may be used by EchoNet-LVH for hypertrophy etiology classification, R 2 +1D, and MC3
  • input video clip length 8, 16, 32, 64, 96 frames
  • model processing time and model memory usage e.g., a hyperparameter search for a model architecture
  • the etiology prediction model is a trained neural network model, wherein the training is based on a hyperparameter sweep on frame rate to determine a number of frames to skip.
  • the dataset for training the ventricular dimension assessment model typically includes of a plurality of echocardiograms.
  • Each echocardiogram preferably includes a plurality of frames, and includes at least two annotated frames, one at systole and one at diastole.
  • LVID measurements may be annotated in both systole and diastole frames, while IVS and LVPW measurements are may be made once at diastole.
  • the labels used for training the assessment model are four channel images that contain all zeros except for the pixels closest to the endpoints of measurements being ones.
  • PLAX echo frames may be fed through the ventricular dimension assessment model to predict heat-maps representing the location of each of the four measurement points.
  • the model is initialized with random weights and trained for a desired number of epochs using an optimizer with an initial learning rate.
  • random scale, shift and rotate data augmentation may be applied to both input images and labels.
  • the centroid of each of the output channels may thereby be calculated as the predicted location for each point.
  • the LVPW, LVID, and IVS may further be calculated using these four measurement points.
  • MSE Mean Square Error
  • Equation 1 below may be used as a modified weighted MSE loss function to weighs the losses associated with false positives and negatives differently.
  • alpha “ ⁇ ” the amount that false positive labels are penalized can be scaled and prediction size can be optimized.
  • predictions based on distance from original point labels may be penalized and anatomic measurements may be derived.
  • Loss is a summation of the difference between the label and model prediction at each pixel which is scaled by alpha such that the relative weighting of importance for negative and positively labeled pixels are optimized.
  • the positive pixels can be heavily weighted to prioritize model performance for those good performance on those pixels while negative pixels can be relatively unprioritized.
  • centroid of each of the output channels was also calculated as the predicted location for each point.
  • the LVPW, LVID, and IVS are calculated from these four points.
  • using machine learning to process information like echocardiograms has the benefit of being able to label an entire video and calculate other practical values associated therewith. For instance, in addition to being able to determine relevant measurements for every frame, the displacement of each of the points along the measurement axis over time may also be calculated to show how the heart is moving over time.
  • training dataset for ventricular dimension assessment model comprises sparsely annotated echocardiograms.
  • the weak supervision refers to the fact that in each video, only 1 or 2 frames (e.g., only about 0.5 – 1% of the data) are actually labeled. Despite this sparse amount of information, machine learning processes described herein are able to utilize these labeled frames.. For instance, in some approaches, a weighted loss function is used. Further, false positives/negatives are weighted and a hyperparameter sweep of the weighted parameters may be performed. [0096] Example 1 [0097] It should be noted that the following example is presented for illustrative purposes and is in no way intended to limit the scope of the invention.
  • EchoNet-LVH predicted ventricular dimensions with a coefficient of determination (R 2 ) value of 0.97. This coefficient is typically determined by measuring the amount of variance in the results produced by a given dataset. In other words, R 2 it is the difference between the samples in a dataset and the predictions made by the deep learning model using the dataset. Looking to the results depicted in FIGS. 5A, 5B, and 5C, applying EchoNet-LVH in the present example also resulted in a mean absolute error (MAE) of 1.2mm for IVS, 2.4mm for LVID, and 1.4mm for LVPW.
  • MAE mean absolute error
  • EchoNet-LVH was additionally tested, without any tuning, on an external test dataset of 1,791 videos from Unity Collaborative and 13,796 videos from Cedars-Sinai Medical Center (CSMC). On the Unity external test dataset, EchoNet-LVH showed a robust prediction accuracy with an overall R 2 of 0.90, MAE of 1.6mm for IVS, 3.6mm for LVID, and 2.1cm for LVPW.
  • a standard full resting echocardiogram study may include of a series of about 50- 100 videos, as well as still images visualizing the heart from different angles, locations, and image acquisition techniques (2D images, tissue Doppler images, color Doppler images, and others).
  • the network architecture was trained on parasternal long axis images to minimize a weighted mean square error loss was used to identify key points used for measuring ventricular dimensions.
  • An Adam optimizer was used with a learning rate of 0.001 was used and the model was trained for 50 epochs with early stopping based on the validation loss.
  • an 18-layer ResNet 3D24 architecture was used to classify videos as either amyloid or not amyloid.
  • This model was trained to minimize binary cross-entropy loss using an Adam optimizer with a learning rate of 0.01. The model was trained for 100 epochs with a batch size of 14 with early stopping based on AUC on the validation set.
  • EchoNet-LVH model trained on 28,201 echocardiogram videos, EchoNet-LVH model accurately measures intraventricular wall thickness (mean absolute error 1.4mm, 95% CI 1.2- 1.5mm), left ventricular diameter (MAE 2.4mm, 95% CI 2.2-2.6mm), and posterior wall thickness (MAE 1.2cm, 95% CI 1.1-1.3mm) and classifies cardiac amyloidosis (AUC of 0.83) and aortic stenosis (AUC 0.89) from other etiologies of LVH.
  • AUC cardiac amyloidosis
  • AUC 0.89 aortic stenosis
  • EchoNet-LVH accurately quantified ventricular parameters (R 2 of 0.90) and detected cardiac amyloidosis (e.g., AUC 0.79) and aortic stenosis. Leveraging measurements across multiple heart beats, EchoNet-LVH model can more accurately identify subtle changes in left ventricular (LV) geometry and its causal etiologies. Compared to human experts, EchoNet-LVH is more reproducible than human evaluation and provides improved precision in diagnosis of cardiac hypertrophy. [00117] Example 2 [00118] It should be noted that the following example is presented for illustrative purposes and is in no way intended to limit the scope of the invention.
  • a standard full resting echocardiogram study may include a series of about 50-100 videos, as well as still images visualizing the heart from different angles, locations, and image acquisition techniques (2D images, tissue Doppler images, color Doppler images, and others). Patients were identified by physician curated cohorts from the Stanford Amyloid Center and CSMC Advanced Heart Disease Clinic for cardiac amyloidosis and the Stanford Center for Inherited Cardiovascular Disease and CMSC Hypertrophic Cardiomyopathy Clinic for hypertrophic cardiomyopathy. [00122] In this study, relevant PLAX and A4C 2D videos were extracted from each study. Human clinician annotations of IVS, LVID, and LVPW were used as training labels to assess ventricular hypertrophy.
  • PLAX videos were split 9,600, 1,200, and 1,200 patients respectively for the training, validation, and test sets.
  • An additional 7,767 SHC studies were of patients with defined disease characteristics, including cardiac amyloidosis, hypertrophic cardiomyopathy, and severe aortic stenosis. From these studies, the A4C videos were extracted and used as input data for the hypertrophic disease classification task. Videos were processed in a previously described automated preprocessing workflow removing identifying information and human labels. This research was approved by the Stanford University and Cedars-Sinai Medical Center Institutional Review Boards.
  • Deep Learning Algorithm Development and Training [00126] Model design and training was done in Python using the PyTorch deep learning library. A modified DeepLabV3 26 architecture trained on parasternal long axis images to minimize a weighted mean square error loss was used to identify key points used for measuring ventricular dimensions. 3D implementations of segmentation model took substantially more computational resources without significant improvement in performance. An Adam optimizer was used with a learning rate of 0.001 was used and the model was trained for 50 epochs with early stopping based on the validation loss. Different video lengths, resolutions, and temporal resolutions were evaluated as hyperparameters to optimize model performance. Computational cost was evaluated using one NVIDIA GeForce GTX 3090.
  • an 18-layer ResNet 3D architecture was used to classify videos. Given the potential for overlap patients with multiple etiological diagnoses for LVH 28 , parallel binary classification deep learning models were trained to predict probability of amyloid, hypertrophic cardiomyopathy, aortic stenosis, secondary to uncontrolled hypertension, and in setting of end-stage kidney disease (ESRD) independently. Distinct from prior literature, for each classification task, the negative controls were images from patients with other causes of LVH to mimic the clinical workflow. For example, during amyloid classification, the negative training examples included patients with diagnosed HCM, aortic stenosis, hypertension, and in setting of ESRD as other etiologies of LVH.
  • ESRD end-stage kidney disease
  • This model was trained to minimize binary cross- entropy loss using an Adam optimizer with a learning rate of 0.01. The model was trained for 100 epochs with a batch size of 14 with early stopping based on AUC on the validation set.
  • Comparison with Variation in Human Measurement [00129] Using the reporting database of Stanford Echocardiography Laboratory, paired studies of the same patient were identified, for which the reviewing cardiologist determined there was no significant change from the current study to the prior study by structured reporting element. Of these studies with clinical stability, the subset of 23,874 studies were analyzed for which LVID, IVS, and LVPW at diastole was measured for both the current and subsequent study.
  • Deep learning workflows and other information associated with screening of hypertrophic cardiomyopathy and cardiac amyloidosis in the various approaches herein preferably have two components.
  • the deep learning model is provided with atrous convolutions for semantic segmentation of PLAX various echocardiogram videos, and identification of the IVS, LVID, and LVPW.
  • atrous convolutions to capture longer range features full resolution PLAX frames were used as input images for higher resolution assessment of LVH in some illustrative approaches.
  • FIG. 4A includes a graph showing a correlation between human annotations and model predictions for ventricular dimensions in datasets from two independent healthcare systems.
  • FIG.4B shows model variation compared to human variation in annotation. Boxplot represents the median as a thick line, 25th and 75th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the median.
  • FIG.4D graphically depicts the correlation between human annotations and model predictions using various ones of the approaches included herein.
  • the data displayed in the graph was derived from ventricular dimensions in datasets from three different healthcare systems.
  • FIG.4E model variation is depicted with respect to datasets from three healthcare systems compared to human clinical annotation variation.
  • the boxplot graph represents the mean as a thick line, 25 th and 75 th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the mean.
  • a neural network as described in the various approaches herein may generally be trained on these sparse annotations, and be able to make accurate measurement predictions for every frame of the entire video in dynamically efficient manner to allow for beat-to-beat estimation of ventricular wall thickness and dimensions. Pre-processing the sparse annotations may be able to further improve accuracy, while also reducing the computational throughput associated with achieving this improved accuracy. Accordingly, the neural network models in the various approaches herein are desirably are able to implement processes that ultimately improve operation of the computing components included therein, e.g., as would be appreciated by one skilled in the art after reading the present description. [00138] After detection of LVH, identifying the specific etiology (e.g.
  • FIGS. 6A – 6C show performance of disease etiology classification based on this example.
  • FIG. 6B shows representative images for selected cases and controls for each etiology.
  • FIG. 6B shows representative images for selected cases and controls for each etiology.
  • FIGS. 6D – 6F show performance of disease etiology classification across two independent domestic institutions.
  • representative images for selected cases and controls for each etiology are presented in FIG.6E, but are in no way intended to be limiting.
  • the model By integrating spatial as well as temporal information, the model expands the video-based model interpretation of echocardiograms and classifies videos based on probability of hypertension, aortic stenosis, hypertrophic cardiomyopathy, or cardiac amyloidosis as etiology of ventricular hypertrophy. Additionally, a video-based model architecture and hyperparameter search was performed to identify desirable base architecture for the deep learning algorithm.
  • FIGS.7A, 7B, and 7C depict AUC with respect to various chip lengths, time steps, and image resolutions, respectively.
  • the deep learning algorithm according to the present example was trained on a dataset of 17,802 echocardiogram videos from Stanford Health Care (SHC), and then evaluated on held out test cohorts from SHC, CSMC, and Unity Imaging Collaborative.
  • SHC Stanford Health Care
  • EchoNet-LVH The deep learning algorithm (EchoNet-LVH) in the present example had a mean absolute error (MAE) of 1.2mm for IVS, 2.4mm for LVID, and 1.4mm for LVPW. This compares favorably with clinical inter- provider variation, which had a MAE of 1.3mm for IVS, 3.7mm for LVID, and 1.3mm for LVPW. EchoNet-LVH also performed desirably when compared to the prospective consensus annotation of two level 3 echocardiography certified cardiologists in 99 random studies from CSMC, e.g., as depicted in FIGS.12A and 12B.
  • the deep learning algorithm was additionally tested, without any tuning, on an external test dataset of 1,791 videos from Unity Imaging Collaborative and 3,660 videos from CSMC.
  • our deep learning algorithm showed a robust prediction accuracy with an overall R 2 of 0.90, MAE of 1.6mm for IVS, 3.6mm for LVID, and 2.1 mm for LVPW.
  • a rapid, high-throughput automated approach allows for measurement of every individual frame that would be tedious for manual tracing (FIGS.5A, 5B, and 5C). Differences in filling time and irregularity in the heart rate can cause variation in measurement but beat-to-beat model assessment can provide higher fidelity overall assessments. While the SHC and Unity datasets were directly compared on annotated individual frames, evaluation of the deep learning algorithm’s beat-to-beat was performed on the CSMC dataset in comparison with study-level annotations of ventricular dimensions. In this dataset, human measurements were not associated with specific frames of the echocardiogram video, and beat-to-beat analysis was used to predict diastole and average measurements from each heart beat across the entire video.
  • our deep learning algorithm classifies cardiac amyloidosis with an AUC of 0.83, hypertrophic cardiomyopathy with an AUC 0.98, and aortic stenosis with an AUC 0.89 from other etiologies of LVH.
  • the area under the precision-recall curve (AUPRC) of our deep learning algorithm for cardiac amyloidosis was 0.77
  • hypertrophic cardiomyopathy was 0.95
  • aortic stenosis was 0.79.
  • the proposed ensemble of binary classification video-based deep learning classifiers in our deep learning algorithm was similar in performance to a multi-label, multi-class deep learning model for disease detection, however had the flexibility of being able to identify overlap patients with multiple diagnoses.
  • Phenotypic Mimics and Disease Specific Training Pipeline [00147] To highlight the benefit of training a model with negative controls derived from other etiologies of LVH instead of normal controls, a series of experiments was performed to see how a model trained without seeing other phenotypic mimics would perform when encountering phenotypic mimics. A confusion matrix was generated in the two experimental settings (Table 5), where a higher AUC outside the diagonal shows the model confusion and a lower AUC suggests improved discrimination between phenotypic mimics.
  • the deep learning algorithm performs measurements of ventricular thickness and diameter well within the variance of human clinical test-retest assessment – while concurrently aiding the detection of subtle ventricular phenotypes that tend to be challenging for human readers.
  • This integration of left ventricular measurement and prediction of etiology offers an automated workflow for disease screening from echocardiography, the most frequently used form of cardiac imaging.
  • echocardiography-based screening can provide high index of suspicion that can facilitate more efficient clinical evaluation, diagnosis, and care.
  • Assimilation of automated diagnostic algorithms with widely available clinical imaging can reduce physician burden while streamlining opportunities for more targeted cardiovascular care. [00149] Rather than rare, there is reason to believe that diseases such as cardiac amyloidosis are underdiagnosed 28–30 .
  • the systems and methods represent an important step towards the automated assessment of cardiac structures in echocardiogram videos through deep learning.
  • individual linear measurements take only seconds to measure, there is inherent variation in frame and video selection that sets a floor to the precision of manual measurements derived from echocardiography.
  • echocardiographic labels may be augmented with annotations and information from cardiac MRI and other imaging modalities to have more precision automation.
  • deep learning models on echocardiogram images can automate an increasingly larger proportion of tasks for assessing cardiac form and structure to provide more holistic evaluation of cardiovascular disease 8 .
  • Table 1 below includes baseline characteristics of patients with Parasternal Long Axis Videos from Stanford Healthcare and Cedars-Sinai Medical Center echocardiograms, which are displayed for exemplary purposes and are in no way intended to limit the invention. Looking to the bottom section of the leftmost column, it should be noted that the left ventricular ejection fraction (LVEF) is included. Moreover, Table 2 includes baseline characteristics of patients with Apical-4-Chamber Videos from Stanford Healthcare and Cedars-Sinai Medical Center echocardiograms. It should again be noted that the information shown is in no way intended to limit the invention and is displayed for exemplary purposes. Looking now to the upper section of the leftmost column, it should be noted that hypertrophic cardiomyopathy (HCM), aortic stenosis (AS), and left ventricular hypertrophy (LVH) are included. Table 1
  • HCM hypertrophic cardiomyopathy
  • AS aortic stenosis
  • LH left ventricular hypertrophy
  • Table 2 [00153] Table 3 below illustrates performance on Unity Imaging Collaborative External Test Dataset with and without fine-tuning.
  • the deep learning models (EchoNet-LHV) that were trained using Stanford test datasets were able to achieve a favorable R 2 (R2) value of 0.81, while the deep learning models trained using Unity test datasets were able to achieve an increased R 2 value of 0.89.
  • R2 R 2
  • the deep learning models that were first pre-trained using Stanford test datasets, and subsequently fine-tuned using the Unity test datasets were able to achieve a elevated R 2 (R2) value of 0.92. It follows that, in addition to being able to detect heart structural conditions as well as the causes of the conditions more efficiently and even accurately than experts in the field of cardiology, the manner in which the deep learning models are trained also has a notable effect on performance.
  • echocardiogram data may be echocardiogram frames derived from one or more examination procedures performed on the patient, and these frames may be pre-processed prior to passing through a neural network model (e.g., see 200 of FIG.2).
  • Pre-processing the echocardiogram waveform data may include filtering, for example. In some instances, filtering may be performed to remove very low quality video signals.
  • pre-processing of the input echocardiogram frames may be based on the pre-processing operations performed during training the neural network model.
  • the process of training, pre-training, and/or fine-tuning a deep learning model may incorporate any of the approaches described herein.
  • a deep learning model may be processed using any one or more of the operations included in FIGS. 8 and 9.
  • Table 4 performance across different body mass index values is presented with respect to a deep learning model that was trained using a SHC test dataset.
  • Table 5 displays information associated with minimzing confusion of alternative etiologies of hypertrophy. More specifically, this may be achieved when a model is trained on age and sex matched control cases vs. other hypertrophic control. For instance, some instances include a model trained with age and sex matched controls without selection for LVH and introduced other etiologies of LVH at test time not seen during training.
  • non-transitory memory may store training module includes instructions for training one or more neural network models stored in the module.
  • the training module may include instructions that, when executed by processor, cause an echocardiogram processing system to conduct one or more of the steps of methods 800 and 900 below.
  • FIG.8 is a high-level flow chart showing a method 800 for performing LVH etiology prediction, according to an embodiment.
  • FIG.9 depicts a method 900 that may be used to train one or more neural network models with corresponding training data sets, e.g., as will be described in further detail below.
  • FIG.8 a flowchart of a method 800 is shown according to one embodiment.
  • the method 800 may be performed in accordance with the present invention in any of the environments depicted in FIGS. 1-3, among others, in various embodiments. Of course, more or less operations than those specifically described in FIG.8 may be included in method 800, as would be understood by one of skill in the art upon reading the present descriptions.
  • Each of the steps of the method 800 may be performed by any suitable component of the operating environment.
  • the method 800 may be partially or entirely performed by a controller, a processor, a computer, etc., or some other device having one or more processors therein.
  • method 800 may be a computer-implemented method.
  • the terms computer, processor and controller may be used interchangeably with regards to any of the embodiments herein, such components being considered equivalents in the many various permutations of the present invention.
  • the processor e.g., processing circuit(s), chip(s), and/or module(s) implemented in hardware and/or software, and preferably having at least one hardware component may be utilized in any device to perform one or more steps of the method 800.
  • Illustrative processors include, but are not limited to, a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., combinations thereof, or any other suitable computing device known in the art.
  • operation 802 of method 800 includes acquiring echocardiogram data.
  • the echocardiogram data may be acquired directly from an echocardiogram system having components capable of generating the echocardiogram data, e.g., such as an echocardiogram testing facility.
  • Operation 804 includes pre-processing the acquired echocardiogram data.
  • the pre-processing may be performed using any of the approaches described herein.
  • a pre-processing module such as 108 of FIG.1 may be used to pre-process the acquired echocardiogram data.
  • Operation 806 further includes inputting the pre-processed echocardiogram data into a trained ventricular assessment model, from which method 800 proceeds to operation 810.
  • operation 810 includes actually obtaining an indication of LVH condition being present in the echocardiogram data being evaluated.
  • Decision 812 includes determining whether the presence of LVH is confirmed, and if not, an indication of absence of LVH is output. See operation 814. It should also be noted that operation 814 does not involve providing any input to the etiology prediction model.
  • cardiac structural conditions may be referred to specifically as herein as a LVH condition, but this is done by way of example, and is in no way intended to be limiting.
  • operation 810 and/or decision 812 may correspond to a different type of cardiac health condition depending on the instance.
  • method 800 proceeds to operation 816 in response to determining that LVH is confirmed to be present.
  • operation 816 includes inputting the pre- processed echocardiogram data into a trained etiology prediction model, e.g., according to any of the approaches included herein.
  • Operation 818 further includes obtaining (e.g., receiving) a predicted etiology of the LVH present, while operation 820 includes determining treatment based on the predicted etiology of the LVH.
  • the models in various approaches herein are able to achieve earlier detection and treatment of subclinical cardiovascular disease than is otherwise possible.
  • FIG.9 a flowchart of a method 900 is shown according to one embodiment.
  • the method 900 may be performed in accordance with the present invention in any of the environments depicted in FIGS. 1-3, among others, in various embodiments. Of course, more or less operations than those specifically described in FIG.9 may be included in method 900, as would be understood by one of skill in the art upon reading the present descriptions.
  • Each of the steps of the method 900 may be performed by any suitable component of the operating environment.
  • the method 900 may be partially or entirely performed by a controller, a processor, a computer, etc., or some other device having one or more processors therein.
  • method 900 may be a computer-implemented method.
  • the terms computer, processor and controller may be used interchangeably with regards to any of the embodiments herein, such components being considered equivalents in the many various permutations of the present invention.
  • the processor e.g., processing circuit(s), chip(s), and/or module(s) implemented in hardware and/or software, and preferably having at least one hardware component may be utilized in any device to perform one or more steps of the method 900.
  • Illustrative processors include, but are not limited to, a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., combinations thereof, or any other suitable computing device known in the art.
  • operation 902 of method 900 includes acquiring a training dataset having a plurality of echocardiograms.
  • operation 902 includes acquiring a training dataset having a plurality of data entries therein, the data entries corresponding to a plurality of different echocardiograms performed.
  • operation 904 includes pre- processing the acquired training dataset. Again, pre-processing may be performed using any of the approaches described herein.
  • a pre-processing module such as 108 of FIG.1 may be used to pre-process the acquired training dataset.
  • Operation 906 includes training a risk prediction neural network model with the pre-processed training data set.
  • the neural network model may correspond to any of the approaches included herein.
  • operation 908 includes validating and updating hyperparameters of the neural network model, while operation 910 includes testing the trained and validated neural network model.
  • Computer & Hardware Implementation of Disclosure [00171] It should initially be understood that the disclosure herein may be implemented with any type of hardware and/or software, and may be a pre-programmed general purpose computing device.
  • the system may be implemented using a server, a personal computer, a portable computer, a thin client, a wearable device, a digital stethoscope, or any suitable device or devices.
  • the disclosure and/or components thereof may be a single device at a single location, or multiple devices at a single, or multiple, locations that are connected together using any appropriate communication protocols over any communication medium such as electric cable, fiber optic cable, or in a wireless manner.
  • the disclosure is illustrated and discussed herein as having a plurality of modules which perform particular functions. It should be understood that these modules are merely schematically illustrated based on their function for clarity purposes only, and do not necessary represent specific hardware or software.
  • the computing system can include clients and servers.
  • a client and server are generally remote from each other and typically interact through a communication network.
  • the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
  • a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device).
  • data generated at the client device e.g., a result of the user interaction
  • Implementations of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components.
  • the components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network.
  • Examples of communication networks include LANs, WANs, an inter-network (e.g., the Internet), peer-to-peer networks (e.g., ad hoc peer-to-peer networks), any desired type of wireless networks, etc.
  • Implementations of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus.
  • the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine- generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
  • an artificially-generated propagated signal e.g., a machine- generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
  • a computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them.
  • a computer storage medium is not a propagated signal
  • a computer storage medium can be a source or destination of computer program instructions encoded in an artificially-generated propagated signal.
  • the computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, flash memory, or other storage devices).
  • the operations described in this specification can be implemented as operations performed by a “data processing apparatus” on data stored on one or more computer-readable storage devices or received from other sources.
  • the term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, multi-core processors, GPUs, AI-accelerators, In- memory computing architectures or combinations, of the foregoing.
  • the apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
  • the apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them.
  • the apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures, and deep learning and artificial intelligence computing infrastructure.
  • a computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment.
  • a computer program may, but need not, correspond to a file in a file system.
  • a program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).
  • a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
  • Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
  • a processor will receive instructions and data from a read- only memory or a random access memory or both.
  • the essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data.
  • a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, flash memory or optical disks.
  • mass storage devices for storing data
  • a computer need not have such devices.
  • a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), smart watch, smart glasses, patch, wearable devices, a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.
  • PDA personal digital assistant
  • GPS Global Positioning System
  • USB universal serial bus
  • Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
  • semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices
  • magnetic disks e.g., internal hard disks or removable disks
  • magneto-optical disks e.g., CD-ROM and DVD-ROM disks.
  • the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

Abstract

Disclosed herein are systems and methods for evaluating cardiac structural health condition based echocardiogram signals. In one example, a cardiac structural health condition for a patient is predicted on a beat-by-beat basis via a convolutional neural network model comprising at least one atrous layer. Further, in one example, responsive to determining the cardiac health condition, determining one or more causes for the cardiac structural health condition and identifying a treatment for the cardiac health condition based on the one or more causes.

Description

PRECISION PHENOTYPING OF LEFT VENTRICULAR HYPERTROPHY WITH ECHOCARDIOGRAPHIC DEEP LEARNING STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT [0001] This invention was made with government support under Grant No. HL157421 awarded by the National Institutes of Health. The government has certain rights in the invention. CROSS-REFERENCE TO RELATED APPLICATIONS [0002] This application claims priority from and benefit of U.S. Provisional Patent Application Serial No.63/212,045, filed June 17, 2021, titled “PRECISION PHENOTYPING OF LEFT VENTRICULAR HYPERTROPHY WITH ECHOCARDIOGRAPHIC DEEP LEARNING”, as well as U.S. Provisional Patent Application Serial No.63/298,562, filed January 11, 2022, titled “PRECISION PHENOTYPING OF LEFT VENTRICULAR HYPERTROPHY WITH ECHOCARDIOGRAPHIC DEEP LEARNING”, each of which are hereby incorporated by reference herein in their entirety. FIELD OF INVENTION [0003] The present invention relates to cardiac structural assessment and predicting etiology of cardiac health conditions using echocardiograms. BACKGROUND [0004] The following description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art. [0005] Left ventricular hypertrophy (LVH) results from chronic remodeling caused by a broad range of systemic and cardiovascular disease conditions including hypertension, aortic     stenosis, hypertrophic cardiomyopathy, and cardiac amyloidosis. Early detection and characterization of LV wall thickness can significantly impact patient care. However, current approaches are limited by under-recognition of hypertrophy, measurement error and variability, and difficulty differentiating etiologies of increased wall thickness, such has hypertrophy cardiomyopathy and cardiac amyloidosis. Further, current approaches are time-consuming. Accordingly, there is a need for more accurate, reliable, and faster cardiac health evaluation and characterization of disease etiology. SUMMARY [0006] Methods and systems are provided to address at least some of the above-mentioned disadvantages. Further, the inventors have recognized that deep learning may be used to automate measurements of left ventricular dimensions while also identifying patients who could benefit from screening for underdiagnosed diseases. Accordingly, in various implementations, systems and methods are provided for automatically measuring left ventricular dimensions and also identifying patients with increased wall thickness who could benefit from additional screening for hypertrophic cardiomyopathy and cardiac amyloidosis. In one example, a deep learning workflow is provided that automatically quantifies ventricular hypertrophy with precision equal to human experts and predicts etiology of increased wall thickness. The systems and methods described herein perform consistently across multiple cohorts while also delivering results in a fraction of the time required for human assessment. [0007] In one example, the deep learning workflow can automate wall thickness evaluation while facilitating identification of hypertrophic cardiomyopathy and cardiac amyloidosis. [0008] In one example, a deep learning model is provided to quantify ventricular hypertrophy with precision equal to human experts, estimate left ventricular mass, and predict etiology of LVH. It should be noted that deep learning models may vary in terms of organization, structure, accuracy, input/output information, etc., and are referred to generally herein as “EchoNet-LVH”. Accordingly, while the term EchoNet-LVH may be used in conjunction with specific approaches herein, this is done by way of example and is in no way intended to limit the invention. [0009] Trained on a plurality of echocardiogram videos, EchoNet-LVH model accurately measures one or more of intraventricular wall thickness, left ventricular diameter, and posterior wall thickness. Further, the EchoNet-LVH model classifies cardiac amyloidosis and aortic stenosis from other etiologies of LVH. For example, through external datasets from independent international and domestic healthcare systems, EchoNet-LVH accurately quantified ventricular parameters and detected cardiac amyloidosis and aortic stenosis. EchoNet-LVH model utilizes measurements across multiple heart beats. Accordingly, EchoNet-LVH model can more accurately identify subtle changes in left ventricular (LV) geometry and its causal etiologies. Compared to human experts, EchoNet-LVH is more reproducible than human evaluation and provides improved precision in diagnosis of cardiac hypertrophy. [0010] In one example, a method for determining etiology of a heart structural health condition, comprises: acquiring one or more echocardiograms; inputting the one or more echocardiograms into an etiology prediction model; and generating an output indication including one or more predicted caused of the heart structural health condition; wherein the etiology prediction model is trained on a plurality of echocardiograms that have been identified to indicate left ventricular hypertrophy. [0011] In this way, leveraging measurements across multiple heart beats, the deep learning model can more accurately identify subtle changes in LV geometry and its causal etiologies. Compared to human experts, the deep learning workflow is fully automated, allowing for reproducible, precise measurements, and lays the foundation for precision diagnosis of cardiac hypertrophy. [0012] In one embodiment, a method for assessing structural condition of a heart of a patient includes: receiving echocardiogram data acquired by an echocardiogram system. The acquired echocardiogram data is used to determine one or more of a heart structural condition and one or more causes of the heart structural condition associated with the acquired echocardiogram data via a trained neural network model. Moreover, the trained neural-network model receives the echocardiogram data as an input (e.g., for continued training). The trained neural network model also includes a first neural network model for determining the heart structural condition and a second neural network model for predicting one or more causes of the heart structural condition. [0013] In some approaches, the first neural network model includes at least one atrous convolutional layer performing an atrous convolutional operation on the echocardiogram data. In other approaches, the first neural network model is a segmentation model configured to identify two or more structural key points on the input echocardiogram data. In still other approaches, the first neural network model is configured to determine the heart structural condition based on distances between at least two of the two or more structural key points. The first neural network model may also be trained based on sparse labeling of a training echocardiogram dataset. [0014] In other approaches, one or more of the first neural network and the second neural network models are optimized according to a number of skipped frames. Moreover, in some approaches, the heart structural condition is determined based on a desired scan plane, the desired scan plane based on parasternal long axis. [0015] The trained neural network model is trained in still other approaches based on one or more negative controls, the one or more negative controls including images from patients with other causes of LVH. [0016] A system, according to another embodiment, is for cardiac structure assessment. The system includes: at least one memory storing a trained neural network model and executable instructions, as well as at least one processor communicably coupled to the at least one memory. The trained neural network model is preferably including at least one atrous convolutional layer. Moreover, when executing the instructions, the processor is configured to: receive a set of echocardiogram frames of a patient from an echocardiogram system, and process the set of echocardiogram frames via the trained neural network model. This outputs a set of segmented echocardiogram frames which may be further processed to to output a set of segmented echocardiogram frames. These segmented frames may identify a plurality of key structural points on the set of segmented echocardiogram frames. The processor is also configured to obtain as output from the trained neural network model, an indication of a cardiac structural condition based on the set of segmented echocardiogram frames. Furthermore, the indication of the cardiac structural condition and/or segmented echocardiogram frames is displayed on a beat-by-beat basis. This may be achieved via a display portion of a user interface coupled to the at least one processor in some approaches. In some approaches the cardiac health condition may be determined on a beat- by-beat basis. [0017] The trained neural network model includes an etiology prediction classification model in some approaches based on a ResNet architecture. Moreover, the plurality of key structural points may be identified via corresponding heat-map representations in some approaches. The plurality of key structural points may also be based on centroids of the corresponding heat-maps. [0018] In some approaches, the trained neural network model is trained based on sparse labeling of a training echocardiogram dataset. A decision may also be made in some approaches regarding one or more causes of the cardiac structural condition via the trained neural network model. The one or more causes of the cardiac structural condition may include hypertension, aortic stenosis, hypertrophic cardiomyopathy, cardiac amyloidosis, etc. and/or combinations thereof. [0019] In still other approaches, the trained neural network model is trained for amyloid classification based on one or more negative training controls, the one or more negative training controls including images from patients with diagnosed HCM, aortic stenosis, hypertension, and ESRD. [0020] The above advantages and other advantages and features of the present description will be readily apparent from the following Detailed Description when taken alone or in connection with the accompanying drawings. It should be understood that the summary above is provided to introduce in simplified form a selection of concepts that are further described in the detailed description. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is defined uniquely by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS [0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, exemplify the embodiments of the present invention and, together with the description, serve to explain and illustrate principles of the invention. The drawings are intended to illustrate major features of the exemplary embodiments in a diagrammatic manner. The drawings are not intended to depict every feature of actual embodiments nor relative dimensions of the depicted elements, and are not drawn to scale. [0022] FIG.1 is a schematic diagram illustrating an overarching echocardiogram system for LVH assessment and corresponding LVH etiology prediction. [0023] FIG.2 is a schematic diagram illustrating an architecture of a convolutional neural network model for LVH assessment and corresponding LVH etiology prediction, which can be implemented in the echocardiogram system of FIG. 1, according to an embodiment of the disclosure; [0024] FIG.3 is a schematic diagram illustrating an architecture of an etiology prediction module included in the convolutional neural network model of FIG.2, according to an embodiment of the disclosure; [0025] FIG. 3B is a schematic diagram of a bottleneck module included in the inverted residual and max pooling module of FIG.3A, according to an embodiment of the disclosure; [0026] FIG. 4A shows correlation of human annotations vs. model predictions for ventricular dimensions in datasets from two independent healthcare systems (n = 2,320 for SHC (a first healthcare system) and n = 1,791 for ICL (a second healthcare system)). [0027] FIG. 4B shows model variation compared to human variation in annotation. Boxplot represents the median as a thick line, 25th and 75th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the median. [0028] FIG.4C shows receiver operating characteristic curves for diagnosis of amyloidosis on Stanford validation (n = 813) and test (n = 812) datasets; [0029] FIG. 4D shows correlation of human annotations vs. model predictions for ventricular dimensions in datasets from three healthcare systems (n = 1,200 for SHC, n = 1,309 for CSMC, and n = 1,791 for Unity). [0030] FIG.4E shows model variation on datasets from three healthcare systems compared to human clinical annotation variation. Boxplot represents the mean as a thick line, 25th and 75th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the mean. [0031] FIG. 4F shows Receiver operating characteristic curves for diagnosis of amyloidosis on Stanford validation (n = 813) and test (n = 812) datasets. HCM = Hypertrophic cardiomyopathy, SHC = Stanford Health Care, CSMC = Cedars-Sinai Medical Center. [0032] FIG.5A shows model prediction of key points (in green, purple, blue, and yellow) on individual frame of parasternal long video; [0033] FIG. 5B. shows frame-by-frame prediction of wall thickness and ventricular dimension and automated detection of systole and diastole allowing for beat-to-beat prediction of ventricular hypertrophy; [0034] FIG. 5C. shows waterfall plot of individual video variation in beat-to-beat evaluation of ventricular hypertrophy (n = 2,320) across the internal test dataset. Each video is represented by multiple points along a line representing the measurement of each beat and a line signifying range of predictions; [0035] FIGS.6A – 6C show performance of disease etiology classification. In particular, FIG. 6A shows receiver operating characteristic curves for detection of amyloidosis and aortic stenosis in Stanford test dataset (n = 812); FIG.6B. shows representative images for selected cases and controls for each etiology. FIG.6C shows precision-recall curves for detection of amyloidosis and aortic stenosis in Stanford test dataset (n = 812); [0036] FIGS. 6D – 6F show Performance of disease etiology classification across two independent domestic institutions. A. Receiver operating characteristic curves for detection of cardiac amyloidosis and aortic stenosis in SHC internal test dataset (n = 765) and CSMC external test set (n = 2351). B. Representative images for selected cases and controls for each etiology. C. Precision-recall curves for detection of amyloidosis and hypertrophic cardiomyopathy in SHC test dataset (n = 765). [0037] FIG. 7A – 7C example Hyperparameter Search of Video-based Classification Model. In particular, hyperparameter search for model architecture (R3D, which is used by EchoNet-LVH for hypertrophy etiology classification, R2+1D, and MC3), input video clip length (8, 16, 32, 64, 96 frames) and impact on model processing time and model memory usage are shown; [0038] FIG.8 is a high-level flow chart showing an example method for performing LVH etiology prediction, according to an embodiment of the disclosure; [0039] FIG. 9 is a high-level flow chart showing an example method for training a convolutional neural network model for etiology prediction, according to an embodiment of the disclosure; and [0040] FIGS. 10A – 10D are echocardiogram and corresponding plots showing beat-by- beat ventricular dimension assessment. [0041] FIGS. 11A and 11B show comparison of Model Performance with Human Variation. A. Correlation of EchoNet-LVH measurements vs. human annotations for ventricular dimensions in datasets from three healthcare systems (n = 1,200 for SHC, n = 1,309 for CSMC, and n = 1,791 for Unity). B. Correlation of clinician reported measurements compared to prior study for studies without significant change (n = 23,874 at SHC). [0042] FIGS. 12A and 12B show Comparison of Model Performance with Prospective Consensus Annotation of Two Level III Echocardiography Certified Cardiologists. [0043] In the drawings, the same reference numbers and any acronyms identify elements or acts with the same or similar structure or functionality for ease of understanding and convenience. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the Figure number in which that element is first introduced. DETAILED DESCRIPTION [0044] Unless defined otherwise, technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Szycher’s Dictionary of Medical Devices CRC Press, 1995, may provide useful guidance to many of the terms and phrases used herein. One skilled in the art will recognize many methods and materials similar or equivalent to those described herein, which could be used in the practice of the present invention. Indeed, the present invention is in no way limited to the methods and materials specifically described. [0045] In some embodiments, properties such as dimensions, shapes, relative positions, and so forth, used to describe and claim certain embodiments of the invention are to be understood as being modified by the term “about.” [0046] Various examples of the invention will now be described. The following description provides specific details for a thorough understanding and enabling description of these examples. One skilled in the relevant art will understand, however, that the invention may be practiced without many of these details. Likewise, one skilled in the relevant art will also understand that the invention can include many other obvious features not described in detail herein. Additionally, some well-known structures or functions may not be shown or described in detail below, so as to avoid unnecessarily obscuring the relevant description. [0047] The terminology used below is to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the invention. Indeed, certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section. [0048] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular implementations of particular inventions. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination. [0049] Similarly, while operations may be depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. [0050] Despite rapidly advancing developments in targeted therapeutics and genetic sequencing, persistent limits in the accuracy and throughput of clinical phenotyping has led to a widening gap between the potential and the actual benefits realized by precision medicine. This conundrum is exemplified by current approaches to assessing morphologic alterations of the heart. If reliably identified, certain cardiac diseases (e.g. cardiac amyloidosis and hypertrophic cardiomyopathy) could avoid misdiagnosis and receive efficient treatment initiation with specific targeted therapies. Systematic screening paradigms, including through imaging and automated chart feature review, have shown the opportunity to identify patients for underdiagnosed diseases increasingly recognized as more prevalent than previously thought. The ability to reliably distinguish between cardiac disease types of similar morphology but different etiology would also enhance specificity for linking genetic risk variants and determining mechanisms. [0051] Over the last two decades, the cost of genetic sequencing has dropped precipitously, making possible the application of genetics clinical practice and polygenic risk scores for complex diseases. While sequencing technology has improved cost-effectiveness and efficiency, clinical phenotyping has only risen in price and been stagnant in throughput. As such, phenotyping has become the bottleneck in future applications of genomics to clinical medicine, and the heterogeneity and imprecision of clinical measurements limit the power of genetic studies. While abundant, low cost, and without ionizing radiation, echocardiography is the most common form of cardiovascular imaging. However, echocardiography relies on expert interpretation and has measurement variation. Given the labor cost of clinical phenotyping and intra-clinician variability in annotation, previous genetic studies from echocardiography parameters have suggested genetic associations were small, and sometimes were not replicated. [0052] The heart is a dynamic organ capable of remodeling and adaption due to physiologic stress or extra-cardiac perturbation. Both intrinsic cardiac disease as well as systemic insults can result in left ventricular hypertrophy (LVH) and phenotypic mimics that is indistinguishable on routine imaging. Long standing hypertension and aortic stenosis can cause cardiac remodeling to compensate for additional physiologic work, while hypertrophic cardiomyopathy and cardiac amyloidosis can similarly manifest with an increase in left ventricular mass without the need to compensate for physiologic stress. Even with heterogeneity in measurement, there is a strong genetic component. [0053] In addition to the presence of LVH, the degree of ventricular thickness also has significant prognostic value in many diseases. Ventricular wall thickness is used to risk stratify patients for risk of sudden cardiac death and help determine which patients should undergo defibrillator implantation1. Nevertheless, quantification of ventricular thickness remains subject to significant intra- and inter-provider variability across imaging modalities. Even with the high image resolution and signal-to-noise ratio of cardiac magnetic resonance imaging, there is significant test-retest variability due to the laborious, manual nature of wall thickness measurement. Although abundant, low cost, and without ionizing radiation, echocardiography relies on expert interpretation and its accuracy is dependent on careful application of measurement techniques. [0054] Recent work has shown that deep learning applied to medical imaging can identify clinical phenotypes beyond conventional image interpretation and with higher accuracy than human experts. The inventors herein have recognized that echocardiography, when enhanced with artificial intelligence models, can provide additional value in understanding disease states by predicting both the presence of left ventricular hypertrophy in a screening population as well as the potential etiology of diagnosis. To overcome current limitations in the assessment of ventricular hypertrophy and disease diagnosis, systems and methods are provided for end-to-end deep learning approach for labelling the left ventricle dimensions, quantifying ventricular wall thickness, and predicting etiology of LVH. In one example, a method comprises performing frame- level semantic segmentation of the left ventricular wall thickness from parasternal long axis echocardiogram videos and then performing beat-to-beat evaluation of ventricular hypertrophy. After identifying left ventricular hypertrophy, a three-dimensional convolutional neural network with residual connections is used to predict the etiology of the LVH, including predictions for cardiac amyloidosis and aortic stenosis among a background of other hypertrophic diseases. [0055] In this way, deep learning may be applied to cardiac imaging to identify clinical phenotypes beyond conventional use of the imaging test and further provide higher accuracy than human expert interpretation. Further, echocardiography enhanced with artificial intelligence models can provide improved value in understanding disease states (e.g., etiology of diseases). [0056] In various implementations, methods and systems are provided for a deep learning approach to detect the presence of left ventricular hypertrophy in a screening population and classify potential etiology of diagnosis. In one example, EchoNet-LVH, an end-to-end deep learning model is provided for labelling the left ventricle, quantifying ventricular wall thickness, and predicting etiology of LVH. [0057] First, using parasternal long axis echocardiogram as input, a beat-by-beat assessment of ventricular hypertrophy is performed. In particular, a frame-level semantic segmentation of the left ventricular wall thickness for input parasternal long axis echocardiogram video is performed to determine beat-to-beat evaluation of ventricular hypertrophy. After identifying left ventricular hypertrophy, a three-dimensional convolutional neural network with residual connections predict the etiology of the LVH, including predictions for cardiac amyloidosis and aortic stenosis among a background of other hypertrophic diseases. [0058] A technical advantage of the convolutional network model, such as EchoNet-LVH, described herein is improved LVH assessment and prediction of one or more causes for LVH. In some examples, when LVH is not detected, a risk of LVH may be determined based on a degree of ventricular wall thickness. [0059] In one example, EchoNet-LVH is a deep learning system for the quantification of left ventricular hypertrophy on echocardiography and automated prediction of etiology of the hypertrophy. EchoNet-LVH achieves state-of-the-art performance in assessment of ventricular thickness and diameter, within the variance in clinical test-retest assessment, and aids in detection of subtle phenotypes particularly challenging for human readers. Integrating the steps of identifying ventricular hypertrophy and subsequent downstream prediction of etiology, EchoNet- LVH provides a fully automated workflow for disease screening in the most common form of cardiac imaging and can potentially greatly improve access to cardiovascular care. Whether used to triage patients for systemic therapy for cardiac amyloidosis or percutaneous valve replacement for aortic stenosis, EchoNet-LVH facilitate early expedited care end-stage heart disease with ventricular remodeling. [0060] In one example, EchoNet-LVH is tested in both domestic and international datasets not seen during model training, EchoNet-LVH’s performance in assessing ventricular thickness was robustly accurate without additional fine-tuning. These different testing regimens span across continents, clinical practice patterns, and instrumentation for image acquisition. EchoNet-LVH performs these tasks in real time with only one GPU; each prediction is more rapid than human assessment, allowing for real-time screening of cardiovascular disease in the clinic setting. Further, EchoNet-LVH is robust to variation in practice patterns across continents. [0061] The methods and systems herein provide an important improvement in assessment of cardiac structures in echocardiogram videos through deep learning. EchoNet-LVH augments current methods for assessing cardiac form and structure to provide more holistic evaluation of cardiovascular disease. With improved precision to detect ventricular remodeling and cardiac dysfunction, EchoNet-LVH enables earlier detection and treatment of subclinical cardiovascular disease. [0062] In this way, the deep neural network models described herein provide an important step towards phenotyping cardiac health evaluation. As a result, the convolutional neural network models described herein provide significant improvement in cardiac health evaluation and disease etiology, which in turn enables improved patient outcome. [0063] Referring to FIG.1, an echocardiogram system 100 is shown, in accordance with an exemplary embodiment. In some embodiments, as shown, the echocardiogram system 100 includes an echocardiogram processing system 102 which is communicatively coupled to an echocardiogram acquisition system 120 and user interface 130. Looking to the echocardiogram acquisition system 120 specifically, echocardiogram transducers 122 and at least one power supply 124 are included. In some approaches, at least some of the transducers 122 include ultrasound transducers, but any one or more desired types of echocardiogram transducers 122 may be used. Moreover, the power supply 124 preferably provides electrical power to the transducers 122. In one example, the system 120 may be a clinical grade echocardiogram system. In another example, the system 120 may be configured as a mobile ultrasound system. [0064] The power supply 124 and transducers 122 may thereby be able to capture echocardiogram data from a sample (e.g., patient) in proximity to the transducers 122. The echocardiogram data generated by the transducers 122 may further be stored in memory 123 at the echocardiogram acquisition system 120. Medical information, e.g., like the echocardiogram data generated by the transducers 122, is sensitive and is often associated with heightened scrutiny in terms of how it is stored. It follows that in some approaches, the process of storing the generated echocardiogram data in memory 123 includes converting the data into a different, and preferably more secure form. According to an example, which is in no way intended to limit the invention, generated echocardiogram data is encrypted prior to being stored in memory, thereby transforming the echocardiogram data from the general form it was created in, into a more secure form. The encrypted echocardiogram data may thereby be sent from the echocardiogram acquisition system 120 to the echocardiogram processing system 102 with minimal risk of the data being intercepted and improperly accessed. [0065] Depending on the approach, the encrypted echocardiogram data may be processed further before being sent between systems 120, 102. In some approaches, the encrypted data may be processed by an integrated processing unit (not shown) and subsequently transmitted via wired and/or wireless connections to the echocardiogram processing system 102 for cardiac structure evaluation. For instance, systems 120 and 102 may be connected over a network capable of transferring data that has been configured in a particular format. According to an example, which is in no way intended to limit the invention, the encrypted echocardiogram data may be divided into packets, each of which are uniquely numbered and associated with a destination address before being sent from the echocardiogram acquisition system 120 to the echocardiogram processing system 102. In some approaches having systems connected over networks like the Internet, the echocardiogram data may be divided into transmission control protocol/Internet protocol (TCP/IP) packets by a TCP layer, e.g., as would be appreciated by one skilled in the art after reading the present description. In response to receiving the encrypted data at the echocardiogram processing system 102, the data may be decrypted for further use. Additional processes may also be performed on the received data in some approaches, e.g., such as recombining packets of data. [0066] In some embodiments, the echocardiogram processing system 102 is disposed at a device (e.g., edge device, server, etc.) communicably coupled to the echocardiogram system 120 via wired and/or wireless connections. In some embodiments, the echocardiogram processing system 102 is disposed at a separate device (e.g., a workstation) which can acquire echocardiogram data from the echocardiogram system 120 or from a storage device which stores the echocardiogram data acquired by the echocardiogram acquisition system 120. [0067] The echocardiogram processing system 102 may comprise at least one processor 104, and be in communication with a user interface 130. The user interface 130 may include a user input device (not shown), and a display device 132. User input devices that may be used include one or more of a touchscreen, a keyboard, a mouse, a trackpad, a motion sensing camera, etc., or other device configured to enable a user to interact with and manipulate data within the echocardiogram system 100. [0068] The at least one processor 104 is preferably configured to execute machine readable instructions which may be stored in non-transitory memory 106. The processor 104 may be single core or multi-core, and the programs executed thereon may be configured for parallel or distributed processing. In some embodiments, the processor 104 may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and/or configured for coordinated processing. In some embodiments, one or more aspects of the processor 104 may be virtualized and executed by remotely-accessible networked computing devices configured in a cloud computing configuration. According to other embodiments, the processor 104 may include other electronic components capable of carrying out processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphic board. According to other embodiments, the processor 104 may include multiple electronic components capable of carrying out processing functions. For example, the processor 104 may include two or more electronic components selected from a list of electronic components including: a central processor, a digital signal processor, a field-programmable gate array, and a graphic board. [0069] In still further embodiments the processor 104 may be configured as a graphical processing unit (GPU) including parallel computing architecture and parallel processing capabilities. However, it will be appreciated that a trained neural network model as described herein may be implemented in a processor that does not have GPU processing capabilities. [0070] Non-transitory memory 106 may store a pre-processing module 108, a ventricular dimension assessment module 110, an etiology prediction module 111, and echocardiogram data 112. The ventricular dimension assessment 110 may include a neural network model comprising a plurality convolutional layers. The module 110 may further include instructions for implementing the neural network model to receive an echocardiogram video data of a patient acquired and output a corresponding LVH classification (e.g., presence or absence of LVH). For example, the module 110 may store instructions for implementing a neural network model, such as an exemplary model 210 shown at FIG. 2. The module 110 may include trained and/or untrained neural networks and may further include various data, or metadata pertaining to the one or more neural networks stored therein. [0071] Non-transitory memory 106 may further store training module 114, which comprises instructions for training one or more neural network models stored in the module 110. Training module 114 may include instructions that, when executed by processor 104, cause echocardiogram processing system 100 to conduct one or more of the steps of method 900 (FIG. 9) for training one or more neural network models with corresponding training data sets, discussed in more detail below. In some embodiments, training module 114 includes instructions for implementing one or more gradient descent algorithms, applying one or more loss functions, and/or training routines, for use in adjusting parameters of one or more neural network models of the module 110. [0072] Non-transitory memory 106 also stores an inference module 116 that comprises instructions for validating and testing new data with the trained neural network model. Non- transitory memory 106 further stores echocardiogram data 112. In some embodiments, echocardiogram data 112 may include a plurality of training sets, each comprising a plurality of echocardiograms. [0073] In some embodiments, the non-transitory memory 106 may include components disposed at two or more devices, which may be remotely located and/or configured for coordinated processing. In some embodiments, one or more aspects of the non-transitory memory 106 may include remotely-accessible networked storage devices configured in a cloud computing configuration. [0074] Non-transitory memory 106 may further store a model interpretable explanation generator module 118. The model interpretable explanation generator module 118 may include an interpretable model configured to provide one or more visual indications for explaining each individual prediction of the neural network model for LVH and etiology classification. The interpretable model may be based on a model interpretability technique (e.g., Local Interpretable Model-Agnostic Explanations (LIME)) provides data that explains how a model made a determination. The interpretable model may be a local interpretable model. In some examples, a global interpretable model may be used. Thus, via the model interpretable explanation generator, one or more important echocardiogram features having a weightage greater than a threshold weightage and contributing to LVH and corresponding etiology prediction may be identified and indicated. While the above examples describe using the LIME technique for interpreting the neural network model output, other interpretability techniques, such as Shapely Additive Explanations, may be used and are within the scope of the disclosure. [0075] Display 132 may include one or more display devices utilizing virtually any type of technology. In some embodiments, display device 132 may comprise a computer monitor, and may display unprocessed and processed echocardiogram frames. Display device 132 may be combined with processor 104, non-transitory memory 106, and/or user input device in a shared enclosure, or may be peripheral display devices and may comprise a monitor, touchscreen, projector, or other display device known in the art, which may enable a user to view echocardiogram frames, and/or interact with various data stored in non-transitory memory 106. In some approaches, the display 132 is used to display the indication of a cardiac structural condition and/or segmented echocardiogram frames on a beat-by-beat basis. [0076] It should be understood that echocardiogram processing system 100 shown in FIG. 1 is for illustration, not for limitation. Another appropriate image processing system may include more, fewer, or different components. [0077] The echocardiogram processing system 100 may be used to train and deploy a neural network model, such as an example neural network model discussed below at FIG.2. Neural Network Architecture. [0078] Turning to FIG.2, it shows a high-level block diagram of a neural network model 200 that may be implemented for identifying LVH and subsequently, classifying one or more causes (e.g., etiology-based causes) for LVH. According to some approaches, the neural network model 200 includes EchoNet-LVH. The neural network model 200 comprises a first neural network model which is depicted in the present approach as being a ventricular dimension assessment model 210, but other types of neural network models which would be apparent to one skilled in the art after reading the present description may be implemented in other approaches. [0079] The ventricular dimension assessment model 210 itself may also be configured to classify a cardiac structural condition (e.g., such as LVH) based on one or more heart structural parameters. With respect to the present description, the “cardiac structural conditions” may be referred to specifically as herein as a LVH condition, but this is done by way of example, and is in no way intended to be limiting. The heart structural parameters may include one or more of the intraventricular septum (IVS), left ventricular internal dimension (LVID), and left ventricular posterior wall (LVPW). In one example, the ventricular dimension assessment model 210 includes a weak supervision layer 220 and an atrous convolution layer 230. The ventricular dimension assessment model 210 receives echocardiogram frames that have been captured (e.g., created) by a system having the components associated with actually capturing the frames (e.g., see 120 of FIG. 1 above). These frames are passed through the weak supervision layer 220 which may be used to apply different types of supervision (e.g., labeling functions) to the echocardiogram frames and/or information included therein to form numerous unsupervised labeled data sets. From there, the frames are sent through layer 225 which combines information from either one frame or multiple frames across a video, of which some can be labeled and others are not labeled. The echocardiogram frames are also passed through an atrous convolution layer 230 which may be used to apply one or more dilated convolutions to the data sets, e.g., to increase the field of view which is particularly valuable in situations involving real-time segmentation. The frames are additionally input to layer 235 which produces a heat map or probability function of regions were key points may be that can be averaged to a maximally likely key point. From layer 235, the echocardiogram frames are output with corresponding key points identified in the frames themselves 240. These key points may actually be structural key points in some instances, any may be used to measure one or more ventricular dimensions, e.g., as will be described in further detail below. [0080] The model 210 may be trained for semantic segmentation of parasternal long axis (PLAX) echocardiogram videos and identification of one or more of IVS, LVID, and LVPW. With atrous convolutions to capture longer range features, full resolution PLAX frames may be used as input images for higher resolution assessment of LVH. Accordingly, the echocardiogram frames having the corresponding key points identified may further be provided to an additional assessment module 250 capable of performing assessments associated with LVH. It follows that the assessment module 250 may be able to utilize key points identified in the frames to determine a presence and/or type of LVH that may be affecting the heart that was examined to produce the echocardiogram frames (i.e., the heart shown in the echocardiogram frames). An etiology prediction model 290 may also be implemented before an etiology classification is output, e.g., as will be described in further detail below with respect to FIG.3. [0081] Due to the tedious nature of annotation, labels may only be provided for a subset of the frames in a video (e.g., one or two frames of a video) by the weak supervision layer 220. However, the assessment model 210 may transform these sparse annotations into measurement predictions for every frame of the entire video to allow for beat-to-beat estimation of ventricular wall thickness and dimensions. An example method for performing beat-by-beat ventricular assessment is described by Ouyang, D., He, B., Ghorbani, A. et al. in “Video-based AI for beat- to-beat assessment of cardiac function” published in Nature 580, 252–256 (2020), the content of which is incorporated by reference herein in its entirety. [0082] In one embodiment, the ventricular dimension assessment model 210 is a segmentation model, which is trained to identify a plurality of key points on the echocardiogram frames and measure one or more ventricular dimensions using the plurality of key points. In one example four key points are used. Further, one or more ventricular dimensions are measured on a beat-by-beat basis. Example video frames showing the key points and corresponding plots are shown in FIGS.10A – 10D. In this way, accuracy and speed of ventricular dimension measurement is improved, thereby reducing overall compute time, processing bandwidth, network traffic, etc. In other approaches trained neural network models may be trained based on one or more negative controls. Depending on the approach, these negative controls may include images from patients with different causes of LVH. In other approaches, the one or more negative training controls may include images from patients with diagnosed HCM, aortic stenosis, hypertension, and/or ESRD. It follows that the neural network model may be trained for amyloid classification based on one or more negative training controls in some approaches. [0083] The model 200 may be stored in a memory of a processing system, e.g., such the echocardiogram processing system 100 at FIG.1. The neural network model 200 may receive as input, echocardiogram data of a patient, and output one or more of an indication of a LVH condition and one or more etiology of the LVH condition. In one example, the echocardiogram data may be echocardiogram frames derived from one or more examination procedures performed on the patient, and these frames may be pre-processed prior to passing through the neural network model 200. Pre-processing echocardiogram waveform data may include filtering, for example. In some examples, filtering may be performed to remove very low quality video signals. [0084] It follows that by pre-processing echocardiogram waveform data, the approaches herein are able to desirably reduce the unnecessary consumption of computational resources. In other words, because pre-processing echocardiogram data can remove low quality video signals, the approaches herein are desirably able to reduce the amount of processing power that is dedicated to interpreting subpar data. In fact, if this subpar data were to actually be evaluated and incorporated in the training, accuracy of the overall deep learning model may be sacrificed. Pre- processing is thereby able to further improve accuracy, while also reducing the computational throughput associated with achieving this improved accuracy. Accordingly, the neural network model 200, as well as other approaches included herein (e.g., see 108 of FIG.1 above), are able to implement processes that ultimately improve operation of the computing components included therein, e.g., as would be appreciated by one skilled in the art after reading the present description. In any case, pre-processing of the input echocardiogram frames may be based on the pre- processing operations performed during training the neural network model. [0085] The model 200 may include a second video-based convolutional neural network (CNN) model. The second video-based CNN model may be an etiology prediction model 290 configured to classify etiology of LVH. The details of the etiology model 290 are shown at FIG.3 in accordance with an exemplary approach which are in no way intended to limit the invention. As an option, the present etiology model 290 may be implemented in conjunction with features from any other embodiment listed herein, such as those described with reference to the other FIGS., such as FIG. 2. However, such etiology model 290 and others presented herein may be used in various applications and/or in permutations which may or may not be specifically described in the illustrative embodiments listed herein. Further, the etiology model 290 presented herein may be used in any desired environment. Thus FIG. 3 (and the other FIGS.) may be deemed to include any possible permutation. [0086] Looking now to FIG. 3, the etiology prediction model 290 may include layers performing spatiotemporal convolutions to predict etiology of LVH. Integrating spatial as well as temporal information, the model 200 combines LVH assessment with video-based model interpretation of echocardiograms and classify videos based on probability of hypertension, aortic stenosis, hypertrophic cardiomyopathy, or cardiac amyloidosis as etiology of ventricular hypertrophy. For instance, spatial and temporal information may be extracted from a graphical representation 302 of the relevant data. This extracted information may be passed to a first spatial convolution module 304, prior to a number of convolution layers 306, 308, 310, 312. As shown, each of these convolution layers 306, 308, 310, 312 may further include a desired number and order of spatial and/or temporal sub-layers 316, 318 respectively. From the convolution layers 306, 308, 310, 312, the processed information is provided to a spatiotemporal pool 314 which may be used to collect and organize the information, e.g., based on relationships that have been determined and/or defined. [0087] Further, during training, a hyperparameter search may be conducted to identify the optimal base architecture for EchoNet-LVH, e.g., as seen in the graphs depicted in FIGS.7A – 7C. Performing a hyperparameter search for a model architecture (e.g., such as R3D, which may be used by EchoNet-LVH for hypertrophy etiology classification, R2+1D, and MC3), input video clip length (8, 16, 32, 64, 96 frames) and impact on model processing time and model memory usage. [0088] Inventors have identified that by performing hyperparameter sweep on frame rate, speed and accuracy of etiology prediction is improved. Accordingly, in one example, the etiology prediction model is a trained neural network model, wherein the training is based on a hyperparameter sweep on frame rate to determine a number of frames to skip. [0089] The dataset for training the ventricular dimension assessment model typically includes of a plurality of echocardiograms. Each echocardiogram preferably includes a plurality of frames, and includes at least two annotated frames, one at systole and one at diastole. LVID measurements may be annotated in both systole and diastole frames, while IVS and LVPW measurements are may be made once at diastole. The labels used for training the assessment model are four channel images that contain all zeros except for the pixels closest to the endpoints of measurements being ones. [0090] To generate PLAX measurements, PLAX echo frames may be fed through the ventricular dimension assessment model to predict heat-maps representing the location of each of the four measurement points. In some approaches, the model is initialized with random weights and trained for a desired number of epochs using an optimizer with an initial learning rate. To add domain diversity, random scale, shift and rotate data augmentation may be applied to both input images and labels. The centroid of each of the output channels may thereby be calculated as the predicted location for each point. The LVPW, LVID, and IVS may further be calculated using these four measurement points. [0091] Because the labels contain many more ones than zeros, to optimize semantic segmentation for sparse labels, a modified weighted Mean Square Error (MSE) loss function was used that weighs the loss from false positives and false negatives differently, more aggressively penalizing false negative predictions. For instance, in some approaches Equation 1 below may be used as a modified weighted MSE loss function to weighs the losses associated with false positives and negatives differently. By varying alpha “α”, the amount that false positive labels are penalized can be scaled and prediction size can be optimized. Further, predictions based on distance from original point labels may be penalized and anatomic measurements may be derived. Loss is a summation of the difference between the label and model prediction at each pixel which is scaled by alpha such that the relative weighting of importance for negative and positively labeled pixels are optimized. In sparsely labeled images or videos, the positive pixels can be heavily weighted to prioritize model performance for those good performance on those pixels while negative pixels can be relatively unprioritized. Early stopping may be performed based on validation dataset’s loss. ^
Figure imgf000026_0001
Equation 1 [0092] By decreasing alpha α, how much the false positives are penalized may be decreased and thus drive the model to predict larger heat-maps. Using the centroid of the heat- maps to predict the measurement end points serves as a relatively accurate approximation. A benefit of using the centroid, is that it is differentiable. This means that downstream operations can be performed that can inform gradients to the model. With this in mind, the MSE loss for the points predicted by the centroids relative to the labelled points is added to the loss function. This supplemental loss highly penalizes outliers far from the true point. Adding this loss virtually eliminated the underestimate problem and further improved accuracy of the system as a whole. [0093] The centroid of each of the output channels was also calculated as the predicted location for each point. The LVPW, LVID, and IVS are calculated from these four points. [0094] Beyond calculating measurements at systole and diastole as a cardiologist would, using machine learning to process information like echocardiograms has the benefit of being able to label an entire video and calculate other practical values associated therewith. For instance, in addition to being able to determine relevant measurements for every frame, the displacement of each of the points along the measurement axis over time may also be calculated to show how the heart is moving over time. [0095] In one example, training dataset for ventricular dimension assessment model comprises sparsely annotated echocardiograms. In this example, the weak supervision refers to the fact that in each video, only 1 or 2 frames (e.g., only about 0.5 – 1% of the data) are actually labeled. Despite this sparse amount of information, machine learning processes described herein are able to utilize these labeled frames.. For instance, in some approaches, a weighted loss function is used. Further, false positives/negatives are weighted and a hyperparameter sweep of the weighted parameters may be performed. [0096] Example 1 [0097] It should be noted that the following example is presented for illustrative purposes and is in no way intended to limit the scope of the invention. [0098] Evaluation of Hypertrophy Detection [0099] For the held-out test dataset not seen during model training, EchoNet-LVH predicted ventricular dimensions with a coefficient of determination (R2) value of 0.97. This coefficient is typically determined by measuring the amount of variance in the results produced by a given dataset. In other words, R2 it is the difference between the samples in a dataset and the predictions made by the deep learning model using the dataset. Looking to the results depicted in FIGS. 5A, 5B, and 5C, applying EchoNet-LVH in the present example also resulted in a mean absolute error (MAE) of 1.2mm for IVS, 2.4mm for LVID, and 1.4mm for LVPW. This compares favorably with clinical inter-provider variation, which has produced a MAE of 1.3mm for IVS, 3.7mm for LVID, and 1.3mm for LVPW. [00100] To assess the cross-healthcare-system and international reliability of the model, EchoNet-LVH was additionally tested, without any tuning, on an external test dataset of 1,791 videos from Unity Collaborative and 13,796 videos from Cedars-Sinai Medical Center (CSMC). On the Unity external test dataset, EchoNet-LVH showed a robust prediction accuracy with an overall R2 of 0.90, MAE of 1.6mm for IVS, 3.6mm for LVID, and 2.1cm for LVPW. While both SHC and ICL datasets were chosen for direct comparison, matching frame of the video was annotated, a comparison of beat-to-beat evaluation on the CSMC dataset was performed by masking which frame was annotated and only providing the study-level annotations of ventricular dimensions. On the CSMC external test dataset, EchoNet-LVH showed a robust prediction accuracy with an overall MAE of 3.5mm for IVS, 11.6 for LVID, and 3.1 for LVPW. [00101] Prediction of Etiology of Hypertrophy [00102] After detection of LVH, a natural question arises of the etiology of LVH, as treatments vary drastically whether the hypertrophy is due to aortic stenosis, cardiac amyloidosis, and long-standing hypertension. The etiology derivation, validation, and test cohorts from SHC had 6,496, 813, and 812 videos respectively. On the held-out test cohort, EchoNet-LVH classifies cardiac amyloidosis with an area under the curve (AUC) of 0.83 and aortic stenosis with an AUC of 0.89 from other etiologies of LVH. On an external test dataset, videos from CSMC with 358 videos of cardiac amyloidosis, 146 videos of aortic stenosis, 468 videos of hypertrophic cardiomyopathy, and 1,379 videos of other etiologies of LVH, EchoNet-LVH had an AUC of 0.79 for predicting cardiac amyloidosis. [00103] Data Curation [00104] A standard full resting echocardiogram study may include of a series of about 50- 100 videos, as well as still images visualizing the heart from different angles, locations, and image acquisition techniques (2D images, tissue Doppler images, color Doppler images, and others). Patients were identified by physician curated cohorts from the Stanford Amyloid Center and CSMC Advanced Heart Disease Clinic for cardiac amyloidosis and the Stanford Center for Inherited Cardiovascular Disease and CMSC Hypertrophic Cardiomyopathy Clinic for hypertrophic cardiomyopathy. In this study, relevant PLAX and apical-4-chamber (A4C) 2D videos were extracted from each study. Human expert annotations of IVS, LVID, and LVPW were used as training labels to assessing ventricular hypertrophy. PLAX videos were split 18,582, 2,310, and 2,320 patients respectively for the training, validation, and test sets. An additional 8,448 studies of patients with defined disease characteristics, including cardiac amyloidosis, severe aortic stenosis, hypertension, and other phenotypes of LVH were identified through the electronic healthcare system. From these studies, the A4C videos were extracted and used as input data for the hypertrophic disease classification task. Videos were processed in a previously described automated preprocessing workflow removing identifying information and human labels. Videos were spot checked for quality control, confirm view classification, and exclude videos with color Doppler. [00105] EchoNet-LVH Development and Training [00106] Model design and training was done in Python using the PyTorch deep learning library. The network architecture was trained on parasternal long axis images to minimize a weighted mean square error loss was used to identify key points used for measuring ventricular dimensions. An Adam optimizer was used with a learning rate of 0.001 was used and the model was trained for 50 epochs with early stopping based on the validation loss. For video-based disease classification, an 18-layer ResNet 3D24 architecture was used to classify videos as either amyloid or not amyloid. This model was trained to minimize binary cross-entropy loss using an Adam optimizer with a learning rate of 0.01. The model was trained for 100 epochs with a batch size of 14 with early stopping based on AUC on the validation set. Different video lengths, resolutions, and temporal resolutions as hyperparameters were evaluated to optimize model performance (FIGS.7A – 7C). Computational cost was evaluated using one NVIDIA GeForce GTX 3090. [00107] Test Time Augmentation with Beat-by-Beat Assessment [00108] For PLAX measurement prediction, test-time augmentation was performed with aggregating predictions across the entire echocardiogram video. The predicted LVID measurement was used to determine frames of peak systole and peak diastole. These frames were used to generate systolic and diastolic measurements for every beat of an echocardiogram video. These measurements were compared to the human labelled measurements (FIG.5C). Variation from beat to beat in a single echocardiogram is used to evaluate the precision of the method. [00109] Comparison with Human assessment [00110] Using the reporting database of Stanford Echocardiography Laboratory, paired studies of the same patient were identified for which the reviewing cardiologist determined there was no significant change from the current study to the prior study. Of these studies with clinical stability, the subset of 23,874 studies were analyzed for which left ventricle internal dimension diastole (LVIDd), intraventricular septum diastole (IVSd), and left ventricular posterior wall diastole (LVPWd) was measured for both the current and subsequent study. The variance in measurement between the previous and subsequent study was used as a surrogate of clinical expert variation and compared with EchoNet-LVH variation. [00111] It should be noted that while results indicating how accurately various ones of the approaches herein are able to operate in identifying conditions, are presented in addition to related accuracy rates of experts in the respective medical fields. However, this is in no way intended to be limiting, and rather is providing context in terms of the accuracy the various approaches herein are able to achieve. As noted herein, various processes included herein are able to significantly improve operating efficiency of the system as a whole, while maintaining accuracy. This is particularly desirable in view of the shortcomings experienced by conventional systems. [00112] Domestic and International External Health Care System Test Datasets [00113] Transthoracic echocardiogram studies from Cedars-Sinai Medical Center were used to evaluate EchoNet-LVH’s performance in predicting ejection fraction. Previously described methods were used to identify apical-4-chamber view videos22. The same automated preprocessing workflow was used to convert DICOM files to AVI files, mask information outside of the scanning sector, and resize input to 112x112 pixel videos of variable length. After manual exclusion of incorrect view classifications, long cine loops of bubble studies, videos with injection of ultrasonic contrast agents, and videos with color doppler, a plurality of videos were identified from a plurality of patients. Labeled echocardiogram images from the Unity Imaging Collaborative were used as a separate held-out test data not seen during model training. Given that EchoNet- LVH was trained on a separate training set, the entirety of the Unity Imaging dataset with PLAX annotations were used as an international external test set. The echocardiogram videos were already pre-processed to identify labeled frames and saved as PNG files [00114] Statistical Analysis [00115] Confidence intervals were computed using 10,000 bootstrapped samples and obtaining 95 percentile ranges for each prediction. The performance of the semantic segmentation task was evaluated using the Dice Similarity Coefficient compared to human labels in the hold- out test dataset. The performance of ejection fraction task was evaluated by calculating the mean absolute difference between EchoNet-Dynamic’s prediction and the human calculation of ejection fraction as well as calculating the R2 between EchoNet-Dynamic’s prediction and the human calculation. Prospective comparison with human readers was performed with the uniformly most powerful invariant equivalence test for two-sample problems. [00116] In summary, trained on 28,201 echocardiogram videos, EchoNet-LVH model accurately measures intraventricular wall thickness (mean absolute error 1.4mm, 95% CI 1.2- 1.5mm), left ventricular diameter (MAE 2.4mm, 95% CI 2.2-2.6mm), and posterior wall thickness (MAE 1.2cm, 95% CI 1.1-1.3mm) and classifies cardiac amyloidosis (AUC of 0.83) and aortic stenosis (AUC 0.89) from other etiologies of LVH. Through external datasets from independent international and domestic healthcare systems, EchoNet-LVH accurately quantified ventricular parameters (R2 of 0.90) and detected cardiac amyloidosis (e.g., AUC 0.79) and aortic stenosis. Leveraging measurements across multiple heart beats, EchoNet-LVH model can more accurately identify subtle changes in left ventricular (LV) geometry and its causal etiologies. Compared to human experts, EchoNet-LVH is more reproducible than human evaluation and provides improved precision in diagnosis of cardiac hypertrophy. [00117] Example 2 [00118] It should be noted that the following example is presented for illustrative purposes and is in no way intended to limit the scope of the invention. [00119] Methods [00120] Data Curation [00121] A standard full resting echocardiogram study may include a series of about 50-100 videos, as well as still images visualizing the heart from different angles, locations, and image acquisition techniques (2D images, tissue Doppler images, color Doppler images, and others). Patients were identified by physician curated cohorts from the Stanford Amyloid Center and CSMC Advanced Heart Disease Clinic for cardiac amyloidosis and the Stanford Center for Inherited Cardiovascular Disease and CMSC Hypertrophic Cardiomyopathy Clinic for hypertrophic cardiomyopathy. [00122] In this study, relevant PLAX and A4C 2D videos were extracted from each study. Human clinician annotations of IVS, LVID, and LVPW were used as training labels to assess ventricular hypertrophy. From SHC, PLAX videos were split 9,600, 1,200, and 1,200 patients respectively for the training, validation, and test sets. An additional 7,767 SHC studies were of patients with defined disease characteristics, including cardiac amyloidosis, hypertrophic cardiomyopathy, and severe aortic stenosis. From these studies, the A4C videos were extracted and used as input data for the hypertrophic disease classification task. Videos were processed in a previously described automated preprocessing workflow removing identifying information and human labels. This research was approved by the Stanford University and Cedars-Sinai Medical Center Institutional Review Boards. [00123] Domestic and International External Health Care System Test Datasets [00124] Transthoracic echocardiogram studies from CSMC and the Unity Imaging Collaborative were used to evaluate performance of the various approaches included herein in identifying key points in PLAX videos and measuring ventricular dimensions. Previously described methods were used to identify PLAX and apical-4-chamber view videos and convert DICOM files to AVI files. In total, 3,660 total videos were extracted from CSMC as a domestic held out test dataset. Labeled echocardiogram images from the Unity Imaging Collaborative were used as an additional held-out international test dataset not seen during model training. These echocardiogram videos were obtained from British echocardiography labs, retrospectively annotated by echocardiography certified cardiologists. [00125] Deep Learning Algorithm Development and Training [00126] Model design and training was done in Python using the PyTorch deep learning library. A modified DeepLabV326 architecture trained on parasternal long axis images to minimize a weighted mean square error loss was used to identify key points used for measuring ventricular dimensions. 3D implementations of segmentation model took substantially more computational resources without significant improvement in performance. An Adam optimizer was used with a learning rate of 0.001 was used and the model was trained for 50 epochs with early stopping based on the validation loss. Different video lengths, resolutions, and temporal resolutions were evaluated as hyperparameters to optimize model performance. Computational cost was evaluated using one NVIDIA GeForce GTX 3090. [00127] For video-based disease classification, an 18-layer ResNet 3D architecture was used to classify videos. Given the potential for overlap patients with multiple etiological diagnoses for LVH28, parallel binary classification deep learning models were trained to predict probability of amyloid, hypertrophic cardiomyopathy, aortic stenosis, secondary to uncontrolled hypertension, and in setting of end-stage kidney disease (ESRD) independently. Distinct from prior literature, for each classification task, the negative controls were images from patients with other causes of LVH to mimic the clinical workflow. For example, during amyloid classification, the negative training examples included patients with diagnosed HCM, aortic stenosis, hypertension, and in setting of ESRD as other etiologies of LVH. This model was trained to minimize binary cross- entropy loss using an Adam optimizer with a learning rate of 0.01. The model was trained for 100 epochs with a batch size of 14 with early stopping based on AUC on the validation set. [00128] Comparison with Variation in Human Measurement [00129] Using the reporting database of Stanford Echocardiography Laboratory, paired studies of the same patient were identified, for which the reviewing cardiologist determined there was no significant change from the current study to the prior study by structured reporting element. Of these studies with clinical stability, the subset of 23,874 studies were analyzed for which LVID, IVS, and LVPW at diastole was measured for both the current and subsequent study. The variance in measurement between the previous and subsequent study was used as a surrogate of clinician variation and compared with the variation of the approaches described herein. On the CSMC dataset, 99 random studies were identified and blinded re-labeling was performed by two level III echocardiography certified cardiologists and compared the performance against the performance of the deep learning algorithms disclosed herein on the consensus label. [00130] Statistical Analysis [00131] Confidence intervals were computed using 10,000 bootstrapped samples and obtaining 95 percentile ranges for each prediction. The performance of the semantic segmentation task was evaluated comparing the length of LVID, LVPW, and IVS to human labels in the hold- out test dataset. The centroid of each predicted key point was used to calculate measurements. [00132] Results [00133] Deep learning workflows and other information associated with screening of hypertrophic cardiomyopathy and cardiac amyloidosis in the various approaches herein (e.g., such as those depicted in FIGS. 1-4C) preferably have two components. First, that the deep learning model is provided with atrous convolutions for semantic segmentation of PLAX various echocardiogram videos, and identification of the IVS, LVID, and LVPW. With atrous convolutions to capture longer range features, full resolution PLAX frames were used as input images for higher resolution assessment of LVH in some illustrative approaches. [00134] For instance, FIG. 4A includes a graph showing a correlation between human annotations and model predictions for ventricular dimensions in datasets from two independent healthcare systems. A first of the datasets has a sample size of n = 2,320 and corresponds to Stanford Health Care (SHC), while the other of the datasets has a sample size of n = 1,791 and corresponds to ICL. Moreover, FIG.4B shows model variation compared to human variation in annotation. Boxplot represents the median as a thick line, 25th and 75th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the median. [00135] The graph in FIG.4C shows receiver operating characteristic curves for diagnosis of amyloidosis on Stanford validation (n = 813) and test (n = 812) datasets. FIG.4D graphically depicts the correlation between human annotations and model predictions using various ones of the approaches included herein. The data displayed in the graph was derived from ventricular dimensions in datasets from three different healthcare systems. A first of the datasets had a sample size of n = 1,200 and corresponded to SHC, another of the datasets had a sample size of n = 1,309 and corresponded to CSMC, while the remaining dataset had a sample size of n = 1,791 and was associated with Unity. [00136] Looking now to FIG.4E, model variation is depicted with respect to datasets from three healthcare systems compared to human clinical annotation variation. As noted above, the boxplot graph represents the mean as a thick line, 25th and 75th percentiles as upper and lower bounds of the box, and individual points for instances greater than 1.5 times the interquartile range from the mean. Furthermore, FIG.4F shows receiver operating characteristic curves for diagnosis of amyloidosis on Stanford validation (n = 813) and test (n = 812) datasets. [00137] Given the tedious nature of annotation, the standard clinical workflow often only labels in one or two frames of a video, while each video records multiple heart beats that can be used for clinical measurements (e.g., as seen in FIGS.5A, 5B, and 5C). Therefore, a neural network as described in the various approaches herein, may generally be trained on these sparse annotations, and be able to make accurate measurement predictions for every frame of the entire video in dynamically efficient manner to allow for beat-to-beat estimation of ventricular wall thickness and dimensions. Pre-processing the sparse annotations may be able to further improve accuracy, while also reducing the computational throughput associated with achieving this improved accuracy. Accordingly, the neural network models in the various approaches herein are desirably are able to implement processes that ultimately improve operation of the computing components included therein, e.g., as would be appreciated by one skilled in the art after reading the present description. [00138] After detection of LVH, identifying the specific etiology (e.g. infiltrative disease, inherited cardiomyopathies, chronic elevated afterload, etc.) can help guide therapy. According to one example, a video-based CNN model was trained with spatiotemporal convolutions to predict etiology of LVH. FIGS. 6A – 6C show performance of disease etiology classification based on this example. In particular, FIG. 6A shows receiver operating characteristic curves for detection of amyloidosis and aortic stenosis based on a Stanford test dataset with a sample size of n = 812. FIG. 6B. shows representative images for selected cases and controls for each etiology. Furthermore, FIG. 6C shows precision-recall curves for detection of amyloidosis and aortic stenosis based on a Stanford test dataset with a sample size of n = 812 as well. [00139] FIGS. 6D – 6F show performance of disease etiology classification across two independent domestic institutions. The receiver operating characteristic curves for detection of cardiac amyloidosis and aortic stenosis corresponded to an internal test dataset from SHC having a sample size of n = 765, and an external test dataset from CSMC having a sample size of n = 2351. It should also be noted that representative images for selected cases and controls for each etiology are presented in FIG.6E, but are in no way intended to be limiting. Precision-recall curves for detection of amyloidosis and hypertrophic cardiomyopathy corresponding to analyzing a test dataset from SHC having a sample size of n = 765 are also depicted in FIG.6F. [00140] By integrating spatial as well as temporal information, the model expands the video-based model interpretation of echocardiograms and classifies videos based on probability of hypertension, aortic stenosis, hypertrophic cardiomyopathy, or cardiac amyloidosis as etiology of ventricular hypertrophy. Additionally, a video-based model architecture and hyperparameter search was performed to identify desirable base architecture for the deep learning algorithm. For instance, FIGS.7A, 7B, and 7C depict AUC with respect to various chip lengths, time steps, and image resolutions, respectively. The deep learning algorithm according to the present example was trained on a dataset of 17,802 echocardiogram videos from Stanford Health Care (SHC), and then evaluated on held out test cohorts from SHC, CSMC, and Unity Imaging Collaborative. [00141] Evaluation of Hypertrophy Detection [00142] From the held-out test dataset from SHC (n = 1,200) not seen during model training, the deep learning algorithm predicted ventricular dimensions with a R2 of 0.97 compared to annotations by human experts, e.g., as depicted in FIGS. 11A and 11B). The deep learning algorithm (EchoNet-LVH) in the present example had a mean absolute error (MAE) of 1.2mm for IVS, 2.4mm for LVID, and 1.4mm for LVPW. This compares favorably with clinical inter- provider variation, which had a MAE of 1.3mm for IVS, 3.7mm for LVID, and 1.3mm for LVPW. EchoNet-LVH also performed desirably when compared to the prospective consensus annotation of two level 3 echocardiography certified cardiologists in 99 random studies from CSMC, e.g., as depicted in FIGS.12A and 12B. To assess the cross-healthcare-system and international reliability of the model, the deep learning algorithm was additionally tested, without any tuning, on an external test dataset of 1,791 videos from Unity Imaging Collaborative and 3,660 videos from CSMC. On the Unity external test dataset, our deep learning algorithm showed a robust prediction accuracy with an overall R2 of 0.90, MAE of 1.6mm for IVS, 3.6mm for LVID, and 2.1 mm for LVPW. Highlighting data shift and potential variations in practice across institutions and continents, when fine-tuned using the training split of the Unity dataset, the deep learning algorithm showed an improved performance with an overall R2 of 0.92, median absolute error of 1.1mm for IVS, 1.7mm for LVID, and 1.5mm for LVPW on the Unity validation data split (Table 3 below). [00143] A rapid, high-throughput automated approach allows for measurement of every individual frame that would be tedious for manual tracing (FIGS.5A, 5B, and 5C). Differences in filling time and irregularity in the heart rate can cause variation in measurement but beat-to-beat model assessment can provide higher fidelity overall assessments. While the SHC and Unity datasets were directly compared on annotated individual frames, evaluation of the deep learning algorithm’s beat-to-beat was performed on the CSMC dataset in comparison with study-level annotations of ventricular dimensions. In this dataset, human measurements were not associated with specific frames of the echocardiogram video, and beat-to-beat analysis was used to predict diastole and average measurements from each heart beat across the entire video. On the CSMC external test dataset, the deep learning algorithm showed a robust prediction accuracy with an overall R2 of 0.96, MAE of 1.7mm (95% CI 1.6–1.8mm) for IVS, 3.8mm (95% CI 3.5-4.0mm) for LVID, and 1.8mm (95% CI 1.7-2.0mm) for LVPW with beat-to-beat evaluation. [00144] Prediction of Etiology of Hypertrophy [00145] The etiology derivation, validation, and test cohorts from SHC had 6,215, 787, and 765 videos respectively. On the held-out test cohort, our deep learning algorithm classifies cardiac amyloidosis with an AUC of 0.83, hypertrophic cardiomyopathy with an AUC 0.98, and aortic stenosis with an AUC 0.89 from other etiologies of LVH. On the held-out test cohort, the area under the precision-recall curve (AUPRC) of our deep learning algorithm for cardiac amyloidosis was 0.77, hypertrophic cardiomyopathy was 0.95, and aortic stenosis was 0.79. The proposed ensemble of binary classification video-based deep learning classifiers in our deep learning algorithm was similar in performance to a multi-label, multi-class deep learning model for disease detection, however had the flexibility of being able to identify overlap patients with multiple diagnoses. On an external test dataset of 2,351 A4c videos from CSMC with 358 videos of cardiac amyloidosis, 146 videos of aortic stenosis, 468 videos of hypertrophic cardiomyopathy, and 1,379 videos of other etiologies of LVH, our deep learning algorithm had an AUC of 0.79 for predicting cardiac amyloidosis and an AUC of 0.89 for hypertrophic cardiomyopathy. On the CSMC cohort, the AUPRC of the deep learning algorithm for cardiac amyloidosis was 0.54, hypertrophic cardiomyopathy was 0.69, and aortic stenosis was 0.08. The model performance was consistent across body mass index and image quality (Table 4). [00146] Phenotypic Mimics and Disease Specific Training Pipeline [00147] To highlight the benefit of training a model with negative controls derived from other etiologies of LVH instead of normal controls, a series of experiments was performed to see how a model trained without seeing other phenotypic mimics would perform when encountering phenotypic mimics. A confusion matrix was generated in the two experimental settings (Table 5), where a higher AUC outside the diagonal shows the model confusion and a lower AUC suggests improved discrimination between phenotypic mimics. In this experiment, while such a model can produce a higher AUC (AUC of 0.96 for cardiac amyloid, 0.98 for AS, and 0.97 for HCM), there was significant confusion when introducing other etiologies, suggesting a model trained only on age and sex matched controls primarily identifies hypertrophy. [00148] Systems and methods are provided for an artificial intelligence guided workflow, which is a deep learning system that automatically quantifies left ventricular wall thickness on echocardiography while also predicting etiology of LVH as attributable to either hypertrophic cardiomyopathy or cardiac amyloidosis. The deep learning algorithm performs measurements of ventricular thickness and diameter well within the variance of human clinical test-retest assessment – while concurrently aiding the detection of subtle ventricular phenotypes that tend to be challenging for human readers. This integration of left ventricular measurement and prediction of etiology offers an automated workflow for disease screening from echocardiography, the most frequently used form of cardiac imaging. As such, echocardiography-based screening can provide high index of suspicion that can facilitate more efficient clinical evaluation, diagnosis, and care. Assimilation of automated diagnostic algorithms with widely available clinical imaging can reduce physician burden while streamlining opportunities for more targeted cardiovascular care. [00149] Rather than rare, there is reason to believe that diseases such as cardiac amyloidosis are underdiagnosed28–30. Particularly given the large heterogeneous population of patients with heart failure with preserved ejection fraction31, methodologies that might appropriately as well as efficiently increase suspicion for under-recognized etiologies – such as subtypes of amyloidosis with newly available targeted therapies - can help address a persistent unmet need. Accordingly, methods and systems are provided for application of efficient AI algorithms to increase recognition of historically underdiagnosed disease conditions across stored images in databases of large echocardiography laboratories. Notwithstanding the fact that all patient data should be interpreted in clinical context, the findings described herein suggest that automated image analysis workflows could be feasibly implemented to rapidly identify patients who could benefit from follow-up screening across large populations. As such, more prospective work is needed to evaluate the potential of such algorithms to expedite appropriate clinical evaluation, targeted testing, and confirmation prior to eventual diagnosis and initiation of disease-modifying therapy. [00150] The systems and methods described herein offer several strengths. A key challenge in AI in healthcare has been the lack of benchmark datasets for direct comparison of models and engineering workflows across institutions. Dataset inclusion criteria, differences in annotations and disease definitions, and protocols of how to annotate images are all sources of dataset shift that limit the direct comparison of model performance34,35. With fine-tuning on site-specific data, the deep learning model provided herein compares favorably with prior state-of-the-art approaches to assessing ventricular wall thickness and hypertrophy on open benchmarks. [00151] The systems and methods represent an important step towards the automated assessment of cardiac structures in echocardiogram videos through deep learning. Although individual linear measurements take only seconds to measure, there is inherent variation in frame and video selection that sets a floor to the precision of manual measurements derived from echocardiography. In some examples echocardiographic labels may be augmented with annotations and information from cardiac MRI and other imaging modalities to have more precision automation. By leveraging an automated method, potentially more precise measurements can be obtained in both busy clinical and research settings. Further, deep learning models on echocardiogram images can automate an increasingly larger proportion of tasks for assessing cardiac form and structure to provide more holistic evaluation of cardiovascular disease8. With improved precision to detect ventricular remodeling and cardiac dysfunction, artificial intelligence systems offer the potential for earlier detection and treatment of subclinical cardiovascular disease including less common or underdiagnosed conditions. [00152] Table 1 below includes baseline characteristics of patients with Parasternal Long Axis Videos from Stanford Healthcare and Cedars-Sinai Medical Center echocardiograms, which are displayed for exemplary purposes and are in no way intended to limit the invention. Looking to the bottom section of the leftmost column, it should be noted that the left ventricular ejection fraction (LVEF) is included. Moreover, Table 2 includes baseline characteristics of patients with Apical-4-Chamber Videos from Stanford Healthcare and Cedars-Sinai Medical Center echocardiograms. It should again be noted that the information shown is in no way intended to limit the invention and is displayed for exemplary purposes. Looking now to the upper section of the leftmost column, it should be noted that hypertrophic cardiomyopathy (HCM), aortic stenosis (AS), and left ventricular hypertrophy (LVH) are included.
Figure imgf000040_0001
Table 1
Figure imgf000041_0001
Table 2 [00153] Table 3 below illustrates performance on Unity Imaging Collaborative External Test Dataset with and without fine-tuning. As shown, the deep learning models (EchoNet-LHV) that were trained using Stanford test datasets were able to achieve a favorable R2 (R2) value of 0.81, while the deep learning models trained using Unity test datasets were able to achieve an increased R2 value of 0.89. Finally, deep learning models that were first pre-trained using Stanford test datasets, and subsequently fine-tuned using the Unity test datasets were able to achieve a elevated R2 (R2) value of 0.92. It follows that, in addition to being able to detect heart structural conditions as well as the causes of the conditions more efficiently and even accurately than experts in the field of cardiology, the manner in which the deep learning models are trained also has a notable effect on performance.
Figure imgf000042_0001
Figure imgf000042_0003
Table 3 [00154] As noted above, in some approaches echocardiogram data may be echocardiogram frames derived from one or more examination procedures performed on the patient, and these frames may be pre-processed prior to passing through a neural network model (e.g., see 200 of FIG.2). Pre-processing the echocardiogram waveform data may include filtering, for example. In some instances, filtering may be performed to remove very low quality video signals. In any case, pre-processing of the input echocardiogram frames may be based on the pre-processing operations performed during training the neural network model. [00155] It should also be noted that the process of training, pre-training, and/or fine-tuning a deep learning model, may incorporate any of the approaches described herein. For instance, a deep learning model may be processed using any one or more of the operations included in FIGS. 8 and 9. [00156] Looking to Table 4, performance across different body mass index values is presented with respect to a deep learning model that was trained using a SHC test dataset.
Figure imgf000042_0002
Table 4 [00157] Meanwhile, Table 5 displays information associated with minimzing confusion of alternative etiologies of hypertrophy. More specifically, this may be achieved when a model is trained on age and sex matched control cases vs. other hypertrophic control. For instance, some instances include a model trained with age and sex matched controls without selection for LVH and introduced other etiologies of LVH at test time not seen during training. Outside the diagonal, a higher AUC shows confusion of the phenotypic mimics with a high degree of inappropriate confidence, e.g., as would be appreciated by one skilled in the art after reading the present description.
Figure imgf000043_0001
Figure imgf000043_0002
Table 5 [00158] As noted above, non-transitory memory (e.g., see 106 of FIG.1) may store training module includes instructions for training one or more neural network models stored in the module. The training module may include instructions that, when executed by processor, cause an echocardiogram processing system to conduct one or more of the steps of methods 800 and 900 below. For instance, FIG.8 is a high-level flow chart showing a method 800 for performing LVH etiology prediction, according to an embodiment. Moreover, FIG.9 depicts a method 900 that may be used to train one or more neural network models with corresponding training data sets, e.g., as will be described in further detail below. [00159] Referring now to FIG.8, a flowchart of a method 800 is shown according to one embodiment. The method 800 may be performed in accordance with the present invention in any of the environments depicted in FIGS. 1-3, among others, in various embodiments. Of course, more or less operations than those specifically described in FIG.8 may be included in method 800, as would be understood by one of skill in the art upon reading the present descriptions. [00160] Each of the steps of the method 800 may be performed by any suitable component of the operating environment. For example, in various embodiments, the method 800 may be partially or entirely performed by a controller, a processor, a computer, etc., or some other device having one or more processors therein. Thus, in some embodiments, method 800 may be a computer-implemented method. Moreover, the terms computer, processor and controller may be used interchangeably with regards to any of the embodiments herein, such components being considered equivalents in the many various permutations of the present invention. [00161] Moreover, for those embodiments having a processor, the processor, e.g., processing circuit(s), chip(s), and/or module(s) implemented in hardware and/or software, and preferably having at least one hardware component may be utilized in any device to perform one or more steps of the method 800. Illustrative processors include, but are not limited to, a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., combinations thereof, or any other suitable computing device known in the art. [00162] As shown in FIG. 8, operation 802 of method 800 includes acquiring echocardiogram data. The echocardiogram data may be acquired directly from an echocardiogram system having components capable of generating the echocardiogram data, e.g., such as an echocardiogram testing facility. Operation 804 includes pre-processing the acquired echocardiogram data. The pre-processing may be performed using any of the approaches described herein. For example, a pre-processing module such as 108 of FIG.1 may be used to pre-process the acquired echocardiogram data. [00163] Operation 806 further includes inputting the pre-processed echocardiogram data into a trained ventricular assessment model, from which method 800 proceeds to operation 810. There, operation 810 includes actually obtaining an indication of LVH condition being present in the echocardiogram data being evaluated. Decision 812 includes determining whether the presence of LVH is confirmed, and if not, an indication of absence of LVH is output. See operation 814. It should also be noted that operation 814 does not involve providing any input to the etiology prediction model. However, it should again be noted that cardiac structural conditions may be referred to specifically as herein as a LVH condition, but this is done by way of example, and is in no way intended to be limiting. Thus, operation 810 and/or decision 812 may correspond to a different type of cardiac health condition depending on the instance. [00164] Returning to decision 812, method 800 proceeds to operation 816 in response to determining that LVH is confirmed to be present. There, operation 816 includes inputting the pre- processed echocardiogram data into a trained etiology prediction model, e.g., according to any of the approaches included herein. Operation 818 further includes obtaining (e.g., receiving) a predicted etiology of the LVH present, while operation 820 includes determining treatment based on the predicted etiology of the LVH. As noted above, the models in various approaches herein are able to achieve earlier detection and treatment of subclinical cardiovascular disease than is otherwise possible. [00165] Referring now to FIG.9, a flowchart of a method 900 is shown according to one embodiment. The method 900 may be performed in accordance with the present invention in any of the environments depicted in FIGS. 1-3, among others, in various embodiments. Of course, more or less operations than those specifically described in FIG.9 may be included in method 900, as would be understood by one of skill in the art upon reading the present descriptions. [00166] Each of the steps of the method 900 may be performed by any suitable component of the operating environment. For example, in various embodiments, the method 900 may be partially or entirely performed by a controller, a processor, a computer, etc., or some other device having one or more processors therein. Thus, in some embodiments, method 900 may be a computer-implemented method. Moreover, the terms computer, processor and controller may be used interchangeably with regards to any of the embodiments herein, such components being considered equivalents in the many various permutations of the present invention. [00167] Moreover, for those embodiments having a processor, the processor, e.g., processing circuit(s), chip(s), and/or module(s) implemented in hardware and/or software, and preferably having at least one hardware component may be utilized in any device to perform one or more steps of the method 900. Illustrative processors include, but are not limited to, a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., combinations thereof, or any other suitable computing device known in the art. [00168] As shown in FIG. 9, operation 902 of method 900 includes acquiring a training dataset having a plurality of echocardiograms. In other words, operation 902 includes acquiring a training dataset having a plurality of data entries therein, the data entries corresponding to a plurality of different echocardiograms performed. Moreover, operation 904 includes pre- processing the acquired training dataset. Again, pre-processing may be performed using any of the approaches described herein. For example, a pre-processing module such as 108 of FIG.1 may be used to pre-process the acquired training dataset. [00169] Operation 906 includes training a risk prediction neural network model with the pre-processed training data set. The neural network model may correspond to any of the approaches included herein. Furthermore, operation 908 includes validating and updating hyperparameters of the neural network model, while operation 910 includes testing the trained and validated neural network model. [00170] Computer & Hardware Implementation of Disclosure [00171] It should initially be understood that the disclosure herein may be implemented with any type of hardware and/or software, and may be a pre-programmed general purpose computing device. For example, the system may be implemented using a server, a personal computer, a portable computer, a thin client, a wearable device, a digital stethoscope, or any suitable device or devices. The disclosure and/or components thereof may be a single device at a single location, or multiple devices at a single, or multiple, locations that are connected together using any appropriate communication protocols over any communication medium such as electric cable, fiber optic cable, or in a wireless manner. [00172] It should also be noted that the disclosure is illustrated and discussed herein as having a plurality of modules which perform particular functions. It should be understood that these modules are merely schematically illustrated based on their function for clarity purposes only, and do not necessary represent specific hardware or software. In this regard, these modules may be hardware and/or software implemented to substantially perform the particular functions discussed. Moreover, the modules may be combined together within the disclosure, or divided into additional modules based on the particular function desired. Thus, the disclosure should not be construed to limit the present invention, but merely be understood to illustrate one example implementation thereof. [00173] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some implementations, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server. [00174] Implementations of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. As noted above, examples of communication networks include LANs, WANs, an inter-network (e.g., the Internet), peer-to-peer networks (e.g., ad hoc peer-to-peer networks), any desired type of wireless networks, etc. [00175] Implementations of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine- generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially-generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, flash memory, or other storage devices). [00176] The operations described in this specification can be implemented as operations performed by a “data processing apparatus” on data stored on one or more computer-readable storage devices or received from other sources. [00177] The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, multi-core processors, GPUs, AI-accelerators, In- memory computing architectures or combinations, of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures, and deep learning and artificial intelligence computing infrastructure. [00178] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network. [00179] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). [00180] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read- only memory or a random access memory or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, flash memory or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), smart watch, smart glasses, patch, wearable devices, a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. [00181] The various methods and techniques described above provide a number of ways to carry out the invention. Of course, it is to be understood that not necessarily all objectives or advantages described can be achieved in accordance with any particular embodiment described herein. Thus, for example, those skilled in the art will recognize that the methods can be performed in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other objectives or advantages as taught or suggested herein. A variety of alternatives are mentioned herein. It is to be understood that some embodiments specifically include one, another, or several features, while others specifically exclude one, another, or several features, while still others mitigate a particular feature by inclusion of one, another, or several advantageous features. [00182] Furthermore, the skilled artisan will recognize the applicability of various features from different embodiments. Similarly, the various elements, features and steps discussed above, as well as other known equivalents for each such element, feature or step, can be employed in various combinations by one of ordinary skill in this art to perform methods in accordance with the principles described herein. Among the various elements, features, and steps some will be specifically included and others specifically excluded in diverse embodiments. [00183] Although the application has been disclosed in the context of certain embodiments and examples, it will be understood by those skilled in the art that the embodiments of the application extend beyond the specifically disclosed embodiments to other alternative embodiments and/or uses and modifications and equivalents thereof. [00184] In some embodiments, the terms “a” and “an” and “the” and similar references used in the context of describing a particular embodiment of the application (especially in the context of certain of the following claims) can be construed to cover both the singular and the plural. The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (for example, “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the application and does not pose a limitation on the scope of the application otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the application. [00185] Certain embodiments of this application are described herein. Variations on those embodiments will become apparent to those of ordinary skill in the art upon reading the foregoing description. It is contemplated that skilled artisans can employ such variations as appropriate, and the application can be practiced otherwise than specifically described herein. Accordingly, many embodiments of this application include all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the application unless otherwise indicated herein or otherwise clearly contradicted by context. [00186] Particular implementations of the subject matter have been described. Other implementations are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. [00187] All patents, patent applications, publications of patent applications, and other material, such as articles, books, specifications, publications, documents, things, and/or the like, referenced herein are hereby incorporated herein by this reference in their entirety for all purposes, excepting any prosecution file history associated with same, any of same that is inconsistent with or in conflict with the present document, or any of same that may have a limiting affect as to the broadest scope of the claims now or later associated with the present document. By way of example, should there be any inconsistency or conflict between the description, definition, and/or the use of a term associated with any of the incorporated material and that associated with the present document, the description, definition, and/or the use of the term in the present document shall prevail. [00188] In closing, it is to be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of the application. Other modifications that can be employed can be within the scope of the application. Thus, by way of example, but not of limitation, alternative configurations of the embodiments of the application can be utilized in accordance with the teachings herein. Accordingly, embodiments of the present application are not limited to that precisely as shown and described.

Claims

CLAIMS What is claimed is: 1. A method for assessing structural condition of a heart of a patient, the method comprising: receiving echocardiogram data acquired by an echocardiogram system; and determining one or more of a heart structural condition and one or more causes of the heart structural condition via a trained neural network model, the trained neural-network model receiving the echocardiogram data as input; and wherein the trained neural network model includes a first neural network model for determining the heart structural condition and a second neural network model for predicting one or more causes of the heart structural condition.
2. The method of claim 1, wherein the first neural network model includes at least one atrous convolutional layer performing an atrous convolutional operation on the echocardiogram data.
3. The method of claim 1, wherein the first neural network model is a segmentation model configured to identify two or more structural key points on the input echocardiogram data.
4. The method of claim 2, wherein the first neural network model is configured to determine the heart structural condition based on distances between at least two of the two or more structural key points.
5. The method of claim 1, wherein the first neural network model is trained based on sparse labeling of a training echocardiogram dataset.
6. The method of claim 1, wherein the heart structural condition is left ventricular hypertrophy.
7. The method of claim 6, wherein the one or more causes of the heart structural condition is one or more of hypertension, aortic stenosis, hypertrophic cardiomyopathy, or cardiac amyloidosis.
8. The method of claim 3, wherein the one or more structural key points are based on identification of the intraventricular septum (IVS), left ventricular internal dimension (LVID), and left ventricular posterior wall (LVPW).
9. The method of claim 1, wherein one or more of the first neural network and the second neural network models are optimized according to a number of skipped frames.
10. The method of claim 1, wherein the input echocardiogram is pre-processed to video files.
11. The method of claim 1, wherein the heart structural condition is determined based on a desired scan plane, the desired scan plane based on parasternal long axis.
12. The method of claim 1, wherein the trained neural network model is trained based on one or more negative controls, the one or more negative controls including images from patients with other causes of LVH.
13. A system for cardiac structure assessment, the system comprising: at least one memory storing a trained neural network model and executable instructions, the trained neural network model including at least one atrous convolutional layer; at least one processor communicably coupled to the at least one memory and when executing the instructions, configured to: receive a set of echocardiogram frames of a patient from an echocardiogram system; process the set of echocardiogram frames via the trained neural network model to output a set of segmented echocardiogram frames; wherein processing the set of echocardiogram frames to output a set of segmented echocardiogram frames includes identifying a plurality of key structural points on the set of segmented echocardiogram frames; obtain as output from the trained neural network model, an indication of a cardiac structural condition based on the set of segmented echocardiogram frames; and display, via a display portion of a user interface coupled to the at least one processor, the indication of the cardiac structural condition and/or segmented echocardiogram frames on a beat- by-beat basis.
14. The system of claim 13, wherein the cardiac health condition is determined on a beat-by-beat basis.
15. The system of claim 13, wherein the trained neural network model includes an etiology prediction classification model based on a ResNet architecture.
16. The system of claim 13, wherein the plurality of key structural points are identified via corresponding heat-map representations.
17. The system of claim 16, wherein the plurality of key structural points are based on centroids of the corresponding heat-maps.
18. The system of claim 13, wherein the trained neural network model is trained based on sparse labeling of a training echocardiogram dataset.
19. The system of claim 13, wherein the cardiac structural condition is left ventricular hypertrophy.
20. The system of claim 13, further comprising determining one or more caused of the cardiac structural condition via the trained neural network model; and wherein the one or more causes of the cardiac structural condition is one or more of hypertension, aortic stenosis, hypertrophic cardiomyopathy, or cardiac amyloidosis.
21. The method of claim 13, wherein the plurality of structural key points are based on identification of the intraventricular septum (IVS), left ventricular internal dimension (LVID), and left ventricular posterior wall (LVPW).
22. The method of claim 13, wherein the trained neural network model is trained for amyloid classification based on one or more negative training controls, the one or more negative training controls including images from patients with diagnosed HCM, aortic stenosis, hypertension, and ESRD.
PCT/US2022/072977 2021-06-17 2022-06-16 Precision phenotyping of left ventricular hypertrophy with echocardiographic deep learning WO2022266651A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202163212045P 2021-06-17 2021-06-17
US63/212,045 2021-06-17
US202263298562P 2022-01-11 2022-01-11
US63/298,562 2022-01-11

Publications (1)

Publication Number Publication Date
WO2022266651A1 true WO2022266651A1 (en) 2022-12-22

Family

ID=84527664

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/072977 WO2022266651A1 (en) 2021-06-17 2022-06-16 Precision phenotyping of left ventricular hypertrophy with echocardiographic deep learning

Country Status (1)

Country Link
WO (1) WO2022266651A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116704305A (en) * 2023-06-20 2023-09-05 华中科技大学同济医学院附属协和医院 Ultrasonic cardiogram multi-mode multi-section classification method based on deep learning algorithm
CN117114072A (en) * 2023-08-31 2023-11-24 四川维思模医疗科技有限公司 Method for simulating system training application by using ultrasonic image

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018204698A1 (en) * 2017-05-04 2018-11-08 Arterys Inc. Medical imaging, efficient sharing and secure handling of medical imaging information
WO2019153039A1 (en) * 2018-02-06 2019-08-15 Alerte Echo IQ Pty Ltd Systems and methods for ai-assisted echocardiography
WO2020037121A1 (en) * 2018-08-15 2020-02-20 Hyperfine Research, Inc. Deep learning techniques for suppressing artefacts in magnetic resonance images
US20210000449A1 (en) * 2018-03-14 2021-01-07 The Regents Of The University Of California Automated cardiac function assessment by echocardiography
US20210118559A1 (en) * 2019-10-22 2021-04-22 Tempus Labs, Inc. Artificial intelligence assisted precision medicine enhancements to standardized laboratory diagnostic testing

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018204698A1 (en) * 2017-05-04 2018-11-08 Arterys Inc. Medical imaging, efficient sharing and secure handling of medical imaging information
WO2019153039A1 (en) * 2018-02-06 2019-08-15 Alerte Echo IQ Pty Ltd Systems and methods for ai-assisted echocardiography
US20210000449A1 (en) * 2018-03-14 2021-01-07 The Regents Of The University Of California Automated cardiac function assessment by echocardiography
WO2020037121A1 (en) * 2018-08-15 2020-02-20 Hyperfine Research, Inc. Deep learning techniques for suppressing artefacts in magnetic resonance images
US20210118559A1 (en) * 2019-10-22 2021-04-22 Tempus Labs, Inc. Artificial intelligence assisted precision medicine enhancements to standardized laboratory diagnostic testing

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116704305A (en) * 2023-06-20 2023-09-05 华中科技大学同济医学院附属协和医院 Ultrasonic cardiogram multi-mode multi-section classification method based on deep learning algorithm
CN117114072A (en) * 2023-08-31 2023-11-24 四川维思模医疗科技有限公司 Method for simulating system training application by using ultrasonic image

Similar Documents

Publication Publication Date Title
Ghorbani et al. Deep learning interpretation of echocardiograms
Bello et al. Deep-learning cardiac motion analysis for human survival prediction
US10984905B2 (en) Artificial intelligence for physiological quantification in medical imaging
US10499857B1 (en) Medical protocol change in real-time imaging
US10853449B1 (en) Report formatting for automated or assisted analysis of medical imaging data and medical diagnosis
WO2021097460A1 (en) Systems and methods for a deep neural network to enhance prediction of patient endpoints using videos of the heart
Ilardi et al. Myocardial work by echocardiography: principles and applications in clinical practice
WO2022266651A1 (en) Precision phenotyping of left ventricular hypertrophy with echocardiographic deep learning
Sander et al. Automatic segmentation with detection of local segmentation failures in cardiac MRI
Seoni et al. Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013–2023)
US20150095069A1 (en) Algorithms to Identify Patients with Hepatocellular Carcinoma
JP2022553906A (en) Systems, methods and programs for developing disease detection models
WO2022110525A1 (en) Comprehensive detection apparatus and method for cancerous region
Yoon et al. Artificial intelligence and echocardiography
Lu et al. Understanding heart failure patients EHR clinical features via SHAP interpretation of tree-based machine learning model predictions
Zhang et al. A computer vision pipeline for automated determination of cardiac structure and function and detection of disease by two-dimensional echocardiography
Sirjani et al. Automatic cardiac evaluations using a deep video object segmentation network
Guo et al. Survival prediction of heart failure patients using motion-based analysis method
Choudhury et al. Using predictive analytics for cancer identification
Shamrat et al. Analysing most efficient deep learning model to detect COVID-19 from computer tomography images
Abdulkareem et al. Generalizable framework for atrial volume estimation for cardiac CT images using deep learning with quality control assessment
Kanwar et al. Machine learning, artificial intelligence and mechanical circulatory support: A primer for clinicians
Zhang et al. Artificial intelligence-enhanced echocardiography for systolic function assessment
Hagberg et al. Semi-supervised learning with natural language processing for right ventricle classification in echocardiography—a scalable approach
Bhan et al. An assessment of machine learning algorithms in diagnosing cardiovascular disease from right ventricle segmentation of cardiac magnetic resonance images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22826028

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE