WO2024145685A1 - Neural network-based electrocardiogram interpretation - Google Patents

Abstract

Aspects of the present disclosure relate to neural network-based electrocardiogram (ECG) interpretation. In examples, a machine learning model may be trained using a set of training data that includes one or more normal ECGs and one or more abnormal ECGs. The machine learning model may be trained using augmented training data that accounts for potential variability of ECG images that will be processed by the machine learning model. Accordingly, an image representation of an ECG for a patient may be obtained (e.g., as may be captured by a user' s mobile computing device) and processed by the trained machine learning model, thereby determining whether a condition is present or classifying the ECG image as indicating one or more conditions. The model processing result may be presented to a user (e.g., by the user's mobile computing device), which may further include a likelihood or confidence score associated with the determination, accordingly.

Description

NEURAL NETWORK-BASED ELECTROCARDIOGRAM INTERPRETATION

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Application No. 63/477,951, titled “Neural Network-Based Electrocardiogram Interpretation,” filed on December 30, 2022, the entire disclosure of which is hereby incorporated by reference in its entirety.

BACKGROUND

[0002] Interpretation of an electrocardiogram (ECG) is typically performed by an individual having extensive training and experience. However, the availability of such individuals may be a limiting factor for the utility of ECGs. Additionally, it may be difficult to reliably and accurately interpret subtle characteristics or differences of an ECG, which may be indicative of associated issues and/or conditions. The identification of false positives may result in unnecessary hospital admissions and wasted hospital resources, while the identification of false negatives may lead to complications and other unfavorable patient outcomes.

[0003] It is with respect to these and other general considerations that embodiments have been described. Also, although relatively specific problems have been discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the background.

SUMMARY

[0004] Aspects of the present disclosure relate to neural network-based electrocardiogram (ECG) interpretation. In examples, a machine learning model may be trained using a set of training data that includes one or more normal ECGs and one or more abnormal ECGs. The machine learning model may be trained using augmented training data that accounts for potential variability of ECG images that will be processed by the machine learning model. Accordingly, an image representation of an ECG for a patient may be obtained (e.g., as may be captured by a user’ s mobile computing device) and processed by the trained machine learning model, thereby determining whether a condition is present or classifying the ECG image as indicating one or more conditions. The model processing result may be presented to a user (e.g., by the user’s mobile computing device), which may further include a likelihood or confidence score associated with the determination, accordingly.

[0005] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Non-limiting and non-exhaustive examples are described with reference to the following Figures.

[0007] Figure 1 illustrates an overview of an example system in accordance with aspects described herein.

[0008] Figure 2 illustrates an overview of an example method for generating a machine learning model with which to evaluate an electrocardiogram according to aspects described herein.

[0009] Figure 3 illustrates an overview of example method for processing an image representation of an electrocardiogram at a computing device of a user according to aspects described herein.

[0010] Figure 4 illustrates an overview of an example method for processing an image representation of an electrocardiogram using a machine learning model trained according to aspects described herein.

[0011] Figure 5 illustrates an example of a suitable operating environment in which one or more aspects of the present application may be implemented.

DETAILED DESCRIPTION

[0012] In the following detailed description, references are made to the accompanying drawings that form a part hereof, and in which are shown by way of illustrations specific embodiments or examples. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the present disclosure. Embodiments may be practiced as methods, systems or devices. Accordingly, embodiments may take the form of a hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0013] In examples, an electrocardiogram (ECG) may be used to detect any of a variety of arrythmias that could be life-threatening or related to a patient’s underlying condition. The ECG may be generated using data that was obtained by an electrocardiograph machine or from any of a variety of other monitoring devices. For example, the ECG may be generated by a 12-lead electrocardiograph machine or by a smart watch or other wearable computing device, among other examples. However, in order to identify such life-threatening or underlying conditions, the ECG may first be evaluated or interpreted by a skilled individual, such as a physician or a cardiologist.

[0014] As a result, the utility of an ECG may be at least in part limited by the availability of an individual that is able to interpret it. Additionally, variability across individuals may result in different or inconsistent interpretations, as well as potentially incorrect interpretations in some instances. These and other detriments may reduce the ability to obtain meaningful and actionable information from an ECG, may result in unnecessary hospital admissions and wasted hospital resources, and/or may lead to complications and other unfavorable patient outcomes, among other issues.

[0015] Accordingly, aspects of the present disclosure relate to neural -network based ECG interpretation. In examples, an image representation of an ECG is processed using one or more machine learning models to determine the presence or absence of a given condition and/or to classify the ECG image accordingly. Example conditions (for example, as compared to a normal sinus rhythm or “NSR”) include, but are not limited to, myocardial infarction (MI), sinus bradycardia (SB), sinus tachycardia (STach), QT prolongation (QTP), atrial fibrillation (AFIB), atrial flutter (AFL), anteroseptal myocardial infarction (ASMI), inferior myocardial infarction (IMI), and/or first degree atrioventricular block (IAVB), among other examples.

[0016] Example image representations include, but are not limited to, a portable document format (PDF) representation, a Joint Photographic Experts Group (JPEG) image representation, or a portable network graphics (PNG) image representation, among other examples. As another example, the image representation may be extracted from a video stream or file. In some instances, the ECG image may be an image of an ECG that was captured using a camera of a mobile computing device. In a further example, the ECG image may have been obtained from a monitoring device or may have been generated based on ECG signal data, among other examples. Thus, it will be appreciated that any of a variety of techniques may be used to generate or otherwise obtain an image representation of an ECG according to aspects described herein.

[0017] A convolutional neural network (CNN), such as GoogLeNet, ResNet50, DenseNet, or AlexNet, may be used to process an ECG and generate a model processing result accordingly. While example neural networks and associated machine learning techniques are described with reference to the examples described herein, it will be appreciated that any of a variety of alternative networks, models, and/or techniques may be used in other examples.

[0018] A machine learning model may be trained based on a set of ECGs that includes a set of normal ECGs and a set of ECGs associated with one or more conditions. In instances where the training data is associated with a plurality of conditions, the resulting machine learning model may be used to classify an ECG as either normal or as indicating one or more of the plurality of conditions. In another example where a set of normal ECGs is used in conjunction with a set of ECGs associated with a condition, the resulting machine learning model may be used to perform a binary classification, indicating that either the ECG is normal or abnormal (e.g., thus indicating the presence of the condition).

[0019] An ECG (e.g., with which a machine learning model is trained or as may be classified or otherwise processed according to aspects described herein) may include data associated with one or more leads, such as I, II, III, aVR, aVL, aVF, VI, V2, V3, V4, V5, and/or V6. In some examples, a machine learning model may be trained using data associated with a predetermined set of leads, such that the machine learning model may be used to process an ECG having the same or a similar set of leads. For example, the machine learning model may be selected from a set of available machine learning models based on the set of leads that was used to capture an ECG, such that the ECG may be processed using the selected machine learning model accordingly. Such a selection may be made based on an indication received from a user or may be made automatically (e.g., as a result of identifying the data contained within the ECG), among other examples. It will be appreciated that, in some examples, data corresponding to a reduced number of leads may be processed (e.g., one or two leads of an ECG) accordingly to aspects described herein, which may reduce the amount of computational resources associated with such processing.

[0020] In examples, a machine learning model is trained based on an image representation of the ECG (e.g., as may be obtained from a monitoring device or as may be captured by a user of a computing device). In instances where ECG signal data is obtained, the ECG signal data may be used to generate an image representation accordingly, such that the resulting ECG image may be processed using the machine learning model according to aspects described herein. Thus, it will be appreciated that while examples are discussed with respect to ECG images, similar techniques may be applied additionally or alternatively to ECG signal data. For example, an ECG image may have associated ECG signal data, such that both the image representation and the signal data are processed using a machine learning model. As another example, signal data may be extracted from an ECG image and processed accordingly. In some examples, a set of ECG images is used according to aspects described herein and, if accuracy is below a predetermined threshold, ECG signal data may be used in addition to or as an alternative to ECG images.

[0021] Variability may exist in the quality of ECG images. For example, if an ECG image is obtained as a digital file from a monitoring device, very little, if any, noise, skew, or other distortion may be present. By contrast, if the ECG image is an image that was captured of a physical ECG printout, an ECG displayed on a computer monitor, or another physical representation, any of a variety of distortion may be present. Accordingly, the training data used to train a machine learning model may be processed to generate additional or alternative training data (also referred to herein as “augmented training data”) that includes one or more distortions that may be present when the machine learning model is used to process an ECG image. Example distortions include, but are not limited to, image skew, artificial or simulated sensor noise, and/or changes in white balance or orientation, among other examples. As an example, gaussian blur, gaussian noise, rotation, resizing and/or rescaling, flipping, cropping and/or zooming, color space changes (e.g., from RGB to grayscale or according to a single color channel), sharpness changes, contrast changes, saturation changes, brightness changes, hue changes, image encoding quality changes, and/or changes to gamma may be used to generate such training data accordingly. Such changes may be programmatically determined or may be random, among other examples.

[0022] Thus, as compared to instances where a skilled individual evaluates an ECG to make a determination whether one or more conditions are present, aspects of the present application enable more consistent, nuanced, and expedient processing of ECGs by a larger population of users, thereby increasing access to and the associated utility of using ECGs for patient treatment.

[0023] Figure 1 illustrates an overview of an example system 100 in accordance with aspects described herein. As illustrated, system 100 comprises data processing platform 102, computing device 104, monitoring device 106, and network 108. In examples, data processing platform 102, computing device 104, and monitoring device 106 communicate via network 108. For example, network 108 may comprise a local area network, a wireless network, or the Internet, or any combination thereof, among other examples.

[0024] The connection with monitoring device 106 is illustrated using dashed line 116 to indicate that, in other examples, a data connection may not be present, as may be the case when a physical representation of an ECG is generated by monitoring device 106 or monitoring device 106 saves the ECG to removable storage. As another example, computing device 104 may more directly communicate with monitoring device 106, for example using a wired or wireless data connection. In these and other instances, computing device 104 may obtain an image representation of the ECG (e g., by capturing an image of a physical representation of an ECG), which may be processed by computing device 104 or provided for processing by data processing platform 102, among other examples.

[0025] It will be appreciated that while system 100 is illustrated as comprising one data processing platform 102, one computing device 104, and one monitoring device 106, any number of such elements may be used in other examples. For example, ECGs from multiple monitoring devices may be processed by a single computing device and/or data processing platform, or, as another example, different monitoring devices may each have an associated data processing platform.

[0026] Further, the functionality described herein may be distributed among or otherwise implemented on any number of different computing devices in any of a variety of other configurations in other examples. For example, computing device 104 may implement aspects associated with machine learning engine 114 discussed below, such that an ECG may be processed by computing device 104 in addition to or as an alternative to processing by data processing platform 102.

[0027] Monitoring device 106 may be any of a variety of monitoring devices, including, but not limited to, a 12-lead electrocardiograph or a wearable device that has electrocardiograph functionality. As noted above, monitoring device 106 may communicate with network 108 via connection 116, may communicate more directly with computing device 104, and/or may generate a physical representation of an ECG, among other examples.

[0028] Data processing platform 102 may include one or more server computing devices and is illustrated as comprising training data store 110, augmented data generator 112, and machine learning engine 114. In examples, training data is acquired from one or more sources and stored in training data store 110. Training data store 110 may store ECG data as an image representation and/or as a signal representation. Further, at least some of the training data may be obtained from monitoring device 106, as may be the case when an ECG is annotated and added to training data store 110 to provide feedback to a machine learning model managed by machine learning engine 114. As another example, at least some of the training data may be from an existing dataset (e.g., from an external data source, not pictured). In some instances, the training data may have been annotated by one or more individuals that are skilled in the interpretation of ECGs and/or may be associated with an indication as to a patient outcome for the patent from which the ECG was generated.

[0029] Augmented data generator 112 may generate augmented training data (e.g., based on data from training data store 110). For example, augmented data generator 112 may process an image representation of an ECG to introduce one or more distortions, thereby generation additional or alternate training data. The augmented training data may thus be used to train a machine learning model (e.g., by machine learning engine 114) according to aspects described herein.

[0030] Machine learning engine 114 may train a machine learning model according to a set of ECGs (e.g., as may be stored by training data store 110 and/or as may be generated by augmented data generator 112). In examples, the set of ECGs is split into a subset of training ECGs and a subset of validation ECGs, such that the performance of a machine learning model trained using the subset of training ECGs may be evaluated using the subset of validation ECGs. As noted above, a variety of training data may be used, such that the training data used to train a machine learning model includes a set of normal ECGs and a set of ECGs having one or more associated conditions. Thus, machine learning engine 114 may train a machine learning model to identify the absence or presence of a condition (e.g., MI vs. NSR, SB vs. NSR, STach vs. NSR, SR vs. AFP, SR vs. AFL, or ASMI vs. IMI) and/or to classify an ECG as having zero or more indicated conditions (e.g., whether QTP or AFIB is present), among other examples.

[0031] In some examples, machine learning engine 114 maintains a set of models, where each model of the set of models is trained to make a binary determination (e.g., the presence or absence of a condition) or is trained to generate a classification (e.g., identifying the presence of one or more conditions from a set of conditions), among other examples. Thus, models of data processing platform 102 may be applicable to any of a variety of contexts and may be selected and applied accordingly.

[0032] Computing device 104 may be a mobile computing device, a tablet computing device, a laptop computing device, or a desktop computing device, among other examples. As illustrated, computing device 104 comprises application 118. In examples, application 118 obtains an ECG (e.g., as may have been generated by monitoring device 106). For example, application 118 may instruct a user to capture an image of a physical representation of an ECG, which may be captured using an image sensor (not pictured) of computing device 104. As another example, the ECG may be obtained digitally from monitoring device 106.

[0033] The obtained ECG may be processed according to aspects described herein, such that application 118 may provide an indication as to one or more model processing results associated with the ECG. For example, application 118 may provide the obtained ECG to data processing platform 102 for processing (e.g., by machine learning engine 114). In examples, the ECG is provided in conjunction with an indication as to one or more conditions for which the ECG should be evaluated. The ECG may then be processed based on one or more associated machine learning models to generate a model processing result, which may be provided to computing device 104 in response. Additional examples of such aspects are discussed below with respect to Figures 3 and 4. It will be appreciated that, in other examples, at least a part of such processing may be performed by application 118. For example, application 118 may obtain a machine learning model from data processing platform 102, which may thus be used to process an ECG accordingly. [0034] Accordingly, application 118 may present a display that the ECG indicates a normal sinus rhythm or that the ECG indicates one or more conditions (e.g., as may be indicated by the interpretation having the highest associated percentage likelihood). In some examples, the indication includes a percentage likelihood and/or a confidence level that is associated with the presented model processing result. As another example, one or more other conditions may be presented for consideration by the user, which may be ranked according to an associated percentage likelihood or confidence level, among other examples.

[0035] Figure 2 illustrates an overview of an example method 200 for generating a machine learning model with which to evaluate an ECG according to aspects described herein. In examples, aspects of method 200 are performed by a data processing platform, such as data processing platform 102 discussed above with respect to Figure 1. It will be appreciated that similar aspects may be performed by a computing device in other examples.

[0036] Method 200 begins at operation 202, where ECG data is obtained. For example, ECG data may be obtained from or otherwise associated with a monitoring device, such as monitoring device 106 in Figure 1. As another example, at least a part of the ECG data may be obtained from a training data store, such as training data store 110. In examples, the obtained ECG data may be ECG signal data and/or in an image representation.

[0037] At operation 204, ECG image data may be generated based on ECG signal data. As an example, operation 204 may comprise processing the ECG signal data to generate an image including a representation for signal data associated with one or more leads of the ECG signal data. Operation 204 is illustrated using a dashed box to indicate that, in some examples, operation 204 may be omitted. For example, in instances where the obtained ECG data does not include ECG signal data, operation 204 need not be performed.

[0038] Flow may progress to operation 206, where the ECG image data is preprocessed. It will be appreciated that any of a variety of preprocessing operations may be performed, including, but not limited to, resizing the ECG image data, cropping the ECG image data, and/or introducing one or more distortions (e.g., as may be performed by an augmented data generator such as augmented data generator 112 discussed above with respect to Figure 1). Operation 206 is illustrated using a dashed box to indicate that, in some examples, operation 206 may be omitted. [0039] Moving to operation 208, a subset of training data and a subset of validation data is determined. For example, 80% of the ECG image data may be used as training data, while 20% of the ECG image data may be used as validation data. Additionally, the subsets may be determined so as to maintain a similar composition of normal (e.g., NSR) ECG data and abnormal (e.g., indicating one or more conditions) ECG data in both subsets. While example compositions are described, it will be appreciated that any of a variety of alternative compositions may be used in other examples. Similarly, it will be appreciated that the subset of training data and the subset of validation data need not be mutually exclusive.

[0040] At operation 210, the machine learning model is trained based on the determined subset of training data. For example, the machine learning model may be trained according to a categorical cross-entropy loss function or a binary cross-entropy loss function, as may be the case when the machine learning model is trained to classify an ECG (e.g., as being normal or having one or more conditions) or to determine whether a given condition is present or absent, respectively. In examples, each ECG with which the machine learning model is trained may have an associated label, such as whether the ECG indicates a normal sinus rhythm or whether one or more conditions are present and/or an outcome for a patent from which the ECG was obtained. As an example, a label may be associated with a corresponding heartbeat segment of an ECG image representation. As noted above, the machine learning model may be trained according to ECG image data (e.g., as may have been generated at operation 204 and/or preprocessed at operation 206). In some examples, the ECG image data may further have associated ECG signal data with which the model is trained.

[0041] Flow progresses to operation 212, where the trained machine learning model is provided for evaluating ECGs according to aspects described herein. In examples, the trained machine learning model may be stored by a data processing platform (e.g., data processing platform 102 in Figure 1) and/or provided to a computing device (e.g., computing device 104) for subsequent use. Method 200 terminates at operation 212.

[0042] Figure 3 illustrates an overview of example method 300 for processing an image representation of an ECG at a computing device of a user according to aspects described herein. In examples, aspects of method 300 are performed by a computing device, such as computing device 104 discussed above with respect to Figure 1. [0043] As illustrated, method 300 begins at operation 302, where a user selection of captured image data is received. The captured image data may comprise an image representation of an ECG (e.g., as may be captured by a monitoring device, such as monitoring device 106). For example, the user may select the captured image data from a set of available fdes (e.g., on the computing device, on removable storage, or fdes that are otherwise accessible by the computing device). As another example, operation 302 may comprise capturing the image data using an image sensor of the computing device. Thus, it will be appreciated that any of a variety of techniques may be used to obtain such image data at operation 302.

[0044] Flow progresses to operation 304, where an indication of the image data is provided to a processing platform, such as data processing platform 102 discussed above with respect to Figure 1. For example, the indication may include at least a part of the captured image data or an indication as to a location from which the captured image data may be obtained. In some instances, the indication further comprises an indication of one or more conditions for which the image data should be evaluated.

[0045] Moving to operation 306, a processing result is received from the processing platform. For example, the processing result may include an indication as to whether a condition is indicated by the captured image data and/or a categorization for the captured image data, and, in some examples, a likelihood or confidence level associated with such indications. In some instances, the indication may further include a recommendation or a specific indication as to one or more regions of the image data that are associated with the indication.

[0046] At operation 308, a display of the received processing result is generated. For example, a screen of the computing device may be updated to include an indication of an identified condition, a likelihood or confidence level associated with the identified condition, and/or a region of the captured image data that was identified as being indicative of the condition, among other examples. While example processing results and associated user experience aspects are described, it will be appreciated that any of a variety of additional or alternative processing results may be obtained from the data processing platform and presented to the user accordingly. Method 300 terminates at operation 308.

[0047] While method 300 is described as an example in which the image data is provided to a data processing platform for processing using a machine learning model, it will be appreciated that, in other examples, at least a part of such processing may be performed local to the computing device. For example, the computing device may perform aspects of method 400 discussed below in addition to or as an alternative to processing performed by the data processing platform.

[0048] Figure 4 illustrates an overview of an example method 400 for processing an image representation of an ECG using a machine learning model trained according to aspects described herein. In examples, aspects of method 400 may be performed by a machine learning engine (e.g., machine learning engine 114 in Figure 1) and/or an application (e.g., application 118), among other examples.

[0049] Method 400 begins at operation 402, where ECG data is obtained. In examples, the ECG data is obtained from a computing device that is performing aspects of operation 302 discussed above with respect to method 300 of Figure 3. In other examples, the ECG data may be signal data of the ECG, such that the received ECG signal data is processed to generate an image representation according to aspects described herein. As another example, at least a part of the ECG data may be obtained from a remote data store, as may have been indicated by a request received form the computing device. In some examples, the ECG data is obtained in association with an indication as to one or more conditions for which the ECG data should be evaluated.

[0050] At operation 404, a machine learning model is selected from a set of machine learning models. For instance, the machine learning model may be selected according to one or more conditions for which the ECG data is to be evaluated. As an example, the selected machine learning model may have been trained to identify the presence or absence of a given condition or to classify the ECG as indicating the presence of one or more conditions, among other examples. Operation 404 is illustrated as using a dashed box to indicate that, in some examples, operation 404 may be omitted. For example, operation 404 may be omitted in examples where a single machine learning model is available or, as another example, where the ECG is to be evaluated using each machine learning model of the set of machine learning models.

[0051] Flow progresses to operation 406, where a model processing result is generated using the model that was selected at operation 404. For example, the model processing result may comprise a determination whether a condition is present within the ECG that was obtained at operation 402 or may comprise a classification of the ECG as indicating one or more conditions, among other examples. As another example, the model processing result comprises a confidence score or a likelihood for one or more identified conditions. While example model processing results are described, it will be appreciated that additional, fewer, or alternative processing results may be generated in other examples.

[0052] At operation 408, an indication of the processing result is provided. For example, the indication may be provided to a computing device (e.g., computing device 104, by data processing platform 102 in Figure 1), such that an application thereon may present an indication of the model processing result to a user of the device accordingly. In instances where aspects of method 400 are performed by such a computing device, operation 408 may comprise generating such a display based on the model processing result. As noted above, a likelihood value or a confidence score may be presented with such an indication. Method 400 terminates at operation 408.

[0053] Figure 5 illustrates an example of a suitable operating environment 500 in which one or more of the present embodiments may be implemented. This is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality. Other well-known computing systems, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics such as smart phones, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

[0054] In its most basic configuration, operating environment 500 typically may include at least one processing unit 502 and memory 504. Depending on the exact configuration and type of computing device, memory 504 (storing, among other things, APIs, programs, etc. and/or other components or instructions to implement or perform the system and methods disclosed herein, etc.) may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in Figure 5 by dashed line 506. Further, environment 500 may also include storage devices (removable, 508, and/or nonremovable, 510) including, but not limited to, magnetic or optical disks or tape. Similarly, environment 500 may also have input device(s) 514 such as a keyboard, mouse, pen, voice input, etc. and/or output device(s) 516 such as a display, speakers, printer, etc. Also included in the environment may be one or more communication connections, 512, such as LAN, WAN, point to point, etc. [0055] Operating environment 500 may include at least some form of computer readable media. The computer readable media may be any available media that can be accessed by processing unit 502 or other devices comprising the operating environment. For example, the computer readable media may include computer storage media and communication media. The computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium, which can be used to store the desired information. The computer storage media may not include communication media.

[0056] The communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” may mean a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. For example, the communication media may include a wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.

[0057] The operating environment 500 may be a single computer operating in a networked environment using logical connections to one or more remote computers. The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above as well as others not so mentioned. The logical connections may include any method supported by available communications media. Such networking environments are commonplace in offices, enterprisewide computer networks, intranets and the Internet.

[0058] The different aspects described herein may be employed using software, hardware, or a combination of software and hardware to implement and perform the systems and methods disclosed herein. Although specific devices have been recited throughout the disclosure as performing specific functions, one skilled in the art will appreciate that these devices are provided for illustrative purposes, and other devices may be employed to perform the functionality disclosed herein without departing from the scope of the disclosure.

[0059] As stated above, a number of program modules and data fdes may be stored in the system memory 504. While executing on the processing unit 502, program modules (e.g., applications, Input/Output (I/O) management, and other utilities) may perform processes including, but not limited to, one or more of the stages of the operational methods described herein such as the methods illustrated in Figures 2, 3, or 4, for example.

[0060] Furthermore, examples of the invention may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, examples of the invention may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in Figure 5 may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or "burned") onto the chip substrate as a single integrated circuit. When operating via an SOC, the functionality described herein may be operated via application-specific logic integrated with other components of the operating environment 500 on the single integrated circuit (chip). Examples of the present disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. In addition, examples of the invention may be practiced within a general purpose computer or in any other circuits or systems.

[0061] Aspects of the present disclosure, for example, are described above with reference to block diagrams and/or operational illustrations of methods, systems, and computer program products according to aspects of the disclosure. The functions/acts noted in the blocks may occur out of the order as shown in any flowchart. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved.

[0062] The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The aspects, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the best mode of claimed disclosure. The claimed disclosure should not be construed as being limited to any aspect, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Having been provided with the description and illustration of the present application, one skilled in the art may envision variations, modifications, and alternate aspects falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed disclosure.

Claims

1. A system comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations comprising: obtaining an image representation of an electrocardiogram (ECG); processing the image representation of the ECG using a machine learning model to generate a model processing result, wherein the machine learning model was trained using a training dataset including a subset of normal ECG data and a subset of abnormal ECG data; and providing an indication of the model processing result for the image representation of the ECG.

2. The system of claim 1, wherein: the image representation of the ECG is received from a mobile computing device; and the indication of the model processing result is provided to the mobile computing device.

3. The system of claim 1, wherein: the image representation of the ECG is obtained from an image sensor of the system; and the indication of the model processing result is presented on a display of the system.

4. The system of claim 3, wherein the machine learning model is obtained from a data processing platform.

5. The system of claim 1, wherein the indication of the model processing result includes at least one of an indication of a likelihood associated with the model processing result or a recommendation associated with the model processing result.

6. The system of claim 1, wherein obtaining the image representation of the ECG comprises: obtaining a signal representation of the ECG; and generating, based on the signal representation of the ECG, the image representation of the ECG.

7. The system of claim 1, wherein the training dataset includes augmented training data having one or more distortions selected from the group of distortions consisting of image skew, sensor noise, a change in contrast, a change in sharpness, a change in white balance, a change in resolution, gaussian blur, gaussian noise, rotation, and a change in gamma.

8. The system of claim 1, wherein the set of operations further comprises selecting the machine learning model from a set of machine learning models based on a condition for which the image representation of the ECG is to be evaluated.

9. A method for generating a machine learning model to characterize electrocardiogram (ECG) data, the method comprising: obtaining an image representation of an ECG; generating, based on the image representation, augmented training data comprising a distortion of the image representation; training the machine learning model using the image representation and the augmented training data, wherein the augmented training data is annotated the same as the image representation; and processing new ECG data using the trained machine learning model to generate a model processing result indicating whether the new ECG data is indicative of a normal ECG or an abnormal ECG.

10. The method of claim 9, wherein the distortion is one or more of: image skew; simulated sensor noise; gaussian blur; gaussian noise; rotation; resizing/rescaling; flipping; cropping; zooming; a change in white balance; a change in orientation; a color space change; a sharpness change; a contrast change; a saturation change; a brightness change; a hue change; an image encoding quality change; or a change to gamma.

11. The method of claim 9, wherein the image representation of the ECG comprises at least one of a digital file from a monitoring device; an image that was captured of a physical ECG printout; or an image of an ECG displayed on a computer monitor.

12. The method of claim 9, wherein the machine learning model is further trained using signal data of the obtained ECG that is further associated with the image representation of the ECG.

13. The method of claim 9, further comprising providing an indication of the model processing result to a computing device from which the new ECG data was obtained.

14. A method for processing an electrocardiogram (ECG), the method comprising: obtaining an image representation of an ECG; processing the image representation of the ECG using a machine learning model to generate a model processing result, wherein the machine learning model was trained using a training dataset including a subset of normal ECG data and a subset of abnormal ECG data; and providing an indication of the model processing result for the image representation of the ECG

15. The method of claim 14, wherein: the image representation of the ECG is received from a mobile computing device; and the indication of the model processing result is provided to the mobile computing device.

16. The method of claim 14, wherein: the image representation of the ECG is obtained from an image sensor of the system; the machine learning model is obtained from a data processing platform; and the indication of the model processing result is presented on a display of the system.

17. The method of claim 14, wherein the indication of the model processing result includes at least one of an indication of a likelihood associated with the model processing result or a recommendation associated with the model processing result.

18. The method of claim 14, wherein obtaining the image representation of the ECG comprises: obtaining a signal representation of the ECG; and generating, based on the signal representation of the ECG, the image representation of the ECG.

19. The method of claim 14, wherein the training dataset includes augmented training data having one or more distortions selected from the group of distortions consisting of image skew, sensor noise, a change in contrast, a change in sharpness, a change in white balance, a change in resolution, gaussian blur, gaussian noise, rotation, and a change in gamma.

20. The method of claim 14, wherein further comprising selecting the machine learning model from a set of machine learning models based on a condition for which the image representation of the ECG is to be evaluated.