CN110691548B - System and method for predicting and summarizing medical events from electronic health records - Google Patents

Abstract

A system for predicting and summarizing medical events from electronic health records includes a computer memory storing aggregated electronic health records from a large number of patients with different ages, health conditions, and demographics, including medications, laboratory values, diagnostics, vital signs, and medical notes. The aggregated electronic health records are converted into a single standardized data structure format and arranged in patient order, e.g., into chronological order. A computer (or computer system) runs one or more deep learning models trained on the aggregated health records to predict one or more future clinical events and summarize related past medical events related to the predicted events based on input electronic health records of patients having standardized data structure formats and ordered chronologically. An electronic device configured with a healthcare provider-oriented interface displays one or more future clinical events and related past medical events of a patient.

Description

System and method for predicting and summarizing medical events from electronic health records

Cross Reference to Related Applications

The present application claims priority from 35U.S. c. ≡119 to U.S. provisional application serial No. 62/538,112 filed on 7, 28. The entire contents of the' 112 provisional application, including appendix a and appendix B, are incorporated herein by reference.

Technical Field

The present disclosure relates to a system and method for predicting and summarizing medical events from electronic health records using a deep learning model. The present disclosure also relates to several constituent aspects and combinations thereof, including merging electronic health records into a single format for model generation and training, a deep learning model for predicting health events from medical records, and a provider-oriented interface on an electronic device for displaying clinical predictions obtained through deep learning and potentially related medical events related to the predictions.

Background

Nobel Laureate Herbert Simon once said: "information consumption is quite obvious: it consumes the attention of the recipient. Thus, the rich information creates a lack of attention. "in a clinical setting, the management and presentation of information about a patient is an important aspect of patient care and healthcare decisions, such as how to treat the patient or when to discharge the patient. In the case of a busy hospital or clinic, information management is a particular problem, in which case a healthcare provider (such as a nurse or physician) is paying attention to many patients simultaneously. For example, information contained in a patient's electronic health record consumes the attention of the recipient (e.g., a nurse or physician). Rich information, such as information contained in a broad medical history (medical history) of a particular patient for many years, or more generally, information in a medical history of a large number of patients, causes a lack of attention.

There is a need for a system and method to help healthcare providers effectively distribute their attention among the plethora of information from different sources and to timely provide predictions of future clinical events and highlighting (highlight) of relevant potential medical events that contribute to these predictions. The present disclosure addresses the urgent problem faced by physicians in hospitals, which patients now most need my attention, and on a personal level, which information in patient medical records should i be noted?

Prior Art

The rapid adoption of electronic health records (Electronic Health Record, EHR) has made routine clinical data plentiful and digital. Henry J et al Adoption of Electronic Health Record Systems among u.s.no-Federal Acute Care Hospitals:2008-2015,Office of the National Coordinator for Health Information Technology,ONC data brief no.35,May 2016.Adler-Milstein J, desroche CM, electronic Health Record Adoption In US Hospitals: progress Continues, known as KraloVec P et al, but changes personast.health aff.2015;34 (12):2174-2180. Adoption of Electronic Health Record Systems among U.S. No. Federal Acute Care Hospitals:2008-2015,Office of the National Coordinator for Health Information Technology,ONC, published under 35, month 5 of Henry J et al. There have been breakthrough efforts to address this phenomenon with algorithms to intervene for patients predicted to be at high risk (see, integrating Predictive Analytics Into High-Value Care: the Dawn of Precision delivery. JAMA.2016, 315 (7): 651-652) by Parikh RB, kakad M, bates DW, triage for patients at risk of adverse effects or decompensation (see, bates DW, saria S, ohno-Machado L, shah A, escobar G by Big Data in healthcare: using analytics to identify and manage high-risk and high-cost reagents. Health Aff.2014;33 (7): 1123-1131; obimeyer Z, emanuel EJ, predicting the Future-Big Data, machine Learning, and Clinical media. N Engl J.2016; 375 (13): 1216-1219), or even recommended specific cancer treatments. See Artificial Intelligence by Kantarjian H, yu PP, big Data, and cancer jama oncocol 2015;1 (5):573-574.

Traditionally, these predictive models were created separately for each task by collecting variables that were measured continuously in a pre-specified queue, often ensuring high quality data collection in clinical registries or trials. In contrast, data generated in daily care may result in incomplete, inaccurate, and inconsistent data sets. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care.2013 by Hersh WR, weiner MG, embi PJ et al; 51 (8 Suppl 3) S30-S37; validation of electronic medical record-based phenotyping algorithms by Newton KM, peissig PL, kho AN et al results and lessons learned from the eMERGE network.J Am Med Inform assoc.2013;20 (e 1) e147-e154; electronic Health Records as Sources of Research data.JAMA.2016 by Opmeer BC; 315 (2):201-202. Thus, in order to create a predictive model, researchers expend a great deal of effort to define variables, normalize data, and process missing measurements (see, for example, newton and Opmeer references), complicating deployment, as these steps must be reconstructed in real-time based on real-time data. Opportunities and challenges in developing risk prediction models with electronic health records data by Goldstein BA, navar AM, pencina MJ, ioanidis JPA: a systematic review.J AM Med information assoc.2017;24 (1):198-208. In view of the above, the median of the variables in the predictive model is 27 (see Goldstein et al, supra), and therefore most data, especially unstructured data (e.g., notes) and repeated measurements (e.g., vital signs and laboratory results), are ignored.

Disclosure of Invention

As a summary and summary, one aspect of the present disclosure relates to a system for predicting and summarizing medical events from an electronic health record. The system comprises three parts:

first, the system includes a computer memory, e.g., one or more mass data storage devices, that store aggregated electronic health records from a large number (e.g., millions) of patients of different ages, health conditions, and demographics (demographics), including in particular medications, laboratory values, diagnostics, vital signs, and medical notes (i.e., free text entered by the provider). The aggregated health records are patient-unidentified (patient-identified) and obtained from one or more sources, and potentially organized in different data structure types due to differences in legacy systems. The aggregated electronic health record is converted into a single standardized data structure format and preferably placed in an ordered format, such as, for example, chronological order.

Second, the system includes a computer (which term is intended to refer to a single computer or system or processing unit of computers sharing processing tasks, including auxiliary memory) running one or more machine learning models trained on aggregated health records converted into standardized data structure formats and ordered formats. The deep learning model is trained to predict one or more future clinical events based on the input electronic health record for a given patient and to summarize or highlight related past medical events (pertinent past medical event) (e.g., diagnosis, medication, notes, or excerpts thereof) related to the predicted one or more future clinical events. The entered electronic health records are in a standardized data structure format and are ordered chronologically, such as summarized health records for model training.

Third, the system includes an electronic device for use by a healthcare provider who treats the patient, such as a computer terminal or workstation, tablet, smartphone, or other type of computing device with a screen display configured with a customer-oriented interface that displays one or more future clinical events and related past medical events of the patient's predictions generated by one or more predictive models.

In the detailed description, we describe that the aggregated health record may take the form of health records from a large number of patients (hundreds of thousands or even millions of patients), such health records being obtained in de-identified form from a plurality of different institutions (e.g., hospitals or medical systems). Because of the lack of standardization in the industry, data from different institutions may take different data formats. These records are converted into a standardized data structure format. In one embodiment, they are arranged chronologically on a per patient basis. There is de-identification of the patient in the aggregated health record. In one particular embodiment, the standardized data structure format is a fast health interoperability resource (Fast Health Interoperability Resources, FHIR) format, which is a known format, see SMART on FHIR, by Mandel JC et al, a standards-based, interoperable apps platform for electronic health records.J Am Med information assoc.2016;23 899-908, wherein the EHR is formatted as a time-sequential (time-ordered) FHIR "resource" bundle.

In one embodiment, the aggregated health record contains variable names that are inconsistent with standard terms, except for variables required to define the primary outcome and exclusion criteria (exclusion criteria), i.e., criteria for excluding a given EHR from model training. In one embodiment, the aggregated health record contains hospitalization diagnoses, and the diagnoses are mapped to single level clinical classification software (Clinical Classification Software, CCS) codes.

In one aspect, the one or more deep learning models contain a "attentiveness mechanism" (a technique known in the deep learning art and described in detail below, sometimes referred to as an "attribution mechanism"), wherein, when invoked, the "attentiveness mechanism" instructs the one or more models to give how much attention or equivalent "weight" to a particular "token" corresponding to an atomic element (single word in a note, laboratory measurement, prescription, etc.) in the electronic health record to reach a prediction of one or more future clinical events and related past medical events. The provider-oriented interface preferably includes a display of the outcome of the attention mechanism, such as by providing a degree of highlighting or emphasis of elements in the health record (i.e., past medical events) associated with a particular prediction, particularly those elements that are high in the attention mechanism. In addition to predicting and correlating medical events, displaying the results of the attention mechanism on the electronic device also provides the healthcare provider with confidence in the predictions and their basis and directs them to notice the relevant elements or features of the health record associated with the predictions to inform and guide their patient care.

Aspects of the present disclosure are directed to a deep learning model for making predictions. In one embodiment, we consider the use of a set of deep-learning neural network models, each model trained individually on a summarized EHR. In one embodiment, we use (1) a Long-Short-Term Memory (LSTM) Model, (2) a time-aware feedforward Model (FFM), also referred to herein as a feedforward Model with time-aware attention, and (3) an embedded enhanced time-series Model, also referred to herein as an embedded time-aware enhancement Model. Alternatives to these models may be suitable for use in the present system, such as, for example, an autoregressive convolutional neural network model with attention, see Attention is all you need by A.Vaswani et al, arXiv:1706.03762[ cs.CL ] (month 6 2017). The predictions of one or more future clinical events and the summary of past medical events related to the predicted one or more future clinical events may be obtained from a collective average (ensemble average) of three deep learning models. In some cases, predictions from members of the set may be excluded for various reasons.

We disclose various possible predictions of future clinical events, and in one embodiment, the deep learning model(s) predicts at least one of: an unplanned transfer to an intensive care unit, a stay exceeding 7 days, an unplanned stay, an ER (Emergency Room) visit or readmission within 30 days after patient discharge, an inpatient mortality rate, a primary diagnosis, or a complete set of primary and secondary billing diagnoses at patient discharge. We also disclose the ability to predict atypical laboratory values, including potential factors such as acute kidney injury, hypokalemia, hypoglycemia, or hyponeutrophilia. We will describe further additional prediction tasks for which the model may be used.

Other aspects of the disclosure are directed to an electronic device and its provider-oriented interface. In one embodiment, the interface includes a display of: (1) alerts for one or more future clinical events predicted, (2) critical medical problems or conditions associated with the alert (i.e., past medical events), and (3) notes or excerpts thereof, e.g., words or phrases, associated with the alert. In one configuration, the deep learning model contains an attention mechanism that instructs one or more models to pay more attention to tokens corresponding to elements in the electronic health record to predict one or more future clinical events. The display of notes or excerpts thereof is displayed in a manner that indicates results from the application of the attention mechanism, e.g., by using a degree of highlighting or emphasis on particular words, phrases, or other text in the note, e.g., by changing font size, color, shading, boldness, italics, underlining, strikethrough, blinking, highlighting, font selection, etc., thereby drawing the attention of the provider to the most important past medical event in the EHR that is related to the predicted future clinical event. In yet another configuration, the display may also include a timeline or graph of information inferred from the patient's electronic health record (e.g., a preliminary diagnosis inferred from past medical events) and the risk or probability of certain clinical events occurring in the future, such as death or transfer to an ICU (intensive care unit ).

In one possible configuration, the display allows a user of the electronic device to select one of the key questions or conditions, and the selection triggers further display of information related to the selected key question or condition, such as display of medications prescribed to the patient and notes or excerpts thereof related to the selected key question or condition.

In another aspect of the disclosure, a method for predicting and summarizing a medical event from an electronic health record is described. The method comprises the following steps:

a) Summarizing electronic health records from a large number of patients with different ages, health conditions, and demographics, the electronic health records including some or all of medications, laboratory values, diagnoses, vital signs, and medical notes;

b) Converting the summarized electronic health records into a single standardized data structure format and arranging the electronic health records according to the order of patients;

c) Training one or more deep learning models on the summarized health records converted into a single standardized data structure format and arranged in order;

d) Predicting one or more future clinical events from the input electronic health record of the chronologically ordered patient in a standardized data structure format and summarizing related past medical events related to the predicted one or more future clinical events using the trained one or more deep learning models; and

e) Data is generated for a healthcare provider-oriented interface of the electronic device for use by the healthcare provider to display one or more future clinical events predicted by the patient and related past medical events.

In yet another aspect, a system is described that includes a combination of:

a) Computer memory storing aggregated electronic health records, including some or all of medications, laboratory values, diagnoses, vital signs, and medical records, from a large number of patients of different ages, health conditions, and demographics, and obtained in different formats, wherein the aggregated electronic health records are converted into a single standardized data structure format and placed in an ordered arrangement (such as a chronological order); and

b) A computer (as defined above) that runs one or more deep learning models to predict future clinical events based on input electronic health records of a patient, the deep learning models trained on aggregated health records that are converted to a single standardized data structure format and ordered. In one aspect, the one or more deep learning models contain an "attention mechanism" that indicates how much attention the one or more models pay to a particular "token" corresponding to an atomic element (single word in notes, laboratory measurements, prescriptions, etc.) in the electronic health record to arrive at a prediction of one or more future clinical events and a summary of past medical events related to the predicted one or more future clinical events. In one embodiment, we consider the use of a set of deep-learning neural network models, where each model is trained separately on summarized EHRs. In one embodiment, we use (1) a long-term short-term memory (LSTM) model, (2) a time-aware feedforward model (FFM), and (3) an embedded enhanced time-series model.

In yet another aspect of the disclosure, a method for predicting a medical event from an electronic health record is described. The method comprises the following steps:

a) Summarizing electronic health records from a large number of patients of different ages, health conditions and demographics and obtained in different formats, the electronic health records including some or all of medications, laboratory values, diagnostics, vital signs and medical notes;

b) Converting the aggregated electronic health record into a single standardized data structure format and ordering into an ordered arrangement, such as, for example, chronological order, per patient; and

c) One or more deep learning models are trained on the aggregated health records converted to a single standardized data structure format and ordered, wherein the trained one or more deep learning models predict future clinical events based on the input electronic health records of the patient in the standardized data structure format and ordered chronologically.

In yet another aspect, we have described an improved computer (as previously defined) that runs on one or more deep learning models trained on aggregated electronic health records converted into a single standardized data structure format and chronologically ordered to predict one or more future clinical events and to summarize related past medical events related to the predicted one or more future clinical events based on input electronic health records of patients having the standardized data structure format and chronologically ordered.

In a preferred embodiment, the deep learning models each contain an "attention mechanism" that indicates how much attention one or more models pay to a particular "token" corresponding to an electronic health record element (single word in notes, laboratory measurements, prescriptions, etc.) in the electronic health record to predict one or more future clinical events and summarize related past medical events related to the predicted one or more future clinical events.

In yet another aspect, a system is disclosed that includes a combination of: a) A computer running one or more deep learning models trained on aggregated health records converted to a single standardized data structure format and chronologically ordered to predict one or more future clinical events based on input electronic health records of patients having the standardized data structure format and chronologically ordered and to summarize related past medical events related to the predicted one or more future clinical events; and b) a customer-facing interface of an electronic device used by a healthcare provider of the treatment patient, the interface configured to display one or more future clinical events predicted by the patient and related past medical events.

In yet another aspect, an electronic device (e.g., workstation, tablet computer, or smart phone) is disclosed having a healthcare provider-oriented interface that displays a display of predictions of one or more future clinical events for at least one patient in substantially real-time. The display is also configured to display elements from the electronic health record (past medical events) that correspond to applications of the attention mechanism on the predictive model that operates on the electronic health record related to the prediction. In one embodiment, the element of the electronic health record is a note or a snippet thereof with a highlighting or emphasis level for a particular word, phrase, or other text in the note. The elements of the electronic health record may also be, for example, laboratory values, past medications, vital signs, and the like. The highlighting or emphasis level may take the form of at least one of font size, font color, shading, bolding, italics, underlining, strikethrough, highlighting with color, and font selection, or some combination thereof, such as red and bolded. The predicted one or more future clinical events may include an unplanned transfer to an intensive care unit, a stay in hospital exceeding 7 days, an unplanned readmission within 30 days after discharge, an inpatient mortality rate, a primary diagnosis, a complete set of primary and secondary billing diagnoses, or atypical laboratory values such as acute kidney injury, hypokalemia, hypoglycemia, and hyponeutrophilia.

In one embodiment, the interface is further configured to display a graph plotting risk or probability of at least one patient over time, e.g., risk of transition to the ICU or risk of hospitalization for more than 7 days or risk of death. The electronic device may be used in a hospital or clinical setting in which the system is used to predict future clinical events for multiple patients simultaneously, in which case the interface is further configured to display a timeline simultaneously plotting risk or probability of at least one patient over time for multiple patients.

In yet another aspect, a method of assisting a healthcare provider in providing care to a patient is disclosed.

The method comprises the following steps:

a) Generating (1) predictions of future clinical events for the patient using a predictive model trained from the aggregated electronic health records, and (2) identifying from the patient's input electronic health records related to past medical events;

b) Generating data related to both the predicted and identified past medical events; and

c) Transmitting the generated data to an electronic device used by the healthcare provider for display on the electronic device;

wherein:

the predictive model uses an attention mechanism to indicate how much attention the predictive model pays to elements in the input electronic health record to predict future clinical events and identify related past medical events, and wherein the generated data includes results of the attention mechanism.

In one embodiment, the relevant past medical event includes notes (e.g., text input from a physician or nurse) or excerpts thereof. In one embodiment, the prediction is selected from the group (group) comprising: an unplanned transfer to an intensive care unit, a stay exceeding 7 days, an unplanned readmission within 30 days after discharge, an inpatient mortality rate, a complete set of primary diagnoses, primary and secondary billing diagnoses, and atypical laboratory values.

The generated data may also include a timeline of probabilities or risks of events occurring over time.

In one embodiment, steps a), b), c) and d) are performed simultaneously in real time for a large number of patients based on a large number of input electronic health records. A healthcare provider caring for at least two patients of the plurality of patients receives the generated data of the at least two patients in real-time, thereby helping the healthcare provider provide care for the at least two patients simultaneously, and allowing patient care for the at least two patients to be prioritized based on the respective predictions.

In another aspect, the present disclosure is directed to a system comprising:

a) Computer memory storing aggregated electronic health records, including some or all of medications, laboratory values, diagnoses, vital signs, and medical records, from a large number of patients of different ages, health conditions, and demographics, and obtained in different formats, wherein the aggregated electronic health records are converted into a single standardized data structure format and ordered into an ordered arrangement by patient; and

b) A computer running one or more deep learning models to predict future clinical events on a patient's input electronic health record, the deep learning models trained on aggregated health records converted to a single standardized data structure format and ordered.

In one embodiment, the aggregated health records are in the form of health records arranged in different data formats. The standardized data structure format may take the form of a fast health interoperability resource (Fast Health Interoperability Resource, FHIR). The aggregated health record contains variable names that are inconsistent with standard terms, except for variables required to define the primary outcome and exclude the standard. The aggregated health record may contain hospitalization diagnoses, and the diagnoses are mapped to a single level Clinical Classification Software (CCS) code. In one embodiment, the electronic health records are ordered into a temporal order by patient.

In another aspect, a method for generating training data for machine learning in different data formats from an original set of electronic health records for a large number of patients from different sources is disclosed.

The method comprises the following steps:

a) Obtaining an original electronic health record set;

b) Converting the original electronic health record set into a single standardized data structure format;

c) Sequencing the electronic health records converted into a single standardized data structure format according to the time sequence of the patient; and

d) An ordered electronic health record of the timing of the standardized data structure format is stored in the data storage device.

The collection of electronic health records may include structured data and unstructured data containing free text notes. In one embodiment, the converting step b) is performed without unifying the terms in the original electronic health record to standard terms.

Drawings

FIG. 1 is a schematic diagram of an overall system including a summarized electronic health record, a computer running a trained deep learning model, and an electronic device used by a healthcare provider to receive predictions and related past medical events related to the predictions from the deep learning model and having an interface presenting such information on its display.

Figure 2 is a diagram of a procedure for converting an original electronic health record into FHIR resources in chronological order in the system of figure 1.

FIG. 3A is a flow chart illustrating the design and operation of the time-aware feedforward model of FIG. 1; FIG. 3B is a flow chart illustrating the design and operation of the embedded enhanced time series model of FIG. 1; fig. 3C is a flowchart illustrating the design and operation of the LSTM model of fig. 1.

FIG. 4 is an illustration of a form of display of data on a provider-oriented interface that displays the results of the attention mechanisms in a deep learning model in the form of a patient timeline or a series of events (including medications, meetings, procedures, notes, indications, etc.).

FIG. 5 is a graphical representation of another form of display of data in an EHR showing the results of the attention mechanisms in a deep learning model in the form of an excerpt of a note where individual words or phrases are given a degree of emphasis (size, bolstering, color, etc.) that corresponds to the attention (importance or weight) of clinical predictions generated by the deep learning model to the words and the attention of the predictions to specific medications in the medical history.

FIG. 6 is another example of the results of the attention mechanism in the deep learning model, showing that different words or phrases found in the EHR notes are given different degrees of emphasis (bolstering) related to the prediction. A deeper highlighting corresponds to a higher attention score.

FIG. 7 illustrates different types of predictions that the model of the present disclosure may be used to make, including atypical laboratory results, and accuracy statistics obtained by retrospectively applying the model to the test set portion of the initial patient record set for model training.

Fig. 8A is an illustration of a healthcare provider facing interface of an electronic device (e.g., a computer terminal, tablet, smartphone, or other type of computing device with a screen display) used by a healthcare provider for treating a patient. The interface of this configuration is designed for a hospital environment, displaying a graph of the risk of two patients at the same time. The interface displays two predicted future clinical events for a particular patient, in this case an unplanned transfer to an Intensive Care Unit (ICU) and a delayed discharge. The display of fig. 8A is designed to alert healthcare providers to a patient at risk at an early stage.

Fig. 8B is an illustration of the interface of fig. 8A after the provider has selected a patient with an alert. This display helps the provider now learn about the patient-reminding them of critical medical problems (medical events related to predictions), learn in depth about what he or she may need to look at or other data (including notes with model attention mechanism results) to make decisions about patient care and not let them miss critical information.

FIG. 9 illustrates displaying tools on an interface related to the use of the interface in an outpatient prognosis.

Fig. 10-13 illustrate hypothetical examples of the vast amount of information that a healthcare provider can obtain from an EHR and why features of the present disclosure are needed. Figure 10 shows an excerpt of a four year history of the patient listing 400 more diagnoses. Fig. 11 shows an excerpt of a patient's diagnosis, but lacks important accompanying information, such as whether the patient is an outpatient, inpatient, or receiving treatment in an ICU or other setting. Fig. 12 shows 150 different meetings of a particular patient within a given time span, but without the details of each meeting. Fig. 13 shows only a small portion of the notes taken by the provider in a single hypothesized four-day hospitalization; the display of all notes requires 60 different screens of a standard mobile device.

Fig. 14 shows an example of an interface for the device of fig. 3 to track data and risk of four patients in real time.

FIG. 15 illustrates the interface of FIG. 8B showing only selected, prediction-related critical events (ICU transfer, delayed discharge) in the past 152 meetings in the EHR and presented in the patient timeline.

FIG. 16 shows the interface of FIG. 8B showing a critical issue (ICU transfer, delayed discharge) associated with a prediction selected from a list of past 433 diagnoses in the EHR. These key questions (i.e., about past medical events) are presented as summaries in the question list area on the left side of the display.

FIG. 17 illustrates the interface of FIG. 8B showing only the prediction-related key important snippets or words (ICU transfer, delayed discharge) selected from the 12,000 words in the EHR note. In generating predictions, key snippets (words and phrases) with some degree of highlighting of particular words or phrases are presented in the lower right region of the interface, due to the use of attention mechanisms in the deep learning model.

FIG. 18 illustrates the interface of FIG. 8B showing the ability of the interface to summarize each of the listed medical problems. In this case, the provider clicks on the "alcohol break" key question in the display of FIG. 8B, and the display shows the medication, notes, and event timeline associated with the key question.

Fig. 19 shows a summary of the key problem "cardiomyopathy" in the form of a timeline, medication and related notes.

In the drawings and accompanying description, all patient and provider names and medical data are fictitious and do not reveal any confidential patient information.

Detailed Description

A. Summary of the invention

The present disclosure describes a new method of configuring EHR data for predictive model training. These models use all recorded data about the patient, including clinical notes, variables in raw non-uniform format or terminology, and preserve the temporal order of data collection. We further apply one aspect of deep learning to generate and train models for clinical prediction from EHR data. We choose deep learning because it handles millions of variables, automatically unifies data from different sources, and adapts to variable length data sequences. Deep learning techniques have achieved advanced performance in other complex fields such as medical image recognition (e.g., detection of diabetic retinopathy and cancerous skin lesions) and language translation. These deep learning models are considered new for many applications and implementations in the field of current problems.

This document further demonstrates the technical feasibility and clinical applicability of our method. We describe predictive models of multiple clinical tasks, including predicting hospital stays to increase receptionists and reduce costs; predicting an unscheduled readmission for intervention for the high risk patient; predicting hospitalized patient mortality to aid in deploying early interventions; predictive and qualitative (phenotype) diagnostics are based on routine clinical data to support clinical decisions. Furthermore, we describe the use of the model for predicting the unplanned transfer and primary diagnosis of patients to an intensive care unit in a hospital. Furthermore, we describe a user interface of a provider-oriented electronic device (e.g., a computer terminal, tablet, or smartphone) that presents these predicted and potentially relevant medical events to assist the provider in timely treatment of the patient.

Fig. 1 illustrates a system 10 for predicting and summarizing medical events from electronic health records. The system comprises three parts:

first, a computer memory 24, such as a mass data storage device, is described that stores aggregated electronic health records 22 from a large number of patients of varying ages, health conditions, and demographics, including medications, laboratory values, diagnoses, vital signs, and medical records (e.g., free text records written by attending physicians and nurses). The aggregated electronic health records are converted to a single standardized data structure format and ordered by patient, e.g., in chronological order. Because of the wide variety of legacy electronic health record systems currently in use, raw electronic health records 12 from a large number of patients in different institutions 14 (e.g., university medical centers, hospital systems, etc.) may be formatted into a variety of different electronic formats. The raw health record is patient-de-identified, transmitted over the computer network 16, stored in the relational database (Relational Database, RDB) 20, converted to a standardized format by the computer system 18 acting as a converter, and stored in the memory 24. In a preferred embodiment, the records are converted to a standardized data structure format and arranged in chronological order. In one particular embodiment, the standardized data structure format is a Fast Health Interoperability Resource (FHIR) format, a known format in which EHRs are formatted into a time-ordered FHIR "resource" bundle as shown at 22 in FIG. 1. This will be described later in connection with fig. 2.

Second, the system includes a computer 26 (which term is intended to refer to a single computer or system of computers or processing units that share processing tasks, as well as secondary memory) that runs one or more deep learning models (28, 30, 32, described below) trained on aggregated health records 22 that are converted into a single standardized data structure format and chronological. The deep learning model is trained to predict one or more future clinical events based on the input electronic health record 38 for a given patient 36 and to summarize related past medical events (e.g., questions, conditions, test results, medications, etc.) related to the predicted one or more future clinical events. The input health records 38 are in a standardized data structure format and are arranged in a chronological order, such as summarized health records for model training. As shown by the dashed line 39, the input health record 38 may be converted to FHIR format by the converter 18, if desired.

It should be appreciated that while fig. 1 shows the receipt of an input electronic health record 38 from a single patient, in practice, this may occur substantially simultaneously on many other patients in a medical system or hospital, depending on the degree of deployment (roll-out) of the system. The system of fig. 1 preferably employs sufficient computing resources for the computer 26 (or computer system) to operate the model on the input health record, and generates data about the predictions and past medical events related to the predictions simultaneously for all of these patient EHRs in real time, and transmits the data to the electronic device(s) 40 for display on the device's customer-facing interface.

Third, the system includes an electronic device 40 for use by a healthcare provider who treats the patient, such as a computer terminal or workstation, tablet, smartphone, or other type of computing device with a screen display, configured with a customer-oriented (healthcare provider) interface (fig. 8A-8B, 9, 14, etc.) that displays one or more future clinical events and related past medical events of the patient. The display of future predicted clinical events and related past medical events helps healthcare provider 42 (e.g., a nurse or doctor) focus their attention on highly relevant information related to the predictions in the patient's electronic health record, such as predictions of ICU metastasis, delayed discharge, mortality, etc. Examples of using the device 40 and interface to provide such assistance will be described later in connection with fig. 8A-19 and in the examples.

In another aspect of the disclosure, a method for predicting and summarizing a medical event from an electronic health record is described. The method comprises the following steps:

a) Summarizing electronic health records from a large number of patients with varying ages, health conditions, and demographics, the electronic health records including medications, laboratory values, diagnoses, vital signs, and medical notes;

b) Converting the aggregated electronic health records into a single standardized data structure format and ordering the patients into an ordered arrangement (see time-ordered FHIR resource bundles 22 generated by the converter 18);

c) Training one or more deep learning models 28, 30, and 32 on the aggregated health record 22 converted to a single standardized data structure format and arranged in an order;

d) Using the trained one or more deep learning models 28, 30, and 32, predicting one or more future clinical events from an input electronic health record 38 of the patient 36 having a standardized data structure format and ordered into a time series and summarizing related past medical events related to the predicted one or more future clinical events; and

e) Data (fig. 8A-8B, 14, 19, etc.) is generated for a healthcare provider-oriented interface of the electronic device 40 for use by a healthcare provider for treating a patient, the data displaying one or more future clinical events and related past medical events of the patient.

The constituent aspects of the systems and methods will now be described in more detail.

B. Integrating electronic health records into a single format for model generation

As described above, the raw electronic health record 12 may take the form of health records from a large number of patients (hundreds of thousands or even millions of patients). The aggregated health records may be obtained from one or more different institutions. EHR data may take different data formats due to industry lack of standardization. The records are converted to a standardized format and arranged in an ordered arrangement. This is shown in fig. 2, where the patient's raw electronic health record 12 includes a meeting table 50 (all visits by the patient to a doctor's office, laboratory, hospital, etc.), a laboratory table 52 containing all laboratory tests and results, and other tables (not shown) containing data such as vital sign data, medical notes (free text), demographic data, diagnostics, flowcharts, etc. Patient data is anonymous; no personal identification data is included. Permission is obtained from the institution to receive the data and train the model using the data. These tables 50, 52 representing the raw data are stored in the RDB 20 of fig. 1. The converter 18 then converts the raw data into a standardized format, in this example a set of FHIR resources 22A, 22B, 22C, 22D, etc. as shown in fig. 2, and for each patient there is a "bundle" or set 22 of such FHIR resources. These resources are then placed in chronological order, as shown at 54 in FIG. 2, to create a timeline or chronological order of all the data in the EHR.

Details of our dataset for generating predictive models are presented in annex a of U.S. provisional application serial No. 62/538,112, which we previously filed on day 28, 7, 2017. Briefly, in our model development, we obtained electronic health record data from california university san francisco (UCSF), chicago university medical center (UCM) in chicago, il and belleville medical center (MIMIC-III (Medical Information Mart for Intensive Care III, intensive care medical information marketplace 3)) in boston, ma. These electronic health records are shared in the form of de-identified or limited data sets in accordance with all state and federal privacy laws, including HIPAA (Health Insurance Portability and Accountability Act, health insurance circulation and liability act). We also used a de-identified national medical insurance and business statement (claims) database, internally called "stars of heaven (pirans)", recording 20 hundred million meetings for 7000 ten thousand patients between 2013 and 2015. The data for UCSF contains all patients meeting between 2011 and 2016 from several different scale hospital academic medical systems. The de-identified data for UCMs contains all adult patient meetings from several hospitals between 2009 and 2016. The de-identified dataset of MIMIC contained data from the intensive care unit of boston, ma and 2012 related to patient meetings. Of course, electronic health records may be aggregated and obtained from other institutions, so the details of developing the collection are not particularly important, but a large enough collection should be used to improve the accuracy of the model.

Each EHR dataset contains patient demographics, all inpatients and outpatients meetings, indications entered at EHR, diagnosis, procedure, medication, laboratory values, vital signs, and flow chart data representing all other structured data elements (e.g., care flow charts). In addition, the data sets from UCM and MIMIMIMIMI-III contain de-identified medical records, and the data sets from UCM also contain intra-operative vital sign and outpatient surgical flow chart data.

The Uranus claim dataset includes patient demographics, all hospitalization and outpatient meetings, diagnostic codes, program codes, and outpatient medication prescriptions.

Other data in the UCM dataset, except for the date, are de-identified, which data is consistent with the HIPAA present disclosure and all requirements for using a limited dataset. The ethical review and institutional review board obtains an exemption from each institution. Patient data is not linked to any google user data. Furthermore, for an aggregated electronic health record for creating a model, our system includes a sandbox infrastructure that separates each EHR dataset from each other according to regulations, data permissions, and/or data usage protocols. The data in each sandbox is encrypted; all data accesses are controlled, recorded and audited at separate levels.

We developed a single data structure for the aggregated EHR based on Fast Health Interoperability Resources (FHIR) to store data from each system for all health systems and predictions. FHIR is an open source framework that allows standardized representation of clinical data as a collection of resources-modular items containing specific data types, such as single meetings or laboratory tests. Various types of data collected by the health system are converted into corresponding FHIR resources.

When converting data into FHIR format ("resources", see fig. 2), we do not unify variable names with standard terms, but rather use raw terms provided by the health system, bypassing traditional time-consuming data unification. The only exception is the variables required to define the primary outcome and exclusion criteria: discharge dispositions, hospital services, diagnostic codes, and program codes. Hospitalization diagnosis is provided in ICD-9/10 codes, which we map to single level clinical classification software categories (CCS; healthcare research and quality institutions); hospitalization procedures are provided as ICD-9/10 and current procedural terminology (Current Procedural Terminology, CPT) program codes and mapped to CCS codes.

Next, the resource sets for a given patient are integrated in chronological order. This series of events faithfully reveals the timeline for each patient in the EHR. The billing code is immediately time stamped after the meeting is completed.

Certain elements, such as vital signs, may be entered into the EHR after collection. We model the data using the time stamp of the care document and vital sign, as this time stamp would be available in real time in the EHR, which corresponds to the data being entered into the EHR rather than when the data is recorded as being collected.

C. Deep learning model for predicting health events from medical records

As shown in fig. 1, our system includes a computer 28 (or an equivalent set of computers or processors and auxiliary memory) running a deep learning model 28, 30, and 32, the deep learning model 28, 30, and 32 being trained on a aggregated health record 22 that is converted into a single standardized data structure format and chronological. The model predicts one or more future clinical events based on the input electronic health record 38 of the patient 36 and summarizes related past medical events related to the predicted one or more future clinical events. The incoming EHRs are formatted and chronologically ordered in the same standardized data structure format, either locally or after conversion by the converter 18, if desired.

While it is theoretically possible to use only a single trained model, to avoid overfitting and provide high accuracy in predicting future clinical events, we have found that it is advantageous to use three different models, each trained on the data sets that make up the aggregated electronic health record, respectively. At least one deep learning model contains an attention mechanism that instructs the model how much attention (or equivalently, importance) to "tokens" (i.e., atomic elements in an electronic health record, such as individual words in notes, medications, laboratory results, etc.) to predict one or more future clinical events and related past medical events related to the predicted one or more future clinical events. The use of the attention mechanism in deep learning neural networks is described in the conference statement of d.bahdanau et al: neural Machine Translation by Jointly Learning to Align and Translate, jan.2014 (arXiv: 1409.0473[ cs.CL ]). Further explanation of the attentive mechanisms in the context of healthcare includes GRAM: graph-based attention model for Healthcare Representation Learning, arXiv:1611.07012v3[ cs.lg ] (month 4 2017), and RETAIN: an Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism, arXiv:1608.05745v3[ cs.gl ] (month 2 2017), by Choi et al. The contents of the Choi et al and Bandanau references are incorporated herein by reference.

In our preferred embodiment, we use three different models: a long-term short-term memory (LSTM) model 28, which is a weighted recurrent neural network model, a time-aware feedforward model (FFM) 30, also referred to herein as a feedforward model with time-aware attention, and an embedded enhanced time-series model 32, also referred to herein as a feedforward model with enhanced time-aware stumps (stumps). Further details of the architecture, design, and implementation of the three models are given in appendix B of our earlier U.S.' 112 provisional application and the description of FIGS. 3A-3C.

These models may perform a variety of predictive tasks; several of which are described in detail herein and in annex a of our earlier U.S. 112 provisional application. Including extended hospital stay, unscheduled readmission, unscheduled transfer to ICU, inpatient mortality, primary diagnostic codes, and a complete set of billing diagnostic codes at discharge. These predictions are made without task-wise selection or design of predictor (predictor) variables.

The results of the five prediction tasks are defined as follows. For each prediction we use all the information available in the EHR (except the claim database) until the prediction is made: at admission, after 24 hours or at discharge. We selected 24 hours because this is typically used in clinical predictive models such as APACHE, e.g., acute Physiology and Chronic Health Evaluation (APACHE) IV by Zimmerman et al hospital mortality assessment for today's critically ill patients, crit. Care med, 2006.

The time of admission is defined as the beginning of the hospitalization state, which means that data from emergency department and outpatient surgery will be available prior to admission.

For the MIMIC dataset, the time point is relevant for ICU admission. Since the declaration data has only a day-level attribute, predictions made on the day of admission include declarations submitted at the same calendar time of admission.

Mortality rate of inpatients

We predict the death of inpatients, defined as the discharge disposition of "expired".

Long stay inpatient

We predict a stay longer than 7 days, which is selected as a stay for most services of about 75% of the data set. The hospital stay is defined as the time between admission and discharge.

30-day unplanned readmission

Taking into account all of the above data elements during and prior to admission, we predict future unplanned readmission within the next 30 days after discharge. There is no accepted "off-plan" definition, so we use a modified form of the medical insurance and healthcare service center (Centers for Medicare and Medicaid Services, CMS) definition: readmission is excluded if it is a planned procedure for the admission of no acute complications, chemotherapy, transplantation or rehabilitation, see in detail annex B of our earlier U.S.' 112 provisional application. Readmission is counted (i.e., readmission can be counted only once) if the time of admission is within 30 days of discharge prior to a eligible index hospitalization (index hospitalization) and without any intervening hospitalization.

diagnosis-Main and complete set

For each hospitalization we classify the patient most likely to receive treatment for what by predicting the primary diagnosis (using CCS classification, the relevant diagnosis and procedure are divided into about 250 groups, such as sepsis or tuberculosis). We also predicted the complete set of primary and secondary ICD-9 billing diagnostics (i.e., a large set from 14,025 codes). We use CCS categories to make major diagnostics to simulate tasks available for decision support, which do not require accurate ICD-9 codes.

Inclusion and exclusion criteria in study queue (study mask)

We include all serial admissions of patients 18 years or older than 18 years except for one dataset that we did not use age restriction to compare with literature. We included only hospitalizations for 24 hours or more to ensure that predictions at different time points had the same cohort.

To simulate the actual accuracy of a real-time prediction system, we have not excluded patients that are typically removed by re-entry studies, such as patients discharged against orders, because these exclusion criteria are not known at the time of early prediction of hospitalization.

To predict the complete set of ICD-9 diagnoses, we exclude meetings without any ICD-9 diagnoses, which is approximately 2-12% of each dataset. These are typically meetings when the hospital switches to ICD-10 after 10 months 2015. However, for all other predictions we include such hospitalization.

For comparison with existing literature we also created a limited set of indexed hospitalizations for medical or surgical services (i.e. not including obstetrics).

Model design and training

We use three types of deep learning architectures for models 28, 30, and 32 (fig. 1) that are adapted to model a series of patient events for EHR. We have used a well-known version of a recurrent neural network called Long Term memory (LSTM) (see Hochretiter S, schmidhuber J. Long Short-Term memory, journal computer 9pp.1735-1780 (1997)), the contents of which are incorporated herein by reference. We created two new approaches, we call the time-aware feedforward model (FeedForward Model, FFM) used to create model 30, and the embedded enhanced time-series model used to create model 32, which we describe in annex B of the earlier U.S. 112 provisional application. For the first two models 28 and 30, we implement the attention mechanism (see the paper by Bahdanau et al cited earlier) to highlight the data elements that have the greatest impact on the predictions. Each model aims to address specific challenges with EHR data: long patient event sequences, dynamic changes in variables, and the effects of remote historical patient data.

Each model 28, 30, and 32 is trained on each dataset in the queue, respectively. For prediction, in most cases we average the predictions from each model, resulting in a final prediction score. In other cases, we exclude results from one of the models, e.g., where no tuning is done for a particular task or prediction, and the prediction scores for the remaining models are averaged.

Patient EHRs were randomly divided into development (80%), validation (10%) and test (10%). To prevent any implicit overfitting, the test set remains unused (and hidden) prior to final evaluation. Model accuracy was reported on the test set and bootstrap (bootstrapping) 999 times of the test set was used to calculate the 95% confidence interval. Since the goal is to create personalized predictions, rather than evaluate the contribution of individual predictors, we ignore clusters inside the patient.

For each prediction task, we create a baseline model based on existing literature using artificial (hand-spent) variables to determine the performance of the backtracking model. Details about the baseline model are described in annex B of our earlier U.S. 112 provisional application. LSTM model 28 and feedforward model 30 are trained with TensorFlow (version 1.0) and enhancement model 32 is implemented with custom C++ code. Statistical analysis and baseline model were completed in SciKit learn Python (0.18.1).

All models learn an embedded vector to represent each token (e.g., an atomic element of EHR). For example, the token may be a word in a note, the name of a medication, or a discrete value for a particular laboratory test. The embedding is randomly initialized and model training updates the embedding to improve predictive performance.

Fig. 3A is a flow chart illustrating a design and implementation of FFM 30 of fig. 1. These steps are essentially as follows

Step (1) shows the data in the original EHR with a relative timestamp (delta time), such as a predicted time of day.

Step (2) shows that each data element is embedded, which means a conversion to a d-dimensional vector (the conversion is learned by a model).

Step (3) shows that each delta time is embedded, which means that k functions are used to convert into a k-dimensional vector, which together encode a piecewise linear division (the conversion is learned by a model, yielding a predefined or learned function a ₁ ...A _k A library of (c) a).

Step (4) shows multiplying the learned projection matrix by data embedding to produce an attention data projection matrix that uses a column dot product operator to multiply the time embedding matrix to produce an alpha (alpha) vector.

Step (5) shows that the alpha vector is put into a softmax function, resulting in a beta (beta) vector.

In step (6) the beta vector is multiplied with the data embedding matrix resulting in a reduced record vector of dimension D, which is input to the feed forward network (i.e. several layers of internal ReLu (rectifying linear unit)) and at the end a sigmoid or softmax function is used to generate the prediction output.

The output of the model is the output of sigmoid, plus the attention vector learned from step (4).

Fig. 3B is a flow chart illustrating the design and operation of the embedded enhanced time series model 32 of fig. 1. The method basically comprises the following steps: step (1) shows the data of the original EHR with a relative time stamp (delta time) to, for example, the predicted time instant. In step 2), each data element is converted into a binary feature f ₀ ...f _N Indicating the presence of a particular value/token at a particular (relative) point in time. Wherein each feature represents a time t>V at T>V form. These together form an N-bit vector V.

In step (3), vector V is multiplied by (learned) embedded vector E of dimension D and summed (e.g., summed) to produce a D-dimensional vector.

In step (4), the D-dimensional vector is input into a network of e.g. several layers of ELUs (exponential linear unit, exponential linear units), ending with sigmoid. The output of the network is the output of the sigmoid function.

Fig. 3C is a flow chart illustrating the design and operation of LSTM model 28 of fig. 1. These steps are essentially as follows:

(1) For each feature class (e.g., drug, note, vital sign), each data point is embedded in D _category In a vector of dimensions.

(2) All data are considered in a bag (bag) of, for example, 1 day. For each feature type, a weighted average is calculated for all vectors in the bag, resulting in, for example, an average drug vector for the bag.

(3) For each bag, e.g. 1 day bagsSub-all average feature vectors are concatenated to produce a vector of size D, for all feature types d=d _medication +D _note +D _vital Etc.

(4) These average vectors are input into the LSTM model, each vector representing a step in the sequence.

(5) The output of the LSTM is input to a softmax function (for multi-class classification, e.g., identifying primary diagnostics) or a logic function (for probabilistic tasks, e.g., mortality).

Alternatives to these models may be suitable for use in the present system, such as, for example, an autoregressive convolutional neural network model using attention, see Attention is all you need by A.Vaswani et al, arXiv:1706.03762[ cs.CL ] (month 6 2017).

As described above, the model uses an attention mechanism that enables visualization of the granularity of "attention" or weights for specific tokens used by the model for specific predictions of the patient. Several examples will now be described in connection with fig. 4, 5 and 6.

FIG. 4 is a graphical illustration of one form of display 64 of data in an EHR showing the results of the attention mechanism in a deep learning model in a patient timeline or series of events (including medications, meetings, procedures, notes, indications, etc.). In this particular example, each circle indicates the occurrence of a particular event associated with the prediction (in this case, the risk of predicted inpatient mortality), such as a drug administration, laboratory test, procedure, note, or indication. In this example, the timeline indicates the patient's date of admission and their records (e.g., medications, notes, reports, etc.) are extracted at a particular point in time within two days. FIG. 5 is a graphical representation of another display of the form of data in an EHR showing the results of the attention mechanisms in a deep learning model in the form of an excerpt of a note where individual words are given a degree of emphasis (size, bolstering, color, etc.) that corresponds to the attention (importance, or weight) of clinical predictions generated by the deep learning model to the words and the attention of the predictions (diagnosis of metastatic melanoma with pneumonia and anemia) to specific medications in medical history. Terms from free text notes in EHRs, "melanoma," "metastatic," "wrapped," "hemoptysis," etc., are displayed in larger fonts and darker colors to direct the attention of the provider to the elements of these EHRs that are relevant or relevant to the model-generated predictions. FIG. 6 is another example of the results of the attention mechanism in the deep learning model, showing that different words found in the notes of the EHR are given different degrees of emphasis (bolded, font size) related to diagnosing the prediction of an alcohol-related disorder. A deeper highlighting corresponds to a word in the medical note with a higher attention score: abuse, withdrawal, alcoholism, and the like. Further examples of attention mechanisms in the model for driving the display of medical events in the prediction-related EHR will be discussed in the subsequent section of the provider-oriented interface.

Examples of how the attention results of fig. 4, 5 and 6 are generated are as follows: first, to identify past medical problems for a patient associated with a prediction, we run a model that identifies diagnostic codes (ICD 9 code prediction and primary diagnostic CCS code prediction, as previously described) over all historical time periods of the patient (e.g., once per historical meeting, or once per historical week). From this we have a list of predicted/identified past medical problems that we have inferred from drugs, laboratory results, vital signs, notes, etc.

These questions are ranked and presented to the physician. Ranking depends on several factors such as (1) how much evidence supports the medical problem (e.g., whether it is only mentioned in notes, or is also observed in laboratory results/vital signs, and is also treated with drugs), (2) whether the problem is explicitly accounted for and coded in the primary EHR, or what we infer but not explicitly coded or accounted for "embedded" diagnostics, and (3) how rare and severe the medical problem (e.g., aneurysms and hypertension), and potentially other factors.

Next, for each question we need to summarize key facts, such as key medications and key note excerpts and words. We interrogate the above model (which classifies the patient as, for example, a hypertensive patient) using an attention mechanism to indicate how much attention (a number between 0-1) the model gives to each word (or intuitively, a weight or importance) for example for each drug or for each word in the note. The word with the highest score is shown in the diagrams of fig. 5 and 6.

As another example, as will be described in later figures, the provider-oriented interface of the electronic device (fig. 1, 40) shows note excerpts from the EHR related to predictions of ICU transfers (rather than historical medical problems). Here again we use exactly the same attention mechanism to obtain a number between 0-1 for each input token seen by the model (e.g., each word in the note, each drug prescribed, etc.), indicating how much weight the model has applied to the word/drug/etc. while making predictions of ICU metastasis.

Further details regarding model performance, study cohorts, dataset characteristics, and results compared to baseline models are given in appendix a of our earlier U.S. 112 provisional application, which is not particularly relevant. The performance results of our model in a retrospective study of test sets in the cohort are summarized in figure 7. Fig. 7 shows different types of predictions made by the model, including readmission, mortality, unscheduled ER/hospital visits, etc. The "AUC" (area under the curve) performance metric represents the observer operating characteristic (receiver operating characteristic) area under the curve, which is a standard performance metric in machine learning.

Our findings in model development and testing are summarized below. Using deep learning on electronic health record data we show high predictive performance in predicting inpatient mortality, long hospitalization, 30 days out of plan readmission, identifying major diagnoses and assigning billing codes at discharge. We show that the results are consistent across healthcare systems and clinical tasks, increasing with the availability of new patient data, and can be interpreted using the attentive mechanisms. We have four key findings, as follows. See annex a of our earlier U.S. 112 provisional application for more details.

The results may be spread across different data sets

First, our method accommodates unstructured data, such as free text notes across multiple clinical sites, and can use all of the data of the EHR for model training and creating accurate predictions. Our method does not require manual selection of variables nor decision on how to clean, extract and unify them from the raw data of a particular site. The predictive model in the literature uses a median of 27 variables, whereas we use a median of over 100,000 data points, including variables that are typically difficult to incorporate, such as clinical notes and flowcharts.

Predictive performance shows color in different tasks

Second, our results indicate that our method of representing and modeling EHR data can be extended across clinical tasks, and we believe our results are superior to comparable studies in terms of mortality (0.94-0.98 to 0.91), readmission (0.74-0.75 to 0.69), and hospital stay (0.86-0.92 to 0.77). Our performance in terms of ICU mortality and readmission is also superior to physician discrimination.

However, considering that performance varies based on cohort selection and study design, it is difficult to compare our results with other studies; many results have incomplete descriptions of the queues and results, are predicted based on smaller, disease-specific queues, or use data that is not conventionally available in real-time.

To address this limitation, we implemented versions of the hospital, NEWS score (NEWS score), and Liu's model as baselines, see annex B of our earlier U.S. provisional application, 112, and demonstrated excellent performance. We also evaluated a cohort designed to be more similar to related studies of medical or surgical service patients and found similar benefits to our approach.

In addition, we used an open dataset MIMIC, where we outperformed the existing literature with 0.91 to 0.80 in AUC of mortality and 0.4 to 0.28 in micro F1 of ICD classification.

Modeling utilizes values from a complete data sequence

Third, our modeling technique successfully updates predictions as new data becomes available, rather than using fixed points in time. In all tasks, the model uses hundreds of thousands of patient attributes to significantly improve the performance of all tasks. Interestingly, our model extracted almost equivalent discriminative performance from declarative data as EHR data; in fact, when an unscheduled readmission is predicted, the declared performance exceeds that of the EHR data, most likely due to a complete view of readmission at other hospitals.

The output of the complex model is interpretable

Fourth, we demonstrate an attention mechanism that enables visualization of the granularity of the data used by the model to make specific predictions for the patient (see fig. 4-6 and the discussion of interfaces in fig. 8-9 and 14-19 below). Because we explicitly model the order and time of patient events, our method does show what, time and place in the patient history are relevant to the predictions. While there is no simulation of the yield ratios (odds ratios) used to describe how each variable contributes to the results, we believe that attention techniques can alleviate the concern that deep learning is a "black box" and can be used in the future to extract prominent information for the clinician. Presentation of potential past medical events associated with predicted future clinical events in the interface gives the healthcare provider confidence: deep learning models are actually providing timely and useful information.

Limitations of

Tags in the dataset used for model development and training may be clinically incorrect or missing. Billing diagnosis may not reflect clinical diagnosis; for example, pneumonia is increasingly coded as sepsis for regulatory reasons. Similarly, readmission typically occurs in a separate medical system, and these records are typically not shared with the hospital system that is discharged. In the absence of complete data sharing between health institutions or data sets with study-level qualitative, this limitation can affect all data collected in real-time clinical care.

A second limitation is that our approach relies on large data sets, powerful computing infrastructure, and complex algorithms, which require complex engineering to replicate. However, this approach allows a single modeling architecture to achieve excellent predictive performance in a range of predictive tasks, and is within the ability of one skilled in the art in view of this disclosure and the attached appendix.

Finally, there is concern that using many variables will always result in an overfitting. We alleviate this concern by reporting the results of the retention (hold-out) test set of unused patients during training, which estimates real world performance, and by displaying the results of the 3 independent data sets. Furthermore, the design of the model may include techniques to avoid overfitting.

Although several types of predictions have been described above, these models may be used for other prediction tasks, including:

medicaments and doses, with the aim of automatically completing and alerting to abnormal doses or accidental prescriptions (sources of medical errors).

The next word, sentence or paragraph in the physician note, such as a discharge summary, is intended to automatically complete or suggest a document template or section for review, editing and submission by the physician (writing a document is a major time burden).

A wide variety of life threatening events are predicted, such as intubation, ventilation, changes in sensitivity of care (e.g., ICU metastasis), organ support, transplantation, etc., with the aim of monitoring and alerting such events.

Physiological deterioration is predicted, for example daily or before instructing laboratory tests, or for example before administering glucose (in order to prevent e.g. hyperglycemia/hypoglycemia). The total cost of care is predicted with the aim of risk stratification for high cost patients.

Admission and screening (how many patients each ward will take) are predicted for capacity planning.

D. Provider interface for clinical prediction and understanding through deep learning

Once the predictive models 28, 30, and 32 have been developed, tested, and validated as described above, they may then be used to predict an EHR from patient input, as shown in FIG. 1, to improve patient care. In this section of this document, we will describe how these predictions, as well as the identification of past medical events (test results, diagnoses, notes, medications, etc.) in the EHR, are presented to the healthcare provider. In essence, the computer 26 of FIG. 1 uses the model(s) 28, 30, and or 32 to generate data from the input health record regarding predicted and related past medical events and provides the data to the electronic device 40 for rendering on the interface.

Fig. 8A is an illustration of a healthcare provider facing interface 100 of an electronic device 40 (e.g., a computer terminal, tablet, smartphone, or other type of computing device with a screen display) used by a healthcare provider of a treating patient. The interface 100 of this configuration is designed for use in a hospital environment. The interface includes two patient display areas 102 and 104. For patient "Mark Smith", the display includes an alert 104 that instructs the predictive model to predict two future clinical events for that particular patient, in this case an unplanned transfer to an Intensive Care Unit (ICU) and a delayed discharge. The interface of fig. 8A is designed to alert healthcare providers to a patient at risk early. The system of fig. 1 accurately predicts certain events where something is "off", "abnormal" or "attention needed". From the perspective of physicians, the interface meets the need for early alerts when they have time to take action. Furthermore, as will be explained in connection with fig. 8B, the interface explains why the predictive model believes/predicts that an alarm condition will occur.

FIG. 8A also shows other aspects of interest, including a toolbar 108 that allows a physician to select a graphical display of different probabilities (or risks) on a Y-axis scale of 0-100 in the timeline area 105 of the display 102. In this case, the physician has switched the risk/probability of death, discharge and ICU transfer to the "on" position. Line 110 depicts the probability of discharge. Line 112 plots the probability of an ICU transition. Line 114 plots the risk of death. Note that about 16: at 00, the risk of ICU metastasis increases dramatically, and shortly thereafter, the risk of death increases slightly. Physicians can explore these risk/probability graphs by clicking or selecting alert icon 104 and find more information about past medical events related to ICU transfer and delayed discharge risk.

When the alert icon 104 is selected, the interface 100 changes to the display shown in fig. 8B. Basically, this version of the interface helps the physician now know about the patient, including the predictions made and the related previous medical events. The physician is considering: what are the key medical problems i need to know? Helping me get insight into the conditions or other data that i may need to view in making a decision. I do not let me miss critical information. "these needs are met by the display of FIG. 8B. In particular, in region 130, a list of questions associated with the alert is displayed in real-time based on the EHR: current hospitalized complaints (abdominal pain), previously hospitalized complaints (alcohol withdrawal, atrial fibrillation), critical inferential diagnosis (i.e., diagnosis inferred by models 28, 30, 32), and other major medical conditions of the patient (pre-diabetes, hypertension, and tinea cruris). In region 140, current laboratory results are displayed. In region 150, an excerpt of a medical note is displayed relating to the prediction of ICU diversion and delayed discharge, the results of the attention mechanisms in the model highlighting in red font the particular element or word in the note that is scored high by the attention mechanisms (in this case, "alcohol withdrawal", "fear of alcohol withdrawal"). In region 160, the current vital sign is displayed. In region 120, a timeline showing previous hospitalization, ER visits, and outpatient activities is displayed. Line 122 is a visualization of the patient's healthcare utilization intensity, as well as the amount of data available to the patient, e.g., how frequently they visit the healthcare facility, how many laboratory/vital sign tests were performed, how much medications were prescribed, etc.

The interfaces of fig. 8A and 8B are discussed further below in the description of the examples.

As previously mentioned, predictive models may also be used in an outpatient setting to predict patients. For example, fig. 9 shows a display of the interface 100 and the tool 200 on the interface, the tool 200 will be related to the use of the interface in the physician's office. Tool 200 allows a physician and his or her care team to draw a timeline of risk/probability of emergency department visits, hospitalizations, deaths (similar to that shown in area 105 of fig. 8A) and general cost/utilization of medical resources similar to the display line in fig. 8B.

Example 1-what happens today without the benefit of the present disclosure

An example of this assumption will illustrate the difficulty of patient care without the benefit of the present disclosure.

The patient "Mark Smith" arrived in the emergency room at 7 a.m. due to severe abdominal pain.

He performed a comprehensive examination, including laboratory tests and abdominal ultrasonography, but did not find an explicit cause. His abdominal exam is relatively benign, but he still needs Intravenous (IV) analgesics.

At 10 am, he was treated by the medical team for observation and pain control. The admission team suspects that it is nonspecific gastritis and they expect to discharge the hospital the next day.

The main team signs off (sign-out) at 4 pm, hands over to the attending physician (responsible for 130 patients), and warns: mr. Smith may develop alcohol withdrawal but there is no sign yet. "8 pm, the attending physician will sign all 130 patients back to the night shift physician, kingsley physician. At 8 pm, the Kingsley doctor enters her night shift. She was responsible for 130 patients she had not been cared for before. She starts a shift and transfers the first-call pagers of all 130 patients to her own pagers. At 10 pm, the Kingsley doctor receives the page:

the heart rate of the patient Smith in floor 21, room 14, was 99, and the patient was comfortable watching television in bed. BP 115/79, RR 20, 98% RA. For reference, the call parameter is 90

At 1 a.m., the Kingsley doctor receives another page:

the Smith of the 14 th floor 21 room has septicemia alarm and please dial back 3-9124

The Kingsley doctor logs in to the EHR and sees an alarm:

sepsis alarm. Patients meet SIRS criteria. According to national guidelines, 30cc/kg intravenous fluid and antibiotic are administered within 1 hour

After intensive investigation, the Kingsley physician found that the patient's heart rate had risen from 70 to 99 in the daytime, now to 110 again, and his respiratory rate (respiratory rate) was recorded as 20 (normal number of breaths recorded at normal). Lactate (indicated by nurses) was 2.5 (slightly elevated). The Kingsley doctor's pager now sounds every 45 seconds so she must split (triage) her time. She returned her electricity to the nurse reporting "he looked less good, somewhat shivering and sweating". When she speaks, she takes the daytime note where the main problem is "blind abdominal pain". The above is continued to write "patients had nonspecific abdominal pain and slight liver enzyme elevation, and ultrasound examination had nonspecific gallbladder thickening. Suspected gastritis may be due to alcohol consumption, but is denied by the patient. Monitoring for intra-abdominal pathology. "

Sepsis alarms reminiscent of clinical rules that the mortality rate increased by 7.5% for sepsis with antibiotics every hour of delay. She wants the patient, but may not be able to give him an additional 30 minutes of examination, as this would likely delay the use of antibiotics by more than 1 hour. She was minded for intra-abdominal infection. She looks at whether he has made an echocardiogram, but he does not.

In the morning 1:00, she indicated 2L IVF (intravenous fluids, intravenous), vancomycin and zosyn (antibiotics), and abdominal-pelvic CT contrast.

An alarm sounds overhead 2:10 a.m..

Emergency rescue (code blue): 14 building 21 room

The Kingsley doctor runs to floor 14, room 21, and finds mr. Smith dyspnea. The second bag IV fluid is almost complete. She auscultates his lungs and notices no obvious sound changes (crack) recorded by the daytime team. His JVD was significantly elevated. She also noted his apparent tremors and tremors. When again queried, the patient acknowledges that there was a great deal of alcohol consumption in the past week, but stopped by two days because of abdominal pain. She stopped IV fluid, called an Intensive Care Unit (ICU) team, and transferred the patient to the ICU because of iatrogenic acute pulmonary edema and alcohol withdrawal.

The solution for this example is as follows: the patient actually causes gastric irritation due to the use of alcohol and ibuprofen, resulting in abdominal pain. While at the hospital he starts to get alcohol withdrawal, which is why his heart rate is fast, shivering and sweating. The clinician is also unaware that his outpatient physician is concerned about alcoholic cardiomyopathy due to the progressive deterioration of exercise tolerance and has indicated an as yet ongoing echocardiographic examination.

After the actual examination of the patient from the time of emergency rescue, the physician diagnoses acute pulmonary edema from her extracted fluid and recognizes alcohol withdrawal. Patients were transferred to the ICU, treated, and discharged 4 days later. After 3 weeks, the patient was readmitted for clostridium difficile colitis, most likely due to improper antibiotic administration.

What is the root cause analysis asking the Kingsley doctor to miss? The patient is evaluated on the fly only after the patient has deteriorated. The patient had alcoholic gastritis and alcohol withdrawal, which was mistaken for sepsis and improper treatment. The patient is suspected of having cardiomyopathy: and should not receive infusion without physical examination and ECG. What should happen?

1. It should be predicted and prevented from waiting for ICU transfer due to alcohol withdrawal.

2. IV fluid should not be administered to prevent ICU transfer due to excess fluid.

3. Antibiotics should not be administered to prevent hospital acquired infections.

4. Should prevent subsequent readmission

Both the framing bias (scaling bias) and the validation bias (confirmation bias) help explain why this occurs. The framing deviation is: is i refused to rescue treatment for patients who may have sepsis? The confirmation bias is that, given the information density, the physician only looks for the source of possible abdominal sepsis.

Fig. 10-13 illustrate examples of the vast amount of information that a healthcare provider can obtain from an EHR and why features of the present disclosure are needed. Figure 10 shows an excerpt of the patient's four year history listing 433 diagnoses. Fig. 11 shows an excerpt of the diagnosis of this patent, but the lack of important accompanying information (such as whether the patient is an outpatient, an inpatient, or is receiving treatment in an ICU or other environment) limits the usefulness of the information. Fig. 12 shows a vast list listing the different meetings of the particular patient over a given time span, but lacking details of each meeting. Fig. 13 shows only a small fraction of notes taken by the provider within 4 days of a single hospitalization, a total of 33 notes, about 10,000 words, that can fill up 60 different screens of a standard mobile device.

In short, there is a need to assist the Kingsley physician in focusing her attention on only those elements of the EHR that are actually relevant to the patient's current condition. Patient care in example 1 may be improved, thus developing the system of the present disclosure.

Example 2-prediction of clinical events: ICU transfer and delayed discharge

This example will illustrate the benefits of the system of fig. 1 in treating a patient "Mark Smith" in example 1. In summary, the system alerts physicians to notice at an early stage patients at risk by accurately predicting specific events; when they have time to take action, they are alerted early and explain why the system made the predictions. Once they have attention (e.g., by using the alert in fig. 8A), this will help the physician now know what the patient-critical issue is, what conditions and other data the physician may need to see when making the decision, rather than letting them miss critical information.

In fig. 14, an example of the interface 100 of the device of fig. 3 tracks data and risk of four patients in real time. Physicians have switched tools 108 to customize tracking risk or probability in real time. In fig. 14, the interface includes four display areas 300, 302, 304, and 306 for four different patients, the display area 300 being that for the patient Mark Smith, and the graph and alarm 104 being as described in fig. 8A.

In our hypothetical example, 8:02 pm, the kingsley doctor starts her shift at 8 pm and logs into the system that provides the interface 100 of the present disclosure, which is referred to as the "guardian" in this document. She first looks at Jerry mashitar, who is informed that Jerry mashitar is an "observer", which is confirmed by the guardian, because the graph shows an increased risk of death as indicated by line 303. At 8:03 night, alarm 104 is activated and she notices Mark Smith is first in the patient list. The warning is that the patient is at risk of ICU transfer and delayed discharge.

The Kingsley doctor had immediately formed a question in the brain: what is the patient's current medical problem? How severe they were alcohol to stop in the past? Do they need to stay in the ICU? What is they receiving treatment for heart failure? Is their ejection fraction reduced? Is they recently infected or received antibiotics? Is there a positive bacterial response? Is atrial fibrillation difficult to control? Is the patient suddenly stopped taking the beta blocker? In other words, what is the critical issue for the patient, when it appears, what is there evidence?

The Kingsley physician activates icon 104 and the display of fig. 8B appears. This interface shows the risk of ICU transfer and directs her attention to the concern about alcohol withdrawal that caused the risk, as note area 150 shows notes of "possibly having a history of alcohol withdrawal episodes", "severe alcohol use", and "fear of alcohol withdrawal". The phrases "alcohol withdrawal" and "fear of alcohol withdrawal" are shown in red and bold. This is a result of using the attention mechanism in the predictive model, as previously described. Thus, the display of FIG. 8B summarizes the past medical events of the predicted current risk (ICU transfer).

FIG. 15 illustrates the interface of FIG. 8B showing only selected, prediction-related critical events (ICU transfer, delayed discharge) in the past 152 meetings in the EHR and presented in the patient timeline area.

Fig. 16 shows the interface of fig. 8B showing only the critical questions (ICU transfer, delayed discharge) relevant to the prediction selected from the list of past 433 diagnoses or questions in the EHR. These key questions (i.e., about past medical events) are presented as summaries in the question list area on the left side of the display.

FIG. 17 illustrates the interface of FIG. 8B showing only the prediction-related key important snippets or words (ICU transfer, delayed discharge) selected from the 12,000 words in the EHR note. In generating predictions, key snippets (words and phrases) with some degree of highlighting of particular words or phrases are presented in the lower right region of the interface, due to the use of attention mechanisms in the deep learning model.

FIG. 18 illustrates the interface of FIG. 8B showing the ability of the interface to summarize each of the listed medical problems. In this case, the provider clicks on the "alcohol break" key question 400 in the display area 130 of FIG. 8B, and the display shows the medication in field 402, notes or excerpts thereof in field 404, and the event timeline in field 406 associated with the key question "alcohol break".

FIG. 19 shows what happens when the user selects the "Critical inference" question "cardiomyopathy" and the display shows a summary of the critical question "cardiomyopathy" in the form of a timeline 506, medication 502, and related notes in field 504 or excerpts thereof. The notes or snippets in fields 504 and 404 again use highlighting (bold, font size, etc.) to indicate the outcome of the attention mechanism in the model to again display to the physician the EHR elements that are most important in generating the prediction.

Returning again to the description of the treatment of patient Mark Smith using features of the present disclosure, in example 1, the Kingsley doctor receives a sepsis page at 1 am, with features of the present disclosure, the Kingsley doctor indicates the desired intervention in advance. Immediately after presenting the alarm she looks at the patient, considering that the risk is very high, she indicates the CIWA protocol for withdrawal. She found that the outpatient patient was suspected of having cardiomyopathy and decided not to give 2L IV in case the patient had heart failure. Taking into account the uncertainty of the diagnosis, she decides to also instruct the ECG and check the patient based on the medical history of atrial fibrillation.

In summary, the system of the present disclosure avoids the need to transfer the patient to the ICU, and also avoids the need to re-hospital later. In this example, physicians are given timely alerts of predicted clinical events, presenting predicted critical medical events, enabling physicians to improve their care for patients, avoid ICU metastasis, avoid unscheduled readmission, and avoid complications from antibiotic administration.

Example 3-out-of-office alarms for risk of ER or hospitalization

This example will explain the use of the system of fig. 1 in an outpatient setting.

Jennifer Choi is an 83 year old female with a history of heart failure (EF 30%), with warfarin for atrial fibrillation, hypertension and pre-diabetes, currently seen as a new patient in the clinical cardiac clinic of Key doctors. The Keyes doctor is required to treat her heart failure. Choi women want to ensure that the Keyes doctor knows about her other conditions to ensure that the treatments do not interfere with each other.

Earlier, choi women underwent laboratory examination before the primary care physician (primary care physician, PCP) visit, and she was found to have mild acute kidney injury. Her PCP felt that her body fluid state was stable so he reduced the dose of diuretic and recommended that laboratory tests be repeated within a week.

The daughter's private bottom of Choi women, when out after meeting with the key doctor, presents a concern that her mother is increasingly confused and that her mother may not take the medication correctly. The Keyes doctor has been 30 minutes later than the original program, she says she will further investigate this in the meeting after 3 weeks and ask her to solve the confusion the next visit in the note: "daughter worry about the patient becoming confused. MOCA was planned and cognitive impairment was assessed at the next visit.

Both PCP and Key doctors participate in the system of FIG. 1 and forward the Choi lady's EHR information to the computer 26 of FIG. 1 to apply the predictive model. Both the PCP and the keys doctor have electronic devices (workstations) that include the interface of fig. 9 for outpatients.

The model predicts that Choi women will have a risk of ED visits/hospitalizations within the next 14 days. The alarm is presented on the display screen of fig. 9. The team has expertise in managing these high risk situations. The display will show a timeline (including recent hospitalization), such as shown in column 406 of fig. 18, which will show inferred questions: CHF (congestive heart failure ), AKI (acute kidney injury, acute kidney injury), AFib (atrial fibrillation ), pre-diabetes, hypertension in column 130 of fig. 8B, and will include note excerpts in column 150 (fig. 8B):

Note 1 (PCP): "daughter worry about the patient becoming confused. MOCA was scheduled for the next visit and cognitive impairment [.] patients were assessed for worsening renal function, possibly due to excessive urination. The dose of furosemide will be reduced and the experiment repeated within 1 week. Telling girl to monitor weight and respiration "

Note 2 (nurse): patient-to-furosemide dose very paste "

Note 3 (nurse): "I don't know what dosage I'm should serve"

Due to the use of the attention model, the key parts of these notes are shown in bold- "attention", "paste", "MOCA", "worsening of renal function", "reducing furosemide dose", "very paste to furosemide dose", etc.

Example 4-busy emergency department

This example will illustrate the use of the features of the present disclosure in examples 1 and 2 for a hypothetical patient "Mark Smith".

"Mark Smith walks into the emergency room and covers the abdomen, complaining of pain. His heart rate was 110, body tremors, sweating, dyspnea. Nurses call the ED hospitalization Peters doctor to help ascertain what has happened.

Peters doctors have many problems. Is he previously hospitalized? What is he suffering from? How severe these diseases are? How are these diseases treated? The ED extracts the Mr. Smith EHR to which the predictive model of FIG. 1 is applied. The interface of the terminal or other electronic device presenting the interface extracts and displays information related to these questions and his current complaints and includes predicted diagnostic and critical potential medical events, as shown in fig. 8B. When the current vital signs are obtained, they are added to the display of the relevant chart information.

Further consider

The exact physical location and implementation of the predictive model and associated computer or computer system 26 may vary. In some cases, it may be physically located in a hospital, primary care physician office, and related clinic, etc. of a medical system or service accessory facility. In other cases, it may be centrally located and receive EHRs, transmit predicted future clinical events and related prior medical events over a wide area network, and serve a number of unrelated healthcare institutions with service fees, subscription fees, independent product fees, or other business models. In all cases, there is an appropriate data security and HIPPA compliance program.

Claims (28)

1. A system for predicting and summarizing medical events from electronic health records, comprising:

a) Computer memory storing aggregated electronic health records from a large number of patients of different ages, health conditions, and demographics, including some or all of medications, laboratory values, diagnostics, vital signs, and unstructured free-text medical records, wherein the aggregated electronic health records are converted into a single standardized data structure format and ordered by patient;

b) A computer running one or more deep learning models to predict one or more future clinical events based on the patient's input electronic health records, the deep learning models trained on aggregated electronic health records converted to a single standardized data structure format and ordered; and

c) An electronic device for use by a healthcare provider for treating a patient is configured with a healthcare provider-oriented interface that displays one or more future clinical events predicted by the patient,

wherein a particular one of the at least one of the one or more deep learning models contains an attention mechanism that indicates how much attention the particular deep learning model gives to elements in the electronic health record to predict one or more future clinical events, wherein the particular deep learning model generates an output that includes both the predicted one or more future clinical events and attention mechanism results, wherein the computer running a particular one of the one or more deep learning models comprises: (1) Generating a respective data embedding vector and a respective time embedding vector for a plurality of elements of the input electronic health record, wherein each data element is embedded and converted to a d-dimensional vector and each delta time is embedded and converted to a k-dimensional vector, (2) multiplying the learned projection matrix by the data embedding to produce an attention data projection matrix, the attention data projection matrix being combined with the time embedding matrix to produce an alpha (alpha) vector; (3) Placing the alpha vector into a softmax function to produce a beta (beta) vector; and (4) multiplying the beta vector with the data embedding matrix to obtain a reduced record vector of dimension D, which is input to the feed forward network, using a sigmoid or softmax function at the end, producing a predicted output.

2. The system of claim 1, wherein the aggregated electronic health record comprises health records arranged in at least two different data formats.

3. The system of claim 1 or claim 2, wherein the standardized data structure format comprises Fast Health Interoperability Resources (FHIR).

4. The system of claim 1, wherein each of the deep learning models is trained on a data set that constitutes a summarized electronic health record, respectively.

5. The system of claim 1, wherein the predictions of one or more future clinical events and summarized related past medical events related to the predicted one or more future clinical events are obtained from a collective average of the deep learning models.

6. The system of claim 1 or claim 2, wherein the predicted one or more future clinical events include at least one of: an unplanned transfer to an intensive care unit, an out-of-hospital stay exceeding 7 days, an out-of-plan readmission within 30 days after patient discharge, an in-patient mortality rate, a complete set of primary diagnoses, primary and secondary billing diagnoses, or atypical laboratory values obtained from acute kidney injury, hypokalemia, hypoglycemia, or hyponeutrophilia.

7. The system of claim 1, wherein the interface of the electronic device comprises a display of:

(1) An alert to a predicted one or more future clinical events,

(2) Critical medical problems or conditions associated with alarms

(3) Notes related to the alert or excerpts thereof.

8. The system of claim 7, wherein at least one of the one or more deep learning models each includes an attention mechanism that indicates how much attention the one or more deep learning models pay to elements in an electronic health record to predict one or more future clinical events and summarize related past medical events related to the predicted one or more future clinical events, and wherein the display of the notes or excerpts thereof is displayed in a manner that indicates results from application of the attention mechanism.

9. An apparatus for predicting and summarizing medical events from an electronic health record, comprising:

a) Means for aggregating electronic health records from a large number of patients of different ages, health conditions, and demographics, the electronic health records including some or all of medications, laboratory values, diagnostics, vital signs, and unstructured free-text medical notes;

b) Means for converting the aggregated electronic health records into a single standardized data structure format and ordering the aggregated electronic health records into an ordered arrangement according to patient order;

c) Means for training one or more deep learning models on the aggregated electronic health records converted to a single standardized data structure format and arranged in an order;

d) Means for predicting one or more future clinical events from the input electronic health records of the patient ordered in time series using the trained one or more deep learning models; and

e) Means for generating data for an electronic device for use by a healthcare provider for treating a patient, the electronic device having a healthcare provider-oriented interface for displaying one or more future clinical events predicted by the patient,

wherein a particular one of the one or more deep learning models includes an attention mechanism that indicates how much attention the particular deep learning model gives to elements in the electronic health record to predict one or more future clinical events, wherein the particular deep learning model generates an output that includes both the predicted one or more future clinical events and attention mechanism results, wherein using a particular one of the one or more deep learning models includes: (1) Generating a respective data embedding vector and a respective time embedding vector for a plurality of elements of the input electronic health record, wherein each data element is embedded and converted to a d-dimensional vector and each delta time is embedded and converted to a k-dimensional vector, (2) multiplying the learned projection matrix by the data embedding to produce an attention data projection matrix, the attention data projection matrix being combined with the time embedding matrix to produce an alpha (alpha) vector; (3) Placing the alpha vector into a softmax function to produce a beta (beta) vector; and (4) multiplying the beta vector with the data embedding matrix to obtain a reduced record vector of dimension D, which is input to the feed forward network, using a sigmoid or softmax function at the end, producing a predicted output.

10. The apparatus of claim 9, wherein the aggregated electronic health records comprise health records from a large number of patients arranged in different data formats.

11. The apparatus of claim 9, wherein the standardized data structure format comprises a Fast Health Interoperability Resource (FHIR).

12. The apparatus of claim 9, wherein each of the deep learning models is trained on a data set that constitutes a summarized electronic health record, respectively.

13. The apparatus of claim 9, wherein the predictions of one or more future clinical events and summarized related past medical events related to the predicted one or more future clinical events are obtained from a collective average of the deep learning models.

14. The apparatus of any of claims 9-13, wherein the one or more deep learning models predict one or more future clinical events comprising at least one of: an unplanned transfer to an intensive care unit, an out-of-hospital stay exceeding 7 days, an out-of-plan readmission within 30 days after patient discharge, an in-patient mortality rate, a complete set of primary diagnoses, primary and secondary billing diagnoses, or atypical laboratory values obtained from acute kidney injury, hypokalemia, hypoglycemia, or hyponeutrophilia.

15. The device of claim 9, wherein the device further comprises means for generating data for display on an interface of the electronic device:

(1) Alarms for one or more future clinical events,

(2) Critical medical problems or conditions associated with alarms

(3) Notes related to the alert or excerpts thereof.

16. The apparatus of claim 15, wherein the one or more deep learning models each contain an attention mechanism that instructs the one or more models how much attention to elements in an electronic health record to predict one or more future clinical events and summarize related past medical events related to the predicted one or more future clinical events, and wherein the display of the notes or extracts thereof is displayed in a manner that instructs results from the application of the attention mechanism.

17. The device of claim 16, wherein the displaying of the note or snippet thereof indicating a result from the application of the attention mechanism includes displaying the note or snippet thereof using at least one of the following to provide a highlighting or emphasis level for a particular word, phrase, or other text in the note: font size, font color, shading, bold, italics, underline, strikethrough, blinking, highlighting with color, and font selection.

18. An improved computer for predicting and summarizing medical events from electronic health records, comprising:

a processor running one or more deep learning models to predict one or more future clinical events based on input electronic health records of patients ordered in time series, the deep learning models trained on aggregated electronic health records converted into a single standardized data structure format and ordered, and

wherein a particular one of the one or more deep learning models contains an attention mechanism that indicates how much attention the particular deep learning model gives to elements in the electronic health record to predict one or more future clinical events, wherein the particular deep learning model generates an output that includes both the predicted one or more future clinical events and attention mechanism results, wherein the processor runs a particular one of the one or more deep learning models comprises: (1) Generating a respective data embedding vector and a respective time embedding vector for a plurality of elements of the input electronic health record, wherein each data element is embedded and converted to a d-dimensional vector and each delta time is embedded and converted to a k-dimensional vector, (2) multiplying the learned projection matrix by the data embedding to produce an attention data projection matrix, the attention data projection matrix being combined with the time embedding matrix to produce an alpha (alpha) vector; (3) Placing the alpha vector into a softmax function to produce a beta (beta) vector; and (4) multiplying the beta vector with the data embedding matrix to obtain a reduced record vector of dimension D, which is input to the feed forward network, using a sigmoid or softmax function at the end, producing a predicted output.

19. The improved computer of claim 18, wherein each of the deep learning models is trained on a data set that constitutes a summarized electronic health record, respectively.

20. The improved computer of claim 18, wherein the predicted one or more future clinical events and summarized related past medical events related to the predicted one or more future clinical events are obtained from a collective average of the deep learning models.

21. The improved computer of claim 18, wherein the predicted one or more future clinical events comprise at least one of: an unplanned transfer to an intensive care unit, an out-of-hospital stay exceeding 7 days, an out-of-plan readmission within 30 days after patient discharge, an in-patient mortality rate, a complete set of primary diagnoses, primary and secondary billing diagnoses, or atypical laboratory values obtained from acute kidney injury, hypokalemia, hypoglycemia, or hyponeutrophilia.

22. The improved computer of any of claims 18-21, wherein the processor is operative to generate predictions of a plurality of input electronic health records for different patients in substantially real-time.

23. A system for predicting a future clinical event from an electronic health record, the system comprising a combination of:

a) Computer memory storing aggregated electronic health records from a large number of patients of different ages, health conditions and demographics and obtained in different formats, including some or all of medications, laboratory values, diagnostics, vital signs and unstructured free-text medical records, wherein the aggregated electronic health records are converted into a single standardized data structure format and ordered into an ordered arrangement by patient; and

b) A computer running one or more deep learning models to predict future clinical events based on the patient's input electronic health records, the deep learning models trained on aggregated electronic health records converted to a single standardized data structure format and ordered,

wherein a particular one of the at least one of the one or more deep learning models contains an attention mechanism that indicates how much attention the particular deep learning model gives to elements in the electronic health record to predict one or more future clinical events, wherein the particular deep learning model generates an output that includes both the predicted one or more future clinical events and attention mechanism results, wherein the computer running a particular one of the one or more deep learning models comprises: (1) Generating a respective data embedding vector and a respective time embedding vector for a plurality of elements of the input electronic health record, wherein each data element is embedded and converted to a d-dimensional vector and each delta time is embedded and converted to a k-dimensional vector, (2) multiplying the learned projection matrix by the data embedding to produce an attention data projection matrix, the attention data projection matrix being combined with the time embedding matrix to produce an alpha (alpha) vector; (3) Placing the alpha vector into a softmax function to produce a beta (beta) vector; and (4) multiplying the beta vector with the data embedding matrix to obtain a reduced record vector of dimension D, which is input to the feed forward network, using a sigmoid or softmax function at the end, producing a predicted output.

24. The system of claim 23, wherein the aggregated electronic health records comprise health records arranged in different data formats.

25. The system of claim 23, wherein the standardized data structure format comprises Fast Health Interoperability Resource (FHIR).

26. The system of claim 23, wherein the aggregated electronic health record contains variable names that are inconsistent with standard terms, except for variables required to define primary outcome and exclude standards.

27. The system of claim 23, wherein the aggregated electronic health record contains hospitalization diagnoses, and wherein the diagnoses are mapped to a single level Clinical Classification Software (CCS) code.

28. The system of any of claims 23-27, wherein the electronic health records are ordered in chronological order by patient.

Images