JP2020529057A - Systems and methods for predicting and summarizing medical events from electronic health records - Google Patents

Abstract

A system for predicting and summarizing medical events from electronic health records is an aggregation of diverse age, health status, and demographics from a large number of patients, including medications, test values, diagnoses, vital signs, and medical notes. Includes computer memory for storing electronic health records. Aggregated electronic health records are converted, for example, in order of occurrence into a single standard data structure format and a regular sequence for each patient. The computer (or computer system) has a standard data structure format and predicts one or more future clinical events based on the patient's input electronic health records arranged in order of occurrence and is related to the predicted events. Run one or more deep learning models trained on an aggregated health record to summarize a past medical event. An electronic device consisting of a healthcare provider-opposed interface displays one or more predicted future clinical events for a patient, as well as related past healthcare events.

Description

Priority This application claims the benefit of priority under 35 USC 119 over US Provisional Application No. 62 / 538,112 filed on July 28, 2017. The entire contents of this provisional application No. 62 / 538,112, including Appendix A and Appendix B, are incorporated herein by reference.

The present disclosure covers systems and methods for predicting and summarizing medical events from electronic health records using deep learning models. This disclosure provides the basis for the integration of electronic health records into a single format for model generation and training, deep learning models for predicting health events from medical records, and clinical predictions and predictions obtained through deep learning. Relationships Also covered are aspects of several components and combinations thereof, including provider-opposed interfaces on electronic devices for the display of medical events.

Nobel laureate Herbert Simon said, "What information consumes is rather obvious, which consumes the attention of the recipient of the information. Therefore, a large amount of information can result in a lack of attention and consume it. It creates the need to efficiently allocate this attention in the face of a plethora of sources of sexuality, "he once said. In the clinical setting, managing and presenting information about a patient is an important aspect of patient care and medical decision making, such as how to treat a patient or when to leave a hospital. .. Management of information is a particularly serious problem in the context of busy hospitals or clinics where healthcare providers such as nurses or doctors are treating many patients at the same time. For example, the information contained in a patient's electronic health record consumes the attention of the recipient (eg, a nurse or doctor). For example, the large medical history of a particular patient over the years, or more commonly the large amount of information contained in the medical history of a large number of patients, results in a lack of attention.

Relevant underlying healthcare that efficiently allocates healthcare provider attention in the face of information overload from diverse sources, as well as predicting future clinical events, and contributing to these predictions in a timely manner. Systems and methods are needed to help healthcare providers do what they do to highlight the event. This disclosure presents an urgent issue for hospital physicians: which patients now have the greatest need for their attention, and at the individual level, what information on the patient's charts they treat. Deal with what should be done.

Related technologies The rapid adoption of electronic health records (EHRs) has digitally enriched daily clinical data. Adler-Milstein J, DesRoches CM, Kralovec P, et al. Electronic Health Record Adoption In US Hospitals: Progress Continues, But Challenges Persist. Health Aff. 2015; 34 (12): 2174-2180. Henry J, et al., Adoption of Electronic Health Record Systems among US Non-Federal Acute Care Hospitals: 2008-2015, Office of the National Coordinator for Health Information Technology, ONC data brief no. 35, May 2016. This phenomenon is intended for interventions in patients who are predicted to be at high risk of readmission (Parikh RB, Kakad M, Bates DW. Integrating Predictive Analytics Into High-Value Care: The Dawn of Precision Delivery. JAMA. 2016; 315 (7): 651-652), Triaging Patients at Risk for Adverse Events or Compensation Failure (Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data) in mobility: using analytics to identify and manage high-risk and high-cost patients. Health Aff. 2014; 33 (7): 1123-1131; Obermeyer Z, Emanuel EJ. Predicting the Future --Big Data, Machine Learning, and Clinical Medicine. N Engl J Med. 2016; 375 (13)
): 1216-1219), and even recommending specific cancer treatments (Kantarjian H, Yu PP. Artificial Intelligence, Big Data, and Cancer. JAMA Oncol. 2015; 1 (5): 573- It has spurred efforts to utilize electronic health records with algorithms that do (see 574).

Traditionally, these predictive models often collect variables that are consistently measured for a pre-specified cohort in a clinical registry or clinical trial to ensure a high quality data collection. Created separately for the task. In contrast, the data generated in routine care can produce incomplete, inaccurate, and inconsistent datasets. Hersh WR, Weiner MG, Embi PJ, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013; 51 (8 Suppl 3): S30-S37; Newton KM, Peissig PL, Kho AN, et al. Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network. J Am Med Inform Assoc. 2013; 20 (e1): e147-e154; Opmeer BC. Electronic Health Records as Sources of Research Data. JAMA. 2016; 315 (2): 201-202. Therefore, in order to create a predictive model, researchers have to define variables, normalize the data, and complicate the implementation, and thus recreate the steps in real time based on the raw data, missing. We spend a great deal of effort to respond well to the measurements we make (see, for example, the Newton and Opmeer reference). Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc. 2017; 24 (1): 198-208. Considering the above, the central number of variables in the prediction model is 27 (see Goldstein et al., Above), so unstructured data, especially notes, as well as vital signs and test results. Ignore most data, such as repetitive measurements such as.

Adler-Milstein J, DesRoches CM, Kralovec P, et al. Electronic Health Record Adoption In US Hospitals: Progress Continues, But Challenges Persist. Health Aff. 2015; 34 (12): 2174-2180 Henry J, et al., Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008-2015, Office of the National Coordinator for Health Information Technology, ONC data brief no. 35, May 2016 Parikh RB, Kakad M, Bates DW. Integrating Predictive Analytics Into High-Value Care: The Dawn of Precision Delivery. JAMA. 2016; 315 (7): 651-652 Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in healthcare: using analytics to identify and manage high-risk and high-cost patients. Health Aff. 2014; 33 (7): 1123-1131 Obermeyer Z, Emanuel EJ. Predicting the Future --Big Data, Machine Learning, and Clinical Medicine. N Engl J Med. 2016; 375 (13): 1216-1219 Kantarjian H, Yu PP. Artificial Intelligence, Big Data, and Cancer. JAMA Oncol. 2015; 1 (5): 573-574 Hersh WR, Weiner MG, Embi PJ, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013; 51 (8 Suppl 3): S30-S37 Newton KM, Peissig PL, Kho AN, et al. Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network. J Am Med Inform Assoc. 2013; 20 (e1): e147-e154 Opmeer BC. Electronic Health Records as Sources of Research Data. JAMA. 2016; 315 (2): 201-202 Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc. 2017; 24 (1): 198-208 Mandel JC, et al., SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016; 23 (5): 899-908 A. Vaswani et al., Attention is all you need, arXiv: 1706.03762 [cs.CL] (June 2017) D. Bahdanau et al., Neural Machine Translation by Jointly Learning to Align and Translate, Jan. 2014 (arXiv: 1409.0473 [cs.CL]] Choi et al., GRAM: Graph-based attention model for Healthcare Representation Learning, arXiv: 1611.07012v3 [cs.LG] April 2017 Choi et al., RETAIN: an Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism, arXiv: 1608.05745v3 [cs.GL] February 2017 Zimmerman et al. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients, Crit. Care Med., 2006 Hochreiter S, Schmidhuber J., Long Short-Term Memory. Neural Comput. 9 pp. 1735-1780 (1997) A. Vaswani et al., Attention is all you need, arXiv: 1706.03762 [cs.CL] (June 2017)

As a whole and overview, one aspect of the disclosure is directed to a system for predicting and summarizing medical events from electronic health records. This system includes the following three components.

First, the system stores aggregated electronic health records from many (eg millions) patients of diverse age, health status, and demographics, eg computers such as one or more mass data storage devices. Includes memory, records include, among other things, medications, test values, diagnostics, vital signs, medical notes (ie, free text entered by the provider). Aggregate health records are patients obtained unspecified from one or more sources and may be organized into different data structure types due to differences in older systems. Aggregated electronic health records are converted to a single standard data structure format, preferably arranged in a regular format, for example in order of occurrence.

Second, the system is a computer that runs one or more machine learning models trained with aggregated health records transformed into a standard data structure format (this term includes auxiliary memory, single). Is intended to refer to a computer, or system of a computer, or a processing unit that shares a processing task). A deep learning model predicts one or more future clinical events based on a given patient's input electronic health record and is relevant past medical events for the predicted one or more future clinical events. Trained to summarize or highlight (eg, diagnostics, medications, notes, or note excerpts). Input electronic health records are in a standard data structure format and are arranged in order of occurrence, similar to the aggregated health records used for model training.

Third, the system is associated with one or more predicted future clinical events for a patient, eg, a computer terminal or workstation, a tablet, a smartphone, or one or more predictive models. Includes electronic devices for use by healthcare providers treating patients, such as other types of computing devices that have screen displays consisting of client-facing interfaces that display certain past medical events.

In a detailed description, we from a large number of patients (hundreds of thousands, or even millions) obtained in a non-specific format from multiple different institutions, such as hospitals or healthcare systems. Explain that the health records may be aggregated in the form of health records. Data from different institutions may be in different data formats due to lack of standardization in the industry. The recording is converted to a standard data structure format. In one embodiment, these records are arranged in chronological order for each patient. Patients are unspecified in the aggregate health record. In one particular embodiment, the standard data structure format is a known format, the Fast Healthcare Interoperability Resources (FHIR) format (Mandel JC, et al., SMART on FHIR: a standards-based, Interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016; 23 (5): 899-908), in which EHR is formatted with a bunch of FHIR "resources" in chronological order. ing.

In one embodiment, the aggregated health record excludes the key outcomes and the variables necessary to define the exclusion criteria, namely the criteria for excluding the inclusion of a given EHR for model training. Contains variable names that are not harmonized with standard terms. In one embodiment, the aggregated health record comprises an inpatient diagnosis, which diagnosis is mapped to a single level Clinical Classification Software (CCS) code.

In one embodiment, one or more of the deep learning models are referred to as "attention mechanisms" (known in the field of deep learning, described in detail below, and referred to as "attribution mechanisms". How much attention or "weight" to reach one or more future clinical event predictions, and related past medical events when activated, including (some techniques) Or indicate whether multiple models have been shown to a particular "token" corresponding to a sub-element in an electronic health record (individual words in a note, test measurements, drugs, etc.). The provider-opposed interface provides a degree of highlighting or emphasis on elements in the health record (ie, past medical events) associated with a particular prediction, especially those highly scored by the mechanism of interest. It is preferable to include the display of the result of the mechanism of interest, such as. In addition to predictions and related health care events, the display of the results of attention mechanisms on electronic devices provides predictions and confidence in their foundations to health care providers and to inform and guide health care providers' patient care. Bring the healthcare provider's attention to the relevant elements or characteristics of the health record.

Aspects of the present disclosure are directed to deep learning models used to make predictions. In one embodiment, we consider using an ensemble of deep learning neural network models, each of which is trained individually in an aggregated EHR. In one embodiment, we also refer to (1) a Long-Short-Term Memory (LSTM) model and (2) a feedforward model with Time-Aware Attention herein. The time-aware feed-forward model (FFM), which is also called, and (3) an embedded time-aware time-series model, which is also called an embedded time-aware boosting model in the present specification. Use (embedded boosted time-series model). Alternatives to these models, such as the autoregressive convolutional neural network model with attention, may be suitable for use in this system (A. Vaswani et al., Attention is all). You need, arXiv: 1706.03762 [cs.CL] (see June 2017)). Prediction of one or more future clinical events, and summarized and relevant past medical events for one or more predicted future clinical events should be taken from the ensemble average of three deep learning models. Is possible. In some cases, predictions from members of the ensemble may be excluded for a variety of reasons.

We disclose various possible predictions of future clinical events, and in one embodiment, the deep learning model includes unplanned transport to an intensive care room, longer hospital stays than 7 days, unplanned hospitalization, patient discharge. Predict at least one of the entire set of ER visits or readmissions, inpatient death, primary diagnosis, or primary and secondary billing diagnoses at patient discharge within 30 days. We also disclose the ability to predict abnormal laboratory values, including potential ones such as acute renal injury, hypokalemia, hypoglycemia, or hyponatremia. We describe below additional predictive tasks in which the model can be used.

A further aspect of the present disclosure is directed to an electronic device and its provider facing interface. In one embodiment, the interface comprises (1) alerting to one or more predicted future clinical events, (2) major medical issues or conditions relating to alerting (ie, past medical events), and (3) Includes display of notes or note excerpts related to alerts, such as words or phrases. In one configuration, a deep learning model draws as much attention as one or more to predict future clinical events, tokens in which one or more models correspond to elements in an electronic health record. Includes a mechanism of interest to indicate what is shown in. Displaying a note or note excerpt, for example, by changing font size, color, shading, bold, italics, underlining, strikethrough, blinking, highlighting, font selection, etc., for example, a specific word or phrase in a note. , Or by using the degree of highlighting or emphasis on other texts, displayed in a manner that indicates the outcome of the application of the attention mechanism, thereby the most important past medical event in the EHR related to the predicted future clinical event. Attract the attention of the provider. In yet one further configuration, the display displays inferred information from the patient's electronic health records (eg, provisional diagnosis inferred from past medical events), and certain clinical events such as death or transport to the ICU in the future. It can further include a timeline or plot of risks or probabilities to do.

In one possible configuration, the display allows the user of the electronic device to select one of the major problems or conditions, the selection being, for example, the drug prescribed to the patient, and the selected major problem. Or trigger further display of information related to the selected major problem or condition, such as displaying a note or note excerpt about the condition.

In another aspect of the disclosure, a method for predicting and summarizing medical events from electronic health records is described. The method is
a) Aggregating electronic health records from a large number of patients of diverse ages, health conditions, and demographics, where electronic health records are drug, test values, diagnostics, vital signs, and some of the medical notes or Including everything, aggregating and
b) Converting aggregated electronic health records into a single standard data structure format and into regular patient-specific sequences,
c) Training one or more deep learning models with a regular array of aggregated health records converted to a single standard data structure format.
d) Predict one or more future clinical events from patient input electronic health records, having a standard data structure format and arranged in chronological order, for one or more predicted future clinical events. Using one or more trained deep learning models to summarize relevant past medical events, and
e) Data for the provider-opposed interface of the electronic device for use by the healthcare provider to display one or more predicted future clinical events for the patient and related past healthcare events. To generate and
Including the step.

In yet another embodiment
a) Aggregate electronic health from a large number of patients of diverse ages, health conditions, and demographics obtained in various formats, including some or all of medications, test values, diagnoses, vital signs, and medical notes. Computer memory that stores records, in which aggregated electronic health records are converted to a single standard data structure format and arranged in a regular array, such as order of occurrence.
b) One or more deep learning models trained with regular sequence aggregate health records converted to a single standard data structure format to predict future clinical events based on patient input electronic health records A system provided in combination with a computer (as defined above) that runs is described. In one embodiment, one or more deep learning models reach predictions of one or more future clinical events, respectively, and relevant past medical events for one or more future clinical events predicted. To summarize, how much attention is paid to a particular "token" where one or more models correspond to the microelements in an electronic health record (individual words in a note, test measurements, drugs, etc.) Includes "attention mechanism" to indicate whether to indicate. In one embodiment, we consider using an ensemble of deep learning neural network models, each of which is trained individually in an aggregated EHR. In one embodiment, we use (1) long short-term memory (LSTM) model, (2) time-aware feedforward model (FFM), and (3) embedded boosted time series model.

In yet another aspect of the disclosure, a method for predicting a medical event from an electronic health record is described. The method is
a) Aggregating electronic health records from a large number of patients of diverse age, health status, and demographics, where the electronic health records are drugs, laboratory values, diagnoses, vital signs, and some of the medical notes or Aggregation and aggregation, including all, acquired in various formats,
b) Converting aggregated electronic health records into a single standard data structure format, arranged by patient in a regular sequence, such as order of occurrence.
c) Training one or more deep learning models with a regular array of aggregated health records converted to a single standard data structure format, where one or more trained deep learning models are standard. A data structure format that predicts and trains future clinical events based on patient input electronic health records arranged in order of occurrence.
Including the step.

In yet another embodiment, we have a standard data structure format and predict and predict one or more future clinical events based on patient input electronic health records sorted in order of occurrence. One or more deep learning trained with aggregated electronic health records transformed in order of occurrence into a single standard data structure format to summarize relevant past medical events for multiple future clinical events Described an improved computer (as previously defined) that runs the model.

In a preferred embodiment, the deep learning model predicts one or more future clinical events, respectively, and which to summarize the relevant past medical events for the predicted one or more future clinical events. Only one or more models show a particular "token" corresponding to a microelement in an electronic health record element in an electronic health record (individual words in a note, test measurements, drugs, etc.) Includes a mechanism of interest to indicate.

In yet another embodiment, a) one or more predicting and predicting one or more future clinical events based on the patient's input electronic health records, which have a standard data structure format and are arranged in order of occurrence. Run one or more deep learning models trained with aggregated health records transformed in order of occurrence into a single standard data structure format to summarize relevant past medical events for future clinical events in For use by healthcare providers treating patients, configured to display computers and b) one or more predicted future clinical events for the patient, as well as related past medical events. A system provided in combination with a client-opposed interface of an electronic device is disclosed.

In yet another embodiment, an electronic device having a healthcare provider-opposed interface (eg, a workstation, etc.) that provides a substantially real-time display of a prediction of one or more future clinical events for at least one patient. A tablet computer or smartphone) will be disclosed. The display is further configured to display elements (past medical events) from the electronic health record that correspond to the application of the feature mechanism on the predictive model that operates on the electronic health record for the prediction. In one embodiment, an element of an electronic health record is a note or note citation with a gradation of highlights or highlights for a particular word, phrase, or other text in the note. Elements of the electronic health record can also be test values, previous medications, vital signs, and so on. Highlight or highlight gradients include font size, font color, shading, bold, italics, underlining, strikethrough, blinking, colored highlights, and font selection, or, in some cases, red and bold fonts. It can take the form of at least one of several combinations. Predicted one or more future clinical events include unplanned transport to the intensive care unit, hospital stay longer than 7 days, unplanned readmission within 30 days of patient discharge, death of inpatient, primary diagnosis , The entire set of primary and secondary claims, or abnormal laboratory values such as acute renal injury, hypokalemia, hypoglycemia, and hyponatremia can be included.

In one embodiment, the interface is time to plot the risk or probability of at least one patient over time, for example plotting the risk of transport to the ICU, or the risk of hospitalization longer than 7 days, or the risk of death. Further configured to display lines. Electronic devices can be used in hospital or clinic environments where the system is functioning to simultaneously predict future clinical events for multiple patients, in which case the interface will be at least one. A timeline that plots the risk or probability of a given patient over time is further configured to be displayed simultaneously for multiple patients.

In yet another aspect, methods are disclosed that help healthcare providers provide care to patients. The method is
Training from aggregated electronic health records to (1) generate predictions of future clinical events for patients and (2) identify relevant past medical events from input electronic health records for patients Using the predicted model
b) To generate data on both predictions and identified related past medical events.
c) Sending the generated data to an electronic device used by a healthcare provider for display on the electronic device,
Including the step
Predictive models are focused mechanisms to predict future clinical events and to show how much attention the predictive model has shown to the elements in the input electronic health record in order to identify relevant past medical events. The data generated using is containing the results of the mechanism of interest.

In one embodiment, the relevant past medical event includes a note (eg, text entered by a doctor or nurse) or an excerpt of the note. In one embodiment, the predictions are unplanned transport to the intensive care unit, length of stay longer than 7 days, unplanned readmission within 30 days of discharge of the patient, death of the inpatient, primary diagnosis, primary and Selected from the entire set of secondary billing diagnoses and a group of abnormal test values.

The generated data may further include a timeline of probability or risk that the event will occur over time.

In one embodiment, steps a), b), c), and d) are performed simultaneously in real time for multiple patients from multiple input electronic health records. A healthcare provider who cares for at least two of a large number of patients receives the data generated for at least two patients in real time so that the healthcare provider cares for at least two patients at the same time. Helps to prioritize patient care for at least two patients based on individual predictions.

In another aspect, the present disclosure
a) Aggregate electronic health from a large number of patients of diverse ages, health conditions, and demographics obtained in various formats, including some or all of medications, test values, diagnoses, vital signs, and medical notes. Computer memory that stores records, in which aggregated electronic health records are converted to a single standard data structure format and arranged in a regular array for each patient.
b) One or more deep learning models trained with regular sequence aggregate health records converted to a single standard data structure format to predict future clinical events based on patient input electronic health records Targets systems that include computers running.

In one embodiment, the aggregated health record is a form of health record arranged in various data formats. The standard data structure format may take the form of Fast Healthcare Interoperability Resources (FHIR). Aggregate health records include variable names that are out of harmony with standard terminology, excluding the variables needed to define key outcomes and exclusion criteria. Aggregate health records may include inpatient diagnoses, which are mapped to a single level clinical classification software (CCS) code. In one embodiment, electronic health records are sorted by patient in order of occurrence.

In another aspect, a method for generating training data for machine learning from a set of raw electronic health records of a large number of patients from different sources in different data formats is disclosed. The method is
a) Obtaining a set of raw electronic health records and
b) Converting a set of raw electronic health records into a single standard data structure format,
c) Arrange electronic health records converted to a single standard data structure format in chronological order by patient.
d) Includes the step of storing time-series electronic health records in standard data structure formats on a data storage device.

A set of electronic health records can include both structured data and unstructured data, including free text notes. In one embodiment, the conversion step b) is performed without harmonizing the terms in the raw electronic health record with standard terms.

To receive aggregated electronic health records, computers running trained deep learning models, and predictions from deep learning models, and relevant past medical events related to predictions, and to present such information on the display. It is a schematic diagram of the whole system including the electronic device used by the medical provider having the interface of. FIG. 5 illustrates an illustration of the procedure used in the system of FIG. 1 for converting raw electronic health records into FHIR resources in chronological order. It is a flow chart which shows the design and operation of the time recognition feed forward model of FIG. It is a flow chart which shows the design and operation of the time recognition feed forward model of FIG. It is a flow chart which shows the design and operation of the time recognition feed forward model of FIG. It is a flow chart which shows the design and operation of the embedded boost type time series model of FIG. It is a flow chart which shows the design and operation of the embedded boost type time series model of FIG. It is a flow chart which shows the design and operation of the embedded boost type time series model of FIG. It is a flow chart which shows the design and operation of the LSTM model of FIG. It is a flow chart which shows the design and operation of the LSTM model of FIG. It is a flow chart which shows the design and operation of the LSTM model of FIG. A diagram illustrating one form of display of data on a provider-opposed interface showing the results of a mechanism of interest in a deep learning model in a patient's timeline or series of events, including medications, encounters, treatments, notes, instructions, etc. Is. Corresponds to the attention (importance or weight) that the word was for clinical prediction generated by a deep learning model, and the attention that a particular drug in the medical history was for prediction. Representation of data in EHR showing the results of attention mechanisms in a deep learning model in the form of note excerpts with the degree of emphasis (size, thickness, color, etc.) given to individual words or phrases in the note. It is a figure which shows the illustration of another form. It is a diagram showing another example of the result of the attention mechanism in a deep learning model, showing the various words or phrases found in EHR notes that result in different degrees of emphasis (bold) associated with prediction. The darker the highlight, the higher the number of points of interest. The model of the present disclosure, including anomalous test results, performs with accuracy statistics obtained from retroactively applying the model to the test set portion of the original set of patient records used for model training. FIG. 5 illustrates illustrations of various types of predictions that can be used for. FIG. 6 illustrates an illustration of a healthcare provider facing interface of an electronic device for use by a healthcare provider treating a patient, for example, a computer terminal, tablet, smartphone, or other type of computing device with a screen display. .. The interface in this configuration is designed for hospital use and presents a risk plot for two patients simultaneously. The interface displays two predicted future clinical events for a particular patient, in this case unplanned transport to the intensive care unit (ICU), and delayed discharge from the hospital. The display in Figure 8A is designed to alert healthcare providers to early attention to at-risk patients. FIG. 5 illustrates an illustration of the interface of FIG. 8A after the donor has selected a patient with alert. The display alerts the provider to a major medical issue (predictive medical event) so that the provider now understands the patient and makes patient care decisions, including notes with the results of the model's attention mechanism. To help, we delve into the conditions or other data that providers may need to look at, and ensure that providers do not miss important information. FIG. 5 shows the display of tools on the interface related to the use of the interface when predicting outpatients. It is a figure which shows the large amount of information available to the medical provider from EHR, and the example of the assumption of the reason why the feature of this disclosure is necessary. Figure 10 shows an excerpt from four years of a patient's medical history with over 400 listed diagnoses. It is a figure which shows the large amount of information available to the medical provider from EHR, and the example of the assumption of the reason why the feature of this disclosure is necessary. FIG. 11 shows an excerpt of a diagnosis for a patient, excluding a lack of important accompanying information such as whether the patient was treated as an outpatient, inpatient, or in the ICU or other situations. It is a figure which shows the large amount of information available to the medical provider from EHR, and the example of the assumption of the reason why the feature of this disclosure is necessary. FIG. 12 shows more than 150 different encounters for a particular patient over a given time span, but there is no detail about what happened in each encounter. It is a figure which shows the large amount of information available to the medical provider from EHR, and the example of the assumption of the reason why the feature of this disclosure is necessary. Figure 13 shows only one few notes taken by the provider at the time of one hypothetical four-day hospitalization, and all views of the notes require 60 different screens on a standard mobile device. And. FIG. 3 illustrates an example of the device interface in Figure 3, which tracks data and risks for four patients in real time. It is a diagram showing the interface of FIG. 8B showing the selection of only major events in 150 past encounters within the EHR presented on the patient's timeline in relation to the prediction (ICU transport, delayed discharge). FIG. 8 shows the interface of FIG. 8B showing the selection of only the major problems from the list of 400 past diagnoses in the EHR related to prediction (ICU transport, delayed discharge). The main issues (ie, related past medical events) are presented as a summary on the left side of the display in the issue list area. FIG. 5 shows the interface of FIG. 8B showing major and important excerpts or word-only selections from 10,000 words in a note in an EHR related to prediction (ICU transport, delayed discharge). Key excerpts (words and phrases) are presented in the lower right area of the interface, with the degree of highlighting to a particular word or phrase, as a result of the use of attention mechanisms in the deep learning model when generating predictions. To. It is a figure which shows the interface of FIG. 8B which shows the ability of the interface for summarizing each listed medical problem. In this example, clicking on the main issue of "alcohol withdrawal symptoms" in the display of FIG. 8B shows the drug, notes, and timeline of the event for the main problem. It is a diagram outlining the main problem of "cardiomyopathy" in the form of a timeline, drugs, and associated notes.

In the figures and accompanying descriptions, the names and medical data of all patients and providers are fictitious and do not reveal any confidential patient information.

A. Overview This disclosure describes a new way of constructing EHR data for use during training of predictive models. The model uses all data recorded about the patient, including clinical notes, variables in raw, unharmonized formats or terms, and maintains the temporal order of the data collection. We further applied aspects of deep learning to generate and train models to make clinical predictions from EHR data. We chose deep learning because it handles millions of variables, can automatically harmonize data from different sources, and contains a set of variable-length data. Deep learning techniques have achieved state-of-the-art performance in other complex domains such as medical imaging recognition (for the detection of diabetic retinopathy and cancerous skin lesions), and language translation. Many of the applications and implementations of these deep learning models for this problem domain are believed to be new.

This document further demonstrates the technical feasibility and clinical utility of our approach. We support predicting length of stay to improve throughput and reduce costs, predicting unplanned readmissions to target interventions in high-risk patients, and deploying early interventions. For multiple clinical tasks, including predicting the death of inpatients, predicting diagnoses from routine clinical data to enable clinical decision support, and phenotyping. The prediction model of is explained. In addition, we have described the unplanned transport of patients in the hospital to the intensive care unit and the application of the model to predict the primary diagnosis. In addition, we face providers of electronic devices (eg, computer terminals, tablets, or smartphones) that present these predictions and the underlying relevant medical events that help providers treat patients in a timely manner. The user interface will be described.

Figure 1 shows a system 10 for predicting and summarizing medical events from electronic health records. The system contains three components:

First, a large number of patients of diverse ages, health conditions, and demographics, including medications, test values, diagnoses, vital signs, and medical notes (eg, free text notes written by physicians and nurses). A computer memory 24, such as a mass data storage device, that stores an aggregated electronic health record 22 from is described. Aggregated electronic health records are converted into a single standard data structure format, for example sorted by patient in order of occurrence. A large number of patient raw electronic health records 12 from various institutions 14 (eg, university medical centers, hospital systems, etc.) have a variety of different electronic health records due to the wide variety of outdated electronic health record systems currently in use. It may be formatted in the format. Raw health records are unspecified by the patient, transmitted over computer network 16, stored in a relational database (RDB) 20, converted to standard format by computer system 18 acting as a converter, and into memory 24. It is stored. These records are converted to a standard data structure format, arranged in a regular sequence, and in preferred embodiments arranged in chronological order. In one particular embodiment, the standard data structure format is the known format, the Fast Healthcare Interoperability Resources (FHIR) format, in which the EHR is the time series FHIR shown in Figure 1. Formatted into a bunch of "resources". This will be explained later with FIG.

Second, the system is one or more deep learning models trained in chronologically-ordered aggregated health records 22 transformed into a single standard data structure format (28, 30, 32 described below). 26 (the term is intended to refer to a single computer, or system of computers, or a processing unit that shares a processing task with auxiliary memory). A deep learning model predicts one or more future clinical events based on the input electronic health record 38 of a given patient 36, and is relevant past for one or more predicted future clinical events. Trained to summarize medical events (eg, problems, conditions, test results, medications, etc.). The input health record 38 is a standard data structure format similar to the aggregate health record used for model training and is arranged in order of occurrence. The input health record 38 can be converted to the FHIR format by the converter 18 if desired, as indicated by the dashed line 39.

Figure 1 shows the reception of an input electronic health record 38 from a single patient, but in reality this is many others throughout the medical system or hospital, depending on the extent of the spread of the system. It will be correctly recognized that it may occur essentially at the same time for patients with. The system in Figure 1 operates a model based on input health records to generate predictions and data on corresponding past medical events to simultaneously predict the EHR of all these patients in real time. It is preferable to use sufficient computing resources for the computer 26 (or computer system) to send data to the electronic device 40 for display on the device's client-facing interface.

Third, the system includes an electronic device 40 for use by a healthcare provider treating a patient, for example a computer terminal or workstation, tablet, smartphone, or other type of computing device with a screen display. Consists of a client (healthcare provider) opposite interface (Figures 8A-8B, Figure 9, Figure 14, etc.) that displays one or more predicted future clinical events for a patient and related past medical events. Will be done. A display of upcoming clinical events and corresponding past medical events provides medical care to highly relevant information in the patient's electronic health records related to predictions such as predictions of ICU transport, late discharge, death, etc. Assist 42 (eg, nurses or doctors) in focusing their attention. An example of the use of device 40 and an interface for providing this assistance will be described later with and in FIGS. 8A-19.

In another aspect of the disclosure, a method for predicting and summarizing medical events from electronic health records is described. The method is
a) Aggregating electronic health records 12 from a large number of patients of diverse age, health status, and demographics, where the electronic health records include drugs, test values, diagnoses, vital signs, and medical notes. To aggregate and
b) Converting aggregated electronic health records into a single standard data structure format, per-patient arranged in a regular sequence (see Bundle of Time Series FHIR Resources 22 Generated by Converter 18),
c) Training one or more deep learning models 28, 30, and 32 with a regular array of aggregated health records 22 converted to a single standard data structure format.
d) Predict one or more future clinical events from the input electronic health records 38 of 36 patients, arranged in order of occurrence, with a standard data structure format, and for one or more predicted future clinical events. Using one or more trained deep learning models 28, 30, and 32 to summarize relevant past medical events, and
e) To generate data for the healthcare provider facing interface of the electronic device 40 for use by the healthcare provider 42 treating the patient (FIGS. 8A-8B, 14, 19, 19 etc.). The data includes the steps of displaying and generating one or more predicted future clinical events for the patient and related past medical events.

Aspects of the components of the system and method will be described in more detail below.

B. Integration of electronic health records into a single format for model generation As mentioned above, raw electronic health records 12 are available from a large number of patients (hundreds of thousands, or even millions). It may take the form of a health record. Aggregate health records can be obtained from one or more different institutions. EHR data may be in different data formats due to lack of standardization in the industry. These records are converted to standard format and arranged in a regular array. This is shown in Figure 2, in which the patient's raw electronic health record 12 includes the encounter table 50 (all patient visits to the doctor's office, laboratory, hospital, etc.), all laboratory tests and Includes test table 52 containing results, as well as other tables (not shown) containing data such as vital sign data, medical notes (free text), demographic data, diagnostics, flow sheets, etc. Patient data is anonymized and does not include personal identification data. Permission is obtained from the institution to receive the data and use it to train the model. These tables 50 and 52 representing the raw data are stored in RDB20 in FIG. The converter 18 then converts the raw data into a standard format, in this example a collection of FHIR resources 22A, 22B, 22C, 22D, etc. as shown in Figure 2, and for each patient such FHIR There is a "bundle" of resources or set 22. As shown in 54 in Figure 2, these resources are then placed in chronological order to create a timeline, or in the order in which all data in the EHR occurs.

Details of the dataset we used to generate our predictive model are given in Appendix A of our earlier US Provisional Application No. 62 / 538,112, filed July 28, 2017. In short, in our model development, we have the University of California, San Francisco (UCSF) in San Francisco, California, the University of Chicago Medicine (UCM) in Chicago, Illinois, and Boston, Massachusetts (MIMIC-III). Electronic health record data was obtained from the Beth Israel Deaconess Medical Center. These electronic health records were in a non-specific or limited dataset format shared in compliance with all state and federal privacy laws (including HIPAA). We hold a record of 2 billion encounters across 70 million patients between 2013 and 2015, internally known as "Uranus", a non-specification of Medicare and commercial rights. The national database was also used. UCSF data includes all patients encountered between 2011 and 2016 from the academic healthcare system, with several hospitals of various sizes. UCM non-specific data include encounters with all adult patients from 2009 to 2016 from several hospitals. MIMIC's non-specific dataset contains data associated with patient encounters at the Lifesaving Medical Unit in Boston, Massachusetts, from 2001 to 2012. Of course, electronic health records can be aggregated and obtained from other institutions, so the details of the development set are not considered to be particularly important and improve the accuracy of the model. A set large enough to be used should be used.

Each EHR dataset contains patient demographics, all inpatient and outpatient encounters, the order entered in the EHR, diagnosis, treatment, medications, test values, vital signs, and all other structured data elements ( For example, it contained flow sheet data representing a nursing flow sheet). In addition, the datasets from UCM and MIMIC-III included non-specific medical notes, and the datasets from UCM also included intraoperative vital signs and outpatient surgical flowsheet data. ..

The Uranus rights dataset included patient demographics, all inpatient and outpatient encounters, diagnostic codes, treatment codes, and outpatient prescriptions.

The data were unspecified except for dates within the UCM dataset and adhered to all requirements for disclosure and use of the limited dataset under HIPAA. The exemption from the ethical review and the institutional review committee was obtained from each institution. Patient data was not linked to any Google user data. In addition, for aggregated electronic health records used to create models, our system is a sandbox infrastructure that keeps each EHR dataset separate from each other in accordance with regulations, data licenses, and / or data usage agreements. including. The data in each sandbox is encrypted and all data access is controlled, logged and inspected at the individual level.

We have developed a single data structure for aggregated EHR based on the Fast Healthcare Interoperability Resources (FHIR) to store data from all health systems and each system used for prediction. FHIR is an open source framework that allows standard representation of clinical data as a set of resources (entry of modules containing specific data types such as single encounters or laboratory tests). Various types of data collected by the health system were transformed into these corresponding FHIR resources.

When converting data to FHIR format (“resources”, see Figure 2), we did not harmonize variable names with standard terms, instead using the raw terms provided by the health system, Refrained from the conventional time-consuming data harmonization. The only exception was made to the variables needed to define the key outcomes and exclusion criteria: discharge measures, hospital services, diagnostic codes, and treatment codes. Inpatient diagnosis is provided as an ICD-9 / 10 code, we map these to a single-level Agency for Healthcare Research and Quality (CCS), and inpatient procedures are ICD-9 / 10, It was provided as a treatment code for the Current Procedural Terminology (CPT) and was also mapped to the CCS code.

A set of resources for a given patient was then assembled in order of occurrence. This series of events provided a faithful representation of each patient's timeline within the EHR. The billing code is assigned a time stamp immediately after the end of the encounter.

Certain elements, such as vital signs, can be entered into the EHR after they have been collected. We respond to the input of data into the EHR when the data becomes available within the EHR in real time, rather than when the data was recorded as collected to model the data. Created and used vital sign timestamps.

C. Deep Learning Model for Predicting Health Events from Medical Records As shown in Figure 1, our system is trained with time-series aggregated health records 22 converted to a single standard data structure format. Includes computers 26 (ie, a set of computers or processors and auxiliary memory) that run deep learning models 28, 30, and 32. The model predicts one or more future clinical events based on patient 36's input electronic health record 38 and summarizes relevant past medical events for the predicted one or more future clinical events. The input EHRs are formatted in the same standard data structure format as they were, or after conversion by converter 18 if necessary, and arranged in order of occurrence.

In theory, only a single trained model could be used, but to avoid overfitting and to provide high accuracy in predicting future clinical events, we separate aggregated electronic health records. We found it advantageous to use three different models, each trained in the dataset we created. At least one of the deep learning models is to predict one or more future clinical events and related related past medical events for one or more predicted future clinical events. Attention showing how much attention (ie, how significant) the model has shown to "tokens" (ie, minute elements in electronic health records such as individual words in notes, drugs, test results, etc.) Including the mechanism. The use of featured mechanisms in deep learning neural networks is explained in a conference presentation by D. Bahdanau et al., Neural Machine Translation by Jointly Learning to Align and Translate, Jan. 2014 (arXiv: 1409.0473 [cs.CL]). Further explanation of the mechanism of interest in the medical background is Choi et al., GRAM: Graph-based attention model for Healthcare Representation Learning, arXiv: 1611.07012v3 [cs.LG] April 2017, and Choi et al., RETAIN: an Interpretable. Predictive Model for Healthcare using Reverse Time Attention Mechanism, arXiv: 1608.05745v3 [cs.GL] Including February 2017. References to Choi et al. And Bahdanau are incorporated herein by reference.

In our preferred embodiment, we use a weighted recurrent neural network model, a long-short-term memory (LSTM) model 28, a time-aware feedforward (also referred to herein as a feedforward model with time-aware attention). We use three different models: model (FFM) 30, and embedded boosted time series model 32, also referred to herein as feedforward models with boosted time recognition stamps. Appendix B of our US '112 provisional application and the description of Figures 3A-3C above provide further details on the architecture, design, and implementation of the three models.

There are various predictive tasks that can be performed by these models, some of which are described here in a little more detail and in Appendix A of our US '112 provisional application earlier. These include the entire set of prolonged hospital stays, unplanned readmissions, unplanned transport to the ICU, inpatient death, primary diagnostic code, and billing diagnostic code at discharge. These predictions are made without the selection or engineering of predictive variables for each task.

The conclusions of the five predictive tasks are defined below. For all predictions, we use all information available within the EHR (except the rights database) by the time the predictions are made, such as on admission, 24 hours or after discharge. We chose 24 hours because 24 hours is commonly used in clinical prediction models such as APACHE. For example, Zimmerman et al. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients, Crit. Care Med., 2006.

The time of admission is defined as the start of inpatient status, meaning that data from the emergency department and outpatient surgery is available prior to admission.

For the MIMIC dataset, this time was related to ICU admission. Forecasts made on the day of admission include rights issued on the same calendar date as admission, as rights data have only day-level attributes.

Inpatient Death We predicted inpatient death and defined it as an "expired" discharge measure.

Long hospital stay We predicted a hospital stay longer than 7 days, which was elected as the approximately 75th percentile hospitalization for most services across the dataset. The length of hospital stay was defined as the time between admission and discharge.

30-day unplanned readmissions We have predicted future unplanned readmissions within 30 days following discharge from admission, taking into account all of the above data elements during and before admission. Since there is no accepted definition of "unplanned," we admit that readmission is detailed in Appendix B of our US '112 provisional application earlier, acute complications, chemotherapy, transplantation, or rehabilitation. We used a modified version of the Centers for Medicare and Medicaid Services (CMS) definition that readmissions are excluded if they are for planned treatment, not for hospitalization. Re-hospitalization was included if the admission time was within 30 days of the previous discharge time of a qualified indicator admission without any intervening admission (ie, readmission is counted only once). There is no).

Diagnosis-Primary and whole set For each hospitalization, we predict the primary diagnosis (using the CCS category of which clusters associate diagnosis and treatment with approximately 250 groups such as sepsis or tuberculosis). By doing so, we classified what the patient was most likely to be treated for. We also predicted the entire set of primary and secondary ICD-9 claim diagnoses (ie from a population of 14,025 codes). We used the CCS category for primary diagnostics to mimic the quotas that can be used for decision support that does not require an accurate ICD-9 code.

Inclusion and Exclusion Criteria in the Study Cohort We included all consecutive hospitalizations for patients 18 years and older, with the exception of one dataset for which we did not use age restrictions, for comparison with the literature. We included only hospitalizations of 24 hours or more to ensure that the predictions at different time points had the same cohort.

To resemble the practical accuracy of a real-time prediction system, we are not aware of these exclusion criteria when predicting early in hospitalization, so we are discharged from the hospital against medical advice. The study did not exclude patients typically removed.

To predict the complete set of ICD-9 diagnoses, we ruled out encounters without any ICD-9 diagnosis, which existed approximately 2 to 12 percent per dataset. These are generally encounters since October 2015 when hospitals switched to ICD-10. But we included such hospitalizations for all other predictions.

To compare with the existing literature, we have also created a constrained set of indicator hospitalizations for medical or surgical services (ie, excluding obstetrics).

Model Design and Training We used three types of deep learning architectures for models 28, 30, and 32 (Figure 1) that consider modeling a series of patient events within the EHR. We would like to refer to Long Short-Term Memory (LSTM) (Hochreiter S, Schmidhuber J., Long Short-Term Memory. Neural Comput. 9 pp. 1735-1780 (1997)) to create Model 28. A well-known version of a recurrent neural network called (incorporated by reference in the specification) was used. We have created two new methods, called the Time-Aware Feed Forward Model (FFM) to create Model 30, and the Embedded Boosted Time Series to create Model 32, for which we have created our earlier US '112 tentative. It is explained in Appendix B of the application. For the first two models 28 and 30, we provide a mechanism of interest (see the previously cited Bahdanau et al. Document) to highlight the data elements that most affect the prediction. I ran it. Each model was tailored to address the specific challenges of EHR data: long series of patient events, dynamic changes in variables, and the impact of distant historical patient data.

Each model 28, 30, and 32 was trained separately for each dataset in the cohort. For predictions, in almost some examples, we averaged the predictions from each model to find the final prediction score. In other examples, we exclude results from, for example, one of the models that are not tuned for a particular task or prediction, and average the prediction scores for the remaining models.

Patient EHR was randomly divided into a development set (80%), an activation set (10%), and a test set (10%). To prevent any implicit overfitting, the test set was left unused (and hidden) until final evaluation. Model accuracy was reported for the test set, and the test set bootstrap was used 999 times to calculate the 95% confidence interval. We ignored intrapatient clustering because the goal is to produce personalized predictions, not to assess the contributions of individual predictors.

For each predictive task, we created a baseline model with hand-crafted variables based on existing literature to determine retrospective model performance. Details about the baseline model are described in Appendix B of our earlier US '112 provisional application. LSTM and feedforward models 28 and 30 were trained in Tensorflow (version 1.0) and boost model 32 was run in custom C ++ code. Statistical analysis and baseline models were performed in SciKit learn Python (0.18.1).

All models train embedded vectors to represent each token (eg, a fine element of EHR). The token can be, for example, a word in a note, the name of a drug, or a discretized value of a particular laboratory test. The implants were randomly initialized and model training updated the implants to improve predictive performance.

FIG. 3A is a flow diagram showing the design and implementation of FFM30 in FIG. The steps are essentially as follows:

Step (1) shows, for example, the data in the original EHR with a time stamp (delta time) relative to the moment of prediction.

Step (2) indicates that each data element is embedded and has been transformed into a d-dimensional vector (this transformation is trained by the model).

Step (3) shows that each delta time is embedded and means that it was transformed into a k-dimensional vector using the k-functions that mutually encode the piecewise linear divisions (this transformation depends on the model). learned, it produces a predetermined or learned bank functions a ₁ ... a _k).

Step (4) shows that the trained projection matrix is multiplied by the data embedding so that it yields the data projection matrix of interest, which is multiplied by the time embedding matrix using the column dot product operator to alpha (4). α) Generates a vector.

Step (5) shows that the α vector is positioned in the softmax function, giving rise to the beta (β) vector.

In step (6), the beta vector is multiplied by the data embedding matrix and finally the number of reduced record vectors of dimension D (ie, the internal ReLu (rectifier linear unit)) input to the feedforward network by the sigmoid or softmax function. The layer) is generated, and the predicted output is generated.

The output of the model is the output of the sigmoid, which is the vector of interest learned from step (4).

FIG. 3B is a flow chart showing the design and operation of the embedded boosted time series model 32 of FIG. The steps are essentially as follows, and step (1) shows, for example, the data in the original EHR with a time stamp (delta time) relative to the moment of prediction. In step 2), each data element is adjusted to a binary feature f ₀ ... f _N that indicates the existence of a particular value / token at a particular (relative) point in time. Each predicate of the form in some t> T is v> V. Together they form an N-bit vector V.

In step (3), the vector V is multiplied by the (learned) embedded vector E of dimension D to be aggregated (eg summed) to yield a D-dimensional vector.

In step (4), this D-dimensional vector is input to a network of several ELU (exponential linear unit) layers ending in, for example, a sigmoid. The output of the network is the output of the sigmoid function.

FIG. 3C is a flow chart showing the design and operation of the LSTM model 28 of FIG. The steps are essentially as follows:
(1) For each feature category (eg, drug, note, vital signs), each _datapoint is embedded in a vector of magnitude D _category .
(2) All data can be considered, for example, in a daily bag. For each feature type, a weighted average is calculated for all vectors in the bag, yielding, for example, the average drug vector for this bag.
(3) For each bag, for example, one day, all average feature vectors are combined to generate a vector of magnitude D = D _medication + D _note + D _vital for all feature types.
(4) These average vectors are input to the LSTM model, and each vector represents one step in the sequence.
(5) The output of the LSTM is input to the softmax function (for example, for multiclass classification to identify the primary diagnosis) or the logistic function (for example, for stochastic tasks such as death).

Alternatives to these models may be suitable for use in this system, for example, autoregressive convolutional neural network models with attention (A. Vaswani et al., Attention is all you need, arXiv: 1706.03762 [cs.CL] (see June 2017)).

As mentioned above, the model uses an attention mechanism that allows fine-grained visualization of weights or "attention" for specific tokens used by the model to make specific predictions for the patient. Some examples will be described below with FIGS. 4, 5, and 6.

Figure 4 illustrates one form of data display 64 in EHR showing the results of attention mechanisms in a deep learning model in a patient's timeline or series of events, including medications, encounters, treatments, notes, instructions, etc. is there. In this particular example, each circle indicates the occurrence of a particular event relating to the prediction of drug administration, laboratory tests, treatments, notes, or instructions (in this case, the predictive risk of death of an inpatient). The timeline shows the date the patient was hospitalized and, in this example, an excerpt of these records at a particular point in time over a two-day period (eg, medications, notes, reports, etc.). Figure 5 shows the attention in a deep learning model in the form of a note excerpt with the degree of emphasis (size, thickness, etc.) given to each word in the note corresponding to the attention (importance or weight). An illustration of another form of display of data in EHR that shows the results of the mechanism, the words are for clinical predictions generated by deep learning models, and certain drugs in the medical history are predictions (pneumonia and anemia). (Diagnosis of metastatic melanoma with). The terms "melanoma," "metastatic," "encasement," "hemoptysis," etc. from free text notes within the EHR are related to or related to the predictions generated by the model for these elements of the EHR. Shown in larger fonts and darker colors to draw the attention of the provider. Figure 6 is another example of the results of attention mechanisms in a deep learning model of EHR notes given different degrees of emphasis (thickness, font size) associated with predicting the diagnosis of alcohol-related disorders. Shows the various words found in. Darker highlights correspond to words in higher-focused medical notes such as misuse, withdrawal symptoms, drinkers, and so on. Further examples of featured mechanisms in the model for facilitating the display of medical events within the EHR related to prediction will be discussed in a later section on provider-opposed interfaces.

Examples of how the attention results of FIGS. 4, 5, and 6 are generated are as follows. First, to identify the patient's past medical problems with prediction, we for all historical times of the patient (ie, once per historical encounter, or for each week in history. Once), run a model that identifies the diagnostic code (ICD9 code prediction, and primary diagnostic CCS code prediction, as previously explained). From this, we obtained a list of predicted / identified past medical problems, and in doing so we estimated drugs, tests, vitals, notes, etc.

These issues are ranked and presented to the doctor. The rankings are as follows: (1) How much evidence supports this medical problem (eg, this is only mentioned in the notes, or is also examined / examined in tests / vitals and treated with drugs. (Is it?), (2) This issue has been explicitly claimed and coded in the main EHR, or this is what we presume, but explicitly coded. Is it an unclaimed "implanted" diagnosis, and (3) how rare and serious this medical problem is (eg, aneurysm vs. hypertension), and other potential factors. It depends on several factors such as ,.

Next, for each issue, we need to summarize key facts such as key drugs and key note excerpts and words. We give the number between 0 and 1 how much attention (or intuitively, weight or importance) the model showed to this word, for example, for each drug or word in the note. To demonstrate, the mechanism of interest is used to query the above model (which classified this patient as having, for example, hypertension). The words with the highest scores are shown in these illustrations in Figures 5 and 6.

As another example, as described in the later figure, the provider-opposed interface of the electronic device (40 in Figure 1) is a note from EHR regarding the prediction of ICU transport (instead of historical medical issues). Here is an excerpt. Here we have the same attention mechanism to get a number between 0 and 1 for each input token referenced by the model (for example, for each word in the note, for each defined drug). Is used again correctly to show how much weight the model puts on this individual word / drug / etc while making predictions of ICU transport.

Further details on model performance, study cohorts, dataset characteristics, and results compared to baseline models are provided in Appendix A of our earlier US '112 provisional application and are not particularly relevant. A summary of the performance results of our model in a retrospective study of the test set in the cohort is shown in Figure 7. Figure 7 shows various types of predictions made by the model, including readmissions, deaths, unplanned ER / hospital visits, and so on. The "AUC" performance measurement standard represents the area under the examiner's operating characteristic concentration curve, which is the standard performance measurement standard in machine learning.

An overview of our findings from model development and testing is as follows: Using deep learning based on electronic health record data, we predict in-hospital death, long hospital stays, unplanned 30-day readmissions, identify primary diagnoses, and bill at discharge. Demonstrated high predictive performance when making allocations. We have shown that the results are consistent across healthcare systems and clinical tasks, improved with the availability of first-visit patient data, and can be interpreted with the use of attention mechanisms. We have four major discoveries described below. Further details can be found in Appendix A of our earlier US '112 provisional application.

Results are extensible across completely different datasets First, our method accepts unstructured data such as free text notes across multiple clinical sites and within EHR for model training. All of the data in can be used to generate accurate predictions. Our approach does not require careful selection of variables from the raw data of a particular site, and how to clean, extract, and harmonize them. Predictive models in the literature use the median of 27 variables, while we have over 100,000 data, including variables that are typically difficult to include, such as clinical notes and flow sheets. -Used the median points.

Predictive performance is excellent across completely different tasks Second, our results show that our method of representing and modeling EHR data is extensible across clinical tasks, we We believe our results are superior to similar studies on death (0.94 to 0.98 to 0.91), readmission (0.74 to 0.75 to 0.69), and length of stay (0.86 to 0.92 to 0.77). Our ability to die and readmission in the ICU surpasses the ability of physicians to discriminate.

However, comparing our results with other studies is difficult given the differences in performance based on cohort selection and study design, and many results provide incomplete descriptions of cohorts and conclusions. Have, predict small disease-specific cohorts, or use data that is not routinely available in real time.

To address this limitation, we have implemented the HOSPITAL version, NEWS score, and Liu model as baselines (see Appendix B of our earlier US '112 Provisional Application) for superior performance. To demonstrate. We also evaluated cohorts designed to resemble those in patient-related studies of medical or surgical services and found similar benefits to our approach.

In addition, we used an open dataset called MIMIC, where we outperformed the existing literature with an AUC for deaths of 0.91 vs 0.80 and a micro-F1 for ICD classifications of 0.4 vs 0.28.

Modeling utilizes values from a complete set of data Third, our modeling technique predicts when new data will be available, as opposed to using fixed time points. Update successfully. For all tasks, the model uses hundreds of thousands of patient attributes to obtain substantial performance benefits for all tasks. Interestingly, our model derives distinctive performance from rights data, much like performance for EHR data, but in fact, when it comes to predicting unplanned readmissions, rights performance is EHR data. Exceeds the performance of and is likely due to the big picture of readmissions at other hospitals.

The output of a complex model is interpretable Fourth, we demonstrate a notable mechanism that allows a fine-grained visualization of the data used by the model to make specific predictions for the patient (Figure 4). -See Figure 6 and the following discussion of interfaces in Figures 8--9 and 14-19). Since we explicitly model the sequence and timing of patient events, our method actually shows what, when, and where in the patient's history associated with the prediction. Despite not having an analog-to-odds ratio to explain how each variable contributes to the conclusion, we alleviate the concern that deep learning is a "black box". We believe that it may be used in the future to extract important information for clinicians. The presentation of past medical events at the interface, which is the basis for predicting future clinical events, gives healthcare providers confidence that deep learning models are actually providing timely and useful information.

Labels in datasets used for marginal model development and training may be clinically incorrect or missing. The claim diagnosis may not reflect the clinical diagnosis, for example pneumonia is increasingly encoded as sepsis for administrative reasons. Similarly, readmissions generally occur in separate health systems, and these records are typically not shared with the discharge system. In the absence of complete data shared among health institutions, or datasets with research-grade phenotypes, this limitation affects all data collected during active clinical care.

The second limitation is that our approach relies on large datasets, powerful computing infrastructure, and complex algorithms, which require sophisticated engineering to replicate. However, this approach allows a single modeling architecture to achieve excellent predictive performance over the entire range of predictive tasks, and given the disclosure and accompanying appendices, those skilled in the art will be able to do so. Is within the range of.

Finally, there was concern that the continued use of many variables would lead to overfitting. We estimate real-world performance, by reporting results for provided test sets of patients not used during training, and also to show that results are retained for three separate datasets. By doing so, this concern is alleviated. In addition, the design of the model may include techniques to avoid overfitting.

Although some types of predictions have been mentioned above, the model can be used for other prediction tasks, including:

Drugs and dosages for both unusual dosages or auto-completion and alerting for unexpected prescriptions (sources of medical errors).

The following words, sentences, or paragraphs (documents) in a doctor's notebook for the doctor to review, edit, and auto-complete or suggest parts of the template or document for submission, for example a discharge summary: Writing is a major time burden).

Predict a wide variety of life-threatening events such as intubation, aeration, changes in care intensity (eg ICU transport), organ support, transplantation, etc. to monitor and alert to such events.

Predict physiological deterioration, for example daily, or before giving laboratory test instructions, or, for example, before administering glucose (eg, to prevent hyperglycemia / hypoglycemia). Predict the total cost of care for risk stratification of high-cost patients.

Prediction of hospitalization and census for capacity planning (how many patients will be admitted to each ward).

D. Provider's interface for clinical prediction and understanding through deep learning Once the prediction models 28, 30, and 32 have been developed, tested, and found to be valid as described above, they are then referred to as patients. To improve care, it can be used to make predictions based on input EHR from patients as shown in Figure 1. In this section of this document, we present these predictions to healthcare providers along with the identification of relevant past medical events (test results, diagnoses, notes, medications, etc.) within the EHR. Will explain if it is possible. In essence, the computer 26 in Figure 1 uses models 28, 30, and / 32 to generate data from input health records for predictions and corresponding past medical events and render them on the interface. In addition, this data is provided to the electronic device 40.

FIG. 8A illustrates an example of a healthcare provider facing interface 100 of an electronic device 40 for use by a healthcare provider treating a patient, such as a computer terminal, tablet, smartphone, or other type of computing device with a screen display. Is. Interface 100 in this configuration is designed for use in hospitals. The interface includes display areas 102 and 103 for two patients. For a patient named "Mark Smith," the display predicts two future clinical events for this particular patient, in this case unplanned transport to the intensive care unit (ICU) and delayed discharge from the hospital. Includes alert 104 to indicate that you are doing. The interface in Figure 8A is designed to alert healthcare providers to early attention to patients at risk. The system in Figure 1 accurately predicts a particular event where something is "off," "unusual," or "noticeable." From the physician's point of view, the interface meets the need for early attention when the physician still has time to act. In addition, the interface explains why the predictive model thinks / predicts that an alert condition will occur, as will be explained with Figure 8B.

FIG. 8A includes a toolbar 108 that allows the physician to select a graph display of various probabilities (or risks) on the Y-axis scale 0-100 in the timeline area 105 of display 102. Aspects are also shown. In this example, the physician switched the risks / probabilities of death, discharge, and ICU transport to the "on" position. Line 110 plots the probability of discharge from the hospital. Line 112 plots the probability of ICU transport. Line 114 plots the risk of death. Note that there is a surge in the risk of ICU transport around 16:00, followed by a slight increase in the risk of death. Physicians can investigate these plots / probabilities of risk and gain more information about past medical events regarding the risk of ICU transport and delayed discharge by clicking or selecting the alert icon 104.

When the alert icon 104 is selected, interface 100 changes to the display shown in FIG. 8B. Basically, this version of the interface, which includes predictions to be made and related previous medical events, helps physicians understand the patient now. The doctor said, "What are the major medical issues I need to know? Help me dig into the conditions or other data I may need to see to make a decision. I But don't miss important information. " These needs are met by the display in Figure 8B. In particular, area 130 includes the chief complaints of current hospitalization (abdominal pain), major previous hospitalizations (alcohol withdrawal symptoms, atrial fibrillation), and major inferred diagnoses based on real-time EHR (ie, models 28, 30, 32). A list of problems associated with alerts, such as (diagnosis estimated by) and other major medical conditions of the patient (prediagnosis, hypertension, and stubbornness). Area 140 has a display of current test results. Area 150 contains an excerpt of a medical note that was involved in predicting ICU transport and delayed discharge, and the result of the attention mechanism in the model was a specific element or word in the note that was highly scored by the attention mechanism. Is highlighted in red font (in this case, "EtOH withdrawal symptoms", "Concerns about ETOH withdrawal symptoms"). Area 160 has a display of the current vital signs. Area 120 has a timeline display showing previous admissions, ER visits, and outpatient activities. Line 122 indicates the amount of medical use of the patient, such as how often the patient visited the medical facility, how many test values and vitals were taken, how many drugs were prescribed, and so on. A visualization of the size of the data available for this patient.

Further discussion of the interfaces of FIGS. 8A and 8B will be made below in the description of the examples.

As mentioned above, the predictive model can also be used in outpatient situations to make predictions about the patient. For example, FIG. 9 shows a display of interface 100 and tool 200 on an interface related to the use of the interface in a doctor's office. Tool 200 has similar risks / risks (similar to those shown in area 105 in Figure 8A) for general cost / utilization of emergency department visits, hospitalizations, deaths, and medical resources, similar to line 202 in Figure 8B. Allows physicians and physician care teams to plot a timeline of probabilities.

What Happens On This Day Without The Benefits of The Disclosure The examples of this assumption will demonstrate the difficulty in patient care in the absence of the benefits of the disclosure.

Patient "Mark Smith" comes to the emergency room at 7 am due to severe abdominal pain.
He performs a complete work-up, including examination and abdominal ultrasonography, without clear evidence.
Although his abdominal examination is relatively benign, he still needs IV painkillers.

At 10 am, he is admitted to the internal medicine team for medical examination and pain control. The hospitalized team suspects unspecified gastritis and they expect to be discharged the next day.

The primary team left at 4 pm and handed it over to their doctor (who is responsible for 130 patients), saying, "Smith may have alcohol withdrawal symptoms, but there are still no signs. No "warns. At 8 pm, the attending physician will hand over all 130 patients to Dr. Kingsley, a night shift doctor, and leave. At 8 pm, Dr. Kingsley enters his midnight shift. She is responsible for 130 patients, but she has never previously cared for any of these patients. She begins her shift by transferring to herself a first-call pager for all 130 patients. At 10 pm, Dr. Kingsley receives a pager.

Patient Smith in Room 14L-21 has a heart rate of 99 and is comfortably watching TV in bed. BP115 / 79, RR20, 98% RA. FYI as a call parameter is 90.

At 1:00 am, Dr. Kingsley receives another pager.

Patient Smith in Room 14L-21 has a sepsis alert and should call back at 3-9124.

Dr. Kingsley logs in to the EHR and sees a warning.

Calling attention to sepsis. Patients meet SIRS criteria. According to national guidelines, administer 30cc / kg IV infusion and antibiotics within 1 hour.

Upon closer examination, Dr. Kingsley learns that the patient's heart rate had risen over time from 70 to 99 during the day, and is now 110, with the patient's respiratory rate recorded as 20. (The normal number recorded when the respiratory rate is normal). Lactate (prescribed by the nurse) was 2.5 (slightly higher). Dr. Kingsley's pager is now every 45 seconds, so she has to triage her time. At 1:05 am, she calls back to the nurse who reported, "He's not looking good, he's shaking a little and sweating a lot." While she was speaking, she drew notes of the day, where the primary problem was "unspecified abdominal pain." The note states, "Patients have unspecified abdominal pain, slightly higher liver enzymes, and unspecified gallbladder thickening on ultrasound. Although suspected gastrointestinal inflammation may be from alcohol addiction. , The patient denies. Watch for intraperitoneal lesions. "

The sepsis alert reminded her of the clinical rule that mortality increases by 7.5% every hour when antibiotics are delayed for sepsis. She wants to see the patient, but may not be able to test him for another 30 minutes, and antibiotics may be delayed for more than an hour. She is worried about intra-abdominal infections. She sees if he has ever taken an echocardiogram, but he has never.

At 1:10 am, she prescribed 2 L of IVF, vancomycin, and zocin (antibiotic) and directed abdominal pelvic contrast CT (CT abdomen-pelvis with contrast).

At 2:10 am, the overhead alarm sounds.

Emergency: 14L Room 21

Dr. Kingsley runs to Room 14-L 21 and finds Smith having difficulty breathing. The second bag of IV infusion is almost complete. She hears the sound of his lungs and notices a noticeable crepitus that was not recorded by the day shift team. His JVD is clearly high. She also notices his noticeable tremors and tongue-wag. When asked again, the patient admits that he has drunk heavily this week, but stopped due to abdominal pain two days ago. She stops IV fluids and orders the intensive care unit (ICU) team to transport the patient to the ICU due to iatrogenic acute pulmonary edema and alcohol withdrawal symptoms.

The clarification of this example is as follows. The patient actually had stomach discomfort due to regular use of alcohol and ibuprofen, which caused his abdominal pain. While in the hospital, he began to develop alcohol withdrawal symptoms, which were responsible for his high heart rate, tremors, and sweating. The clinician also overlooked that his outpatient was worried about alcohol cardiomyopathy due to worsening exercise tolerance and had indicated an echocardiogram that had not yet been performed.

After actually entering the patient's examination due to an emergency, the doctor diagnosed her with infusion-induced acute pulmonary edema and recognized alcohol withdrawal. The patient was taken to the ICU for treatment and discharged 4 days later. The patient was readmitted 3 weeks later due to Clostridium difficile enteritis, but may be due to inaccurately given antibiotics.

Root cause analysis asks what Dr. Kingsley missed. The patient was evaluated carelessly after he got worse. The patient had alcoholic gastritis and withdrawal symptoms, which were mistaken for sepsis and were treated incorrectly. The patient was suspected of having cardiomyopathy but should not have received fluids without physical examination and ECG. What was supposed to happen?
1. It should have predicted and prevented the postponement of ICU transport due to alcohol withdrawal symptoms.
2. IV fluid was not given and therefore should have prevented ICU transport against fluid overload.
3. No antibiotics were given and therefore should have prevented nosocomial infections.
4. It should have prevented subsequent readmissions.
Both framing bias and confirmation bias help explain why this happened. Framing bias is whether I withhold life-saving treatment for patients with possible sepsis. Confirmation bias is that doctors look only for possible sources of abdominal sepsis, taking into account the density of information.

Figures 10 to 13 show examples of the large amount of information available to healthcare providers from EHR and why the features of this disclosure are needed. Figure 10 shows an excerpt from the patient's 4-year medical history with 433 listed diagnoses. Figure 11 shows an excerpt of the diagnosis for this patient, but the lack of important accompanying information such as whether the patient was treated as an outpatient, inpatient, or in the ICU or other circumstances makes the information useful. Limited. Figure 12 shows a huge list of various encounters with this particular patient over a given time span, but lacks details about what happened during each encounter. Figure 13 is a small fragment of one of the notes taken by the provider in a single hospitalization over a four-day period (33 notes with a total of ~ 10,000 words filling 60 different screens on a standard mobile device). Only shown.

Simply put, there is a need to help Dr. Kingsley focus his attention only on these factors within the EHR that are actually related to the patient's current condition. The patient care in Example 1 can be improved and therefore the system of the present disclosure can be developed.

Predicted clinical events for ICU transport and delayed discharge This example would show the benefits of the system in Figure 1 in the treatment of patient "Mark Smith" in Example 1. In summary, the system alerts the doctor early to the patient at risk by accurately predicting a particular event, the doctor still has time to act, and the system makes the prediction. Call your doctor early while you have time to explain why you are. When doctors pay attention (for example, by using the alert in Figure 8A), this is what the main problem is, what conditions the doctor may need to look at to make a decision, and what other data It helps doctors understand the patient's current situation and that doctors do not miss important information.

In FIG. 14, the example of interface 100 of the device of FIG. 3 tracks data and risks of 4 patients in real time. Physicians switched tools 108 to customize real-time risk or probability tracking. In FIG. 14, the interface includes four display areas 300, 302, 304, and 306 for four different patients, display area 300 is a display area for patient Mark Smith, and plots and alerts 104. It is as described in Figure 8A.

In our hypothetical embodiment, at 8:02 pm, Dr. Kingsley begins his 8 pm shift and logs in to a system that provides Interface 100 of the disclosure, referred to in this document as the "Guardian." .. She first sees Jerry Mashokitar, who was told she was a "watcher" identified by the Guardian, as the plot shows the increased risk of death indicated by line 303. At 8:03 pm, alert 104 is activated and she finds Mark Smith at the top of the patient list. The reminder is that the patient is at risk of ICU transport and delayed discharge.

The question immediately arises in Dr. Kingsley's mind what is the patient's ongoing medical problems, how serious these alcohol withdrawal symptoms were in the past, and whether they required an ICU stay.
What kind of treatment did they give for heart failure? Did these have a decrease in ejection fraction?
Did they have a previous infection or have they recently received antibiotics? Was there any positive culture?
Was atrial fibrillation difficult to control? Did the patient suddenly stop taking beta-blockers? So what are the major problems with the patient, when, and what is the evidence?

Dr. Kingsley activates Icon 104 and the display shown in Figure 8B appears. The interface shows the risk of ICU transport and this by note area 150 showing excerpts from the notes "Possibility of Hx for Et-OH-withdrawal attacks", "Abuse of EtOH", and "Concerns about ETOH withdrawal symptoms". Attracts her attention to concerns about alcohol withdrawal symptoms that exacerbate the risk. The phrases "EtOH-withdrawal symptoms" and "concerns about ETOH withdrawal symptoms" are shown in red font and in bold. This is the result of the use of attention mechanisms in predictive models as previously described. Thus, the display in Figure 8B summarizes past medical events regarding the predicted current risk (ICU transport).

Figure 15 shows the interface in Figure 8B showing the selection of only the major events out of 152 past encounters in the EHR presented in the patient's timeline area in relation to predictions (ICU transport, delayed discharge). ..

FIG. 16 shows the interface of FIG. 8B showing the selection of only major problems from a list of 433 past diagnoses or problems within the EHR related to prediction (ICU transport, delayed discharge). The main issues (ie, related past medical events) are presented as a summary on the left side of the display in the issue list area.

FIG. 17 shows the interface of FIG. 8B showing major and important excerpts from 12,000 words or word-only selection in notes in the EHR related to prediction (ICU transport, delayed discharge). Key excerpts (words and phrases) are presented in the lower right area of the interface with a degree of highlighting for a particular word or phrase as a result of the use of attention mechanisms in the deep learning model when generating predictions.

FIG. 18 shows the interface of FIG. 8B showing the ability of the interface to summarize each listed medical issue. In this example, the donor clicks on the main problem 400 of "alcohol withdrawal symptoms" in the display area 130 of FIG. 8B, and the display is the drug in field 402, in field 404, associated with the main problem of alcohol withdrawal symptoms Shows a note or note excerpt, and a timeline of events in field 406.

FIG. 19 shows what happens when the user selects the "major guessed" problem of cardiomyopathy, and the display is an excerpt of the associated note or note in timeline 506, drug 502, and field 504. In form, it outlines the main problem "cardiomyopathy". The notes or excerpts in fields 504 and 404 are highlights (bold, font size, etc.) to show the results of the mechanism of interest in the model to remind the physician of the elements of EHR that were most important in generating the predictions. ) Will be used again.

Returning to the description of the treatment of patient Mark Smith using the features of the present disclosure, in Example 1, Dr. Kingsley gets a page of sepsis at 1 am, while Dr. Kingsley desires due to the features of the present disclosure. Instruct early intervention. She goes to see the patient shortly after the alert is presented and directs the CIWA protocol for alcohol withdrawal symptoms, given the very high risk. She sees outpatient suspicion of cardiomyopathy and decides not to give 2 L IV in case the patient actually has heart failure. Given the uncertainty of the diagnosis, she also decides to direct an ECG that takes into account the history of atrial fibrillation and examines the patient.

In summary, the system of the present disclosure has avoided the need to transport the patient to the ICU and to be admitted to the hospital later. In this example, the physician is timely alerted to the predicted clinical event, presented with a major medical event for the prediction, improving the physician's care for the patient, avoiding ICU transport, and to the hospital. Allows physicians to avoid unplanned readmissions and to avoid complications from antibiotic administration.

Outpatient alerts to the risk of ER or hospitalization This example would illustrate the use of the system in Figure 1 in outpatient situations.

Jennifer Choi, an 83-year-old woman with a history of heart failure (EF30%), warfarin-induced atrial fibrillation, hypertension, and prediabetes, visits Dr. Keyes' outpatient heart disease clinic as a first-time patient. Dr. Keyes was asked to manage her heart failure. Choi wants Dr. Keyes to understand her other conditions to ensure that none of the treatments interfere with each other.

Previously, Choi had been examined before seeing her primary care physician (PCP), where she noticed that she had mild acute kidney injury. Her PCP felt her volume status was stable, so he recommended reducing the dose of diuretics and repeating the test for a week.

On her way out of meeting with Dr. Keyes, Choi's daughter said she was worried that her mother was becoming more and more confused and that she might not be taking her medication correctly. Secretly expressed concern. Dr. Keyes said he was already 30 minutes behind his schedule and would investigate this further at a visit three weeks later, saying, "My daughter is getting more and more confused about her patients at her next visit. I am concerned about it. I plan to have a MOCA and evaluation for cognitive dysfunction at the next visit. ”I wrote a note in my notebook.

Both PCP and Dr. Keyes will join the system in Figure 1 and transfer Choi's EHR to computer 26 in Figure 1 to utilize the predictive model. Both PCP and Dr. Keyes have electronic devices (workstations) that include the interface of Figure 9 used for outpatients.

The model predicts that Choi is at risk for ED consultation / hospitalization in the next 14 days. A warning is presented on the display in Figure 9. This team has the expertise to manage these high-risk situations. The display shows a timeline (including recent hospitalizations), such as that shown in field 406 of FIG. 18, which is presumed to be CHF, AKI, AFib, prediabetes, hypertension in field 130 of FIG. 8B. Indicates a problem and includes an excerpt from the following note in field 150 (Figure 8B).
Note 1 (pcp): "Daughter is concerned that the patient is becoming more and more confused. At the next visit, MOCA and assessment for cognitive dysfunction is planned [..]. The patient has worsened renal function. However, it may be due to over-diuresis. I will reduce the dose of Lasix and repeat the test for a week. I told my daughter to monitor her weight and breathing. "
Note 2 (Nurse): "Patients are confused about the dose of Lasix"
Note 3 (Nurse): "I don't know how much dose my mother should take"
As a result of the use of the attention model, the main parts of these notes ("concerned", "confused", "MOCA", "deteriorated renal function", "reduce the dose of Lasix", "I'm confused about the dose of Lasix ", etc.) is depicted in bold font.

Busy Emergency Department This example will demonstrate the use of the features of this disclosure by the hypothetical patient "Mark Smith" in Example 1 and Example 2.

Mark Smith walks into the emergency room, holding his stomach and complaining of pain. His heart rate is 110 and he is shivering, sweating and diuretic. The nurse brings in Dr. Peters, an ED resident, to help understand what is happening.

Dr. Peters has a great many questions. Has he ever been? What kind of illness does he have? How serious are these? How were these treated? ED takes out Mr. Smith's EHR, and the prediction model in Fig. 1 is used for Mr. Smith's EHR. The interface of the terminal or other electronic device presenting the interface retrieves and displays information related to these questions and his current chief complaint, with a predictive diagnosis, as shown in Figure 8B, and the main basis. Include medical events. Once the vital signs are obtained, they are added to the display of the relevant chart information.

Further Examination The precise physical location and implementation of the predictive model and associated computer or computer system 26 may vary. In some cases, it may be physically located in the medical system or related equipment provided by the hospital, the attending physician's office, and the associated clinic. In other situations, it may be central, receive EHRs on a wide area computer network, send predicted future clinical events and related previous medical events, pay for services, subscribe. , Standalone products, or other business models may serve many unrelated medical institutions. In all situations, reasonable data security and HIPPA compliance procedures will be implemented.

10 system
12 Raw electronic health records, electronic health records
14 institutions
16 Computer network
18 converter, computer system
20 RDB, relational database
22 Time-series bundles of FHIR resources, aggregated electronic health records, aggregated health records
22A FHIR resource
22B FHIR resource
22C FHIR resource
22D FHIR resource
24 memory, computer memory
26 Computer
28 LSTM model, deep learning model, long short-term memory (LSTM) model, LSTM, model
30 FFM model, deep learning model, time recognition feed forward model (FFM), model, FFM
32 Embedded boosted TS model, deep learning model, embedded boosted time series model, model, boosted model
36 patients
38 EHR, Input Electronic Health Record, Input Health Record
39 dashed line
40 Electronic devices, devices
42 Providers, Healthcare Providers
50 encounter table, table
52 Inspection table, table
100 Healthcare provider facing interface, interface
102 Display area
103 Display area
104 Attention icon, Attention
105 Timeline area, area
108 Toolbars, tools
110 line
112 line
114 lines
120 areas
122 lines
130 areas, display areas, fields
140 areas
150 areas, note areas, fields
160 areas
200 tools
202 display line
300 display area
302 display area
303 line
304 Display area
306 Display area
400 Major issues
402 field
404 field
406 field
502 drug
504 field
506 timeline

Claims (60)

A system for predicting and summarizing medical events from electronic health records.
a) Computer memory that stores aggregated electronic health records from a large number of patients of diverse ages, health conditions, and demographics, including some or all of medications, test values, diagnostics, vital signs, and medical notes. The aggregated electronic health records are converted into a single standard data structure format and are a regular array for each patient, with computer memory.
b) Predict one or more future clinical events based on the input electronic health records of patients with said standard data structure format, and relevant past for the predicted one or more future clinical events. To summarize a medical event, a computer running one or more deep learning models trained on the aggregated health record in the regular sequence converted to the single standard data structure format, and
c) Use by a healthcare provider treating the patient, consisting of a provider-facing interface displaying the predicted one or more future clinical events and the relevant past healthcare events for the patient. A system with electronic devices for. The system of claim 1, wherein the aggregated health records include health records arranged in at least two different data formats. The system according to claim 1 or 2, wherein the standard data structure format comprises Fast Healthcare Interoperability Resources (FHIR). At least one of the one or more deep learning models predicts the one or more future clinical events and is relevant past medical care for the predicted one or more future clinical events. To summarize the event, claim one of claims 1 to 3, comprising a focus mechanism indicating how much attention the one or more models have shown to the elements in the electronic health record. System. The system of claim 4, wherein one or more deep learning models include a long short-term memory (LSTM) model, a time-aware feedforward model (FFM), and an embedded boosted time series model. The system of claim 5, wherein each of the deep learning models is trained separately in the dataset that produces the aggregated electronic health record. Predictions of one or more future clinical events, and summarized and relevant past medical events for the predicted one or more future clinical events are obtained from the ensemble average of the deep learning model. The system according to claim 5. One or more of the predicted future clinical events include unplanned transport to the intensive care unit, length of stay longer than 7 days, unplanned readmission within 30 days of discharge of the patient, death of the inpatient, 1 Claims that include the entire set of secondary, primary and secondary claims, or at least one of abnormal test values such as acute renal injury, hypokalemia, hypoglycemia, or hyponatremia. The system according to any one of items 1 to 7. The interface of the electronic device
(1) Attention to one or more of the predicted future clinical events and
(2) Major medical problems or conditions related to the above alert, and
(3) The system according to claim 1, comprising displaying a note relating to the alert or an excerpt of the note. At least one of the one or more deep learning models predicts the one or more future clinical events, respectively, and the relevant past with respect to the predicted one or more future clinical events. In order to summarize a medical event, the indication of the note or excerpt of the note includes a focus mechanism indicating how much attention the one or more models have shown to an element in the electronic health record. , The system according to claim 9, which is displayed by a method showing the result of applying the mechanism of interest. A method for predicting and summarizing medical events from electronic health records.
a) A step in aggregating electronic health records from a large number of patients of diverse age, health status, and demographics, said electronic health records are some of the medications, test values, diagnoses, vital signs, and medical notes. Or including all, steps and
b) The steps to convert the aggregated electronic health record into a single standard data structure format, arranged patient by patient in a regular sequence, and
c) With the step of training one or more deep learning models with the aggregated health record of the regular sequence converted to the single standard data structure format.
d) Predict one or more future clinical events from the input electronic health records of patients with the standard data structure format and arranged in order of occurrence, and predict one or more future clinical events. With the steps of using the trained one or more deep learning models to summarize relevant past medical events related to the event,
e) by a healthcare provider treating the patient having a healthcare provider-facing interface to display the predicted one or more future clinical events and the relevant past healthcare event for the patient. A method that includes steps to generate data for an electronic device for use. 11. The method of claim 11, wherein the aggregated health records include health records from a large number of patients arranged in various data formats. The method of claim 11 or 12, wherein the standard data structure format comprises Fast Healthcare Interoperability Resources (FHIR). At least one of the one or more deep learning models predicts the one or more future clinical events and relevant past medical events relating to the predicted one or more future clinical events. 11. The method of claim 11, comprising an attention mechanism indicating how much attention the one or more models have shown to an element in the electronic health record to summarize. 15. The method of claim 14, wherein one or more deep learning models include a long short-term memory (LSTM) model, a time-aware feedforward model (FFM), and an embedded boosted time series model. 15. The method of claim 15, wherein each of the deep learning models is trained separately in the dataset that produces the aggregated electronic health record. Predictions of one or more future clinical events, and summarized and relevant past medical events for the predicted one or more future clinical events are obtained from the ensemble average of the deep learning model. The method of claim 15. The one or more deep learning models include unplanned transport to the intensive care unit, length of stay longer than 7 days, unplanned readmission within 30 days of discharge of the patient, death of the inpatient, primary diagnosis, One or more of the entire set of primary and secondary claim diagnoses, or at least one of abnormal test values such as acute renal injury, hypokalemia, hypokalemia, or hyponatremia. The method of any one of claims 11-17, which predicts future clinical events. The above method
(1) Attention to one or more future clinical events and
(2) Major medical problems or conditions related to the above alert, and
(3) The method of claim 11, further comprising generating data for display on the interface of the electronic device, such as a note relating to the alert or an excerpt of the note. The one or more deep learning models predict the one or more future clinical events, respectively, and summarize the relevant past medical events for the predicted one or more future clinical events. In order to include an attention mechanism indicating how much attention the one or more models have shown to an element in the electronic health record, the indication of the note or excerpt of the note is of the attention mechanism. The method of claim 20, which is displayed in a manner that indicates the result of the application. The display of the note or an excerpt of the note showing the result of the application of the attention mechanism is a font to provide a gradation of highlights or highlights for a particular word, phrase, or other text in the note. Claims include size, font color, shading, bold, italics, underlining, strikethrough, blinking, colored highlights, and display of said note or excerpt of said note using at least one of the font selections. The method described in 20. Predict one or more future clinical events based on the patient's input electronic health records, which have a standard data structure format and are arranged in order of occurrence, and for the predicted one or more future clinical events. Equipped with a processor that runs one or more deep learning models trained in a regular array of aggregated electronic health records transformed into a single standard data structure format to summarize relevant past medical events. Improved computer. At least one of the one or more deep learning models predicts the one or more future clinical events and is relevant past medical care for the predicted one or more future clinical events. 22. To summarize an event, claim 22 includes a focus mechanism that indicates how much attention was given to an element in the electronic health record by at least one of the one or more models. Improved computer. 22. The improved computer of claim 22, wherein the one or more deep learning models include a long short-term memory (LSTM) model, a time-aware feedforward model (FFM), and an embedded boosted time series model. 24. The improved computer of claim 24, wherein each of the deep learning models is trained separately in the dataset that produces the aggregated electronic health record. Predictions of one or more future clinical events, and summarized and relevant past medical events for the predicted one or more future clinical events are obtained from the ensemble average of the deep learning model. The improved computer according to claim 24. One or more of the predicted future clinical events include unplanned transport to the intensive care unit, length of stay longer than 7 days, unplanned readmission within 30 days of discharge of the patient, death of the inpatient, 1 Claims that include the entire set of secondary, primary and secondary claims, or at least one of abnormal test values such as acute renal injury, hypokalemia, hypoglycemia, and hyponatremia. Item 24. The improved computer according to item 24. The improved computer according to any one of claims 22 to 27, wherein the processor operates to generate predictions for multiple input electronic health records of different patients in substantially real time. a) Aggregate electronic health from a large number of patients of diverse ages, health conditions, and demographics obtained in various formats, including some or all of medications, test values, diagnoses, vital signs, and medical notes. A computer memory for storing records, wherein the aggregated electronic health records are converted into a single standard data structure format and arranged in a regular arrangement for each patient.
b) One or more trained in the aggregated health record in the regular sequence converted to the single standard data structure format to predict future clinical events based on the patient's input electronic health records. A system equipped in combination with a computer that executes a deep learning model. 29. The system of claim 29, wherein the aggregated health records include health records arranged in various data formats. The system of claim 29 or 30, wherein the standard data structure format comprises Fast Healthcare Interoperability Resources (FHIR). 29. The system of claim 29, wherein the aggregated health record contains variable names that are not harmonized with standard terms, excluding the variables necessary to define key outcomes and exclusion criteria. 29. The system of claim 29, wherein the aggregated health record comprises an inpatient diagnosis, the diagnosis being mapped to a single level clinical classification software (CCS) code. The system according to any one of claims 29 to 33, wherein the electronic health records are arranged by patient in order of occurrence. A method for generating training data for machine learning from a large set of raw electronic health records of a large number of patients from a variety of sources in a variety of data formats.
a) Steps to obtain the set of raw electronic health records and
b) The steps to convert the set of raw electronic health records into a single standard data structure format,
c) A step of arranging the electronic health records converted into the single standard data structure format in chronological order for each patient.
d) A method comprising storing the time-series electronic health records of the standard data structure format in a data storage device. 35. The method of claim 35, wherein the standard data structure format comprises Fast Healthcare Interoperability Resources (FHIR). 36. The method of claim 36, wherein the set of electronic health records includes both structured data and unstructured data, including free text notes. 35. The method of claim 35, wherein the conversion step b) is performed without harmonizing the terms in the raw electronic health record with standard terms. a) Predict one or more future clinical events and predict one or more future clinical events based on patient input electronic health records having the standard data structure format and arranged in order of occurrence. To summarize relevant past medical events related to clinical events, one or more deep learning models trained with an aggregated health record of regular sequences per patient transformed into the single standard data structure format. The computer to run and
b) For use by a healthcare provider treating the patient, configured to display the predicted one or more future clinical events for the patient, and the relevant past medical events. A system that combines an electronic device with a healthcare provider facing interface. The interface of the electronic device
(1) Attention to one or more future clinical events and
(2) Major medical problems or conditions related to the above alert, and
(3) The system of claim 39, comprising displaying the note relating to the alert or an excerpt of the note. At least one of the one or more deep learning models predicts the one or more future clinical events, respectively, and the relevant past with respect to the predicted one or more future clinical events. To summarize a medical event, the note or note comprises a focus mechanism indicating how much attention the at least one of the one or more models has shown to an element in the electronic health record. 40. The system of claim 40, wherein the display of the excerpt is displayed in a manner that indicates the result of the application of the mechanism of interest. 40. The system of claim 40, wherein said display further comprises estimated information from the patient's electronic health record and at least one display of a timeline of probability or risk that a given event will occur in the future. The display allows the user of the electronic device to select one of the major problems or conditions, and the selection triggers further display of information related to the selected major problem or condition. The system according to any one of claims 39 to 42. 43. The system of claim 43, wherein the further indication comprises an indication of a drug prescribed to the patient and a note or excerpt of the note associated with the selected major problem or condition. The display of the note or an excerpt of the note showing the result of the application of the attention mechanism is a font to provide a gradation of highlights or highlights for a particular word, phrase, or other text in the note. Claims include size, font color, shading, bold, italics, underlining, strikethrough, blinking, colored highlights, and display of said note or excerpt of said note using at least one of the font selections. The system described in 41. An electronic device with a healthcare provider-facing interface that provides a virtually real-time display of predictions of one or more future clinical events for at least one patient.
The display is further configured to display elements containing past medical events from the electronic health record, corresponding to the application of a mechanism of interest on a predictive model that operates on the electronic health record for the prediction.
Electronic device. 46. The element of the electronic health record comprises the note or a citation of the note with a gradation of highlights or highlights for a particular word, phrase, or other text in the note. Electronic device. 46. The highlight or highlight gradient comprises the use of at least one of font size, font color, shading, bold, italics, underlining, strikethrough, blinking, colored highlights, and font selection. The electronic device described in. 46. The electronic device of claim 46, wherein the electronic device includes a workstation, tablet computer, or smartphone. One or more of the predicted future clinical events include unplanned transport to the intensive care unit, length of stay longer than 7 days, unplanned readmission within 30 days of discharge of the patient, death of the inpatient, 1 Claims that include the entire set of secondary, primary and secondary claims, or at least one of abnormal test values such as acute renal injury, hypokalemia, hypoglycemia, and hyponatremia. The electronic device according to item 46. 46. The electronic device of claim 46, wherein the interface is further configured to display a timeline that plots the risk or probability of at least one patient for an event over time. 46. The electronic device of claim 46, wherein the interface is further configured to simultaneously display to a plurality of patients a timeline that plots the risk or probability of at least one patient for an event over time. 46. The electronic device of claim 46, wherein said display of a prediction of one or more future clinical events is in the form of a warning display. A way to help healthcare providers care for their patients
From aggregated electronic health records to (1) generate predictions of future clinical events for the patient and (2) identify relevant past medical events from the input electronic health records for the patient. Steps to use the trained predictive model and
b) Steps to generate data for both the predictions and the identified related past medical events.
c) Including the step of transmitting the generated data to the electronic device used by the healthcare provider for display on the electronic device.
Shows how much attention the predictive model has given to the elements in the input electronic health record to predict the future clinical events and identify relevant past medical events. A method of using a mechanism of interest for the purpose of which the generated data comprises said result of the mechanism of interest. 54. The method of claim 54, wherein the relevant past medical event comprises a note or an excerpt of the note. The predictions are unplanned transport to the intensive care unit, length of stay longer than 7 days, unplanned readmission within 30 days of discharge of the patient, death of the inpatient, primary diagnosis, primary and secondary claims. 54. The method of claim 54, selected from the entire set of diagnoses, as well as a group of abnormal test values. 54. The method of claim 54, wherein the generated data further comprises a timeline of probability or risk that the event will occur over time. Steps a), b), c), and d) are performed simultaneously in real time for a large number of patients from a large number of input electronic health records to provide healthcare delivery that cares for at least two of the large number of patients. Receiving the generated data to the at least two patients in real time, thereby helping the healthcare provider to care for the at least two patients at the same time and to the individual predictions. The method of claim 54, which allows for prioritization of patient care for at least two of the patients. 54. The method of claim 54, wherein the predictive model comprises an ensemble of deep learning models individually trained on the aggregated electronic health record, at least one of which incorporates the mechanism of interest. 59. The method of claim 59, wherein the ensemble comprises (1) a long short term memory (LSTM) model, (2) a time recognition feed forward model (FFM), and (3) an embedded boosted time series model.

Images