JP2022545017A - Unsupervised Learning and Treatment Line Prediction from High-Dimensional Time Series Drug Data - Google Patents

Abstract

In one aspect, the present disclosure provides a method for labeling one or more agents co-administered to a patient as a line of therapy. The method comprises: identifying a patient medical record from a plurality of digital records; creating from a subset of the medical records a plurality of treatment intervals including at least one drug administered to the patient and a time interval; Associating one or more therapeutic agents with their respective treatment intervals when the administration of the agents falls within the time interval and one treatment of the one or more treatments falling outside that time interval , refining the time interval for each treatment interval when within the extension period, identifying one or more potential treatment lines from multiple treatment intervals, and identifying the potential treatment line(s) with the highest maximum likelihood estimate. and labeling the therapeutic line of treatment as a therapeutic line.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is the subject of U.S. Provisional Patent Application No. 62/890,178, entitled "Unsupervised Learning And Prediction Of Lines," filed Aug. 22, 2019, the entire contents of which are incorporated herein by reference. of Therapy From High-Dimensional Longitudinal Medications Data”.

FIELD OF THE DISCLOSURE The present disclosure relates to computer-implemented methods and systems for labeling lines of care present in patient records, and more particularly, unsupervised approaches for labeling lines of care from drug data. related to

Line of treatment (LoT) is the standard nomenclature for discussing antineoplastic therapy. Both the National Comprehensive Cancer Network and the Association for Clinical Oncology (ASCO), bodies that publish treatment guidelines for standard of care (SoC), present findings within the LoT framework. Oncologists consider these guidelines closely when planning treatment for their patients. Additionally, the composition of the LoT will be considered by regulators, payers (both private and institutional), and provider groups in planning, approving, and paying for new anticancer agents. As such, pharmaceutical companies also approach planning and pilot design with consideration of the potential impacts/benefits to patients realized by LoT and new drugs. Physician highlights LoT prescribed for patient, any adverse, exacerbation, or intervening events, and any subsequent changes to LoT to compensate or adapt treatment to improve patient outcome Frequently summarize the patient's history to another physician by Unfortunately, this type of informal summary is never entered into a patient's Electronic Medical/Health Record (EMR/EHR). When a physician consents to provide a patient's EMR, the provided records are analyzed to determine the LoT as well as the significant intervening (superseding) events (progression, regression, metastasis, length of time). ) and provided to physicians for their convenience to improve physicians' understanding of the LoT history for each patient.

This is a difficult task to achieve because the information recoverable from EHRs and/or progress notes alone is never complete. There are many inaccuracies, discrepancies, missing records, and other imperfections that may (or may not) appear in the record, and these need to be considered. For example, an oncologist may consider two LoTs, one drug/treatment/therapy combination of cyclophosphamide, doxorubicin, paclitaxel as a line of treatment or series of monotherapy. A patient's insurance company may refuse to take paclitaxel because of the cost, so the patient receives cyclophosphamide and doxorubicin. After a series of doses, the patient may decide that the combination is adversely affecting their overall health and may transition to maintenance therapy with paclitaxel alone. On the EMR, all these agents can appear to be registered for several months, but this is because the patient did not even receive paclitaxel for the first few items of the EMR and only received cyclophosphamide for one portion of that period. Even if it was only in the department. A CT scan may then be ordered by an oncologist to monitor tumor growth. When the patient returns 6 months later with worsening symptoms, the physician may record the worsening of symptoms as an exacerbation event and list the drugs carboplatin and gemcitabine, or the physician may give the drug gemcitabine alone. and keep the record obscure for previous medications. From an abstraction point of view, the EMR simply records that the prior medication was prescribed and that gemcitabine was prescribed 6 months later. EMR can document CT scans and even worsening symptoms over a 6-month time frame. The difficulty of creating LoT from these records is manifold.
1) If a patient does not take or discontinues their medication, the LoT that indicates that they have taken them is unreliable from a data science perspective.
2) From an industry perspective, a change in LoT that merely adjusts a drug to avoid adverse drug effects is not a new LoT, but the same LoT. Therefore, a drug change does not necessarily represent a new LoT. Identifying whether a change in medication is consistent with an exacerbation event, worsening symptoms, or any other significant interventional event can be tricky, for example, when patients If not taken, this can be difficult to correct for exacerbation of symptoms as a change in LoT, or just avoid side effects relative to the original LoT.
3) From a data science point of view, it can be difficult to impute whether a drug was continued partly or wholly throughout the year, even after a new drug was prescribed.
4) From a clinician's point of view, some drugs can be considered essentially the same drug despite name changes.
5) Patients receive a number of medications, termed 'palliative care' medications, as part of their treatment, but are irrelevant to LoT assignment. Moreover, distinguishing between these is not always straightforward, as agents considered "palliative therapy" versus "primary therapy" vary by cancer type.
6) Heterogeneity of data sources. Curation from EHRs and follow-up records varies from source to source, requiring harmonization to common standards prior to LoT decisions.
7) Overcoming the burden and complexity of spotty data. Few patients have complete cancer treatment records covered by both the EHR and curated follow-up records. Often only one or the other is available, and when both are present, they describe inconsistent parts of the patient's timeline. This complicates matters, especially when the records generally record the beginning of a set of drugs, but rarely when they are stopped.

Currently, no algorithms exist to predict, digest, or impute LoT from EMR. This generally requires a trained practitioner to manually review files and make these decisions on a case-by-case basis for every patient, which is costly and time consuming. Machine learning cross-sectionally for all patients based on the frequency of medication changes for a particular diagnosis due to intervening events, which typically reflect that LoT can be predicted from imperfect data, is common occurrence. It can be applied to discuss drugs. To address this, machine learning approaches that synthesize heuristics such as rule-based structures with clinical insights and expectation maximization (EM) that use machine learning algorithms (MLA) to make effective predictions. Algorithms can be considered.

U.S. Patent Application No. 16/732,168

In one aspect, the present disclosure provides a method for labeling one or more agents co-administered to a patient as a line of therapy. The method comprises identifying a patient's medical record from a plurality of digital records, wherein a subset of the medical records reflects the patient's medical history regarding one or more treatments after diagnosis of the patient's medical condition. and from a subset of the medical records, creating a plurality of treatment intervals comprising at least one drug administered to the patient and the time interval; and administering one or more treatments when the drug administration falls within the time interval. drugs to their respective treatment intervals and the time interval of each treatment interval is refined when one of the one or more treatments falls outside the time interval but within the extension period and no interval precedence events occur in the refined time interval, refining and, via an unsupervised artificial intelligence engine, one or more potential treatment lines from multiple treatment intervals. wherein each potential treatment line includes one or more of a plurality of treatment intervals and maximum likelihood estimation, and through an unsupervised artificial intelligence engine, the best labeling potential lines of therapy with maximum likelihood estimates as lines of therapy.

The method chronologically generates additional potential lines of treatment in consecutive time intervals after each preceding line of treatment, via an unsupervised artificial intelligence engine, and generates potential lines of treatment within each consecutive time interval. It may further include labeling the treatment lines as incrementally numbered from the potential treatment lines having the highest maximum likelihood estimates for each of the lines. The method includes, via an unsupervised artificial intelligence engine, labeling each incrementally numbered treatment line for multiple patients, identifying multiple cohorts of patients, and multiple cohorts of patients. reporting the incrementally numbered lines of treatment for each of the cohorts. The method may further include reporting facility-wide and physician-specific compliance to standardized treatment guidelines. The method may further include reporting changes in treatment lines over time based at least in part on respective time intervals for each treatment line. The method may further comprise reporting an analysis of differences in treatment impact between a first cohort of patients and a second cohort of patients, the cohorts comprising one or more similar treatment days, treatment Identify patients as having line, diagnosis, outcome, geographic location, treating physician, treatment site, genomic marker, or clinical characteristics. The method may further comprise reporting an analysis of progression-free survival between the first cohort and the second cohort. The method comprises generating a progression-free survival curve for a first cohort of patients having at least one labeled line of therapy in common and patients having at least one other labeled line of therapy in common. and displaying the progression-free survival curves for the first and second cohorts on the same survival graph. In this method, the labeled therapeutic line may include one or more of drug name, therapeutic rollup class, dosage, or method of administration.

The method may further include generating a report, the report including each treatment line labeled for the patient and the respective time interval.

In this method, medical condition diagnoses can include cancer, heart disease, depression, mental health, diabetes, infectious diseases, epilepsy, dermatology, and autoimmune diseases.

In this method, the medical condition diagnosis may be cancer.

In this method, the plurality of digital records may include one or more EHR medical records and one or more curated medical records. In this method, the curated medical records may be images of records that are not included in one or more EHR medical records. The method may further include binarizing, optical character recognition, and encoding the curated medical records into a structured format.

The method may further include identifying the therapy administered to the patient and the administration date on which the therapy was administered by the therapy identifier.

In this method, treatment includes administration of drugs, drugs, molecules or chemicals, implantation of medical devices, use of medical devices, biotherapy, viral therapy, phage therapy, phytotherapy, gene therapy, epigenetic therapy, protein therapy. , enzyme replacement therapy, hormone therapy, cell therapy, immunotherapy, antibody therapy, nutritional therapy, use of electromagnetic radiation therapy or radiation therapy, surgical procedures, or radiosurgery.

In another aspect, the present disclosure provides a system for labeling one or more drugs co-administered to a patient as a line of therapy. The system is at least one computer having at least one processor for identifying patient medical records from a plurality of digital records, wherein a subset of the medical records includes identifying, reflecting a patient's medical history regarding one or more treatments, and creating from a subset of the medical records a plurality of treatment intervals including at least one drug administered to the patient and a time interval; , associating the drugs of one or more treatments with their respective treatment intervals when the administration of the drugs falls within the time interval, and the time interval of each treatment interval to one of the one or more treatments refining when the treatment is out of the time interval but within the extension period, wherein no interval precedence event occurs in the refined time interval, refining. and at least one computer, the at least one computer including an unsupervised artificial intelligence engine, the unsupervised artificial intelligence engine to identify one or more potential treatment lines from the plurality of treatment intervals. each potential treatment line includes one or more of the plurality of treatment intervals and a maximum likelihood estimate; and labeling the potential treatment line with the highest maximum likelihood estimate as that treatment line. It is programmed to attach and

In the system, an unsupervised artificial intelligence engine chronologically generates an additional potential line of treatment in successive time intervals after each preceding line of treatment, and generates an additional potential line of treatment in each successive time interval. can be programmed to label as the treatment line numbered incrementally from the potential treatment line with the highest maximum likelihood estimate of . In the system, an unsupervised artificial intelligence engine labels each incrementally numbered treatment line for multiple patients, identifies multiple cohorts of patients, and incrementally identifies multiple cohorts of patients for each of the multiple cohorts of patients. can be programmed to report the line of treatment numbered to. In the system, at least one computer may be further programmed to report facility-wide and physician-specific compliance to standardized treatment guidelines. In the system, at least one computer can be programmed to report changes in treatment lines over time based, at least in part, on respective time intervals for each treatment line. In the system, the at least one computer may be further programmed to report an analysis of differences in treatment impact between a first cohort of patients and a second cohort of patients, the cohorts comprising: Identify patients as having one or more similar treatment dates, lines of treatment, diagnoses, outcomes, geographic location, treating physicians, treatment centers, genomic markers, or clinical characteristics. In the system, at least one computer can be further programmed to report an analysis of progression-free survival between the first cohort and the second cohort. In the system, at least one computer generates a progression-free survival curve for a first cohort of patients having at least one labeled line of therapy in common and at least one other labeled line of therapy in common. and to display the progression-free survival curves for the first and second cohorts on the same survival graph. In the system, a labeled therapeutic line may include one or more of drug name, therapeutic rollup class, dosage, or method of administration.

In the system, medical condition diagnoses can include cancer, heart disease, depression, mental health, diabetes, infectious diseases, epilepsy, dermatology, and autoimmune diseases.

In the system, the multiple digital records may include one or more EHR medical records and one or more curated medical records. In the system, at least one computer may be further programmed to binarize, optical character recognize, and encode the curated medical records into a structured format.

In the system, treatment includes administration of drugs, drugs, molecules or chemicals, implantation of medical devices, use of medical devices, biological therapy, viral therapy, phage therapy, plant therapy, gene therapy, epigenetic therapy, protein therapy, It may include one or more of enzyme replacement therapy, hormone therapy, cell therapy, immunotherapy, antibody therapy, nutritional therapy, use of electromagnetic radiation therapy or radiation therapy, surgical procedures, or radiosurgery.

The file of this patent contains at least one drawing/photograph executed in color. Copies of this patent with color drawings/photographs will be provided by the Office upon request and payment of the necessary fee.

[0014] Figure 4 is a flowchart illustrating one embodiment of line of treatment (LoT) prediction preparation, training, and generation. [0013] Figure 1 is a flow chart illustrating one embodiment of preparing, training, and generating line of care (LoT) predictions using the MLA approach with rule-based preprocessing. FIG. 4 is an illustration of LoT prediction for a first patient diagnosed with metastatic non-small cell lung cancer. FIG. 10 is an illustration of LoT prediction for a second patient diagnosed with ovarian cancer. FIG. 11 is an illustration of LoT prediction for a third patient diagnosed with breast cancer; An artificial intelligence engine approach with rule-based preprocessing that implements a method to impute complex curated fields from curated patient histories and apply them to non-curated patients or patients with small amounts of data. 6 is a flowchart illustrating one embodiment of the system 600 used to prepare, train, and assign lines of treatment (LoT). [0014] Figure 4 is a flow chart showing one embodiment of preparing, training and generating a line of treatment (LoT) model over three stages. FIG. 4 is an illustration of LoT prediction for a first patient diagnosed with metastatic non-small cell lung cancer. FIG. 10 is an illustration of LoT prediction for a second patient diagnosed with ovarian cancer. FIG. 11 is an illustration of LoT prediction for a third patient diagnosed with breast cancer; FIG. 4 illustrates the first step of the flow; 11B illustrates the number of dosing intervals included in the second step of the flow of FIG. 11A. FIG. 11B illustrates applying heuristics to refine one or more dosing intervals in the third step of the flow of FIG. 11A. FIG. 11B illustrates outputting line assignments in the fourth step of the flow of FIG. 11A; FIG. FIG. 4 illustrates the first step of the flow; 12B illustrates the number of dosing intervals included in the second step of the flow of FIG. 12A. FIG. 12B illustrates applying heuristics to refine one or more dosing intervals in the third step of the flow of FIG. 12A. FIG. FIG. 12B illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of the flow of FIG. 12A. FIG. 12B illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of the flow of FIG. 12A. 12B illustrates outputting line assignments in the fifth step of the flow of FIG. 12A; FIG. FIG. 4 illustrates the first step of the flow; 13B illustrates the number of dosing intervals included in the second step of the flow of FIG. 13A. FIG. FIG. 13B illustrates applying heuristics to refine one or more dosing intervals in the third step of the flow of FIG. 13A. FIG. 13B illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of the flow of FIG. 13A. FIG. 13B illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of the flow of FIG. 13A. 13B illustrates outputting line assignments in the fifth step of the flow of FIG. 13A; FIG. 1 is a Sankey diagram illustrating lines of therapy as applied to patients with at least immuno-oncology (IO) regimen therapy. 1 is a progression-free survival (PFS) graph depicting survival curves for patients receiving different first lines of therapy. FIG. 12 is an illustration of a line of treatment editing tool for user editing of a line of treatment for a patient according to one embodiment. FIG. 10 is an illustration of a structured data representation of a patient timeline after LoT labeling has been performed according to one embodiment; Boxplot of average duration of LoT identified from cohorts of patients in lung cancer cohort and breast cancer cohort. FIG. 4 is an explanatory diagram of LoT frequencies based on the first LoT for breast cancer therapeutics administered to patients. FIG. 10 is an explanatory diagram of LoT frequencies based on the second LoT for breast cancer therapeutics administered to patients. FIG. 4 is an illustration of LoT frequencies based on the first LoT for non-small cell lung cancer (NSCLC) therapeutics administered to patients. FIG. 4 is an illustration of LoT frequencies based on the second LoT for non-small cell lung cancer therapeutics administered to patients. FIG. 10 is a bar graph representation of the percentage of patients for each primary cancer assigned to cohorts containing at least one of the listed LoTs. 1 is an illustration of a block diagram of one implementation of a computer system upon which some implementations of the disclosure may operate; FIG.

The phrase "medical condition" means a disease state such as cancer, heart disease, depression, mental health, diabetes, infectious disease, epilepsy, dermatology, autoimmune disease, or other disease. A medical condition may reflect the presence or absence of a disease in a subject, and may further reflect the severity of the disease when it is present.

FIG. 1 illustrates one embodiment of a computer-implemented system 100 for labeling treatment lines for patients based on drugs administered and clinical records. A line of treatment may be generated from patient information represented by the clinical health record 110 implemented by the system architecture 100 . System 100 may be a content server (also called a LoT engine), which is hardware or a combination of both hardware and software. A user, such as a healthcare provider or patient, is given remote access through the GUIs 170a-170n to obtain information about the patient's medical condition using their own local device (e.g., personal computer or wireless handheld device). Browse, update and analyze. A user can interact with the system to generate electronic records, update electronic records, and issue instructions to perform other actions. The content server is configured to receive a variety of information in different formats, which is put into a standardized format suitable for processing by the operation of the line of care module 120 on or in conjunction with the content server. to convert Thus, information obtained from a patient's electronic medical record (EMR), unstructured text, genetic sequencing, image processing, and a variety of other information is used to train one or more artificial intelligence engines or machine learning models. can be converted into features used to

Information retrieved, processed, and generated by the content server 100 is stored in one or more of the network-based storage devices. A user can interact with a content server to access information stored in a network-based storage device, the content server receiving user-provided information stored in the network-based storage. One or more models can be applied to the information and, in electronic form, the results of the model application can be provided to the user on the graphical user interface of the user device. Electronic information is transmitted in a standardized format over computer networks to users who access that information. In this way, the user can readily adapt medical diagnosis and treatment strategies according to the system's predictions, which can be automatically generated. In addition, the system generates recommendations to the user regarding patient diagnosis and treatment.

In some embodiments, the systems and methods described are implemented as part of a digital and laboratory healthcare platform. The platform can automatically generate molecular reports as part of targeted medical care precision drug therapy. In some embodiments, systems according to embodiments of the present disclosure operate on one or more microservices, which may be microservices of an order management system. In some embodiments, the system is implemented in conjunction with one or more microservices of cell type profiling services.

A clinical health record 110 may store a collection of features, or condition characteristics, generated for some or all patients for whom information exists in the system 100 . These characteristics can be used to identify drugs and priority events using system 100 . Although the feature coverage across all patients is informationally dense, the Patient Feature Set may be sparsely populated across the collective feature coverage of all features across all patients. For example, feature coverage across all patients may grow to tens of thousands of features, but a patient's unique feature set may be a subset of the collective feature coverage of hundreds or thousands based on available records for that patient. can contain.

The multiple features present within the clinical health record 110 may include a diverse set of fields available within a patient's health record. Clinical information fields entered into an electronic medical record (EMR) or electronic health record (EHR), which may be performed automatically or manually by, for example, a doctor, nurse, or other medical professional or representative may be based on Other clinical information may be obtained from other sources such as, for example, genetic sequencing reports (e.g., from molecular fields), progress reports, laboratory results, insurance claims codes and/or records, and other medical documentation for patients. It may also be digitally curated information 114 that is published. Sequencing may include next generation sequencing (NGS) and may be long reads, short reads, or other forms of sequencing the patient's somatic cells and/or normal genome. A comprehensive collection of features within an additional clinical health record may include diagnostics, response to therapeutic regimens, genetic profiles, clinical and phenotypic features, and/or other medical, geographic, demographic, and clinical characteristics. Various features may be combined together across various medical disciplines, which may include genetic, molecular, or genetic features. For example, the subset of features may include molecular data features such as features derived from RNA or DNA sequencing, pathological features derived from cellular review by a pathologist, or other images.

Image features may include, for example, features identified through review of specimens by a pathologist, such as review of stained H&E or IHC slides. As another example, a subset of features may include derived features obtained from analysis of individual and combined results of such feature sets. Features derived from DNA and RNA sequencing can include genetic variants that are identifiable in the sequenced samples. Further analysis of genetic variants includes identifying single or multiple nucleotide polymorphisms, identifying whether the mutation is an insertion or deletion event, identifying loss-of-function or gain-of-function, identifying fusions. steps, calculating copy number changes, calculating microsatellite instability, calculating tumor gene mutational burden, or other structural changes in DNA and RNA. Analysis of slides for H&E or IHC staining reveals features such as tumor infiltration, programmed cell death ligand 1 (PD-L1) status, human leukocyte antigen (HLA) status, or other immunologically relevant features can be

Features derived from structured, curated, and/or electronic medical or health records 112 and 114 include clinical features such as diagnoses, symptoms, treatments, outcomes, patient name, date of birth, , gender, ethnicity, date of death, address, smoking status, cancer, illness, disease, diabetes, depression, patient demographics such as date of diagnosis of other physical or mental illness, medical history, family medical history, date of first medical examination; Clinical diagnosis such as date of diagnosis of metastases, cancer stage, tumor characterization, tissue of origin, treatment line, treatment group, clinical trials, prescribed or taken medications, surgery, radiotherapy, imaging, side effects, related Treatment and outcomes such as outcomes, performance scores, laboratory tests, pathology results, genetic testing and laboratory information such as prognostic indicators, genetic test dates, test providers used, uses such as genetic sequencing methods or gene panels The method of testing performed, genes included, variants, genetic results such as expression levels/status, or corresponding dates associated with any of the above.

Machine learning models and/or artificial intelligence engines may perform classification labeling and/or analysis by selecting from arbitrary features. One method for enriching the patient feature set may include dimensionality reduction, such as collapsing a feature set from tens of thousands of features into a handful of features. Performing dimensionality reduction without losing information can be approached in an unsupervised or supervised manner. Unsupervised methods may include RNA Variational Autoencoder, SVD, PCA, KernelPCA, SparsePCA, DictionaryLearning, Isomap, NMF, UMAP, Feature Aggregation, Patient Correlated Clustering, KMeans, Gaussian Mixture, or Spherical KMeans. Performing dimensionality reduction in a supervised manner may include linear discriminant analysis, neighborhood component analysis, MLP transfer learning, or tree supervised embedding.

Machine learning models or artificial intelligence engines can be implemented via gradient boosting models, random forest models, neural networks (NN), regression models, naive Bayes models, or machine learning algorithms (MLA). An MLA or NN can be trained from a training data set such as multiple matrices with feature vectors for each patient or image and features. In an exemplary prediction profile, the training data set may include patient images, pathology, clinical, and/or molecular reports and details, such as curated from EHRs or genetic sequencing reports. MLA includes linear regression, logistic regression, decision trees, classification and regression trees, naive Bayes, supervised algorithms using nearest neighbor clustering (such as algorithms where features/classifications in the dataset are annotated), a priori, means clustering , principal component analysis, random forests, unsupervised algorithms using adaptive boosting (e.g. algorithms where the features/classifications in the dataset are not annotated), generative approaches (gaussian mixtures, multinomial mixtures, hidden markov models), sparse separation, graph-based approaches (mincut, harmonic functions, manifold regularization, etc.), heuristic approaches, or semi-supervised algorithms using support vector machines (e.g. algorithms whose features/classifications are annotated). NNs include conditional random fields, convolutional neural networks, attention-based neural networks, deep learning, long short-term memory networks, or other neural models, and the training dataset consists of multiple tumor samples, RNA expression data for each sample, and a pathology report covering imaging data for each sample. MLA and neural networks identify different approaches to machine learning, but the terms may be used interchangeably herein. Thus, unless otherwise noted, references to an MLA may include a corresponding NN, or references to an NN include a corresponding MLA and an artificial intelligence engine embodying one or more of each in a hierarchical architecture. obtain.

In some embodiments, the artificial intelligence engine may be a trained machine learning model. In some embodiments, a trained machine learning model may be generated by training a machine learning model on medical records having one or more LoTs. In some embodiments, the trained machine learning model may be a regression model. In some embodiments, the trained machine learning model may be a clustering model. In some embodiments, the trained machine learning model may be a reduced dimensionality model. In some embodiments, the trained machine learning model may be a classification model.

Training consists of providing the optimized dataset as a matrix of feature vectors for each patient, labeling these features as supervisory signals when they occur in patient records, and output classification, prediction, or and training the MLA to predict the labels. Artificial NNs are powerful computational models that show their strength in solving difficult problems for artificial intelligence. It has also been shown that they are universal approximators (a wide range of functions can be represented given suitable parameters). Some MLAs identify important features and identify coefficients, or weights, for them. The coefficients are multiplied by the frequency of occurrence of the feature to produce a score, and some classification can be predicted by MLA when the score of one or more features exceeds a threshold. Coefficient schemas may be combined with rule-based schemas to generate more complex predictions, such as predictions based on multiple features. For example, 10 significant features can be identified in different classifications. A list of coefficients may exist for important features and a rule set may exist for classification. A rule set may be based on feature occurrence counts, feature scaled weights, or other qualitative and quantitative evaluations of features encoded in logic known to those skilled in the art. In other MLAs, features may be organized in a binary tree structure. For example, the main feature that distinguishes most classes may exist as the root of a binary tree and as each subsequent branch in the tree, until a class can be given based on reaching the terminal node of the tree. For example, a binary tree may have a root node that tests the first feature. Occurrence or non-occurrence of this feature must be present (binary decision) and the logic may be to traverse branches that are true for the item being classified. Additional rules may be based on thresholds, ranges, or other qualitative and quantitative tests. Supervised methods are useful when the training dataset has a large number of known values or annotations, but due to the nature of EMR/EHR documents, many annotations may not be provided. Unsupervised methods are beneficial for binning/bucketing instances in datasets when exploring large amounts of unlabeled data. A single instance of the above model, or two or more such instances may be combined to form a model for the purposes of the model or engine described herein.

The line of treatment module 120 may include multiple modules: a record harmonization module 122 , an interval generation module 124 , and a line of treatment assignment module 126 . A record reconciliation module 122 reconciles the records contained in the EMR 112 and the digitally curated records 114 to remove duplicates and avoid similar records, as disclosed in more detail below with respect to FIG. combination, which may take additional harmonization steps. Interval generation module 124 may receive the reconciled records and generate and refine multiple treatment intervals based on the reconciled records, as disclosed in more detail below with respect to FIG. A treatment line assignment module 126 receives a plurality of treatment intervals, identifies potential treatment lines, and activates a trained artificial intelligence engine to identify and label one or more treatment lines to be administered to the patient. It can be applied to therapeutic lines of therapy.

A line of care (LoT) store 150 may receive LoTs generated from the LoT module 120 and store them for use in the system 100 . The LoT may be stored in a structured format for retrieval by, for example, a user interface such as a web forms-based interactive user interface, which in some embodiments may include web forms 160a-160n. A web form may be used to initiate or view an immediate LoT from module 120 or multiple analyzes, including initiating or coordinating patient cohorts from which module 120 may perform an analysis. It may support a GUI that can be displayed by the computer to the user of the computer system to perform the functions. Electronic reports 170a-170n may be generated and provided to users via a graphical user interface (GUI) 165. FIG. It should be appreciated that GUI 165 may be presented on a user device that is connected to content server/prediction engine 100 via a network.

Reports 170a-170n collect patient information from various physicians and healthcare providers (including laboratories), convert it into a standardized format, consolidate it, store it in a network-based storage device, and implement the present disclosure. After the report is generated according to the form, it can be provided to the user as part of a network-based patient management system that generates a message containing the electronic report. In this way, a user (e.g., a physician, oncologist, or any other healthcare provider, or patient) receives computer-generated predictions, analysis, and other reports related to the generated LoT.

In some embodiments, the electronic report is an indication to the physician for treating the patient using the next LoT or treatment that correlates with the predicted response based on the response of similar patients who have received similar treatments. May contain recommendations.

FIG. 2 illustrates a flowchart of a method for generating one or more lines of therapy from multiple digital records. At step 210, the system 200 may identify medical records for the patient from the plurality of digital records. These records may be recorded within the EMR/EHR or digitally curated from unstructured data. Medical records may be limited to those from a particular time period, including time periods beginning with the diagnosis of a medical condition such as cancer. Additional time periods may include specific yearly intervals when performing an analysis of LoT development over time.

At step 220, system 200 may create multiple treatment intervals having at least one drug in the time interval. These agents have been identified from patient records and further identified as relevant to diagnosis. Time intervals are associated with start and/or end dates, and dates can be assigned directly from patient records or learned from an artificial intelligence engine trained on similar medications, standard practice, or similar patient records. It can be imputed from additional supporting information, such as the pre-interval.

At step 230, the system 200 may associate medications with respective treatment intervals when administration of the medication falls within the time interval. For example, if there is a treatment interval from June 1 to June 23, each drug administered on June 15 can be associated with the treatment interval, but the drug administered on September 1 cannot be associated with a treatment interval as its administration does not fall within the time interval.

At step 240, system 200 may refine the time interval for each respective treatment interval. Time intervals may be refined by combining overlapping intervals, splitting intervals, or adjusting start and/or end dates according to one or more heuristics, rules, or trained models.

At step 250, system 200 may identify one or more potential treatment lines from multiple treatment intervals. Identification may be performed using trained models, exhaustive enumeration with likelihood-based ranking, or other methods disclosed herein.

At step 260, system 200 may label the potential treatment line with the highest maximum likelihood estimate as the treatment line. For example, given a ranked list of potential treatment lines, the treatment line with the highest ranking (highest confidence) may be selected as the treatment line.

FIG. 3 illustrates an exemplary record reconciliation module 300 that implements the sub-processes of record reconciliation 122 of FIG.

Record reconciliation may include one or more modules such as record whitelist and/or record blacklist module 322 , record association module 324 , record combination module 326 , and heuristic processing module 328 .

A recording whitelist and/or recording blacklist module 322 may include or exclude recordings based on their content, quality, reliability, or other rating metrics. For example, a record is included when its source can be identified as imaging results from the respective institution, or is excluded if the image resolution is below an acceptable threshold for readability, or if a record is classified according to gender, ethnicity, etc. , or patient inconsistencies such as inconsistencies in other invariant characteristics/demographics.

The record association module 324 may check records for redundancy and remove records that appear multiple times or contain the same information as other records. Associations with drugs, dosing start dates, or dosing end dates can be assigned or imputed.

A record combination module 326 may combine records that have sufficiently overlapping associations. Sufficiently overlapping associations may involve the same drug or similar drugs (generic versus brand name, competing brand name, known equivalent, or other analog, etc.). Multiple records with the same start date for similar medications may be "rolled up" into a single record.

The heuristic processing module 328 may incorporate rule-based or artificial intelligence driven record reconciliation structures. In one example, the artificial intelligence engine is trained on drug names to identify similar drugs, or trained on dosing intervals to predict the duration of the most commonly prescribed drugs for patients for each drug type. may be The model may additionally include a set of rules for applying set thresholds to the allocation of start dates, end dates, or for enforcing minimum or maximum time periods for each recording.

Additional record reconciliation steps are disclosed below with respect to data capture and reconciliation.

FIG. 4 illustrates an exemplary interval generation module 400 that implements the sub-process of interval generation 124 of FIG.

Interval generation may include one or more modules such as diagnostic relevance filter 422, interval date assignment 424, interval date imputation 426, interval date extension 428, or interval date segmentation 430.

Diagnosis relevance filter 422 may incorporate a knowledge database containing agents known to treat diagnoses. These agents may be curated from FDA approvals, clinical trials, publications, or artificial intelligence analysis of patient records. Agents not relevant to the treatment of diagnosis may be excluded from treatment for LoT as they are unlikely to be administered for the purpose of treating the diagnosis.

Interval date assignment 424 may include assigning an interval to each medication record that has not been deleted from one of the preceding modules in record curation, reconciliation, or relevance filtering. A medical record containing a drug and at least a start date can be assigned to an interval with no drug or with the same or similar drug.

Interval date imputation 426 may include imputing a start date or end date for an interval to reflect an accurate estimate of drug administration for that interval. For example, if medication is expected to continue for a set number of days (a course of antibiotics, steroids, or other drugs with a known periodicity), the end dates are 7 days and 14 days after each drug. , can be imputed for intervals assigned based on expected duration, such as after 3 months.

Interval date extension 428 identifies when successive intervals overlap and extends the duration of the first overlapping interval to encompass other overlapping intervals when the drugs are the same or similar. and In addition, when medications are assigned or imputed for an interval, the start or end date of the respective interval indicates the date of additional medication when the extension falls within the threshold time period. can be extended to include In one example, the threshold time period may include a set duration, such as 22 days, or a time period for each medication based on the interval dosing, similar to record combination 326 above.

Interval date segmentation 430 may include identifying when an interval priority event occurs in one or more intervals and dividing each respective interval by the date of the priority event. For example, in cancer diagnostics, the occurrence of metastasis may be referred to as an interval-priority event, and any interval that extends beyond the date of metastasis will have the date of metastasis as the original interval start and end date. 1 interval and a second interval with the date of transition as the interval start date and the original interval end date. Priority events include events that trigger a change in treatment line and are identified through artificial intelligence analysis of patient records or through rulesets or through those events as they may be curated from FDA approvals, clinical trials, publications. can be

FIG. 5 illustrates an exemplary line of treatment assignment module 500 for implementing the sub-processes of line of treatment assignment 126 of FIG.

Treatment line assignment may include one or more modules such as LoT assignment estimation 522 , LoT assignment probability 524 , LoT ranking 526 , or LoT labeling 528 .

LoT assignment estimation 522 may include performing an integral combination evaluation of the available intervals so that the intervals may be considered in part and as a whole with respect to each other to identify potential treatment lines. . In another embodiment, the artificial intelligence engine may receive multiple treatment intervals and output multiple potential treatment lines.

The LoT assignment probability 524 is the likelihood that each potential line of treatment will be included as a sequentially numbered line of treatment (first line of treatment, second line of treatment, third line of treatment, etc.). calculating the expected maximum probability for each potential treatment line to identify .

The LoT ranking 526 is multiple based on their likelihood such that the potential treatment line with the highest probability is at the top of the ranking and the potential treatment line with the lowest probability is at the bottom of the ranking. may include ranking potential treatment lines for

LoT labeling 528 identifies the most likely potential line of treatment as that line of treatment, or evaluates consecutive potential lines of treatment, occurring chronologically after each preceding line of treatment, Consecutive, incrementally numbered treatment lines with the highest maximum likelihood estimates for each of the potential treatment lines in each consecutive time interval may be identified. For example, if a first potential line of treatment starts in January and a second potential line of treatment starts in May, the first potential line of treatment is labeled as the first line of treatment, A second potential line of treatment may be labeled as the second line of treatment when both lines of treatment are the highest ranking potential lines of treatment in the pre-May and post-May intervals, respectively. Additionally, the artificial intelligence engine may be trained to identify and label lines of treatment and/or iteratively consecutive lines of treatment.

The modules of FIGS. 3-5 may be configured to perform one or more of the following steps or processes.

In one example, a set of rules can guide the process of labeling one or more LoTs from patient records. Rules may include one or more "hard" or "soft" rules.

Hard Rules It may be common for patients to be prescribed leuprolide for life after initial diagnosis of prostate cancer. Therefore, after leuprolide appears in the EMR, it can always remain part of the LoT, even if it changes. Another exemplary hard rule may include some intervening event that requires modification of LoT, such as treatment discontinuation, metastasis, or progressive disease outcome. The opposite can also be encoded as a hard rule. For example, patients documenting a dosing change to abemaciclib generally only present with breast cancer metastasis. Even if the EMR is incomplete and does not describe that the patient's cancer has spread by metastasis, this is imputed and added to the EMR, as well as segmenting the respective treatment interval, latent LoT, or post-detection LoT. can become Similarly, progression from one class of drug to another class of drug may also be implemented as a hard rule, enforcing each treatment interval, potential LoT, or segmentation of the LoT.

Soft Rules When rules are formed, but can be applied conditionally, they can be encoded as soft rules. In one example, oncologists may each define LoT differently, i.e., consider part carboplatin and cisplatin together as LoT, but not the subsequent maintenance pembrolizumab as a core part of LoT. Note that some may consider cisplatin and carboplatin together with maintenance pembrolizumab as one LoT. Some people consider deviation to be a new LoT, even if it's to avoid side effects, while others don't. These types of preferences can be generated as soft rules that give additional weight to LoT, but do not require new LoT to be generated when viewed in EMR. If two drugs are similar, such as having the same active ingredient or achieving the same effect with different active ingredients, the change from one to the other may be weighted to determine whether there is a LoT change. . Additionally, it is noted that the application of one soft rule is preferred over the application of another, and is applied on a per-physician or per-institution basis, whereby a particular soft rule is enforced by the use of that soft rule. may be applied conditionally based on the patient under the care of the particular physician or institution that has chosen to do so.

Algorithm Methodology and Example Output In one approach, EMR and curated progression reports are generated based on all diagnoses, agents, significant events, and any LoT combinations based on combinations of agents, significant events, and other combinations of agents. can be analyzed for other features that may be relevant to identifying the . MLA works in two main steps. The first step consists of synthesizing and harmonizing disparate data sources, including medications, outcomes, and diagnoses across EHRs and curated progress notes, to create unique intervals of patient care. The second step considers all possible combinations of these intervals and assigns the LoT accordingly. In the lexical example where the medication history is represented as the string THECATSAT, each letter may represent a unique drug combination after digesting by the first step. After training across patient data, the goal of the second step is to consider heuristics to recognize common dosing patterns and separate them into 'THE' + 'CAT' + 'SAT'. . Examples based on patient EMR are described below with respect to FIGS. 8-13F.

Model Training Model training consists of defining a unique treatment interval (a "letter" from the example above, such as "T" or "H") for each patient within a large training cohort, all of the aggregates Considering combinations (T+HEC+AT vs. THE+CAT) iteratively, assigning these aggregates, calculating the frequency of these resulting aggregates (words), until the termination condition is reached. It may consist of reassigning aggregates. These combinations of aggregates are enumerated using an integer join approach. In a simplified example with such five drug LoT representations, we divide them into 16 different possible LoT groupings using an integer combination approach.

The 16 syntheses of 5 are as follows.
● 5
● 4+1
● 3 + 2
● 3 + 1 + 1
● 2 + 3
● 2 + 2 + 1
● 2 + 1 + 2
● 2 + 1 + 1 + 1
● 1 + 4
● 1 + 3 + 1
● 1 + 2 + 2
● 1 + 2 + 1 + 1
● 1 + 1 + 3
● 1 + 1 + 2 + 1
● 1 + 1 + 1 + 2
● 1 + 1 + 1 + 1 + 1.

This frequency analysis can be combined with hard/soft rules along with diagnostic information, clinical information, and significant events to provide global probability estimates for each combination. This most likely combination (THE+CAT+SAT) is then output as the assigned LoT.

One goal, after studying a large corpus of text, is to identify a cut point for this string that reads THE|CAT|SAT. Such a problem would be simpler if each letter of the alphabet were mapped to some sub-hierarchy such as consonant (C) and vowel (V), the above would be T→C, H→V, E→C, C →C, A→V, T→C, S→C, A→V, T→C. In this case it is much easier to learn the separator CVC|CVC|CVC. To this end, the Anatomical Therapeutic Chemistry (ATC) classification system is a drug classification system that classifies the active ingredients of drugs according to the organ or system they act on and their therapeutic, pharmacological, and chemical properties. It is administered by the World Health Organization's Collaborating Center for Pharmacostatistics and Methodology (WHOCC) and was first published in 1976. This drug coding system divides drugs into different groups according to the organ or system they act on, their therapeutic intent or nature, and the drug's chemical properties. Different brands share the same code if they have the same active substance and directions. Each sublevel ATC code represents a substance, or combination of substances, that is used as a drug in a single indication (or use). This means that one drug can have multiple codes, for example, acetylsalicylic acid (aspirin) has A01AD05 (WHO) as a drug for oral topical therapy, B01AC06 (WHO) as a platelet inhibitor, N02BA01 ( WHO) as an analgesic and antipyretic, and one code can represent multiple active ingredients, for example, C09BB04 (WHO) is a combination of perindopril and amlodipine, the two active ingredients alone They have their own codes when prescribed (C09AA04 (WHO) and C08CA01 (WHO) respectively). The ATC classification system is strictly hierarchical, ie each code has exactly one parent code, except for the top 14 codes which have no parent. These codes are semantic identifiers, ie they represent the complete lineage of the parent as such. The ATC hierarchy provides a third level hierarchy and a fourth level hierarchy for a given drug. An improvement in defining LoT in these simpler representations could be realized if each drug could be remapped to this hierarchy, especially in contexts with smaller datasets or very high drug cardinality. A notable specific case could be hormone maintenance therapy and other therapies that are very common in one cancer but difficult to learn using conventional MLA techniques.

For example, in the Breast Cancer LoT, a cohort may have approximately 55 unique antineoplastic agents associated with breast cancer. Grouping these drugs into their underlying ATC hierarchies, either through levels 3 or 4, results in a significant increase in this drug space of 20 under ATC level 4 and 9 under ATC level 3. A decrease occurs. The number of identified LoT states (classifications of LoT from the entire breast cancer patient set) is 1117 using drug name only, 656 under ATC level 4 and 289 under ATC level 3. Roughly speaking, adopting increasingly non-specific drug groupings reduces the number of unique 'drugs' by a factor of two, and correspondingly, the number of drug states in the output model is roughly halved. Decrease.

The equivalence provided by the ATC hierarchy was implemented primarily to solve the "maintenance" problem observed in breast cancer. Patients are often seen with first-line chemotherapy (doxycycline and cyclophosphamide) followed by tamoxifen or anastrozole (hormonal therapy), all of which are considered one LoT. However, pure frequency-based approaches to discerning LoT from baseline patient data almost always refer to chemotherapy and hormone therapy as separate LoTs because hormone therapy is ubiquitous. , half of all patient medications are on hormonal therapy (i.e., if the patient is taking at least one drug, half the time is on hormonal therapy), and second, chemotherapy as the first choice. Patients who receive it transition to hormone therapy within 180 days, half the time. Returning to our verbal example (THECATSAT), this is like saying that one sees E almost all the time, TH at a regular pace, and THE about half the time. . In pure frequency, TH|E is the result. This is partly because prob(TH)*(E)>prob(THE) due to the ubiquity of E (prob(E) is high). In fact, this is because we observe hundreds of variations that are followed by an E or simply an E. In this case, one would consider E to be only a single letter.

The maintenance problem is therefore a consequence of the core problem of the pure frequency model, when the occurrence of E is insignificant, and second, when E is commonly seen alone, and TH, in fact one It must be separated whenever there are several cases that can be a "word". Both of these may be encoded into a probability representation, where THE is a word if prob_1(TH)*prob_2(E)<prob_1(THE) and prob_1 is the first (1 ), the probability of observing the second (2), etc. The latter means that the "word-ness" of THE is prob(E|TH)>prob(X|TH)*prob(E|X), i.e., preceded by a given TH and the presence of E happens to be (or any other given agent, X), possibly modulated by the frequency of E.

Choosing which model to implement is a case of evaluating the performance of the underlying system. Lines of treatment are often the result of NCCN guidelines, in which oncologists select a "plan" at a given time point, change it after outcomes are observed, and/or make changes to treatment, with only minor changes. patients do not respond well. Therefore, the latter is a more appropriate way of expressing the probability space. The former expression implies prob_1(TH)*prob_2(E)!=prob_2(TH)*prob_3(E), implying that time ordering strictly varies probabilities. This is likely to be the case as well, but the more desirable features to capture are that seeing only E or TH is not important to predict whether THE is a word, but rather TH(X) and ( Z) is important for comparing E and predicting whether there is significant enrichment when X=E and Z=TH.

As explained above, the probabilistic model, in its raw form, has an imprecise mapping to the way in which progress events are usually imputed. At worst, it consistently imputes Progressive Events when it rarely actually sees Progressive Events. To further improve the agreement between models and outcomes, an enhanced model and corresponding model training are performed using an underlying probabilistic model as follows.
prob(A→B)=prob(A)*prob(B)*prog_w(A,B) vs prob(AB)*[1-prog_w(A,B)],
where prog_w(A,B) represents an estimate of the observed incidence of progression between medication state A and medication state B; The condition above initiates a LoT break is modified by the incidence of progress normally observed in A→B transitions, and the more common this condition becomes, the more common the situation is when both are one LoT. more likely compared to The model may implement prog_w(A,B) as the number of progressions observed in this transition over the total number of times this transition was observed. However, the majority of transitions are largely unseen, thus significantly biasing the model. A "Bayesian" estimate (ie, leaving the decision to the underlying frequency space of the model rather than the progressive incidence) is made with a trend toward 0.5. The larger the sample size, the more likely we want to use the observed estimate, the smaller the more likely we want to use 0.5. The classical way to implement this is additive smoothing, where prog_w could be:
prog_w(A→B)=(prog_f)±abs(prog_f-0.5)*(upper_95(prog_f)-lower_5(prog_f)).
where prog_f is the observed number of progressions on the A→B transition divided by the overall incidence, upper_95(prog_f), lower_5(prog_f) are the 95th and 5th progressions based on the Clopper Pearson method for prog_f. th binomial proportion CI estimate. As the sample size increases, the "centering" applied to prog_f decreases as the confidence interval approaches 0. As the sample size increases, the confidence interval diverges (up to 1.0), which means that the maximum shift is applied, prog_w tends towards 0.5.

ATC and enhanced models show common transitions and highest agreement. Most importantly, the model then showed the least significant discrepancy with the data (trastuzumab→trozole), with an incidence of 1/319 in the data comparable to 30 of 319 in the output, which Either a very small difference or an infusion of ˜29 additional events such as taxane→trozole. This is very common as a transition from an intensive, rapid starting therapy such as chemotherapy to maintenance therapy. An advantage of the instant LoT labeling artificial intelligence engine is that it allows nuances in the soft rules covering the three possible LoTs based on the request entity. The LoT can include initiation therapy, maintenance therapy, or a combination of initiation and maintenance therapy.

Thus, the enhanced model either removes spurious LoT changes related to maintenance therapy or reduces the occurrence of discrepancies between the model and observed data while maintaining the potential for higher-order interactions. Let

In another embodiment, an artificial intelligence model, such as an unsupervised model, can be trained on patient health information or other treatment information. Unsupervised models can be trained on information about medications or other therapies received by patients and progress, or information including treatment interval preference events. In some examples, a progressive event associated with model training is a progressive event that occurred after the first therapy was discontinued but before the second therapy was initiated.

In some embodiments, the treatment line identified by the model trained using patient health data is based on guidelines published by the National Comprehensive Cancer Network (NCCN), FDA guidelines, clinical trials, or other Follow established standard treatment guidelines, including published publications.

In some embodiments, a semi-supervised approach is implemented when multiple lines of therapy are curated from NCCN guidelines, FDA recommendations, clinical trials, drug fact sheets, or other publications, and then semi-supervised It can be used as a starting point to identify labels in patients for whom labels were generated that did not match the curated labels in the unsupervised portion of the ALI approach. Those skilled in the art will appreciate that the semi-supervised approach takes additional time during operation to set up and run, but provides a more reliable LoT given the starting criteria, including an accurate LoT designation. will recognize that

In some embodiments, treatments may be generated by generating a concatenated treatment from treatments with the same treatment identifier when the administration dates of two or more treatments are within a time period of each other. In some embodiments, the time period can be based at least in part on medical conditions. In some embodiments, the time period can be based at least in part on the treatment identifier. In some embodiments, the time period can be based at least in part on the medical condition and treatment identifier.

For any particular patient record, the lines of therapy identified from the record may align with the lines of therapy published in standard therapeutic guidelines. For example, in ovarian cancer, the standard of care guideline is maintenance for patients with platinum-sensitive disease who have received 2 or more lines of platinum-based therapy and who have had a complete or partial response to a recent recurrent line of therapy. Niraparib may be recommended as therapy. Analyzes such as those performed via web pages 160A-160N or GUI 170 may provide physicians or institutions with compliance statistics for guidelines from the NCCN, FDA, or other guideline-setting entity.

Some of the lines of therapy conform to standard treatment guidelines, such as those identified in the NCCN Guidelines. Others of the lines of treatment may not conform to standard treatment guidelines. In these cases, the LoT determination can be used for quality improvement purposes to identify data within the medical record that may be erroneous. In other cases, LoT determinations may identify patients who have participated in clinical trials, such as trials testing FDA-approved therapeutic combinations. In other examples, LoT determinations may identify new trends in clinical practice that are not yet reflected in current standard of care guidelines. For example, a line of treatment in ovarian cancer in patients undergoing a LoT chemotherapy regime followed by LoT maintenance targeted therapy (such as targeted inhibitors such as PARP inhibitors or EGFR inhibitors).

Another semi-supervised approach can force bucketing on the LoT timeline. For example, in 1980 the original LoT with 2 drugs may be the norm, but in 1993 those 2 drugs will be discontinued and replaced as LoT. A semi-supervised approach would identify the year in which new treatments were released based on incorporating guideline changes from public records or inclusion in clinical trials, and unsupervised so that development of LoT would be tracked over the years. LoT curation could be bounded to come between years of LoT changes. In one embodiment, those bounds can be set as a sliding window and automatically determined based on an optimized EM similar to the EM as shown in more detail with respect to FIG. Similar bucketing can be performed based on geographic criteria such as by physician, institution, state, region, country, or other common characteristic for cohorts of patients. Unsupervised learning excels in these analyzes due to the inherently variable nature of attempting to identify new windows based on time or location in order to derive meaningful LoT from existing patient records. . Analysis may be performed using windowing to identify areas, facilities, or years where a particular LoT would be most effective.

In some embodiments, the unsupervised approach allows fine-tuning the system according to different cancers and their medications. For example, treatment of prostate cancer involves oral medications taken at longer intervals than, for example, intravenous medications traditionally administered to treat lung cancer, which has short intervals. Restrictive structured models limit cancer subtypes and label nuances to only those categories of drugs that are enumerated by the programmer during training or are available from a system of curated data. It can be difficult to learn labels.

Figure 6 shows an artificial field with rule-based preprocessing that implements a method to impute complex curated fields from curated patient histories and apply them to non-curated patients or patients with small amounts of data. 6 is a flowchart illustrating system 600, one embodiment of preparing, training, and assigning lines of treatment (LoT) using an intelligence engine approach.

A patient's medication history is analyzed in clinical practice through a combination of machine learning of digital documents such as progress notes and electronic health records (EHRs) MRs112, and curated medical records (MRs)114 derived from a combination of medical expert annotations. Captured as record 110. Characteristics of a patient's medical record may be structured and transmitted to multiple modules within system 600 . The system 600 takes this redundant, repetitive, and sometimes inconsistent raw input and transforms it into a series of condensed records that capture the most prominent drugs used to treat a patient's cancer. Focused. The following steps are an exemplary method for performing this conversion.

In another embodiment, record reconciliation 122 may include drug filtering module 605 , drug combination module 610 , and drug reconciliation module 615 . Drug filtering 605 may include filtering clinical health record 110 drugs from curated records 114 and EHR 112 to drug cancer therapies, such as antineoplastic drugs. Filters may also be applied to other drug characteristics, such as to identify drugs associated with a line of therapy. Filtering drugs that are not typically part of a line of therapy reduces the computational load required to process the vast number of available patient records. For example, the Electronic Health Record (EHR) consists of all medications ordered and administered to a patient, varying from pain relievers, antibiotics, placebos, multivitamins, and so on. A filtering step 605 removes any drugs that have been identified as anti-neoplastic drugs (anti-cancer drugs) and not stored in the internal value set. This value set may be defined by the clinical team or curated from machine learning models as well as internal and external publications, clinical trials, FDA approvals, NCCN guidelines, or pharmaceutical reports.

A medication combination 610 may combine medication records (MRs) within a threshold number of days (d), such as 22 days. Thresholds may be set by drug, drug class, active ingredient, or learned from patient records via an artificial intelligence engine. EHR MRs are highly repetitive periodic records that occur for each drug order and administration. Many antineoplastic therapies are administered every 22d (referred to as one cycle), so by combining recordings within the 22d, significant recording compression can be achieved that captures the entire duration the patient is on drug. . Procedurally, one process for date imputation may be run for each record, which includes:
a) If MR has an annual start or end date, or no start date, go to reconciliation 615.
b) If MR has a month-only start or end date, impute on the 15th of that month.
c) If the MR has a missing end date, set the end date to the start date.
Separately for native and curated recordings (i.e. perform the following steps for all drugs in native recordings and also for all drugs in curated recordings), for each drug: do.
i) Sort each MR for a drug by start date.
ii) For each MR of the drug:
(1) Compare the end of a record with the beginning of a subsequent record. Combine records if the MRs are within 22d of each other. This can be accomplished by setting the end of the record to the end of the subsequent record and deleting the intervening MR for that drug.
(2) advance to the next record;

Reconciliation 615 may identify and correct redundant and sometimes conflicting information. “Harmony” refers to a set of machine learning algorithms, heuristics learned from internal and external publications and medical experts, removing these redundancies or inconsistencies, and combining the high temporal resolution of EHR MR112 with the curated MR114 expert Establish a consistent set of MRs that combine knowledge and Procedurally, one embodiment of reconciliation may involve the following.
a) Correct the poor temporal resolution of any curated EMR and adjust the next inset to meet the resolution of the MR to be accurate within 1 day, 1 week, or 1 month as needed for harmonization. impose a putation rule. The following approach assumes the maximum duration of these records, possibly allowing EHR MRs with higher resolution to clarify these dates.
i) For month-only starts, set the day to the 1st of the month. For year only, set month to January and day to 1.
ii) For month only ends, set the day to the 15th. For year only, set month to December and day to 31st.
b) If all records are native or curated and there is no need for harmonization, proceed to interval generation 124;
c) Sort the combined records by start date.
d) For each drug type in the patient record:
i) Create an empty "output list". (The last entry in the list is referenced as output[-1])
ii) For each record of this drug type:
(1) If this is the first entry, add it to the output.
(2) Append the record to the output if the current record occurs after output[-1]. Continue.
(3) If the record has a start or end date with higher resolution than output[-1] (such as month or year resolution, and the record has day resolution), replace the low resolution date with the high resolution date. replace with
(4) If the record appears within the output[-1] timeframe (such as a one-day record that appears within months of curated records), exclude this extraneous record from the output list.
e) Return the output list as a harmonized drug table 620 with each MR's drug classified and structured.

In another embodiment, interval generation 124 may include modules for primary cancer (PC) relevance filter 625 , date padding 630 , MR combination using PC curation model 635 , and interval generation 640 . A line of therapy is best described by a period of continuous medical care that describes the administration (such as, but not limited to, a "regimen") of one or more drugs. Here, these periods of continuous care are referred to as "intervals" or "treatment intervals." "Interval generation" is the process of converting harmonized MRs into these treatment/medication intervals (MI). They simply represent the duration that the patient has been on one set of drugs, and a new MI is initiated when there is a change, addition, or subtraction of antineoplastic drugs related to the patient's primary cancer type. do.

A PC relevance filter 625 may further filter the reconciled drug table by drugs associated with the patient's primary cancer (PC), diagnosis, or other identifiers to bias the line of treatment. The list is defined by machine learning models, internal and external publications, and medical experts. This removes drugs such as denosumab, which are considered supportive in some cancers (such as prostate cancer) but prominent in others (such as osteosarcoma). One embodiment of PC filtering may involve the following.
a) Filter patient MRs using the PC relevance filter.
b) Filter patients with significant uncertainty within the remaining MR. This includes:
i) year-only medication records;
ii) Ambiguous or generic drug names that exist from curated progress notes (such as “platinum compounds” or “antineoplastic agents”) rather than actual drug names.

Date padding 630 may identify and pad the date. A number of MRs consist only of a start date or an end date corresponding to a start date. A minimum duration is required to construct a dosing interval, so these records are set by an interval discrimination threshold (DATE_PAD, typically 21d, where DATE_PAD is set by the user based on the user's threshold selection). given the end date using a tunable hyperparameter that can be One embodiment may involve the following.
A) If the record has no end date or end date == start date, set end date = start date + DATE_PAD.

MR combinations using the PC curated model 635 may perform drug-specific rollups. While many antineoplastic agents are administered approximately every 21 d, some hormone therapy agents are administered monthly or less frequently. To account for this, drug-specific rollups (typically 22d-180d) can be learned from a combination of machine learning and medical expert knowledge for each drug. This process is described in more detail below with respect to the PC Curation Model section. One embodiment may involve the following.
a) A learned drug-specific rollup is applied to each drug record. For each drug and each record:
i) If the end of the previous record and the start of the next record are within this rollup, combine the records and stitch the two records together.

Interval generation 640 may include conversion from MR to dosing interval. A patient's medical history typically consists of multiple antineoplastic MRs that overlap in time. This conversion process produces a defined interval of homogeneous treatment for one or more drugs and creates a new dosing interval whenever an antineoplastic agent is added or subtracted from a patient's medication record. be started. One embodiment may involve the following.
a) Sort the resulting medication records by start date and for each record do the following:
i) Create a new interval if one of the following applies:
(1) first medication record;
(2) the medication record begins after the end of the last interval, or
(3) The medication record starts within the interval discrimination threshold (eg, 22 days) of the end of the last interval end, and either the start of the record or the end of the previous interval has monthly resolution.
ii) If a medication record overlaps the last interval, add this medication record to that interval.
iii) If an interval of at least the interval discrimination threshold cannot be constructed due to multiple overlapping records, then LoT cannot be determined for that patient and returns a failure status.
iv) Otherwise, the medication record will span multiple other intervals, thus adding this medication to any overlapping records. If recording continues after the end of the last interval, create a new interval for the remainder of the recording.
b) Output Medication Intervals 645, describing all records containing treatment intervals, start and end, and associated medications.

Those skilled in the art will appreciate that the above steps represent embodiments in which the present disclosure may work. Other systems and methods consistent with the disclosure herein are also available.

Line of treatment (LoT) assignment 126 may comprise one or more modules for refining intervals with outcomes 650, modeled LoT frequencies 655, estimated probabilities 660, LoT labeling 665, or post-processing LoT 670. A patient's line of treatment captures the therapeutic strategies employed by an oncologist to manage that person's cancer. Each line is one or more of the planned antineoplastic agents, and a new set of agents or A new "line of treatment" is proposed. Due to incomplete response data (often present in follow-up reports alone), these unplanned events are often missed in patient MR. The main goal is to learn common treatment patterns and apply these when response data are scarce to generate computationally derived LoT assignments. Here, the intervals generated from the LoT assignments 126 are annotated with response data from the patient's clinical health record 110, when present, to produce a refined interval list. Using a combination of integers (COI) approach, the most probable combination of these interval sets is determined, taking into account the patient's incrementally quantified LoT. In some embodiments, one or more LoTs may be annotated. In some embodiments, one or more LoTs may not be annotated. In some embodiments, treatments can be annotated. In some embodiments, treatments may not be annotated.

Interval refinement 650 by outcome may identify some “treatment interval priority events” or patient events, such as outcomes, that are absolute indicators of change in LoT. These are added to the dosing intervals to generate a refined interval list consisting of one or more partitioned sets of intervals. One such process for completion may include the following.
a) Collect all patient outcomes and filter to LoT relevant outcomes and interventions. These may be cancer-type specific, such as castration-resistant prostate cancer (CRPC) diagnosis, or generally metastatic diagnosis or progressive disease outcome. Commonly excluded outcomes include complete and partial responses (indicating that the LoT was successful and should be continued).
b) Repeat each outcome and patient interval and separate this patient interval list into two separate sets of intervals if the outcome occurs within the interval discrimination threshold of the start of a new interval. If the outcome is within the threshold of two intervals, separate by the interval closer in time and choose the latter.
c) Repeat the patient refinement interval list, splitting into additional sets if treatment line maximum duration threshold (eg, 180 days) separation occurs.

The modeled LoT frequencies 655 may enumerate a combination of integer divisions for each treatment interval or generate all sequential permutations of drugs that divide into potential treatment lines. In one example, this may include an artificial intelligence engine trained on patient records.

Estimated probability 660 may take a collection of potential LoTs and treatment intervals to generate respective likelihoods for each LoT. In some embodiments, 655 and 660 indicate that the artificial intelligence engine receives refined intervals based on outcome data, separates potential LoTs from the intervals, and determines the likelihood that each potential LoT is a LoT. It can be combined into a single step to produce probabilities to measure. In one example, the process may include the following.
a) After separating the refined interval list into sets given the outcome, repeat for each set and do the following (set LoT counter=1).
i) Compute all possible combinations of intervals (potential LoT) using an integer combination approach or an unsupervised artificial intelligence engine trained to identify LoT from patient data.
ii) Compute the probability of each of these interval connections using the frequencies learned during model training. The total probability of a given interval combination is the product of the individual combined intervals (for 2 intervals this is Prob(interval 1)*Prob(interval 2) and Prob(interval 1 followed by interval 2) be).
iii) Consider the combination of intervals with the highest probability as the LoT assignment. (If Prob(interval 1)*Prob(interval 2) > Prob(interval 1 followed by interval 2) then interval 1 is the first LoT and interval 2 is the second LoT.)

LoT labeling 665 identifies the most probable combination of the set of dosing intervals (the potential LoT with the highest likelihood for each successive time period), which can be considered the patient's LoT. In one example, the process may include the following.
i) Number each of these interval combinations using the LoT counter and increment each time. (In the case of two intervals, interval 1 is assigned LoT2 and interval 2 is assigned LoT3 if it was preceded by an interval assigned LoT1).

Post-processing LoT 670 may include identifying LoTs that can be remedied or modified. For example, in rare cases, consecutive combined intervals may have the same set of drugs, but were separated by a period of time exceeding the combination threshold for the drug or interval, but had no treatment interval precedence event. not, and therefore can be considered to be the same LoT. Any LoT that falls into a pattern that can be corrected or adjusted in post-processing may be assigned to the same LoT, and subsequent LoTs may be adjusted incrementally. In one example, this may include:
a) Combine all most probable interval joins into a final LoT list.
b) Iterate through each of the interval joins and do the following:
i) If two consecutive LoTs have the same drug, assign them the same LoT (e.g. [doxorubicin, cyclophosphamide and doxorubicin]|[cyclophosphamide and doxorubicin, cyclophosphamide and doxorubicin and paclitaxel) , anastrozole] produces the final output list [doxorubicin, cyclophosphamide and doxorubicin, cyclophosphamide and doxorubicin, cyclophosphamide and doxorubicin and paclitaxel, anastrozole], which produces the final output list [doxorubicin, cyclophosphamide famid and doxorubicin, cyclophosphamide and doxorubicin and paclitaxel, anastrozole], where '|' bars indicate outcomes, ',' commas indicate new LoTs, grouped by 'and' drugs in combination).

In some embodiments, an output LoT table, such as a LoT table stored in LoT store 150, may be in a structured format, such as a json file, xml file, or other format, and is structured as follows: It can have a subset of data fields.
● patient_id: A unique patient_id across tables.
● interval_start: start of dosing interval.
● interval_end: End of dosing interval.
● medication: Relevant medication (maximum one per line).
● lot: LoT allocated. (1-n, integer).
● complete_lot_start: Start of a comprehensive LoT, spanning one or more intervals.
● complete_lot_end: Comprehensive LoT end, spanning one or more intervals.
- complete_lot_medications: Lists the associated medications in a given LoT, concatenated with a '+' sign.
● emr_derived: 1 if the entire LoT is derived from a medication record that exists only in the EHR, 0 otherwise.
● success: Whether the LoT was successfully defined for the patient. This is unsuccessful when patients have only the required years of medication for LoT or have medication records resulting in treatment intervals.

Once generated, the LoT table may be stored in the LoT store 150 or combined with structured data of the patient's EHR for subsequent use in analytical models.

Primary Cancer (PC) Curation Model Algorithm As explained above, different agents have different cycles that may vary depending on the primary cancer type. The next step can be used to estimate this cycle time of primary cancer types to specific drugs of interest by leveraging the medical knowledge that exists in curated MR. Capture the typical duration of drug administration as described by the curated MR and use this to cycle that drug to repeat this typical duration when applied to the EHR MR It is by suggesting time. This has the effect of converting a highly repetitive EHR dosing record into a curated MR duration 'look' that an abstractor can curate. Without loss of generality, this process is described below for a single agent against a single primary cancer type, but is extended to multiple cancer types and multiple agents without detracting from the present disclosure. obtain.
1. Input: curated MRs and EHR MRs for a given drug against a given primary cancer type.
2. Filter curated MRs with the following characteristics:
a. an end date that is different from the start date;
b. Start and end dates with at least monthly resolution.
3. Calculate the duration of drug administration across these curated MRs, defined as MR end date to MR start date.
4. Calculate the median duration of drug administration across the curated MRs.
5. Starting with the minimum interval threshold (typically 22d), do: a.
a. Combine the EHR records within this threshold for each patient to generate the duration of drug administration described by (optionally) one or more records.
b. Calculate the duration of drug administration (end to start) for each of these records.
c. Compute the median duration across all of these recordings.
d. If this median is equal to the curated MR median (calculated in 4), then return this interval threshold as the "cycle time"; otherwise, increment the interval threshold.
e. If the increment threshold is greater than the maximum interval threshold (typically set to 180d), exit the loop.

The above approach computes the kernel density approximation of the curated duration distribution, rather than the median, and minimizes the distance metric (Euclidean distance, geometric distance, or Kullback-Leibler divergence) between the two distributions. It can be generalized as returning a threshold.

FIG. 7 is a flowchart illustrating one embodiment of preparing, training, and generating a line of care (LoT) model over three stages 710, 720, and 730. FIG.

In the model data preparation phase 710, sources of patient information related to diagnosis, prognosis, treatment, and outcome are aggregated across sources in curated clinical health records 110, including EMR/EHR records. Analysis from these records produces cancer-specific clinical insights 705, patient medications 714, and curated progression and treatment deviations 712. For example, if a patient is diagnosed with prostate cancer, prostate cancer clinical insights (types of drugs, treatments, or therapies that are directly linked to treating prostate cancer, effects of specific drugs, treatments, or therapies on prostate cancer, prostate cancer list of drugs, treatments, or therapy names that link to codes associated with the LoT in the element At 705 and 712, it may be retrieved from internal or external sources such as one or more databases or publications. Drugs curated from documents such as follow-up notes, lab results, or other handwritten documents are then to include only drugs associated with diagnosed cancers for further processing, such as drugs that have an effect on treatment. can be filtered to For example, unrelated drugs such as amphetamines for headaches, antihistamines for allergies, serotonin receptor blockers for depression, or drugs the patient is taking for purposes other than treating the diagnosed cancer , can be filtered from the LoT prediction. Filtering to LoT-related drugs by cancer type in module 716 filters depression drugs from cancer diagnoses such that modeled dates and intervals 718 are calculated only from diagnosis-related treatments for each patient. can be run as Modeled dates and intervals 718 may include corresponding start and end dates, which may be generated for each of the drugs remaining in the filtered LoT drug list, such that initial timeline intervals are generated. may be A collection of modeled dates and intervals 718, for example, all cancers, organ-specific cancers (breast, lung, bone, blood, liver, prostate, etc.), or even non-small cell lung cancer, ductal non-invasive Curated for all patients in the dataset with an associated diagnosis of a specific diagnosis such as cancer (DCIS), invasive ductal carcinoma (IDC), triple-negative breast cancer, or other diagnosis.

In a training phase 720 , additional cancer-specific clinical insights are retrieved in element 705 and the dosing intervals calculated in steps 710 to 718 are received in drug measurement interval frequency (w) 722 . An additional cancer-specific clinical insight is the number of days that constitute an automatic LoT break (e.g. for rapidly progressing cancers this could be 90 days and for slower paced cancers this could be 180 days). and even progression events that can establish a LoT break for each cancer type, such as CRPC diagnosis for prostate cancer). At element 722, treatment/dosage intervals can be compared across all patient data to identify frequencies of occurrence. For example, in patients with prostate cancer, the treatment/dosage interval can also be refined according to a rolling window with width corresponding to days (w). This process can be run over one or more window sizes (w=30 days, 90 days, 120 days, etc.) to identify the most representative set of LoTs in the patient population. For each dosing interval identified in the patient population, an expectation maximization (EM) can be calculated to identify the reliability of the interval. These intervals may be assigned in LoT partition assignment 724 after looping through the EM of frequencies. After the optimal window size is determined, the dosing frequency can be refined at dosing frequency refinement 726 . Refinement may involve looping the EM of frequencies one more time until convergence with a specified window size (w) is achieved. The set of treatment intervals with the highest expected value can be selected as the best representative set of LoT (Potential LoT). After all patients have been processed to identify a base set of LoT, stage 130 may output LoT frequencies estimated from modeled LoT frequencies 655 .

At the model LoT allocation stage 730, the collection of estimated LoT frequencies 655 across all patients can be compared to each new patient's dosing interval. In the LoT generation stage 732, treatment intervals designated as potential LoTs may then be assigned to first, second, third, etc. LoT partitions corresponding to each patient's ranking of potential LoTs. Each potential LoT assignment 732 , listed according to the integer combining method, may be ranked by the corresponding popularity of the most likely LoT labeled as the patient's LoT in the LoT and LoT labeling 734 .

Exemplary applications of LoT labeling are described below with respect to FIGS. 8-13. Exemplary embodiments may be generated and labeled according to any of the methods and systems disclosed herein, including the systems and methods of FIGS. 1-7. The following examples are provided to illustrate how the different methods and approaches herein can be applied to different examples of patient records.

8-10 illustrate patient medical records, calculated dosing intervals, and predicted lines of treatment (LoT) from a combined EMR and curated record system, according to one embodiment. Patient record type and source may be considered when extracting medications, outcomes, treatment interval preferences, and other LoT features. These illustrations highlight the interaction between EMR and curated recordings for identifying one or more treatment intervals and potential LoTs within the recording.

FIG. 8 is an illustration of LoT prediction for a first patient diagnosed with metastatic non-small cell lung cancer. The first patient was diagnosed with NSCLC in August 2016 at primary diagnosis 802. Throughout the next year in the course of treatment, the oncologist treating the first patient maintains several follow-up notes (referred to above as "follow-up notes") describing the patient, the first patient's medications, and related outcomes. ) was created. This data is complemented by electronic medical health records (EHRs) of medication administration 804 .

A medical professional (curator) reviewed each of these transcripts and recorded the indicated medications and associated outcomes 806 from each notebook. Next, the contents of the progress record will be explained.
Follow-up note 1 (PN1): The curator documented the date of the patient's primary diagnosis and first dose of triplet chemotherapy 808 of pemetrexed, bevacizumab, and carboplatin in August 2016.
PN2: The curator recommended completion of triplet therapy 808 for PN1 in October 2016, partial response to related therapy (not shown, October 2016), and initiation of pemetrexed/bevacizumab maintenance therapy 810 in November 2016 Recorded.
PN3: In January 2017, the curator recorded progressive disease outcomes 812 to pemetrexed/bevacizumab maintenance therapy 810 and their associated terminations in December 2016. The oncologist also noted his intention to administer nivolumab 814 to the patient in February 2017.
PN4: The curator documented a progressive disease outcome 816 (meaning that this therapy was unsuccessful in treating the patient's cancer) for nivolumab 814 in February 2017. Initiation of gemcitabine therapy818 was also recorded beginning in March 2017.
PN5: Gemcitabine 818 termination due to toxicity in late March 2017 was documented, and paclitaxel 820 initiation was documented at the same time.
PN6: End of paclitaxel 820 was documented in June 2017 and an associated "complete response" to therapy (indicating that the patient was found to be cancer-free).
PN7: At the August 2017 follow-up visit, another "complete response" 822 was recorded, indicating that the patient remained cancer-free. These curated drugs were entered into the drug table and outcomes into the Outcomes SQL table for downstream analysis.

Inclusion and Reconciliation with EHR Next, the drug administration records present in the EHR from the hospital were added to the medication data table (all drug administrations 804 and event bars according to record reconciliation 122 in FIG. 1). Dexamethasone administration 824 was additionally recorded. These agents are not curated because they are supportive care agents that patients commonly receive in combination with chemotherapy to alleviate side effects. While displayed, dexamethasone was filtered and not considered in subsequent calculations because supportive care medications are not considered significant to LoT interval generation module 124 . The September and October 2016 EHR dosing records for pemetrexed, bevacizumab, and carboplatin 808 were added to the curated records for these agents, and the “native and curated ” generated a record. An EHR dose of paclitaxel 820 was used to augment a curated record in April-June 2017, generating another 'native and curated' record. The August paclitaxel booster (in 804a) was added to the patient's record in record reconciliation 122.

Presentation of Dosage Intervals The written medication records were then converted to unique medication intervals (dotted bars represented by 830a-830e). These intervals represent collections of drugs taken at the same time with defined start and end dates. Medication intervals are created whenever a medication change occurs (see interval generation module 124).
Interval 830a: The first dosing interval begins with triplet therapy pemetrexed/bevacizumab/carboplatin and ends with discontinuation of carboplatin in October 2016 (interval generation module 124).
Interval 830b: Consisting of pemetrexed/bevacizumab, this interval begins with discontinuation of carboplatin and continues until the end of this doublet therapy (interval generation module 124).
Interval 830c: This captures the February 2017 patient administration of nivolumab (interval generation module 124).
Interval 830d: This interval begins with gemcitabine administration in March 2017 and ends with its discontinuation and paclitaxel initiation (interval generation module 124).
Interval 830e: Curated, beginning with native paclitaxel recordings and continuing through July 2017 EHR recordings of paclitaxel administration. Although the patient received dexamethasone beginning in May 2017, this drug is not considered relevant for LoT assignment and therefore does not cause a sixth interval from May 2017 to July 2017 to be generated ( interval generation module 124).

LoT Allocation for Intervals A combination of probabilistic selection and heuristics is then applied to determine the LoT for the generated dosing intervals (1-5) (LoT allocation module 126).

Outcomes are considered to separate different intervals. The January 2017 outcome is used to separate interval 830b from interval 830c (LoT allocation module 126).
The late February 2017 outcome is used to separate intervals 830c and 830d (LoT allocation module 126).
Next, consider the intervals 830a-830b. In this case, a probabilistic selection is made considering the relative frequency of each of these intervals across the training population. The probability of seeing Interval 830a (representing triplet therapy) alone is 10% and the probability of seeing Interval 830b (pemetrexed/bevacizumab) alone is 5%, thus the combined probability is 0.5% (10%*5%) becomes. However, 2% see interval 830a followed by interval 830b. Since 2%>0.5%, the intervals are combined and defined as 1 LoT (LoT allocation module 126).
Since interval 830c is separated in outcomes by interval 830b and interval 830d, this single interval becomes LoT 2 (LoT allocation module 126).
Intervals 830d and 830e are now considered in terms of probabilistic selection. The probability of seeing intervals 830d and 830e alone is 7.5% (30% and 25% respectively, 30%*25%=7.5%), but the probability of seeing intervals 830d-830e across the dataset is 1%. Since 7.5%>1%, the intervals are considered isolated LoT, interval 830d is assigned to LoT3 and interval 830e is assigned to LoT4 (LoT assignment module 126).

FIG. 9 is an illustration of LoT prediction for a second patient diagnosed with ovarian cancer.
Patient demographics and clinical data sources.
PN1, March 2015: Primary diagnosis and surgery, initiation of carboplatin, bevacizumab, and paclitaxel.
PN2, September 2015: Start of bevacizumab, end of carboplatin, paclitaxel.
PN3, March 2016: Progressive disease outcome
PN4, June 2016: Progressive disease outcome, end of bevacizumab (month only), start of carboplatin, bevacizumab, gemcitabine.
PN5, October 2016: End of carboplatin, bevacizumab, gemcitabine.
EHR Source: Anastrozole, April 2017.

Interval Construction Interval 930a: Triplet therapy carboplatin/bevacizumab/paclitaxel.
Interval 930b: bevacizumab (separated from interval 930a by interval generation module 124).
Interval 930c: Carboplatin/Bevacizumab/Gemcitabine.
Interval 930d: Anastrozole (separated from interval 930a by interval generation module 124).

LoT Allocation:
The June 2016 outcome is used to inform the separation between intervals 930a-930b and 930c-930d (LoT allocation module 126). Intervals 930c and 930d are separated due to the separation between the end of the previous and the start of the next (LoT allocation module 126). The refined interval list then consists of [(interval 930a, interval 930b), (interval 930c), (interval 930d)].

For the first set, the probabilities of intervals 930a and 930b alone and in combination are compared. Since the combined interval has a higher probability, the set becomes (interval 930a+interval 930b) and is assigned LoT1 (LoT assignment module 126).
Interval 930c is assigned LoT2 and interval 930d is assigned LoT3 (LoT assignment module 126, only one possible combination).

FIG. 10 is an illustration of LoT prediction for a third patient diagnosed with breast cancer.
Patient demographics and clinical data sources.
PN1, January 2017: Primary diagnosis, breast, and initiation of docetaxel, trastuzumab, pertuzumab, and carboplatin.
PN2, July 2017: End of docetaxel, trastuzumab, pertuzumab, and carboplatin.
EMR, July 2017: Administration of Trastuzumab
EMR, December 2017-June 2018: Administration of capecitabine, several doses of trastuzumab, and two doses of tamoxifen.

Interval Construction Interval 1030a: Quadruplet therapy of docetaxel, trastuzumab, pertuzumab, and carboplatin.
Interval 1030b: Trastuzumab (separated by interval 1030a via interval generation module 124).
Interval 1030c: Trastuzumab and Capecitabine (separated from interval 1030b by new drug and interval generation module 124).
Interval 1030d: Trastuzumab (separated from interval 1030c by capecitabine instillation, interval generation module 124 sees continuation of trastuzumab).
Interval 1030e: Trastuzumab and Tamoxifen (separated from interval 1030d by the introduction of tamoxifen, interval generation module 124, continuing through interval generation module 124, tamoxifen adopts drug-specific rollup of 207d).

LoT Assignment: Since this record has no outcome, all possible combinations of the 5 intervals are considered probabilistically (in this case 2(5-1) or 16 possible combinations, LoT assignment module 126 ), the one with the highest probability is selected as the set of LoTs. In this case, 2+1+2 was chosen as the most likely interval bond.

Figures 11-13 illustrate treatment line labeling from a patient record according to another embodiment involving four steps.
1) Combining medical records that identify outcomes such as medications, cancer diagnoses, and interval priority events.
2) Define multiple dosing intervals at the start of each new therapy or medication in the patient record.
3) Applying an artificial intelligence engine trained unsupervised from multiple patient records and applying learned heuristics to the complete listing of LoT.
4) outputting LoT assignments according to the LoT identified from the artificial intelligence engine;

Figures 11-13 show the primary diagnosis at time 0 day (pDX), multiple medications/therapies occurring after the primary diagnosis at a time and having a duration of at least 1 day, and any interval such as surgery. Contains listings of priority events or outcome listings from patient records. Each has a pDX start and end date to indicate the interval between therapy, dosing, or other events.

11A-11D, an exemplary flow 1100 for line of treatment generation is shown. FIG. 11A illustrates the first step of flow 1100. FIG. A first step of flow 1100 combines one or more medications and/or treatments, one or more diagnoses (eg, cancer diagnosis), and one or more outcomes (eg, progressive disease outcome). can include In exemplary flow 1100, primary diagnosis may define the beginning of a timeline to which events and/or treatments may be mapped to provide one or more time intervals to a user (eg, an oncologist). can. As shown, one or more of the medications and/or treatments can include carboplatin, pemetrexed, nivolumab, and surgery. In some embodiments, flow 1100 as shown in FIG. 11A may be generated using steps 210 and 220 of FIG.

FIG. 11B illustrates the number of dosing intervals involved in the second step of flow 1100. FIG. Flow 1100 of FIG. 11B may be generated based on one or more medications and/or treatments of FIG. 11A. Some agents can be associated with a common interval for one or more new treatments. As shown, carboplatin and pemetrexed can be associated with a first dosing interval of approximately 70 days after primary diagnosis. A first dosing interval can define the start of a first new therapy. Other medications can be associated with a second dosing interval. As shown, nivolumab can be associated with a second dosing interval. A second dosing interval can define the start of a second new therapy. In some embodiments, flow 1100 as shown in FIG. 11B may be generated using step 230 of FIG.

FIG. 11C illustrates applying heuristics to refine one or more dosing intervals in the third step of flow 1100 . The heuristic divides one or more dosing intervals by combining overlapping intervals, dividing intervals, or adjusting start and/or end dates according to one or more heuristics, rules, or trained models. can be refined. For example, carboplatin and pemetrexed can be associated with separate dosing intervals, and flow 1100 can associate both carboplatin and pemetrexed with a first dosing interval. A heuristic can refine one or more dosing intervals based on rules associated with progressive disease outcome. For example, the heuristic may set the second dosing interval to begin about 130 days after diagnosis based on progressive disease outcome about 130 days after diagnosis. For continuation of the same drug (eg, nivolumab), the treatment line may be the same for all dosing intervals, such as a third interval associated with a second treatment using nivolumab. In some embodiments, flow 1100 as shown in FIG. 11C may be generated using step 240 of FIG.

11D illustrates outputting line assignments in the fourth step of flow 1100. FIG. A line assignment can be associated with one or more potential lines of therapy from one or more dosing intervals. Identification may be performed using trained models, exhaustive enumeration with likelihood-based ranking, or other methods disclosed herein. The lines of therapy include a first line (e.g., carboplatin and pemetrexed therapy) associated with a first dosing interval and a second line (e.g., nivolumab therapy) associated with a second dosing interval. can be done. In some embodiments, flow 1100 as shown in FIG. 11D may be generated using steps 250 and 260 of FIG.

12A-12F, an exemplary flow 1200 for line of treatment generation is shown. FIG. 12A illustrates the first step of flow 1200. FIG. A first step of flow 1200 can include combining one or more medications and/or treatments and one or more diagnoses (eg, cancer diagnosis). In exemplary flow 1200, primary diagnosis may define the beginning of a timeline to which events and/or treatments may be mapped to provide one or more time intervals to a user (eg, an oncologist). can. As shown, one or more medications and/or treatments can include doxorubicin, cyclophosphamide, paclitaxel, tamoxifen, and surgery. In some embodiments, flow 1200 as shown in FIG. 12A may be generated using steps 210 and 220 of FIG.

FIG. 12B illustrates the number of dosing intervals involved in the second step of flow 1200. FIG. Flow 1200 of FIG. 12B may be generated based on one or more medications and/or treatments of FIG. 12A. Some agents can be associated with a common interval for one or more new treatments. As shown, doxorubicin and cyclophosphamide can be associated with a first dosing interval of approximately 50 days after primary diagnosis. A first dosing interval can define the start of a first new therapy. Other medications can be associated with a second dosing interval or a third dosing interval. As shown, paclitaxel can be associated with a second dosing interval approximately 120 days after the primary diagnosis. A second dosing interval can define the start of a second new therapy. As shown, tamoxifen can be associated with a third dosing interval approximately 150 days after primary diagnosis. A third dosing interval can define the start of a third new therapy. In some embodiments, flow 1200 as shown in FIG. 12B may be generated using step 230 of FIG.

FIG. 12C illustrates applying heuristics to refine one or more dosing intervals in the third step of flow 1200 . The heuristic divides one or more dosing intervals by combining overlapping intervals, dividing intervals, or adjusting start and/or end dates according to one or more heuristics, rules, or trained models. can be refined. For example, doxorubicin and cyclophosphamide can be associated with separate dosing intervals, and flow 1200 can associate both doxorubicin and cyclophosphamide with a first dosing interval. A heuristic can refine one or more dosing intervals based on rules associated with hormone therapy. For example, a heuristic may set the second dosing interval to include a third therapy (eg, tamoxifen) because the third therapy is hormone therapy. In some embodiments, flow 1200 as shown in FIG. 12C may be generated using step 240 of FIG.

FIG. 12D illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of flow 1200. FIG. For each dosing interval, an expectation maximization model can be used to calculate the EM and identify the reliability of the dosing interval. In some embodiments, flow 1200 as shown in FIG. 12D may be generated using step 240 of FIG.

FIG. 12E illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of flow 1200. FIG. Specifically, flow 1200 applies an expectation maximization model to the first dosing interval transition "A" and the second dosing interval transition "B" such that the first dosing interval is equal to the second dosing interval. should be combined with or left separate to maximize estimation performance. The probability of observing one or more given intervals can be denoted as P(X). The probability of observing a progression event between two intervals can be denoted as P_prog(X→Y). The probability of seeing a progress event when dividing the interval into two lines (P(A)P(B)P_prog(A→B)) is the probability of seeing a progress event when keeping one line (P(AB )(1-P_prog(A→B))) and the more probable option can be selected as the line assignment. In the example shown in FIG. 12E, it was decided that the more probable option would be to split the interval into two lines, so each of interval "A" and interval "B" would have a corresponding medication line. have

FIG. 12F illustrates outputting line assignments in the fifth step of flow 1200. FIG. A line assignment can be associated with one or more potential lines of therapy from one or more dosing intervals. Identification may be performed using trained models, exhaustive enumeration with likelihood-based ranking, or other methods disclosed herein. The lines of therapy can include first lines associated with first, second, and third dosing intervals. In some embodiments, flow 1200 as shown in FIG. 12F may be generated using steps 250 and 260 of FIG.

13A-13F, an exemplary flow 1300 for line of treatment generation is shown. FIG. 13A illustrates the first step of flow 1300. FIG. A first step of flow 1300 can include combining one or more medications and/or treatments, one or more progressive disease outcomes, and one or more diagnoses (eg, cancer diagnosis). can. In exemplary flow 1300, primary diagnosis may define the beginning of a timeline to which events and/or treatments may be mapped to provide a user (eg, an oncologist) with one or more time intervals. can. As shown, one or more medications and/or treatments may include dexamethasone, lenalidomide, bortezomib, carfilzomib, pomalidomide, daratumumab, and stem cell transplantation. In some embodiments, flow 1300 as shown in FIG. 13A may be generated using steps 210 and 220 of FIG.

FIG. 13B illustrates the number of dosing intervals involved in the second step of flow 1300. FIG. Flow 1300 of FIG. 13B may be generated based on one or more medications and/or treatments of FIG. 13A. Some agents can be associated with a common interval for one or more new treatments. As shown, dexamethasone, lenalidomide, and bortezomib can be associated with a first dosing interval approximately 5 days after primary diagnosis. A first dosing interval can define the start of a first new therapy. Other drugs can be associated with additional dosing intervals. As shown, carfilzomib and pomalidomide can be associated with a second dosing interval approximately 180 days after primary diagnosis. A second dosing interval can define the start of a second new therapy. As shown, stem cell transplantation can be associated with a third dosing interval approximately 360 days after primary diagnosis. A third dosing interval can define the start of a third new therapy. As shown, lenalidomide may be associated with a fourth dosing interval approximately 540 days after primary diagnosis. A fourth dosing interval can define the initiation of a fourth new therapy. As shown, daratumumab and dexamethasone can be associated with a fifth dosing interval approximately 700 days after primary diagnosis. A fifth dosing interval can define the start of a fifth new therapy. As shown, pomalidomide can be associated with a sixth dosing interval approximately 750 days after primary diagnosis. A sixth dosing interval can define the start of a sixth new therapy. As shown, daratumumab can be associated with a seventh dosing interval approximately 930 days after primary diagnosis. A seventh dosing interval can define the start of a seventh new therapy. In some embodiments, flow 1300 as shown in FIG. 13B may be generated using step 230 of FIG.

FIG. 13C illustrates applying heuristics to refine one or more dosing intervals in the third step of flow 1300 . The heuristic divides one or more dosing intervals by combining overlapping intervals, dividing intervals, or adjusting start and/or end dates according to one or more heuristics, rules, or trained models. can be refined. A heuristic can refine one or more dosing intervals based on rules associated with progressive disease outcome. For example, the heuristic may set the fifth dosing interval to begin about 700 days after diagnosis based on progressive disease outcome about 700 days after diagnosis. In some embodiments, flow 1300 as shown in FIG. 13C may be generated using step 240 of FIG.

FIG. 13D illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of flow 1300. FIG. For each dosing interval, an expectation maximization model can be used to calculate the EM and identify the reliability of the dosing interval. In some embodiments, flow 1300 as shown in FIG. 13D may be generated using step 240 of FIG.

FIG. 13E illustrates applying an expectation maximization model to refine one or more dosing intervals in the fourth step of flow 1300. FIG. Specifically, flow 1300 applies an expectation maximization model to the first dosing interval transition 'C', the second dosing interval transition 'D', and the third dosing interval transition 'E' to Determining whether dexamethasone, pomalidomide, and daratumumab should be combined in a single interval or kept separate can include maximizing estimated performance. The probabilities of new lines at transitions C-D and D-E can be denoted as P(C)P(D)P(E)P_prog(C→D)P_prog(D→E). The probability of a new line at transition C-D can be written as P(CD)P(E)(1-P_prog(C→D))P_prog(D→E). The probability of a new line at transition D-E can be written as P(C)P(DE)P_prog(C→D)(1−P_prog(D→E)). The probability that transitions C, D, and E are on one line can be written as P(CDE)(1-P_prog (C→D))(1-P_prog (D→E)). In the example shown in Figure 13E, it was determined that C, D, and E were most likely to be on one line.

FIG. 13F illustrates outputting line assignments in the fifth step of flow 1300. FIG. A line assignment can be associated with one or more potential lines of therapy from one or more dosing intervals. Identification may be performed using trained models, exhaustive enumeration with likelihood-based ranking, or other methods disclosed herein. The lines of therapy are a first line associated with the first dosing interval, a second line associated with the second, third, and fourth dosing intervals, and a fifth, sixth, and fourth dosing interval. A third line associated with 7 dosing intervals can be included. In some embodiments, flow 1300 as shown in FIG. 13F may be generated using steps 250 and 260 of FIG.

FIG. 14 is a Sankey diagram 1400 illustrating lines of therapy as applied to patients having at least an immuno-oncology (IO) regimen.

For the patient record in this example, the first line of therapy is IO monotherapy such as atezolizumab, nivolumab, or pembrolizumab, combination IO and IO such as ipilimumab and nivolumab, or combination IO monotherapy such as carboplatin, pembrolizumab, pemetrexed and chemotherapy, carboplatin, paclitaxel, pembrolizumab, carboplatin, protein-bound paclitaxel, pembrolizumab, carboplatin, pembrolizumab, pemetrexed, or any other combination listed. A second line of treatment includes additional combinations of IO therapy, hormone therapy, or maintenance therapy. The third line of therapy includes docetaxel, ramucirumab, vinorelbine, and gemcitabine. And the fourth line of therapy includes docetaxel and gemcitabine.

After the dataset is labeled according to which line of therapy each patient received, additional analysis can be performed.

FIG. 15 is a progression-free survival (PFS) graph 1500 depicting survival curves for patients receiving different first lines of therapy.

Cohorts of patients can be generated based on a common diagnosis and one or more lines of therapy. Cohorts can also be filtered based on any of the features contained in the patient record. Exemplary lines of therapy 1510, 1520, and 1530. A physician may access a web portal, such as web portals 160A-160N of FIG. 1, containing collections of features found from patient records containing LoT labeled according to any of the systems and methods disclosed herein. Cohort selection may be used to populate the PFS graph to compare patient survival over time for cohorts of patients with the identified characteristics. Here, patients receiving the line of treatment associated with PFS curve 1510 responded better to treatment than patients receiving the line of treatment associated with PFS curve 1520, and both cohorts were positive for patients receiving PFS curve 1530. It can be empirically observed to respond better than to therapy receiving regimens of the associated line of treatment. Based on this information and the cohort population, physicians are more likely to avoid prescribing LoT associated with curve 1530 and more likely to recommend prescribing LoT associated with curve 1510.

An example of cohort selection based on characteristics found in patient records, via an online web portal, is US patent application Ser. 732,168 entitled "A Method And Process For Predicting And Analyzing Patient Cohort Response, Progression And Survival".

Referring to the exemplary online web portals of US patent application Ser. No. 16/732,168, cohort selection in FIGS. 1-9, analyzes in FIGS. A book) implementation may, for example, extract features from the clinical health record 110 of FIG. Selected and referred to in its entirety as the cohort selection to receive. A customizable notebook suitable for displaying customized analyzes to the user includes PFS, outliers, surgical efficacy, patient molecular characteristics on LoT performance, adverse event analysis, effect from duration of LoT. , or other cohort analysis, etc., which may include custom target generation, which may be based on labeled LoT. The exemplary notebook is a programming approach that allows designing custom interfaces with R, SQL, or other database software backends for analysis. The custom interface may include identifying off-label uses of drugs and treatments that may be particularly effective. The notebook may identify and present the informatics for initiating and establishing investigations into new clinical trials to test the efficacy of the approved off-label use of the treatment. In one example, in the notebook, each LoT has a generic drug name, a brand name, and a therapeutic rollup class (therapeutic type [target, initiation, maintenance, chemotherapy, radiotherapy, tyrosine kinase inhibitor (TKI), etc.) ], the dosage of the drug within the LoT, and/or the method of administration of the treatment, such as oral, intravenous, radiation exposure, etc., may be identified.

FIG. 16 is an illustration of an edit line of treatment tool 1600 for user editing of a line of treatment for a patient according to one embodiment.

The edit line of care tool may receive any patient from the set of patients who have lines of care identified according to the methods and systems herein. After the patient is loaded, a timeline similar to that of Figures 11-13 is loaded. In another embodiment, timelines similar to those of FIGS. 8-10 may be loaded as well. The user can use the cursor to perform some manipulations that correct perceived deficiencies in the LoT labeling performed by the unsupervised artificial intelligence engine.

For example, the user may drag and drop 1610 any treatment line designation to recalibrate the new LoT based on the new location of the listed LoT. The user may edit the drug name 1620 to correct spelling errors or replace brand names with generic names or drug classes. The user can right click 1630 on the primary diagnostic axis to insert or delete LoT at the mouse cursor position. The user may drag and drop 1640 a dosing interval to edit the start or end date of that interval. The user may right click 1650 on a drug to insert a new drug or delete a drug. The user may right-click 1660 Outcomes to insert new outcomes into the record, such as surgeries, procedures, or other priority events.

Controls 1610-1660 are illustrated in terms of mouse events such as click and drag and right click, but those skilled in the art will appreciate that these controls are keyboard controls, touch screen controls, or input/output devices for PCs or mobile devices. may be manipulated using different events, such as similar interactive manipulations of the interface.

FIG. 17 is an illustration of a structured data representation of patient timeline 1700 after LoT labeling has been performed according to one embodiment.

Patient timeline 1700 represents all significant interval anchors to patient records that have LoT labels inserted. In this example, a patient received a primary cancer diagnosis on 1/1/2000, started first line of therapy on 2/29/2000, and had a progressive event approximately 16 months later on 8/11/2001. was observed and a progressive event was recognized as an intra-treatment priority event, a second line of therapy was administered beginning on 12/9/2001, and the patient underwent gene sequencing on 12/16/2002, which is likely due to Due to progressive events such as metastasis of the tumor to another organ site. As of the date of sequencing, timelines are made available to the treating physician, for example, through in-depth web portals 160A-160N. Physicians can then utilize the portal to review future LoT outcomes based on similar cohorts of patients and personalize the patient's next LoT to improve the likelihood of a positive response. Other analyzes may include assessment of other treatments, procedures other than those labeled as LoT, assessment of genetic variants in treatment outcomes, and the level of risk patients have of developing metastases or progression at any time during treatment. Including identification and other analyzes to personalize patient treatment and improve positive outcomes.

FIG. 18 is a box plot 1800 of the average duration of LoT identified from cohorts of patients in the lung cancer cohort and the breast cancer cohort.

As mentioned above, cancer-specific intervals can be learned from training the artificial intelligence engine on multiple patients. After such learning relates the average duration of treatment so that the correct dates can be imputed, the intervals are appropriate to provide meaningful potential LoT choices for the model to label from. set to a suitable size. As an example, lung cancer patients experience most treatment intervals at about 70 days, and few exceed 120 days. Analysis of LoT1-4 shows that there is negligible variation across each successive treatment line in diagnosing lung cancer. Therefore, the lung cancer interval threshold is approximately 70 days to 120 days and does not change depending on the LoT interval being evaluated. However, for breast cancer diagnosis, the majority of treatment intervals are around 100 days, with 300 days for the first LoT, 290 days for the second LoT, 260 days for the third LoT, and 210 days for the fourth LoT. degree. Analyzes for breast cancer identify thresholds for drug duration and treatment interval that are two to three times longer than those for lung cancer.

FIG. 19 is an illustration of LoT frequencies 1900 based on the first LoT for breast cancer therapeutics administered to a patient.

Based on models generated through unsupervised learning, the majority of breast cancer patients have anastrozole, tamoxifen, or AC-T (the drugs doxorubicin hydrochloride (adriamycin) and cyclophosphamide, followed by paclitaxel (Taxol)). therapy) will be initiated. Although there appears to be a large variability in the first LoT selection for breast cancer patients, Figure 20 represents the second LoT frequency distribution for breast cancer patients.

FIG. 20 is an illustration of LoT frequencies 2000 based on the second LoT for breast cancer drugs administered to patients.

Based on models generated through unsupervised learning, the majority of breast cancer patients receive the first line of therapy in FIG. 19, followed by a second line of anastrozole, tamoxifen, letrozole. Note that similarly, AC-T regimens are never prescribed as a second LoT for a patient, and thus it makes sense not to appear in the learned labeling space for a second line of treatment. want to be There is less variability in second LoT selection for breast cancer patients, which is expected as maintenance therapy is commonly prescribed in the third line of therapy.

FIG. 21 is an illustration of LoT frequencies 2100 based on the first LoT for non-small cell lung cancer (NSCLC) therapeutics administered to a patient.

Based on models generated through unsupervised learning, the majority of NSCLC patients are started on carboplatin and paclitaxel, and some patients are started on carboplatin and either pemetrexed or etoposide. Another point of accuracy includes that none of the NSCLC patients started on targeted therapies such as those listed in FIG.

FIG. 22 is an illustration of LoT frequencies 2200 based on the second LoT for non-small cell lung cancer therapeutics administered to the patient.

Based on models generated via unsupervised learning, the majority of NSCLC cancer patients receive the first line of therapy in FIG. Try targeted therapy with gemcitabine, docetaxel, or topotecan as a second LoT with an even distribution.

Figures 19-22 illustrate the frequency of occurrence of a particular LoT across all patients in each primary diagnostic cohort. Although illustrative examples were limited to breast cancer and lung cancer, the methods and systems disclosed herein can be applied to any cancer diagnosis without departing from the embodiments described herein. . Additionally, rankings for intervals and potential treatment lines may be derived directly from frequency-based likelihoods. For example, if the first LoT occurs in 50% of patients as the first LoT, a 50% likelihood may be given directly when each treatment interval corresponds to it in LoT labelling. In addition, the artificial intelligence engine may generate further likelihood estimates based on the frequency of LoTs that are consecutive with each other. For example, if the first LoT of carboplatin and paclitaxel was followed by the second LoT of nivolumab 80% of the time, and both carboplatin and paclitaxel and nivolumab were in consecutive treatment intervals, then the first LoT was carboplatin and paclitaxel and the second line of treatment is nivolumab is scaled higher than if the record included only the first line of carboplatin and paclitaxel or only the second line of nivolumab. Become.

FIG. 23 is a bar graph diagram 2300 of the proportion of patients for each primary cancer assigned to cohorts containing at least one of the listed LoTs.

In this example, breast cancer patients are distributed such that 70% are labeled 1st LoT, 15% are labeled 2nd LoT, and only 5% are labeled 3rd LoT. . Conversely, for primary diagnosis of colorectal cancer (CRC), only 60% were labeled 1st LoT, 15% were labeled 2nd LoT, and 5% were labeled 3rd LoT. be done. Each label-free patient can be manually evaluated to identify what label, if any, is applied.

FIG. 24 is an illustration of an exemplary machine of computer system 2400 on which a set of instructions may be executed to cause the machine to perform any one or more of the methods described herein. In alternative implementations, the machine may be connected (eg, networked) to other machines in a LAN, intranet, extranet, and/or the Internet.

A machine may operate in the capacity of a server or client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment. A machine can be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, server, network router, switch or bridge, or an action performed by that machine. It may be any machine capable of executing the specified instruction set (sequentially or otherwise). Further, although a single machine is illustrated, the term "machine" may refer to a set of instructions (or a set of instructions) for performing any one or more of the methods described herein. set), either individually or in conjunction, shall be construed to include any collection of machines.

The exemplary computer system 2400 includes a processing device 2402, a main memory 2404 (dynamic random access memory (DRAM) such as read-only memory (ROM), flash memory, synchronous dynamic random access memory (DRAM) (SDRAM) or DRAM, etc.), static memory 2406 (flash memory, static random access memory (SRAM), etc.), and data storage device 2418 , which communicate with each other via bus 2430 .

Processing device 2402 represents one or more general purpose processing devices such as a microprocessor, central processing unit, or the like. More specifically, the processing device implements a Complex Instruction Set Computing (CISC) microprocessor, Reduced Instruction Set Computing (RISC) microprocessor, Very Long Instruction Word (VLIW) microprocessor, or other instruction set. It can be a processor or a processor implementing a combination of instruction sets. Processing device 2402 may be one or more dedicated processing devices such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, or the like. good. Processing device 2402 is configured to execute instructions 2422 to perform the operations and steps described herein.

Computer system 2400 may further include a network interface device 2408 for connecting to LANs, intranets, the Internet, and/or extranets. Computer system 2400 includes a video display unit 2410 (such as a liquid crystal display (LCD) or cathode ray tube (CRT)), an alphanumeric input device 2412 (such as a keyboard), a cursor control device 2414 (such as a mouse), and a signal generation device 2416 (such as a speaker). ), and a graphics processing unit 2424 (such as a graphics card).

Data storage device 2418 includes machine-readable storage medium 2428 (computer readable storage medium 2428) in which is stored one or more sets of instructions or software 2422 for using any one or more of the methods or functions described herein. readable medium). The instructions 2422 may also reside wholly or at least partially in the main memory 2404 and/or the processing device 2402 during execution by the computer system 2400, which also constitute machine-readable media. do.

In one implementation, instructions 2422 comprise a software library containing instructions for and/or methods of functioning as a line of care module (such as line of care module 120 of FIG. 1). Instructions 2422 may further include instructions for record harmonization module 122, interval generation module 124, and LoT allocation module 126 (such as record harmonization module 122, interval generation module 124, and LoT allocation module 126 of FIG. 1). Although machine-readable storage medium 2428 is shown in exemplary implementations as being a single medium, the term "machine-readable storage medium" refers to a single medium that stores one or more instruction sets. It should be interpreted as including a medium or multiple mediums (such as centralized or distributed databases and/or associated caches and servers). The term "machine-readable storage medium" refers to any medium capable of storing or encoding a set of instructions to be executed by a machine and causing the machine to perform any one or more of the methods of this disclosure. should also be construed as including medium. The term "machine-readable storage medium" shall be construed accordingly to include, but not be limited to, solid state memory, optical media, and magnetic media. The term "machine-readable storage medium" shall accordingly exclude temporary storage media, such as signals, unless otherwise specified by identifying the machine-readable storage medium as a temporary storage medium or transitory machine-readable storage medium. It shall be.

In another implementation, virtual machine 2440 may comprise modules that execute instructions for record harmonization module 122 , interval generation module 124 , and LoT allocation module 126 . In computing, a virtual machine (VM) is an emulation of a computer system. A virtual machine is based on computer architecture and provides the functionality of a physical computer. These implementations may involve dedicated hardware, software, or a combination of hardware and software.

Some portions of the preceding detailed description are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is contemplated herein, and generally, to be a self-consistent sequence of operations that yields a desired result. These operations are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

It should be remembered, however, that all of these and similar terms are to be associated with appropriate physical quantities and are merely convenient labels applied to these quantities. Throughout the description, words such as "identifying" or "providing" or "calculating" or "determining" or similar terms are used unless otherwise noted, as is clear from the description above. describes manipulating data represented as physical (electronic) quantities in computer system registers and memory; It is understood to refer to the actions and processes of a computer system, or similar electronic computing device, that transform other data similarly expressed as a quantity.

The present disclosure also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the intended purposes, or it may comprise one or more general purpose computers selectively activated or reconfigured by a computer program stored in the computers. . Such computer programs may be stored on any type of disk, including, but not limited to, floppy disks, optical disks, CD-ROMs, and magneto-optical disks, read-only memory (ROM), random-access memory (RAM), EPROM, EEPROM. , magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each of which is coupled to the computer system bus.

The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the methods. The structure for a variety of these systems appears as set forth in the discussion below. Additionally, the present disclosure has not been described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the disclosure as described herein.

The present disclosure may include a computer program product or machine-readable medium having instructions stored thereon that may be used to program a computer system (or other electronic device) to perform processes in accordance with the present disclosure. It can be provided as software. A machine-readable storage medium provides any mechanism for storing information in a form readable by a machine (such as a computer). For example, a machine-readable (e.g., computer-readable) medium includes read-only memory ("ROM"), random-access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory devices, etc. Including storage media.

In the foregoing specification, implementations of the present disclosure have been described with reference to specific example implementations. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of implementations of the disclosure as defined in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

100 computer implemented system
110 Clinical Health Record
112 EMRs
114 records, information
120 Therapeutic Line Module
122 Record Harmonization Module
124 interval generation module
126 Line of Treatment Assignment Module
150 Line of Treatment (LoT) Store
160a-160n web forms
160A~160N Web page
165 Graphical User Interface (GUI)
170a-170n GUI, electronic reporting
200 systems
300 Record Harmonization Module
322 recording whitelist and/or recording blacklist modules
324 Record Association Module
326 recording combination module
328 Heuristic Processing Module
400 interval generation module
422 Diagnostic Relevance Filter
424 interval date allocation
426 Interval Date Imputation
428 Interval Date Extension
430 interval date segmentation
500 interval generation module, treatment line assignment module
522 LoT Allocation Estimation
524 LoT allocation probability
526 LoT ranking
528 LoT Labeling
600 system
605 Drug Filtering Module
610 drug combination module
615 Drug Harmonization Module
620 Harmonized Drug Table
625 Primary Cancer (PC) Association Filter
630 date padding
635 PC Curation Model
640 interval generation
645 Dosing interval
650 Outcome
655 modeled LoT frequencies
660 estimated probability
665 LoT Labeling
670 Post Processing LoT
705 Clinical Insights
710 Model data preparation stage
712 Curated Progressions and Treatment Deviations
714 Patient Medication
716 modules
718 Modeled Dates and Intervals
720 training phase
722 drug measurement interval frequency (w)
724 LoT split allocation
726 Dosing Frequency Refinement
730 model LoT allocation stage
732 LoT generation stage
734 LoT Labeling
802 Primary Diagnosis
804 Drug Administration
806 Outcomes
808 triplet chemotherapy
810 Pemetrexed/bevacizumab maintenance therapy
812 Progressive disease outcome
814 Nivolumab
816 Progressive disease outcome
818 Gemcitabine therapy
820 Paclitaxel
822 complete response
824 Dexamethasone administration
830a spacing
830b interval
830c spacing
830d spacing
830e spacing
930a spacing
930b spacing
930c spacing
930d spacing
1030a spacing
1030b interval
1030c spacing
1030d spacing
1030e spacing
1100 flow
1200 flow
1300 flow
1400 Sankey diagram
1500 Progression Free Survival (PFS) Graph
1510, 1520, 1530 Lines of Treatment
1600 treatment line editing tools
1610 control
1620 drug name, control
1630 control
1640 control
1650 control
1660 control
1700 patient timeline
1800 boxplot chart
1900 LoT frequency
2000 LoT frequency
2100 LoT frequency
2200 LoT frequency
2300 bar chart chart
2400 computer system
2402 processing device
2404 main memory
2406 static memory
2408 network interface device
2410 Video Display Unit
2412 alphanumeric input device
2414 cursor control device
2416 signal generation device
2418 data storage device
2422 instruction set or software, instructions
2424 graphics processing unit
2428 machine-readable storage medium
2430 Bus
2440 virtual machines

Claims (30)

A method for labeling one or more drugs co-administered to a patient as a line of therapy, comprising:
identifying the patient's medical records from a plurality of digital records, wherein the subset of medical records reflects the patient's medical history regarding one or more treatments after diagnosis of the patient's medical condition; ,
creating from the subset of medical records a plurality of treatment intervals comprising at least one drug administered to the patient and a time interval;
associating said one or more therapeutic agents with respective treatment intervals when said administration of said agent falls within said time interval;
refining the time interval of each treatment interval when one of the one or more treatments falls outside the time interval but within an extended period of time, said refining a refining step in which no interval precedence event occurs in the selected time interval;
identifying, via an unsupervised artificial intelligence engine, one or more potential treatment lines from the plurality of treatment intervals, each potential treatment line being one of the plurality of treatment intervals or steps, including multiple and maximum likelihood estimation;
labeling, via the unsupervised artificial intelligence engine, the potential treatment line with the highest maximum likelihood estimate as the treatment line. Via said unsupervised artificial intelligence engine, additional potential lines of treatment in successive time intervals are chronologically generated after each preceding line of treatment; 2. The method of claim 1, further comprising labeling treatment lines as incrementally numbered from said potential treatment lines having said respective highest maximum likelihood estimates. labeling, via said unsupervised artificial intelligence engine, each incrementally numbered treatment line for a plurality of patients;
identifying multiple cohorts of patients;
and reporting the incrementally numbered lines of treatment for each of the plurality of cohorts of patients. 4. The method of claim 3, further comprising reporting facility-wide and physician-specific compliance to standardized treatment guidelines. 4. The method of claim 3, further comprising reporting changes in treatment lines over time based at least in part on the respective time intervals for each treatment line. further comprising reporting an analysis of differences in treatment impact between a first cohort of patients and a second cohort of patients, the cohorts having one or more similar treatment dates, lines of treatment, diagnoses, outcomes 4. The method of claim 3, wherein the patient is identified as having the , geographic location, treating physician, treatment facility, genomic marker, or clinical characteristic. 7. The method of claim 6, further comprising reporting an analysis of progression-free survival between said first cohort and said second cohort. generating a progression-free survival curve for a first cohort of patients having at least one labeled line of therapy in common;
generating a progression-free survival curve for a second cohort of patients having at least one other labeled line of treatment in common;
4. The method of claim 3, further comprising displaying said progression-free survival curves for said first cohort and said second cohort on the same survival graph. 9. The method of claim 8, wherein the labeled therapeutic line includes one or more of drug name, therapeutic rollup class, dosage, or method of administration. 2. The method of claim 1, further comprising generating a report, said report including each treatment line labeled for said patient and said respective time interval. 2. The method of claim 1, wherein medical condition diagnosis includes cancer, heart disease, depression, mental health, diabetes, infectious diseases, epilepsy, dermatology, and autoimmune diseases. 2. The method of claim 1, wherein the medical condition diagnosis is cancer. 2. The method of claim 1, wherein the plurality of digital records comprises one or more EHR medical records and one or more curated medical records. 14. The method of claim 13, wherein curated medical records are images of records not included in the one or more EHR medical records. 14. The method of claim 13, further comprising binarizing, optical character recognition, and encoding the curated medical records into a structured format. 2. The method of claim 1, further comprising identifying by a therapy identifier the therapy administered to the patient and the administration date on which the therapy was administered. treatment is
administration of drugs, drugs, molecules, or chemicals;
implanting medical devices,
use of medical devices;
Use of biotherapy, virus therapy, phage therapy, plant therapy, gene therapy, epigenetic therapy, protein therapy, enzyme replacement therapy, hormone therapy, cell therapy, immunotherapy, antibody therapy, nutritional therapy, electromagnetic radiation therapy, or radiotherapy ,
2. The method of claim 1, comprising one or more of: a surgical procedure; or radiosurgery. 1. A system for labeling one or more agents co-administered to a patient as lines of therapy, comprising:
1. At least one computer having at least one processor, said processor for identifying said patient's medical record from a plurality of digital records, said subset of said medical records being one post-diagnosis of said patient's medical condition. identifying, reflecting the patient's medical history regarding one or more treatments; and creating, from the subset of medical records, a plurality of treatment intervals including at least one drug administered to the patient and a time interval. When,
associating the one or more therapeutic agents with respective treatment intervals when the administration of the agent falls within the time interval;
refining the time interval of each treatment interval when one of the one or more treatments falls outside the time interval but within an extended period of time, said refining at least one computer programmed to refine, wherein no interval precedence event occurs in a set time interval;
The at least one computer includes an unsupervised artificial intelligence engine, the unsupervised artificial intelligence engine to identify one or more potential treatment lines from the plurality of treatment intervals, each potential treatment line comprising: , one or more of the plurality of treatment intervals, and maximum likelihood estimation;
and labeling said potential treatment line with the highest maximum likelihood estimate as said treatment line. The unsupervised artificial intelligence engine chronologically generates additional potential lines of treatment in successive time intervals after each preceding line of treatment; 19. The system of claim 18, programmed to label as the treatment line numbered incrementally from the potential treatment line having the highest maximum likelihood estimate of . The unsupervised artificial intelligence engine includes:
labeling each incrementally numbered line of treatment for multiple patients;
identify multiple cohorts of patients,
20. The system of claim 19, programmed to report the incrementally numbered lines of treatment for each of the plurality of cohorts of patients. The at least one computer is
21. The system of claim 20, further programmed to report facility-wide and physician-specific compliance to standardized treatment guidelines. The at least one computer is
21. The system of claim 20, programmed to report changes in treatment lines over time based at least in part on the respective time intervals for each treatment line. The at least one computer is
further programmed to report an analysis of differences in treatment impact between a first cohort of patients and a second cohort of patients, wherein the cohorts are classified according to one or more similar treatment dates, treatment lines, diagnoses, 21. The system of claim 20, wherein the patient is identified as having an outcome, geographic location, treating physician, treatment facility, genomic marker, or clinical characteristic. The at least one computer is
24. The system of claim 23, further programmed to report an analysis of progression-free survival between said first cohort and said second cohort. The at least one computer is
generating a progression-free survival curve for a first cohort of patients having at least one labeled line of therapy in common;
generating a progression-free survival curve for a second cohort of patients having at least one other labeled line of treatment in common;
21. The system of claim 20, further programmed to display said progression-free survival curves for said first cohort and said second cohort on the same survival graph. The labeled lines of treatment are
26. The system of claim 25, comprising one or more of drug name, therapeutic rollup class, dosage, or method of administration. 19. The system of claim 18, wherein medical condition diagnoses include cancer, heart disease, depression, mental health, diabetes, infectious diseases, epilepsy, dermatology, and autoimmune diseases. 19. The system of claim 18, wherein the plurality of digital records includes one or more EHR medical records and one or more curated medical records. The at least one computer is
29. The system of claim 28, further programmed to binarize, optical character recognize, and encode the curated medical records into a structured format. treatment is
administration of drugs, drugs, molecules, or chemicals;
implanting medical devices,
use of medical devices;
Use of biotherapy, virus therapy, phage therapy, plant therapy, gene therapy, epigenetic therapy, protein therapy, enzyme replacement therapy, hormone therapy, cell therapy, immunotherapy, antibody therapy, nutritional therapy, electromagnetic radiation therapy, or radiotherapy ,
19. The system of claim 18, comprising one or more of: a surgical procedure; or radiosurgery.

Images