US20160246945A1 - System and method for weighting manageable patient attributes during criteria evaluations for treatment - Google Patents
System and method for weighting manageable patient attributes during criteria evaluations for treatment Download PDFInfo
- Publication number
- US20160246945A1 US20160246945A1 US14/692,504 US201514692504A US2016246945A1 US 20160246945 A1 US20160246945 A1 US 20160246945A1 US 201514692504 A US201514692504 A US 201514692504A US 2016246945 A1 US2016246945 A1 US 2016246945A1
- Authority
- US
- United States
- Prior art keywords
- patient
- attribute
- treatment
- attributes
- attribute value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000011282 treatment Methods 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000011156 evaluation Methods 0.000 title description 9
- 230000008859 change Effects 0.000 claims description 12
- 238000003058 natural language processing Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 2
- 238000004590 computer program Methods 0.000 abstract description 11
- 238000003860 storage Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 230000036772 blood pressure Effects 0.000 description 7
- 229940079593 drug Drugs 0.000 description 7
- 239000003814 drug Substances 0.000 description 7
- 238000005303 weighing Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000002483 medication Methods 0.000 description 3
- 206010028980 Neoplasm Diseases 0.000 description 2
- 230000037213 diet Effects 0.000 description 2
- 235000005911 diet Nutrition 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000035488 systolic blood pressure Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G06F19/3481—
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G06F19/322—
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/20—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for electronic clinical trials or questionnaires
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H70/00—ICT specially adapted for the handling or processing of medical references
- G16H70/20—ICT specially adapted for the handling or processing of medical references relating to practices or guidelines
Definitions
- Embodiments disclosed herein relate to computer software. More specifically, embodiments disclosed herein relate techniques for identifying criteria for a patient treatment, whether a patient meets the criteria, and whether the patient is likely to be able meet the required criteria if they do not currently meet the requirements.
- Determining an appropriate treatment based on the patient is an important part of providing medical care.
- patients In order to have safe, relevant, and effective treatment, patients generally need to satisfy criteria before receiving a particular treatment. Criteria for each treatment typically varies and may include multiple attributes used to evaluate the suitability of a treatment for a given patient. Each attribute may specify an associated value or range which a patient should meet in order to qualify for a given treatment. Additionally, patient medical data may change over time or a course of treatment. Because of these factors, matching patients to treatments can be a time-consuming process. Accordingly, techniques for weighing manageable patient attributes during criteria evaluations for treatment are needed.
- Embodiments disclosed herein provide a system, method, and computer program product for evaluating attribute values to determine treatment eligibility, the method by receiving a set of required attributes associated with a treatment protocol, receiving a case, wherein the case includes a patient history containing patient attribute values, identifying a patient attribute value that does not satisfy a required attribute specified by the treatment protocol, determining a likelihood that the patient could meet the required attribute based upon the patient history, and providing an indication of the likelihood that the patient could satisfy the required attribute specified by the treatment protocol.
- FIG. 1 is a block diagram illustrating a networked system for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment.
- FIG. 2 is a flow chart illustrating a method for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment.
- FIG. 3 is a flow chart illustrating a method for determining whether required attribute values are met according to one embodiment.
- FIG. 4 is a flow chart illustrating a method for determining how likely a required attribute value will be met according to one embodiment.
- FIG. 5 is a block diagram illustrating components of a system for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment.
- Embodiments disclosed provide techniques for identifying required criteria for a patient treatment protocols, determining whether a patient meets the required criteria, and predicting whether the patient is likely to be able meet the required criteria if they do not currently meet the required criteria.
- Patient treatments may include existing treatment options as well as clinical trials of new medications or techniques. Matching patients to clinical trials allows for safer and more effective clinical trials.
- the required criteria for a particular clinical trial treatment protocol includes attributes with an associated required value or range which must be met by the patient in order to participate.
- a particular clinical trial may include a treatment protocol calling for patients under 50 years of age, where age is an attribute requiring a value under 50 for eligibility to participate in the trial. In such a case, a 51 year old person would not be able to participate in the trial.
- a second trial may require patients with a systolic blood pressure between 140 and 159.
- a patient with a blood pressure of 170, but starting blood pressure medication may be excluded from this second trial where an attribute range for blood pressure is strictly evaluated.
- a patient's attributes may be evaluated and scored based upon how closely they align with the required attributes.
- an attribute can be changed does not mean that the attribute can be changed for a particular patient. That is, it can be difficult to determine the likelihood that the particular patient's attribute value can be changed to meet a required attribute value. For example, a particular patient may have been unresponsive to multiple blood pressure medications and thus less likely to be able to adjust their blood pressure to the required range. Another patient may be steadily losing weight through a diet and exercise regime. Analyzing the medical history of the particular patient may therefore yield a more accurate estimate of whether a patient's attribute value may meet the required attribute value in the future.
- embodiments disclosed herein provide techniques for identifying required criteria for a patient treatment, determining whether a patient currently meets the required criteria, and predicting whether the patient is likely to be able meet the required criteria if they do not currently meet the required criteria.
- An automated threshold value prediction system may process cases through an analysis “pipeline.”
- a pipeline may represent the execution of various analysis programs, or engines, using both the medical history of a patient as well as the criteria for a treatment protocol of a clinical trial.
- Information about the criteria, including the required attributes associated with the treatment protocol, for a trial may be presented as either structured or unstructured data. With structured data, data conversion may be applied to the required attributes and attribute values.
- Unstructured data may also be processed using direct information analysis, for example, using natural language processing (NLP), of the data to extract the required attributes and required attribute values of the clinical trial.
- NLP natural language processing
- a pipeline may be created for each domain or problem space (e.g. a different pipeline is used for supporting cancer treatments, diagnoses, predicting opinions, etc.).
- analysis engines themselves may be directed to a particular domain (e.g., identification of a tumor stage or size, identification of health issues, predicting opinions, etc.).
- Question and answer analysis within a pipeline may also include complex NLP algorithms, used, for example, to identify deep semantic relationships within the text.
- the pipeline may also analyze the attributes of a patient and determine which attributes may change. While certain patient attributes are not variable and cannot change, other patient attributes can change over time. For example, the fact that a patient once broke their arm cannot change, but that patient's weight or blood pressure can change over time. That is, a patient's weight is manageable as the patient can control their weight such as through diet and exercise. Other patient attributes may be mutable as they may be changed through medical interventions. For example, a patient's blood pressure may be mutable through medication. Further, certain attributes may change over time, but this change may or may not matter depending on the required attribute value. For example a patient's age changes over time.
- the pipeline may take into account the information related to required attributes values in a given trial in order to determine which attributes that may change are relevant for the given trial.
- a patient's eligibility for a particular clinical trial may be scored based on how close a match the patient's attribute values are to the particular trial's required attribute values as specified by the treatment protocol. Where a patient's attribute values all meet the required attribute values, the patient is deemed eligible for the trial. Where a patient's attribute values does not meet all the required attribute values specified by the treatment protocol, the deep QA system can make a determination of how likely a patient will be able to meet an attribute and score the patient based on the determination.
- the scoring phase of a deep QA system such as IBM's Watson, may call various scoring algorithms to help identify an answer (or response) to a case.
- a scoring algorithm may generate one or more feature scores to indicate a measure of confidence for each answer.
- the deep QA system may also use a training phase to learn which features, or combinations of features, are best at predicting a correct answer for different questions. Once the deep QA system has been trained, questions flowing through the pipeline may use the machine-learned model for finding the most likely correct answer.
- FIG. 1 is a block diagram illustrating a networked system 100 for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment.
- the system 100 includes a computer 102 .
- the computer 102 may also be connected to other computers via a network 130 .
- the network 130 may be a telecommunications network and/or a wide area network (WAN).
- the network 130 is the Internet.
- the computer 102 generally includes a processor 104 connected via a bus 120 to a memory 106 , a network interface device 118 , a storage 108 , an input device 122 , and an output device 124 .
- the computer 102 is generally under the control of an operating system (not shown). Examples of operating systems include the UNIX operating system, versions of the Microsoft Windows operating system, and distributions of the Linux operating system. (UNIX is a registered trademark of The Open Group in the United States and other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.) More generally, any operating system supporting the functions disclosed herein may be used.
- the processor 104 is included to be representative of a single CPU, multiple CPUs, a single CPU having multiple processing cores, and the like.
- the memory 106 may be a random access memory. While the memory 106 is shown as a single identity, it should be understood that the memory 106 may comprise a plurality of modules, and that the memory 106 may exist at multiple levels, from high speed registers and caches to lower speed but larger DRAM chips.
- the network interface device 118 may be any type of network communications device allowing the computer 102 to communicate with other computers via the network 130 .
- the storage 108 may be a persistent storage device. Although the storage 108 is shown as a single unit, the storage 108 may be a combination of fixed and/or removable storage devices, such as fixed disc drives, solid state drives, removable memory cards or optical storage. The memory 106 and the storage 108 may be part of one virtual address space spanning multiple primary and secondary storage devices.
- the memory 106 contains the QA application 112 , which is an application generally configured to operate a deep question answering (QA) system.
- QA deep question answering
- One example of a deep question answering system is Watson, by the IBM Corporation of Armonk, N.Y.
- a user may submit a case (also referred to as a question) to the QA application 112 , which provides an answer to the case.
- the QA system may predict whether a patient may be able to change their patient attribute values to meet the required attribute values of a given treatment.
- the QA application 112 may execute a pipeline to generate the prediction, which may be returned to the user.
- storage 108 contains an ontology 110 , corpus 114 , feature store 115 , ML models 116 , and evidence classification 117 .
- the ontology 110 provides a structural framework for organizing information.
- An ontology formally represents knowledge as a set of concepts within a domain, and the relationships between those concepts.
- the corpus 114 is a body of information used by the QA application 112 to generate answers to cases.
- Evidence classification 117 stores relationships between evidence from the corpus 114 , the question context, and the predictive features.
- ontology 110 may take any form sufficient to store data, including text files, xml data files, and the like.
- the ontology 110 is part of the corpus 114 .
- any combination of the QA application 112 , the ontology 110 , corpus 114 , feature store 115 , ML models 116 , and evidence classification 117 may reside on the same or different computers.
- the input device 122 provides input to the computer 102 .
- a keyboard and/or a mouse may be used.
- the output device 124 may be any device for providing output to a user of the computer 102 .
- the output device 124 may be any conventional display screen.
- the output device 124 and input device 122 may be combined.
- a display screen with an integrated touch-screen may be used.
- FIG. 2 is a flow chart illustrating a method 200 for use in weighing manageable patient attributes during criteria evaluations for treatment, according to one embodiment.
- the QA application receives a question from a user seeking to determine the clinical trials a patient may be eligible for.
- the QA application 112 determines that the question seeks to determine the clinical trials the patient is eligible to participate in. This may be done by processing the question to identify topics present in the question and identify what answer is being sought, e.g. whether a patient is eligible for a particular trial. A person may also be identified from the question. Any suitable method may be used to identify the question context, including the use of natural language processing (NLP) to extract normalized terms and concepts from the question.
- NLP natural language processing
- the QA application 112 receives clinical trial criteria information. This information may be received from the user or previously stored in, for example, the corpus 114 , and is associated with treatment protocols of a clinical trial, or a set of clinical trials.
- the QA application 112 may parse the clinical trial criteria to identify attributes required by the trial along with the required attribute values. This information may be in the form of structured or unstructured data. Structured data is data formatted or marked up to make the information easier to parse by computers, such as with the addition of tagged or XML data fields. Where the clinical trial criteria is stored in structured data, the data may be converted into a data format compatible with the QA application 112 .
- Unstructured data may be formatted for human readability and is not optimized for data input or modeling.
- the QA application 112 may parse the data, using techniques such as NLP, to extract the required attributes and associated attribute values.
- the QA application 112 may identify and label each required attribute that is mutable or manageable. This identification may be based upon, for example, database comparisons, predicative algorithms or earlier trained models of how mutable or manageable a given required attribute may be.
- the QA application 112 receives a patient record. This patient record may also be structured or unstructured data.
- the QA application determines whether the patient satisfies the required attribute values of a particular clinical trial. Step 270 is discussed in greater detail with reference to FIG. 3 . If it is determined that the patient satisfies the criteria specified by the treatment protocol for a clinical trial, then the patient is marked as eligible to participate in the clinical trial. If there are additional clinical trial to evaluate for the patient, the method loops back to step 210 . Otherwise, where the patient does not satisfy some of the required attribute values, the QA application, at step 280 , determines how likely it is that each criteria not satisfied can (or will) be later met by the patient. Step 280 is discussed in greater detail with reference to FIG. 4 .
- the weighted answer is returned to the user indicating how likely the patient is to be eligible for the clinical trial.
- the weighted answer may be the full weight (i.e. a maximum score) or simply an indication that the patient is qualified for the clinical trial.
- the weighted answer may also be processed for ease of use. For example, threshold may be set such that weights above a certain weight may result in an answer of “Potentially Eligible,” “Probably Not Eligible,” or the like.
- FIG. 3 is a flow chart illustrating a method 300 for determining whether required attribute values are met according to one embodiment.
- the patient's medical record is parsed to extract values for the attributes used to determine eligibility of a patient to participate in a clinical trial.
- a patient's medical records may include both structured and unstructured data and may be parsed using techniques such as NLP, predicative algorithms and the like.
- previous values for some patient attributes may also be identified.
- the attribute values identified in the patient's medical record are compared against the criteria for eligibility. The patient's eligibility for the clinical trial being evaluated is then scored based on the required attribute values met and the weights of those attributes.
- Scoring may comprise an aggregation score where scores are aggregated and when the patient's eligibility score reaches a particular threshold, the patient may be determined to be eligible for the trial. This threshold may be set such that all of the required attribute values are met to reach the threshold. The set of required attribute values that are not met by the patient may also be identified and stored.
- the eligibility score along with attribute information are returned for further processing, if needed.
- the weighted answer may be based on the eligibility score.
- FIG. 4 is a flow chart illustrating a method 400 for determining how a required attribute value will be met according to one embodiment.
- the attribute values, criteria, and the parsed patient medical records are received.
- attributes which the patient does not satisfy are evaluated based on whether they are mutable or manageable attributes with respect to the clinical trial. Where an attribute is determined to be unmanageable or immutable and the patient attribute does not satisfy the attribute, the QA application 112 returns an answer indicating that the patient is ineligible for the clinical trial.
- the patient's past values for the attribute may be evaluated to determine how likely it is that the patient may satisfy the criteria for the mutable or manageable attribute in the future.
- the patient's eligibility score may be adjusted based on how far the patient's attribute value is from the required attribute value. For example, where the difference between the patient's attribute value and the required attribute value is large, the patient's eligibility score may be adjusted such that the patient is less likely to ever be eligible even if all other required attributes are met. Conversely, where the distance is smaller the patient's eligibility score may be adjusted such that the patient is more likely to eventually be eligible.
- the patient's eligibility score would be further from the eligibility threshold than if the patient were 2 lbs under the required weight.
- the patient's attribute and attribute value history may be evaluated.
- the evaluation may include a time dimension and account for trends or changes to an attribute value over time. For example, where a patient is above a required weight, but the patient's weight is steadily trending down, the patient's eligibility score may be adjusted to reflect this progress. However, where a patient has an extensive medical history indicating the patient has never satisfied the required attribute value, the trend indicates that the patient is less likely to meet the required attribute values and the patient's eligibility score may be adjusted to reflect this.
- the patient's eligibility score may be adjusted such that the patient is more likely to be eligible.
- Other factors may also be correlated to attribute values over time. Where a record reflects treatments or attempts to adjust an attribute, the eligibility score may be adjusted to take into account these factors.
- These factors may be associated with attribute values using, for example, data stored in the corpus, such as a medical treatment data.
- a patient's medical record may indicate that the patient has taken several medications used for treating a particular attribute. Any recorded changes (if any) to the attribute value while the patient is on the medication may indicate how the patient responds to the medication and correlate to how manageable or mutable the attribute value may be for the patient.
- Treatments associated with managing or mutating the attribute value may also be compared against the required attribute values to avoid treatments with side-effects that may disqualify the patient for the trial.
- factors that influence the patient's eligibility score may be cumulative. That is, a first patient's fluctuating historical attribute value and a large difference between their attribute value and the required attribute value likely will be less likely to be eligible as compared to a second patient with a similarly fluctuating historical attribute value, but a small difference between their attribute value and the required attribute value. Other factors not directly related to a medical attribute may also be considered. For example, a patient's history may indicate how willing a patient is to travel to obtain treatment, or how willing to consent to various procedures may also be considered.
- the patient's eligibility score may be returned as a weighted answer based on the eligibility score.
- FIG. 5 is a block diagram illustrating components of a system for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment.
- the deep question answering system is the QA application 112 .
- the QA application 112 contains a question classifier component 501 , a partial and parallel pipeline execution component 502 , and an attribute verification component 503 .
- the question classifier component 501 may parse questions to determine that the question seeks to identify trials in which a patient may be eligible for. Once identified, the topic may be stored in a feature store, such as feature store 115 .
- the partial and parallel pipeline execution component 502 may, when presented with a case, access the topic defined in the feature store 115 to reduce processing of sources that do not discuss the topic identified.
- the attribute verification component 503 may identify relevant pieces of information from the patient medical records, parse the information, and store these pieces of information in the corpus 114 .
- each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s).
- the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
- aspects of the present disclosure may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
- the computer readable medium may be a computer readable signal medium or a computer readable storage medium.
- a computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
- a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
- a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof.
- a computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
- Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
- Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages.
- the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
- the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
- LAN local area network
- WAN wide area network
- Internet Service Provider for example, AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
- These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
- the computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- Embodiments of the disclosure may be provided to end users through a cloud computing infrastructure.
- Cloud computing generally refers to the provision of scalable computing resources as a service over a network.
- Cloud computing may be defined as a computing capability that provides an abstraction between the computing resource and its underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction.
- cloud computing allows a user to access virtual computing resources (e.g., storage, data, applications, and even complete virtualized computing systems) in “the cloud,” without regard for the underlying physical systems (or locations of those systems) used to provide the computing resources.
- cloud computing resources are provided to a user on a pay-per-use basis, where users are charged only for the computing resources actually used (e.g. an amount of storage space consumed by a user or a number of virtualized systems instantiated by the user).
- a user can access any of the resources that reside in the cloud at any time, and from anywhere across the Internet.
- a user may access a deep question answering system or related data available in the cloud.
- the deep question answering system could execute on a computing system in the cloud and provide question classification and feature mapping.
- the deep question answering system could classify questions, map features and store the resultant data sets at a storage location in the cloud. Doing so allows a user to access this information from any computing system attached to a network connected to the cloud (e.g., the Internet).
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Pathology (AREA)
- Bioethics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Surgery (AREA)
- Urology & Nephrology (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
Description
- This application is a continuation of co-pending U.S. patent application Ser. No. 14/630,751, filed Feb. 25, 2015. The aforementioned related patent application is herein incorporated by reference in its entirety.
- Embodiments disclosed herein relate to computer software. More specifically, embodiments disclosed herein relate techniques for identifying criteria for a patient treatment, whether a patient meets the criteria, and whether the patient is likely to be able meet the required criteria if they do not currently meet the requirements.
- Determining an appropriate treatment based on the patient is an important part of providing medical care. In order to have safe, relevant, and effective treatment, patients generally need to satisfy criteria before receiving a particular treatment. Criteria for each treatment typically varies and may include multiple attributes used to evaluate the suitability of a treatment for a given patient. Each attribute may specify an associated value or range which a patient should meet in order to qualify for a given treatment. Additionally, patient medical data may change over time or a course of treatment. Because of these factors, matching patients to treatments can be a time-consuming process. Accordingly, techniques for weighing manageable patient attributes during criteria evaluations for treatment are needed.
- Embodiments disclosed herein provide a system, method, and computer program product for evaluating attribute values to determine treatment eligibility, the method by receiving a set of required attributes associated with a treatment protocol, receiving a case, wherein the case includes a patient history containing patient attribute values, identifying a patient attribute value that does not satisfy a required attribute specified by the treatment protocol, determining a likelihood that the patient could meet the required attribute based upon the patient history, and providing an indication of the likelihood that the patient could satisfy the required attribute specified by the treatment protocol.
- So that the manner in which the above recited aspects are attained and can be understood in detail, a more particular description of embodiments of the disclosure, briefly summarized above, may be had by reference to the appended drawings.
- It is to be noted, however, that the appended drawings illustrate only typical embodiments of this disclosure and are therefore not to be considered limiting of its scope, for the disclosure may admit to other equally effective embodiments.
-
FIG. 1 is a block diagram illustrating a networked system for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment. -
FIG. 2 is a flow chart illustrating a method for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment. -
FIG. 3 is a flow chart illustrating a method for determining whether required attribute values are met according to one embodiment. -
FIG. 4 is a flow chart illustrating a method for determining how likely a required attribute value will be met according to one embodiment. -
FIG. 5 is a block diagram illustrating components of a system for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment. - Embodiments disclosed provide techniques for identifying required criteria for a patient treatment protocols, determining whether a patient meets the required criteria, and predicting whether the patient is likely to be able meet the required criteria if they do not currently meet the required criteria. Patient treatments may include existing treatment options as well as clinical trials of new medications or techniques. Matching patients to clinical trials allows for safer and more effective clinical trials. Generally, the required criteria for a particular clinical trial treatment protocol includes attributes with an associated required value or range which must be met by the patient in order to participate. For example, a particular clinical trial may include a treatment protocol calling for patients under 50 years of age, where age is an attribute requiring a value under 50 for eligibility to participate in the trial. In such a case, a 51 year old person would not be able to participate in the trial. However, a second trial may require patients with a systolic blood pressure between 140 and 159. A patient with a blood pressure of 170, but starting blood pressure medication may be excluded from this second trial where an attribute range for blood pressure is strictly evaluated. To broaden the pool of eligible patients, when determining eligibility for a clinical trial, it may be desirable to include patients who do not strictly meet every required attribute of the treatment protocol, but may be able to meet the required attributes in the future. To determine which patients may be able to meet the required attributes, a patient's attributes may be evaluated and scored based upon how closely they align with the required attributes.
- However, just because an attribute can be changed does not mean that the attribute can be changed for a particular patient. That is, it can be difficult to determine the likelihood that the particular patient's attribute value can be changed to meet a required attribute value. For example, a particular patient may have been unresponsive to multiple blood pressure medications and thus less likely to be able to adjust their blood pressure to the required range. Another patient may be steadily losing weight through a diet and exercise regime. Analyzing the medical history of the particular patient may therefore yield a more accurate estimate of whether a patient's attribute value may meet the required attribute value in the future.
- Generally, embodiments disclosed herein provide techniques for identifying required criteria for a patient treatment, determining whether a patient currently meets the required criteria, and predicting whether the patient is likely to be able meet the required criteria if they do not currently meet the required criteria. An automated threshold value prediction system may process cases through an analysis “pipeline.” A pipeline may represent the execution of various analysis programs, or engines, using both the medical history of a patient as well as the criteria for a treatment protocol of a clinical trial. Information about the criteria, including the required attributes associated with the treatment protocol, for a trial may be presented as either structured or unstructured data. With structured data, data conversion may be applied to the required attributes and attribute values. Unstructured data may also be processed using direct information analysis, for example, using natural language processing (NLP), of the data to extract the required attributes and required attribute values of the clinical trial. A pipeline may be created for each domain or problem space (e.g. a different pipeline is used for supporting cancer treatments, diagnoses, predicting opinions, etc.). Further, analysis engines themselves may be directed to a particular domain (e.g., identification of a tumor stage or size, identification of health issues, predicting opinions, etc.). Question and answer analysis within a pipeline may also include complex NLP algorithms, used, for example, to identify deep semantic relationships within the text.
- The pipeline may also analyze the attributes of a patient and determine which attributes may change. While certain patient attributes are not variable and cannot change, other patient attributes can change over time. For example, the fact that a patient once broke their arm cannot change, but that patient's weight or blood pressure can change over time. That is, a patient's weight is manageable as the patient can control their weight such as through diet and exercise. Other patient attributes may be mutable as they may be changed through medical interventions. For example, a patient's blood pressure may be mutable through medication. Further, certain attributes may change over time, but this change may or may not matter depending on the required attribute value. For example a patient's age changes over time. For a trial where eligibility depends on whether a patient is currently under the given age, that the patient continues to age does not matter. However, where a trial requires participants within a certain age range, a patient may become eligible for the trial at a later date. The pipeline may take into account the information related to required attributes values in a given trial in order to determine which attributes that may change are relevant for the given trial.
- A patient's eligibility for a particular clinical trial may be scored based on how close a match the patient's attribute values are to the particular trial's required attribute values as specified by the treatment protocol. Where a patient's attribute values all meet the required attribute values, the patient is deemed eligible for the trial. Where a patient's attribute values does not meet all the required attribute values specified by the treatment protocol, the deep QA system can make a determination of how likely a patient will be able to meet an attribute and score the patient based on the determination. The scoring phase of a deep QA system, such as IBM's Watson, may call various scoring algorithms to help identify an answer (or response) to a case. A scoring algorithm may generate one or more feature scores to indicate a measure of confidence for each answer. The deep QA system may also use a training phase to learn which features, or combinations of features, are best at predicting a correct answer for different questions. Once the deep QA system has been trained, questions flowing through the pipeline may use the machine-learned model for finding the most likely correct answer.
-
FIG. 1 is a block diagram illustrating anetworked system 100 for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment. Thesystem 100 includes acomputer 102. Thecomputer 102 may also be connected to other computers via anetwork 130. In general, thenetwork 130 may be a telecommunications network and/or a wide area network (WAN). In a particular embodiment, thenetwork 130 is the Internet. - The
computer 102 generally includes aprocessor 104 connected via a bus 120 to amemory 106, anetwork interface device 118, astorage 108, aninput device 122, and anoutput device 124. Thecomputer 102 is generally under the control of an operating system (not shown). Examples of operating systems include the UNIX operating system, versions of the Microsoft Windows operating system, and distributions of the Linux operating system. (UNIX is a registered trademark of The Open Group in the United States and other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.) More generally, any operating system supporting the functions disclosed herein may be used. Theprocessor 104 is included to be representative of a single CPU, multiple CPUs, a single CPU having multiple processing cores, and the like. Similarly, thememory 106 may be a random access memory. While thememory 106 is shown as a single identity, it should be understood that thememory 106 may comprise a plurality of modules, and that thememory 106 may exist at multiple levels, from high speed registers and caches to lower speed but larger DRAM chips. Thenetwork interface device 118 may be any type of network communications device allowing thecomputer 102 to communicate with other computers via thenetwork 130. - The
storage 108 may be a persistent storage device. Although thestorage 108 is shown as a single unit, thestorage 108 may be a combination of fixed and/or removable storage devices, such as fixed disc drives, solid state drives, removable memory cards or optical storage. Thememory 106 and thestorage 108 may be part of one virtual address space spanning multiple primary and secondary storage devices. - As shown, the
memory 106 contains theQA application 112, which is an application generally configured to operate a deep question answering (QA) system. One example of a deep question answering system is Watson, by the IBM Corporation of Armonk, N.Y. A user may submit a case (also referred to as a question) to theQA application 112, which provides an answer to the case. In one embodiment, for example, the QA system may predict whether a patient may be able to change their patient attribute values to meet the required attribute values of a given treatment. TheQA application 112 may execute a pipeline to generate the prediction, which may be returned to the user. - As shown,
storage 108 contains anontology 110, corpus 114,feature store 115,ML models 116, and evidence classification 117. Theontology 110 provides a structural framework for organizing information. An ontology formally represents knowledge as a set of concepts within a domain, and the relationships between those concepts. The corpus 114 is a body of information used by theQA application 112 to generate answers to cases. Evidence classification 117 stores relationships between evidence from the corpus 114, the question context, and the predictive features. Although depicted as a database,ontology 110, corpus 114,feature store 115,ML models 116, and evidence classification 117 may take any form sufficient to store data, including text files, xml data files, and the like. In one embodiment, theontology 110 is part of the corpus 114. Although depicted as residing on the same computer, any combination of theQA application 112, theontology 110, corpus 114,feature store 115,ML models 116, and evidence classification 117 may reside on the same or different computers. - The
input device 122 provides input to thecomputer 102. For example, a keyboard and/or a mouse may be used. Theoutput device 124 may be any device for providing output to a user of thecomputer 102. For example, theoutput device 124 may be any conventional display screen. Although shown separately from theinput device 122, theoutput device 124 andinput device 122 may be combined. For example, a display screen with an integrated touch-screen may be used. -
FIG. 2 is a flow chart illustrating amethod 200 for use in weighing manageable patient attributes during criteria evaluations for treatment, according to one embodiment. Atstep 210, the QA application receives a question from a user seeking to determine the clinical trials a patient may be eligible for. Atstep 220, theQA application 112 determines that the question seeks to determine the clinical trials the patient is eligible to participate in. This may be done by processing the question to identify topics present in the question and identify what answer is being sought, e.g. whether a patient is eligible for a particular trial. A person may also be identified from the question. Any suitable method may be used to identify the question context, including the use of natural language processing (NLP) to extract normalized terms and concepts from the question. Atstep 230, theQA application 112 receives clinical trial criteria information. This information may be received from the user or previously stored in, for example, the corpus 114, and is associated with treatment protocols of a clinical trial, or a set of clinical trials. Atstep 240, theQA application 112 may parse the clinical trial criteria to identify attributes required by the trial along with the required attribute values. This information may be in the form of structured or unstructured data. Structured data is data formatted or marked up to make the information easier to parse by computers, such as with the addition of tagged or XML data fields. Where the clinical trial criteria is stored in structured data, the data may be converted into a data format compatible with theQA application 112. Unstructured data may be formatted for human readability and is not optimized for data input or modeling. Where the clinical trial criteria is specified using unstructured data, theQA application 112 may parse the data, using techniques such as NLP, to extract the required attributes and associated attribute values. Atstep 250, theQA application 112 may identify and label each required attribute that is mutable or manageable. This identification may be based upon, for example, database comparisons, predicative algorithms or earlier trained models of how mutable or manageable a given required attribute may be. Atstep 260, theQA application 112 receives a patient record. This patient record may also be structured or unstructured data. Atstep 270, the QA application determines whether the patient satisfies the required attribute values of a particular clinical trial. Step 270 is discussed in greater detail with reference toFIG. 3 . If it is determined that the patient satisfies the criteria specified by the treatment protocol for a clinical trial, then the patient is marked as eligible to participate in the clinical trial. If there are additional clinical trial to evaluate for the patient, the method loops back tostep 210. Otherwise, where the patient does not satisfy some of the required attribute values, the QA application, atstep 280, determines how likely it is that each criteria not satisfied can (or will) be later met by the patient. Step 280 is discussed in greater detail with reference toFIG. 4 . Atstep 290, the weighted answer is returned to the user indicating how likely the patient is to be eligible for the clinical trial. Where a patient meets all required attributes, the weighted answer may be the full weight (i.e. a maximum score) or simply an indication that the patient is qualified for the clinical trial. The weighted answer may also be processed for ease of use. For example, threshold may be set such that weights above a certain weight may result in an answer of “Potentially Eligible,” “Probably Not Eligible,” or the like. -
FIG. 3 is a flow chart illustrating amethod 300 for determining whether required attribute values are met according to one embodiment. Atstep 310, the patient's medical record is parsed to extract values for the attributes used to determine eligibility of a patient to participate in a clinical trial. As with the clinical trial criteria, a patient's medical records may include both structured and unstructured data and may be parsed using techniques such as NLP, predicative algorithms and the like. In addition to the patient's current attribute values, previous values for some patient attributes may also be identified. Atstep 320, the attribute values identified in the patient's medical record are compared against the criteria for eligibility. The patient's eligibility for the clinical trial being evaluated is then scored based on the required attribute values met and the weights of those attributes. Scoring may comprise an aggregation score where scores are aggregated and when the patient's eligibility score reaches a particular threshold, the patient may be determined to be eligible for the trial. This threshold may be set such that all of the required attribute values are met to reach the threshold. The set of required attribute values that are not met by the patient may also be identified and stored. Atstep 330, the eligibility score along with attribute information are returned for further processing, if needed. The weighted answer may be based on the eligibility score. -
FIG. 4 is a flow chart illustrating amethod 400 for determining how a required attribute value will be met according to one embodiment. Atstep 410, the attribute values, criteria, and the parsed patient medical records are received. Atstep 420, attributes which the patient does not satisfy are evaluated based on whether they are mutable or manageable attributes with respect to the clinical trial. Where an attribute is determined to be unmanageable or immutable and the patient attribute does not satisfy the attribute, theQA application 112 returns an answer indicating that the patient is ineligible for the clinical trial. - At
step 430, if any of the unmet required attributes are mutable or manageable attributes, the patient's past values for the attribute may be evaluated to determine how likely it is that the patient may satisfy the criteria for the mutable or manageable attribute in the future. The patient's eligibility score may be adjusted based on how far the patient's attribute value is from the required attribute value. For example, where the difference between the patient's attribute value and the required attribute value is large, the patient's eligibility score may be adjusted such that the patient is less likely to ever be eligible even if all other required attributes are met. Conversely, where the distance is smaller the patient's eligibility score may be adjusted such that the patient is more likely to eventually be eligible. Thus where a patient is 150 lbs under the required weight, all else being equal, the patient's eligibility score would be further from the eligibility threshold than if the patient were 2 lbs under the required weight. Atstep 440, the patient's attribute and attribute value history may be evaluated. The evaluation may include a time dimension and account for trends or changes to an attribute value over time. For example, where a patient is above a required weight, but the patient's weight is steadily trending down, the patient's eligibility score may be adjusted to reflect this progress. However, where a patient has an extensive medical history indicating the patient has never satisfied the required attribute value, the trend indicates that the patient is less likely to meet the required attribute values and the patient's eligibility score may be adjusted to reflect this. In comparison, where a patient's attribute value fluctuates over time between meeting and not meeting the required attribute value, the patient's eligibility score may be adjusted such that the patient is more likely to be eligible. Other factors may also be correlated to attribute values over time. Where a record reflects treatments or attempts to adjust an attribute, the eligibility score may be adjusted to take into account these factors. These factors may be associated with attribute values using, for example, data stored in the corpus, such as a medical treatment data. For example, a patient's medical record may indicate that the patient has taken several medications used for treating a particular attribute. Any recorded changes (if any) to the attribute value while the patient is on the medication may indicate how the patient responds to the medication and correlate to how manageable or mutable the attribute value may be for the patient. Treatments associated with managing or mutating the attribute value may also be compared against the required attribute values to avoid treatments with side-effects that may disqualify the patient for the trial. - Additionally, factors that influence the patient's eligibility score may be cumulative. That is, a first patient's fluctuating historical attribute value and a large difference between their attribute value and the required attribute value likely will be less likely to be eligible as compared to a second patient with a similarly fluctuating historical attribute value, but a small difference between their attribute value and the required attribute value. Other factors not directly related to a medical attribute may also be considered. For example, a patient's history may indicate how willing a patient is to travel to obtain treatment, or how willing to consent to various procedures may also be considered. At
step 450, the patient's eligibility score may be returned as a weighted answer based on the eligibility score. -
FIG. 5 is a block diagram illustrating components of a system for use in weighing manageable patient attributes during criteria evaluations for treatment according to one embodiment. In one embodiment, the deep question answering system is theQA application 112. As shown, theQA application 112 contains aquestion classifier component 501, a partial and parallelpipeline execution component 502, and anattribute verification component 503. Thequestion classifier component 501 may parse questions to determine that the question seeks to identify trials in which a patient may be eligible for. Once identified, the topic may be stored in a feature store, such asfeature store 115. The partial and parallelpipeline execution component 502 may, when presented with a case, access the topic defined in thefeature store 115 to reduce processing of sources that do not discuss the topic identified. Theattribute verification component 503 may identify relevant pieces of information from the patient medical records, parse the information, and store these pieces of information in the corpus 114. - The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
- In the preceding, reference is made to embodiments of the disclosure. However, it should be understood that the disclosure is not limited to specific described embodiments. Instead, any combination of the preceding features and elements, whether related to different embodiments or not, is contemplated to implement and practice the disclosure. Furthermore, although embodiments of the disclosure may achieve advantages over other possible solutions and/or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the disclosure. Thus, the preceding aspects, features, embodiments and advantages are merely illustrative and are not considered elements or limitations of the appended claims except where explicitly recited in a claim(s). Likewise, reference to “the invention” shall not be construed as a generalization of any inventive subject matter disclosed herein and shall not be considered to be an element or limitation of the appended claims except where explicitly recited in a claim(s).
- As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
- Any combination of one or more computer readable medium(s) may be used. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
- A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
- Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
- Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
- Aspects of the present disclosure are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
- The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- Embodiments of the disclosure may be provided to end users through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing capability that provides an abstraction between the computing resource and its underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. Thus, cloud computing allows a user to access virtual computing resources (e.g., storage, data, applications, and even complete virtualized computing systems) in “the cloud,” without regard for the underlying physical systems (or locations of those systems) used to provide the computing resources.
- Typically, cloud computing resources are provided to a user on a pay-per-use basis, where users are charged only for the computing resources actually used (e.g. an amount of storage space consumed by a user or a number of virtualized systems instantiated by the user). A user can access any of the resources that reside in the cloud at any time, and from anywhere across the Internet. In context of the present disclosure, a user may access a deep question answering system or related data available in the cloud. For example, the deep question answering system could execute on a computing system in the cloud and provide question classification and feature mapping. In such a case, the deep question answering system could classify questions, map features and store the resultant data sets at a storage location in the cloud. Doing so allows a user to access this information from any computing system attached to a network connected to the cloud (e.g., the Internet).
- While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/692,504 US10431338B2 (en) | 2015-02-25 | 2015-04-21 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/630,751 US10431337B2 (en) | 2015-02-25 | 2015-02-25 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
US14/692,504 US10431338B2 (en) | 2015-02-25 | 2015-04-21 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/630,751 Continuation US10431337B2 (en) | 2015-02-25 | 2015-02-25 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
Publications (2)
Publication Number | Publication Date |
---|---|
US20160246945A1 true US20160246945A1 (en) | 2016-08-25 |
US10431338B2 US10431338B2 (en) | 2019-10-01 |
Family
ID=56690473
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/630,751 Expired - Fee Related US10431337B2 (en) | 2015-02-25 | 2015-02-25 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
US14/692,504 Expired - Fee Related US10431338B2 (en) | 2015-02-25 | 2015-04-21 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/630,751 Expired - Fee Related US10431337B2 (en) | 2015-02-25 | 2015-02-25 | System and method for weighting manageable patient attributes during criteria evaluations for treatment |
Country Status (1)
Country | Link |
---|---|
US (2) | US10431337B2 (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190006024A1 (en) * | 2016-12-23 | 2019-01-03 | Trishul Kapoor | Methods and systems for matching patients with clinical trials |
US20190361980A1 (en) * | 2018-05-22 | 2019-11-28 | International Business Machines Corporation | Inferring confidence and need for natural language processing of input data |
US11176326B2 (en) * | 2019-01-03 | 2021-11-16 | International Business Machines Corporation | Cognitive analysis of criteria when ingesting data to build a knowledge graph |
US11557381B2 (en) * | 2019-02-25 | 2023-01-17 | Merative Us L.P. | Clinical trial editing using machine learning |
US11587647B2 (en) * | 2019-08-16 | 2023-02-21 | International Business Machines Corporation | Processing profiles using machine learning to evaluate candidates |
US20230137326A1 (en) * | 2019-06-18 | 2023-05-04 | Cigna Intellectual Property, Inc. | Machine learning system for generating predictions according to varied attributes |
US20230207071A1 (en) * | 2021-12-29 | 2023-06-29 | Microsoft Technology Licensing, Llc | Knowledge-grounded complete criteria generation |
US20230215552A1 (en) * | 2021-12-31 | 2023-07-06 | Cerner Innovation, Inc. | Early detection of patients for coordinated application of healthcare resources based on bundled payment |
US11790018B1 (en) * | 2022-07-25 | 2023-10-17 | Gravystack, Inc. | Apparatus for attribute traversal |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11263405B2 (en) * | 2018-10-10 | 2022-03-01 | Healthpointe Solutions, Inc. | System and method for answering natural language questions posed by a user |
US11302423B2 (en) * | 2019-01-23 | 2022-04-12 | International Business Machines Corporation | Predicting patients who can benefit from a clinical trial |
US11302424B2 (en) * | 2019-01-24 | 2022-04-12 | International Business Machines Corporation | Predicting clinical trial eligibility based on cohort trends |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU2001265205A1 (en) | 2000-05-31 | 2001-12-11 | Fasttrack Systems, Inc. | Clinical trials management system and method |
US20060025931A1 (en) * | 2004-07-30 | 2006-02-02 | Richard Rosen | Method and apparatus for real time predictive modeling for chronically ill patients |
US20070067189A1 (en) * | 2005-09-16 | 2007-03-22 | Numoda Corporation | Method and apparatus for screening, enrollment and management of patients in clinical trials |
US8265948B2 (en) | 2006-09-29 | 2012-09-11 | Cerner Innovation, Inc. | Proactive and interactive clinical decision support |
US7809660B2 (en) | 2006-10-03 | 2010-10-05 | International Business Machines Corporation | System and method to optimize control cohorts using clustering algorithms |
WO2009076227A2 (en) * | 2007-12-06 | 2009-06-18 | Pain Therapeutics, Inc. | Methods for conducting a clinical trial |
US8239216B2 (en) | 2009-01-09 | 2012-08-07 | Cerner Innovation, Inc. | Searching an electronic medical record |
US20140316793A1 (en) * | 2013-03-14 | 2014-10-23 | nPruv, Inc. | Systems and methods for recruiting and matching patients for clinical trials |
US20120316898A1 (en) * | 2011-06-08 | 2012-12-13 | Levitt Tod S | Scalable determination of probable patient eligibility for clinical trials and associated process for active solicitation of patients for clinical trials via their healthcare providers |
CN102903008B (en) * | 2011-07-29 | 2016-05-18 | 国际商业机器公司 | For the method and system of computer question and answer |
US20130132308A1 (en) * | 2011-11-22 | 2013-05-23 | Gregory Jensen Boss | Enhanced DeepQA in a Medical Environment |
US9208693B2 (en) * | 2012-04-30 | 2015-12-08 | International Business Machines Corporation | Providing intelligent inquiries in question answer systems |
US20130332190A1 (en) * | 2012-06-06 | 2013-12-12 | Cerner Innovation, Inc. | Providing indications of clinical-trial criteria modifications |
US20130332191A1 (en) | 2012-06-06 | 2013-12-12 | Cerner Innovation, Inc. | Identifying patient eligibility for clinical trials |
US20140074510A1 (en) * | 2012-09-07 | 2014-03-13 | Jennifer Clement McClung | Personalized Health Score Generator |
US9443005B2 (en) * | 2012-12-14 | 2016-09-13 | Instaknow.Com, Inc. | Systems and methods for natural language processing |
WO2014133825A1 (en) * | 2013-03-01 | 2014-09-04 | 3M Innovative Properties Company | Classifying medical records for identification of clinical concepts |
US11139050B2 (en) * | 2013-08-23 | 2021-10-05 | Transparency Life Sciences, Inc. | Systems and methods for pre-qualifying clinical trial populations |
-
2015
- 2015-02-25 US US14/630,751 patent/US10431337B2/en not_active Expired - Fee Related
- 2015-04-21 US US14/692,504 patent/US10431338B2/en not_active Expired - Fee Related
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190006024A1 (en) * | 2016-12-23 | 2019-01-03 | Trishul Kapoor | Methods and systems for matching patients with clinical trials |
US20190361980A1 (en) * | 2018-05-22 | 2019-11-28 | International Business Machines Corporation | Inferring confidence and need for natural language processing of input data |
US11010566B2 (en) * | 2018-05-22 | 2021-05-18 | International Business Machines Corporation | Inferring confidence and need for natural language processing of input data |
US11176326B2 (en) * | 2019-01-03 | 2021-11-16 | International Business Machines Corporation | Cognitive analysis of criteria when ingesting data to build a knowledge graph |
US11557381B2 (en) * | 2019-02-25 | 2023-01-17 | Merative Us L.P. | Clinical trial editing using machine learning |
US20230137326A1 (en) * | 2019-06-18 | 2023-05-04 | Cigna Intellectual Property, Inc. | Machine learning system for generating predictions according to varied attributes |
US11848104B2 (en) * | 2019-06-18 | 2023-12-19 | Cigna Intellectual Property, Inc. | Machine learning system for generating predictions according to varied attributes |
US11587647B2 (en) * | 2019-08-16 | 2023-02-21 | International Business Machines Corporation | Processing profiles using machine learning to evaluate candidates |
US20230207071A1 (en) * | 2021-12-29 | 2023-06-29 | Microsoft Technology Licensing, Llc | Knowledge-grounded complete criteria generation |
US20230215552A1 (en) * | 2021-12-31 | 2023-07-06 | Cerner Innovation, Inc. | Early detection of patients for coordinated application of healthcare resources based on bundled payment |
US11790018B1 (en) * | 2022-07-25 | 2023-10-17 | Gravystack, Inc. | Apparatus for attribute traversal |
Also Published As
Publication number | Publication date |
---|---|
US20160246932A1 (en) | 2016-08-25 |
US10431338B2 (en) | 2019-10-01 |
US10431337B2 (en) | 2019-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10431338B2 (en) | System and method for weighting manageable patient attributes during criteria evaluations for treatment | |
US11545173B2 (en) | Automatic speech-based longitudinal emotion and mood recognition for mental health treatment | |
Fonseka et al. | The utility of artificial intelligence in suicide risk prediction and the management of suicidal behaviors | |
US11176325B2 (en) | Adaptive evaluation of meta-relationships in semantic graphs | |
US11200968B2 (en) | Verifying medical conditions of patients in electronic medical records | |
US9141662B2 (en) | Intelligent evidence classification and notification in a deep question answering system | |
US9158773B2 (en) | Partial and parallel pipeline processing in a deep question answering system | |
US9911082B2 (en) | Question classification and feature mapping in a deep question answering system | |
US9087304B2 (en) | Concept noise reduction in deep question answering systems | |
US9773044B2 (en) | Multi-dimensional feature merging for supporting evidence in a question and answering system | |
US9672475B2 (en) | Automated opinion prediction based on indirect information | |
US10031912B2 (en) | Verification of natural language processing derived attributes | |
US9129213B2 (en) | Inner passage relevancy layer for large intake cases in a deep question answering system | |
US11532387B2 (en) | Identifying information in plain text narratives EMRs | |
US20240120109A1 (en) | Artificial intelligence architecture for providing longitudinal health record predictions | |
WO2022233421A1 (en) | Systems and methods for real-time determinations of mental health disorders using multi-tier machine learning models based on user interactions with computer systems | |
US11301772B2 (en) | Measurement, analysis and application of patient engagement | |
Munshi et al. | A deep learning-based approach for predicting COVID-19 diagnosis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ALLEN, CORVILLE O.;PAYTON, ASPEN L.;REEL/FRAME:035461/0923 Effective date: 20150224 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20231001 |