WO2023174779A1 - Methods and systems utilizing objective transfer criteria for improving acute care transitions - Google Patents

Abstract

A method for developing a set of consensus criteria for patient analysis, comprising: (i) generating an initial set of criteria for a selected patient analysis; (ii) recruiting a plurality of experts in the field of the patient analysis; (iii) receiving a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria; (iv) revising the initial set of criteria to generate a revised set of criteria; (v) repeating the receiving and revising steps to generate a final set of criteria; (vi) receiving a parameter vote regarding one or moreof the criterion in the initial set of criteria and/or the final set of criteria for the selected patient analysis; and (vii) generating a consensus set of criteria for the patient analysis.

Description

METHODS AND SYSTEMS UTILIZING OBJECTIVE TRANSFER CRITERIA FOR IMPROVING ACUTE CARE TRANSITIONS

Field of the Invention

[0001] The present disclosure is directed generally to methods and systems for developing a set of consensus criteria for a patient care analysis.

Background

[0002] A growing group of elderly, more fragile and multimorbid patients will increase the future need for intensive care capacities. Adding intensive care unit (ICU) beds to meet this demand is often not an option due to financial limitations and the lack of specialized and highly skilled care givers to staff the beds. Therefore, optimizing the utilization of available ICU resourcesis a high priority for hospital management to avoid bottleneck situations. Capacity strain is often created by pending discharges and flow delays which could account to 15-25% of the total ICU length of stay (LOS). On the other hand, high census levels can cause premature discharges resulting in instable patients at lower levels of care together with an overestimation of the receivingward capacities. That ultimately results in readmissions and even increases capacity strain, LOS, mortality and costs.

[0003] Optimal patient outflow can be achieved by identifying those patients early that are stable enough to transition to the next lower level of care and keeping boarding time to a minimumthrough close cooperation with the receiving unit. Discharging the right patient at the right time reduces LOS, readmission rates, and costs, where an inappropriate discharge will achieve the opposite and increase risk of mortality.

[0004] However, discharge readiness assessment is often rushed, subjective, and not transparent, and is based on a limited amount of available aggregated data and decision criteria. The need for a comprehensive proposal of discharge criteria for adult ICUs that is widely applicable in daily clinical practice throughout Europe has been phrased in a variety of studies, and is even more relevant today in light of the ongoing COVID-pandemic and extremely strainedICU capacities. But what does it take to identify stable patients timely, minimize their boarding time, and ensure safe and efficient care transitions? Basically, the set of criteria for an objective evaluation of patient discharge readiness should satisfy two purposes: First, patient specific criteria such as patient status, interventions and medications, diagnosis, and prognosis as well as a patient’s preferences should indicate a stable state of the patient for at least the next 48 hours that allows safe discharge with a minimized risk of readmission. Second, the set of criteria should incorporate system-specific criteria such as nursing workload related criteria at the discharging and the receiving unit, and institutional factors such as available technical infrastructure, skill sets, patient/nurse ratios, protocols, and processes.

[0005] Providing a consented and standardizable set of criteria for use in daily clinical practice should provide a holistic view to the interdisciplinary care team on individual patient discharge readiness and organizational capabilities. Further, it should guarantee equity in care provision by improving objectivity and comparability in clinical decision making, and increase quality of care transitions and efficient use of ICU capacities in the interest of the patient and the society.

Summary of the Invention

[0006] Accordingly, there is a continued need for systems and methods that generate and utilizea set of consensus criteria for a patient care analysis.

[0007] The present disclosure is directed to an inventive method for consensus criteria development. An initial set of criteria for a selected patient analysis is generated based on a literature review and/or an initial selection by a first expert in the field of the patient analysis. A panel comprising a plurality of experts in the field of the patient analysis is recruited, and the panel makes a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria. Based on the received votes, the initial set of criteria is revised to generate a revised set of criteria, where a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, and where a criterion is excluded from the revised set of criteria if the received exclusion votes are above a predetermined threshold. The voting and revising is repeated one or more times to generate a final set of criteria for the selected patient analysis. The plurality of experts also makes a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final set of criteria for the selected patient analysis, the parameter comprising one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology. A consensus set of criteria for the patient analysis is then generated from the final set of criteria and the received parameter votes.

[0008] Generally in one aspect, a method for developing a set of consensus criteria for patient analysis is provided. The method includes: (i) generating an initial set of criteria for a selected patient analysis, based on a literature review and/or an initial selection by a first expert in the field of the patient analysis; (ii) recruiting a plurality of experts in the field of the patient analysis; (iii) receiving, from each of the plurality of experts, a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria; (iv) revising, based on the received votes, the initial set of criteria to generate a revised set of criteria, wherein a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, and wherein a criterion is excluded from the revised set of criteria if the received exclusion votes are above a predetermined threshold; (v) repeating the receiving and revising steps to generate a final set of criteria for the selected patient analysis; (vi) receiving, from each of the plurality of experts, a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final set of criteria for the selected patient analysis, the parameter comprising one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology; and (vii) generating, from the final set of criteria and the received parameter votes, a consensus set of criteria for the patient analysis.

[0009] According to an embodiment, the method further includes editing, by one or more additional experts in the field of the patient analysis, the generated initial set of criteria for the selected patient analysis.

[0010] According to an embodiment, the recruited plurality of experts in the field of the patient analysis are a diverse plurality.

[0011] According to an embodiment, the recruited plurality of experts in the field of the patient analysis perform their steps of the method online.

[0012] According to an embodiment, the patient analysis is patient discharge readiness.

[0013] According to an embodiment, the criteria vote and the parameter vote are made with anonymity between experts in the plurality of experts. [0014] According to an embodiment, the predetermined threshold for inclusion or exclusion changes from the first criteria vote to a repeated criteria vote. According to an embodiment, the predetermined threshold for exclusion for the initial criteria vote is < 75%, and wherein the predetermined threshold for inclusion for a final criteria vote is > 90%.

[0015] According to an embodiment, the method further includes analyzing a patient status using the consensus set of criteria to generate a patient recommendation. According to anembodiment, the method further includes implementing the generated patient recommendation.

[0016] According to another aspect is a method for treating a patient. The method includes receiving a patient recommendation following an analysis of the patient’s status using a set of consensus criteria, the consensus criteria developed by: (i) generating an initial set of criteria for a selected patient analysis, based on a literature review and/or an initial selection by a first expert in the field of the patient analysis; (ii) recruiting a plurality of experts in the field of the patient analysis; (iii) receiving, from each of the plurality of experts, a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria; (iv) revising, based on the received votes, the initial set of criteria to generate a revised set of criteria, wherein a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, and wherein a criterion is excluded from the revised set of criteria if the received exclusion votes are above a predetermined threshold; (v) repeating the receiving and revising steps to generate a final set of criteria for the selected patient analysis; (vi) receiving, from each of the plurality of experts, a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final set of criteria for the selected patient analysis, the parameter comprising one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology; and (vii) generating, from the final set of criteria and the received parameter votes, a consensus set of criteria for the patient analysis. The method further includes implementing the generated patient recommendation.

[0017] According to another aspect is a method for generating an ICU dischargerecommendation. The method includes: (i) providing a set of ICU discharge readiness consensus criteria, the set of ICU discharge readiness consensus criteria comprising at least the 28 criteria inTABLE 1; (ii) receiving patient input regarding some or all of the set of ICU discharge readiness consensus criteria; (iii) analyzing the received patient input using the set of ICU discharge readiness consensus criteria; (iv) generating, based on the analysis, an ICU discharge readiness recommendation; (v) providing the generated ICU discharge readiness recommendation to a clinician; and (vi) administering the provided ICU discharge readiness recommendation.

[0018] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

[0019] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

Brief Description of the Drawings

[0020] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

[0021] FIG. 1 is a flowchart of a method for developing a set of consensus criteria for patient analysis, in accordance with an embodiment.

[0022] FIG. 2 is a schematic representation of a consensus criteria generation system, in accordance with an embodiment.

[0023] FIG. 3 is a flowchart of a method for implementing a set of consensus criteria for patient analysis, in accordance with an embodiment.

Detailed Description of Embodiments

[0024] The present disclosure describes various embodiments of a method and system for generating a set of consensus criteria for a patient care analysis. More generally, Applicant has recognized and appreciated that it would be beneficial to provide a scientifically rigorous process to arrive at a set of consensus criteria and associated criteria parameters for a patient care analysis. According to an embodiment, an initial set of criteria for a selected patient analysis is generated based on a literature review and/or an initial selection by a first expert in the field of the patient analysis. A panel comprising a plurality of experts in the field of the patient analysis is recruited, and the panel makes a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria. Based on the received votes, the initial set of criteria is revised to generate a revised set of criteria, where a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, and where a criterion is excluded from the revised set of criteria if the received exclusion votes are above a predetermined threshold. The voting and revising is repeated one or more times to generate a final set of criteria for the selected patient analysis. The plurality of experts also makes a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final set of criteria for the selected patient analysis, the parameter comprising one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology. A consensus set of criteria for the patient analysis is then generated from the final set of criteria and the received parameter votes.

[0025] The embodiments and implementations disclosed or otherwise envisioned herein can be utilized with any patient care system, including but not limited to clinical decision support tools, among other systems. For example, one application of the embodiments and implementations herein is to improve analysis systems such as, e.g., the Philips® IntelliSpace® products (manufactured by Koninklijke Philips, N.V.), among many other products. However, the disclosure is not limited to these devices or systems, and thus disclosure and embodiments disclosed herein can encompass any device or system capable of generating and/or utilizing consensus criteria in patient analysis.

[0026] There is a continued need in the art for consensus criteria, including but not limited to consensus criteria for a holistic discharge readiness evaluation tool. As described herein, in order generate a set of consensus criteria, a study group condensed evidence-based criteria and recommendations, structured them, and referenced them in a table format as a first proposal for a set of discharge criteria. This proposal was then subject to a 5-level Delphi study involving a multiprofessional panel of European intensive care experts to reach consensus on a standard set of discharge criteria. Although the methodology is described in reference to the generation of discharge readiness consensus criteria, this example does not limit the scope of the underlying methodology. [0027] Referring to FIG. 1, in one embodiment, is a flowchart of a method 100 for developinga set of consensus criteria for patient analysis using a consensus criteria generation system. The methods described in connection with the figures are provided as examples only, and shall be understood not to limit the scope of the disclosure. The consensus criteria generation system can be any of the systems described or otherwise envisioned herein. The consensus criteria generation system can be a single system or multiple different systems.

[0028] At step 110 of the method, a consensus criteria generation system is provided. Referringto an embodiment of a consensus criteria generation system 200 as depicted in FIG. 2, for example, the system comprises one or more of a processor 220, memory 230, user interface 240, communications interface 250, and storage 260, interconnected via one or more system buses 212. It will be understood that FIG. 2 constitutes, in some respects, an abstraction and that the actual organization of the components of the system 200 may be different and more complex than illustrated. Additionally, consensus criteria generation system 200 can be any of the systems described or otherwise envisioned herein. Other elements and components of the consensus criteria generation system 200 are disclosed and/or envisioned elsewhere herein.

[0029] At step 120 of the method, an initial set of criteria for a selected patient analysis is generated based on a literature review and/or an initial selection by a first expert in the field of thepatient analysis. The literature review can be any review of scientific or other literature, preferablyin the field of the selected patient analysis. The expert or experts can be any expert in the field of the selected patient analysis. The patient analysis can be any analysis for patient care that utilizesone or more criteria for the analysis. For example, the analysis can be patient discharge or admission readiness, patient risk analysis, patient treatment analysis, patient diagnosis, or any other analysis. The initial set of criteria can comprise one or many criteria, and the criteria can berelated or unrelated to each other. Notably, one or more of the criterion can also include a criterionparameter. For example, a criterion parameter may be a criterion importance ranking, a criterion range or value, a criterion evaluation time, or a criterion calculation or evaluation methodology, among many other criterion parameters. Once generated, the initial set of criteria can be utilized or deployed immediately, or it may be stored in local and/or remote memory for future use and/or deployment. [0030] At optional step 122 of the method, the generated initial set of criteria for the selected patient analysis can be edited by one or more (additional) experts in the field of the patient analysis. For example, once the initial set of criteria is generated from the literature review or by an initial expert(s), that initial set of criteria can be edited by more experts before downstream voting. Editing can comprise, for example, the removal of a criterion, the addition of a criterion, editing a parameter of a criterion, or another edit.

[0031] At step 130 of the method, a plurality of experts in the field of the patient analysis are recruited. The plurality of experts can be two or more experts. Diversity of the expert panel may increase the robustness of the final generated set of consensus criteria. According to an embodiment, the experts can be identified and recruited using a wide variety of methods. For example, potential experts can be identified based on a combination of proven research activities (e.g., topic-related publications in PubMed and on ResearchGate, topic related congress presentations, and through peer recommendation of being a practice specialist in the specific field. In addition, the expert panel can be targeted to represent a diversity of countries, regions, and/or healthcare systems as well as a balance in professions. Communication with potential experts can be performed using a wide variety of communication methods. For example, email invitations with background and introduction to the study can be sent to the potential panelists. Upon acceptance to participate, enrolled participants can receive one or more documents or other preparatory information, including personalized sign-in credentials to access an online tool facilitating the remainder of the process.

[0032] At step 140 of the method, a first criteria vote for inclusion or exclusion of each criterionin the initial set of criteria is received from each of the plurality of experts. If an expert fails to vote, that expert can be removed from the panel. The first criteria vote can be made via any methodfor voting, including but not limited to in-person voting, online voting, and other voting forms. Voting equipment and/or software can be utilized to facilitate the vote. The first criteria vote can be made with anonymity with regard to the other experts, meaning that each expert in the plurality of experts will not be made aware of the vote of each of the other of the experts.

[0033] At step 150 of the method, the initial set of criteria is revised based on the received votesto create a revised set of criteria. According to an embodiment, a criterion is included in the revisedset of criteria if the received inclusion votes are above a predetermined threshold, and wherein a criterion is excluded from the revised set of criteria if the received exclusion votes are above a predetermined threshold. The predetermined thresholds can be derived experimentally, based on known statistical methods, obtained from the literature, and/or generated by experts, among other methods. As just one example, for an initial round of voting a predetermined threshold of at least 75% of votes voting for inclusion may be necessary for inclusion of the criterion in the revised set of criteria. However, lower (such as at least 50% or more) or higher (such as at least 90% or more) thresholds are possible. Once generated, the revised set of criteria can be utilized or deployed immediately, or it may be stored in local and/or remote memory for future use and/or deployment.

[0034] At step 160 of the method, steps 140 (voting) and 150 (revising) are repeated one or more times. This can be a single repeated round of voting and revision, or two or more additionalrounds of voting and revision. The number of rounds of voting can be based on a predetermined number of rounds, or can be derived experimentally, based on known statistical methods, obtainedfrom the literature, and/or generated by experts, among other methods. The predetermined thresholds utilized for inclusion or exclusion during each voting round may be the same as a previous round or may be modified, including raising or lowering the predetermined threshold.

[0035] At step 170 of the method, which can occur before, concurrent with, or after any of the rounds of voting described or otherwise envisioned herein, the plurality of experts vote on a parameter of one or more of the criteria, such as the initial set of criteria, a revised set of criteria, and/or a final set of criteria for the selected patient analysis. A parameter is any metric or associated element or measure of a criterion. For example, the parameter can comprise one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology, among many other parameters.

[0036] At step 180 of the method, a consensus set of criteria for the patient analysis is generatedfrom the final set of criteria and received parameter votes. Generation of the final set of criteria may comprise one or more thresholds, such as an inclusion threshold and/or a parameter threshold. Once generated, the consensus set of criteria can be utilized or deployed immediately, or it may be stored in local and/or remote memory for future use and/or deployment.

[0037] At optional step 182 of the method, the consensus set of criteria is implemented and utilized to analyze a patient status in order to generate a patient recommendation. As just one example, the consensus set of criteria may be implemented in a patient analysis tool such as a clinical decision support tool, which may be implemented at one or more locations such as a clinical care facility like a hospital. For example, the clinical decision support tool may analyze a patient risk and generate a patient risk recommendation such as a cardiological analysis, among many other implementations. Accordingly, the consensus set of criteria may be utilized by the clinical decision support tool to perform a patient analysis and generate a patient recommendation. Once generated, the recommendation can be utilized or deployed immediately, such as providing the patient recommendation via a user interface, or it may be stored in local and/or remote memory for future use and/or deployment.

[0038] At optional step 184 of the method, a clinician implements the generated patient recommendation. For example, for a clinical decision support tool may analyze a patient risk and generate a patient risk recommendation such as a cardiological risk. Based on the patient’s cardiological risk as generated from the consensus set of criteria, the clinician implements a treatment to prevent or minimize the patient’s risk. For example, the clinical decision support tool may analyze a patient risk and reveal a very high cardiological risk. The clinician may receive that generated risk and implement a plan to prevent or minimize the risk, such as providing blood pressure management or medication to raise or lower blood pressure, providing heart rate management or medication to lower or raise heart rate, among many other treatments.

[0039] Thus, according to one implementation of the method, which is a non-limiting exampleof the methods and systems envisioned herein, a group of consensus criteria are generated. In the online expert group consensus can be built through an iterative process that uses systematic progression through two to five rounds of voting on questions, statements, or criteria. The enrolled panelists can vote on every criterion for inclusion in a standard set of discharge criteria that is necessary and suitable to evaluate individual patient’s discharge readiness for adult patients in any type of intensive care setting and not specific to any individual disease process or specialty. Consensus can be defined through agreement on proposed criteria by > 90% of the panelists. For the first one to three rounds, exclusion of criteria from subsequent rounds can be defined by reaching < 75% of agreement per criterion. In round one, agreement for inclusion for round two can be defined per criterion through not editing / leaving as is, editing, or adding a criterion to the list. The remark “removal ” per criterion can be counted as disagreement and a criterion would have been removed from the list if > 25% of the experts voted for removal in round one. In round two, agreement for further inclusion of a criterion in the list can be defined as answering either very relevant or relevant per criterion on the provided Likert scale (5 values: “very relevant”, “relevant”, “cannot judge”, “not relevant”, “completely irrelevant”). Criteria that met consensus of > 90% through answers of “very relevant” or “relevant” can be already approved to enter round four. Criteria that reached agreement between 75 - 89% on being “very relevant” or “relevant”, can go into round three, where experts can be confronted with the voting results on group level compared to their own results and outliers have the chance to change their vote towards the groups opinion. In round four, only those criteria having reached consensus in round two and the additional criteria finally having reached consensus of > 90% in round three, can go through further fine-tuning on criteria importance rank, criteria evaluation time frames, specific values, and calculation method as well who from the care team could best evaluate per criterion if it has been met. For round five, the panelists can receive a statistical summary report of the group voting on criteria phrasing, importance ranking, evaluation time windows, criteria calculation, and preferred decision makers per criterion. Further, they can be asked to agree / disagree on the final criteria phrasing, value calculation method, and criteria importance ranking. The aspects “criteria evaluation time frame” and “who could best evaluate if a specific criterion has been met” can be excluded from further voting due to the heterogeneity of the round four results. Based on the voting results and provided comments, the investigators can derive the final list of discharge criteria and share the results with the expert panel.

[0040] Referring to FIG. 2 is a schematic representation of a consensus criteria generation system 200. System 200 may be any of the systems described or otherwise envisioned herein, andmay comprise any of the components described or otherwise envisioned herein. It will beunderstood that FIG. 2 constitutes, in some respects, an abstraction and that the actual organization of the components of the system 200 may be different and more complex than illustrated.

[0041] According to an embodiment, system 200 comprises a processor 220 capable ofexecuting instructions stored in memory 230 or storage 260 or otherwise processing data to, for example, perform one or more steps of the method. Processor 220 may be formed of one or multiple modules. Processor 220 may take any suitable form, including but not limited to a microprocessor, microcontroller, multiple microcontrollers, circuitry, field programmable gate array (FPGA), application-specific integrated circuit (ASIC), a single processor, or plural processors. [0042] Memory 230 can take any suitable form, including a non-volatile memory and/or RAM.The memory 230 may include various memories such as, for example LI, L2, or L3 cache or system memory. As such, the memory 230 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read only memory (ROM), or other similar memory devices. The memory can store, among other things, an operating system. The RAM is used by theprocessor for the temporary storage of data. According to an embodiment, an operating system may contain code which, when executed by the processor, controls operation of one or more components of system 200. It will be apparent that, in embodiments where the processor implements one or more of the functions described herein in hardware, the software described as corresponding to such functionality in other embodiments may be omitted.

[0043] User interface 240 may include one or more devices for enabling communication with a user. The user interface can be any device or system that allows information to be conveyed and/or received, and may include a display, a mouse, and/or a keyboard for receiving user commands. In some embodiments, user interface 240 may include a command line interface or graphical user interface that may be presented to a remote terminal via communication interface 250. The user interface may be located with one or more other components of the system, or may located remote from the system and in communication via a wired and/or wireless communications network.

[0044] Communication interface 250 may include one or more devices for enabling communication with other hardware devices. For example, communication interface 250 may include a network interface card (NIC) configured to communicate according to the Ethernet protocol. Additionally, communication interface 250 may implement a TCP/IP stack for communication according to the TCP/IP protocols. Various alternative or additional hardware or configurations for communication interface 250 will be apparent.

[0045] Storage 260 may include one or more machine-readable storage media such as read- only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash-memory devices, or similar storage media. In various embodiments, storage 260 may store instructions for execution by processor 220 or data upon which processor 220 mayoperate. For example, storage 260 may store an operating system 261 for controlling various operations of system 200. [0046] It will be apparent that various information described as stored in storage 260 may be additionally or alternatively stored in memory 230. In this respect, memory 230 may also be considered to constitute a storage device and storage 260 may be considered a memory. Various other arrangements will be apparent. Further, memory 230 and storage 260 may both be considered to be non-transitory machine-readable media. As used herein, the term non-transitory will be understood to exclude transitory signals but to include all forms of storage, including both volatile and non-volatile memories.

[0047] While system 200 is shown as including one of each described component, the various components may be duplicated in various embodiments. For example, processor 220 may include multiple microprocessors that are configured to independently execute the methods described herein or are configured to perform steps or subroutines of the methods described herein such that the multiple processors cooperate to achieve the functionality described herein. Further, where one or more components of system 200 is implemented in a cloud computing system, the various hardware components may belong to separate physical systems. For example, processor 220 may include a first processor in a first server and a second processor in a second server. Many other variations and configurations are possible.

[0048] According to an embodiment, literature corpus 270 is a body of literature that may include publications, studies, or other literature regarding the selected patient analysis. The literature corpus may be a single database of literature, or multiple databases of literature. For example, the corpus may comprise scientific sources of literature such as PubMed, Google Scholar, and other databases. The databases may be public and/or private. The database may be alocal or remote database and is in direct and/or indirect communication with system 200.

[0049] According to an embodiment, storage 260 of system 200 may store one or more algorithms, modules, and/or instructions to carry out one or more functions or steps of the methodsdescribed or otherwise envisioned herein. For example, the system may comprise, among other instructions or data, an initial criteria set 262, voting instructions 263, and/or a set of revised, final, and/or consensus criteria 265.

[0050] According to an embodiment, the initial criteria set 262 comprises an initial set of criteria for a selected patient analysis generated based on a literature review and/or an initial selection by a first expert in the field of the patient analysis. The literature review can be any review of scientific or other literature, preferably in the field of the selected patient analysis. The expert or experts can be any expert in the field of the selected patient analysis. The patient analysis can be any analysis for patient care that utilizes one or more criteria for the analysis. Notably, one or more of the criterion can also include a criterion parameter. For example, a criterion parameter may be a criterion importance ranking, a criterion range or value, a criterion evaluation time, or a criterion calculation or evaluation methodology, among many other criterion parameters. Once generated, the initial set of criteria can be utilized or deployed immediately, or it may be stored in local and/or remote memory for future use and/or deployment.

[0051] According to an embodiment, voting instructions 263 direct the system to provide voting information and/or instructions to the recruited plurality of experts, and/or to receive votesfrom the recruited plurality of experts. The voting instructions may interface with voting software, or may be the voting software. The voting instructions may statistically analyze the votes, and canprovide reports of the voting outcomes along with a statistical analysis. The voting instructions may comprise the predetermined thresholds describe or otherwise envisioned herein.

[0052] According to an embodiment, the set of revised, final, and/or consensus criteria 265 comprises any of the consensus criteria other than the initial set of criteria. For example, criteria 265 may be the first set of revised criteria formed after the first round of voting, or a subsequent set of criteria formed during a round of voting. Criteria 265 may be the final set of criteria formed after the final round of criteria voting. Criteria 265 may be the consensus set of criteria formed after both the final round of criteria voting and associated parameter voting. Once generated, a set of criteria can be utilized or deployed immediately, or it may be stored in local and/or remote memory for future use and/or deployment.

[0053] According to an embodiment, the consensus criteria generation system is configured to process many thousands or millions of datapoints in the input data used to perform the steps of the method. For example, scouring literature, forming criteria sets, providing and receiving voting for criteria and parameters, and performing statistical analysis, among other steps, requires processing of millions of datapoints. This can require millions or billions of calculations to generate a novel set of consensus criteria from those millions of datapoints and millions or billions of calculations. Thus, generating this novel set of consensus criteria comprises a process with a volume of calculation and analysis that a human brain cannot accomplish in a lifetime, or multiple lifetimes. [0054] By providing the novel set of consensus criteria for patient analysis as described or otherwise envisioned herein, this novel patient analysis system has an enormous positive effect on patient analysis and care compared to prior art systems.

[0055] EXAMPLE

[0056] The following is a non-limiting example of a method for patient analysis using a set of consensus criteria, in accordance with the methods and systems described or otherwise envisioned herein. In this example, the set of consensus criteria apply to the evaluation of discharge readiness for adult ICU patients to be discharged to a general ward.

[0057] Discharge decisions in Intensive Care Unit (ICU) patients are frequently taken under pressure to free up ICU beds. In the absence of established guidelines, the evaluation of discharge readiness commonly underlies subjective judgements. The challenge is to come to the right decision at the right time for the right patient. A premature care transition puts patients at risk of readmission to the ICU. Delayed discharge is a waste of resources and may result in over-treatment and suboptimal patient flow. More objective decision support is required to assess the individual patient’s discharge readiness but also the current care capabilities of the receiving unit.

[0058] Methods

[0059] According to the modified Delphi process, an international panel of 27 intensive care experts reached consensus on a set of 28 intensive care discharge criteria. An initial evidencebased proposal was developed further through the panelists’ edits, adding, comments and voting over a course of 5 rounds. Consensus was defined as achieved when > 90% of the experts voted for a given option on the Likert scale or in a multiple-choice survey. Round 1 to 3 focused on inclusion and exclusion of the criteria based on the consensus threshold, where round 3 was a reiteration to establish stability. Round 4 and 5 focused on the exact phrasing, values, decision makers and evaluation time frames per criterion.

[0060] Results

[0061] Consensus was reached on a standard set of 28 ICU discharge criteria for adult ICU patients, that reflect the patient’s organ systems ((respiratory (7), cardiovascular (9), central nervous (1), and urogenital system (2)), pain (1), fluid loss and drainages (1), medication and nutrition (1), patient diagnosis, prognosis and preferences (2) and institution-specific criteria (4). All criteria have been specified in a binary decision metric (fit for ICU discharge vs. needs further intensive therapy/monitoring), with consented value calculation methods where applicable and a criterion importance rank with “mandatory to be met” flags and applicable exceptions.

[0062] Conclusion

[0063] For a timely identification of stable intensive care patients and safe and efficient care transitions, a standardized discharge readiness evaluation should be based on patient factors as well as organizational boundary conditions, and involve multiple stakeholders.

[0064] Background

[0065] A growing group of elderly, more fragile and multimorbid patients will increase the future need for intensive care capacities. To meet this demand, adding more intensive care unit (ICU) beds is often not an option due to financial limitations and the lack of specialized and highly skilled care givers to staff the beds. Therefore, optimizing the utilization of given ICU resources is a high priority for hospital management to avoid bottleneck situations. Capacity strain is often created by pending discharges and flow delays which could account to 15-25% of the total ICU length of stay (LOS). On the other hand, high census levels can cause premature discharges resulting in instable patients at lower levels of care together with an overestimation of the receivingward capacities. That ultimately results in readmissions and even increases capacity strain, LOS, mortality and costs. Optimal patient outflow can be achieved by identifying those patients early that are stable enough to transition to the next lower level of care and keeping boarding time to a minimum through close cooperation with the receiving unit. Discharging the right patient at the right time reduces LOS, readmission rates, and costs, where an inappropriate discharge will achieve the opposite and increase risk of mortality. Until today, discharge readiness assessment isoften rushed, subjective and untransparent, and based on a limited amount of available aggregateddata and decision criteria. The need for a comprehensive proposal of discharge criteria for adult ICUs that is widely applicable in daily clinical practice throughout Europe has been phrased in a variety of studies, and is even more relevant today in light of the ongoing COVID-pandemic and extremely strained ICU capacities. But what does it take to identify stable patients timely, minimize their boarding time and ensure safe and efficient care transitions? Basically, the set of criteria for an objective evaluation of patient discharge readiness should satisfy two purposes: First, patient specific criteria such as patient status, interventions and medications, diagnosis and prognosis as well as patient’s preferences should indicate a stable state of the patient for at least the next 48 hours that allows safe discharge with a minimized risk of readmission. Second, the set of criteria should incorporate system-specific criteria such as nursing workload related criteria at the discharging and the receiving unit, and institutional factors such as available technical infrastructure, skill sets, patient/nurse ratios, protocols and processes.

[0066] The absence of such a holistic discharge readiness evaluation tool was the motivation behind this study. The study group condensed evidence-based criteria and recommendations, structured and referenced them in a table format as a first proposal for a set of discharge criteria. This proposal was then subject to a 5-level Delphi study involving a multi -professional panel of European intensive care experts to reach consensus on a standard set of discharge criteria. Providing a consented and standardizable set of criteria for use in daily clinical practice should provide a holistic view to the interdisciplinary care team on individual patient discharge readiness and organizational capabilities. Further, it should guarantee equity in care provision by improving objectivity and comparability in clinical decision making, and increase quality of care transitions and efficient use of ICU capacities in the interest of the patient and the society.

[0067] Methods

[0068] With the study aim to reach consensus on a standardized set of ICU discharge criteria, the research group selected a modified online Delphi process. The Delphi method was chosen as it is a suitable research tool, specifically in areas where there is limited scientific evidence, lack of agreement, incomplete knowledge or uncertainty and conclusions are heavily relying on expert opinion. In this case, it should help to build on the limited scientific evidence for established and well-defined ICU discharge criteria that was identified by the previously performed scoping literature review. Further, the Delphi technique had four main characteristics that suited the objective of this study: anonymity between participants through a non-face-to-face format, iteration with controlled feedback of group opinion, statistical aggregation of group responses and expert input. The study was realized via an online voting platform designed for Delphi studies (welphi.com) to include a geographically spread panel of experts and to allow participation in an asynchronous, online, participatory and interactive way, at comparatively low cost and time investment. [0069] Generally, the Delphi study was conducted in three stages: 1.) Scoping literature review, criteria preselection and panelist recruitment, 2.) Online Delphi process (detailed process description in online supplement b, doc. 2), 3.) Conclusion on final results.

[0070] In the 1^st stage, the investigators derived a preselection of criteria from the earlier performed scoping literature review that was then reviewed by selected experts for suitability and comprehensiveness. Items, values and ranges were added or edited where applicable. Scientific evidence was referenced per criterion and a first proposal of an ICU discharge criteria checklist was structured by criteria categories (online supplement a, doc. 1, tab. SI). Potential experts were identified based on a combination of proven research activities (topic-related publications in Pubmed and on ResearchGate, topic related congress presentations (ESICM, ISICEM) and through peer recommendation of being a practice specialist in the specific field. In addition, the investigators aimed for an expert panel representing a diversity of European countries and healthcare systems as well as a balance in professions. Email invitations with background and introduction to the study were sent out to the potential panelists. Upon acceptance to participate, enrolled participants received a pre-read document (online supplements a, doc. 1) and their personalized sign-in credentials to access the online Delphi tool.

[0071] In the online Delphi process (2^nd stage), expert group consensus was built through an iterative process that used systematic progression through five rounds of voting on questions, statements, or criteria in this case. The individual rounds of the process were set-up based on available literature on Delphi studies in similar settings, process reviews with appointed methodologists and the technical support team of the online Delphi platform. The enrolled panelists voted on every criterion for inclusion in a standard set of discharge criteria that is necessary and suitable to evaluate individual patient’s discharge readiness for adult patients in any type of European intensive care setting and not specific to any individual disease process or specialty. The investigators did not actively participate in the online Delphi process, but reviewed, summarized and discussed the results after each round in order to set up the following rounds. Consensus was defined through agreement on proposed criteria by > 90% of the panelists. For the first three rounds, exclusion of criteria from subsequent rounds was defined by reaching < 75% of agreement per criterion with the cut-off value oriented on comparable research. In round 1, agreement for inclusion for round 2 was defined per criterion through not editing / leaving as is, editing, or adding a criterion to the list. The remark “removal ” per criterion was counted as disagreement and a criterion would have been removed from the list if > 25% of the experts voted for removal in round 1. In round 2, agreement for further inclusion of a criterion in the list was defined as answering either very relevant or relevant per criterion on the provided Likert scale (5 values: “very relevant”, “relevant”, “cannot judge”, “not relevant”, “completely irrelevant”). Criteria that met consensus of > 90% through answers of “very relevant” or “relevant” were already approved to enter round 4. Criteria that reached agreement between 75 - 89% on being “very relevant” or “relevant”, went into round 3, where experts were confronted with the voting results on group level compared to their own results and outliers had the chance to change their vote towards the groups opinion. In round 4, only those criteria having reached consensus in round2 and the additional criteria finally having reached consensus of > 90% in round 3, went through further finetuning on criteria importance rank, criteria evaluation time frames, specific values, and calculation method as well who from the care team could best evaluate per criterion if it has been met. For round 5, the panelists received a statistical summary report of the group voting on criteria phrasing, importance ranking, evaluation time windows, criteria calculation, and preferred decision makers per criterion. Further, they were asked to agree / disagree on the final criteria phrasing, value calculation method and criteria importance ranking. The aspects “criteria evaluation time frame” and “who could best evaluate if a specific criterion has been met” where excluded from further voting due to the heterogeneity of the round 4 results (online supplements c, results round 4). Based on the voting results and provided comments, the investigators derived the final list of discharge criteria, shared the results with the expert panel and prepared the study manuscript for publication.

[0072] In the 3rd stage, based on the voting results of the 5th round and the provided comments, the team of investigators concluded on the final list of 28 ICU discharge criteria with their binary decision metric (values for Fit for ICU discharge and Needs further intensive care therapy / monitoring). Where appropriate - the preferred value calculation method to evaluate discharge was proposed, and a criterion importance ranking was provided with exceptions when the mandatory to be met - requirement could be overruled.

[0073] Results

[0074] Of the 54 experts (33 medical doctors and 21 nurses) approached, 17 doctors (2 female)and 11 nurses (6 female) agreed to participate. One participant left the panel after the second round. The final panel represented 12 different healthcare systems (11 European, one Canadian) and brought in experience from different ICU types and sizes and professional backgrounds. Participation with completion of the entire survey per round ranged from 78-90% completion rate over the five rounds of voting.

[0075] At the end of the Delphi process, 28 ICU discharge criteria reached final consensus, thereof 23 criteria reached consensus > 90% and 5 criteria reached consensus of 75 - 88%. For 5 criteria a deletion from the list was agreed with > 92% consensus, because these criteria became obsolete as aspects were finally covered through other rephrased criteria. Based on comments received in the 5^th round and discussion among the team of investigators, 8 criteria got changed inphrasing for simplification for the final criterial list although they have all met already the consensus threshold. The set of proposed ICU discharge criteria for adult patients in any type of ICU covers patientspecific as well as organization-specific discharge criteria to evaluate dischargereadiness of the individual patient but also to assess organizational capabilities to allow patient discharge to the next lower level of care (general ward in this study context). 24 patient-specific criteria reflect on the different organ systems (respiratory system, cardiovascular system, central nervous system, urogenital system), pain, fluid loss and drainages, medication and nutrition, patient diagnosis, prognosis, and patient preferences. 8 out of the 24 patient-specific criteria evaluate as part of their criteria phrasing whether currently available capacities and competencies as well as available technical infrastructure at the receiving unit are in place to safely discharge the patient. 4 criteria focus on the institution specific boundary conditions that allow or don’t allowpatient discharge (institution’s specific admission criteria of receiving unit, safety standards such as isolation or support measures for out of hours discharges, and current capacity, acuity, and workload levels at the receiving unit). For every criterion a binary decision metric was defined with values for “Fit for discharge” and “Needs further intensive care therapy / monitoring”. For consistency and ease of use in daily clinical practice, all criteria have been phrased in a way that they can be answered with the values “yes” or “no” in the binary decision metric. For 6 criteria concerning the patient’s vital parameters, a value calculation method has been defined: For “bloodoxygenation”, the worst value must be above the defined threshold and the trend must be stable over a defined time frame. For “respiratory rate”, “heart rate” and “mean arterial pressure”, the worst value must be within the acceptable range and the trend must be stable over a defined time frame. For “cardiac rhythm” and “hemoglobin” the trend must be stable over a defined time frame. For a definition of an appropriate value evaluation time frame per criterion that indicates stability, thus preventing patient readmission within 48 hours after ICU discharge, agreement within one of the proposed time frame categories was low. 42% has been reached as a maximum agreement level in only one criterion, followed by 2 criteria reaching 38% and 8 criteria reaching 33% agreement in one category. All other criteria reached lower agreement levels per proposed evaluation time frame category. Based on the heterogeneity of the answers, the investigators decided to take this aspect out from any further voting round as they didn’t expect reaching a significantly higher agreement level within the following two rounds. As a result, no specific criterion evaluation time frames can be proposed. However, as the phrasing of the value calculation method refers to a defined time frame, for implementation in daily clinical routine, this time frame needs to be specified once by the clinical decision makers and in context of the institution specific workflows of vital parameter measurement, documentation, and visualization. The votes distribution per criterion could serve as a first orientation.

[0076] Also, the question in the fourth round, who of the stakeholder group can evaluate best if the patient meets a criterion, resulted in very heterogenous results. Only for 9 out of 33 criteria an agreement of > 70% was reached on one stakeholder. A general insight from the panelists’ comments and the voting distribution was, that a lot of criteria need to be evaluated by an interdisciplinary team, that often should even involve a clinician or nurse from the receiving unit. In the performed survey, there was no selection option for the interdisciplinary team. That is why panelists commented via the comments field on the need to have an interdisciplinary team (13 related comments). Based on these results, it was decided to not further iterate on the decision maker. The investigators would rather recommend for criteria implementation in daily clinical practice to define the best decision maker(s) per criterion based on institution specific roles and collaboration aspects. Based on the comments, for some criteria certain roles can equally assess discharge readiness, often depending more on competency and experience level than on the specific role. An interdisciplinary team including representatives from the receiving unit should be consulted for criteria that reflect on patient specific criteria in context with capabilities of the receiving unit. For clinical practice implementation, it is also important to know the importance rank of each criterion, meaning if it is mandatory to meet the criterion or not and if there are exceptions in place that allow to overrule the criterion. After some simplification of the ranking, 17 out of 28 criteria reached > 90% consensus on “mandatory to be met”. 11 criteria reached 73% - 89% consensus on “mandatory to be met”. End-of-life care and agreed treatment limits where the exceptions mentioned the most (in 12 criteria), when “mandatory to be met” can be overruled. Looking at the voting alternatives, the highest vote for “not mandatory to be met” with 26% was reached for the criterion “Patient’s preference is to stop intensive care therapy and to leave the ICU”.

[0077] Finally, over a course of 5 rounds of voting on a basis of 40 initial ICU discharge criteria, where several criteria have been added, edited and deleted along the way through the panel’s consensus, the study resulted in 28 in-depth defined ICU discharge criteria for adult patients, that should be applicable to any type of ICU.

[0078] Discussion

[0079] The objective of this Delphi study to reach consensus on a standardized set of ICU discharge criteria has been achieved. In the voting process, 28 clinicians and nurses were involved with a dedicated expertise and research interest in ICU transfer processes, coming from divers geographical and professional backgrounds. That allowed the process to get a comprehensive evaluation of the provided first proposal of ICU discharge criteria that were derived from previous literature research.

[0080] From the beginning, patient as well as process-related conditions were in focus of the consensus process. As phrased in earlier studies, only the two perspectives together could give a holistic view on discharge readiness for an individual patient to a particular next lower level of care. Patient-specific criteria should indicate a stable state of the patient for at least the next 48 hours that allows safe discharge with a minimized readmission risk. Organization-specific aspects affect the quality and success of care transitions from an ICU to a lower level of care and should reflect on current ICU capacity strain driven by census level and nursing workload as well as the capabilities of the next lower level of care, communication and handover practices between departments, as well as an early definition of patient-centric care goals that are aligned throughout the patient pathway.

[0081] The first round of voting was started with two blocks of criteria: the patient-specific block with 30 criteria and the organization-specific block with 10 criteria. The study wassuccessfully ended after the 5th round with 24 patient-specific and 4 organization-specific criteria, all with a consensus level of > 90%. Patient-specific criteria were represented by a holistic view on the different organ systems and therapeutic interventions as well as patient’s autonomy, continuous care needs, the patient’s wish, and therapeutic susceptibility. Organization-specific criteria were aligned institution specific admission and discharge criteria, discharge timing and accompanying safety measures, available technology, and care capacities at the next lower level of care. Throughout the various rounds of voting, it became clear that patient- and organizationspecific discharge readiness is deeply intertwined as among the patient-specific criteria, 12 criteria also reflect on the capabilities of the receiving unit in the way they were finally phrased.

[0082] Multi-parameter and multi-stakeholder approach

[0083] With its multi-parameter and multi-stakeholder approach, this study uniquely corresponds to the request from several publications, that ICU discharge criteria should not only determine when a patient is no longer in need of intensive care but also whether the receiving unitis capable to take appropriate care of that particular patient. Here, the checklist format could serve as a structured decision support tool to evaluate both perspectives and to facilitate a handshake on patient transition between the sending and receiving unit, thus improving current discharge practices. The high completion rate through all five rounds supports robust results and reflects the interest and dedication of the panelists for this area of research. Throughout the voting process, the panelists’ comments, and perspectives, based on their diverse geographical and professional background, lead to a comprehensive result. ICU clinicians and critical care nurses were purposely in the definition of ICU discharge criteria as the formalization of multidisciplinary input in the ICU discharge decision-making has been recommended in earlier work. Research has also repeatedly shown that bedside nurses can offer a unique perspective on the type and amount of nursing care each patient needs, and nurse-physician collaboration in decision-making at the time of ICU discharge is associated with better patient outcomes, reduction in ICU readmission and hospital mortality. Those studies also proposed that discharge readiness evaluation needs to consider nursing workload related criteria at the discharging and at the receiving unit, as well as available skill sets and patient-nurse ratios. Surprisingly, none of the proposed scores or ratios reflecting on nursing workload or patient-nurse ratio were finally consented. One criterion was formulated rather generic without any patient-nurse ratio, acuity level or workload scores, but specifically to the care capacities at the receiving unit (“Do current acuity and dependency levels and current workload at the receiving unit allow to admit and take care of this patient?”). This particular criterion would help to facilitate a discussion and ultimate handshake for the care transition but leave it still to the decision-making subjectivity and argumentation skills of the different stakeholders. For many of the other criteria in the final list, it was reflected per criterion if the patient status or the needed continuous interventions could be managed at the receiving unit. Having such a focus on the organizational aspects of discharge readiness, where different units need to evaluate if the discharge criteria are met, also brings the challenge of structuring access to the clinical decision support tool (“Who ticks the box?”) and organizing information flow around it (“Who gets notified?”). The questions, who can best evaluate the particular discharge criterion and who ultimately takes the discharge decision, leads to another remarkable result of the study: No clear preference for a particular decision maker per criterion and several comments to better have an interdisciplinary team deciding whether the criteria are met, also supports a multistakeholder approach in clinical decision making. Retrospectively, there should have been a selection option in the questionnaire for “interdisciplinary team”, which wasn’t there but would have potentially brought even clearer results for the multi-stakeholder approach. Another insight was, that clinicians and nurses from the general ward environment should have been included in the panel. Especially, as so many criteria were reflecting on the capabilities of the receiving unit, one assumes that their input and vote would have brought an even more comprehensive result. With that insight, clinical implementation and validation of the criteria should be undertaken by a multidisciplinary and interdepartmental team.

[0084] High consensus level with the focus on generally available and applicable criteria

[0085] This study results could serve as a starting point for implementation in daily clinical practice in many European healthcare systems. The definition of a rather high consensus level with > 90% for criteria inclusion and 5 rounds of partially reiterations of voting enabled consistency checks and fine-tuning of the proposed criteria. Throughout the different rounds of voting, the panelists were focused on the aim that the criteria should be widely available in clinical practice and as broadly applicable as possible, concerning the patient group, type of illness and type of ICU. Further, the inclusion of criteria around patient wishes, prognosis and therapeutic susceptibility, as well as the criteria importance ranking including the exceptions in case of a palliative care pathway, builds a standardized form that was asked for in earlier studies to evaluate the adequacy of the current treatment with the team, the patient and his relatives. However, the results of the criteria importance ranking were quite heterogenous and therefore should serve as a first orientation whether some criteria are more important to be met than others. Implementation in clinical routine needs to show how useful the criteria importance ranking is. Capturing retrospectively how often a “mandatory to be met” criteria was actually not met when a patient was discharged and combining that with patient outcome data will bring more insights into how to apply this rule and how to further fine-tune the weighting of the criteria against each other or even combining some.

[0086] Aspects for implementation in daily clinical practice

[0087] A possible operationalization of the criteria list could be in a kind of dashboard view inthe Patient Data Management System (PDMS), where discharge readiness status is visualized as a summary visual per patient. There, it could automatically flag patients that are “fit for discharge”, when the required criteria thresholds are met over the institution-specific defined time frames. That would help the care team to quickly assess current capacity requirements when they are asked to admit a new patient. Furthermore, in a single patient view, the different factors impacting discharge readiness can be reviewed in more detail and current discharge barriers can be depicted. For hospitals still documenting in paper-based formats, a color-coded discharge readiness checklist could support individual patient assessment. This perspective could guide morning rounds to focus the attention to the most likely-dischargeable patients and on given discharge barriers. Further, the variety in clinical decision making could be reduced through guiding especially junior clinicians through a structured decision criteria catalogue. Implementing this list in daily clinical routine would help to drive multidisciplinary discussions around care goals, early consideration of alternative care pathways and, with that, adapting the discharge criteria to the patient-individual treatment plan. A continuous visualization of the discharge readiness assessment could prompt clinicians to consider patient discharge outside the morning rounds and throughout the day and help them plan and facilitate the actual patient transition with the related care team. Discharging patients as soon as they are stable enough would have great potential to optimize the use of limited ICU resources, to reduce waste in terms of overtreatment, waiting and avoidable complications and to reduce costs.

[0088] Criteria reporting automation

[0089] With 28 criteria, there is still a rather long list of discharge criteria that need to be checked by the decision makers in a recurring manner to determine discharge readiness. Former publications also formulated the need that ideally, most of the defined criteria should be auto- fillable with data from PDMS and Electronic Medical Record (EMR) systems. Although throughout the different rounds of voting, criteria were in discussion that could have been very well derived in electronic and automated form from the PDMS and EMR systems, most of those criteria didn’t make it to the final list. Remarkably, none of the proposed numeric scores and scales (from initial proposal and proposed by the panelists throughout the voting process), like Glasgow Coma Scale (GCS), Richmond Agitation Sedation Scale (RASS) and Confusion Assessment Method for Intensive Care Units (CAM-ICU) , Sequential Organ Failure Assessment (SOFA) and delta SOFA, Pain and Frailty scales as well as nursing workload related scores reached final consensus. For those criteria on the final list, that require a calculation method to determine whether the criterion is within the threshold values (SpO2, respiratory rate (RR), heart rate (HR), cardiac rhythm, mean arterial pressure (MAP), hemoglobin (Hb)), a calculation and reporting automation connected with the criteria catalogue is critical for implementation success in daily clinical practice.

[0090] Continuous discharge readiness evaluation

[0091] With the resulting list of discharge criteria, the request for a continuous evaluation of discharge readiness can still not be met. Ultimately, only 6 criteria are truly continuously and automatically reportable. On the other hand, some of the criteria that are closely related to treatment capabilities or needed infrastructure at the receiving unit, only need to be checked once for a particular patient or don’t account for a particular receiving unit or institution specific workflow and could therefore be excluded when applied in that environment. For the remaining criteria it may help to define evaluation time frames on an institution level. That means, over which time frame a certain criterion needs to be met to indicate patient stability. The results from the 4th round on suitable evaluation time frames didn’t show strong preferences for particular evaluation time frames per criterion. Within the investigators team, it was suspected that also further iterations on this question wouldn’t bring any significantly clearer results. For the sake of survey simplification, it was decided to stop the query on the criteria evaluation time frames after only one round of results. However, the vote distribution on proposed criteria evaluation time frames could serve as a first orientation to define institution specific time frames that match the related workflows, and to trigger future research on this aspect. Further, for daily practice implementation in PDMS- as well as paper-based ICUs, it remains a challenge to provide the basis for a discharge readiness evaluation possible at any time. Different types of data and resources need to be linked to provide one comprehensive view for all involved stakeholders on the patient’s and organization’s progress towards discharge readiness, decisions taken and access to underlying data.

[0092] Need for clinical implementation research

[0093] A multicenter point-prevalence study could compare actual discharge decision making criteria against the consented list and illustrate current differences in discharge practices. Implementation studies should further demonstrate if this consented standardized set of discharge criteria can adequately assess ICU patient’s discharge readiness, by reviewing fit for discharge status, patient flow, capacity utilization and patient outcomes key performance indicators (KPIs) retrospectively. A defined and clinically validated Fit for discharge-status could help future root cause analysis to identify discharge barriers and to measure process related waste of ICUcapacities. Ultimately, it needs to be researched in how far the implementation of objective and standardized ICU discharge criteria can reduce waste in the ICU discharge process, increase overall ICU capacity utilization and workflow efficiency. Clinical practice implementation may also stimulate future research on how this set of discharge criteria can further be improved towardsan automated and intelligent clinical decision support tool, suitable to integrate aggregated data inform of scores and ratios, see trends, predict patient individual discharge readiness, and learn retrospectively about factors that determine successful patient discharge and pathway selection.

[0094] Strengths and limitations

[0095] This study has several strengths. The expert panel included a decent number of subject matter experts in the field of acute care transitions, coming from different professional and geographical backgrounds, so that the results of the study should be robust against regional practice differences. The group was also not too large, so that the high proportion of provided qualitative answers could be handled well. Anonymity of the experts and their individual responses were preserved throughout the entire Delphi process, to avoid bias due to individual’s dominance and group pressure. Several iteration steps and related fine-tuning, the high completion rate throughout the five rounds of voting as well as the high level of consensus all helped to build a robust and consistent result.

[0096] However, the work may have certain limitations. Having a multinational panel answering an online questionnaire in English language with no possibility to clarify doubts could lead to misinterpretation of questions or also provided statements. Some comments were provided in national language and needed to be translated. Also, throughout the rounds, a few experts were confronted with criteria, they had no experience with in their own clinical practice. So, their ability to judge on its general usability could be limited. Further, the inclusion of the multidisciplinary team in the 4^th and 5^th round, could have brought a stronger agreement on a potential decision maker per criterion. But the need for this was only revealed through the provided comments. Also, a reiteration on the criteria evaluation time frames in further rounds could have brought clearer results, that could be then linked to the value calculation method for clinical practice implementation. This missing link requires further research on this detail. In general, several concerns around clinical practice implementation have been raised via the panelists comments and have been summarized in the discussion part and translated into suggestions for future implementation research.

[0097] Conclusion

[0098] In daily clinical practice, there is an absence of evidence-based and well-defined ICU discharge criteria that reflect a holistic assessment on the patient’s fit for discharge status as well as on the organizational capabilities that allow a safe and timely transition to the next lower level of care. A modified online Delphi process reached a consensus level of > 90% on a final list of 28 criteria to evaluate discharge readiness in adult ICU patients for a care transition to a general ward environment. The set of criteria covers patient-specific aspects, such as a holistic view on organ systems, therapeutic interventions as well as patient’s autonomy, continuous care needs, patient’s preferences, and therapeutic susceptibility. The consented organization-specific criteria focus on the underlying framework conditions, like discharge timing, safety measures, available care capacities, skill sets and technology. First clinical practice implementation studies are recommended to further define criteria evaluation time frames, the role of the different stakeholders in the decision process and the criteria importance ranking. Future research shall focus on validation of the criteria set, utility, criteria reporting and decision support automation, and visualization.

[0099] In a broader perspective, applying clearly defined ICU discharge criteria may reduce decision making subjectivity, improve patient safety and workflows in daily care transitions, support efficient use of limited ICU resources and equity of care, but also prevent avoidable patient deterioration and overburdening of lower levels of care. That means, patients and organizations could benefit from the implementation of such discharge criteria as a clinical decision support in daily clinical practice.

[00100] Referring to TABLE 1, therefore, is the final set of consensus criteria for ICU discharge readiness as generated from the methods and systems described or otherwise envisioned herein. Referring to TABLE 2 is the final set of consensus criteria for ICU discharge readiness as generated from the methods and systems described or otherwise envisioned herein, along with one or more parameters that were also derived using the methods and systems described or otherwise envisioned herein.

[00101] According to an embodiment, the final set of consensus criteria for ICU discharge readiness is implemented in a clinical decision support tool or similar system or process to facilitate clinician decision-making. For example, the clinical decision support tool can be programmed to utilize the consensus criteria to provide an ICU discharge readiness decision or recommendation to the clinician. The clinical decision support tool can obtain the necessary input information for the consensus criteria automatically from an EMR system, monitors, or other input sources. The clinical decision support tool can additionally or alternatively obtain some or all input information for the consensus criteria from a clinician or other caregiver. Once input for all the consensus criteria is received, or alternatively once a sufficient amount of input for enough of the consensus criteria is received, such as a threshold percentage or predetermined # or identity of the consensus criteria is received, the clinical decision support tool can process the information according to the criteria and parameters in TABLE 2 to arrive at an ICU discharge readiness recommendation.

[00102] Referring to FIG. 3, in one embodiment, is a method 300 for generating an ICU discharge recommendation, implementing the final set of consensus criteria for ICU discharge from TABLES 1 and 2. At step 310, a clinical decision support tool or similar tool is provided. The clinical decision support tool comprises programming with the final set of consensus criteriafor ICU discharge from TABLE 2, including both the 28 criteria and their associated parameters. At step 320 of the method, the clinical decision support tool or similar tool receives, obtains, or otherwise gets input relevant to some or all of the 28 criteria. The clinical decision support tool can obtain the necessary input information for the consensus criteria automatically from an EMR system, monitors, or other input sources. The clinical decision support tool can additionally or alternatively obtain some or all input information for the consensus criteria from a clinician or other caregiver.

[00103] At step 330 of the method, the clinical decision support tool analyzes the received criteria input. The analysis is performed using any method for input analysis. At step 340 of the method, the result of the analysis is utilized to generate an ICU discharge readiness recommendation, and at step 350 of the method the generated ICU discharge readiness recommendation is reported to a clinician via a user interface or other method as described or otherwise envisioned herein. At step 360 of the method, the ICU discharge readiness recommendation is implemented by the clinician. For example, the ICU discharge readiness recommendation - as generated based on the 28 consensus criteria - may be ‘do not discharge’ orsomething similar and may include reasons why not to discharge, based on the criteria. The ICU discharge readiness recommendation may alternatively be ‘discharge recommended’ or somethingsimilar and may include reasons why to discharge, based on the criteria.

[00104] TABLE 1. Final Set of Consensus Criteria for ICU Discharge Readiness

[00105] TABLE 2. Final Set of Consensus Criteria for ICU Discharge Readiness with

Associated Parameters

[00106] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms.

[00107] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[00108] The phrase “and/or,” as used herein in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and/or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and/or” clause, whether related or unrelated to those elements specifically identified.

[00109] As used herein in the specification and in the claims, “or” should be understood to havethe same meaning as “and/or” as defined above. For example, when separating items in a list, “or” or “and/or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of’ or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.”

[00110] As used herein in the specification and in the claims, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily includingat least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elementsspecifically identified.

[00111] It should also be understood that, unless clearly indicated to the contrary, in any methods claimed herein that include more than one step or act, the order of the steps or acts of the method is not necessarily limited to the order in which the steps or acts of the method are recited.

[00112] In the claims, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of’ and “consisting essentially of’ shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.

[00113] While several inventive embodiments have been described and illustrated herein, thoseof ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantagesdescribed herein, and each of such variations and/or modifications is deemed to be within the scopeof the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the inventive teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.

Claims

Claims What is claimed is:

1. A method for developing a set of consensus criteria for patient analysis, comprising: generating an initial set of criteria for a selected patient analysis, based on a literature review and/or an initial selection by a first expert in the field of the patient analysis; recruiting a plurality of experts in the field of the patient analysis; receiving, from each of the plurality of experts, a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria; revising, based on the received votes, the initial set of criteria to generate a revised set of criteria, wherein a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, and wherein a criterion is excluded from therevised set of criteria if the received exclusion votes are above a predetermined threshold; repeating the receiving and revising steps to generate a final set of criteria forthe selected patient analysis; receiving, from each of the plurality of experts, a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final set of criteria for the selectedpatient analysis, the parameter comprising one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology; and generating, from the final set of criteria and the received parameter votes, a consensus set of criteria for the patient analysis.

2. The method of claim 1, further comprising editing, by one or more additionalexperts in the field of the patient analysis, the generated initial set of criteria for the selected patientanalysis.

3. The method of claim 1, wherein the recruited plurality of experts in the field of the patient analysis are a diverse plurality.

4. The method of claim 1, wherein the recruited plurality of experts in the field of the patient analysis perform their steps of the method online.

5. The method of claim 1, wherein the patient analysis is patient discharge readiness.

6. The method of claim 1, wherein the criteria vote and the parameter vote are made with anonymity between experts in the plurality of experts.

7. The method of claim 1, wherein the predetermined threshold for inclusion or exclusion changes from the first criteria vote to a repeated criteria vote.

8. The method of claim 1, wherein the predetermined threshold for exclusion for the initial criteria vote is < 75%, and wherein the predetermined threshold for inclusion for a final criteria vote is > 90%.

9. The method of claim 1, further comprising analyzing a patient status using the consensus set of criteria to generate a patient recommendation.

10. The method of claim 9, further comprising implementing the generated patient recommendation.

11. A method for treating a patient, comprising: receiving a patient recommendation following an analysis of the patient’s status using a set of consensus criteria, the consensus criteria developed by: generating an initial set of criteria for a selected patient analysis, basedon a literature review and/or an initial selection by a first expert in the field of the patient analysis; recruiting a plurality of experts in the field of the patient analysis; receiving, from each of the plurality of experts, a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria; revising, based on the received votes, the initial set of criteria to generate a revised set of criteria, wherein a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, andwherein a criterion is excluded from the revised set of criteria if the received exclusion votes are above a predetermined threshold; repeating the receiving and revising steps to generate a final set of criteria for the selected patient analysis; receiving, from each of the plurality of experts, a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final setof criteria for the selected patient analysis, the parameter comprising one or more of criterion importance ranking, criterion range or value, criterion evaluation time, and/or criterion calculation or evaluation methodology; and generating, from the final set of criteria and the received parameter votes, a consensus set of criteria for the patient analysis; and implementing the generated patient recommendation.

12. The method of claim 11, further comprising editing, by one or more additional experts in the field of the patient analysis, the generated initial set of criteria for the selected patient analysis.

13. The method of claim 11, wherein the recruited plurality of experts in the field of the patient analysis perform their steps of the method online.

14. A method for generating an ICU discharge recommendation, comprising: providing a set of ICU discharge readiness consensus criteria, the set of ICU discharge readiness consensus criteria comprising at least the 28 criteria in TABLE 1; receiving patient input regarding some or all of the set of ICU discharge readiness consensus criteria; analyzing the received patient input using the set of ICU discharge readiness consensus criteria; generating, based on the analysis, an ICU discharge readiness recommendation; providing the generated ICU discharge readiness recommendation to a clinician; and administering the provided ICU discharge readiness recommendation.

15. The method of claim 14, wherein the ICU discharge readiness consensus criteria are derived using the following method: generating an initial set of criteria for a selected patient analysis, based on a literature review and/or an initial selection by a first expert in the field of the patient analysis; recruiting a plurality of experts in the field of the patient analysis; receiving, from each of the plurality of experts, a first criteria vote for inclusion or exclusion of each criterion in the initial set of criteria; revising, based on the received votes, the initial set of criteria to generate a revised set of criteria, wherein a criterion is included in the revised set of criteria if the received inclusion votes are above a predetermined threshold, and wherein a criterion is excluded from therevised set of criteria if the received exclusion votes are above a predetermined threshold; repeating the receiving and revising steps to generate a final set of criteria forthe selected patient analysis; receiving, from each of the plurality of experts, a parameter vote regarding one or more of the criterion in the initial set of criteria and/or the final set of criteria for the selectedpatient analysis; and generating, from the final set of criteria and the received parameter votes, a consensus set of criteria for the patient analysis.