WO2018017927A1

WO2018017927A1 - Systems and methods for analyzing clinical trial data

Info

Publication number: WO2018017927A1
Application number: PCT/US2017/043231
Authority: WO
Inventors: Syed S. Islam; Shihua WEN; Jiabu YE
Original assignee: Abbvie Inc.
Priority date: 2016-07-22
Filing date: 2017-07-21
Publication date: 2018-01-25

Abstract

A computer-implemented method for monitoring clinical trial data is provided. The method includes generating at least one safety boundary for a clinical trial, receiving a real¬ time data feed including a number of subjects/person years of observation in the clinical trial and a number of observed adverse events in the clinical trial up to the observation time point, causing a dashboard to be displayed on a remote computing device, the dashboard including a plot that includes the at least one safety boundary and an observation point, wherein the location of the observation point corresponds to the number of subjects/person years and the number of observed adverse events, determining whether the observation point exceeds the at least one safety boundary, automatically generating an alert when the observation point exceeds the at least one safety boundary, and automatically transmitting the generated alert to a client computing device.

Description

SYSTEMS AND METHODS FOR ANALYZING CLINICAL TRIAL DATA

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 62/365,457, filed July 22, 2016, which is incorporated herein by reference in its entirety.

FIELD OF USE

[0002] The disclosed subject matter relates to data processing between multiple computers in a digital data processing system and, more particularly, to evaluating clinical trial data in real-time and automatically generating and transmitting alerts based on the monitoring.

BACKGROUND

[0003] In digital data processing systems, large amounts of data may be generated relatively quickly. Such digital data processing systems are prevalent in, for example, the healthcare industry, and in scientific research. However, it may be difficult to evaluate the generated data in an efficient and effective manner. For example, in some systems, users must manually review generated data to identify trends in the data, relationships between various data points, generate reports related to the identified trends and/or relationships, and provide those reports to other users. This may be difficult given the large volume of data generated by clinical trials and observational studies using real-world clinical practice or health encounter data sources. Further, this process is time-consuming and subject to human error in the data review and evaluation.

[0004] For example, in clinical research, clinical trials are used to generate data regarding the safety and efficacy of a treatment of interest (e.g., a vaccine, a drug, a dietary supplement, a medical device, etc.). Clinical trials are typically performed by administering the treatment of interest to a predetermined number of participants, or subjects, in a population (e.g., a treatment group), and withholding the treatment of interest from the remaining participants in the population (e.g., a control group or comparator). Clinical trials are often blinded, such that the participants and study administrators are unaware which participants actually receive the treatment of interest, and which do not.

[0005] Clinical trials generally remain blinded until a predetermined time point in the study is reached, or a reason for unblinding occurs. For example, during the clinical trial, one or more adverse events may occur in the trial population. While the study remains blinded, it is unknown whether a particular adverse event occurred within the treatment group or the control group. Thus, it is difficult to determine whether or not an adverse event is associated with the treatment of interest. Accordingly, if a greater than expected number of adverse events occur during the blinded trial phase, the study may need to be unblinded to determine whether the adverse events are truly associated with the treatment of interest. However, it may be difficult to quickly and efficiently determine whether a greater than expected number of adverse events has occurred. Further, in at least some known systems, safety decisions may be made based on subjective judgment without performing a rigorous quantitative evaluation.

BRIEF DESCRIPTION OF THE DISCLOSURE

[0006] In one aspect, a computer-implemented method for monitoring clinical trial data is provided. The method includes generating, using a clinical trial data analyzer, at least one safety boundary for a clinical trial based on at least one user input, receiving, at the clinical trial data analyzer, a real-time data feed that includes i) one of a number of subjects in the clinical trial and a number of person years of observation up to an observation time point and ii) a cumulative number of observed adverse events in the clinical trial up to the observation time point, a dashboard to be displayed on a remote computing device, the dashboard including a plot that includes the at least one safety boundary and an observation point, wherein the location of the observation point corresponds to the number of subjects or person years of observation in the clinical trial and the number of observed adverse events, determining, using the clinical trial data analyzer, whether the observation point exceeds the at least one safety boundary, automatically generating, using the clinical trial data analyzer, an alert when the observation point exceeds the at least one safety boundary, and automatically transmitting the generated alert to a client computing device to prompt a user of the client computing device to review the generated alert.

[0007] In another aspect, a clinical trial data analyzer for monitoring clinical trial data is provided. The clinical trial data analyzer is configured to generate at least one safety boundary for a clinical trial based on at least one user input, receive a real-time data feed including i) one of a number of subjects in the clinical trial and a number of person years of observation up to an observation time point and ii) a cumulative number of observed adverse events in the clinical trial up to the observation time point, cause a dashboard to be displayed on a remote computing device, the dashboard including a plot that includes the at least one safety boundary and an observation point, wherein the location of the observation point corresponds to the number of subjects or person years of observation in the clinical trial and the number of observed adverse events, determine whether the observation point exceeds the at least one safety boundary, automatically generate an alert when the observation point exceeds the at least one safety boundary, and automatically transmit the generated alert to a client computing device to prompt a user of the client computing device to review the generated alert.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIGS. 1-6 show example embodiments of the methods and systems described herein.

[0009] FIG. 1 is a simplified block diagram of an example clinical trial data analysis system that includes a clinical trial data analyzer and other computing devices in accordance with one example embodiment of the present disclosure.

[0010] FIG. 2 is an expanded block diagram of an example embodiment of a server architecture of the clinical trial data analysis system including the clinical trial data analyzer and a plurality of other computing devices in accordance with one example embodiment of the present disclosure.

[0011] FIG. 3 illustrates an example configuration of a clinical trial administrator computing device that may be used with the system shown in FIGS. 1 and 2.

[0012] FIG. 4 illustrates an example configuration of a server system shown in FIGS. 1 and 2.

[0013] FIG. 5 illustrates a flow chart of an example clinical trial data analysis method that may be performed using the system shown in FIGS. 1 and 2.

[0014] FIG. 6 illustrates a screenshot of an example dashboard that may be generated using the system shown in FIGS. 1 and 2.

[0015] Like numbers in the Figures indicate the same or functionally similar components.

DETAILED DESCRIPTION OF THE DISCLOSURE

[0016] Embodiments of the methods and systems described herein enable generating one or more safety boundaries for adverse events during a clinical trial, and monitoring recorded adverse events to determine whether the one or more safety boundaries are exceeded. The safety boundaries may be generated, for example, using various statistical methods. If the safety boundaries are exceeded, an alert is automatically generated and transmitted to a computing device to prompt a user to view the alert and determine what (if any) action to take in regards to the clinical trial. [0017] The methods and systems described herein may be implemented using computer programming or engineering techniques including computer software, firmware, hardware or any combination or subset thereof, wherein the technical effect is achieved by performing at least one of: (a) generating at least one safety boundary for a clinical trial; (b) receiving a realtime data feed including i) one of a number of participants in the clinical trial and a number of person years of observation up to an observation time point and ii) a number of observed adverse events in the clinical trial; (c) causing a dashboard to be displayed on a remote computing device, the dashboard including a plot that includes the at least one safety boundary and an observation point, wherein the location of the observation point corresponds to the number of subjects or person years and the number of observed adverse events; (d) determining whether the observation point exceeds the at least one safety boundary; (e) automatically generating an alert when the observation point exceeds the at least one safety boundary; and (f) automatically transmitting the generated alert to a client computing device.

[0018] More specifically, the clinical trial data analysis system described herein is specially programmed with computer code to perform the above processes. The technical effects described herein apply to the technical field of processing data transmitted through computer networks. The systems and methods described herein provide the technical advantage of generating one or more safety boundaries, causing a plot including the one or more safety boundaries to be displayed and updated in real-time based on statistical interpretation, and automatically generating and transmitting alerts to computing devices when the one or more safety boundaries are exceeded.

[0019] The embodiments described herein provide a clinical trial data analysis system that includes a clinical trial data analyzer. The clinical trial data analysis system facilitates generating one or more safety boundaries for a clinical trial, and determining whether enough adverse events occur during the clinical trial to exceed the one or more safety boundaries. If the one or more safety boundaries are exceeded, an alert may be automatically generated and transmitted to a computing device operated by a user, as described herein. Facilitating early detection of safety issues in clinical trials can avoid undue suffering for subjects and may generate significant cost savings (e.g., if a trial is terminated early due to improved discovery of safety events). Further, such detection may aid in compliance with regulatory policies (e.g., FDA recommended guidelines).

[0020] In the systems and methods described herein, when a statistical inference is made that suggests a potential safety issue, an alert is automatically generated and sent to appropriate users. The alert may not immediately halt the clinical trial, but is a trigger for the trial to be further evaluated by, for example, a data monitoring committee (DMC) or other appropriate party.

[0021] In the example embodiment, the clinical trial data analyzer generates one or more safety boundaries and monitors whether an observed rate of a particular adverse event exceeds the one or more safety boundaries, as described herein. A sequential probability ratio testing (SPRT) safety boundary and a Bayesian safety boundary are both generated using the clinical trial data analyzer in the example embodiment. Further, safety monitoring is performed on sequential data as it is accrued during the clinical trial. Alternatively, any number and type of safety boundaries may be generated by the clinical trial data analyzer.

[0022] The SPRT safety boundary may be a simple SPRT boundary or a maximized SPRT boundary. SPRT is a likelihood ratio-based method in which in-coming (e.g., sequential) data are compared to a null hypothesis and an alternative hypothesis. In contrast, a Bayesian analysis calculates a posterior probability that incorporates prior probability and currently observed data. For a Bayesian analysis, inferential questions may be answered through an appropriate analysis of a posterior distribution.

[0023] The clinical trial data analyzer generates a dashboard for display on one or more client systems. The dashboard may include any type of output displayed on a computing device. For example, in the example embodiment, the dashboard includes a plurality of panels that enable a user to control generation of the safety boundaries and display of the monitored clinical trial data. Further, the clinical trial data analyzer receives a real-time data feed from an adverse event tracking computing device. The adverse event tracking computing device may generate the real-time data feed, for example, based on data stored in a clinical trial database coupled to the adverse event tracking computing device. The real-time data feed may include the number of subjects, or participants, in the clinical trial up to a time point of observation and the number of reported adverse events in the clinical trial up to the time point of observation. Alternatively, instead of the number of subjects, the data feed may include the person years of observation up to the time point of observation. 'Person years' is a parameter combining the number of persons (subjects) in a study and their time in the study. For example, if a study included four subjects each monitored for three months, the study would have one person year. Similarly, a study that followed twenty subjects for two years would have forty person years.

[0024] In the example embodiment, the dashboard includes an output panel that displays monitored clinical trial data. Specifically, the output panel includes a plot that includes a first safety boundary, a second safety boundary, and an observation point. The first safety boundary may be, for example, the SPRT safety boundary and the second safety boundary may be, for example, a Bayesian safety boundary. The observation point indicates the number of subjects/person years of observation up to the observation time point in the clinical trial and the number of cumulated reported events in the clinical trial up to the observation time point. In the example embodiment, the clinical trial data analyzer causes the plot to be updated automatically based on the real-time data feed received from the adverse event tracking computing device. The output panel may also include textual information (e.g., non-graphical statistical results) describing the statistical output of the analysis up to the observation time point. Accordingly, the output pane provides a current and accurate display of statistical inference and interpretation of clinical trial data up to the observation time point.

[0025] Further, in the example embodiment, when the reported number of adverse events exceeds at least one safety boundary, the clinical trial data analyzer automatically generates an alert. The alert may include, for example, any visual or audio notification. The alert may be displayed on the dashboard and/or may be automatically transmitted to a client computing device for display on the client computing device, such. Further, the transmitted alert may be in a report format, providing information on the clinical trial, safety boundaries, number of subjects/person years of observation, cumulative number of adverse events up to the observation time point, and/or prediction of what may be expected at a future observation time point. For example, the transmitted alert may include the plot. Additionally, in some embodiments, the transmitted alert may be in an interactive format, such that a user is able to manipulate and/or interact with the information in the transmitted alert. Accordingly, when at least one safety boundary is exceeded, the clinical trial data analyzer causes an alert to automatically be generated and transmitted for display on a client computing device, prompting a user operating the client computing device to interpret the alert and take appropriate action (e.g., continuing the clinical trial as planned, recommend unblinding the clinical trial, or potentially stopping the clinical trial if unblinded analysis substantiates the risk).

[0026] The following detailed description illustrates embodiments of the disclosure by way of example and not by way of limitation. It is contemplated that the embodiments have general application to processing healthcare data in a variety of applications.

[0027] As used herein, the term "database" may refer to either a body of data, a relational database management system (RDBMS), or to both. As used herein, a database may include any collection of data including hierarchical databases, relational databases, flat file databases, object-relational databases, object-oriented databases, and any other structured collection of records or data that is stored in a computer system. The above examples are example only and thus are not intended to limit in any way the definition and/or meaning of the term database. Examples of RDBMS's include, but are not limited to including, Oracle® Database, MySQL, Teradata, IBM® DB2, Microsoft® SQL Server, Sybase®, and PostgreSQL. However, any database may be used that enables the systems and methods described herein. (Oracle is a registered trademark of Oracle Corporation, Redwood Shores, California; IBM is a registered trademark of International Business Machines Corporation, Armonk, New York; Microsoft is a registered trademark of Microsoft Corporation, Redmond, Washington; and Sybase is a registered trademark of Sybase, Dublin, California.)

[0028] In one embodiment, a computer program is provided, and the program is embodied on a computer-readable medium. In an example embodiment, the system is executed on a single computer system, without requiring a connection to a sever computer. In a further embodiment, the system is being run in a Windows® environment (Windows is a registered trademark of Microsoft Corporation, Redmond, Washington). In yet another embodiment, the system is run on a mainframe environment and a UNIX® server environment (UNIX is a registered trademark of X/Open Company Limited located in Reading, Berkshire, United Kingdom). The application is flexible and designed to run in various different environments without compromising any major functionality. In some embodiments, the system includes multiple components distributed among a plurality of computing devices. One or more components may be in the form of computer-executable instructions embodied in a computer- readable medium.

[0029] As used herein, an element or step recited in the singular and preceded with the word "a" or "an" should be understood as not excluding plural elements or steps, unless such exclusion is explicitly recited. Furthermore, references to "example embodiment" or "one embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

[0030] The systems and processes are not limited to the specific embodiments described herein. In addition, components of each system and each process can be practiced independently and separate from other components and processes described herein. Each component and process also can be used in combination with other assembly packages and processes.

[0031] FIG. 1 is a simplified block diagram of one embodiment of a clinical trial data analysis system 200 that includes a clinical trial data analyzer 215 in communication with a server system 202 that includes a database server 206. Further, a database 208 is in communication with server system 202 in the example embodiment. Clinical trial data analyzer 215 includes a processing device and a memory. System 200 further includes a plurality of client subsystems, also referred to as client systems 204 or client computing devices, connected to server system 202. In one embodiment, client systems 204 are computers including a web browser, such that server system 202 is accessible to client systems 204 using the Internet or another network. Client systems 204 are interconnected to the Internet or another network through many interfaces including a network, such as a local area network (LAN) and/or a wide area network (WAN), dial-in connections, cable modems, wireless-connections, and special high-speed ISDN lines. Client systems 204 may be any device capable of interconnecting to the Internet including a web-based phone, personal digital assistant (PDA), watch, medical device, kiosk, laptop computer, desktop computer, netbook, tablet, phablet, or other web-connectable equipment.

[0032] Database server 206 is connected to database 208 containing information on a variety of matters, as described below in greater detail. In one embodiment, database 208 is stored on server system 202 and may be accessed by potential users at one of client systems 204 by logging onto server system 202 through one of client systems 204. Database 208 is also accessible to clinical trial data analyzer 215. In an alternative embodiment, database 208 is stored remotely from server system 202 and may be non-centralized (e.g., in a cloud computing configuration). Server system 202 could be any type of computing device configured to perform the steps described herein. Additionally, clinical trial data analyzer 215 is in communication with server system 202. In some implementations, clinical trial data analyzer 215 is incorporated into or integrated within server system 202.

[0033] FIG. 2 is an expanded block diagram of an example embodiment of a server architecture of clinical trial data analysis system 200 in accordance with one embodiment of the present disclosure. Clinical trial data analysis system 200 includes client systems 204 and clinical trial data analyzer 215. Server system 202 includes database server 206, an application server 302, a web server 304, a fax server 306, a directory server 308, and a mail server 310. Database 208 (e.g., a disk storage unit), is coupled to database server 206 and directory server 308. Servers 206, 302, 304, 306, 308, and 310 are coupled in a local area network (LAN) 314. In addition, a system administrator's workstation 316, a user workstation 318, and a supervisor's workstation 320 are coupled to LAN 314. Alternatively, workstations 316, 318, and 320 are coupled to LAN 314 using an Internet link or are connected through an Intranet.

[0034] Each workstation, 316, 318, and 320, is a personal computer having a web browser. Although the functions performed at the workstations typically are illustrated as being performed at respective workstations 316, 318, and 320, such functions can be performed at one of many personal computers coupled to LAN 314. Workstations 316, 318, and 320 are illustrated as being associated with separate functions only to facilitate an understanding of the different types of functions that can be performed by individuals having access to LAN 314.

[0035] Server system 202 is configured to be communicatively coupled to various entities, including third parties 334 using an Internet connection 326. Server system 202 is also communicatively coupled to clinical trial data analyzer 215. In some embodiments, clinical trial data analyzer 215 is integrated within server system 202. The communication in the example embodiment is illustrated as being performed using the Internet, however, any other wide area network (WAN) type communication can be utilized in other embodiments, e.g., the systems and processes are not limited to being practiced using the Internet. In addition, and rather than WAN 328, local area network 314 could be used in place of WAN 328.

[0036] In the example embodiment, any authorized individual or entity having a workstation 330 may access system 200. At least one of the client systems includes a manager workstation 332 located at a remote location. Workstations 330 and 332 include personal computers having a web browser. Also, workstations 330 and 332 are configured to communicate with server system 202. Furthermore, fax server 306 communicates with remotely located client systems, including a client system 332, using a telephone link. Fax server 306 is configured to communicate with other client systems 316, 318, and 320 as well.

[0037] FIG. 3 illustrates an example configuration of a clinical trial administrator computing device 402 operated by a user 401. Clinical trial administrator computing device 402 enables user 401 to monitor clinical trial data and view and interact with alerts generated in associated with monitored clinical trial data, as described herein. Clinical trial administrator computing device 402 may include, but is not limited to, client systems ("client computing devices") 204, 316, 318, and 320, workstation 330, and manager workstation 332 (shown in FIG. 2).

[0038] Clinical trial administrator computing device 402 includes one or more processors 405 for executing instructions. In some embodiments, executable instructions are stored one or more memory devices 410. Processor 405 may include one or more processing units (e.g., in a multi-core configuration). One or more memory devices 410 are any one or more devices allowing information such as executable instructions and/or other data to be stored and retrieved. One or more memory devices 410 may include one or more computer-readable media.

[0039] Clinical trial administrator computing device 402 also includes at least one media output component 415 for presenting information to user 401. Media output component 415 is any component capable of conveying information to user 401. In some embodiments, media output component 415 includes an output adapter such as a video adapter and/or an audio adapter. An output adapter is operatively coupled to processor 405 and operatively couplable to an output device such as a display device (e.g., a liquid crystal display (LCD), organic light emitting diode (OLED) display, cathode ray tube (CRT), or "electronic ink" display) or an audio output device (e.g., a speaker or headphones).

[0040] In some embodiments, Clinical trial administrator computing device 402 includes an input device 420 for receiving input from user 401. Input device 420 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad or a touch screen), a gyroscope, an accelerometer, a position detector, an audio input device, or a medical diagnostic device (e.g., a thermometer, blood pressure measuring device, heart rate monitor, etc.). A single component such as a touch screen may function as both an output device of media output component 415 and input device 420.

[0041] Clinical trial administrator computing device 402 may also include a communication interface 425, which is communicatively couplable to a remote device such as server system 202. Communication interface 425 may include, for example, a wired or wireless network adapter or a wireless data transceiver for use with a mobile phone network (e.g., Global System for Mobile communications (GSM), 3G, 4G or Bluetooth) or other mobile data network (e.g., Worldwide Interoperability for Microwave Access (WIMAX)).

[0042] Stored in one or more memory devices 410 are, for example, computer-readable instructions for providing a user interface to user 401 via media output component 415 and, optionally, receiving and processing input from input device 420. A user interface may include, among other possibilities, a web browser and client application. Web browsers enable users, such as user 401, to display and interact with media and other information typically embedded on a web page or a website from server system 202. A client application allows user 401 to interact with a server application from server system 202 or a web server.

[0043] FIG. 4 illustrates an example configuration of a server computing device 452 such as server system 202 (shown in FIGS. 1 and 2). Server computing device 452 may include, but is not limited to, database server 206, application server 302, web server 304, fax server 306, directory server 308, and mail server 310. Server computing device 452 is also representative of clinical trial data analyzer 215.

[0044] Server computing device 452 includes one or more processors 454 for executing instructions. Instructions may be stored in one or more memory devices 456, for example. One or more processors 454 may include one or more processing units (e.g., in a multi-core configuration). [0045] One or more processors 454 are operatively coupled to a communication interface 458 such that server computing device 452 is capable of communicating with a remote device such as clinical trial administrator computing device 402 or another server computing device 452. For example, communication interface 458 may receive requests from client systems 204 via the Internet or another network, as illustrated in FIGS. 1 and 2.

[0046] One or more processors 454 may also be operatively coupled to one or more storage devices 460. One or more storage devices 460 are any computer-operated hardware suitable for storing and/or retrieving data. In some embodiments, one or more storage devices 460 are integrated in server computing device 452. For example, server computing device 452 may include one or more hard disk drives as one or more storage devices 460. In other embodiments, one or more storage devices 460 are external to server computing device 452 and may be accessed by a plurality of server computing devices 452. For example, one or more storage devices 460 may include multiple storage units such as hard disks or solid state disks in a redundant array of inexpensive disks (RAID) configuration. One or more storage devices 460 may include a storage area network (SAN) and/or a network attached storage (NAS) system. In some embodiments, one or more storage devices 460 may include database 208.

[0047] In some embodiments, one or more processors 454 are operatively coupled to one or more storage devices 460 via a storage interface 462. Storage interface 462 is any component capable of providing one or more processors 454 with access to one or more storage devices 460. Storage interface 462 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any component providing one or more processors 454 with access to one or more storage devices 460.

[0048] One or more memory devices 410 and 456 may include, but are not limited to, random access memory (RAM) such as dynamic RAM (DRAM) or static RAM (SRAM), readonly memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and non-volatile RAM (NVRAM). The above memory types are example only, and are thus not limiting as to the types of memory usable for storage of a computer program.

[0049] Clinical trial data analysis system 200 facilitates generating one or more safety boundaries for a clinical trial, and determining whether enough adverse events occur up to an observation time point the clinical trial to exceed the one or more safety boundaries. If the one or more safety boundaries are exceeded, an alert may be automatically generated and transmitted to a computing device operated by a user, as described herein. [0050] Statistically analyzing adverse events in blinded (or unblinded) clinical trials is important in determining the safety of the trials. For example, assume a blinded clinical trial is designed to enroll 1500 total subjects. Further, suppose that for the first 750 subjects (i.e., half of the total subjects), 5 adverse events (e.g., myocardial infarctions) are reported. Initially, because data is only available for half of the total subjects, and because it is unknown whether the 5 adverse events occurred in the treatment or control group, it is unclear whether this number of adverse events is expected or unusual. However, if the data were to be unblinded, it may be that all 5 adverse events occurred in the treatment group, and no adverse events occurred in the control group. This suggests there may be safety issues with the study.

[0051] In the systems and methods described herein, when a statistical inference is made that suggests a potential safety issue, an alert is automatically generated and sent to appropriate users. The alert does not immediately halt the clinical trial, but is a trigger for the trial to be further evaluated by, for example, a data monitoring committee (DMC) or other appropriate party.

[0052] FIG. 5 illustrates a flow chart of an example clinical trial data analysis method 500. At block 502, a clinical team and product safety team (PST) managing the clinical trial define adverse event(s) of special interest (AESI) to be monitored during the clinical trial. The AESI may be defined based on clinical observations from past studies, and/or current PST knowledge or clinical knowledge of the treatment at issue and related treatments (e.g., the mechanism of action of the treatment at issue). If a serious safety event is discovered during the trial that was not previously listed as an AESI, the clinical team may recommend that event be defined as a new AESI. In some embodiments, a list of AESI may be accessed from an electronic database of relevant AESI and the clinical team and PST team may select AESI from the database.

[0053] Once the list of AESIs is agreed upon, at block 504, the PST creates a safety monitoring process and determines parameters for quantitative safety monitoring. Specifically, a pharmacoepidemiology group and/or clinical statistics group may work together to determine the parameters for each AESI to be monitored. The parameters may include background rate, threshold, quantitative method, prior distribution, etc.

[0054] At block 506, safety monitoring is conducted on blinded data. The safety monitoring is conducted at a predetermined frequency, and monitors whether the observed rate of a particular AESI exceeds an associated threshold, or safety boundary. At block 508, it is determined whether or not there is a safety concern. If there is a serious safety concern (e.g., when a safety boundary has been exceeded), flow proceeds to block 510, wherein unblinded analyses determined by the study team are performed. The unblinded analyses may be performed, for example, by a data monitoring committee (DMC). If there is a non-serious safety concern, protocol-defined analyses (e.g., blinded analyses) are continued at block 512. In either situation, at block 514, results of the analyses are reported to a management authority for that authority to render a decision (e.g., continue the trial, implement risk mitigation procedures, stop the trial, etc.).

[0055] In the example embodiment, to facilitate method 500, clinical trial data analyzer 215 generates one or more safety boundaries and monitors whether an observed rate of a particular adverse event exceeds the one or more safety boundaries, as described herein. There are many statistical methods that may be used to generate safety boundaries, for both unblinded and blinded clinical trials. For example, statistical tests for categorical data (e.g., chi-square tests, exact tests, etc.), sequential probability ratio testing (SPRT), meta-analysis techniques, Bayesian methods, etc., may all be used. For blinded safety monitoring, statistical analysis may be conducted in an on-sample case based on the rate of adverse events in the overall trial population without knowing the treatment assignment (e.g., control group or treatment group) of particular participants. For unblinded safety monitoring, statistical analysis may be conducted and based on a two-sample test where summary level data for each of the treatment and control group is used.

[0056] In the example embodiment, a SPRT safety boundary and a Bayesian safety boundary are both generated using clinical trial data analyzer 215. Further, safety monitoring is performed on sequential data as it is accrued during the clinical trial. Altematively, any number and type of safety boundaries may be generated by clinical trial data analyzer 215.

[0057] The SPRT safety boundary may be a simple SPRT boundary or a maximized SPRT boundary. SPRT is a likelihood ratio-based method in which in-coming (e.g., sequential) data are compared to a null hypothesis and an alternative hypothesis. As evidence favors one hypothesis over the other, this likelihood ratio is examined to see if sufficient evidence supports one hypothesis over the other. The generated SPRT safety boundary is a function of that likelihood ratio of the alternative hypothesis over the null hypothesis, as well a type I error, a, and a type II error, β. For blinded monitoring, using SPRT requires prior knowledge of the underlying risk.

[0058] For example, if the null hypothesis is that 8% of a trial population is likely to have an adverse event, and the alternative hypothesis is that 20% is likely to have the adverse event, a sequential comparison of the two hypotheses based on accumulating data takes of the form of a running log likelihood ratio (LLR) and is given by two thresholds (e.g., "a" and "b"). The boundaries of "a" and "b" depend on a and β, wherein a is the probability of rej ecting the null hypothesis when it is true, and β is the probability or rejecting the alternative hypothesis when it is true. Specifically, the thresholds are given by a=log[ /(l - a)] and b=log[(l- )/a] . If the LLR exceeds "b", the null hypothesis is rejected and the alternative hypothesis is accepted, and if the LLR is less than "a", then the null hypothesis is accepted. Accordingly, a and β control the SPRT safety boundary. To avoid falsely discovering a serious safety event, a should be small. On the other hand, to avoid missing a serious safety event, β should be small.

[0059] A Bayesian analysis calculates a posterior probability that incorporates prior probability and currently observed data. For a Bayesian analysis, inferential questions may be answered through an appropriate analysis of a posterior distribution. Once the posterior distribution has been obtained, point and interval estimates of parameters, prediction inferences for future data, and probabilistic evaluation of hypotheses can all be calculated. The posterior distribution can be understood as a weighted average between knowledge about parameters before data is observed (represented by the prior distribution) and information about parameters contained in the observed data (represented by the likelihood function).

[0060] Specifically, the Bayesian safety boundary may be defined as: Prob (Blinded Rate R > C I x, n) > P, wherein C is a critical rate (determined by the pharmacoepidemiology and/or clinical team), P is a statistical probability threshold, x is a number of AESI, n is a number of patients, and R is a blinded rate.

[0061] As noted above, clinical trial data analyzer 215 facilitates generating safety boundaries and monitoring clinical data for safety issues. FIG. 6 is a screenshot of an example dashboard 600 that may be generated by clinical trial data analyzer 215. Dashboard 600 may be displayed, for example, on client systems 204 (shown in FIG. 2). As shown in FIG. 6 dashboard 600 includes a plurality of panels 602 that enable a user to control generation of the safety boundaries and display of the monitored clinical trial data.

[0062] In the example embodiment, dashboard 600 includes a data metrics panel 604. Data metrics panel 604 allows a user to specify a data type (e.g., binary, Poisson, normal) and whether the clinical trial is blinded or unblinded. For example, binary data includes a denominator that is the number of subj ects in the trial up to the observation time point and a numerator that is the cumulative number of events of interest up to the observation time point. In contrast, Poisson data is based on a denominator of person years of observation rather than number of subjects. Further, normal data is based on a numerator that is a continuous distribution (e.g., blood pressure). [0063] Dashboard 600 also includes a current data panel 606 that lists a number of observed adverse events, s_1; and a number of participants in the study, n. To populate current data panel 606, clinical trial data analyzer 215 may receive a real-time data feed from an adverse event tracking computing device (not shown), such as a computing device running JReview. JReview ® is a registered trademark of Integrated Clinical Systems, Inc., of Frenchtown, NJ. In the example embodiment, the real-time data feed includes the number of subjects, or participants, in the clinical trial and the number of reported adverse events. Alternatively, the real-time data feed may include any data that enables clinical trial data analyzer 215 to function as described herein.

[0064] Dashboard 600 also includes an SPRT input panel 608 in the example embodiment. SPRT input panel 608 allows a user to input variables and/or coefficients to generate an SPRT safety boundary. Dashboard 600 further includes an options panel 610 that allows the user to input other variables and/or coefficients that control what boundaries are generated. Options panel 610 facilitates simulating that if the observed numbers are replicated x number of times, what value is obtained at a pre-specified threshold. For, if the safety boundary is crossed at a threshold of 0.8, it means that after simulating x number of times, the event rate exceeded the boundary of the critical value and it will only show that it exceeded the boundary when it met the threshold of 0.8 or above. So, there may be crossing of the safety boundary when the threshold (which may be set by investigators) is 0.8, but if the threshold is reset to 0.9 the safety boundary will not be crossed.

[0065] For the Bayesian safety boundary, dashboard 600 includes a Bayesian prior panel 612 and a posterior criteria panel 614 for inputting pertinent variables and/or coefficients. Dashboard 600 also includes a predictive events panel 616 for inputting additional variables and/or coefficients. For example, for Bayesian prior panel 612, in the example embodiment, a Bayesian prior distribution is the presumed model. If Oo and βο are 1, it means that there is no information on whether there will be any differences between observed and expected numbers of adverse events (also known as a non-informative prior). Informative prior information may be input in which the user specifies, for example, expecting twice the risk based on prior information. For posterior criteria panel 614, a posterior critical value is provided by the user based on the best background information available. That value may be a fixed number or a range of numbers. If the posterior critical value is exceeded an appropriate signal is generated. Information in predictive events panel 616 means that if additional subjects are enrolled (e.g., fifty additional subjects), what is the probability of observing ten more adverse events. Similarly, if we enroll 1515 subjects, given the Wald null hypothesis of 0.1, the probability of exceeding the safety boundary is 100%.

[0066] In the example embodiment, dashboard 600 includes an output panel 620 that displays monitored clinical trial data. Specifically, output panel 620 includes a plot 622 that includes a first safety boundary 624, a second safety boundary 626, and an observation point 628. First safety boundary 624 may be, for example, an SPRT safety boundary automatically generated by clinical trial data analyzer 215 based on information input into SPRT input panel 608 and options panel 610. Similarly, second safety boundary 626 may be a Bayesian safety boundary automatically generated by clinical trial data analyzer 215 based on information input into Bayesian prior panel 612 and posterior criteria panel 614. Alternatively, any suitable safety boundaries may be generated. Observation point 628 indicates the number of subjects in the clinical trial and the number of reported events.

[0067] As shown in FIG. 6, in this example, observation point 628 indicates that both first safety boundary 624 and second safety boundary 626 have been exceeded (e.g., observation point 628 is above first and second safety boundaries 624 and 626). Those of skill in the art will appreciate that if the number of reported events were lower, observation point 628 may be located between first and second safety boundaries 624 and 626, or located below both first and second safety boundaries 624 and 626. In the example embodiment, clinical trial data analyzer 215 causes plot 622 to be updated automatically based on the real-time data feed received from the adverse event tracking computing device, as described above. Output panel 620 may also include textual information 630 describing observation point 628, first and second safety boundaries 624 and 626, or other information associated with the clinical trial. Accordingly, output panel 620 provides a current and accurate display of clinical trial data.

[0068] To display textual information 630, in the example embodiment, clinical trial data analyzer 215 automatically computes over all proportions or incidence rate at a 95% confidence interface, and textually describes that analysis. It also describes if the critical value based on prior experience is .02 (e.g., two adverse events per one thousand subjects), the probability that the critical value will be exceeded is 0.9996, and as the threshold from simulation is set at 0.8, the observation point will be shown above the safety boundaries. The other descriptions in textual information 630 explain that the probability of observing an additional ten events out of fifty subjects in the future is zero in this situation.

[0069] Further, in the example embodiment, when the reported number of adverse events exceeds at least one of first and second safety boundaries 624 and 626, clinical trial data analyzer 215 automatically generates an alert. The alert may include, for example, any visual or audio notification that indicates to a user that at least one of first and second safety boundaries 624 and 626 has been exceeded. Alternatively, in some embodiments, the alert may only be generated when both first and second safety boundaries 624 and 626 have been exceeded.

[0070] In some embodiments, the alert is displayed on dashboard 600. Alternatively or additionally, clinical trial data analyzer 215 causes the alert to automatically be transmitted to a client computing device for display on the client computing device, such as clinical trial administrator computing device 402 (shown in FIG. 3). Further, the transmitted alert may be in a report format, providing information on the clinical trial, safety boundaries, number of subjects, and/or number of adverse events. For example, the transmitted alert may include plot 622 and/or at least a portion of textual information 630. Additionally, in some embodiments, the transmitted alert may be in an interactive format, such that a user operating clinical trial administrator computing device 402 is able to manipulate and/or interact with the information in the transmitted alert. Accordingly, when at least one safety boundary is exceeded, clinical trial data analyzer 215 causes an alert to automatically be generated and transmitted for display on a client computing device, prompting a user operating the client computing device to interpret the alert and take appropriate action (e.g., continuing the clinical trial as planned, switching a blinded clinical trial to an unblinded clinical trial, or stopping the clinical trial).

[0071] The systems and methods described herein facilitate evaluating clinical trial safety data ad hoc or at a predetermined frequency, and generating and transmitting alerts to detect safety signals early. This is accomplished using a data analysis tool that operates using disparate statistical evaluation methods. By applying this tool, clinicians are able to quantitatively assess at least some safety signals early and continuously monitor for such safety signals.

[0072] As will be appreciated based on the foregoing specification, the above-described embodiments of the disclosure may be implemented using computer programming or engineering techniques including computer software, firmware, hardware or any combination or subset thereof, wherein the technical effect of the systems and processes described herein is achieved by creating a system for retrieving and displaying patient data such that trends may be identified. Any such resulting program, having computer-readable code means, may be embodied or provided within one or more computer-readable media, thereby making a computer program product, e.g., an article of manufacture, according to the discussed embodiments of the disclosure. The computer-readable media may be, for example, but is not limited to, a fixed (hard) drive, diskette, optical disk, magnetic tape, semiconductor memory such as read-only memory (ROM), and/or any transmitting/receiving medium such as the Internet or other communication network or link. The article of manufacture containing the computer code may be made and/or used by executing the code directly from one medium, by copying the code from one medium to another medium, or by transmitting the code over a network.

[0073] This written description uses examples to disclose the embodiments, including the best mode, and also to enable any person skilled in the art to practice the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the embodiments is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.

Claims

WHAT IS CLAIMED IS :

1. A computer-implemented method for monitoring clinical trial data, the method comprising:

generating, using a clinical trial data analyzer, at least one safety boundary for a clinical trial based on at least one user input;

receiving, at the clinical trial data analyzer, a real-time data feed that includes i) one of a number of subjects in the clinical trial and a number of person years of observation up to an observation time point and ii) a cumulative number of observed adverse events in the clinical trial up to the observation time point;

causing, using the clinical trial data analyzer, a dashboard to be displayed on a remote computing device, the dashboard including a plot that includes the at least one safety boundary and an observation point, wherein the location of the observation point corresponds to the number of subjects or person years of observation in the clinical trial and the number of observed adverse events;

determining, using the clinical trial data analyzer, whether the observation point exceeds the at least one safety boundary;

automatically generating, using the clinical trial data analyzer, an alert when the observation point exceeds the at least one safety boundary; and

automatically transmitting the generated alert to a client computing device to prompt a user of the client computing device to review the generated alert.

2. The computer-implemented method of Claim 1, wherein generating at least one safety boundary comprises generating a sequential probability ratio testing (SPRT) safety boundary.

3. The computer-implemented method of Claim 1, wherein generating at least one safety boundary comprises generating a Bayesian safety boundary.

4. The computer-implemented method of Claim 1, wherein generating at least one safety boundary comprises generating at least one safety boundary based on at least one user input that is input into the displayed dashboard.

5. The computer-implemented method of Claim 1 , wherein automatically generating an alert comprises automatically generating an alert that includes the plot.

6. The computer-implemented method of Claim 1 , wherein automatically generating an alert comprises automatically generating an interactive alert that allows the user to manipulate information included in the alert.

7. The computer-implemented method of Claim 1, wherein generating at least one safety boundary comprises generating at least one safety boundary for a blinded clinical trial.

8. The computer-implemented method of Claim 1, wherein generating at least one safety boundary comprises generating at least one safety boundary for an unblinded clinical trial.

9. A clinical trial data analyzer for monitoring clinical trial data, the clinical trial data analyzer configured to:

generate at least one safety boundary for a clinical trial based on at least one user input; receive a real-time data feed including i) one of a number of subjects in the clinical trial and a number of person years of observation up to an observation time point and ii) a cumulative number of observed adverse events in the clinical trial up to the observation time point;

cause a dashboard to be displayed on a remote computing device, the dashboard including a plot that includes the at least one safety boundary and an observation point, wherein the location of the observation point corresponds to the number of subjects or person years of observation in the clinical trial and the number of observed adverse events;

determine whether the observation point exceeds the at least one safety boundary;

automatically generate an alert when the observation point exceeds the at least one safety boundary; and

automatically transmit the generated alert to a client computing device to prompt a user of the client computing device to review the generated alert.

10. The clinical trial data analyzer of Claim 9, wherein to generate at least one safety boundary, the clinical trial data analyzer is configured to generate a sequential probability ratio testing (SPRT) safety boundary.

1 1. The clinical trial data analyzer of Claim 9, wherein to generate at least one safety boundary, the clinical trial data analyzer is configured to generate a Bayesian safety boundary.

12. The clinical trial data analyzer of Claim 9, wherein to generate at least one safety boundary, the clinical trial data analyzer is configured to generate at least one safety boundary based on at least one user input that is input into the displayed dashboard.

13. The clinical trial data analyzer of Claim 9, wherein to automatically generate an alert, the clinical trial data analyzer is configured to automatically generate an alert that includes the plot.

14. The clinical trial data analyzer of Claim 9, wherein to automatically generate an alert, the clinical trial data analyzer is configured to automatically generate an interactive alert that allows the user to manipulate information included in the alert.

15. The clinical trial data analyzer of Claim 9, wherein to generate at least one safety boundary, the clinical trial data analyzer is configured to generate at least one safety boundary for a blinded clinical trial.

16. The clinical trial data analyzer of Claim 9, wherein to generate at least one safety boundary, the clinical trial data analyzer is configured to generate at least one safety boundary for an unblinded clinical trial.