WO2013036718A1 - Determining acceptability of physiological signals - Google Patents

Abstract

Often a problem with a physiological signal Indicates presence of a medically relevant condition. Thus monitoring physiological signals is central in patient care. Disclosed is a system and method for evaluating physiological signals to determine whether or not a physiological signal corresponds to a medically relevant condition. In one embodiment of the present disclosure a system and method is provided for determining acceptability of physiological signals that involves monitoring a subject with one or more channels of a physiological data capture or monitoring device or devices, such as an electrocardiogram, a pulse oximeter and/or a respiration trace, and acquiring signals from such device or devices.

Description

DETERMINING ACCEPTABILITY OF PHYSIOLOGICAL SIGNALS

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. provisional patent application serial number 61/532,502, filed 2011 September 8, having the title "System and Method for Determining Acceptability of Physiological Signals and Events Monitored in Noisy

Ambulatory Environments," which is incorporated by reference as if fully set forth herein in its entirety.

[0002] This application also claims the benefit of U.S. provisional patent application serial number 61/697,569, filed 2012 September 6, having the title "Determining

Acceptability of Physiological Signals," which is incorporated by reference as if fully set forth herein in its entirety.

[0003] Additionally, the following documents are incorporated by reference as if expressly and fully set forth herein in their entireties:

[0004] Aboukhalil, A., L. Nielsen, M. Saeed, R. G. Mark and G. D. Clifford,

"Reducing false alarm rates for critical arrhythmias using the arterial blood pressure waveform," J. Biomed. Inform., vol. 41, Mar. 2008, pp. 442-451.

[0005] Bishop, C. M. 'Pattern Recognition and Machine Learning". New York, NY:

Springer, 2006, pp. 325-345.

[0006] Chang, C. C, and C. J. Lin. "LIBSVM: A library for support vector machines." ACM Trans. Intell. Syst. Technol. Vol. 2, 2011, pp. 1-27.

[0007] Clifford, G. D., D. Lopez, Q. Li, and I. Rezek. "Signal Quality Indices and

Data Fusion for Determining Acceptability of Electrocardiograms Collected in Noisy

Ambulatory Environments," Comput. Cardiol., Vol. 38, 2011, pp. 285-288. [0008] Deshmane, A. V. "False Arrhythmia Alarm Suppression Using ECG, ABP, and Photoplethysmogram," M.S. thesis, Dept. Electric. Eng. Comp., MIT, USA, 2009, published at http://lcp.mit.edu/pdf/DeshmaneThesis09.pdf.

[0009] Goldberger, A. L., L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R.

G. Mark, J. E. Mietus, G. B. Moody, C. K. Peng, and H. E. Stanley, "PhysioBank,

PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals," Circulation, vol. 101, no. 23, Jun. 2000, pp. e215-220.

[0010] The MIMIC II Database, available online at:

http://www.physionet.org/physiobank/database/mimic2db/

[0011] Goldberg, D. E. "Genetic Algorithms in Search, Optimization, and Machine

Learning." Reading, MA: Addison- Wesley, 1989.

[0012] Hamilton and Tompkins. "Quantitative Investigation of QRS Detection Rules

Using the MIT/BIH Arrhythmia Database," IEEE Transactions on Biomedical Engineering, no. 12, pp. 1157-1165, 1986.

[0013] Huang, C. L. and C. J. Wang. "A GA-based feature selection and parameters optimization for support vector machines," Expert Systems with Applications. Vol. 31 , pp: 231-240, 2006.

[0014] Leardi, R. R. Boggla, and M. Terrile, "Genetic algorithms as a strategy for feature selection," Journal of Chemometrics . Vol. 6, pp. 267-281, 1992.

[0015] Li, Q., R. G. Mark, and G. D. Clifford. , "Artificial arterial blood pressure artifact models and an evaluation of a robust blood pressure and heart rate estimator," BioMedical Engineering OnLine, 2009, 8:13 doi: 10.1186/1475-925X-8-13. [0016] Li, Q., R. G. Mark, and G. D. Clifford. "Robust heart rate estimation from multiple asynchronous noisy sources using signal quality indices and a Kalman filter." Physiol. Meas., vol. 29, no. 1, p. 15, 2008.

[0017] Mason C. L. and L. Tarassenko. "Quantitative assessment of respiratory derivation algorithms," in Engineering in Medicine and Biology Society, 2001. Proceedings of the 23^rd Annual International Conference of the IEEE, vol. 2, 2001, pp. 1998-2001.

[0018] Moody, G.B., R.G. Mark, A. Zoccola, and S. Mantero. "Derivation of respiratory signals from multi-lead ECGs," Proc. Computers in Cardiology, vol. 12, 1985, pp. 113-116.

[0019] Nemati, S., A. Malhotra, and G. D. Clifford. "Data Fusion for Improved

Respiration Rate Estimation." EURASIP Journal on Advances in Signal Processing. Vol. 2010, 2010.

[0020] Peng, H., F. Long, and C. Ding. "Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy." IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 27, 2005, pp. 1226-1238.

[0021] Saeed, M., M. Villarroel, A. T. Reisner, G. Clifford, L. Lehman, G. Moody, T.

Heldt, T.H. Kyaw, B. Moody, and R. G. Mark. "Multiparameter intelligent monitoring in intensive care II: A public-access intensive care unit database," Crit. Care Med., vol. 39, no. 5, pp. 952-960, 2011.

[0022] Sornmo, L. and P. Laguna. Bioelectrical Signal Processing in Cardiac and

Neurological Applications. Elsevier Academic Press, June 2005.

[0023] Sun, J. X., A. T. Reisner, and R. G. Mark. "A signal abnormality index for arterial blood pressure waveforms," Comput. Cardiol, Vol. 33, 2006, pp. 13-16. BACKGROUND

Technical Field

[0024] The present disclosure relates generally to assessing biomedical signals and, more particularly, to reducing erroneous results when assessing biomedical signals.

Description of the Related Art

[0025] Monitoring of physiological signals is vital to care and diagnosis of a patient.

Many physiological signals exist that are monitored during patient care through various systems and methods that provide valuable information on a status of a patient, thus there are ongoing efforts to improve these systems and methods to allow for superior patient care.

SUMMARY

[0026] In one embodiment of the present disclosure a system and method is provided for determining acceptability of physiological signals that involves monitoring a subject with one or more channels of a physiological data capture or monitoring device or devices, such as an electrocardiogram, a pulse oximeter and/or a respiration trace, and acquiring signals from such device or devices. Signal quality of the underlying data is then measured using signal quality metrics. In one ore more embodiments, signal quality metrics are calculated on each signal. Signal quality metrics are a statistical measure of the underlying noise in the signal. Physiological parameters (or other selected subject parameters) indicative of the state of the system can also be extracted from the one or more acquired signals. A set of labeled data from many patients can then be used to train a machine learning algorithm, such as an Artificial Neural Network (ANN) or a Support Vector Machine (SVM), to estimate the probability that (or classify whether) the underlying signal corresponds to a particular alarm condition or underlying noise. The accuracy of the estimate or classification can then be measured. Next, subsets of features optionally using, for example, a genetic algorithm, can be selected and the accuracy of the subset features assessed. The most accurate classifier can be retained based on an independent test set of data. Once the machine learning algorithm has been trained, and validated on a previously unseen test set of data, it can then be deployed in real time on any new data with as much confidence as the performance reported on the unseen test data.

[0027] In an embodiment of the present disclosure, a system is provided including: a data acquisition system configured to acquire physiological data; and a processing system coupled to the data acquisition system, the processing system being configured to receive data acquired by the data acquisition system, the processing system further being configured to estimate a probability that the acquired physiological data corresponds to either an alarm condition or an underlying noise of the acquired data. The processing system can include a local interface; and a processor, memory, a user interface, and an I/O device, each coupled to the local interface. The processing system can be a mobile application for a mobile device. The data acquisition system and the processing system can be integrated into a single device or can reside on separate devices.

[0028] For example, in an embodiment of the present disclosure, a method is provided including the steps of: acquiring a physiological signal; statistically analyzing signal noise to determine a physiological signal quality; training a machine learning algorithm to estimate probability of whether the physiological signal corresponds to either an alarm condition or an underlying noise using a set of labeled data; measuring accuracy of the estimate; selecting an estimate among a plurality of estimates, that can be for example a most-accurate estimate; validating a trained machine learning algorithm using an independent test dataset; and employing a trained and validated machine learning algorithm in real-time on new data. In one or more embodiments, the method can also include the step of extracting physiological parameters indicative of a system state prior to training the machine learning algorithm. The method can include extracting the physiological parameters followed by combining some or all extracted physiological parameters. The method can include selecting a subset of features using a feature selection algorithm followed by assessing accuracy of the selected subset of features prior to training the machine learning algorithm or selecting the most accurate estimate. The feature selection algorithm can be, for example, a genetic algorithm. The method can be used as one or more of an apnea alarm, an electrocardiogram alarm, or a photoplethysmogram alarm.

[0029] In yet another embodiment, a method is provided including the steps of:

statistically analyzing noise in a physiological signal to determine a signal quality; and training a machine learning algorithm to determine if the physiological signal corresponds to either an alarm condition or an underlying signal noise.

[0030] Other systems, devices, methods, features, and advantages will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure.

Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

[0032] FIG. 1 shows a block diagram of an embodiment of a system for determining acceptability of physiological signals.

[0033] FIG. 2 shows a block diagram of an embodiment of a processing system shown in FIG. 1.

[0034] FIG. 3 shows one embodiment of a system and a method for determining acceptability of physiological signals.

[0035] FIG. 4 shows example receiver operating characteristic (ROC) curves generated when using the embodiments disclosed herein.

[0036] FIG. 5 example sensitivity and specificity curves generated when using the embodiments disclosed herein.

[0037] FIG. 6 shows another example ROC curves associated with Example 2 of the disclosed embodiments.

[0038] FIG. 7 shows another example of sensitivity and specificity curves associated with Example 2 of the disclosed embodiments, but without arterial blood pressure features.

[0039] FIG. 8 shows example sensitivity and specificity curves of all variable selections described in Example 2 using a Multi-Layer Perceptron algorithm. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] Monitoring of physiological signals is vital to care and diagnosis of a patient and thus the systems and methods of doing so are ubiquitous to medical facilities.

Significant advancements in physiological signal monitoring have occurred over the past century, however, major issues still exist with the current systems and methods. A major issue that limits current automated or semi-automated monitoring and interpretation of physiological signals is contamination of the physiological signal by noise and accounting for a relationship between multiple physiological signals and signal noise. Contamination of physiological signals leads to high rates (up to 95%) of false alarms by these systems. The rate of false alarms is so great, that attending medical personnel often ignores the alarms, which is detrimental to patient care.

[0041] Previous methods of assessing quality and normality of signals involved a set of heuristics, such as a threshold on spectral energy density in a given frequency region, or a threshold on a moment statistical distribution. In existing methods, parameters and thresholds are combined either in parallel or sequentially, essentially in a univariate manner. For example, prior monitoring systems may be designed to accept or reject as noise, a signal having a parameter above or below a selected single (or series of) threshold(s). Alarms are triggered when parameters of the physiological signal go beyond the set threshold(s). In practice, there can be wide variations of a given parameter without alteration of an underlying physiological function. Thus, it is often required that attending medical personnel have some knowledge of the patient's physiology or condition in order to set the thresholds. Due to the high false positive rate generated by signal noise, the relationship between multiple physiologic signals and noise, and the wide variation in some parameters, medical personnel often adjust the alarm thresholds up or down to avoid being alerted at inappropriate times. One the one hand, manual threshold adjustment allows for the medical personnel to spend more time with patients in actual need. However, on the other hand, the likelihood of an alarm being adjusted to a level so as to not detect a critical physiological signal increases, which is not ideal. Current monitoring systems also do not allow quality metrics to be recalibrated rapidly and accurately for any given recording situation, patient demographic, patient condition, or data type. Additionally, currents systems do not allow multiple signals to be considered simultaneously, regardless of data type and units of measurement. In light of the limitations of current physiological signal monitoring methods and systems, there exists a need for a system that can achieve a lower false alarm rate while being more robust in the types of signals and data it can handle.

[0045] The present disclosure addresses and overcomes the aforementioned disadvantages. The present system and method does not require explicit knowledge of the patient's physiology or condition. Additionally, the disclosed system and method allows quality metrics to be recalibrated rapidly and accurately for any given recording situation or data type, so long as labeled data is provided. Rapid and efficient real time assessment of data quality and normality is facilitated in this system by rapid classification of the data by involving a simple matrix multiplication.

[0046] The present system and method can simultaneously combine multiple measures of signal quality and physiological variables to determine if a segment of data is usable or not, and if useable, if the segment of data represents a normal or abnormal physiological state, thereby providing an improved physiological signal monitoring system. Since measures of noise and physiology can be combined simultaneously, the present system and method can use the relationship between both the physiological parameters and between the physiological parameters and signal noise to differentiate true events from false events due to noise. It should be noted that noise will occur in both true and false events, so it is not sufficient to just measure the fact that noise is present. Rather a covariance between the noise measurements and the physiological measurements is necessary for differentiate true events from false events due to noise. The present system and method not only measures the covariance between the noise and physiological measurements, but can also learn this covariance for any given population or combination of sensors.

[0047] The present system and method can employ a novel machine learning approach for combining measures of underlying signal quality and physiological parameters together in a multivariate manner, which learns the intrinsic nonlinear interrelationship between the noises and the signals in a multivariate manner. Output of the machine learning classifier can be a class (or probability of belonging to a class) and hence can be combined in an almost unlimited number of physiological parameters and signal quality metrics (or measures of noise) into one useful number which tells a user (or decision algorithm) whether underlying data are providing a truthful assessment of a physiological function.

[0048] The disclosed system and method can, for example, be used a) for determining the quality of physiological signals (such as the electrocardiogram (ECG), pulse oximetry trace, or respiratory trace, for example), b) for determining whether or not a segment of physiological data is exhibiting abnormal behavior, and c) for false alarm reduction in monitoring environments such as in the ICU.

[0049] Reference is now made in detail to the description of the embodiments as illustrated in the drawings. While several embodiments are described in connection with these drawings, there is no intent to limit the disclosure to the embodiment or embodiments disclosed herein. On the contrary, the intent is to cover all alternatives, modifications, and equivalents. Moreover, all reference cited herein are intended to be and are hereby incorporated by reference into this disclosure as if full set forth herein.

[0050] FIG. 1 illustrates a system 100 wherein acceptability of physiological signals can be determined according to an embodiment of the present disclosure. As indicated in FIG. 1, the system 100 generally comprises a signal/data acquisition device or system 102 and a processing device or system 104 that are coupled such that data can be acquired and sent from the data acquisition system 102 to the processing system 104. The processing system 104 may comprise, for example, a hand-held device, a portable device, a computer, server, dedicated processing system, or other system, as can be appreciated. The hand-held device can be, for example, a smart mobile phone. The processing system 104 may include various input devices such as a keyboard, microphone, mouse, touch screen or other device, as can be appreciated. By way of example, the system 100 can comprise a stand-alone device or a part of a network, such as a local area network (LAN) or wide area network (WAN).

[0051] The signal/data acquisition system 102 is configured to acquire physiological signals or data. It can include any one or more of a conventional device used to acquire or extract physiological data concerning, for example, heart rate, breathing rate, respiration rate, blood pressure, and/or oxygen saturation level of a subject. Exemplary devices include the Nellcor OxiMax N-600x Pulse Oximeter, the Welch Allen CP-50 ECG recorder, the Omron R6 Blood Pressure Monitor, the Masimo Rainbow Acoustic Respiration Rate (RRa) system, the Philips IntelliVue MP90 or SureSigns VM6 portable bedside monitor and the GE Dash 3000 or the GE-Marquette Eagle 4000 Vital Signs Patient Monitor with ECG, NIBP, and oxygen saturation (Sp0₂).

[0052] As described below, the processing system 104, in particular software provided on the processing system, is configured to receive the data acquired by the signal/data acquisition system 102 and evaluate the acquired data to determine the acceptability of the physiological signals collected. Notably, although the signal/data acquisition system 102 and the processing system 104 are illustrated as separate components in FIG. 1 , the two components and/or one or more of their respective functionalities can be integrated into a single system or device, if desired.

[0053] FIG. 2 is a block diagram illustrating an architecture for the processing system

104 shown in FIG. 1 according to one embodiment. The processing system 104 of FIG. 2 can comprise a processor 200, memory 202, a user interface 204, and at least one I/O device 206, each of which is connected to a local interface 208. The local interface 208 may be, for example, a data bus with a contra 1/address bus as can be appreciated.

[0054] The processor 200 can include a central processing unit (CPU) or a semiconductor-based microprocessor in the form of a microchip. The memory 202 can include any one of a combination of volatile memory elements (e.g., random access memory (RAM)) and nonvolatile memory elements (e.g., hard disk, read-only memory (ROM), tape, and the like).

[0055] The user interface 204 comprises the components with which a user interacts with the processing system 104 and therefore may comprise, for example, a keyboard, mouse, and a display, such as a liquid crystal display (LCD) monitor. The user interface can also comprise, for example, a touch screen that serves both input and output functions. One or more I/O devices 206 are adapted to facilitate communications with other devices or systems and may include one or more communication components such as a

modulator/demodulator (e.g., modem), wireless (e.g., radio frequency (RF)) transceiver, network card, or other component as can be appreciated by one with ordinary skill in the art.

[0056] The memory 202 comprises various software programs including an operating system 210 and the present system 212 for determining the acceptability of the physiological signals/data acquired by the signal/data acquisition system 102. The operating system 210 controls the execution of these programs as well as other programs and provides scheduling, input-output control, file and data management, memory management, and communication control and related services. The present system 212 for determining the acceptability of the physiological signals/data acquired by the signal/data acquisition system 102 serves to determine the quality of the physiological signals acquired, to determine whether or not a segment of physiological data is exhibiting abnormal behavior, and to reduce false alarms in monitoring environments.

[0057] Components stored in the memory 202 may be executable by the processor

200. In this respect, the term "executable" refers to a program file that is in a form that can ultimately be run by the processor 200. Examples of executable programs may be, for example, a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of the memory 202 and run by the processor 200, or source code that may be expressed in proper format such as object code that is capable of being loaded into a random access portion of the memory 202 and executed by the processor 200, and the like. An executable program may be stored in any portion or component of the memory 200 including, for example, random access memory, read-only memory, a hard drive, compact disk (CD), floppy disk, or other memory components.

[0058] The memory 202 is defined herein as both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon a loss of power. Thus, the memory 202 may comprise, for example, RAM, ROM, hard disk drives, floppy disks accessed via an associated floppy disk drive, compact discs accessed via a compact disc drive, magnetic tapes accessed via an appropriate tape drive, and/or other memory

components, or a combination of any two or more of these memory components. In addition, the RAM may comprise, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. The ROM may comprise, for example, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other like memory device.

[0059] In addition, the processor 200 may represent multiple processors and the memory 202 may represent multiple memories that operate in parallel. In such a case, the local interface 208 may be an appropriate network that facilitates communication between any two of the multiple processors, between any processor and any one of the memories, or between any two of the memories, etc. The processor 200 may be of electrical or optical construction, or of some other construction as can be appreciated by those with ordinary skill in the art.

[0060] The operating system 210 is executed to control the allocation and usage of hardware resources such as the memory, processing time and peripheral devices in the processing system 104. In this manner, the operating system 210 serves as a foundation on which applications depend, as is generally known by those with ordinary skill in the art.

[0061] Various programs (or computer logic) have been described herein. Those programs can be stored on any computer-readable medium for use by or in connection with any computer-related system or method. In the context of this document, a computer- readable medium is an electronic, magnetic, optical, or other physical device or means that contains or stores a computer program for use by or in connection with a computer-related system or method. Those programs can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch instructions from an instruction execution system, apparatus, or device and execute the instructions.

[0062] Now having discussed various embodiments of the system, operation of the system will now be discussed. In the discussion that follows, a flow diagram is provided. It is noted that process steps or blocks in that flow diagram may represent modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps in the process. Although particular example process steps are described, alternative implementations are feasible. Moreover, steps may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.

[0063] Referring next to FIG. 3, shown is a flow chart that provides one example of a number of possible examples of the operation of a system 212 for determining acceptability of the physiological signals acquired by the signal/data acquisition system 102 according to an embodiment of the present invention. Alternatively, the flow chart of FIG. 3 may be viewed as depicting steps of an example of a method implemented in the processor system 104 to evaluate the acquired data. Functionality of the system 212 as depicted by the example flow chart of FIG. 3 may be implemented, for example, in an object-oriented design or in some other programming architecture. Assuming the functionality is implemented in an object oriented design, each block represents functionality that may be implemented in one or more methods that are encapsulated in one or more objects. The system 212 may be implemented using any one of a number of programming languages such as, for example, C, C++, or other programming languages. Alternatively, the system 212 may comprise, for example, such applications as Matlab, Lab View, or any compiled code.

[0064] Beginning with block 310, a subject is monitored with one or more channels of a physiological data capture or monitoring device or devices, such as an

electrocardiogram, a pulse oximeter and/or a respiration trace. Such devices are typical of the ICU, ambulatory monitoring, sleep studies, and ambulatory ECG (Holter) recordings for example. One or more signals are acquired 310 from the device or devices. Signal quality of underlying data is then measured 320 using quality metrics. Physiological parameters (or other selected parameters) are also extracted 330 indicative of the state of the system.

[0065] Signal quality is measured to determine how much the underlying data can be trusted. In one embodiment signal quality metrics are calculated on each acquired signal. Signal quality metrics are a temporal, statistical or other measure of the underlying noise in the acquired signal. One or more quality metrics can be applied to the acquired signal, such as Kurtosis, spectral density, and the like. As an example, if the acquired signal is extremely Gaussian then it is likely that the signal is random noise. The less Gaussian the distribution of the data, then the more likely the signal provides useful information. As another example, the system can also measure how much each heartbeat deviates from an average template.. Thus, the system can measure the cross-correlation of a signal metric against an average signal metric template. A low correlation suggests low quality data. As a further example, the system can also measure spectral density ratios of the acquired signals. At this stage, none of the data is rejected as noisy or not noisy. None of the data is rejected based on a selected threshold. Conversely, all of the data is maintained.

[0066] The present system also extracts 330 and calculates physiological parameters at the same time. For example, it can extract heart rate, breathing rate, respiration rate, blood pressure, and/or oxygen saturation levels and combine them. In an embodiment, at each epoch, for example, every ten seconds, one or more of the physiological parameters can be calculated. Examples of physiological parameters that can be extracted include minimum oxygen saturation, the change in oxygen saturation, minimum heart rate, the change in heart rate, minimum respiration rate and the like. Further examples of physiological parameters that can be extracted are provided in Example 2 and Tables 1 and 2 below.

[0067] The system thus allows several different physiological measurements of the same parameters or variables from different acquired signals. As an example, with reference to Example 2 and Tables 1 and 2 below, the respiration rate (RR) estimated from a spectral or auto-regressive analysis of ECG (RR EDRs^) measures the same thing as RR_PDR«_¾ which is an estimation of respiration rate (RR) from an ECG using frequency analysis of the ECG. The system thus allows two different ways to measure the same parameter from the same signal but two using different estimation methods. The system can then learn which two extracted parameters are better in a given circumstance. [0068] The results of the signal quality measurement 320 and the extraction of the physiological parameters 330 are then provided to a machine learning algorithm, for example an Artificial Neural Network (ANN) or a Support Vector Machine (SVM). A set of labeled data from patients (the larger, the better) is used to train 340 the machine learning algorithm to classify the truth of the events, for example, to estimate whether the underlying acquired signal corresponds to a particular alarm condition or underlying noise. For example, in the case of apnea, the system can classify data as true or false resulting in the classification of thousands of events. This trains the machine learning algorithm to understand not just the physiology measured but also the correlation of the combination of acquired signals regarding heart rate, breathing rate, respiration rate, blood pressure, and/or oxygen saturation that gives rise to apnea in correlation with the noise in the different acquired signals.

[0069] Additionally, the system then learns that noise generally is not independent of a signal assumption. Instead, the noise is correlated with the signal assumption. As an example when one has a heart attack the person typically clutches his or her chest, causing muscle noise. The system, thus, simultaneously learns as well the covariance between different noises and the event monitored by the acquired signal. This covariance is learned without application of heuristics or thresholds. The covariance is learned on a case-by-case basis across a patient call.

[0070] After training, classification of the data is rapid, involving, for example a simple matrix multiplication allowing real time assessment of data quality abnormality. The system then measures 350 the accuracy of the classifier by comparing the outputs of the classifier to the labels on the test data. Optionally, this is followed by selecting 360 subsets of features by employing a selection algorithm, such as a genetic algorithm. The probability is that some of the observed data do not contribute significantly to a data analysis, are independent events that occur with equal probability for all classes of events, or are co-linear with some other features. In such cases, the features are not needed and will reduce accuracy of the classifier.

[0071] Accuracy of the subsets of data is then assessed 370. Non-useful data observations are, for example, 50-70 parameters where it is unknown which subsets of features may be useful. In such cases, optionally employing the feature selection algorithm is preferred. Repeated iterations 375 can result in retention of a most accurate classifier once the selected features are settled. Once a machine learning algorithm is trained and tested on unseen data, it can be deployed in real-time on new data with as much confidence as the performance completed on unseen data.

[0072] One exemplary embodiment of the present system is a neonatal apnea alarm system. In premature infants, there are numerous times when the infant spontaneously stops breathing. This is known as "apnea of prematurity." Unlike in adults, the infant does not always begin breathing again. Instead the infant slowly begins to desaturate until the blood oxygen levels reach 90% or less. At this point the conventional monitoring system creates an alarm and the medical staff then stimulates the infant into breathing again. The respiration signal from the conventional system, however, is so noisy it is of little use in detecting apnea and so any alarm it issues there forth is typically ignored. Unfortunately the oxygen saturation alarm is also often noisy and issues false alarms as often as 90% of the time.

[0073] Example 1. The disclosed embodiment was used for automatic detection of apneic episodes in neonates and was tested on almost 3,000 apnea alarms from 27 patient stays. See Daly et ah, and Monasterio et al. A technique based on the disclosed machine learning algorithm, in particular the SVM, was evaluated using ICU recordings from 27 neonate available from the Multi-Parameter Intelligent Monitoring from Intensive Care II (MIMIC II) database. The MIMIC II database contains physiologic wave form data from over 3500 ICU patients hospitalized at Beth Israel Deaconess Medical Center; Boston, USA. Preliminary results showed a high ability to detect apneic episodes, achieving a sensitivity of 100 %, a specificity of 96%, and an accuracy of 97% in a training set composed of 820 suspected apneic episodes. A sensitivity of 94%>, a specificity of 87%, and an accuracy of 89%) was achieved in a second test set composed of 803 suspected episodes. Data comprised several physiological waveforms sampled at 125 Hz (2 leads of ECG, impedance

pneumogram (IP), and pulse photoplethsmogram (PPG)) as well as 1 Hz numeric time series provided by bedside monitors including heart rate (HR) derived from the ECG, and peripheral Sp0₂ derived from the PPG.

[0074] First, a set of reference annotations was created and served as a gold standard with which to evaluate performance of apnea detection algorithms. An Sp0₂ of 90 % was considered as the threshold to trigger desaturation alarms, as it is an intermediate value within acceptable Sp0₂ levels. In other words, alarms would be triggered at intervals where Sp0₂ was less than 90 %. Two investigators independently annotated desaturation events in all the stays. For each event, the investigators decided among three (3) options: (1) the desaturation is associated to an apnea, which constitutes a positive event, (2) the desaturation is caused by noise or artifacts, which constitutes a negative event, or (3) it cannot be determined whether the desaturation is associated with an apnea or not, which constitutes an unsure event. [0075] Option (1) was chosen if the following conditions were fulfilled: with an interval of 300 seconds before the desaturation event (a) the HR decreases at least 10 beats per minute (bpm), (b) the minimum HR was < 1300 bpm, (c) the quality of the ECT and the PPG waveforms was high, so that one having ordinary skill would expect the waveforms to provide reliable parameter estimates, and (d) no artifacts were present. Option (2) was selected if high levels of noise and/or artifacts were clearly visible in the measured signals. Option 3 was chosen if the event did not meet category either (1) or (2) conditions. The two annotators agreed for 86% of the events, which were then used as the reference set of annotations for classification. This reference set was then split into training and validation subsets for SVM analysis.

[0076] Next, physiological variables were computed. There were four groups of variables: variables related to Sp0₂, HR, RR, and quality of the signals. A total of 20 variables were computed every 5 seconds for a 300-second interval before each desaturation event. Variables related to HR and Sp0₂ were derived from a Sp0₂ and HR numerical series. In each 20 second measurement window, the minimum value and a gradient of the HR and Sp0₂ series were computed. These variables were denoted as min HR, VHR, min S_p0₂, and VS_p0₂ respectively. The gradients were computed using standard least squares regression

[0077] Variables related to respiration rate (RR) were computed in several steps.

First, respiratory signals were derived from ECG and PPG waveforms as follows. ECG beats were detected using an open source implementation of Hamilton and Tompkins' QRS detector. See Hamilton et al. Then, three widely used methods were applied for estimating a respiratory signal from the ECG (ECG-derived respiration, EDR): a method based on the QRS area summation (EDRg), a method based on R-S amplitude tracking (EDRRS), and a method based on an estimation of respiratory sinus arrhythmia (EDR«,¾). See Moody et al. Since PPG waveforms exhibit amplitude fluctuations due to respiration, a similar approach to EDR_RS was considered, and differences between successive peaks and valleys in the signals were computed to estimate a PPG-derived respiratory signal (PDRj_¾).

[0078] Second, RR was estimated from each derived respiratory signal and from IP signal using a breathing rate extinction algorithm described in Nemati et al., which is based on work by Mason and Tarassenko, who utilized autoregressive modeling to estimate the respiratory frequency. The resulting series of derived RR were denoted as RR_EDR«¾ RR_EDRft¾, RR_EDR_G, RR PDR^, and RR IP.

[0079] Third, an improved RR estimation was computed using a data fusion algorithm proposed by Nemati et al. This method is an application of a modified Kalman filter (KF) framework for data fusion to the estimation of RR from multiple physiological sources. See, Li et al., 2008. Kalman filters are employed to obtain independent RR estimates from the series of derived RR, and then the independent estimates are fused taking into account the uncertainty associated with each estimate. In the present work, the fusion algorithm was applied to the series of derived RR for the 300-seconds interval before each desaturation event, and the result was denoted as RR fused. Nemati et al. proposed a variation of the fusion algorithm that makes use of signal quality indexes (SQI), which are explained below. SQI are incorporated in computation of individual Kalman filters and into the fusion step to obtain a more robust RR estimation. In the present study, the fusion algorithm was applied with SQI to the series of derived RR for the 300-seconds interval before each desaturation, and denoted the result as RR_fused¾_/. Finally, a minimum value and a gradient of all RR series every 15 seconds for the 300-seconds interval before each desaturation event was calculated.

[0080] Variables related to signal quality were computed using SQIs as follows. The selected index for determining the quality of PPG, IP, and derived respiratory signals is the spectral purity, an approach proposed in Nemati et al. The spectral purity of a signal is defined in Sornmo and Laguna, as

2 where ω„ is the nth-order spectral moment defined as ω = Γ o)ⁿP(e^iw)do),

with P(d^w) being the power spectrum of the signal. In the case of a periodic signal with a single dominant frequency, Γ stakes the value of one and approaches zero for non-sinusoidal noisy signals. Therefore, in an ideal respiratory waveform one having ordinary skill would expect Ts = 1. The spectral purity was computed for ECG, PPG, and IP signals for the 300- seconds interval before each desaturation using a running window of 20 seconds with a sliding step of 5 seconds. Quality of the ECG signal was determined using an approach proposed by Li et al., who computed the kurtosis of the ECG using a running window of 20 seconds with a sliding step of 5 seconds, and denoted the result as kECG.

[0081] Classification performance individually obtained by each variable was evaluated using univariate receiver operation characteristic (ROC) curve analysis. The temporal relation between apnoea, desaturation, and bradycardia is not completely understood, and significant changes in HR, RR and Sp0₂ do not necessarily appear at the same time before an apnoea-related desaturation event. Therefore, it was necessary to find the optimum evaluation interval for each variable before using it to classify events. To do so, 20 time windows were defined within the 300 seconds interval before each desaturation event. An endpoint of all windows was defined as a beginning of the desaturation event ( mi), and a starting point of each window k was defined as t_end-15 s. In each window k, a minimum value of a variable was selected for classification. This process was repeated for all desaturation events, and a ROC curve was constructed for each variable and each window k. Subsequently, the window corresponding to a maximum area under the curve (AUC) was selected as an optimum evaluation interval for each variable. Finally, for each desaturation event a set of 20 features was created by selecting the minimum value of each variable within its optimum evaluation period.

[0082] Next features were selected. Among the 20 features resulting from ROC analysis, it was not known which of them were most relevant, and which were irrelevant or redundant for false alarm detection. It is preferable to select only relevant features as it results in higher performance with lower computational effort. Therefore, a feature selection algorithm was applied before performing SVM classification. In general there are two types of feature selection methods: filter methods and wrapper methods. For this study a minimum Redundancy Maximum Relevance (mRMR) filter method proposed by Peng et al. , was employed, which computes a rank of most relevant features using mutual information metrics. The result was generation of a feature selection algorithm. The feature with the k^th highest rank as computed by the mRMR algorithm was denoted as _k., and 20 subsets of features were defined as

[0083] After the features were selected, SVM classification was completed. When using SVMs, two questions needed to be addressed: how to select an optimal subset of features, and how to choose an appropriate kernel. For this study, two options for feature and kernel selection were compared. First, an exhaustive search for feature selection with a linear kernel was combined. Next, the feature selection algorithm with a Radial Bias Function (RBF) SVM kernel. These two options are described as follows.

[0084] 1) Exhaustive feature search plus linear SVM: The first option was using a standard SVM with a linear kernel. See Chang and Lin. First, training data were normalized so that features in the training set had zero mean and unit variance, and the test data were scaled to scaling factors used for the training data. Then, an exhaustive search was conducted by training and testing the SV with all possible feature combinations to find those combinations (CK), which provided the best classification performance. Since positive and negative classes in the data were not balanced, a penalty associated with misclassification was multiplied by a factor of r for positive events, and by a factor of 1/r for negative events, with r for equal to the ratio between negative and positive events in the training set.

[0085] 2) mRMR plus RBF-SVM: The second option was using an RBF kernel for the SVM. An RBF kernel has been found to improve classification results over a linear kernel in most cases. See Chang and Lin. When using RBF-SBM, it is necessary to estimate two defining parameters of the RBF: the capacity C and the kernel function parameter γ.

[0086] Performing an exhaustive feature search while looking for the optimum RBF parameters is too computationally expensive, so a different strategy was employed. A feature selection algorithm was applied first, and then the optimum RBF parameters were found using a grid search for each subset of features St as follows: (1) consider a grid space of (C, χ) with log₂C e {-5, -4,...,15} and log₂ γ e {-15, -14,...,3} . ; (2) for each pair (C, ) in the space, perform 10-fold cross validation (CV) on the training set; and (3) choose the pair (C, ) that produces the maximum mean CV accuracy.

[0087] For each subset of features ¾ the selected pair (C, f) was selected to train the

RBF-SVM with the whole training set, and final performance of the classifier with the test set was tested. As in a linear SVM case, every time a classifier was trained and tested (either with data subsets for CV, or with the whole sets for the final evaluation), corresponding training and test inputs were normalized, and a ratio r was recomputed to scale the penalty parameter C.

[0088] Results of the univariate ROC analysis are presented in Table 1 , which contains an optimum evaluation window for each feature and the corresponding AUC. For each feature, a positive (negative) sign in the third column indicates that values above (below) the discrimination threshold are classified as positive events. Maximum AUC, 0.93, was obtained for the minimum HR within an interval of 275 seconds before the desaturation event (feature min HR at window 2). Second highest AUC was obtained for the minimum gradient of HR within an interval of 245 seconds before the desaturation event (feature VHR at window 4) (Table 1).

[0089] The SVM with linear kernel was trained and tested with all possible feature combinations. Prior to RBF-SVM classification, the most relevant features were ranked by applying the mRMR algorithm to the training set, the results of which are set out in Table 2. The four most relevant features were min HR, SQI IP, SQI PPG and VHR. A grid search was conducted for each subset of features to find optimum RBF parameters, the results of which are set out in Table 3. [0090] Table 4 shows 20 different combinations of features. A "1" indicates that a feature is present in the combination, while a "0" indicates that it is excluded from the combination. Tables 5 a and 5b present the classification results obtained with the best 20 feature combinations (those with the highest accuracy in the test set), denoted as C_\ . .. C₂₀. Not all features could be computed for every desaturation event for two reasons. First, there were intermittently missing data in all signals, and second, the appearance of successive desaturation events with less than 20 seconds between them was frequent. Columns 'positive' and 'negative' in Table 5 a and 5b show the number (percentage) of events in which all features of the corresponding combination could be comported. The highest accuracy in the training set (88.6%) was obtained with a combination of 11 features (Ci in Tables 4 and 5a). Seven out of the twenty features are included in all 20 best combinations: min HR, VHR, min RR EDR_RSA, min RR IP, VRR fused, SQI PPG and SQI IP (Tables 5a and 5b).

[0091] A second embodiment of the present system comprises false alarm reduction in the ICU. In this case, 114 signal quality and physiological parameter metrics were extracted from the ECG, blood pressure signal and pulse oximeter signal indicative of heart rate, rhythm and signal quality, and changes in these parameters. Five life threatening arrhythmia alarms were studied, for which a large percentage of the alarms were false. See Tables 6 and 7. Data were broken into testing and training sets again, and a SVM was trained to separate true from false alarms in according with the present disclosure. A genetic algorithm was used to select the most useful parameters from the 114 signal quality and physiological parameter metrics. When blood pressure waveform was available, 56 parameters were chosen. When no blood pressure waveform was available, 27 parameters were chosen. False alarm suppression rates varied from 98% to 38% (depending on alarm) with no true alarms suppressed.

[0092] Example 2. In this example the disclosed system was combined with the

ECG, arterial blood pressure (ABP), PPG, and Sp0₂ signals to suppress false arrhythmia alarms. As the ABP is an invasive measurement, algorithms with ABP and without ABP were compared. First, a novel PPG signal quality assessment method using a dynamic time warping algorithm (See Li and Clifford 2012) and used it to suppress the false alarms, according to the frame which Aboukhalil et al. and Deshmane et al. used.

[0093] Next, HR was estimated from ECG, ABP and PPG separately and fusions of

HR based on Kalman filter and SQIs were made (See Li and Clifford 2012, and Li et al. 2008 and 2009) and used to suppress the false alarms. These traditional methods worked well on asystole and Extreme Bradycardia (EB) alarms, modest on Extreme Tachycardia (ET) alarms, but little on ventricular tachycardia (VT) alarm. To deal with the VT alarm, 114 variables were extracted from ECG, SPB, PPG and Sp0₂ signals, including signal features and SQIs. A genetic algorithm was employed to select optimal variables and an SVM and a MLP ANN were used to classify the alarms between true and false.

[0094] The multi-parameter ICU database (PhysioNet's MIMIC II database, Saeed et al. and Goldberger et al.) was used with ECG, ABP, PPG and Sp0₂ signals and expert annotated alarms were used to develop and evaluate the algorithms. Datasets were similar to those used by Deshmane et al. They included 182 cases and totaled 4107 expert annotated alarms as the gold standard. Alarm types include Asystole, EB, ET, and VT. Each alarm was specified with an availability of different channels of signals and dispatched them into to subsets. A first subset had ECG and PPG available around each alarm. A second subset had ECG, ABP, and PPG available. Table 8 shows a relative frequency of each alarm category and their associated true and false rates. Tables 9a, 9b, and 10 show a distribution of alarms in training, test, and combined sets of the first and second subset.

[0095] Regarding false alarm reductions based on PPG, a novel PPG SQI using a

Dynamic Time Warping algorithm was developed. See Li and Clifford 2012 . A PPG beat dynamic template was built based on 30 second PPG signals as described in Li and Clifford 2012, and a correlation coefficient between each PPG beat and the template was calculated. Three methods were used to fit each PPG beat with the template and three SQI matrices were obtained. A first matrix was a direct comparison. A second matrix was a linear interpolating and re-sampling. A third matrix was a dynamic time warping. A fourth matrix was a clipping detection, which detected a percentage of saturation to a maximum or minimum with a beat duration. These four matrices were fused to classify each beat into excellent (E), acceptable (A), and unacceptable (U). Good beat percentage (E and A) in a 17-second analysis window (13 seconds prior to alarm onset and 4 seconds after alarm) was set as an SQI of PPG.

[0096] An SQI threshold (SQI^) for each type of alarm was set to accept or reject the

PPG as a good quality signal. At first, SQI^ was set strictly to 1 in order to avoid true alarm suppression. The PPG signal with an SQI above SQI^ was considered as a good quality signal and fed into a false alarm suppression procedure as described in Deshmane et al. The Ql_th then decreased gradually and also obeyed a least true alarm suppression rule. The first subset of data was used to evaluate the algorithm in this step.

[0097] HRs and SQIs from PPG, ABP, and ECG were then estimated to suppress false alarms according to the procedure set forth in Li et al., 2008. A 20-second analysis window prior to alarm onset was used to calculate the HR and SQL Seven beat-by-beat HRs were estimated. These were HR_ECG, HRA_BP, HR_PPG, (these three were taken directly from beats interval of corresponding channel), HR_ECG_ABP, HR_ECG_PPG, HRABP PPG, and

HR_ECG_A_{BP PP}G, (these four were taken by fusing two or three signals of corresponding channels using SQIs and Kalman filter). See Li et al, 2008. As there are beats missing of PPG and/or ABP, each beat-by-beat HR was transformed into three kinds of second-by- second HRs by calculating the maximum, minimum, and mean HR from beats around each second.

[0098] These 21 HRs and corresponding SQI were used to suppress false alarms respectively. The second subset of data was employed to evaluate the algorithm in this step. As traditional methods and systems are limited in their ability to deal with VT false alarms, a machine learning algorithm was employed herein in accord with the presently disclosed embodiments.

[0099] First, 114 variables, including 87 features and 27 SQI matrices, were extracted from ECG, ABP, PPG and Sp0₂ signals within the 20-second analysis window. Features included HR, which was extracted from ECG, ABP, and PPG, systolic, diastolic, mean, pulse blood pressure, Sp0₂, amplitude of PPG, and area difference of beat (ADB), with a mean area under the waveform of each beat in the 20-second analysis window of EC, ABP, and PPG. Each feature, except for ADB, had five types, such as maximum, minimum, median, variance, and robustfit. ADB had 4 feature types, including a mean ADB of 5 top beats (ADBmean _to_p5), a maximum of mean ADB of 5 continuous neighbor beats (ADB_max means), variance (ADBvariance), and robustfit (ADBdeita) of beats in the 20-second analysis window, The SQI matrices included SQI matrices from ECG (See Li et al.), ABP (See Sun et al. and Li and Clifford, 2012), and PPG (See Li and Clifford, 2012 and Deshmane).

[00100] A genetic algorithm was used to select optimized variables for alarm classification between true and false alarms. See Goldberg, Leardi et al., and Huang and Wang. Training set of the second subset was used to train and evaluate the algorithm. Fifty chromosomes of a population were selected. A multivariate linear regression was used as a fitness function and root mean squared error (rMSE) was used to estimate error. After each iteration, the chromosomes were sorted by the rMSE. 10 percent of the population with smallest rMSE was kept into a next iteration. A next 45 percent was selected to cross over to create a new 90 percent of the population. Twenty percent of the 90 percent of chromosomes was randomly selected to do mutation with a 2% bit mutation rate. After 100 iterations, genes of a chromosome with smallest rMSE were selected as selected variables for this run. The genetic algorithm was repeated 100 runs and the selected variables were sorted based upon times of being selected to get a variable order of best selected variables for this run. The genetic algorithm was repeated for 100 runs and the selected variables were sorted based upon times of being selected to get a variable order of best selected.

[00101] A SVM algorithm was used to classify the alarms between true (TA) and false

(FA). See Bishop and Chang and Lin. According to an order of the selected variables, various numbers of variables from 1 up to 114 were selected separately to feed into the SVM. A weight from 0.1 up to as large as 10 or even more for true alarm category was used in the SVM to create a ROC curve during the training, and point with no true alarm suppression and best specificity was selected to classify the test dataset. Models were generated and the model with best performance on training and test set together was selected. [00102] The genetic algorithm also was performed with no ABP features present. In this scenario, 60 variables out of 1 14 (48 features and 12 SQI matrices) were used. A standard three-layer feed forward MLP ANN was used to classify the alarms. Input layer nodes were selected from 1 up to 1 14 and increased one-by-one based on output order from the genetic algorithm. Hidden layer nodes were selected from 3 to 30 and the output layer node was the only node to say if it was a true or false alarm.

[00103] Data were resampled to test an effect of applying more weight on the true alarm during training. A number of true alarms from 0.1 times up to 10 times the number of true alarms increasing by a factor of 0.1. False alarm reduction based on PPG was then evaluated. When set SQI^ = 1 , no TA suppression for asystole, EB, and ET on training of the first subset was observed. There was 1 TA suppression for VT when SQI^ = 1. When the Ql_th is gradually subtracted, there was no TA suppression for asystole and EB and only 1 TA suppression for VT until the SQI^ = 0.1 , and the FA suppression rate was increased gradually. However, there was TA suppression for ET when SQI^ was less than 1.

[00104] In response to these results, the SQI^ was set to for ET and 0.1 for asystole,

EB and Vt. With the set SQI^ values, the genetic algorithm obtained the same results on both the test and the training data set. There was no TA suppression for asystole, EB, and ET. Further, there was only 1 TA suppression for VT. The overall FA suppression rates were 83.1% for asystole (80.5%> on training dataset and 86.4%> on test dataset), 90.8%> for EB (93.7% on training dataset and 83.3 on test dataset), and 21.1% for ET (10.3% on training dataset and 26.8% on test dataset). For VT, the overall FA suppression rate was only 1.83% (1.66% on training dataset and 2.01 % on test dataset), making the genetic algorithm, as applied, of marginal use for VT FA alarm suppression. Table 1 1 details the FA and TA suppression performance of the genetic algorithm.

[00105] FA reduction based on HR and SQI from PPG, ABP and ECG was evaluated.

After calculating the 21 HRs and corresponding SQI, each HR and correlated SQI was used to suppress the FAs according to previous procedure. When the HR was fused from several sources, the maximum SQI of these channels was selected as the selected SQI. Table 12 shows a best performance, HR variable selections, and SQI_th on the training set of the second subset of data. With these selected HRs, there was no TA suppression for all types of alarms. For the asystole alarm, the fused HRs of HRECG PPG max, HRABP PPG min, and HRABP PPG mean give best FA suppression results (154 out of 166, 92.8%) with 10% SQI threshold selection. For EB alarm, HR_PPG _mean shows 94.8%> FA suppression rate with SQI threshold from 50%> to 10%). HRA_{BP mean} was selected to create a best result for ET (73.7%>) and VT (3.6%>).

[00106] Using the selected HRs and SQI thresholds, performance of false alarm suppression was evaluated on the test dataset of the second subset. Table 13 shows performance on the test set. Although FA suppression rate for asystole alarm was as high as 96.8%o (91 out of 94), the TA suppression rate is 54.5%> (6 out of 1 1). Further, there was no TA suppression for VT alarm, but FA suppression rate was only 5.1%>.

[00107] False alarm reduction based on the machine learning algorithm (genetic algorithm) was also evaluated. After 100 runs of the genetic algorithm, an order of best selected variables shows that 4 variables were selected 100 times. These variables are ADB_{max me}an5 ECG, ADB_vari_ance ECG from ECG and two SQI matrices of ABP and PPG. Table 14 shows the order of all selected variables. [00108] According to the order of variables, extracted variables were fed from the training set of the second subset of data into the SVM for training, beginning with variable 1 , followed by subsequent additions made one-by-one, until all 114 variables were selected and fed into the SVM. Each time a variable was added, an SVM model was generated and used to classify the training set. With each variable selection, TA data was weighted from 0.1 to at least 10 by a factor of 0.1 each time. ROC curves were generated, an example of which is shown in FIG. 4.

[00109] The model with maximum specificity when sensitivity equals 1 was selected and used to classify the test dataset and obtain the sensitivity and specificity of the signal. FIG. 5 shows sensitivity and specificity curves of all variable selections. Specifically, FIG. 5 shows the sensitivity of the training subset 500, sensitivity of the test subset 501, specificity of the test subset 502, and specificity of the training subset 503. From FIG. 5, a point with 56 selected variables was selected as having maximum sensitivity and specificity. The sensitivity was 1.0 and 0.981 for the training and test dataset, respectively. The specificity was 0.3880 and 0.361 for the training and the test set, respectively. By selecting these 56 variables, the true alarm was weighted, and ROC curves for the training set were obtained. FIG. 6 shows the ROC curves from the selected 56 variables for the training set 600 and the test set 601. Results of alarm suppression are shown in Table 15. When the weight was selected as 4.6, the FA suppression rate is 38.03% and 36.12% for the training set and the test set, respectively. However, there is TA suppression for the test set at a rate of 1.89%). By weighting theTA to 12.3 for the training set, the TA suppression rate was decresed to 0 and the FA suppression rate was decreased to 7.87% and 7.28% for the training set and the test set, respectively. [00110] When ABP features were removed, 60 variables remained to be analyzed.

FIG.7 shows sensitivity and specificity curves of variable selection without ABP features. As shown in FIG. 7, 27 selected variables result in a best specificity of 0.292 and 0.181 for training sets 703 and test sets 702, respectively, and a best sensitivity of 1 and 0.984 for the training set 700 and test set 701 respectively. Results of alarm suppression without ABP features are shown in Table 16.

[00111] The MLP ANN was trained by selecting input layer nodes froml to 114 and hidden layer nodes from 3 to 30, as previously described. Models were generated as previously described and used to classify the training dataset and the test dataset. FIG. 8 shows sensitivity curves of test 800 and training 801 sets and specificity curves of test 802 and training 803 sets of all variable selections with hidden layer nodes of 10.

[00112] In a further embodiment, the presently disclosed system and method were used to detect poor quality ECGs collected in low-resource environments, in particular for intensive care monitoring. The system was adapted for use on short (10 second) 12-lead ECGs. Signal quality metrics used quantified spectral energy distribution, higher order moments, and inter-channel and inter-algorithm agreement. Six metrics were produced for each channel, for a total of 72 features in all. These were then presented to machine learning algorithms for training on provided labeled data. Binary labels were available, indicating whether data were acceptable or unacceptable for clinical interpretation. All data in a first set (training set), and a second set (test set) were re-annotated using two independent annotators as described in Example 1. A third annotator was employed for adjudication of differences between the annotations generated by the first and the second annotators. Events were then balanced and all 1000 subjects in the first data set were used to train classifiers. For this particular embodiment three classifiers were compared. Na^"ive Bayes, SVM, and a MLP ANN classifiers were three chosen. The SVM and MLP provided the best classification accuracies of 99% on the first data set and 95% on the second data set.

[00113] A problem of vetting system and method was specifically directed to ECG quality collected by an untrained user in ambulatory scenarios. The system provided realtime feedback on ECG diagnostic quality and prompted a user to make adjustments in recording data until the ECG quality is sufficient so that an automated algorithm or medical expert may be able to make a clinical diagnosis.

[00114] Example 3. Data were collected by project Sana and freely provided via

PhysioNet. A dataset included 1500 ten-second recordings of standard 12-lead ECGs, age sex, weight, and other possible relevant patient information, such as a photo of electrode placement, were included. Some of the recordings were identified initially as unacceptable or acceptable. Subsequently, participants annotated their own annotations to establish a gold standard reference database of recording quality in the data.

[00115] As stated above, the data were standard 12-lead ECG recordings (leads I, II,

II, aVR, aVL,aVF, VI, V2, V3, V4, V5, and V6) with full diagnostic bandwidth (0.05 through 100 Hz). Each lead was sampled at 500 Hz with 16-bit resolution. The leads were recorded simultaneously for a minimum of 10 seconds by nurses, technicians, and volunteers with varying amounts of training recorded the ECGs, to simulate an intended target user.

[00116] To balance the training and test sets, data were then divided into two sets using a ratio of 2: 1 , with the larger subset used to generate a training set, for which binary annotations (acceptable or unacceptable) were available. The smaller subset was used to generate a test set, for which annotations were not available. Users were required to submit a list of files in the test set together with an estimated classification and an automated scorer immediately posted results.

[00117] ECGs collected were reviewed by a group of annotators with varying amounts of expertise in ECG analysis, in blinded fashion for grading and interpretation. Between 3 and 18 annotators, working independently, examined each ECG, assigning it a letter and a score indicating signal quality according to the following: A (0.95): excellent; B (0.85): good; C (0.75): adequate, D (0.60): poor; and F (0): unacceptable. The average score (^As) was calculated in each case and each record was assigned to one of the three following groups. Group 1 (acceptable) included records with ^As of > 0.70 and NF < 1, wherein NF is a number of grades that were marked as F. Group 2 (indeterminate) included records with ^As > 0.70 and NF > 2. Group 3 (unacceptable) included records with ^As < 0.70.

[00118] Approximately 70% of the collected ECG records were assigned to Group 1, 30% were assigned to Group 3, and fewer than 1%> were assigned to Group 2, reflecting a high degree of agreement among the annotators. Participants were also given an opportunity to grade the ECGs in the data sets for quality control purposes. All data in both the training set and the test set were annotated using two independent annotators with no prior experience in annotating ECGs, and adjudicated by an engineer with over a decade of experience examining and processing ECGs.

[00119] Individual leads were annotated, but due to time constraints, no adjudication of discrepancies was made for individual leads. Further, an extended classification scheme, detailed in Table 17, was employed. This scheme does not render all recordings with a disconnected lead to be unacceptable. This scheme was necessary since a single missing lead should necessarily be cause for rejection. To map the annotations, B-,C-, and D- became D, B+ and C+ became B and C respectively. Also, each class of acceptability was mapped to a numerical score between -1 (worst quality) to +1 (best quality) in order to provide a less quantized set of targets for the MLP and to allow continuous classifiers an option to predict individual classes. The ECGs were no preprocessed prior to annotation.

[00120] Pre-processing of ECGs was completed by downsampling each channel of the

ECG to 125 Hz using an anti-aliasing filter. QRS detection was performed on each ECG channel individually using two open source QRS detectors (eplimited and wqrs) since eplimited is less sensitive to noise. See Li et ah, 2008.

[00121] Six signal quality indices (SQIs) were chosen based on earlier work (See Li et ah, 2008) and run on each of the m = 12 leads separately, producing 72 features per recording:

1. iSQI: The percentage of beats detected on each lead which were detected on all leads.

2. bSQI: The percentage of beats detected by wqrs that were also detected by eplimited.

3. fSQI: The ratio of power P(5-20Hz)/P(0-/„Hz), where„=62.5 Hz is the

Nyquist frequency.

4. sSQI: The third moment (skewness) of the distribution.

5. kSQI: The fourth moment (kurtosis) of the distribution.

6. pSQI: The percentage of the signal xm which appeared to be a flat line

( dx_m I dt <e where e= 25 V ).

[00122] Resulting features were then used to train various machine learning algorithms to classify data as acceptable (1) or unacceptable (-1). To compare possible inconsistencies in labeling between the sets we compared results for training on the balanced training set and testing on the balanced testing set against training on the balanced testing set and testing on the balanced training set. We compared three different classifiers; Na^'ive Bayes (NB), a SVM and a MPL-ANN. Although the example provided in here was performed with 6 SQIs, this number may be greater or less depending on variables chosen. For example, 7 SQIs corresponding to may be used. See Clifford et ah, where all SQIs described above were considered as well as a basSQI, which corresponds to the relative power in a baseline.

[00123] Two classification approaches were tested: a single classifier trained on all 12 leads combined and 12 separate classifiers trained on individual leads. In the 12-lead classifier, input data comprised of 72 features (6 per lead), discussed above, whilst the single lead classifiers were trained on 6 features extracted for each lead individually. All classifiers were provided with class-labels 1 : Acceptable or -1 : Unacceptable, as described above.

[00124] Building classifiers using imbalanced classes, e.g., when one class vastly out numbers other classes, can cause bias and can result in a poor generalization ability of the classification model. When prior probabilities (and a Bayesian training paradigm) are not used to overcome this problem, an alternative is to balance training classes. In a balanced data set, equal numbers of examples are selected from each of the classes allowing for a more accurate model. In this example, the dataset was balanced by rendering or bootstrapping the unrepresented class to be equal to the more numerous class.

[00125] Linear Discriminant Analysis (LDA) attempts to find a linear combination of features that characterize or separate two or more classes. LDA is closely related to analysis of variance (ANOVA) and regression analysis, which also attempts to express one dependent variable as a linear combination of other features or measurements. However, rather than a dependent variable being a numerical quantity, LDA uses categorical variables (the class labels).

[00126] For a vector of observations x, and a scalar class y, LDA assumes that the conditional probability density functions p(x\y =— 1) and/?(x y = 1) are both normally distributed with mean and covariance parameters ( _₁∑_J,₌_₁) and ( ₁∑_J,₌₁), respectively.

Under this assumption, the Bayes optimal solution is to predict points as being from the second class if the ratio of the log-likelihoods is below some threshold S„ so that

(^ - -ι)^Γ ^₌_₁ (^ - -ι) + ^1η

(* - μι f∑ ^~y_=l (^χ - μι ) + ^ln |^∑^=i I < ^δ·

[00127] LDA also makes the simplifying homoscedastic assumption that the class covariances are identical (∑₎₌_₁ =∑_y=i = ∑) and that the covariances have full rank. In this case, several terms cancel and a decision criterion, discussed above, becomes a threshold on a dot product

φ· x < γ

where γ\§ a constant and

[00128] Naive Bayes is a basic probabilistic classifier. For a feature vector x with D dimensions, the Na^'ive Bayes classifier is given in a problem of automatically identifying trust as

D

p(C_k I x)aY[p(x_d I C_k)p(C_k)

d=l where Xd is a <i-th element of a feature vector Xd and Ck is a posterior probability of class k. See Bishop. Class-conditional probabilities p(xd | Ck) were chosen to be Gaussian distributions whose parameters were adjusted in a usual maximum likelihood framework (see Bishop) and is readily implemented in MATLAB. Also, prior class probability p(Ct) was set to be uniform, which is justified because classes were balanced.

[00129] Performance of Support Vector Machines for classification of ECG features was also examined. Support Vector Machine (SVM) classification uses a principle maximum margin hyperplane and uses a "kernel trick" to transform the data into a high-dimensional feature space for linear classification. In this example, a linear kernel SVM which has an objective function

N N,N

/(«) =∑«„— ∑ a_tta_my_tty_mk(x„,x_m)

n=l =l,m=l

was used, wherein the vector x„ is a n-y_n training vector from a set of N training examples, y_n is the associated class label (-1 / +1) and an is a n-y_n Lagrange multiplier and is subject to constraints

N

0 < a_n < C∑a_ny_n = 0.

«=1

[00130] A kernel k x_n;x_m) was chosen to be a radial basis function kernel and training of the classifier (determining the values for a_n) was based on a Sequential Minimal

Optimization (SMO) algorithm according to Bishop, as implemented in Matlab. Slack variables' trade-off parameter C was optimized by grid search within a range of 1 to 10³ and a scale of the RBF kernel was optimized by grid searching within the range of 0.1 to 8. As with the Naive Bayes classifier, a classifier was trained on the 6 features extracted from each of the 12 leads and a single classifier on all 72 features combined.

[00131] A standard three-layer feed-forward MLP was used in which input nodes were fully connected to a next hidden layer and in turn, to an output layer. The output layer consisted of a single node. Thus, full network function for M hidden and D input nodes is given by

1

where σ(α) is the sigmoid mapping function and <» . , ωί are the weights to

1 + exp(-a) the hidden and output layers, respectively, oj and co_j are respectively the input and hidden layer bias terms. Training of the neural network (determining the values for of-) , , and ) was based on a scaled conjugate gradient algorithm according to

Bishop, as implemented in Matlab's Neural Network Toolbox. The stopping criteria were; 100 epochs max, performance-- 10^~3 and gradient-- 10^"6.

[00132] During training, the training set was divided automatically into 70% training,

15% validation and 15% testing. Using a resulting validation set, a number of nodes (A¾ ) in the hidden layer was chosen to be that which provided greatest accuracy within a range of A¾ = 3...30.

[00133] An input layer comprised of 72 nodes and the output layer was a single node, representing the probability of good (1) or bad (-1) quality data. Since there is, at most, 1000 training examples in the training set and it is desired that a number of free parameters (weights in the MLP) to be approximately one tenth of this or less (See Bishop), then the number of hidden nodes must be restricted to about 13 (< 1000/74 if we include the bias weights). With the rendering or bootstrapping of the less frequent class, higher values of Ny are permissible. [00134] For single lead classification, since none of our metrics except skewness were lead-specific, 12,000 training examples were used, and a restriction on a number of possible hidden nodes rose to 162. Classifier training strategy for each lead implies that a suitable classifier fusion strategy must be chosen. Three fusion mechanisms were considered: (a) simple averaging of classification probabilities; (b) averaging of classification log-odds; and (c) empirical density estimation. Given a posterior class probability of a z^'-th test sample ¾, pl(ci I xi), estimated by classifier (the Na^'ive Bayes Method or the ANN) for lead 1 , an average predictive class probability was performed simply by calculating

1 ¹²

Pavgi I x_i) =—∑pl(c_i I .

1=1

[00135] Clearly, this approach takes a very simplistic view and normal-distributed view of predictive class probabilities. Thus, standard deviation estimates using this approach can lead to fused probability estimates outside the range of [0,1] and thus be impossible to interpret. To combine outputs of multiple multilead classifiers (a voting strategy) a simple voting approach was employed. Three methods with highest results on the test set were selected and a majority voting strategy led to a class selection.

[00136] Since the single lead approach produces 12 classifications (one for each lead), and competition requires a single classification per 12 lead recording, the 12 classifications must be combined in some way. This can be treated as either another classification problem, and train a second classifier (with 12 inputs and one output), or an approach previously described may be used.

[00137] A chosen approach involved dividing a sum of scores of each individual channel by 12. An ROC curve was then plotted and an optimal threshold was calculated. An additional step was also added, to override results obtained when a flat line was detected. Results of applying each classifier to training set and test set given in Tables 18 to 22. Table 18 shows classification results of the SVM. Table 19 shows classification results of the Naive Bayes method. Table 20 shows classification results for the MLP. Table 21 shows competition entries with accuracy of classifiers on different data and annotations. Table 22 shows classifier accuracy.

[00138] The SVM and MLP methods performed best, with classification accuracies of

99% on the training data and 95% on the test data using customized labels. This produced a score of 92.6%> on unseen test data labels. Training sets and testing sets were swapped around to compare annotation consistencies between the two sets. Note a drop in

performance when the test set is used for training, indicating that there are inconsistencies between the two data sets, or that only 500 training patterns is insufficient to train the classifiers

[00139] It was then attempted to cohere the customized labels with unseen competition labels by relaxing criteria for rejecting leads. This provided a score of 93.6%. The single channel approach yielded a lower accuracy. For the MLP, Entry 4, Table 21, the number of hidden nodes was 25 (achieving an accuracy of 0.988 on training Set-b and a score of 0.922. For Entry 3, Table 21, the number of hidden nodes was 12, achieving an accuracy of 0.972 on training Set-a, 0.952 testing on Set-b and a challenge score of 0.902. A best result on the balanced data (training accuracy on Set-a of 0.987 and testing on Set-b of 0.948) was achieved using 16 hidden nodes. Given these results the present system and method for classifying the quality of ECGs present a novel and completely general approach to the signals or events. [00140] Example 4. A similar study was conducted using CinC Physionet 2011

Challenge data (See Behar et al. 2012) and improved quality metrics. For this study, 1500 10-second recordings of standard 12- lead ECGs with full diagnostic bandwith were used. Medical personnel and volunteers with varying amounts of training in ECG recording performed the ECG recordings. Similar to Example 3, the data was balanced by generating additional bad quality data from good quality records by adding noise to clean ECGs. Again data was distributed in a 2:1 ratio into two subsets, as in Example 3. Thus, there was a first data set comprised of a training set and a test set. The second dataset comprised a balanced training set and test set. Finally a third dataset was built from a MIT-BIH arrhythmia database.

[00141] In the recordings comprising the third database, locations of arrythmias and premature atrial and ventricular beats were identified. Next, each channel of ECG was downsampled to reduced computation complexity. QRS detection was performed in a similar fashion to that in Example 3. However, different SQIs were employed which included two different SQIs. The chosen SQIs were as follows:

(1) pSQI: Relative QRS complex power.

(2) kSQI: Fourth moment (kurtosis) of the signal.

(3) basSQI: Relative baseline power.

(4) bSQI: Beat percentage detected by eplimited and wqrs.

(5) rSQI: Ratio of number of beats detected by eplimited and wqrs.

(6) pcaSQI: Ratio of a sum of eigenvalues associated with five principal

components of a sum of all eigenvalues obtained by principal component analysis applied to an R-peak aligned ECG cycles detected in a window by eplimitedi segmented 100ms either side of the R-peak.

[00142] For the machine learning step, leads from the balanced and unbalanced dataset were used as the training set and the third set was used as the test set. For each record, all 6 SQIs were computed and used as input features of a SVM classifier. Like Example 3, the LIB-SVM library with a Gaussian kernel was used, employing similar parameters to

Example 3. By adding the rSQI and the pcaSQI, an increase in accuracy was observed. Accuracies of 97.9% and 97.1% were achieved on the CinC training and test sets, respectively. When considering all six SQIs, a 98.0% accuracy was achieved on both the training set and the test set (arrhythmia dataset).

[00143] Although exemplary embodiments have been shown and described, it will be apparent to those of ordinary skill in the art that a number of changes, modifications, or alterations to the disclosure as described may be made.

Claims

What is claimed is 1. A method, comprising:

acquiring a physiological signal;

statistically analyzing signal noise to determine a physiological signal quality;

training a machine learning algorithm to estimate probability of whether the physiological signal corresponds to either an alarm condition or an underlying noise using a set of labeled data;

measuring accuracy of the estimate;

selecting a most-accurate estimate;

validating a trained machine learning algorithm using an independent test dataset; and employing a trained and validated machine learning algorithm in real-time on new data.

2. The method according to claim 1, further comprising the step of extracting physiological parameters indicative of a system state prior to training the machine learning algorithm.

3. The method according to claim 2, wherein extracting the physiological parameters is followed by combining some or all extracted physiological parameters.

4. The method according to any one of claims 1-3, further comprising:

selecting a subset of features using a feature selection algorithm followed by assessing accuracy of the selected subset of features prior to training the machine learning algorithm or selecting the most accurate estimate.

5. The method according to claim 4, the feature selection algorithm comprising a genetic algorithm.

6. The method of any one of claims 1-5, used as one or more of an apnea alarm, an electrocardiogram alarm, or a photoplethysmogram alarm.

7. A method, comprising:

statistically analyzing noise in a physiological signal to determine a signal quality; and

training a machine learning algorithm to determine if the physiological signal corresponds to either an alarm condition or an underlying signal noise.

8. A system, comprising:

a data acquisition system configured to acquire physiological data; and

a processing system coupled to the data acquisition system, the processing system being configured to receive data acquired by the data acquisition system, the processing system further being configured to estimate a probability that the acquired physiological data corresponds to either an alarm condition or an underlying noise of the acquired data.

9. The system of claim 8, the processing system comprising: a local interface; and

a processor, memory, a user interface, and an I/O device, each coupled to the local interface.

10. The system of claim 8 or 9, the processing system comprising a mobile application for a mobile device.

11. The system any one of claims 8-10, the data acquisition system and the processing system being integrated into a single device, or residing on separate devices.