WO2023096632A1

WO2023096632A1 - Method for false alarm prediction and severity classification in event sequences

Info

Publication number: WO2023096632A1
Application number: PCT/US2021/060587
Authority: WO
Inventors: Laleh Jalali; Adriano ARANTES; Dipanjan Ghosh; Ahmed FARAHAT; Chetan Gupta
Original assignee: Hitachi, Ltd.
Priority date: 2021-11-23
Filing date: 2021-11-23
Publication date: 2023-06-01

Abstract

Example implementations described herein provide systems and methods for predicting false alarm events from equipment and involve receiving an event sequence comprising events from the equipment; and generating a true or a false alarm label for each event according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence. Example implementation described herein may provide for classifying severity of events from equipment and involve receiving an event sequence of events from the equipment; identifying true alarm events in the event sequence; and automatically generating a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event.

Description

METHOD FOR FALSE ALARM PREDICTION AND SEVERITY CLASSIFICATION IN EVENT SEQUENCES

BACKGROUND

Field

[0001] The present disclosure is generally directed to event prediction, and more specifically, to systems and methods for predicting false alarm events from equipment generated event sequences. Additionally, the present disclosure provides for systems and methods for predicting event severity for true alarm events.

Related Art

[0002] False alarm reduction is a process of filtering out false alarm events in an event sequence. This prevents time and resources from being wasted due to reacting to the false alarms. Too many false alarms increase chances of analysis mistakes (e.g., ignoring the true alarms) and reduce analysis accuracy (e.g., it is hard to identify true alarms from many false alarms). In addition, identifying false alarms provides an opportunity to issue constructive feedback to equipment designers for rectifying related underlying conditions, for example, by tightening alarm trigger thresholds.

[0003] Moreover, some alarms in an event sequence may require immediate attention from system administrators due to their severe nature, while attention to others can be postponed. For example, a fault code with high severity that leads to a breakdown in a truck in a near future may need to be assessed immediately, while another fault code with low severity can be addressed during the next scheduled maintenance.

[0004] An event may be an observable and measurable signal that is either an occurrence of an activity or an indication of an aberration in the life span of an equipment. There may be two categories of events: 1) normal alarms that indicate an occurrence of an event or activity (e.g., a scheduled maintenance or change of equipment status), and 2) an alarm that indicates an aberration (e.g., a fault code in a piece of equipment or Denial-of-Service (DoS) attack in a network). Events can further be categorized as a true alarm or a false alarm. A false alarm may occur when an alarm is issued for an unexpected hazard/failure, but that hazard/failure never actually materializes. For example, the accumulation of black soot inside a furnace heat exchanger of a truck might generate a fault code. However, when the soot bums away while the truck speeds up in a highway, the fault code may disappear by itself, indicating that the fault code is a false alarm.

[0005] Related art implementations provide for prediction of false alarms in sensor-based security systems. Various systems such as surveillance systems, intrusion detection, and fire systems might have different types of sensors. Failures of these sensors can be significant sources of false alarms. In some related art implementations, collected sensor information may be analyzed to detect changes in the operational characteristics of a sensor device. If a change is detected, a request for maintenance on the sensor device may be generated. To detect changes from normal behavior, unsupervised learning algorithms may be applied to compare historical and current sensor readings. However, these related art implementations utilize an unsupervised clustering technique that clusters the collected information and determines falsity from the clustered information based on deviation from normal behavior.

[0006] Other related art implementations provide for an intrusion detection system alerts mechanism. In such related art implementations, Intrusion Detection System (IDS) alerts are analyzed for identifying false alarms. A machine learning module is configured to take a manually labeled training data set as an input and produce association rules and corresponding purity values as an output. The purity of each rule indicates how likely the alert is of being a true alarm. In these related art implementations, a labeled training data set, manually labeled by a domain subject matter expert, is provided as input. However, not many applications have access to labeled data sets and manual labeling can be very tedious and costly. Also, in many instances, the subject matter expert that is performing the labeling may not know whether the alarm is a true or false at the time the alarm is generated.

[0007] Some other related art implementations provide for false alarm detection in a physical security system that monitors alarms in the context of physical intrusions. According to these related art implementations, when an alarm is triggered at a supervised premise, an Alarm Receiving Center (ARC) tries to contact a customer by phone to verify the alarm. However, more than 90% of the alarms are false. To identify false alarms, these related art implementations defined a heuristic to infer labels of alarms based on a duration of the alarm as a threshold. That is, the quicker the alarm was reset after being triggered, the higher the likelihood that the alarm was false. SUMMARY

[0008] Implementations disclosed herein provide for a false alarm prediction method for determining whether an alarm (referred to herein as an alarm event instance or event instance) is a true alarm or a false alarm. According to various implementations disclosed herein the determination may be based on sequential (e.g., followed by) and/or temporal (e.g., within) relationships between event instances in an event sequence corresponding to a piece of equipment.

[0009] Additionally, some implementations also provide for a severity level prediction method to determine a severity level of true alarm events based on thresholds, for example, how much time and/or how many operating cycles remain to a repair and/or failure.

[0010] Implementations disclosed herein may provide for, not only filtering false alarms and preventing time and resources from being wasted by reacting to them, but also to classify the true alarms to different severity levels, which assist equipment administrators and/or operators in prioritizing interventions and preventing costly breakdowns.

[0011] Thus, various example implementations disclosed herein provide for generating an event sequence by temporal ordering received event instances (e.g., alarms and other normal events) from a piece of equipment and determining whether an event instance is a true alarm or false alarm based on applying a logic process flow that utilizes sequential (e.g., followed by) and/or temporal (e.g., within) relationships between events instances to automatically label each alarm event. Automatic labeling may be very cost effective since it reduces, and may event eliminate, the need for a subject matter expert to manually label each instance of a false alarm. Furthermore, each alarm event instance of an event sequence may be labeled based on the contextual relationships between events, as opposed to specifics of an alarm (such duration of an alarm as relied on in related art implementations) that might not be available in many use cases.

[0012] Example implementations disclosed herein may also learn a function for predicting labels for alarm events based on equipment information, event data, sensor data, and repair history. In addition, implementations herein may also provide a plurality of thresholds may be defined based features of true alarm event instances (e.g., based on time and/or operating cycles to repair/failure) for labeling true alarms to different severity levels. For true alarm event instances, implementations herein may learn a function for predicting severity levels based on equipment information, event data, sensor data, or repair history.

[0013] Accordingly, implementations disclosed herein provide a logic process flow for labeling alarm events as false or true alarms, and utilize supervised machine learning for prediction. Whereas, related art implementations rely on clustering techniques. The logic process flow for labeling alarm events may be used to generate false alarm and true alarm labels to provide a labeled data set, which can be used as an input for training machine learning models. Whereas, related art implementations required a subject matter expert label events, which can be costly and tedious. Furthermore, in many instances, a subject matter expert may not know if a given alarm is true or false. A lot of context to the situation and environment surrounding the alarm may be required in order for the subject matter expert to determine whether the alarm is true or false.

[0014] Aspects of the present disclosure can involve a method for predicting false alarm events from equipment, the method involving receiving an event sequence comprising a plurality of events from the equipment; and generating a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence. The set of conditions comprising one or more of identifying an intervention event in the plurality of events, identifying an escalation event in the plurality of events, and identifying a repetition event in the plurality of events.

[0015] Aspects of the present disclosure can involve a method for classifying severity of events from equipment, the method involving receiving an event sequence comprising a plurality of events from the equipment; identifying true alarm events in the event sequence; and automatically generating a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event.

[0016] Aspects of the present disclosure can involve a system for predicting false alarm events from equipment, the system involving one or more memories configured to store instructions; and one or more processors coupled to the one or more memories. The one or more processors configured to receive an event sequence comprising a plurality of events from the equipment; and generate a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence. The set of conditions comprising one or more of identifying an intervention event in the plurality of events, identifying an escalation event in the plurality of events, and identifying a repetition event in the plurality of events.

[0017] Aspects of the present disclosure can involve a system for classifying severity of events from equipment, the system involving one or more memories configured to store instructions; and one or more processors coupled to the one or more memories. The one or more processors configured to receive an event sequence comprising a plurality of events from an equipment; generate a true alarm event in the event sequence; and automatically generate a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event.

[0018] Aspects of the present disclosure can involve a system for predicting false alarm events from equipment, the system involving one or more memories configured to store instructions; and one or more processors coupled to the one or more memories. The one or more processors configured to receive an event sequence comprising a plurality of events from an equipment; and generates a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence. The set of conditions comprising one or more of identifying an intervention event in the plurality of events; identifying an escalation event in the plurality of events; and identifying a repetition event in the plurality of events. The one or more processors are also configured to automatically generate a severity label for each event in an event sequence for which a true alarm label is generated, based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, wherein the severity label is indicative of a level of severity of each event for which a true alarm label is generated.

[0019] Aspects of the present disclosure can involve an apparatus for predicting false alarm events from equipment, the apparatus involving a means for receiving an event sequence comprising a plurality of events from the equipment; and a means for generating a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence. The set of conditions comprising one or more of identifying an intervention event in the plurality of events, identifying an escalation event in the plurality of events, and identifying a repetition event in the plurality of events.

[0020] Aspects of the present disclosure can involve an apparatus for classifying severity of events from equipment, the method involving a means for receiving an event sequence comprising a plurality of events from the equipment; identifying true alarm events in the event sequence; and a means for automatically generating a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii)operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event.

BRIEF DESCRIPTION OF DRAWINGS

[0021] FIG. 1 illustrates an example tree diagram of a taxonomy for classification of alarm events, in accordance with an example implementation.

[0022] FIG. 2 illustrates an example table of scenarios and outputs for the taxonomy of FIG. 1, in accordance with an example implementation.

[0023] FIG. 3 illustrates an example table of a set of conditions for labeling an alarm event instance as a false alarm, in accordance with an example implementation.

[0024] FIG. 4 illustrates an example table of a set of conditions for labeling an alarm event instance as a true alarm, in accordance with an example implementation.

[0025] FIG. 5 illustrates an example logical process flow for labeling an alarm event instance, in accordance with an example implementation.

[0026] FIG. 6 schematically illustrates an example event sequence, in accordance with an example implementation.

[0027] FIG. 7 illustrates a flowchart of an example process for false alarm prediction, in accordance with an example implementation.

[0028] FIG. 8 schematically illustrates a performance metric of false alarm prediction, in accordance with an example implementation. [0029] FIG. 9 illustrates an example process for labeling event severity, in accordance with an example implementation.

[0030] FIG. 10 illustrates a system involving one or more equipment and a management apparatus, in accordance with an example implementation.

[0001] FIG. 11 illustrates an example computing environment with an example computer device suitable for use in some example implementations.

DETAILED DESCRIPTION

[0002] The following detailed description provides details of the figures and example implementations of the present application. Reference numerals and descriptions of redundant elements between figures are omitted for clarity. Terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the term “automatic” may involve fully automatic or semi-automatic implementations involving user or administrator control over certain aspects of the implementation, depending on the desired implementation of one of the ordinary skills in the art practicing implementations of the present application. Selection can be conducted by a user through a user interface or other input means, or can be implemented through a desired algorithm. Example implementations as described herein can be utilized either singularly or in combination and the functionality of the example implementations can be implemented through any means according to the desired implementations.

[0003] Implementations disclosed herein may be used for any system or equipment that generates alarm events. Example implementations disclosed herein provide for a formal definition of an alarm event and further defines a concept of events as one of a false alarm and true alarm in a system or equipment. Implementations disclosed herein provide for predication of false alarm events in an event sequence. Events that are predicted as false alarm can be safely filtered from the event sequence, thereby reducing and/or preventing time and resources from being wasted due to reacting to false alarms. Some implementations disclosed herein may also classify true alarms events according to levels of severity, which may be utilized by system administrators to assist in prioritizing interventions to address the true alarm events and preventing costly breakdowns and equipment failures. [0004] As used herein, equipment may refer to any physical device, machine, structure for performing an activity. Such equipment can include stationary equipment or equipment with sensor systems such as coolers, water tanks, air compressors, electrical lines, as well as mobile equipment or equipment such as, but not limited to, motor vehicles (e.g., automobiles, trucks, buses, etc.), commuter vehicles (e.g., buses, trains, airplanes, etc.), construction equipment, moving robots, robotic arms, as well as any other equipment that are integrated into the industrial networks or can be part of an industrial network. Other example equipment may include, but not limited to, equipment and components for establishing a network, such as antennas, routes, modems, communication devices, etc.

[0005] According to various example implementations disclosed herein, an event may refer to an observable and/or measurable signal that is either an occurrence of an activity or an indication of an aberration in the life span of a system and/or equipment. The observable and measurable signals may be generated by sensors included in the equipment, in an environment surrounding the equipment to monitor the status thereof, and otherwise communicably (e.g., via wired or wireless communication) coupled to the equipment to monitor the status thereof. Alarm events may refer to encoding of raw signals output from sensors by equipment designers and/or subject matter experts to summarize important domain knowledge that needs be communicated to the equipment operators and repair technicians. The encoding may be in the form of alarm events, such as, fault codes and the like. The equipment may output alarm events that may be temporally ordered as an event sequence comprising a plurality of alarm events (also referred to as alarm event instances). The event sequence provides a symbolic view of dynamics and abnormalities in the equipment.

[0006] Each alarm event of an event sequence may comprise an event type and a timestamp indicating the time that the alarm event occurred. An event alphabet E predefines a set of event types present in such a sequence. The event alphabet of types may be predefined for each use case by subject matter experts. Accordingly, each alarm event instance may be represented as a pair (A, /), where E represents event types in E and where E E£ is an event type and t is the occurrence time of the event E. Event sequence S may be represented as a triplet <t_s , t_e , S> with t_s as the start time of the sequence 5, t_e as the end time of the sequence S, and S is an time ordered sequence of the events occurring within the time window defined by t_s and t_e, for example, <(Ei, Ei), (E2, T2), ...., (EM, TM)> with Ei E E, Vi : t_s< E < t_e and VEt , Ej , i <j : Ei < , where E represents the timestamp for each respective event E and i and j each represents an instance of the timestamp for a respective event E (e.g., Ej corresponds to Tj and Et corresponds to Tt).

[0007] As opposed to time-series data where measurements are taken at successive equally spaced points in time, alarm events can occur at any given point in time in the life span of equipment. An equipment control unit (e.g., as implemented the computing environment 1100 coupled to the equipment as described in connection to FIGS. 10 and 11) can be configured to receive the observable and measurable signals from sensors, record alarm events, and communicate alarm events that reflect important changes in the underlying sensors and other important activities in the equipment. These alarm events may be defined by equipment designers and/or subject matter experts to summarize the raw signals received from sensors monitoring the equipment and encode domain knowledge to be communicated to the equipment users and repair technicians. As the number and varieties of event types increase, it is more challenging for equipment designers and repair technicians to keep track of the event types. There can be thousands of different event types in a given system or equipment.

[0008] Accordingly, implementations disclosed herein provide a taxonomy that is constructed based on event types and the multiple scenarios that can occur due to such event types in equipment, for example, as shown in FIGS. 1 and 2. The taxonomy of FIG. 1 and the scenarios of FIG. 2 establish a structure for collections of events generated by an equipment, which may be unmanageable as the number and varieties of the event types increase. From the taxonomy, equipment operators and/or administrators may be able to make sense of the multiple events.

[0009] For example, alarm events of an event sequence may be labeled (e.g., classified) as true or false alarm events, and, for true alarm events, classified into severity levels according to the taxonomy and scenarios of FIGS. 1 and 2. Each scenario of FIG. 2 may correspond to a node of the taxonomy of FIG. 1, and a corresponding output as shown in FIG. 2 leads to a following node of the taxonomy.

[0010] For example, from the illustrative taxonomy shown in FIGS. 1 and 2, equipment operators and/or administrators can recognize different categorizations for alarm events. If an alarm event is for information only or does not lead to (e.g., is followed by) an intervention (e.g., a repair, replacement, maintenance, equipment failure, etc.), no action need be taken. If the alarm event leads to an intervention, it may be determined whether the alarm event is false or not. If the alarm event is a false alarm, no action is needed. In some implementations, the alarm event need not even be shown to the operator or administrator (e.g., not displayed on a dashboard). If it is a true alarm, then the taxonomy defines a severity level to see if the instance of this alarm event will lead to some breakdown and to prioritize an intervention. In the illustrative example, severity is provided as three levels (e.g., low, medium, high). However, implementations herein are not limited to three levels, and may be any number of severity levels according the use case. For example, severity may be provided as five or more, 10 or more levels, etc. From the defined levels, the taxonomy translates the levels to lower levels corresponding to intervene later, medium levels translates to intervene soon, and higher levels translates to intervene immediately.

[0011] In the implementations disclosed herein, a false alarm may occur when a warning and/or alert is issued that represents an unexpected failure and/or hazard in the equipment, but the represented failure and/or hazard never actually materializes in the equipment. An alarm event might be a false alarm due to one or more of the following: the conditions that triggered the alarm are temporary and will not continue to hold (e.g., cold-temperature start-up with the diesel exhaust fluid (DEF) frozen); the underlying conditions that triggered the alarm will resolve by themselves, without intervention (also referred to as self-heal) (e.g., soot accumulation in a filter that bums out by itself due to truck’s speed on a highway); the definition of the underlying conditions does not reflect a true alarm under the certain circumstances or operating conditions for which the underlying conditions occurred (e.g., low coolant fault occurring when the engine starts, the reporting thresholds should have been tightened), and so on. The preceding list is not comprehensive and the implementations disclosed herein are not limited to these false alarms only. Other definitions of false alarm due to system requirements and components’ characteristics may exist that are within the scope of the implementations disclosed herein.

[0012] False alarm reduction is the process of filtering false alarms in an event sequence. False alarm reduction may reduce and even prevent time and resources from being wasted due to reacting to false events. Labeling false alarms can be done manually by a human or automatically. Manual labeling is very time consuming and costly. In addition, it requires subject matter experts to go through the event sequence and identify false alarms. This task cannot be performed by crowdsourcing or Mechanical Turks since a comprehensive knowledge of the system and the characteristics of the components are required. Moreover, even a human expert may not be able to successfully label an event E in a sequence without considering the context and relationships between events in the sequence (e.g., if E is followed by or preceded by other events). In addition, as the number of the equipment grow, manual labelling is not scalable and it may become unfeasible, unpractical, and costly.

[0013] Due to limitations of manual labeling, implementations disclosed herein provide for and utilize an automatic algorithm for labeling alarm events, in an event sequence, as false or true alarm. A set of conditions may be defined based on sequential (e.g., followed by) relationships and/or temporal (e.g., within) relationships between events of an event sequence to automatically label each event as a true alarm or false alarm. Subsequently, implementations disclosed herein may then be fed the resulting labeled data to a machine learning function to teach and devise a function configured to predict a true or false label for an alarm event based on equipment information, event data, sensor data, and repair history of equipment.

[0014] Example conditions used to label false and true alarms include, but not limited to:

• Intervention: Looking forward in the event sequence, from the viewpoint of an alarm event instance, for an intervention event (e.g., repair, part replacement) occurring subsequent to the alarm event instance. That is, an intervention condition is satisfied if an alarm event instance (E, TE) is followed by an intervention instance (R, TR) within a period Ti TR - TE < TI).

• Escalation: Looking forward in the event sequence, from the viewpoint of an alarm event instance, for an escalation event (e.g., an event of greater severity than the event instance) occurring subsequent to the alarm event instance. For example, an escalation pattern (E*, TE*) is an itemset containing events of greater severity relative to the alarm event instance. If the event instance (E, TE is followed by the escalation pattern E* within Tc TE* -TE < Tc), then the escalation condition is satisfied.

• Repetition: Looking forward in the event sequence, from the viewpoint of an alarm event instance, for a repetition event (e.g., the same event occurs again) subsequent to the alarm event instance. A repetition condition occurs when an alarm event happens again sometime in the future. If the first alarm event instance (E, TE) is followed by a second alarm event instance (E, T'E) within Tp (T'E- TE < Tp), then the repetition condition is satisfied. [0015] More conditions may be defined based on requirements of a given system and specifications of an equipment.

[0016] FIGS. 3 and 4 illustrate example tables providing a set of conditions for labeling an alarm event instance as a false alarm or true alarm, in accordance with example implementations disclosed herein. FIGS. 3 and 4 provide illustrative examples of conditions and rules for generating labels. Other conditions and/or rules may be provided depending on the system and/or equipment of different use cases.

[0017] FIG. 3 shows a set of conditions (also referred to as rules) for labeling an alarm event instance as a false alarm based on the example conditions described above. For example, for each alarm instance (E, TE) in an event sequence, if conditions of any of the rules 1-6 are triggered, then the alarm event instance (E, TE) in question is labeled as a false alarm. The symbol indicates negation. For example, “-Intervention” means an alarm event instance is not followed by an intervention within Ti (e.g., rule number 1). Rule number 4 means if the alarm is not followed by an intervention within Ti, and it is followed by an escalation pattern within 7c, and the escalation pattern is a false alarm, then the alarm event instance is also false alarm and labeled accordingly.

[0018] FIG. 4 shows a set of conditions for labeling an alarm event instance as a true alarm based on the example conditions described above. For example, for each alarm event instance (E, TE) in an event sequence, if conditions of any of the rules 1-3 are triggered, then the alarm event instance (E, TE in question is labeled as a true alarm. For example, rule number 3 of FIG. 4 means if an alarm event instance (E, TE) is not followed by an intervention within Ti, and a repetition instance (E, T'E) follows the alarm event instance (E, TE) in the future within Tp ( where T'E > TE and T'E - TE < TP), and the repetition instance (E, T’E) is a true alarm, then the alarm event instance (E, TE) is also a true alarm.

[0019] Depending on the system and equipment that generate the event sequence, additional rules can be defined by combining conditions with AND or OR operations.

[0020] FIG. 5 illustrates an example logical process flow 500 for labeling an alarm event instance, in accordance with example implementations disclosed herein. The flow 500 demonstrates an example logic for false alarm labeling according to, for example, the set of conditions of FIGS. 3 and 4. Tables of FIGS. 3 and 4 may be derived from the logic flow 500. While three example conditions are provided herein, the present disclosure is not intended to be limited to only these conditions.

[0021] For example, a plurality of alarm event instances is received and temporally and sequentially ordered according the respective timestamps to generate an event sequence. The event sequence is received and an alarm event instance (E, TE) occurs (502). The next relevant repair instance (R, TR) is extracted from the event sequence (504) and a determination is made as to whether the repair instance (R, TR) occurred within time 7/(506) (e.g., does Ti satisfy the condition TR-TE < Ti). If yes, the alarm event instance (E, TE) is labeled as a true alarm (508). If no, a determination is made as to whether the alarm event instance (E, TE is followed by an escalation pattern (E*, TE*) within time Tc (e.g., TE*-TE<TC) (510). If yes, the alarm event instance (E, TE) is labeled as a true alarm in a case where the escalation pattern (E*, TE*) is a true alarm or a false alarm in a case where the escalation pattern (E*, TE*) is a false alarm (512). In some implementations, the escalation pattern (E*, TE*) may have been labeled according to the logic flow 500 and/or otherwise labeled as a true or false label.

[0022] If the determination at 514 is no, a determination is made as to whether the alarm event instance (E, TE) is followed by a repetition instance (E, T'E) within time Tp (e.g., T'E- TE<TP) (514). If yes, the alarm event instance (E, TE) is labeled as a true alarm in a case where the repetition instance (E, T'E) is a true alarm or a false alarm in a case where the repetition instance (E, T'E) is a false alarm (516). In some implementations, the repetition instance E, T'E) may have been labeled according to the logic flow 500 and/or otherwise labeled as a true or false label. If the determination at 514 is no, the alarm event instance (E, TE) is labeled as a false alarm (518).

[0023] Other conditions may be defined, for example, depending on the use case. Conditions for labeling alarm event instances as a true or false alarm can be determined with a chain combination of any of the defined conditions. Accordingly, implementations disclosed herein are not to be restricted to only the three example conditions. For example, another condition that can be defined is an event instance (E, TE) followed by a “de-escalation” pattern (e.g., a de-escalated event of less severity as compared to the event instance), and the “de- escalation” pattern is not followed by any repetition of the event instance.

[0024] FIG. 6 illustrates an example event sequence, in accordance with implementations disclosed herein. The event sequence 600 may be received from a piece of equipment and comprises a plurality of alarm event instance E1-E7, a plurality of fault codes 602a-g, and a plurality of timestamps 604a-g. Each alarm event instance is associated with a fault code and a timestamp indicating a time of occurrence of the alarm event instances, for example, alarm event instance Ei comprises timestamp 604a and fault code 602a, alarm event instance E2 comprises timestamp 604b and fault code 602b, etc. Fault codes 602a-g may be indicative of the error and/or alarm event instance including an indication of relative severity. For example, the fault codes may indicate the event type E. In the illustrative example of FIG. 6, the equipment may be a truck and the alarm event instances may be NOx-related fault codes generated by the truck and sensor replacement alarm event instance (e.g., replacement of NOx sensor) extracted from a repair history in the historical data of the equipment. In the illustrative example of FIG. 6, Ti = 15 days; Tc = 15 days; Tp = 5 days and event type “ Inlet NOx Signal Erratic" may be a more severe fault than event type “ Inlet NOx Circuit Issue" in an inlet NOx fault category.

[0025] Each alarm event instance E1-E7 may be labeled as a true or false alarm according to the example logic flow disclosed herein (e.g., as described in connection with FIGS. 3-5). For example, FIG. 6 provides an illustrative example of labeling each fault code. At SI, from alarm event instance Ee, example implementations look forward in the event sequence 600 to retrieve an intervention instance, escalation pattern, and/or a repetition instance. In this example, an intervention instance E? (e.g., repair including replacement of an Inlet NOx sensor) is found to have occurred at timestamp 604g (e.g., the timestamp for alarm event instance ET) within time Ti since the occurrence timestamp 604f of alarm event instance Ee. Referring to rule 1 of FIG. 4, at S2, the alarm event instance EEs labeled as a true alarm (e.g., 508 of FIG. 5).

[0026] At S3, from alarm event instance Ej, example implementations look forward in the event sequence 600 to retrieve an intervention instance, escalation pattern, and/or a repetition instance. In this case, an intervention instance is not found within time Ti. However, because the fault code 602f of alarm event instance Ee is the same event type (e.g., erratic inlet NOx sensor signal) as fault code 602g of alarm event instance Ej, alarm event instances Ee occurring at time 604f represents a repetition instance occurring within time Tp since the occurrence timestamp 604e of alarm event instance Es. Thus, with reference to rule 3 of FIG. 4, at S4, the alarm event instance Ejis labeled as a true alarm (e.g., 514 and 516 of FIG. 5). [0027] At S5, from alarm event instance E4, example implementations look forward in the event sequence 600 to retrieve an intervention instance, escalation pattern, and/or a repetition instance. In this case, an intervention instance is not found within time Ti. However, because fault code 602e of alarm event instance Es is a more severe event type (e.g., erratic inlet NOx sensor signal) as compared to the fault code 602d of alarm event instance E4 (e.g., issue with the inlet NOx sensor circuit), alarm event instances Es occurring at time 604e represents an escalation pattern instance occurring within time Tc since the occurrence timestamp 604d of alarm event instance E4. Thus, with reference to rule 2 of FIG. 4, at S6, the alarm event instance Evis labeled as a true alarm (e.g., 510 and 512 of FIG. 5).

[0028] At S7, from alarm event instance Ej, example implementations look forward in the event sequence 600 to retrieve an intervention instance, escalation pattern, and/or a repetition instance. In this case, an intervention instance is not found within time Ti, an escalation pattern is not found within time Tc, and a repetition instance is not found within time Tp. Thus, with reference to rule 6 of FIG. 3, at S8, the alarm event instance E3 is labeled as a false alarm (e.g., 518 of FIG. 5).

[0029] At S9, from alarm event instance E2, example implementations look forward in the event sequence 600 to retrieve an intervention instance, escalation pattern, and/or a repetition instance. In this case, while alarm event instance Es may have a timestamp 604c occurring within times Ti, Tc, and Tp since the timestamp 604b of alarm event instance E2, the fault type of alarm event instance Es is not a more severe fault type than E2 nor are the fault types the same. Thus, an intervention instance is not found within time Ti, an escalation pattern is not found within time Tc, and a repetition instance is not found within time Tp. With reference to rule 6 of FIG. 3, at S10, the alarm event instance E2 is accordingly labeled as a false alarm (e.g., 518 of FIG. 5).

[0030] At Si l, from alarm event instance Ei, example implementations look forward in the event sequence 600 to retrieve an intervention instance, escalation pattern, and/or a repetition instance. In this case, an intervention instance is not found within time T However, because E2 is a more severe event type (e.g., erratic inlet NOx sensor signal) as compared to Ei (e.g., issue with the inlet NOx sensor circuit), alarm event instance E2 occurring at timestamp 604b represents an escalation pattern occurring within time Tc since the occurrence timestamp 604a of alarm event instance Ei. Thus, with reference to rule 4 of FIG. 3, at S12, the alarm event instance Ei is labeled as a false alarm due to E2 being labeled as a false alarm (e.g., 510 and 512 of FIG. 5).

[0031] In some implementations, once all alarm event instances of an event sequence are labeled as a true alarm or a false alarm, the data of the event sequence may represent a labeled data set. The labeled data set may be applied to a supervised machine learning technique for false alarm prediction. In false alarm prediction, predictions are made as to a label to be associated with a given feature vector x for each received alarm event instance. False alarm prediction may be a supervised binary classification problem. A set of features can be extracted from a time window preceding each alarm event instance for which a label is to be predicted. Example features may be extracted from an event sequence that is being examined. For example, features may include the fault codes of other events, operating cycles (e.g., mileage, etc.), timestamps for each alarm event, etc. Other examples may be intervention events detected, for example, in historical data of the equipment. Extracted features can be adjusted depending on system requirements and component characteristics.

[0032] FIG. 7 illustrates a flowchart of an example process 700 for false alarm prediction, in accordance with example implementations disclosed herein. The process 700 may be an example of a false alarm reduction formulated as a binary machine learning process.

[0033] At step 710, process 700 receives a set of labeled training data as inputs. For example, a set of training data labeled according to the flow 500 of FIG. 5 based on the conditions of FIGS. 3 and 4 may be utilized for application to a machine learning system. Thus, a labeled event sequence, according to the FIGS. 3-5 as set forth above, may be input data at step 710.

[0034] At step 720, process 700 extracts and calculates event data features. For example, a set of functions may be applied to the set of labeled data input at step 710 that extracts event data features from the input labeled event sequence. For example, the set of functions may be configured to capture a normalized frequency of each preceding event within a time window (/), as an example of an event data feature. The time window t may be set as desired for each application or use case, for example, x may be 10 days, 5 days, 15 days, etc. As another example, the set of functions may be configured to capture a normalized frequency of each preceding event within an operating cycles window (m). The operating cycles window m may be set as desired for each application or user case, for example, m can be 1000 miles, 500 miles, 1500 miles, etc. Example functions that may be included in the set of functions are, but not limited to, a “count” function that counts a frequency of previous events within a proceeding window, a “TF-IDF” function, and the like.

[0035] At step 730, process 700 determines whether the input data includes sensor measurements for each alarm event. As an illustrative example, in FIG. 6, the timestamps 604 and the fault codes 604 may be examples of sensor data. If the determination at step 730 is Yes, process 700 proceeds to step 740 where process 700 calculates sensor data features and merges the calculated sensor data features with the event data features calculated at step 720. Example sensor data features for sensor measurements include, but are not limited to, time related features (e.g., month, day of month, day of week, day of year, etc.); statistical features (e.g., minimum, maximum, median, variance, standard deviation, kurtosis, skewness, interquartile range, etc.); temporal features (e.g., trend, mean difference, median difference, autocorrelation, etc.); and projection-based features (e.g., wavelet projection coefficient mean and standard deviation, Fast Fourier transform projection coefficient mean and standard deviation).

[0036] Merged data features (e.g., output from step 740) or data features in the case that the determination at step 730 is No are applied to a set of learners at step 750 to effectuate the machine learning process. Example learners may include, but are not limited to, decision trees, random forest, XGBoost, and neural networks. Each learner operates on the training data (e.g., the labeled data) according to a set of parameters of the respective learner. The machine learning process runs until a set number of cycles have been executed, a set time period has elapsed, or a desired accuracy level of the learning is obtained. The thresholds for the number of cycles, time period, and accuracy may be set in advance according to the specific user case on which process 700 is being performed.

[0037] For example, at step 760, process 700 evaluates the learned model based on a false alarm reduction rate and/or a rate of mistakes on true alarms. In various implementations, typical accuracy metrics, which indicate accuracy as a number of correct predictions divided by total number of predictions, may not provide a good measure of model performance. For example, if the input data includes 10% false alarm, a naive model that assigns true labels to all the alarms would achieve 90% accuracy. Accordingly, various implementations disclosed herein, may be configured to leverage a performance metric that provides improved insight on the accuracy of the model performance. [0038] FIG. 8 schematically illustrates the performance metric of the model performance, in accordance with various implementations disclosed herein. FIG. 8 schematically shows an example event sequence 800 having a plurality of alarm events. The event sequence 800 has been assigned labels to generate assigned labels sequence 810. Sequence 810 includes alarm events labeled as false alarms and alarm events labeled as true alarms, shown in FIG. 8 as a black box 812 and white box 814, respectively. The relative length of each box 812, 814 may be indicative of the relative number of alarm events labeled accordingly. Thus, while not shown in FIG. 8, each box 812 and 814 contains a number of events and more alarm events are true than false. The assigned label sequence 810 may be generated according to, for example, the flow 500 of FIG. 5 using the rules of FIGS. 3 and 4.

[0039] FIG. 8 also schematically shows an example of labels predicted for the event sequence using the machine learning process of FIG. 7 as predicted labels sequence 820. The assigned labels 810 may be an example of input data at step 710 of FIG. 7 to generate the model, which is used to generate predict labels sequence 820. In the predicted labels sequence 820, boxes 822 and 824 represent alarm events of the event sequence corresponding to the events indicated in boxes 812 and 814. That is, the alarm events of box 822 would all be false alarm events if labels are predicted with 100% accuracy and the events of box 824 would all be true alarm events if predicted 100% accuracy. However, in the illustrative example of FIG. 8, the model is not 100% accurate. Black box A represents false alarm events that are correctly predicted as false alarms (e.g., True Positives), white box B represents false alarm events that are missed (e.g., False Negatives), black box C represents true alarm events that are incorrectly predicted as false alarm (e.g., False Positives), and white box D represents true alarm events that are correctly predicted as true alarms (e.g., True Negative).

[0040] Returning to FIG. 7, at step 760, the model may be refined by increasing the false alarm reduction rate while reducing mistakes on true alarms. The false alarm reduction rate and mistakes on true alarms can be calculated as follows:

False Alarm Reduction Rate = - Eq. 1

A+B

(

Mistakes on True Alarm = - Eq. 2

C+D [0041] Where A is the number of True Positives, B is the number of False Negatives, C is the number of False Positives, and D is the number of True Negatives, as described above in connection to FIG. 8.

[0042] The result of false alarm reduction process 700 leads to the generation of a model that detects false alarms from an input event sequence.

[0043] In addition to predicting false alarms as set forth above, example implementations are able to classify true alarms as one of multiple severity classes based on an amount of time and/or a number operating cycles until a failure occurs.

[0044] FIG. 9 illustrates an example process 900 for labeling event severity, in accordance with implementations disclosed herein. For example, process 900 illustrates an example labeling logic for labeling true alarm severity levels according to three severity classes or severity levels (e.g., low, medium, or and high). While the examples provided herein are described in connection to three severity class, implementation disclosed herein are not to be limited to three classes. Any number of severity classes may be used based on the use case. For example, five or more severity classes, 10 or more severity classes, and so on.

[0045] At step 910, a true alarm sequence is received as an input. The true alarm sequence may be an event sequence comprising only true alarm events. For example, process 500 of FIG. 5 may be executed on an event sequence to remove false alarm events. The resulting event sequence may be referred to as a true alarm sequence, which can be used as the input at step 910. As another example, the true alarm sequence need not be a result of process 500, and may be a sequence of events known to be true alarms (e.g., via manual or other method of ensuring true alarm events).

[0046] At step 920, for each true alarm instance in the true alarm sequence, process 900 calculates a feature difference (d) between each respective alarm event instance and an intervention instances (R) that occurs subsequent to the respective alarm event instance in the true alarm sequence (e.g., the next repair or replacement event in the event sequence). In some implementations, the feature difference d may be a difference in time between the respective alarm event instance and the intervention instance. For example, process 900 may retrieve timestamps for the respective alarm event instance and the intervention instance, and calculate a difference therebetween. In other implementations, the feature difference d may be a difference in operating cycles between the respective alarm event instance and the intervention instances. With reference to the truck example set forth above, an example operating cycle may be mileage (e.g., as read from the odometer of the truck) associated with each alarm event instance and an intervention instance (e.g., the instance E and/or R may be associated with a mileage m corresponding to the mileage of the truck at the time that the instance E and/or R occurred). In this case, the feature difference d may be determined from the difference in mileage between the intervention instance and the respective alarm event instance. Another example of an operating cycle may be time (e.g., seconds, minutes, hours, days, weeks, months, etc.). For example, the difference d may be how may days are remaining until equipment failure.

[0047] At step 930, the difference d is compared against a first threshold difference value DI. First threshold DI may be a value set in advance based on the use case and data underlying the difference d. For example, in the case where difference t/is a time between the intervention instance and the alarm event instances (e.g., TR - TE = d). DI may be set as a threshold time difference. In the case where difference t/is a difference in operation cycles (e.g., WIR - mE=d), DI may be set as a threshold operating difference. The first threshold difference DI may be set, for example, based on requirements of the equipment. For example, in the truck scenario, DI may be set as 1000 miles. As another example, where the operating cycle is in terms of time, DI may be a number of operating days (e.g., three days). In various implementations, step 930 determines whether the difference d is a positive number. For example, a time difference would be positive in the case where the intervention instances occurred temporally subsequent to the alarm event instance in question. Similarly, an operating difference would be positive in the case that that operating cycles of the intervention instance (for example, mileage in the above example) is subsequent to that of the alarm event instance in question. A negative difference would occur, for example, if the intervention instance occurred prior to the alarm event instance in question.

[0048] If the determination at step 930 is Yes, at step 930 the process 900 labels the respective alarm event instance as high severity. Thus, a high severity label applied to a true alarm event instance may be indicative that the true alarm event instance could lead to a breakdown after a short time or small number of operating cycles. Thus, the alarm is labeled as high severity, such that equipment operators and/or administrators may determine to intervene immediately. [0049] If the determination at step 930 is No, process 900 proceeds to step 950. At step 950, a determination is made as to whether the feature difference d is less than or equal to a second threshold difference value D2. Similar to Z>7, second threshold D2 may be a value set in advance based on the use case and data underlying the difference d and may be set, for example, based on requirements of the equipment. For example, in the truck scenario, D2 may be set as 5000 miles. As another example, where the operating cycle is in terms of time, D2 may be a number of operating days greater than DI (e.g., sever days).

[0050] If the determination at step 950 is Yes, at step 960 the process 900 labels the respective alarm event instance as medium severity. Thus, a medium severity label applied to a true alarm instance may be indicative that the true alarm event instance could lead to a breakdown after a reasonable time or medium number of operating cycles. That is, the alarm has medium severity, such that the equipment operators and/or administrators may determine to intervene within a reasonable time, but need to act immediately. For example, when a medium severity fault code occurs in a truck, the truck can finish the current trip and seek maintenance afterwards.

[0051] If the determination at step 950 is No, process 900 proceeds to step 970, where the respective true alarm event instance is labeled low severity. A low severity label applied to a true alarm instance may be indicative that the true alarm event instance could lead to a breakdown after a long time or large number of operating cycles. For example, when a true alarm has low severity, equipment operators and/or administrators can investigate the alarm during a next scheduled maintenance.

[0052] After all true alarms in the event sequence are labeled with a severity class according to process 900, the labeled data may be supplied as an input to a supervised machine learning technique for severity classification.

[0053] In event severity classification, the goal is to predict a severity class label associated with a given feature vector x for each received true alarm event. Severity classification is a supervised multi-class classification problem. A set of features are extracted from a time window preceding each true alarm event instance of a true alarm event sequence input for classification. The time window may be set in advance as desired based on the use case and equipment that generated the event sequence. Extracted features can be adjusted depending on system requirements and equipment component characteristics. Extracted features are then applied to a set of learners. Example learners may include, but are not limited to, decision trees, random forest, XGBoost, and neural networks. Each learner operates on the training data according to a set of parameters. The machine learning process runs until a given number of cycles have run, a set time period has elapsed, or a desired accuracy level is obtained.

[0054] For example, a process for severity classification may be similar to the flowchart shown in FIG. 7, with a difference in evaluation metrics. That is, the process 700 may be applied to generate a model for severity classification where a labeled input data set may be the result of labeling a true alarm sequence according to FIG. 9. The difference between the severity classification and the false alarm prediction may be the metric used to refine the model. Many true alarms may be classified as low and medium severity. This may result in an imbalanced number of records for three severity classes. Due to class imbalance, along with an overall accuracy metric (e.g., Number of Correctly Predicted Events / Total Number of Events), an accuracy per severity level (c) may be calculated according to equation 3 below and an estimate of the overall cost savings may be determined.

. > „ . Number of Correctly Assigned Events to Class c > „

Accuracy Per Severity Class c = - - - - - - - _: - - Eq. 3

Total Number of Events in Class c

[0055] FIG. 10 illustrates a system involving one or more equipment and a management apparatus configured to manage the one or more equipment, in accordance with an example implementation. One or more equipment or equipment systems 1001-1, 1001-2, 1001-3, and 1001-4 are communicatively coupled to a network 1000 which is connected to a management apparatus 1002. The management apparatus 1002 manages a database 1003, which contains historical data collected from the apparatuses and apparatus systems in the network 1000. In alternate example implementations, the data from the apparatuses and apparatus systems 1001- 1, 1001-2, 1001-3, and 1001-4 can be stored to a central repository or central database such as proprietary databases that data from equipment or equipment systems such as enterprise resource planning systems, and the management apparatus 1002 can access or retrieve the data from the central repository or central database. Such equipment can include stationary equipment or equipment with sensor systems such as coolers, water tanks, air compressors, electrical lines, as well as mobile equipment or equipment such as, but not limited to, motor vehicles (e.g., automobiles, trucks, buses, etc.), commuter vehicles (e.g., buses, trains, airplanes, etc.), construction equipment, moving robots, robotic arms, as well as any other equipment that are integrated into the industrial networks or can be part of an industrial network. Other example equipment may include, but not limited to, equipment and components for establishing a network, such as antennas, routes, modems, communication devices, etc. The sensor data provided by the one or more equipment can involve raw sensor data from the equipment while under operation as well as when the equipment undergoes a fault. For integration into training a machine learning classifier and/or regressor for generating false alarm labels (e.g., as described in connection with FIGS. 3-6); false alarm prediction models (e.g., described in connection with FIGS. 7 and 8); and true alarm severity classification (e.g., as described in connection with FIG. 9) in accordance with the example implementations described herein, the sensor data may also be encoded to indicate if the sensor data incorporated from the equipment is when the equipment is operating in a normal condition or undergoing some sort of fault condition (e.g., an alarm event instance).

[0056] FIG. 11 illustrates an example computing environment with an example computer device suitable for use in some example implementations, such as a management apparatus 1002 of FIG. 10. The computing environment may be implemented to perform the functions described in connection to any one or more of FIGS. 1-9. For example, the computing environment may be implemented to label events according to FIGS. 3-6; generate a false events prediction model according to FIGS. 7 and 8; and/or generate an event severity classification model according to FIG. 9. In computer device 1105 in computing environment 1100 can include one or more processing units, cores, or processors 1110, memory 1115 (e.g., RAM, ROM, and/or the like), internal storage 1120 (e.g., magnetic, optical, solid-state storage, and/or organic), and/or IO interface 1125, any of which can be coupled on a communication mechanism or bus 1130 for communicating information or embedded in the computer device 1105. IO interface 1125 is also configured to receive images from cameras or provide images to projectors or displays, depending on the desired implementation. In some implementations, the I/O interface 1025 may be an example of a means for receiving an event sequence of events output from sensors monitoring an equipment.

[0057] Computer device 1105 can be communicatively coupled to input/user interface 1135 and output device/interface 1140. Either one or both of the input/user interface 1135 and output device/interface 1140 can be a wired or wireless interface and can be detachable. Input/user interface 1135 may include any device, component, sensor, or interface, physical or virtual, that can be used to provide input (e.g., buttons, touch-screen interface, keyboard, a pointing/cursor control, microphone, camera, braille, motion sensor, optical reader, and/or the like). Output device/interface 1140 may include a display, television, monitor, printer, speaker, braille, or the like. In some example implementations, input/user interface 1135 and output device/interface 1140 can be embedded with or physically coupled to the computer device 1105. In other example implementations, other computer devices may function as or provide the functions of input/user interface 1135 and output device/interface 1140 for a computer device 1105.

[0058] Examples of computer device 1105 may include, but are not limited to, highly mobile devices (e.g., smartphones, devices in vehicles and other machines, devices carried by humans and animals, and the like), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, and the like), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with one or more processors embedded therein and/or coupled thereto, radios, and the like).

[0059] Computer device 1105 can be communicatively coupled (e.g., via IO interface 1125) to external storage 1145 and network 1150 for communicating with any number of networked components, devices, and systems, including one or more computer devices of the same or different configuration. Computer device 1105 or any connected computer device can be functioning as, providing services of, or referred to as a server, client, thin server, general machine, special-purpose machine, or another label.

[0060] IO interface 1125 can include but is not limited to, wired and/or wireless interfaces using any communication or IO protocols or standards (e.g., Ethernet, 802.1 lx, Universal System Bus, WiMax, modem, a cellular network protocol, and the like) for communicating information to and/or from at least all the connected components, devices, and network in computing environment 1100. Network 1150 can be any network or combination of networks (e.g., the Internet, local area network, wide area network, a telephonic network, a cellular network, satellite network, and the like).

[0061] Computer device 1105 can use and/or communicate using computer-usable or computer readable media, including transitory media and non-transitory media. Transitory media include transmission media (e.g., metal cables, fiber optics), signals, carrier waves, and the like. Non-transitory media include magnetic media (e.g., disks and tapes), optical media (e.g., CD ROM, digital video disks, Blu-ray disks), solid-state media (e.g., RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory. [0062] Computer device 1105 can be used to implement techniques, methods, applications, processes, or computer-executable instructions in some example computing environments. Computer-executable instructions can be retrieved from transitory media, and stored on and retrieved from non-transitory media. The executable instructions can originate from one or more of any programming, scripting, and machine languages (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, and others).

[0063] Memory 1115 can be configured to store one or more programs, such as Operating System (OS), Hypervisor, and applications. Memory 1115 may be configured to store instructions for executing alarm event labeling, for example, as described in connection with FIGS. 3-6; false alarm prediction as described in connection with FIGS. 7 and 8; and/or event severity labeling as described in connection with FIG. 9. One or more of internal storage 1120 and external storage (if applicable) may be configured to store the raw sensor measurements, historical data, alarm events and associated data, the set of conditions (e.g., as described in FIGS. 3 and 4), and other data necessary for executing the processes described herein.

[0064] Processor(s) 1110 can execute under any operating system (OS) (not shown), in a native or virtual environment. One or more applications can be deployed that include logic unit 1160, application programming interface (API) unit 1165, input unit 1170, output unit 1175, and inter-unit communication mechanism 1195 for the different units to communicate with each other, with the OS, and with other applications (not shown). The described units and elements can be varied in design, function, configuration, or implementation and are not limited to the descriptions provided. Processor(s) 1110 can be in the form of hardware processors such as central processing units (CPUs) or in a combination of hardware and software units.

[0065] In some example implementations, when information or an execution instruction is received by API unit 1165, it may be communicated to one or more other units (e.g., logic unit 1160, input unit 1170, output unit 1175). In some instances, logic unit 1160 may be configured to control the information flow among the units and direct the services provided by API unit 1165, the input unit 1170, the output unit 1175, in some example implementations described above. For example, the flow of one or more processes or implementations may be controlled by logic unit 1160 alone or in conjunction with API unit 1165. The input unit 1170 may be configured to obtain input for the calculations described in the example implementations, and the output unit 1175 may be configured to provide an output based on the calculations described in example implementations. [0066] Processor(s) 1110 can be in the form of physical hardware processors (e.g., Central Processing Units (CPUs), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC)) or a combination of software and hardware processors.

[0067] Processor(s) 1110 can be configured to fetch and execute programs stored in memory 1115. When processor(s) 1110 execute programs, processor(s) 1110 fetch instructions of the programs from memory 1115 and execute them, such as programs for performing process as illustrated in FIGS. 1, 5, 7, and 9. When processor(s) 1110 execute programs, processor can load information such as illustrated in FIGS. 2, 3, 4, 6 and 8 from memory. Processor(s) 1110 can pre-fetch and cache instruction of programs and information to improve performance.

[0068] In example implementations, processor(s) 1110 can be configured to receive an event sequence comprising a plurality of events from the equipment, and generate a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence, for example, as illustrated in FIGS. 3-6. The set of conditions may include one or more of identifying an intervention event in the plurality of events, identifying an escalation event in the plurality of events, and identifying a repetition event in the plurality of events. In various example implementations, the processor(s) 1100 (or the components therein) may be an example of means for receiving an event sequence, in accordance with the example implementations disclosed herein.

[0069] In some implementations, processor(s) 1110 may be configured to receive an event sequence comprising a plurality of events from the equipment, identify true alarm events in the event sequence, and automatically generating a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event, for example, as illustrated in FIG. 9. In various example implementations, the processor(s) 1100 (or the components therein) may be an example of means for receiving an event sequence, in accordance with the example implementations disclosed herein.

[0070] In some implementations, processor(s) 1110 may be configured to receive an event sequence comprising a plurality of events from an equipment and generate a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence, for example, as illustrated in FIGS. 3-6. The set of conditions may include one or more of identifying an intervention event in the plurality of events, identifying an escalation event in the plurality of events, and identifying a repetition event in the plurality of events. The processor(s) 110 may also be configured to automatically generate a severity label for each event in an event sequence for which a true alarm label is generated, based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, where the severity label is indicative of a level of severity of each event for which a true alarm label is generated, as described in connection to FIG. 9.

[0071] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the essence of their innovations to others skilled in the art. An algorithm is a series of defined steps leading to a desired end state or result. In example implementations, the steps carried out require physical manipulations of tangible quantities for achieving a tangible result.

[0072] Unless specifically stated otherwise, as apparent from the discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” or the like, can include the actions and processes of a computer system or other information processing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system’s registers and memories into other data similarly represented as physical quantities within the computer system’s memories or registers or other information storage, transmission or display devices.

[0073] Example implementations may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored in a computer readable medium, such as a computer readable storage medium or a computer readable signal medium. A computer readable storage medium may involve tangible mediums such as, but not limited to, optical disks, magnetic disks, read-only memories, random access memories, solid- state devices, and drives, or any other types of tangible or non-transitory media suitable for storing electronic information. A computer readable signal medium may include mediums such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Computer programs can involve pure software implementations that involve instructions that perform the operations of the desired implementation.

[0074] Various general-purpose systems may be used with programs and modules in accordance with the examples herein, or it may prove convenient to construct a more specialized apparatus to perform desired method steps. In addition, the example implementations are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the example implementations as described herein. The instructions of the programming language(s) may be executed by one or more processing devices, e.g., central processing units (CPUs), processors, or controllers.

[0075] As is known in the art, the operations described above can be performed by hardware, software, or some combination of software and hardware. Various aspects of the example implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software), which if executed by a processor, would cause the processor to perform a method to carry out implementations of the present application. Further, some example implementations of the present application may be performed solely in hardware, whereas other example implementations may be performed solely in software. Moreover, the various functions described can be performed in a single unit, or can be spread across a number of components in any number of ways. When performed by software, the methods may be executed by a processor, such as a general-purpose computer, based on instructions stored on a computer readable medium. If desired, the instructions can be stored on the medium in a compressed and/or encrypted format.

[0076] Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the teachings of the present application. Various aspects and/or components of the described example implementations may be used singly or in any combination. It is intended that the specification and example implementations be considered as examples only, with the true scope and spirit of the present application being indicated by the following claims.

Claims

CLAIMS What is claimed is:

1. A method for predicting false alarm events from equipment, the method comprising: receiving an event sequence comprising a plurality of events from the equipment; and generating a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence, the set of conditions comprising one or more of: identifying an intervention event in the plurality of events; identifying an escalation event in the plurality of events; and identifying a repetition event in the plurality of events.

2. The method of claim 1, wherein the set of conditions are based on, from a viewpoint of a given event in the plurality of events, analyzing subsequent events in the event sequence to identify the intervention event.

3. The method of claim 1, wherein the set of conditions are based on, from a view point of a given event of the plurality of events, analyzing subsequent events in the event sequence to identify the escalation event.

4. The method of claim 1, wherein the set of conditions are based on, from a view point of a given event of the plurality of events, analyzing subsequent events in the event sequence to identify the repetition event.

5. The method of claim 1, wherein the set of conditions are based on combinations of two or more of: the identifying the intervention event in the plurality of events; the identifying the escalation event in the plurality of events; and the identifying the repetition event in the plurality of events.

- 29 -

6. The method of claim 1, wherein the set of conditions are based on identifying the intervention event in the plurality of events and at least one of identifying the escalation event in the plurality of events or identifying the repetition event in the plurality of events.

7. The method of claim 1, further comprising predicting the false alarm label or the true alarm label for events in the event sequence based on one or more of equipment information, event data, sensor data, and equipment repair history.

8. The method of claim 1, further comprising labeling events in the event sequence based on a taxonomy that classifies events in the event sequence according to the true alarm label or the false alarm label generated for each event.

9. The method of claim 8, wherein the taxonomy comprises a plurality of scenarios under which ones of the plurality of events occur, the plurality of scenarios corresponding to a plurality of outputs, wherein the taxonomy classifies each event according to an output associated with a scenario for under which each respective event occurs.

10. The method of claim 1, wherein the equipment is a motor vehicle.

11. The method of claim 10 , wherein the motor vehicle is one of an automobile, a truck, and a bus.

12. The method of claim 1, further comprising automatically generating a severity label for each event in an event sequence for which a true alarm label is generated, based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, wherein the severity label is indicative of a level of severity of each event for which a true alarm label is generated.

13. The method of claim 1, further comprising filtering the event sequence according to the generated true alarm labels and false alarm labels to remove events for which a false alarm label is generated, wherein resources for reacting to events in the event sequence are allocated according to the filtered event sequence.

- 30 -

14. A method for classifying severity of events from equipment, the method comprising: receiving an event sequence comprising a plurality of events from the equipment; identifying true alarm events in the event sequence; and automatically generating a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event.

15. The method of claim 14, wherein each severity label is one of a plurality of severity levels indicative of increasing levels of severity of respective true alarm events, each severity level corresponding to a respective threshold value, wherein the severity labels generated for each true alarm event is determined to be one of the plurality of severity levels based on the true alarm event satisfying a respective threshold value.

16. The method of claim 15, further comprising predicting the severity labels based on one or more of equipment information, event data, sensor data, and equipment repair history.

17. The method of claim 15, wherein the severity labels are generated based on a taxonomy that classifies events in the event sequence as the true alarm events or false alarm events and determines the severity labels for the true alarm events.

18. The method of claim 17, wherein the taxonomy comprises a plurality of scenarios, under which ones of the plurality of events occur, the scenarios correspond to a plurality of outputs, wherein the taxonomy determines the severity labels for the true alarm events according to an output associated with a scenario under which each respective true alarm event occurs.

19. The method of claim 14, wherein the equipment is a motor vehicle.

20. The method of claim 19 , wherein the motor vehicle is one of an automobile, a truck, and a bus.

21. The method of claim 14, wherein interventions for each true alarm event in the event sequence are prioritized according to the generated severity labels.

22. A system for predicting false alarm events from equipment, the system comprising: one or more memories configured to store instructions; and one or more processors coupled to the one or more memories and configured to execute the instructions to: receive an event sequence comprising a plurality of events from an equipment; and generate a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence, the set of conditions comprising one or more of: identifying an intervention event in the plurality of events; identifying an escalation event in the plurality of events; and identifying a repetition event in the plurality of events.

23. A system for classifying severity of events from equipment, the system comprising: one or more memories configured to store instructions; and one or more processors coupled to the one or more memories and configured to execute the instructions to: receive an event sequence comprising a plurality of events from an equipment; generate a true alarm events in the event sequence; and automatically generate a severity label for each true alarm event identified in the event sequence based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, the severity label indicative of a level of severity of each true alarm event.

24. A system for predicting false alarm events from equipment, the system comprising: one or more memories configured to store instructions; and one or more processors coupled to the one or more memories and configured to execute the instructions to: receive an event sequence comprising a plurality of events from an equipment; generate a true alarm label or a false alarm label for each event in the event sequence according to a set of conditions and based on sequential relationships and temporal relationships between two or more events in the event sequence, the set of conditions comprising one or more of identifying an intervention event in the plurality of events; identifying an escalation event in the plurality of events; and identifying a repetition event in the plurality of events; and automatically generate a severity label for each event in an event sequence for which a true alarm label is generated, based on one or more of (i) time until failure or repair of the equipment and (ii) operating cycles until failure or repair of the equipment, wherein the severity label is indicative of a level of severity of each event for which a true alarm label is generated.

- 33 -