US20190253440A1 - Defense system and method against cyber-physical attacks - Google Patents
Defense system and method against cyber-physical attacks Download PDFInfo
- Publication number
- US20190253440A1 US20190253440A1 US16/331,409 US201716331409A US2019253440A1 US 20190253440 A1 US20190253440 A1 US 20190253440A1 US 201716331409 A US201716331409 A US 201716331409A US 2019253440 A1 US2019253440 A1 US 2019253440A1
- Authority
- US
- United States
- Prior art keywords
- physical
- devices
- verification devices
- sensors
- utilities
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 376
- 238000004891 communication Methods 0.000 claims description 62
- 230000001276 controlling effect Effects 0.000 claims description 24
- 230000000875 corresponding Effects 0.000 claims description 16
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 96
- 238000005259 measurement Methods 0.000 description 66
- 241000196324 Embryophyta Species 0.000 description 64
- 238000001514 detection method Methods 0.000 description 40
- 230000001010 compromised Effects 0.000 description 30
- 101700039183 PLCD1 Proteins 0.000 description 24
- 101700074329 PLCG1 Proteins 0.000 description 24
- 102100011802 PLCG1 Human genes 0.000 description 24
- 101710029124 RPS3A Proteins 0.000 description 24
- 101700008860 plc1 Proteins 0.000 description 24
- 101710023955 rps101 Proteins 0.000 description 24
- 101710009096 PLC6 Proteins 0.000 description 10
- 238000002347 injection Methods 0.000 description 10
- 239000007924 injection Substances 0.000 description 10
- 230000003993 interaction Effects 0.000 description 10
- 230000004048 modification Effects 0.000 description 6
- 238000006011 modification reaction Methods 0.000 description 6
- 101700010313 PLC2 Proteins 0.000 description 4
- 101710009097 PLC8 Proteins 0.000 description 4
- 101700012550 PLCD4 Proteins 0.000 description 4
- 230000000295 complement Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 238000010248 power generation Methods 0.000 description 4
- 238000004540 process dynamic Methods 0.000 description 4
- 101710023954 rps102 Proteins 0.000 description 4
- 239000000243 solution Substances 0.000 description 4
- 238000000108 ultra-filtration Methods 0.000 description 4
- 241000156978 Erebia Species 0.000 description 2
- 101710009100 PLC5 Proteins 0.000 description 2
- 101700035749 PLCD3 Proteins 0.000 description 2
- 101700047933 PLCE1 Proteins 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000002547 anomalous Effects 0.000 description 2
- 230000004397 blinking Effects 0.000 description 2
- 238000007374 clinical diagnostic method Methods 0.000 description 2
- 230000002708 enhancing Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000000977 initiatory Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000000116 mitigating Effects 0.000 description 2
- 230000003647 oxidation Effects 0.000 description 2
- 238000007254 oxidation reaction Methods 0.000 description 2
- 238000004886 process control Methods 0.000 description 2
- 230000001105 regulatory Effects 0.000 description 2
- 238000001223 reverse osmosis Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000004642 transportation engineering Methods 0.000 description 2
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
- G05B19/0428—Safety, monitoring
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/05—Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts
- G05B19/058—Safety, monitoring
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM]
- G05B19/4184—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM] characterised by fault tolerance, reliability of production system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/554—Detecting local intrusion or implementing counter-measures involving event detection and direct action
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/10—Plc systems
- G05B2219/14—Plc safety
- G05B2219/14006—Safety, monitoring in general
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/45—Nc applications
- G05B2219/45103—Security, surveillance applications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
Description
- The present disclosure claims the benefit of Singapore Patent Application No. 10201607439T filed on 7 Sep. 2016, which is incorporated in its entirety by reference herein.
- The present disclosure generally relates to a defense system and method against cyber-physical attacks. More particularly, the present disclosure describes various embodiments of a defense system and method for defending a utilities system against cyber-physical attacks.
- Utilities systems or public utilities, such as power generation plants, water treatment plants, transportation infrastructure, supply chains, electrical power grids, medical facilities, and smart buildings, are often at risk of cyber-physical attacks or malicious cyber-physical activity. A utilities system comprises a physical subsystem that is controlled by a cyber subsystem. A cyber-physical attack on the utilities system refers to an attempt to disrupt the operation of the physical system through the manipulation of cyber and/or physical components of the utilities system. The utilities system may also be referred to as a Cyber Physical System (CPS) or Industrial Control System (ICS). The physical subsystem comprises components such as generators, transformers, pumps, and tanks, for operating physical processes in the utilities system. The cyber subsystem comprises networked embedded computation and communication devices, e.g. programmable logic controllers (PLCs) and a Supervisory Control and Data Acquisition (SCADA) system/workstation, as well as sensors and actuators for monitoring, measure, and control of the physical processes.
- The cyber subsystem of a utilities system may comprise one or more control stages, each control stage for controlling one physical process. The control actions by the PLCs are based on the current process state obtained through the sensors, and the control actions subsequently alter the process state. For example, in a water treatment plant, the PLCs may communicate instructions to an actuator, e.g. a pump, to fill a tank with water. The pump is instructed to stop when the tank reaches a predetermined level. Physical data for the water level in the tank is communicated to the PLCs through a level sensor. Thus, the PLCs receive physical data from the sensors, compute control actions, and apply these actions to the actuators.
- The communications infrastructure of a utilities system, often using wired and/or wireless communications, may be connected to an external network. However, such connections render the utilities system susceptible to cyber attacks. Such attacks may compromise the communication links between sensors, actuators, and the PLCs, as well as across the PLCs and SCADA system. Each such link is considered as an attack point in the utilities system. Once a link has been compromised, an attacker can send fake or false state (sensor) data to the PLCs, or bypass the PLCs and directly control the actuators. Unless the defense system of the utilities system is robust, such attacks are able to cause an undesirable response that may lead to system shutdown and/or component damage. Furthermore, it is assumed that an attacker has access to any one or more control stages of the utilities system. Thus, the attacker is able to compromise components in one or more control stages to which access is available. In many complex utilities systems, the PLCs are often distributed. Although attacking more than one control stage simultaneously might be difficult, it is known to be feasible.
- Many existing cyber-physical attack detection solutions for utilities systems are extensions of the traditional network-centric defense system or infrastructure such as firewalls and other network-based logic to prevent intrusions into a utilities system. Such solutions are aimed at preventing unauthorized access into the utilities system. However, social engineering and software vulnerabilities may still enable an attacker to obtain access to the utilities system, potentially leading to component damage and significant deviation from desired behaviour of the physical processes operative in the utilities system.
- Assuming that an attacker has bypassed the traditional network-centric defense system, a variety of attacks can be launched in such a situation. While some attacks may focus only on a few components of the utilities system, others may compromise the entire utilities system. For example, bypassing the traditional network-centric defense system exposes the PLCs to the attacker, potentially compromising the logics of the PLCs. This may result in the PLCs computing false/fake sensor data that affect performance of the physical processes. For example, the PLCs may be led by the attacker to believe that a physical process is operating normally, when in fact the physical process is behaving anomalously. The actuators may also be directly controlled by the attacker to behave anomalously and affect performance of the physical process.
- Flaws in the design of the cyber subsystem thus expose the utilities system to cyber-physical attacks that may damage its components and/or affect performance of the physical processes. Utilities systems generally operate in real-time, and must be safe, secure, and efficient. As attackers become more skilled in planning and executing cyber-physical attacks, utilities systems continue to be vulnerable to cyber-physical attacks, and such risks and vulnerabilities cannot be undermined or neglected. One of the most dangerous cyber attacks on utilities systems or critical infrastructures is the Stuxnet worm attack. Such cyber attacks may cause devastating consequences and are difficult to detect.
- In various utilities systems, there are many possible types of cyber-physical attacks that lead to undesirable or anomalous behaviour and performance of physical processes operative in the utilities systems. One type of cyber attacks is a man-in-the-middle attack to maliciously manipulate sensor data and/or status of the actuators. For example in a water treatment plant, the intention of such an attack may be to cause a tank to overflow or a reduction in the performance measured as gallons of water produced per minute. Other types of cyber attacks or attack vectors include, but are not limited to, ma are injection into the PLCs, Denial of Service (DOS) or Distributed DOS (DDOS).
- Therefore, in order to address or alleviate at least one of the aforementioned problems and/or disadvantages, there is a need to provide a defense system and method against cyber-physical attacks, in which there is at least one improvement and/or advantage over the prior art.
- According to a first aspect of the present disclosure, there is a defense system for defending a utilities system against cyber-physical attacks associated with anomalies in a physical process operative in the utilities system. The defense system comprises: a set of sensors for collecting physical data associated with the physical process; a set of controller devices for monitoring process states of the physical process based on the physical data from the sensors; a set of verification devices for monitoring the physical process based on the physical data from the sensors, the physical data enabling the verification devices to detect the anomalies based on a set of invariants predefined for the physical process; and a set of actuators controllable by the controller devices or verification devices to remedy the anomalies and regulate the physical process, thereby defending the utilities system against the cyber-physical attacks.
- According to a second aspect of the present disclosure, there is a defense method for defending a utilities system against cyber-physical attacks associated with anomalies in a physical process operative in the utilities system. The defense method comprises: collecting, by a set of sensors, physical data associated with the physical process; monitoring, by a set of controller devices, process states of the physical process based on the physical data from the sensors; monitoring, by a set of verification devices, the physical process based on the physical data from the sensors, the physical data enabling the verification devices to detect the anomalies based on a set of invariants predefined for the physical process; detecting, by the verification devices, the anomalies in the physical process; and controlling, by the controller devices or verification devices, a set of actuators to remedy the anomalies and regulate the physical process, thereby defending the utilities system against the cyber-physical attacks.
- An advantage of the present disclosure is that the defense system can be used to defend the utilities system from cyber-physical attacks such as in situations where the communications network and/or the control logic inside the controller devices are compromised. The defense system uses an orthogonal defense that is separate/independent from the traditional defense system, e.g. network firewalls, of the utilities system. The defense system uses verification devices that are able to detect cyber attacks based on the use of invariants derived from process dynamics and interactions among components across the physical processes operative in the utilities system. The design of the
defense system 300 is not influenced by any specific attacker model or attack model, and is thus able to detect unique and undocumented attacks, also known as zero day attacks, when they cause anomalies that involve deviation of dynamics of the physical processes from the expected behaviour. - A defense system and method against cyber-physical attacks according to the present disclosure are thus disclosed herein. Various features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description of the embodiments of the present disclosure, by way of non-limiting examples only, along with the accompanying drawings.
-
FIG. 1 is an illustration of an architecture schematic of acyber subsystem 100 of a utilities system, in accordance with various embodiments of the present disclosure. -
FIG. 2 is an illustration of a control workflow of a physical process operative in the utilities system, in accordance with various embodiments of the present disclosure. -
FIG. 3 is an illustration of an architecture schematic of adefense system 300 for defending a utilities system against cyber-physical attacks, in accordance with various embodiments of the present disclosure. -
FIG. 4 is another illustration of a control workflow of a physical process operative in a utilities system, in accordance with various embodiments of the present disclosure. -
FIG. 5 is a flowchart illustration of a defense method for defending a utilities system against cyber-physical attacks, in accordance with various embodiments of the present disclosure. -
FIG. 6 is an illustration of an attack workflow in a utilities system, in accordance with various embodiments of the present disclosure. -
FIG. 7 is an illustration of a block diagram of a utilities system (water treatment plant), in accordance with various embodiments of the present disclosure. -
FIG. 8 is an illustration of a table of sample model equations used to derive invariants, in accordance with various embodiments of the present disclosure. -
FIG. 9 is an illustration of a table of sample attack regions in a utilities system (water treatment plant), in accordance with various embodiments of the present disclosure. - In the present disclosure, depiction of a given element or consideration or use of a particular element number in a particular figure or a reference thereto in corresponding descriptive material can encompass the same, an equivalent, or an analogous element or element number identified in another figure or descriptive material associated therewith. The use of “/” in a figure or associated text is understood to mean “and/or” unless otherwise indicated. As used herein, the term “set” corresponds to or is defined as a non-empty finite organization of elements that mathematically exhibits a cardinality of at least one (e.g. a set as defined herein can correspond to a unit, singlet, or single element set, or a multiple element set), in accordance with known mathematical definitions. The recitation of a particular numerical value or value range herein is understood to include or be a recitation of an approximate numerical value or value range.
- For purposes of brevity and clarity, descriptions of embodiments of the present disclosure are directed to a defense system and method against cyber-physical attacks, in accordance with the drawings. While aspects of the present disclosure will be described in conjunction with the embodiments provided herein, it will be understood that they are not intended to limit the present disclosure to these embodiments. On the contrary, the present disclosure is intended to cover alternatives, modifications and equivalents to the embodiments described herein, which are included within the scope of the present disclosure as defined by the appended claims. Furthermore, in the following detailed description, specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be recognized by an individual having ordinary skill in the art, i.e. a skilled person, that the present disclosure may be practiced without specific details, and/or with multiple details arising from combinations of aspects of particular embodiments. In a number of instances, known systems, methods, procedures, and components have not been described in detail so as to not unnecessarily obscure aspects of the embodiments of the present disclosure.
- Utilities System
- Representative or exemplary embodiments of the present disclosure describe a defense system and method for defending a utilities system against cyber attacks and/or physical attacks (collectively referred to as cyber-physical attacks).
FIG. 1 illustrates an architecture schematic of a control orcyber subsystem 100 of the utilities system for controlling a physical subsystem (not shown) of the utilities system. Particularly, thecyber subsystem 100 controlsphysical processes 110 operative or performed in the utilities system. Thephysical processes 110 are affected by cyber-physical attacks in the utilities system, causing anomalies or abnormalities in thephysical processes 110. The cyber-physical attacks are thus associated with anomalies/abnormalities in thephysical processes 110. Eachphysical process 110 may be considered as a process stage of a collectivephysical process 115. Each physical process orprocess stage 110 is controlled by acorresponding control stage 120 of thecyber subsystem 100. - The
cyber subsystem 100 comprises various components/devices for monitoring, including measuring and controlling, thephysical processes 110. Thecyber subsystem 100 comprises one or more sets ofcontroller devices 130 for monitoring process states of thephysical processes 110 based on analysis of physical data associated with thephysical processes 110. Eachcontroller device 130 may comprises a programmable logic controller (PLC). It will be appreciated that the PLCs are generally programmable in a variety of suitable programming languages such as ladder logic, structured text, and functional blocks. Eachcontroller device 130 may additionally or alternatively comprise a remote terminal unit (RTU). - The
cyber subsystem 100 comprises one or more sets ofsensors 140 for collecting the physical data and communicating it to thecontroller devices 130. Thecyber subsystem 100 comprises one or more sets ofactuators 150 controllable by thecontroller devices 130 to execute actions computed/determined by thecontroller devices 130. Thecyber subsystem 100 comprises a set ofsupervisory devices 160 including but not limited to SCADA workstation, Human Machine Interface (HMI), engineering workstation, Historian, and OPC (Object linking and embedding for Process Control) server. The SCADA workstation and HMI are communicatively connected to all thecontroller devices 130 across allcontrol stages 120 for overall monitoring and control of all physical processes/process stages 110 of the collectivephysical process 115. The SCADA workstation and HMI may also be referred to as the control or command centre of thecyber subsystem 100. - In one embodiment, the collective
physical process 115 is a water treatment process segmented into six physical processes/process stages 110—raw water input, chemical dosing, ultrafiltration, dichlorination, reverse osmosis, and backwash. Eachphysical process 110 is controlled by acorresponding control stage 120, and eachcontrol stage 120 comprises its own sets ofcontroller devices 130,sensors 140, andactuators 150. Eachphysical process 110 is thus associated with the corresponding sets ofcontroller devices 130,sensors 140, andactuators 150. For example, in aphysical process 110 for raw water input, various physical components are used, such as a pump, water tank, and valve. Thecontroller devices 130 rely on thesensors 140 to obtain physical data for process state estimation and control theactuators 150, e.g. the pump or valve, to control thephysical process 110, i.e. input flow rate of the water. The placement of thesensors 140 in the utilities system would depend on the functional and cyber security requirements of the collectivephysical process 115 operative therein. - As illustrated in
FIG. 1 , thecyber subsystem 100 is structured hierarchically into several levels/layers of communication networks. Communications within and across the various levels occur via known wired/wireless communication protocols as well as other computational or network devices such as network switches and communications interfaces, as will be readily known to the skilled person. The lowest level (Physical Level) comprises the physical subsystem wherein thephysical processes 110 are performed. The next level (Level 0 or Control Level) comprises a communication network that allows thecontroller devices 130 to communicate with thesensors 140 andactuators 150 to control thephysical processes 110 and implement anomaly detection and process regulatory mechanisms. The next level (Level 1 or Control Level) comprises a communication network that allows thecontroller devices 130 to communicate with one another, enabling access to state information such as water level in a tank, or pH value of water at a particularphysical process 110. The next level (Level 2 or Supervisory Level) comprises diagnostics tools combined with optimal control based tools to provide precise assessment of the situation and ensure timely response to cyber-physical attacks and/or component failures. Communications between thecontroller devices 130 and thesupervisory devices 160, e.g. SCADA workstation, occur via network switch SW1. Thesupervisory devices 160 collect information from allcontroller devices 130 across allcontrol stages 120 and for allphysical processes 110 operative in the utilities system. The highest level (Level 3 or Management Level) comprises a corporate network where the decision makers focus on strategies to maximize profits and minimize losses of the utilities system due to security and reliability risks. Communications between thesupervisory devices 160, e.g. SCADA workstation, and the corporate network occur via network switch SW2. - The control of a
physical process 110 operative in the utilities system can be described as aworkflow 200 with reference toFIG. 2 . For purpose of brevity, theworkflow 200 is described based on onephysical process 110. It will be appreciated that theworkflow 200 is similarly applicable to otherphysical processes 110 of the collectivephysical process 115 operative in the utilities system, as distinct workflows or as part of an integrated workflow. - The
workflow 200 comprises four main steps—monitoringstep 210, networkingstep 220, computingstep 230, andactuation step 240. In themonitoring step 210, thephysical process 110 and environment are monitored by the use of thesensors 140. Particularly, measurements (indicated as y) of physical data of thephysical process 110 collected by thesensors 140. In thenetworking step 220, the physical data collected by sensors is communicated to thecontroller devices 130. In thecomputing step 230, the physical data is analyzed and further actions are computed/determined by thecontroller devices 130. In theactuation step 240, theactuators 150 are controlled by thecontroller devices 130 to execute the computed/determined actions, thereby controlling thephysical process 110. Thesteps physical process 110,controller devices 130,sensors 140, andactuators 150. - Operators and/or engineers of the utilities system use a supervisory device (HMI 162) to monitor the physical subsystem under control of the
cyber subsystem 100. Particularly, the sensor measurements y are communicated from thecontroller devices 130 to the control centre via theHMI 162 in astep 250 of theworkflow 200. In addition, the control centre may be used to communicate the user input and/or feedback (indicated as u) provided by the operators/engineers to thecontroller devices 130. The user input/feedback may be instructions to configure set points, control algorithms, adjust parameters in thecontroller devices 130, and/or display current process status and historical information. - Orthogonal Defense
- Various security approaches and/or defense systems/mechanisms can be implemented at individual or across several levels and/or control
stages 120 for secure communication inside thecyber subsystem 100. In various embodiments of the present disclosure, there is adefense system 300 for acontrol stage 120 of thecyber subsystem 100, as schematically illustrated inFIG. 3 . For purpose of brevity, thedefense system 300 is described based on onecontrol stage 120 for controlling onephysical process 110. It will be appreciated that thedefense system 300 is similarly applicable to otherphysical processes 110, as distinct defense systems or as part of an integrated defense system. - The
defense system 300 comprises a traditional network-centric and layered defense infrastructure or system including network firewalls and other anomaly detection devices. The traditional defense system is indicated as D0, D1, D2, and D3, corresponding to therespective levels 0 to 3 (L0 to L3) of thecyber subsystem 100. Thedefense system 300 further comprises an Orthogonal Defense (OD) system that is separate/independent from, as well as is auxiliary to and complements, the traditional defense system. The OD system is also named or known as “Argus”. - The
defense system 300 enables detection of cyber-physical attacks on thecyber subsystem 100. The detection of cyber-physical attacks is based on redundancy, namely physical and analytical redundancies. With further reference toFIG. 4 , the OD system comprises various types of hardware devices known as Intelligent Checkers (ICs) 170. EachIC 170 is a monolithic device comprising a decision logic component ormicroprocessor 172, one ormore IC sensors 174, a signal conditioner, and an analog-to-digital converter. Thedecision logic component 172 provides analytical redundancy and theIC sensor 174 provides physical redundancy in defending thecyber subsystem 100. In the case of physical redundancy, theIC sensor 174 measures parameters of thephysical process 110, duplicating or emulating the functions of thesensors 140. Analytical redundancy makes use of a mathematical model implemented in an anomaly detection module of thedecision logic component 172. The mathematical model may include one or more estimators/estimation algorithms, such as Kalman filtering or linear quadratic estimation. An estimator performs a comparison of actual measurements of physical data of thephysical process 110 with predefined constraints or conditions. -
FIG. 4 illustrates aworkflow 400 for control of aphysical process 110. For purpose of brevity, theworkflow 400 is described based on onephysical process 110 with oneIC 170. It will be appreciated that theworkflow 400 is similarly applicable to otherphysical processes 110 and withmultiple ICs 170. It will also be appreciated that various steps of theworkflow 200 are similar/analogous to various steps of theworkflow 400. - The
decision logic component 172 of theIC 170 is communicatively connected to one or morealert devices 164, e.g. an audible/visible alarm. Thedecision logic component 172 monitors the status of thephysical process 110 via theIC sensor 174. In astep 410, measurements (indicated as v) of physical data of thephysical process 110 are collected by theIC sensor 174. In astep 420, the measurements v are then periodically communicated to the control centre via theHMI 162. Operators also receive the measurements y (collected by the sensors 140) fromcontroller devices 130, and can compare their values. In astep 430, thedecision logic component 172 activates thealert devices 164 when the measurements v violate or deviate from the predefined constraints, and/or when the measurements v and y deviate from each other. - The predefined constraints are associated with critical properties of the
physical process 110 known as invariants. Particularly, an invariant is defined based on one or more predefined constraints of thephysical process 110 operative in the utilities system. Furthermore, the invariant may be defined based on one or more predefined constraints of otherphysical processes 110. The predefined constraints are conditions for one or more process variables whose values are sampled from thesensors 140. The predefined constraints are derived using fundamental laws of physics and/or chemistry that govern the interactions between various components of a utility of thephysical process 110. The process variables depend on the nature of thephysical process 110 and possibly of one or more otherphysical processes 110. The process variables may include parameters such as temperature, light, pressure, and humidity. A predefined constraint may also specify the maximum and minimum temperatures in a component of the utilities system, such as a boiler. In one example, in a water treatment plant and at a given time instant, a suitable set of invariants constitutes the observable state of the water treatment plant. The invariants may include the correlation between the water level in a tank and the incoming and outgoing flow rates across the tank. - The invariants are programmed and the resulting program code is embedded in the
ICs 170. The coded invariants are checked at all times to ensure that the underlyingphysical processes 110 are behaving as desired. An invariant must be true when evaluated in a given state of the utilities system. When evaluated as false, the invariant leads to an alert indicating that aphysical process 110 has diverged from its intended behaviour, i.e. there are one or more anomalies in the behaviour of thephysical process 110. An alert may lead to a sequence of control actions, e.g. sounding an alarm and/or controlling theactuators 150, aimed at avoiding significant damage to the utilities system and/or reduction in performance of thephysical process 110. - The
ICs 170 are designed to respond quickly to changes in sensor measurements. However, these measurements can be affected by noise, such as from the measurements themselves, operation of thephysical processes 110, and/or failures or malfunctions of theIC sensors 174. Sensitivity to noise and sensor failures may result in false alarms from theICs 170. Noise may be taken into consideration during predefining of the constraints, so that theICs 170 are able to tolerate a certain level of measurement noise to minimize occurrence of false alarms. It will be appreciated that detection of IC sensor failures will be readily known to the skilled person, such as with thesupervisory devices 160 using analytical failure filters. - Accordingly, the
ICs 170 are designed to improve the resilience of a utilities system, specifically thecyber subsystem 100, to cyber attacks by ensuring that any violation of or deviation from the invariants is detected and communicated to the operators via theHMI 162 soon after occurrence, enabling timely initiation of emergency procedures. As shown inFIG. 4 , the measurements v are directly communicated to theHMI 162 and control centre without passing through thecontroller devices 130. Thus, any attack on thecontroller devices 130 will not compromise the IC 170: theIC 170 continues to alert operators of possible cyber attacks even if thecontroller devices 130 have been compromised. The SCADA workstation compares the measurements y (from the sensors 140) and measurements v (from the IC 170), and determine whether they corroborate each other. Inconsistencies in the measurements may indicate abnormal operation of the utilities system. - Therefore, the
defense system 300 is thus able to verify/authenticate the physical data collected by thesensors 140 by validating/corroborating the physical data at the SCADA workstation. Thedefense system 300 provides a holistic view of the states of thephysical process 110 and hence is able to defend and protect the utilities system from cyber-physical attacks. - In various embodiments of the present disclosure with reference to
FIG. 5 , there is adefense method 500 implemented in thedefense system 300, themethod 500 for defending a utilities system against cyber-physical attacks, the cyber-physical attacks associated with anomalies in aphysical process 110 operative or performed in the utilities system. The anomalies may relate to abnormalities in the operation or performance of thephysical process 110, and/or violations/deviations of physical data of thephysical process 110 from the invariants. For purpose of brevity, themethod 500 is described based on onephysical process 110, but it will be appreciated that themethod 500 is similarly applicable to otherphysical processes 110. - The
method 500 comprises astep 510 of collecting, by thesensors 140, the physical data associated with thephysical process 110. Themethod 520 comprises astep 520 of monitoring, by thecontroller devices 130, process states of thephysical process 110 based on the physical data from thesensors 140. Particularly, thecontroller devices 130 monitor and control thephysical process 110 by altering thephysical process 110 such that the physical data follows an intended behaviour. - Referring to
FIG. 3 , thecyber subsystem 100 comprises one or more sets ofverification devices 180 for monitoring thephysical processes 110 based on the physical data associated with thephysical processes 110. Themethod 500 comprises astep 530 of monitoring, by theverification devices 180, based on the physical data of a physical process from thesensors 140, the physical data enabling theverification devices 180 to detect the anomalies based on the invariants predefined for thephysical process 110. - The
verification devices 180 are designed to be behaviourally and structurally independent of the portion of thecyber subsystem 100 comprising thecontroller devices 130,sensors 140, andactuators 150. Eachverification device 180 is an OD element that provides an additional layer of defense to the utilities system, separated and independent from the traditional defense system. Eachverification device 180 comprises anIC 170 that enables the detection of cyber attacks on thecyber subsystem 100. Theverification devices 180 provide an advantage of enabling thecyber subsystem 100 to compare the physical data from thecontroller devices 130 andverification devices 180, and determine whether they corroborate each other. - In some embodiments, the
verification devices 180 comprise a number ofglobal verification devices 182 programmatically integrated with or is resident in thecontroller devices 130 which may comprise PLCs. Eachglobal verification device 182 comprises one ormore ICs 170 for detecting the anomalies. Theglobal verification devices 182 may also be referred to as PLC-resident Global Intelligent Checkers (PGICs), and may be built or implemented using a Linux-based computer or server. For purpose of brevity and as shown inFIG. 3 , there is oneglobal verification device 182 resident in onecontroller device 130 for monitoring thephysical process 110. - The
global verification device 182 is communicatively connected to thesensors 140 to receive the physical data collected by thesensors 140. Particularly, theglobal verification device 182 detect whether the physical data matches the dynamics of thephysical process 110 based on the invariants predefined for thephysical process 110. The control logic of theglobal verification device 182 is executed at all times together with the PLC control logic of thecontroller device 130. The SCADA workstation compares the physical data from thecontroller device 130 andglobal verification device 182, and determine whether they corroborate each other. Inconsistencies in the physical data are associated with the anomalies in thephysical process 110. - The
controller device 130 in onecontrol stage 120 for aphysical process 110 may be communicatively connected toother controller devices 130 associated withother control stages 120 and otherphysical processes 110 of the collectivephysical process 115 operative in the utilities system. This enables thecontroller device 130 to learn the process states of the otherphysical processes 110 and of the collectivephysical process 115. As eachcontroller device 130 has aglobal verification device 182 resident therein, theglobal verification device 182 in onecontrol stage 120 is able to obtain physical data collected byother sensors 140 for the otherphysical processes 110. - In an attack event where all
sensors 140 in onecontrol stage 120 are compromised, the attack cannot be detected by theglobal verification device 182 in thecontrol stage 120. However, theglobal verification device 182 in anothercontrol stage 120, e.g. a neighbouring, preceding, or succeedingcontrol stage 120, may be able to detect the attack as thesensors 140 in theother control stage 120 have not been compromised. Such detection is possible due to the process interactions across the control stages 120. - In some embodiments, the
verification devices 180 comprise a number oflocal verification devices 184 communicatively isolated from thecontroller devices 130. Eachlocal verification device 184 comprises one ormore ICs 170 for detecting the anomalies. Thelocal verification devices 184 may also be referred to as Local Intelligent Checker (LICs), and may be built or implemented using a computer such as Raspberry Pi. For purpose of brevity and as shown inFIG. 3 , there is onelocal verification device 184 for monitoring thephysical process 110. - The
local verification device 184 is communicatively connected to thesensors 140 to receive the physical data collected by thesensors 140. Particularly, thelocal verification device 184 detect whether the physical data matches the dynamics of thephysical process 110 based on the invariants predefined for thephysical process 110. The SCALA workstation compares the physical data from thecontroller device 130 andlocal verification device 184, and determine whether they corroborate each other. Inconsistencies in the physical data are associated with the anomalies in thephysical process 110. - The
local verification device 184 performs both detection of the anomalies and control of thephysical process 110. Thelocal verification device 184 obtains physical data directly from thesensors 140 and can command theactuators 150. An attack on thecontroller device 130 is likely to affect aglobal verification device 182 resident in thecontroller device 130, but is unlikely to affect thelocal verification device 184 as thelocal verification device 184 is communicatively isolated from thecontroller device 130. Thus, thelocal verification device 184 may provide for better corroboration of the physical data for detecting the anomalies. Thelocal verification device 184 is designed to prevent damage to components of the utilities system at a specific control stage 120 (controlling the physical process 110) or across multiple control stages 120 (controlling multiple physical processes 110). - The physical data is obtained by the
local verification device 184 from thesensors 140 using a secure mechanism, such as by using redundant sensors connected directly to thelocal verification devices 182 or by using asplitter 142 at eachsensor 140. As shown inFIG. 3 , thesensors 140 are connected to thesplitter 142 that splits the output from thesensors 140 into two (or more) identical signals carrying the physical data. One signal is communicated to thecontroller device 130 and the other signal is communicated to thelocal verification device 184. - Thus, the physical data is communicated from the
sensors 140 via thesplitter 142 to thecontroller device 130 andlocal verification device 184 separately. Specifically, the physical data is communicated from thesensors 140 to thecontroller device 130 via a communication network, such as or including a remote input/output interface 190. In addition, the physical data (indicated as S) is communicated from thesensors 140 directly to thelocal verification device 184 via awired communication 184 a (e.g. hardwire) without going through thecontroller device 130. - The
local verification device 184 comprises only onecommunication input 184 b for receiving the physical data via thewired communication 184 a. The wired communication 182 a reduces the chances of a man-in-the-middle attack (cyber attack) on thesensors 140 which can compromise the communication connection between thesensors 140 and thelocal verification device 184. Thewired communication 184 a thus mitigates risk of remote intrusion, e.g. hacking, into thecyber subsystem 100 to tamper with thelocal verification device 184, as well as risk of deception (e.g. false physical data/sensor measurements) to thelocal verification device 184, unless thewired communication 184 a is physically tampered with (physical attack). Accordingly, thelocal verification device 184 is inaccessible to an attacker via the normal communication network used by thecyber subsystem 100, and can be attacked only through physical means. - Other than the
communication input 184 b, thelocal verification device 184 has no other communication input or incoming network links. This prevents thecontroller device 130 from communicating with thelocal verification device 184, thereby mitigating risk of further compromising thelocal verification device 184 if thecontroller device 130 has already been compromised. Thecontroller device 130 andlocal verification device 184 are thus communicatively isolated from each other and this “one-way communication” enforces the security of thedefense system 300. An attempt to communicate from thecontroller device 130 to thelocal verification device 184 would be flagged off as malicious activity. Thelocal verification device 184 is configured to detect as an anomaly in response to thelocal verification device 184 detecting an input communication attempt from thecontroller device 130. - In some embodiments, the
verification devices 180 comprise a number of independentglobal verification devices 186 communicatively connected to thelocal verification devices 184 and similarly communicatively isolated from thecontroller devices 130. Each independentglobal verification device 186 comprises one ormore ICs 170 for detecting the anomalies. The independentglobal verification devices 186 may also be referred to as Independent Global Intelligent Checker (IGICs), and may be built or implemented using a computer such as Raspberry Pi. For purpose of brevity and as shown inFIG. 3 , there is one independentglobal verification device 186 communicatively connected to onelocal verification device 184. - The independent
global verification device 186 receives the physical data from thesensors 140 via thelocal verification device 184. Thelocal verification device 184 has exactly one communication output or outgoing network link 184 c that allows it to communicate information to the independentglobal verification device 186. The independentglobal verification device 186 is communicatively connected to otherlocal verification devices 184 andactuators 150 associated with otherphysical processes 110 and controlstages 120 in the utilities system. The independentglobal verification device 186 is thus configured to obtain state information of the utilities system from alllocal verification devices 184 for allphysical processes 110 across allcontrol stages 120, and may control thephysical processes 110 with theactuators 150 to ensure that the dynamics of thephysical processes 110 are consistent with the physical data obtained from thesensors 140 as well as the invariants predefined for thephysical processes 110. The independentglobal verification device 186 is communicatively connected to thesupervisory devices 160 including the SCADA workstation for corroboration of the physical data and detection of the anomalies in one or morephysical processes 110 in the utilities system. - As described in an attack event above, a non-compromised
global verification device 182 in onecontrol stage 120 may detect an attack in anothercontrol stage 120 with a compromisedglobal verification device 182. However, in rare attack scenarios where allglobal verification devices 182 across allcontrol stages 120 are compromised, such attacks may not get detected. In such attack scenarios, thecontroller devices 130 may be compromised and the control actions by thecontroller devices 130 may not be reliable. For example, when an attacker has assumed control of allcontroller devices 130, control actions by thecontroller devices 130 may be potentially compromised and may cause further damage to the utilities system. - The
local verification device 184 and independentglobal verification device 186 are configured for managing such attack scenarios. As they are communicatively isolated from thecontroller devices 130, an attack on thecontroller devices 130 is unlikely to compromise them. Moreover, thelocal verification device 184 and independentglobal verification device 186 have hardwire direct connections to thesensors 140, and thus are able to collect uncompromised physical data from thesensors 140 for detecting such attacks. Thelocal verification device 184 and independentglobal verification device 186 are programmed to check the invariants, trigger alerts, and control/regulate thephysical processes 110 when an attack is detected. - In one embodiment, there is one
local verification device 184 for eachcontrol stage 120. However, multiplelocal verification devices 184 may be used depending on the complexity of the invariants and the code to respond when an attack is detected. In one embodiment, the utilities system is relatively small and there is one independentglobal verification device 186 which is adequate to obtain local state information from thelocal verification devices 184 for state estimation. In another embodiment, the utilities system is more complex and/or distributed. Multiple independentglobal verification devices 186 may be used for state estimation using domain specific distributed state estimation algorithms. - It will be appreciated that in the
defense system 300 for eachcontrol stage 120, theverification devices 180 may comprise one or more of theglobal verification device 182,local verification device 184, and independentglobal verification device 186. In one embodiment, theverification devices 180 comprise all threedevices verification devices 180 comprise only oneglobal verification device 182 resident in thecontroller device 130. In another embodiment, theverification devices 180 comprise only onelocal verification device 184. In another embodiment, theverification devices 180 comprise only onelocal verification device 184 and one independentglobal verification device 186. - Referring to
FIG. 3 andFIG. 5 , themethod 500 comprises astep 540 of detecting, by theverification devices 180, anomalies in thephysical process 110 operative in the utilities system. Themethod 500 further comprises astep 550 of controlling, by thecontroller device 130 orverification devices 180, theactuators 150 to remedy the anomalies and regulate thephysical process 110, thereby defending the utilities system against the cyber-physical attacks associated with the anomalies. - In some embodiments, upon detecting an attack, the
controller device 130 triggers an alarm but does not generate any control action/signal or actuation signal to theactuators 150. Thecontroller device 130 is prevented from taking any control action to be executed by theactuators 150 as theglobal verification device 182 might have been compromised due to a malware in the embeddingcontroller device 130. Upon detecting the attack, thelocal verification device 184 triggers an alarm and may override control actions by thecontroller device 130. Two types of alarms may be generated by thelocal verification devices 184—a physical alarm and a logical alarm. The physical alarm may be a blinking light and/or an audio signal from thealert devices 164. The logical signal is communicated to theHMI 162 via the independentglobal verification device 186. - Depending on the nature of the attack as detected by the
local verification device 184, the control action may be generated by thelocal verification device 184 or the independentglobal verification devices 186. Each of thelocal verification device 184 and independentglobal verification device 186 is thus configured to assume and take over control of theactuators 150 from the initial control by thecontroller devices 130 in response to thelocal verification device 184 detecting the anomalies. More specifically, thelocal verification device 184 is configured to determine which one of thelocal verification device 184 and independentglobal verification device 186 assumes control of theactuators 150. - The
local verification device 184 is designed to defend one or more specific components of the utilities system. During an attack, theactuators 150 may be controllable by thelocal verification device 184 to defend the specific components. For example, if a pump is turned on but there is no water in the input tank, then the pump must be shut off. Thelocal verification device 184 may control therelevant actuators 150 to shut off the pump and communicate this information to the independentglobal verification device 186. The independentglobal verification device 186 can then communicate with otherlocal verification devices 184 andother actuators 150 so that they can perform appropriate control actions given that a portion of the physical process 110 (and thus of the collective physical process 115) may be shut down. Alternatively, the independentglobal verification device 186 may assume control of theactuators 150 to shut off the pump because it needs to know the states of theother control stages 120/physical processes 110 from the otherlocal verification devices 184. Nevertheless, the independentglobal verification device 186 is useful in ensuring that the dynamics of the collectivephysical process 115 are consistent with its intended behaviour according to the physical data and the invariants. - The
defense system 300 comprises a multiplexer (mux) 152 which is a device that selects one of multiple input signals and forwards a selected input signal into a single line. In thedefense system 300 as shown inFIG. 3 , themultiplexer 152 receives multiple input signals and communicates a selected input signal to theactuators 150. These input signals may also be referred to as control actions or actuation signals. In normal operation of thephysical process 110, thecontroller device 130 communicates actuation signals to theactuators 150 via themultiplexer 152 in order to control thephysical process 110. - However, upon detecting the attack, the
local verification device 184/independentglobal verification device 186 assumes control of theactuators 150 from thecontroller devices 130. Particularly, each of thelocal verification device 184 and independentglobal verification device 186 communicates actuation signals (indicated as A) to theactuators 150 via themultiplexer 152. Similar to the wired communication between thesensors 140 and thelocal verification device 184, thelocal verification device 184 and independentglobal verification device 186 are communicatively connected to themultiplexer 152 via a wired communication. - Depending on the nature of the attack, the
local verification device 184 determine which one of thelocal verification device 184 and independentglobal verification device 186 assumes control of theactuators 150. Thelocal verification device 184 communicates a control signal (indicated as C) to themultiplexer 152, the control signal instructing themultiplexer 152 which actuation signals to communicate to theactuators 150. - Accordingly, the
multiplexer 152 prioritizes the source of the control actions/actuation signals communicated to theactuators 150. Thecontroller device 130 has priority under normal operation of thephysical process 110 and control theactuators 150 in accordance with the design of the utilities system. In the event of cyber-physical attacks, the priority shifts to thelocal verification device 184 that decide whether the control action should come from thelocal verification device 184 or the independentglobal verification device 186. Theactuators 150 are thus controllable by thecontroller device 130 or thelocal verification device 184/independentglobal verification device 186 depending on the attack (or non-attack/normal) scenarios to remedy the anomalies and regulate thephysical processes 110, thereby defending the utilities system against the cyber-physical attacks. - Attack Scenarios
- There are various types or scenarios of cyber-physical attacks that may be detected with the
defense system 300 with the OD system including theverification devices 180 andICs 170. Attacks can be classified based on three security objectives of the utilities system—confidentiality, integrity, and availability. A lack of confidentiality results in disclosure when an unauthorized entity gains access to data in the utilities system. A lack of integrity leads to deception when an authorized party receives false data and believes it is true. A lack of availability results in denial of service - (DoS) or distributed DoS (DDoS) when an authorized entity cannot receive commands or data. Deception, disclosure, and DoS/DDoS are three types of cyber attacks the utilities system.
- Complex attacks on the utilities system may be classified based on three dimensions for the attack space—attacker's a priori system model knowledge, attackers disclosure resources, and attacker's disruption resources. On example of a complex attack is a replay attack, during which an attacker uses disclosure resources to hijack the
sensors 140 and record their measurements of physical data for a certain duration, and then uses his disruption resources to repeat the recorded measurements while carrying out his attack. The attacker can attempt to inject false control inputs or carry out physical attacks on the utilities system while replaying the false sensor measurements. - Another example of a complex attack is a false data injection attack, wherein an attacker uses disruption resources and system knowledge to inject false sensor measurements of physical data in order to mislead the
controller devices 130. Mathematical system state estimation, failure detection methods, and/or analytical detection techniques can be used to detect such attacks, however it is also feasible for skilled attackers to use system knowledge to successfully bypass detection. -
FIG. 6 illustrates a schematic of anattack workflow 600 in a utilities system in the event of a cyber-physical attack. Indicators a1, a2, and a3 represent DoS attacks wherein communication channels are hijacked. Theactuators 150 are prevented from receiving control inputs (a1); thecontroller devices 130 are prevented from receiving physical data from thesensors 140; and/or theHMI 162 is prevented from receiving data from the ICs 170 (a3). Indicator a4 represents a deception attack wherein an attacker (a4) causes false information, e.g. actuation signals, to be communicated from thecontroller devices 130 to the actuators 150 (ũ≠u). Indicator a5 represents a deception attack wherein an attacker (a5) causes false information, e.g. physical data, to be communicated from thesensors 140 to the controller devices 130 ({tilde over (y)}≠y). Indicator a6 represents a deception attack wherein an attacker (a6) causes false information to be communicated from theICs 170 to the HMI 162 ({tilde over (v)}≠v). Indicators a7 and a8 represent direct physical attacks on the physical processes 110 (a7) and/or the ICs 170 (a8). Thealert devices 164 may be configured to be automatically activated in the event of a physical attack on theICs 170. - In the example of a replay attack, the attacker sends false sensor measurements {tilde over (y)} to the controller devices 130 (a5), and injects false control inputs ũ to the actuators 150 (a4) and/or carries out physical attacks (a7). During a false data injection attack, the sensor measurements are modified (a5) ({tilde over (y)}≠y).
- The
ICs 170 in theverification devices 180 of thedefense system 300 can be used to detect the following cyber attacks on thecyber subsystem 100 of the utilities system. -
- Deception attacks on sensor measurements (a5), when false physical data is communicated from the
sensors 130 to the controller devices 130 ({tilde over (y)}≠y). Operators can compare sensor measurements v from theICs 170 and sensor measurements {tilde over (y)} from the controller devices 130 (v≠{tilde over (y)}) to identify inconsistencies. - DoS attacks on sensor measurements (a2), when the attacker prevents the sensor measurements from reaching the
controller devices 130. Operators will notice absence of the sensor measurements, but they will be able to estimate the current states of thephysical processes 110 based on measurements received from the ICs 170 (v) and communicate the correct inputs to thecontroller devices 130. - Physical attacks on the physical processes 110 (a7). If the
ICs 170 detect violation of the invariants, theICs 170 trigger thealert devices 164 to notify the operators. Operators will be notified of possible attacks even if the communication channels between theICs 170 and the control center via theHMI 162 are compromised (a3). - Replay attacks. Operators will notice inconsistencies between the sensor measurements from the
controller devices 130 and from the ICs (v≠{tilde over (y)}). Furthermore, if the attacker attempts to send false control inputs ũ to the actuators 150 (a4) while replaying pre-recorded data, theICs 170 will still be able to measure the actual states of thephysical processes 110. If thephysical processes 110 become critical, theICs 170 will activate thealert devices 164. Thealert devices 164 will also be activated if the attacker performs physical attacks (a7) and thephysical processes 110 become critical. - False data injection attacks. Operators will be able to detect such attacks by comparing the sensor measurements from the
controller devices 130 against the sensor measurements from the ICs 170 (v≠{tilde over (y)}).
- Deception attacks on sensor measurements (a5), when false physical data is communicated from the
- Accordingly, the
verification devices 180 are useful in detecting different types or attacks on the utilities system. Particularly, theverification devices 180 can be used to detect complex attacks, such as replay and false data injection attacks. However, theverification devices 180 are unable to detect all types of attacks, such as disclosure attacks. Therefore, theverification devices 180 should be used in combination with other security mechanisms to better protect the utilities system. More generally, thedefense system 300 comprising the traditional defense system and OD system using theverification devices 180 provide the required security level for the utilities system. - Attack Experiment
- An experiment of a cyber attack on a utilities system was performed to test the
defense system 300 anddefense method 500. A model of the utilities system was developed as a six-stage water treatment plant.FIG. 7 shows a block diagram 700 of the water treatment plant model with the six control stages 120 (indicated as PLC1 to PLC6) and various components. InFIG. 7 , the terms S, A, MV, FIT, LIT, DPIT, AIT, T, and P respectively represent sensors, actuators, motorized valve, flow meter, level meter, differential pressure meter, pH meter, water tank, and pump. - A state model of the water treatment plant was created and the invariants were defined for use in the
verification devices 180. The model includes the water level in five tanks and pH values at two points in the water treatment plant. The oxidation/reduction potential (ORP), conductivity, differential pressure across the Ultrafiltration stage PLC4, and dichlorination were excluded from the model. Including water level in tanks and pH values allows detection of attacks aimed at damaging a pump, causing tank overflow, and reducing the production of quality filtered water. -
FIG. 8 shows a table 800 of sample model equations that were used to derive the invariants. The last entry in the table 800 is one of the several invariants derived from the model. Additional invariants were derived through an examination of the cause-effect relationship across several components of the water treatment plant. For example, if valve MV101 is open and pump P101 is off, then the water level in tank T101 must increase. - The model of the water treatment plant was created according to one embodiment of the
defense system 300, wherein theverification devices 180 comprise only theglobal verification devices 182 for the different stages PLC1 to PLC6 of the water treatment plant. The attack surface for the water treatment plant and the attack model described below were developed to test the effectiveness of thedefense system 300 against a variety of cyber attacks. - An attack surface for a utilities system the set of all of its logical and physical elements. An attack region is a subset of an attack surface.
FIG. 9 shows a table 900 of sample attack regions in the utilities system which is a water treatment plant. The element type, element label, and its function in the water treatment plant are listed. Asensor 140 can function as a logical element as it provides data to acontroller device 130, as well as a physical element that could be damaged or replaced by an attacker. Whether to consider an element as logical and/or physical depends on the element itself as well as on the nature of attacks against which a defense mechanism of the element was designed. Thus, acontroller device 130 is a physical element but also a logical element due to the control logic it stores and executes. - A utilities system-centric attacker model is used to assess the effectiveness of the
defense system 300. Such a model was designed specifically for investigating the impact of cyber-physical attacks on the water treatment plant. In this model, the water treatment plant was considered as an attacker's domain and modelled as a triple (E, Pr, Pe), where E is a finite set of elements in the utilities system (i.e. the attack surface). Pr is a finite set of properties of thephysical processes 110, including properties such as the pH values and conductivity of water at a specific point in the water treatment plant. Pe is a finite set of performance metrics, such as the amount of water produced per unit time, e.g. in gallons/minute. - An attacker model is a pair (I, D), where I is a set of attacker intents and D a collection of subsets of the three sets in the domain (E, Pr, Pe). For example, I may refer to “Overflow a Tank” and the domain may be ([LIT101, MV101], [ ], [ ]). Here, the attacker is not interested in changing any property of water or performance metrics of the water treatment plant. Hence, the subsets corresponding to Pr and Pe are empty.
- The following types of attacks are considered.
-
- Single Stage Single Element (SSSE) attack that focuses on exactly one element which is either a logical element or a physical element.
- Single Stage Multiple Element (SSME) attack that focuses on a plurality of elements at any
stage 120. - Multiple Stage Single Element (MSSE) attack that is similar to an SSSE attack except that it is performed on
multiple stages 120. - Multiple Stage Multiple Element (MSME) attack that is similar to an SSME attack except that it is performed on
multiple stages 120.
- Two detection mechanisms were developed. For detecting SSSE attacks, the invariant uses data from
sensors 140 at thestage 120 where the attacked components reside. For detecting SSME attacks, the invariant uses readings and physical data fromsensors 140 in its immediate neighbouringstages 120, i.e. preceding and/or succeedingstages 120. For example, a deception attack using the level sensor LIT101 is detected using the sensor readings from stage PLC1. However, a deception attack that compromises allsensors 140 at stage PLC1 cannot be detected using sensor readings and physical data from the same stage PLC1. In this case, the detection mechanism embedded as aglobal verification device 182 in the neighbouring stage PLC2 is able to detect the attack. Such detection becomes possible due to the process interactions across different stages PLC1 to PLC6 in the water treatment plant. - A total of twelve SSSE attacks were designed and launched. In addition, six attack scenarios were created for SSME attacks, with one SSME attack corresponding to each attack scenario launched. The SSSE attacks were simple and altered readings of
various sensors 140 in various stages PLC1 to PLC6 of the water treatment plant. The SSME attacks were strategic and aimed at deceiving thecontroller devices 130 by capturing theactuators 150 and communicating simulated false readings to thecontroller devices 130 to make thecontroller devices 130 believe what the attacker intends. All attacks were launched across the stages PLC1, PLC3, PLC4, and PLC5 of the water treatment plant. - The experiment was conducted in two phases. In the first phase, attacks were launched without the
defense system 300 to understand the responses of the water treatment plant. The first phase also serves as a reference basis for comparison with the second phase. In the second phase, thedefense system 300 with theglobal verification devices 182 was implemented in the water treatment plant. The attacks were launched again to assess the effectiveness of thedefense system 300. - The experiment results showed that all SSSE and SSME attacks were detected by the
defense system 300 including theglobal verification devices 182 resident in thecontroller devices 130. The invariants derived from the design of each stage PLC1 to PLC6 of the water treatment plant are effective in detecting SSME attacks. A distributed attack detection mechanism was used to detect the SSME attacks. The logic of this detection mechanism was embedded in theglobal verification devices 182 that communicated with theglobal verification devices 182 in the neighbouringcontroller devices 130 to obtain physical data from the sensors of the neighbouring stages. - While replay attacks on the stage PLC1 were not initially detected as the stage PLC1 only has a succeeding stage PLC2 and no preceding stage, the invariants for the stage PLC1 were redefined and the modified
global verification device 182 for thestage PLC 1 was later successful in detecting the replay attacks. - Implementation
- The
defense system 300 described in various embodiments herein is designed to defend a utilities system, such as a water treatment plant or power generation plant, from cyber-physical attacks such as in situations where the communications network and/or the control logic inside thecontroller devices 130, e.g. PLCs, are compromised. Thedefense system 300 uses an orthogonal defense that is separate/independent from, as well as is auxiliary to and complements, the traditional defense system. - The
defense system 300 comprisesverification devices 180 andICs 170 that are able to detect cyber attacks based on the use of invariants derived from process dynamics and interactions among components across thevarious control stages 120 andphysical processes 110 operative in the utilities system. The architecture of thedefense system 300 is layered, such that everycontrol stage 120 of the utilities system comprises one ormore verification devices 180 for monitoring thephysical processes 110 controlled in the control stages 120. Theverification devices 180 cooperate with thecontroller devices 130,sensors 140, andactuators 150 to understand industrial protocols for monitoring and controlling thephysical processes 110. - The
verification devices 180 in the utilities system may comprise multiplelocal verification devices 184 and one independentglobal verification device 186. Thelocal verification devices 184 communicate physical data from thesensors 140 and other state-related information of the respective control stages 120 and for the respectivephysical processes 110 to the independentglobal verification device 186, which is a centrally distributed device in the utilities system. The independentglobal verification device 186 also obtains component state information from other sources, such as the controller devices 130 (e.g. PLCs) and supervisory devices 160 (e.g. SCADA workstation, Historian, and OPC server). The information about the same components obtained from multiple sources is not redundant. Each component state information source can be compromised by a smart attacker, and in the extreme case, by an insider working in the utilities system. By corroborating the information from multiple potential attack points, it is possible for thedefense system 300 to detect a cyber attack, trigger alarms, and pinpoint the source and domain of the attack in the utilities system. - The design of the
defense system 300 is not influenced by any specific attacker model or attack model. Thedefense system 300 is attack-agnostic in its defense mechanism, in that, the corroboration of component state information is focused on the state-correctness of the utilities system, and not on the attacker's approach. Thedefense system 300 is thus able to detect unique and undocumented attacks, also known as zero day attacks, when they involve modification of component values at some level of the utilities system that cause deviation of dynamics of thephysical processes 110 from the expected behaviour. - The
defense system 300 may be implemented in a utilities system that is still in the design stage or one that is already operational in an existing plant or facility. In the former approach, thedefense system 300 can be designed such that the traditional defense system of the utilities system is integrated with the orthogonal defense using theverification devices 180 andICs 170. During the design phase, the hardware and software portions of thedefense system 300 can be incorporated. Implementing thedefense system 300 at the design phase minimizes the risk of malfunction or even failure during operation of the utilities system. - In the latter approach, the
defense system 300 may be implemented when the plant is under maintenance instead of during normal operation when thephysical processes 110 are already being monitored and controlled. In some utilities system, thecontroller devices 130 may utilize dual PLCs. One of these PLCs is the primary PLC while the other (secondary PLC) functions as a hot backup to assume or take over control when the primary PLC malfunctions or fails. Thedefense system 300 may be implemented in the secondary PLC of onecontroller device 130 in onecontrol stage 120. Particularly, aglobal verification device 182 may be implemented in the secondary PLC. Operation of theglobal verification device 182 may be tested by switching it to function as the primary PLC instead. Implementation of thedefense system 300 may be extended to theother control stages 120 of the utilities system upon successful testing. - In the foregoing detailed description, embodiments of the present disclosure in relation to a defense system and method against cyber-physical attacks are described with reference to the provided figures. The description of the various embodiments herein is not intended to call out or be limited only to specific or particular representations of the present disclosure, but merely to illustrate non-limiting examples of the present disclosure. The present disclosure serves to address at least one of the mentioned problems and issues associated with the prior art. Although only some embodiments of the present disclosure are disclosed herein, it will be apparent to a person having ordinary skill in the art in view of this disclosure that a variety of changes and/or modifications can be made to the disclosed embodiments without departing from the scope of the present disclosure. Therefore, the scope of the disclosure as well as the scope of the following claims is not limited to embodiments described herein.
Claims (20)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
SG10201607439T | 2016-09-07 | ||
SG10201607439T | 2016-09-07 | ||
PCT/SG2017/050444 WO2018048351A1 (en) | 2016-09-07 | 2017-09-06 | Defense system and method against cyber-physical attacks |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190253440A1 true US20190253440A1 (en) | 2019-08-15 |
US11431733B2 US11431733B2 (en) | 2022-08-30 |
Family
ID=61561981
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/331,409 Active 2039-11-01 US11431733B2 (en) | 2016-09-07 | 2017-09-06 | Defense system and method against cyber-physical attacks |
Country Status (3)
Country | Link |
---|---|
US (1) | US11431733B2 (en) |
SG (1) | SG10201912502QA (en) |
WO (1) | WO2018048351A1 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190219994A1 (en) * | 2018-01-18 | 2019-07-18 | General Electric Company | Feature extractions to model large-scale complex control systems |
CN112600814A (en) * | 2020-12-08 | 2021-04-02 | 震兑工业智能科技有限公司 | Detection method and system for deception attack of ship CPS (cyber-physical system) |
CN113364796A (en) * | 2021-06-18 | 2021-09-07 | 西安建筑科技大学 | Self-triggering model prediction control security defense method and system under FDI attack |
WO2021177899A1 (en) * | 2020-03-05 | 2021-09-10 | Singapore University Of Technology And Design | Power system security enhancement |
US20210334370A1 (en) * | 2020-04-24 | 2021-10-28 | Honeywell International Inc. | Systems and methods for embedded anomalies detector for cyber-physical systems |
US11212257B2 (en) * | 2018-06-22 | 2021-12-28 | Aeronix, Inc. | Multi-level secure ethernet switch |
US11316883B2 (en) * | 2019-07-17 | 2022-04-26 | Bank Of America Corporation | Cybersecurity—operational resilience of computer networks |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
SG10201912502QA (en) | 2016-09-07 | 2020-02-27 | Univ Singapore Technology & Design | Defense system and method against cyber-physical attacks |
US20190334932A1 (en) * | 2018-04-26 | 2019-10-31 | Abb Schweiz Ag | Technologies for Detecting Cyber-Attacks against Electrical Distribution Devices |
EP3611587A1 (en) * | 2018-08-16 | 2020-02-19 | Siemens Aktiengesellschaft | System for controlling and monitoring of adaptive cyber physical systems |
US10990668B2 (en) | 2018-09-17 | 2021-04-27 | General Electric Company | Local and global decision fusion for cyber-physical system abnormality detection |
US20200099252A1 (en) * | 2018-09-26 | 2020-03-26 | Abb Schweiz Ag | Secure distributed state estimation for networked microgrids |
US20200106301A1 (en) * | 2018-10-01 | 2020-04-02 | Abb Schweiz Ag | Decentralized false data mitigation for nested microgrids |
JP2020095672A (en) * | 2018-11-28 | 2020-06-18 | オムロン株式会社 | Controller system |
RU2724075C1 (en) | 2018-12-28 | 2020-06-19 | Акционерное общество "Лаборатория Касперского" | System and method for determining anomaly source in cyber-physical system having certain characteristics |
JP7255369B2 (en) * | 2019-06-06 | 2023-04-11 | オムロン株式会社 | control system |
US20210034031A1 (en) * | 2019-08-02 | 2021-02-04 | Hitachi, Ltd. | Using ai for ensuring data integrity of industrial controllers |
US20210084056A1 (en) * | 2019-09-18 | 2021-03-18 | General Electric Company | Replacing virtual sensors with physical data after cyber-attack neutralization |
CN111698257B (en) * | 2020-06-17 | 2022-05-20 | 武汉科技大学 | Industrial information physical system security detection method for multi-class malicious attacks |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101081157B1 (en) * | 2010-02-25 | 2011-11-07 | 중앙대학교 산학협력단 | A Dynamic Control Management System for Cyber-Physical System |
JP5731223B2 (en) * | 2011-02-14 | 2015-06-10 | インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation | Abnormality detection device, monitoring control system, abnormality detection method, program, and recording medium |
US9177139B2 (en) * | 2012-12-30 | 2015-11-03 | Honeywell International Inc. | Control system cyber security |
WO2014109645A1 (en) * | 2013-01-08 | 2014-07-17 | Secure-Nok As | Method, device and computer program for monitoring an industrial control system |
US9405900B2 (en) * | 2013-03-13 | 2016-08-02 | General Electric Company | Intelligent cyberphysical intrusion detection and prevention systems and methods for industrial control systems |
US20160330225A1 (en) * | 2014-01-13 | 2016-11-10 | Brightsource Industries (Israel) Ltd. | Systems, Methods, and Devices for Detecting Anomalies in an Industrial Control System |
US10075475B2 (en) * | 2015-02-06 | 2018-09-11 | Honeywell International Inc. | Apparatus and method for dynamic customization of cyber-security risk item rules |
SG10201912502QA (en) | 2016-09-07 | 2020-02-27 | Univ Singapore Technology & Design | Defense system and method against cyber-physical attacks |
-
2017
- 2017-09-06 SG SG10201912502QA patent/SG10201912502QA/en unknown
- 2017-09-06 US US16/331,409 patent/US11431733B2/en active Active
- 2017-09-06 WO PCT/SG2017/050444 patent/WO2018048351A1/en active Application Filing
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190219994A1 (en) * | 2018-01-18 | 2019-07-18 | General Electric Company | Feature extractions to model large-scale complex control systems |
US11212257B2 (en) * | 2018-06-22 | 2021-12-28 | Aeronix, Inc. | Multi-level secure ethernet switch |
US11316883B2 (en) * | 2019-07-17 | 2022-04-26 | Bank Of America Corporation | Cybersecurity—operational resilience of computer networks |
WO2021177899A1 (en) * | 2020-03-05 | 2021-09-10 | Singapore University Of Technology And Design | Power system security enhancement |
US20210334370A1 (en) * | 2020-04-24 | 2021-10-28 | Honeywell International Inc. | Systems and methods for embedded anomalies detector for cyber-physical systems |
CN112600814A (en) * | 2020-12-08 | 2021-04-02 | 震兑工业智能科技有限公司 | Detection method and system for deception attack of ship CPS (cyber-physical system) |
CN113364796A (en) * | 2021-06-18 | 2021-09-07 | 西安建筑科技大学 | Self-triggering model prediction control security defense method and system under FDI attack |
Also Published As
Publication number | Publication date |
---|---|
SG10201912502QA (en) | 2020-02-27 |
WO2018048351A1 (en) | 2018-03-15 |
US11431733B2 (en) | 2022-08-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11431733B2 (en) | Defense system and method against cyber-physical attacks | |
Morris et al. | Industrial control system traffic data sets for intrusion detection research | |
US11093606B2 (en) | System and method for detecting a cyber-attack at SCADA/ICS managed plants | |
US10108168B2 (en) | Industrial control system smart hardware monitoring | |
US10911482B2 (en) | Method of detecting cyber attacks on a cyber physical system which includes at least one computing device coupled to at least one sensor and/or actuator for controlling a physical process | |
US20160330225A1 (en) | Systems, Methods, and Devices for Detecting Anomalies in an Industrial Control System | |
US10592668B2 (en) | Computer system security with redundant diverse secondary control system with incompatible primary control system | |
Adepu et al. | Assessing the effectiveness of attack detection at a hackfest on industrial control systems | |
WO2017160913A1 (en) | Intrusion detection via semantic fuzzing and message provenance | |
Adepu et al. | Argus: An orthogonal defense framework to protect public infrastructure against cyber-physical attacks | |
Robles-Durazno et al. | PLC memory attack detection and response in a clean water supply system | |
Escudero et al. | Process-aware model based IDSs for industrial control systems cybersecurity: approaches, limits and further research | |
Shrivastava et al. | Design and assessment of an orthogonal defense mechanism for a water treatment facility | |
Palleti et al. | Cascading effects of cyber-attacks on interconnected critical infrastructure | |
Sabaliauskaite et al. | Intelligent checkers to improve attack detection in cyber physical systems | |
CA2927826C (en) | Industrial control system smart hardware monitoring | |
EP3646561B1 (en) | A threat detection system for industrial controllers | |
EP3190769B1 (en) | A system and method for detecting malicious re-programming of a plc in scada systems using time deception | |
Takagi et al. | Strategic security protection for industrial control systems | |
El-Kady et al. | Security of digitalized process systems | |
Ghaeini et al. | Zero residual attacks on industrial control systems and stateful countermeasures | |
CN105074833A (en) | Device and method for detecting unauthorised manipulations of the system state of an open-loop and closed-loop control unit of a nuclear plant | |
Sicard et al. | Distance Concept Based Filter Approach for Detection of Cyberattacks on Industrial Control Systems | |
Park et al. | Intrusion Detection System for Industrial Network | |
US20230185986A1 (en) | Anomaly detection system for a cyber-physical system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
AS | Assignment |
Owner name: SINGAPORE UNIVERSITY OF TECHNOLOGY AND DESIGN, SINGAPORE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MATHUR, ADITYA;ADEPU, SRIDHAR;SHRIVASTAVA, SIDDHANT;AND OTHERS;REEL/FRAME:049006/0584 Effective date: 20160201 Owner name: SINGAPORE UNIVERSITY OF TECHNOLOGY AND DESIGN, SIN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MATHUR, ADITYA;ADEPU, SRIDHAR;SHRIVASTAVA, SIDDHANT;AND OTHERS;REEL/FRAME:049006/0584 Effective date: 20160201 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |