CN112740234A - Neuromorphic system for authorized user detection - Google Patents
Neuromorphic system for authorized user detection Download PDFInfo
- Publication number
- CN112740234A CN112740234A CN201980031921.5A CN201980031921A CN112740234A CN 112740234 A CN112740234 A CN 112740234A CN 201980031921 A CN201980031921 A CN 201980031921A CN 112740234 A CN112740234 A CN 112740234A
- Authority
- CN
- China
- Prior art keywords
- neuromorphic
- sensor data
- input
- output
- processing component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 65
- 238000000034 method Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 15
- 230000007704 transition Effects 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000005315 distribution function Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000012986 modification Methods 0.000 claims description 5
- 230000004048 modification Effects 0.000 claims description 5
- 230000003595 spectral effect Effects 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 4
- 230000015654 memory Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 210000002569 neuron Anatomy 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 239000007788 liquid Substances 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000002547 anomalous effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000002964 excitative effect Effects 0.000 description 2
- 230000002401 inhibitory effect Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/70—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
- G06F21/88—Detecting or preventing theft or loss
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/316—User authentication by observing the pattern of computer usage, e.g. typical user behaviour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
A neuromorphic system for authorized user detection is described. The system includes a client device including a plurality of types of sensors providing streaming sensor data and one or more processors. The one or more processors include an input processing component and an output processing component. A neuromorphic electronic component is embedded in or on the client device to continuously monitor streaming sensor data and generate output spikes based on the streaming sensor data. Further, the output processing component classifies the streaming sensor data based on the output spike to detect and classify the anomaly signal.
Description
Government rights
The invention was made with government support under U.S. contract number D15PC 00153. The government has certain rights in the invention.
Cross Reference to Related Applications
This application is a continuation-in-part application of U.S. application No.15/338,228 filed on 28/10/2016 (which is a non-provisional patent application of U.S. provisional application No.62/247,557 filed on 28/10/2015 in the united states), the entire contents of which are incorporated herein by reference.
This application is also a non-provisional patent application of U.S. provisional application No.62/684,515 filed in 2018, 6 and 13 in the united states, the entire contents of which are incorporated herein by reference.
Background
(1) Field of the invention
The present invention relates to anomaly detection systems, and more particularly to low power neuromorphic detection systems that detect categorical changes by autonomously discovering significant differences between sensor data.
(2) Background of the invention
Anomaly detection systems are commonly used to classify or otherwise identify system dynamics that may be designated as anomalous behavior. Conventional automatic detection systems operate on top of conventional electronics and therefore use a large amount of energy to solve complex classification problems. Such systems typically run on large-scale processors or other relatively large computing systems. However, implementing anomaly detection in various mobile and field applications requires a small system that can efficiently handle complex problems. Notably, such detection or classification systems have not been incorporated into small-scale chips that can handle complex problems with little computational overhead.
Thus, there is a continuing need for low power, small neuromorphic abnormality detection systems.
Disclosure of Invention
The present disclosure relates to a neuromorphic system for authorized user detection. The system includes neuromorphic electronic components embedded in or attached to a client device. The neuromorphic electronic component has a neuromorphic chip capable of continuously monitoring streaming sensor data from a client device and generating an output spike (out-spike) based on the streaming sensor data.
In another aspect, the neuromorphic system further includes a client device including an input processing component, an output processing component, and a plurality of types of sensors for providing streaming sensor data.
In another aspect, the output processing component classifies the streaming sensor data based on the output spike to detect a user transition.
In yet another aspect, the input processing component is configured to further perform the following:
normalizing the streaming sensor data from the multiple types of sensors into a normalized time series;
grouping the normalized time series from the multiple types of sensors into a single scalar;
collecting packets of the single scalar sample into a queue;
transforming the queue into discrete one-dimensional frequency domain data;
modifying the one-dimensional frequency domain data with a window function to reduce spectral leakage and generate frequency domain modified data;
filtering the frequency domain modification data to obtain a scaled windowed frequency interval;
scaling both the normalized time series and the scaled windowed frequency to generate an input rate;
mapping the input rate to a distribution function; and
an input spike is generated based on the mapped input rate.
In another aspect, the neuromorphic electronic component generates an output spike based on the input spike.
In another aspect, the neuromorphic electronic component generates output spikes based on the input spikes using a randomly connected excitatory-inhibitory pulse network.
In yet another aspect, the output processing component further performs the following:
smoothing the output spike to a rate;
applying the rate to a linear classifier to generate a reading (readout);
filtering the readings; and
classifying the streaming sensor data as an anomaly signal based on the reading.
In addition, after classifying the exception signal, the output processing component further performs at least one of:
locking or unlocking access to the client device;
starting a new processing task;
executing a new logical branch of executable code;
sending information associated with the exception signal;
saving information associated with the exception signal into a memory device.
Finally, the present invention also includes a computer program product and a computer-implemented method. The computer program product includes computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors such that, when the instructions are executed, the one or more processors perform the operations listed herein. Alternatively, a computer-implemented method includes acts that cause a computer to execute such instructions and perform the resulting operations.
Drawings
The objects, features and advantages of the present invention will become apparent from the following detailed description of the various aspects of the invention, when taken in conjunction with the following drawings, in which:
FIG. 1 is an illustration of a neuromorphic system for authorized user detection according to various embodiments of the present disclosure;
FIG. 2 is a block diagram depicting components of an input processing component and an output processing component according to some embodiments of the present disclosure;
FIG. 3 is an illustrative diagram of a computer program product in accordance with some embodiments of the present disclosure;
FIG. 4 provides a flow diagram depicting the processing flow within an input processing component according to some embodiments of the present disclosure;
FIG. 5 is a graphical representation of a path that an electrical signal may travel through a neuromorphic electronic component, according to some embodiments of the present disclosure;
FIG. 6 illustrates a processing flow of an output processing component according to some embodiments of the present disclosure;
FIG. 7 is a flow diagram illustrating an example of a hierarchical threat detection system in which the neuromorphic system of the present disclosure is implemented;
FIG. 8 shows a schematic diagram of an example mobile device and neuromorphic electronic components in which the neuromorphic detection system of the present disclosure is implemented;
FIG. 9 is an illustration of an example mobile device in which the neuromorphic detection system of the present disclosure is implemented; and
fig. 10 is a graph illustrating exponentially smoothed reading signals from 5 users of a neuromorphic platform depicting detected user transitions.
Detailed Description
The present invention relates to anomaly detection systems, and more particularly to low power neuromorphic detection systems that detect categorical changes by autonomously discovering significant differences between sensor data. The following description is presented to enable any person skilled in the art to make and use the invention and is incorporated in the context of a particular application. Various modifications and uses of the various applications will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to a wide variety of aspects. Thus, the present invention is not intended to be limited to the aspects shown, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
In the following detailed description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without limitation to these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.
The reader is directed to all papers and documents which are filed concurrently with this specification and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference. All the features disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.
Furthermore, any element in the claims that does not explicitly recite "means" or "step" for performing a particular function is not to be construed as an "means" or "step" clause specified in 35u.s.c. section 112, clause 6. In particular, the use of "step … …" or "action … …" in the claims herein is not intended to refer to the provisions of section 6, section 112, 35u.s.c.
Before describing the present invention in detail, a description of the various principal aspects of the invention is provided first, followed by an introduction. In the following, specific details of the invention are provided to give an understanding of specific aspects. Finally, a number of example implementations are provided.
(1) Main aspects of the invention
Various embodiments of the present invention include three "primary" aspects. As shown in fig. 1, the first main aspect is a low power neuromorphic system for authorized user detection. The system has three general components, namely, an input processing component 102, a neuromorphic electronic component 104, and an output processing component 106. Both the input processing 102 and output processing 106 components are implemented in software and/or hardware as computer systems owned by or otherwise within "client" systems (such as memory and processing components within mobile devices, vehicles, etc.). The neuromorphic electronic component 104 is a component that is implemented in neuromorphic hardware and performs most of the computational processing. The second main aspect is a method, typically in the form of software or other program, that runs using a data processing system (computer) and the neuromorphic hardware described herein. A third broad aspect is a computer program product. The computer program product generally represents computer readable instructions stored on a non-transitory computer readable medium, such as an optical storage device, such as a Compact Disc (CD) or Digital Versatile Disc (DVD), a Field Programmable Gate Array (FPGA), or a magnetic storage device, such as a floppy disk or magnetic tape. Other non-limiting examples of computer-readable media include hard disks, Read Only Memories (ROMs), and flash-type memories. These aspects will be described in more detail below.
Fig. 2 provides a block diagram depicting a non-limiting example of a computer system 200, which computer system 200 may be implemented to function as an input processing component and/or an output processing component (i.e., elements 102 and/or 106 of fig. 1). Such a computer system 200 is configured to perform calculations, processes, operations, and/or functions associated with a program or algorithm. In one aspect, certain processes and steps discussed herein are implemented as a series of instructions (e.g., a software program) that reside within a computer-readable storage unit and are executed by one or more processors of the computer system 200. When executed, the instructions cause the computer system 200 to perform specific actions and to exhibit specific behavior as may be required to perform the processes described herein.
In one aspect, the computer system 200 may include an input device 212 coupled to the address/data bus 202, wherein the input device 212 is configured to communicate information and command selections to the processor 200. According to one aspect, the input device 212 is an alphanumeric input device (such as a keyboard) that may include alphanumeric and/or function keys. Alternatively, the input device 212 may be an input device other than an alphanumeric input device. In one aspect, the computer system 200 may include a cursor control device 214 coupled with the address/data bus 202, wherein the cursor control device 214 is configured to communicate user input information and/or command selections to the processor 200. In one aspect, cursor control device 214 is implemented using a device such as a mouse, trackball, track pad, optical tracking device, or touch screen. Notwithstanding the foregoing, in one aspect, cursor control device 214 is directed and/or activated via input from input device 212 (such as in response to the use of special keys and key sequence commands associated with input device 212). In an alternative aspect, cursor control device 214 is configured to be guided or directed by voice commands.
In an aspect, the computer system 200 may also include one or more optional computer usable data storage devices (such as storage device 216) coupled to the address/data bus 202. The storage device 216 is configured to store information and/or computer-executable instructions. In one aspect, the storage device 216 is a storage device such as a magnetic or optical disk drive (e.g., a hard disk drive ("HDD"), a floppy disk, a compact disk read-only memory ("CD-ROM"), a digital versatile disk ("DVD"), according to one aspect, a display device 218 is coupled to the address/data bus 202, wherein the display device 218 is configured to display video and/or graphics.
The computer system 200 presented herein is an example computing environment in accordance with an aspect. However, non-limiting examples of computer system 200 are not strictly limited to computer systems. For example, one aspect provides that computer system 200 represents one type of data processing analysis that may be used in accordance with various aspects described herein. Other computing systems may also be implemented. Indeed, the spirit and scope of the present technology is not limited to any single data processing environment. Thus, in one aspect, one or more operations of various aspects of the present technology are controlled or implemented using computer-executable instructions (such as program modules) executed by a computer. In one implementation, such program modules include routines, programs, objects, components, and/or data structures that are configured to perform particular tasks or implement particular abstract data types. In addition, an aspect provides for implementing one or more aspects of the technology by utilizing one or more distributed computing environments, such as where tasks are performed by remote processing devices that are linked through a communications network, or where program modules are located in both local and remote computer storage media including memory-storage devices.
An illustrative diagram of a computer program product (i.e., a storage device) embodying an aspect of the present invention is shown in FIG. 3. The computer program product is shown as a floppy disk 300 or an optical disk 302 such as a CD or DVD. However, as previously mentioned, the computer program product generally represents computer readable instructions stored on any compatible non-transitory computer readable medium. The term "instructions" as used in relation to the present invention generally represents a set of operations to be performed on a computer and may represent a fragment of an entire program or a single separable software module. Non-limiting examples of "instructions" include computer program code (source or object code) and "hard-coded" electronics (i.e., computer operations encoded into a computer chip or otherwise neuromorphic hardware chip). "instructions" are stored on any non-transitory computer readable medium, such as a memory of a computer or in floppy disks, CD-ROMs, FPGAs, and flash drives.
(2) Introduction to
A neuromorphic system for authorized user detection is described. More specifically and referring again to fig. 1, the system is a low power neuromorphic anomaly detection system 100, the low power neuromorphic anomaly detection system 100 identifying anomalies by detecting categorical changes via significant differences between the autonomously discovered sensor data 108. Once such a change is detected, the system generates an output signal 110 as appropriate for the particular application. The output signal 110 may trigger any number of subsequent actions, such as computer processing tasks, transmitting information related to anomalies or environmental changes, and storing data.
The neuromorphic electronic component 104 described herein may be implemented using any suitable neuromorphic hardware. In one aspect, the neuromorphic electronic component 104 is implemented using neuromorphic hardware described in U.S. patent No.8,977,578, the entire contents of which are incorporated herein by reference. The low power neuromorphic system of the present disclosure uses neuromorphic hardware (such as described in the' 578 patent) along with additional data flow steps to generate a single output signal 110. Furthermore, the (tune)/configuration neuromorphic hardware is tuned in a specific manner to produce the desired result.
The system is unique in that it uses neuromorphic hardware (e.g., an electronically-implemented nonlinear fluid state machine) in conjunction with a software linear classifier to perform classification of sensor data (not just images). This combination of linear and non-linear steps is unique and provides a significant improvement over the prior art. Neuromorphic techniques are scalable in size and perform fast enough for complex problems. Furthermore, the system uses machine learning techniques to discover the differences in normal (normal) and abnormal (off-normal) conditions that may be practically applied to modern "client" systems that include electronic appliances. Importantly, the system applies a linear classifier to a coupled liquid state machine with electronics and software as arranged in the figures filed herewith. In doing so, the system provides low power consumption handling of complex problems occurring on "client" systems and enables the system to use fewer resources than prior art systems. This implementation allows a very low power (less than 100 milliwatts) solution to the complex classification problem with fast response times (milliseconds to seconds depending on the application) and small size and weight.
Conventional automatic detection systems (without neuromorphic hardware) use more energy to solve complex classification problems because their software runs on top of conventional electronics, which are power inefficient and do not parallelize the problem (and therefore run slower). Because the system of the present disclosure can perform activities when a trigger occurs, the client system does not have to perform those activities at all times. Thus, substantial savings in processor utilization, power consumption, data storage, and/or transmission bandwidth are achieved.
(3) Details of various embodiments
As shown in fig. 1, the neuromorphic system 100 includes three serial components: an input processing component 102 (which receives sensor inputs 108), a neuromorphic electronic component 104, and an output processing component 106 (which generates output signals 110). The sensor input 108 includes various types of sensor data. Non-limiting examples of sources of sensor input 106 data include gyroscopes, accelerometers, altimeters, fuel levels, computer network traffic, etc., which feed the input processing component 102.
As described above, both the input processing 102 and output processing 106 components are implemented in software and run on conventional computer processors and may be incorporated into any platform or client device that may receive streaming data. Examples of such client devices include mobile devices (e.g., phones, ipads, etc.), autonomous vehicles, computers, or any other platform/client device that uses a processor and receives streaming data. Accordingly, while a particular client device is described below and illustrated as a mobile device such as a telephone, it is to be understood that the present invention is not so limited, as all of the features described and illustrated can be incorporated into a variety of different applications.
The neuromorphic electronic component 104 is a component that is implemented in neuromorphic hardware and performs most of the computational processing. The neuromorphic electronic component 104 communicates with the other processing components 102 and 106 through a digital interface, such as a Serial Peripheral Interface (SPI). The output process 106 generates a binary output signal 110, the binary output signal 110 representing a normal state (e.g., authorized user) or an anomaly detection state (e.g., unauthorized user).
For further understanding, FIG. 4 provides a flow diagram depicting the processing flow within the input processing 102 components. The input processing 102 component cleans up (i.e., normalizes 400) the sensor inputs 108 (i.e., streams the sensor data) and then maps the data to the frequency domain (fd) or the time domain (td). All data (both td and fd) is then converted to rate and then to input spikes (in-spikes)418 as described in further detail below.
The sensor input 108 data is normalized 400 by mapping the sensor specific range onto a zero to one scale to generate a normalized time series of data. In the frequency domain, the data is combined 402 by grouping the data types into a single scalar by time instance. Thereafter, packets of samples are collected 404 into a queue of sample sizes appropriate for the application. The queue is converted 406 into discrete one-dimensional (1D) frequency domain data. The 4081D frequency domain data is then modified by multiplying the data by a window function, such as a Hamming window (Hamming window) or any other suitable window function, to reduce spectral leakage. Thereafter, the data is filtered 410 by rejecting frequency bins (frequency bins) outside the particular application frequency range, resulting in a scaled windowed frequency bin. Non-limiting examples of such applications and frequency ranges include selecting a frequency between 0.5Hz and 10Hz in gait applications.
The input processing 102 component then scales 412 all values (both the normalized time series and the scaled windowed frequency bins) so that the values are constrained to the maximum execution time to generate the input rate (in-rate). For example, the values are linearly mapped onto a range of spike rates (e.g., 0Hz to 200Hz, etc.). The input rate is mapped 414 to a distribution function (e.g., poisson probability distribution function (P)). Finally, input spikes 418 are generated 416 for each input pad of the neuromorphic electronic component 104 and its neuromorphic chip 821. The input spikes 418 are binary values for each input pad that are generated based on a comparison of a randomly generated number and a poisson probability distribution (P) value associated with the input rate value. Input spikes 418 are sent in a structure equivalent to a one-dimensional binary array. The maximum number of input spikes 418 is equal to the number of input pads present on a neuromorphic chip (i.e., a neuromorphic electronic component having an electronically implemented liquid state machine with leak integrated fire (leak integrated and fire) neurons, such as disclosed in U.S. patent No.8,977,578).
Fig. 5 provides a graphical representation of the neuromorphic electronic component 104, as well as a path that an electronic signal may travel from an input pad 500 to an output pad 502 for a particular configuration. Light squares 504 represent excitatory neurons and dark squares 506 represent inhibitory neurons. The neuromorphic electronic component 104 receives the input spike 418 as an input and generates the output spike 508 as an output pad signal. The output spikes 508 are also sent in a structure equivalent to a one-dimensional binary array. The maximum number of output spikes 508 is equal to the number of output pads 502 present on the neuromorphic chip of the neuromorphic electronic component 104.
Fig. 6 illustrates the processing flow of the output processing 106 component. As shown in fig. 6, the output spikes 508 are smoothed 600 into a release rate or output rate 602, which is a measure of the number of spikes per second. The rate 602 is then applied to a linear classifier 604 to calculate 606 a reading 608. The linear classifier 604 serves as a mapping from the rate 602 to the reading 608. Linear classifiers are generally understood by those skilled in the art (see, e.g., the disclosure by
A defined "linear classifier," the entire contents of which are incorporated herein by reference). The linear classifier 604 is so called because each reading 608 is a linear combination of the rates 602. Thus, this is a simple matrix multiplication operation, where the learning process is to determine a matrix by which the rate 602 is multiplied to obtain the reading 608. Desirably, inDiscriminant training 603 must be performed on the linear classifier 604 before reliable anomaly signal generation occurs. Training 603 is a supervised machine learning technique based on the use of truth values 614. Each training phase is specific to one category (there may be two or more categories) at a time. The difference between the categories (time and/or frequency domain) is what will cause an anomaly or anomalous signal. The frequency domain data is calculated in real time and used in a sliding window fashion. The amount of frequency domain output data size is equal to the queue sample size.
The readings 608 are a plurality of floating point values corresponding to respective categories (2 or more). The readings are then filtered 610 to remove noise and an anomaly detection process 612 is performed, which produces a final output signal 110 (e.g., an anomaly signal specifying the presence of an authorized user or an unauthorized user). The anomaly detection process 612 identifies an unauthorized user or an authorized user by signaling that a user change has occurred. When there is a user transition, the reading 608 signal becomes abnormal and the abnormality (in the reading 608 signal) is detected by the system.
(4) Example implementation
As can be appreciated by those skilled in the art, the neuromorphic abnormality detection system of the present disclosure has many applications. For example, the system has been applied to detect user changes of a mobile device using biometric sensor data. In this implementation, the system is physically attached or embedded into the mobile device, where power usage of the mobile device is a major resource saving concern. When an unauthorized user is detected, the mobile device locks the user from further use of the device until an authorized user is detected (at which time the features/functions of the mobile device are unlocked and accessible). In addition, the system may be used in other applications where power consumption, feature size, and/or device accessibility are very limited. The system is also beneficial in situations where it would be most beneficial if complex anomaly detection (for an off-server/cloud solution) were performed directly on the client system.
As a specific example, the low-power neuromorphic system of the present disclosure is applied in a hierarchical anomaly classification solution, such as in stage 1 block 700 of a hierarchical threat detection algorithm 702 (shown in fig. 7) described further in U.S. application No.15/338,228. In other words, the process described in U.S. application No.15/338,228 is modified to include the low power neuromorphic system of the present disclosure as stage 1 block 700. In this modified system, sensor data is generated from the gyroscope and accelerometer at a frequency of 50 Hz. The queue size is 200 samples. The frequency range of interest for applications is 0.3Hz to 20.0 Hz. The upper rate limit is 200 Hz. The execution time ranges from 1.5ms to 5 ms. About 25 input pads are used effectively (active) and about 50 output pads are used effectively. A Liquid State Machine (LSM) is configured such that 300 excitatory neurons and 25 inhibitory neurons are activated. The network map of the neurons is connected at 1%. The system successfully distinguishes between multiple users operating the phone and issues a phone alert when an unauthenticated user walks with the phone.
Fig. 8 shows a schematic diagram of an example mobile device 800 (e.g., a telephone) and neuromorphic electronic components 104 (i.e., neuromorphic chip 821 in communication with FPGA 823 or any other hardware or components required to allow neuromorphic chip 821 to operate). FPGA 823 is responsible for configuration of the neuromorphic chip 821 (e.g., non-limiting examples of which include loading a particular network, such as the network described above with respect to fig. 5). Subcomponents that have been developed and exist on the mobile device 800 include a sensor data collection application 802 and a data resampler 804, the data resampler 804 converting all sensor streams from non-uniform sensor data 803 to a uniform sampling rate (i.e., uniform data (binary) 805). The output of resampler 804 goes directly to a local EWS application 806 and to a rate-based spike encoder 808 (which serves as the input processing component described above). It should be noted that shielded cable 830 for transmission is illustrated; however, the present invention is not limited to such cables, and the particular cable 830 as shown is provided as a non-limiting example of such a transmission medium.
The sensor signals encoded into input spikes 418 are sent to the neuromorphic electronic component 104 via a Serial Peripheral Interface (SPI) connection 807 and corresponding line driver and receiver 827 on the mobile device 800. The alert generated by the neuromorphic electronic component 104 is encoded into an output spike 518 and sent over the SPI connection 807 (and corresponding line driver and receiver 827) to the mobile device 800 where the output spike 518 is decoded 809, then read by the EWS application 806 (i.e., which serves as the output processing component described above) to determine intent or classification, which is then broadcast 813 to an optional policy engine 815, which optional policy engine 815 maintains a policy regarding acceptable intent.
As a non-limiting example, if the policy engine 815 specifies that the category of "unauthorized users" cannot be allowed to continue using the device 800, various protocols or actions 817 may be sent for implementation by the mobile device 800, such as the following: all device access 819 is locked until an authorized user is detected (e.g., the authorized user enters the appropriate access code into the system). Other examples of exception-based detection (i.e., unauthorized user access, etc.) include starting a new processing task or executing a new logical branch of executable code, sending information associated with the exception, and saving the information associated with the exception to a memory device. In some aspects, the above activities are terminated upon or shortly after transitioning from the abnormal back to the normal state. Or, as yet another example, other features of the device 800 in the device 800 may be unlocked if the signal is classified as an authorized user. As an optional step, activities between devices may be related through EWS 825.
Fig. 9 illustrates an example mobile device 800 implementing the neuromorphic detection system of the present disclosure and the neuromorphic electronic components 104 included on an interface backplane attached to the mobile device 800. The signal output from device 800 via the SPI connection was tested. As shown in fig. 10, the exponentially smooth reading signals from 5 users of the neuromorphic platform were input to the neuromorphic classifier and performed with an average accuracy of 84.16%. In FIG. 10, the "filtered readings" axis 1002 is a dimensionless number. They are the result of multiplying the output rate with a linear classifier weight matrix and then applying a low pass filter. The "sample ID" axis 1004 refers to the index of data points and is therefore also dimensionless. It should be appreciated that since the index and time are isomorphic (isomorphic) in this case, the x-axis may also be generated based on time.
The classification accuracy of each user is 66.53% at the lowest and 99.21% at the highest. The average true positive rate and negative rate were 89.67% and 60.32%, respectively. Since the chip was trained on data from 5 different subjects, the chance classification rate was 20%. These results indicate a reasonably good user classification for low power systems. The sharp transition across the time domain represents a user transition 1000, which is measured by the reading signal. As described above, when there is a user transition 1000, the reading signal becomes abnormal, and the abnormality (in the reading signal) is detected by the system.
In some embodiments, perhaps more important than user classification is the ability of the neuromorphic system to detect changes in user identity. To create meaningful performance estimates, a continuous alarm aggregation strategy is formulated on the user transition output and the time series is divided into equal-length blocks from which a ground-route is determined. First, the user transition policy sets the minimum interval (margin) between detecting false alarms to 8.2 seconds. This prevents unwanted successive alarms from occurring for the same transition event. Next, the total number of experiments for each alarm test is set as: (total time in sample set)/(2 x interval). An alarm is classified as false negative if it is not detected within +/-seconds of the true time associated with the alarm. If an alarm is detected, the alarm is considered to be true positive. Using this experimental setup, the resulting metric yielded 98.74% accuracy in detecting user transitions, with 99.57% true positive and 75% true negative. These results demonstrate the great advantage of using the neuromorphic system as a user-transition detection system because it has a high accuracy rate, minimizing the amount of time that the local EWS will be running, thereby further saving power.
Continuous behavior based authentication of devices (e.g., mobile devices) is an important core technology area. In the field of defense and commerce where the problem of stealing personal data from lost or stolen mobile devices (e.g., telephones) is becoming more serious, the development of improved low-power security and authentication techniques for mobile devices has received great attention. Mobile devices are increasingly embedded in vehicles and airplanes, and authentication of these devices is becoming increasingly important given the intentional adoption of these systems by hostiles. Power-efficient behavior-based inferences are needed for continuously and reliably verifying the security necessary for a user with minimal burden. The invention described herein provides translation capabilities for the development of next generation behavior-based authentication and enhanced security protocols for a variety of devices, including mobile devices. Upon detection of an unauthorized user, the device may be caused to perform various automated operations including stopping the operation, safely pulling the autonomous vehicle to the curb and off, locking the user, and the like.
Finally, while the invention has been described in terms of various embodiments, those skilled in the art will readily recognize that the invention can have other applications in other environments. It should be noted that many embodiments and implementations are possible. Furthermore, the following claims are in no way intended to limit the scope of the present invention to the specific embodiments described above. Additionally, any statement that "means for … …" is intended to cause a reading of the element and the device-plus-function of the claim, and any element not specifically used with the term "means for … …" is not intended to be read as a device-plus-function element even if the claim additionally includes the term "means. Further, although specific method steps have been recited in a particular order, these method steps may occur in any desired order and are within the scope of the present invention.
Claims (18)
1. A neuromorphic system for authorized user detection, the neuromorphic system comprising:
a neuromorphic electronic component for embedding in or attaching to a client device, the neuromorphic electronic component having a neuromorphic chip capable of continuously monitoring streaming sensor data from the client device and generating output spikes based on the streaming sensor data.
2. The neuromorphic system of claim 1, further comprising: a client device comprising a plurality of types of sensors to provide the streaming sensor data, an input processing component, and an output processing component, and wherein the output processing component classifies the streaming sensor data based on the output spikes to detect user transitions.
3. The neuromorphic system of claim 2, wherein the input processing component is configured for further performing the following:
normalizing the streaming sensor data from the multiple types of sensors into a normalized time series;
grouping the normalized time series from the multiple types of sensors into a single scalar;
collecting packets of the single scalar sample into a queue;
transforming the queue into discrete one-dimensional frequency domain data;
modifying the one-dimensional frequency domain data with a window function to reduce spectral leakage and generate frequency domain modified data;
filtering the frequency domain modification data to obtain a scaled windowed frequency interval;
scaling both the normalized time series and the scaled windowed frequency to generate an input rate;
mapping the input rate to a distribution function; and
an input spike is generated based on the mapped input rate.
4. The neuromorphic system of claim 3, wherein the neuromorphic electronic component generates an output spike based on the input spike.
5. The neuromorphic system of claim 4, wherein the neuromorphic electronic component generates the output spikes based on the input spikes using a randomly connected excitatory-inhibitory spike network.
6. The neuromorphic system of claim 2, wherein the output processing component further performs the following:
smoothing the output spike to a rate;
applying the rate to a linear classifier to generate a reading;
filtering the readings; and
classifying the streaming sensor data as an anomaly signal based on the reading.
7. The neuromorphic system of claim 2, wherein after classifying the abnormal signal, the output processing component further performs at least one of:
locking or unlocking access to the client device;
starting a new processing task;
executing a new logical branch of executable code;
sending information associated with the exception signal;
saving information associated with the exception signal into a memory device.
8. A neuromorphic method for authorized user detection, the neuromorphic method comprising the acts of:
continuously monitoring streaming sensor data from a client device using a neuromorphic electronic component having a neuromorphic chip; and
generating an output spike based on the streaming sensor data with the neuromorphic electronic component having the neuromorphic chip.
9. The neuromorphic method of claim 8, further comprising the acts of:
classifying, using the client device, the streaming sensor data based on the output spike to detect a user transition, the client device having a plurality of types of sensors to provide the streaming sensor data, an input processing component, and an output processing component, wherein the output processing component classifies the streaming sensor data.
10. The neuromorphic method of claim 9, further performing, by the input processing component:
normalizing the streaming sensor data from the multiple types of sensors into a normalized time series;
grouping the normalized time series from the multiple types of sensors into a single scalar;
collecting packets of the single scalar sample into a queue;
transforming the queue into discrete one-dimensional frequency domain data;
modifying the one-dimensional frequency domain data with a window function to reduce spectral leakage and generate frequency domain modified data;
filtering the frequency domain modification data to obtain a scaled windowed frequency interval;
scaling both the normalized time series and the scaled windowed frequency to generate an input rate;
mapping the input rate to a distribution function; and
an input spike is generated based on the mapped input rate.
11. The neuromorphic method of claim 10, further comprising the acts of: generating, by the neuromorphic electronic component, the output spike based on the input spike.
12. The neuromorphic method of claim 11, wherein the neuromorphic electronic component generates the output spikes based on the input spikes using a randomly connected excitatory-inhibitory spike network.
13. The neuromorphic method of claim 10, further comprising, by the output processing component:
smoothing the output spike to a rate;
applying the rate to a linear classifier to generate a reading;
filtering the readings; and
classifying the streaming sensor data as an anomaly signal based on the reading.
14. The neuromorphic method of claim 10, wherein after classifying the abnormal signal, the neuromorphic method further comprises performing at least one of:
locking access to the client device;
starting a new processing task;
executing a new logical branch of executable code;
sending information associated with the exception signal; and
saving information associated with the exception signal into a memory device.
15. A computer program product for authorized user detection, the computer program product comprising:
a non-transitory computer-readable medium encoded with executable instructions such that, when executed by one or more processors, the one or more processors perform operations comprising:
receiving an output spike from a neuromorphic electronic component having a neuromorphic chip, the neuromorphic electronic component continuously monitoring streaming sensor data from a client device and generating an output spike based on the streaming sensor data;
classifying, using the client device, the streaming sensor data based on the output spike to detect a user transition, the client device having a plurality of types of sensors to provide the streaming sensor data, an input processing component, and an output processing component, wherein the output processing component classifies the streaming sensor data.
16. The computer program product of claim 15, further comprising instructions for causing the input processing component to:
normalizing the streaming sensor data from the multiple types of sensors into a normalized time series;
grouping the normalized time series from the multiple types of sensors into a single scalar;
collecting packets of the single scalar sample into a queue;
transforming the queue into discrete one-dimensional frequency domain data;
modifying the one-dimensional frequency domain data with a window function to reduce spectral leakage and generate frequency domain modified data;
filtering the frequency domain modification data to obtain a scaled windowed frequency interval;
scaling both the normalized time series and the scaled windowed frequency to generate an input rate;
mapping the input rate to a distribution function; and
an input spike is generated based on the mapped input rate.
17. The computer program product of claim 16, further comprising instructions for causing the output processing component to:
smoothing the output spike to a rate;
applying the rate to a linear classifier to generate a reading;
filtering the readings; and
classifying the streaming sensor data as an anomaly signal based on the reading.
18. The computer program product of claim 17, wherein after classifying an exception signal, the computer program product further causes the one or more processors to perform at least one of:
locking access to the client device;
starting a new processing task;
executing a new logical branch of executable code;
sending information associated with the exception signal; and
saving information associated with the exception signal into a memory device.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201862684515P | 2018-06-13 | 2018-06-13 | |
| US62/684,515 | 2018-06-13 | ||
| PCT/US2019/026830 WO2019240868A1 (en) | 2018-06-13 | 2019-04-10 | Neuromorphic system for authorized user detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112740234A true CN112740234A (en) | 2021-04-30 |
| CN112740234B CN112740234B (en) | 2024-01-02 |
Family
ID=68842032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980031921.5A Active CN112740234B (en) | 2018-06-13 | 2019-04-10 | Neuromorphic system for authorized user detection |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP3807822A4 (en) |
| CN (1) | CN112740234B (en) |
| WO (1) | WO2019240868A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111669514B (en) * | 2020-06-08 | 2021-02-26 | 北京大学 | High dynamic range imaging method and apparatus |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150047040A1 (en) * | 2013-08-09 | 2015-02-12 | Behavioral Recognition Systems, Inc. | Cognitive information security using a behavioral recognition system |
| CN104685516A (en) * | 2012-08-17 | 2015-06-03 | 高通技术公司 | Apparatus and methods for spiking neuron network learning |
| CN107016437A (en) * | 2016-01-13 | 2017-08-04 | 国际商业机器公司 | The method efficiently generated, system and computer-readable recording medium for the random peaks pattern in core base neuromorphic system |
| US20170337469A1 (en) * | 2016-05-17 | 2017-11-23 | Agt International Gmbh | Anomaly detection using spiking neural networks |
| US20170344885A1 (en) * | 2016-05-31 | 2017-11-30 | International Business Machines Corporation | Timing Sequence for Digital STDP Synapse and LIF Neuron-based Neuromorphic System |
| US20180075345A1 (en) * | 2016-09-13 | 2018-03-15 | Sap Se | Spatio-temporal spiking neural networks in neuromorphic hardware systems |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8515885B2 (en) * | 2010-10-29 | 2013-08-20 | International Business Machines Corporation | Neuromorphic and synaptronic spiking neural network with synaptic weights learned using simulation |
| US8977578B1 (en) * | 2012-06-27 | 2015-03-10 | Hrl Laboratories, Llc | Synaptic time multiplexing neuromorphic network that forms subsets of connections during different time slots |
| US10055434B2 (en) * | 2013-10-16 | 2018-08-21 | University Of Tennessee Research Foundation | Method and apparatus for providing random selection and long-term potentiation and depression in an artificial network |
| US10157629B2 (en) * | 2016-02-05 | 2018-12-18 | Brainchip Inc. | Low power neuromorphic voice activation system and method |
| US20170227995A1 (en) * | 2016-02-09 | 2017-08-10 | The Trustees Of Princeton University | Method and system for implicit authentication |
| US10628568B2 (en) * | 2016-03-31 | 2020-04-21 | Fotonation Limited | Biometric recognition system |
-
2019
- 2019-04-10 CN CN201980031921.5A patent/CN112740234B/en active Active
- 2019-04-10 EP EP19820220.2A patent/EP3807822A4/en active Pending
- 2019-04-10 WO PCT/US2019/026830 patent/WO2019240868A1/en active Search and Examination
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104685516A (en) * | 2012-08-17 | 2015-06-03 | 高通技术公司 | Apparatus and methods for spiking neuron network learning |
| US20150047040A1 (en) * | 2013-08-09 | 2015-02-12 | Behavioral Recognition Systems, Inc. | Cognitive information security using a behavioral recognition system |
| CN107016437A (en) * | 2016-01-13 | 2017-08-04 | 国际商业机器公司 | The method efficiently generated, system and computer-readable recording medium for the random peaks pattern in core base neuromorphic system |
| US20170337469A1 (en) * | 2016-05-17 | 2017-11-23 | Agt International Gmbh | Anomaly detection using spiking neural networks |
| US20170344885A1 (en) * | 2016-05-31 | 2017-11-30 | International Business Machines Corporation | Timing Sequence for Digital STDP Synapse and LIF Neuron-based Neuromorphic System |
| US20180075345A1 (en) * | 2016-09-13 | 2018-03-15 | Sap Se | Spatio-temporal spiking neural networks in neuromorphic hardware systems |
Non-Patent Citations (1)
| Title |
|---|
| PHILLIPS MATTHEW E: "Neuromorphic and early warning behavior-based authentication for mobile devices", IEEE, pages 1 - 4 * |
Also Published As
| Publication number | Publication date |
|---|---|
| EP3807822A4 (en) | 2022-03-23 |
| EP3807822A1 (en) | 2021-04-21 |
| WO2019240868A1 (en) | 2019-12-19 |
| WO2019240868A8 (en) | 2020-11-26 |
| CN112740234B (en) | 2024-01-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Azmoodeh et al. | Detecting crypto-ransomware in IoT networks based on energy consumption footprint | |
| Venkatraman et al. | A hybrid deep learning image-based analysis for effective malware detection | |
| Ganju et al. | Property inference attacks on fully connected neural networks using permutation invariant representations | |
| Ramotsoela et al. | Attack detection in water distribution systems using machine learning | |
| Aleesa et al. | Review of intrusion detection systems based on deep learning techniques: coherent taxonomy, challenges, motivations, recommendations, substantial analysis and future directions | |
| Selim et al. | Anomaly events classification and detection system in critical industrial internet of things infrastructure using machine learning algorithms | |
| EP3654216A1 (en) | Computer-security event security-violation detection | |
| Yamanaka et al. | Autoencoding binary classifiers for supervised anomaly detection | |
| US20210406368A1 (en) | Deep learning-based analysis of signals for threat detection | |
| Elnakib et al. | EIDM: Deep learning model for IoT intrusion detection systems | |
| Sahar et al. | Deep learning approach-based network intrusion detection system for fog-assisted iot | |
| Shahin et al. | A novel fully convolutional neural network approach for detection and classification of attacks on industrial IoT devices in smart manufacturing systems | |
| Moustakidis et al. | A novel feature extraction methodology using Siamese convolutional neural networks for intrusion detection | |
| Du et al. | Digital forensics as advanced ransomware pre-attack detection algorithm for endpoint data protection | |
| Assegie | An optimized KNN model for signature-based malware detection | |
| CN115004153A (en) | Demonstration of nerve flow | |
| Roshan et al. | Deep learning approaches for anomaly and intrusion detection in computer network: A review | |
| Li et al. | Deep learning algorithms for cyber security applications: A survey | |
| Raymond et al. | Investigation of Android Malware with Machine Learning Classifiers using Enhanced PCA Algorithm. | |
| WO2022115419A1 (en) | Method of detecting an anomaly in a system | |
| Hassaoui et al. | Domain generated algorithms detection applying a combination of a deep feature selection and traditional machine learning models | |
| CN112740234B (en) | Neuromorphic system for authorized user detection | |
| US10902115B2 (en) | Neuromorphic system for authorized user detection | |
| Khan et al. | DivaCAN: Detecting in-vehicle intrusion attacks on a controller area network using ensemble learning | |
| Azhagiri et al. | An intrusion detection system using ranked feature bagging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |