WO2020111356A1

WO2020111356A1 - Spiking neural network device and intelligent device including same

Info

Publication number: WO2020111356A1
Application number: PCT/KR2018/015188
Authority: WO
Inventors: 황태호; 김병수; 권진산
Original assignee: 전자부품연구원
Priority date: 2018-11-26
Filing date: 2018-12-03
Publication date: 2020-06-04
Also published as: KR20200068051A; KR102464490B1

Abstract

Disclosed are a spiking neural network device and an intelligent device including same. The spiking neural network device according to the present invention includes: a pre-processing unit for receiving multichannel data, filtering and normalizing the received data and converting the filtered and normalized data to multidimensional data, and extracting feature points of the converted multidimensional data; and a spiking neural network unit which is designed to allocate one neuron to each piece of the multidimensional data from which the feature points have been extracted and generate spike signals according to the intensities of signals, connects each of the neurons, allocates one synapse between each of the connected neurons, sends out the spike signals by comparing a preset threshold and the sum of the values of the synapses from which the spike signals have been generated among the synapses, and groups the sent out spike signals to classify and cluster the same.

Description

Spiking neural network device and intelligent device including same

The present invention relates to a neuromorphic technology, and more particularly, to a spiking neural network device for simplifying and modeling the signaling and learning system between neurons based on spiking and an intelligent device including the same.

Recently, neuromorphic technology that mimics the brain of a license has attracted attention. Neuromorphic technology includes multiple pre-synaptic neurons, multiple post-synaptic neurons and multiple synapses.

Meanwhile, the neural network software model such as existing deep learning is a model optimized for a back propagation algorithm and requires a large amount of data and high-performance computing resources for learning.

The network learned offline from the server is limited in terms of versatility because it is difficult to adapt to various environments in a situation where it is applied in the form of being loaded and executed in a small hardware language.

In order to solve this, a method of conducting learning through a separate wired/wireless network with a server is applied, but in low-power IoT devices, there is a problem that requires a high cost of a communication device for intelligent learning and processing.

Recently, neuromorphic technology is being researched to implement intelligence in low-complexity devices through imitation of biosignals. However, mainly the model of the structure of neurons and synapses and the signal transduction system is being studied, so there is no simple and effective model that can be applied immediately to semiconductors.

The technical problem to be achieved by the present invention is to simplify and integrate neuron models, synaptic models, and spike models studied in the neuromorphic field, and spies that support models of cognitive performance satisfactory in intelligent IoT devices through supervised and unsupervised learning. It is an object to provide a King Neural Network device and an intelligent device including the same.

To achieve the above object, the spiking neural network device according to the present invention receives multi-channel data, filters and normalizes the received data, converts it into multi-dimensional data, and features points of the converted multi-dimensional data. Designed to generate a spike signal according to the strength of the signal by allocating a single neuron for each of the multi-dimensional data from which the pre-processing unit and the feature point are extracted, and each neuron is connected, and one synapse is allocated between the connected neurons, Includes a spiked neural network unit that compares a summation value with respect to a value of a synapse in which the spike signal is generated, and a preset threshold value, sends the spike signal, groups and classifies the exported spike signals, and clusters them. do.

In addition, the pre-processing unit, a sensor interface unit that receives multi-channel data in real time, performs noise reduction on the data, amplifies the noise-removed data, normalizes it to data within a certain range, and converts it into the multi-dimensional data It characterized in that it comprises a signal processing unit and a feature point extraction unit for extracting the feature points by measuring the amount of change in the signal in the convolution operation or time window (convolution) operation or a time window (time window) section of the converted multi-dimensional data.

In addition, the spiking neural network unit, one neuron is assigned to each dimension or time to the multidimensional data, and an input layer designed to generate a spike signal based on probability according to signal strength. It is configured, and allocates one synapse between the connected neurons, and when the sum value for the value of the synapse in which the spike signal is generated is higher than the threshold value, sends the spike signal, and when the threshold value is lower than the threshold value It characterized in that it comprises a hidden layer that does not emit a spike signal and an output layer that groups the exported spike signal to classify the data and outputs the result by clustering.

In addition, the hidden layer, it characterized in that it is composed of a single layer or a fully connected (fully connected) multi-layer.

In addition, the output layer is characterized in that if there is labeled data among the grouped data, the spike signal is counted and mapped to the corresponding labeling group for output.

In addition, the spiking neural network unit is characterized by updating the weight value of the synapse by using a time difference between a spike signal emitted by a pre-neuron and a spike signal emitted by a post neuron.

In addition, the spiking neural network unit increases the weight by LTP (Long Term Potentiation) when the difference between the time at which the pre-neuron sends a spike signal and the time at which the post-neuron sends a spike signal is greater than 0, and the difference is less than 0. If small, it is characterized by making LTD (Long Term Digression).

The intelligent device according to the present invention is a communication unit that performs communication with an external device, a sensor unit that measures at least one sensing data among environment information, image information, detection information, and recognition information using a plurality of sensors and is received from the communication unit A control unit that receives the measured data and the sensing data measured from the sensor unit in a multi-channel, and performs learning and arithmetic processing using a spiking neural network device based on neuromorphic technology based on the received multi-channel data Including, the spiking neural network device, receiving the multi-channel data, filtering and normalizing the received data into multi-dimensional data, pre-processing to extract the feature points of the converted multi-dimensional data Designed to generate a spike signal according to the strength of a signal by allocating a single neuron for each of the multi-dimensional data from which the negative and the feature points are extracted, connecting each neuron, allocating one synapse between the connected neurons, and And a spiked neural network unit that compares a summation value with respect to a value of a synapse where a spike signal is generated and a preset threshold value, sends the spike signal, groups the spike signals by grouping them, and clusters them. do.

The spiking neural network device of the present invention and the intelligent device including the same may support a simplified network model structure of spiking-based neurons.

In addition, a simple learning model based on extreme frequency period-dependent plasticity (STDP) with high bio-similarity may be applied, and an additional learning model similar to a living body may be applied.

Through this, the present invention can simplify and integrate neuron models, synaptic models, and spike models, and support satisfactory recognition performance in intelligent IoT devices through supervised and unsupervised learning.

1 is a view for explaining an intelligent device according to an embodiment of the present invention.

2 is a diagram for describing a spiking neural network device according to an embodiment of the present invention.

3 is a view for explaining synaptic neurons and spiking signals according to an embodiment of the present invention.

4 is a view for explaining a histogram of the Poisson distribution according to an embodiment of the present invention.

5 is a view for explaining a complete connection between layers according to an embodiment of the present invention.

6 is a view for explaining a spiking snap neuron according to an embodiment of the present invention.

7 is a view for explaining an extreme wave period dependent plasticity rule according to an embodiment of the present invention.

8 is a view for explaining an operation on the MNIST data of the spiking neural network device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, when adding reference numerals to the components of each drawing, it is noted that the same components have the same reference numerals as possible even though they are displayed on different drawings. In addition, in describing the present invention, when it is determined that a detailed description of related known configurations or functions is apparent to those skilled in the art or may obscure the gist of the present invention, the detailed description thereof will be omitted.

Referring to FIG. 1, the intelligent device 200 is an Internet of Things (IoT) device, and performs intelligent learning and arithmetic processing based on various data. That is, the intelligent device 200 can acquire various information in real time through communication with various devices through IoT communication as well as calculation through learning. The intelligent device 200 may include a refrigerator, a TV, a robot cleaner, an air conditioner, a humidifier, an air cleaner, and the like. The intelligent device 200 includes a communication unit 210, a sensor unit 220 and a control unit 230, and further includes an output unit 240 and a storage unit 250.

The communication unit 210 communicates with an external device. At this time, the communication unit 210 may support IoT communication. The communication unit 210 may receive data from various external devices or transmit data to the external devices. Here, the external device is a device supporting IoT communication, and may include a refrigerator, a TV, a robot cleaner, an air conditioner, a humidifier, and an air cleaner.

The sensor unit 220 includes a plurality of sensors, and uses it to measure sensing data of at least one of environmental information, sensing information, and cognitive information. The sensor unit 220 may include a temperature sensor, a humidity sensor, an infrared sensor, a pressure sensor, a camera, a microphone, an electrocardiogram (ECG) sensor, and a heart rate (PPG) sensor.

The control unit 230 may receive data received from the communication unit 210 and sensing data measured from the sensor unit 220 in real time. The controller 230 performs learning and arithmetic processing using the spiking neural network (SNN) model 100 based on neuromorphic technology based on the received multi-channel data. The controller 230 may perform data classification and clustering and association by performing learning and arithmetic processing through the spiking neural network device 100. The detailed description of the spiking neural network device 100 will be described with reference to FIGS. 2 to 7.

The output unit 240 outputs the results of the learning and calculation processing through the control unit 230. The output unit 240 may output in the form of an image, audio, or operation. To this end, the output unit 240 may include various types of modules such as a display, a speaker, a transfer device, and a robot arm.

The storage unit 250 stores external data received from the communication unit 210 and sensing data measured from the sensor unit 220. The storage unit 250 stores data learned from the control unit 230 and data that has been processed. The storage unit 250 includes a flash memory type, a hard disk type, a media card micro type, and a card type memory (for example, SD or XD memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, It may include at least one storage medium among magnetic disks and optical disks.

2 is a view for explaining a spiking neural network device according to an embodiment of the present invention, FIG. 3 is a view for explaining a synaptic neuron and a spiking signal according to an embodiment of the present invention, and FIG. 4 is a present invention Is a diagram for explaining the histogram of the Poisson distribution according to an embodiment, Figure 5 is a view for explaining the complete connection between the layers according to an embodiment of the present invention, Figure 6 is a spike according to an embodiment of the present invention FIG. 7 is a diagram for explaining snaptic neurons, and FIG. 7 is a diagram for explaining extreme wave duration dependent plasticity rules according to an embodiment of the present invention.

2 to 7, the spiking neural network device 100 includes a pre-processing unit 10 and a spiking neural network unit 30.

The pre-processing unit 10 receives multi-channel data, converts the received data into multi-dimensional data by filtering and normalization, and extracts feature points of the converted multi-dimensional data. To accomplish this, the pre-processing unit 10 includes a sensor interface unit (sensor I/F) 11, a signal processing unit 13, and a feature extraction unit 15.

The sensor interface unit 11 supports a variety of analog/digital interfaces to receive multi-channel data in real time. The sensor interface unit 11 supports I2C, I2S, Universal asynchronous receiver/transmitter (UART), analog-digital converter (ADC), Serial Peripheral Interface (SPI), and general-purpose input/output (GPIO).

The signal processor 13 removes noise included in multi-channel data received from the sensor interface 11. In this case, the signal processor 13 may remove noise using at least one of a low pass filter, a high pass filter, and a band pass filter. The signal processing unit 13 amplifies the signal of the data from which the noise has been removed, and then standardizes the data within a predetermined range to convert it into multidimensional data.

The feature point extraction unit 15 extracts the feature points by measuring the amount of change in the signal in a convolution operation or a time window section of the transformed multidimensional data. Here, the feature point may be a maximum value (peak), a minimum value, or a preset pattern in the data signal.

The spiking neural network unit 30 transmits a spike signal (Fig. 3(b)), when the electrical signals transmitted between the cells of the authorized neurons are combined and exceeds a certain threshold (Fig. 3(a)). It is a control module modeled by simplifying the process of delivering. That is, the spiking neural network unit 30 is designed to generate a spike signal according to the strength of the signal by allocating one neuron for each multi-dimensional data from which feature points are extracted. The spiking neural network unit 30 connects each neuron and allocates one synapse between the connected neurons. At this time, the spiking neural network unit 30 compares the sum value with respect to the value of the synapse that generated the spike signal among the synapses and a preset threshold value, sends the spike signal, groups the classified spike signals, and classifies them. do. The spiking neural network unit 30 includes an input layer 31, a hidden layer 33, and an output layer 35.

The input layer 31 is designed such that one neuron is assigned to each dimension or time to multidimensional data, and generates a spike signal based on probability according to the signal strength. At this time, the input layer 31 may be designed to generate a spike signal based on a Poisson distribution (FIG. 4). That is, the input layer 31 generates a spike signal at a high rate when the signal size is large.

The hidden layer 33 is configured in such a way that all neurons are connected. The hidden layer 33 allocates one synapse between connected neurons. Here, the hidden layer 33 may be composed of a single layer or a multi-layer of a fully connected method. In the case of a fully connected multi-layer, neurons of the first layer may be configured to be connected to all neurons of the second layer (FIG. 5). Here, when the hidden layer 33 is composed of two or more multi-layers, more abstracted information can be obtained, but there is a disadvantage in that the number of synapses and neurons increases exponentially, so that the appropriate number of layers is suitable for the application. Can be configured.

The hidden layer 33 compares the sum value with respect to the value of the synapse in which the spike signal is generated, and a preset threshold value, and sends a spike signal. Here, the synapse is a memory having a weight value, and serves to adjust the frequency of synapses emitted from the hidden layer 33. In detail, the hidden layer 33 sends out a spike signal when the sum value of the synapse value in which the spike signal is generated among the synapses is higher than a preset threshold value, and does not send a spike signal when it is lower than a preset threshold value.

The output layer 35 groups and classifies the spike signals emitted from the hidden layer 33 and clusters them to output a result. If there is labeled data among the grouped data, the output layer 35 counts the spike signal and maps it to the corresponding labeling group to output. At this time, the output layer 35 may be applied with a probability-based mapping method as well as a number of simple spike signals.

For example, since the neurons of the hidden layer 33 that generated the spike signal for the first signal can emit a relatively low frequency spike signal to the second signal, they are converted into probability values for the number of labeled data sets. Judgment can improve accuracy.

Meanwhile, the spiking neural network unit 30 may perform learning. Learning is to update the weight value of the synapse by using a time difference between the spike signal emitted by the pre-neuron and the post signal emitted by the pre-neuron similar to the living body. For example, if the time difference between the time at which a free (synaptic) neuron emits a spike signal and the time at which a post (synaptic) neuron emits a spike is greater than 0, the weight is increased by Long Term Potentiation (LTP), and if the difference is less than 0, LTD ( Long Term Digression) (Fig. 6). At this time, the spiking neural network unit 30 determines the amount of weight to be changed according to LTP and LTD according to Δt as shown in FIG. 7 by a simple equation. That is, the weight value determined according to the spike-timing-dependent plasticity (STDP) rule adds a user input (eg, positive/negative) value to determine a weight value of a reward method.

2 and 8, an embodiment of an application for recognizing handwritten characters of numbers 0 to 9 according to the structure of the spiking neural network device 100 will be described.

The input layer 31 is composed by allocating one neuron per pixel to 28×28 2D image data, and about 300,000 synapses are allocated by connecting the hidden layer 33 composed of 400 neurons in a fully connected manner. The spiking neural network device 100 was constructed.

8 is a result of a process of learning by updating the weight of a synapse according to the STDP rule by a time difference between a spike signal generated based on probability in the input layer 33 and a spike signal generated in the hidden layer 33.

That is, the figure showing the weight of FIG. 8 is a diagram showing 256 grayscales according to the degree of value for 784 points of 28×28 in which each neuron for 400 hidden layers 33 of 20×20 is internally. .

Therefore, when the image of the number 1 is input, the spiking neural network device 100 counts the position and number of neurons of the hidden layer 33 that generates the spike signal, and outputs the result of the number 1 in the output layer 35 as a result. It has a cognitive action.

Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific preferred embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. Of course, various modifications can be made to anyone having ordinary knowledge, and such changes are within the scope of the claims.

[Description of codes]

10: pre-processing unit 11: sensor interface unit

13: signal processing unit 15: feature point extraction unit

30: spiking neural network unit 31: input layer

33: hidden layer 35: output layer

100: spiking neural network device 200: intelligent device

210: communication unit 220: sensor unit

230: control unit 240: output unit

250: storage

Claims

A pre-processor for receiving multi-channel data, filtering and normalizing the received data into multi-dimensional data, and extracting feature points of the converted multi-dimensional data; And

Designed to generate a spike signal according to signal strength by allocating one neuron for each extracted multidimensional data of the feature points, connecting each neuron, allocating one synapse between the connected neurons, and the spike signal among the synapses A spiked neural network unit that compares the sum value of the generated synapse value and a preset threshold value, sends out the spike signal, groups the spike signals, classifies them, and clusters them;

Spiking neural network device comprising a.
According to claim 1,

The pre-processing unit,

A sensor interface unit that receives multi-channel data in real time;

A signal processor that performs noise reduction on the data, amplifies the noise-removed data, and then standardizes the data to a multi-dimensional data by standardizing the data within a predetermined range; And

A feature point extractor for extracting the feature points by measuring the amount of change in a signal in a convolution operation or a time window section of the converted multidimensional data;

Spiking neural network device comprising a.
According to claim 1,

The spiking neural network unit,

An input layer designed to generate a spike signal based on probability according to the strength of the signal, wherein one neuron is allocated to each of the dimensions or time to the multidimensional data;

All of the neurons are configured to be connected, and one synapse is allocated between the connected neurons, and when the sum value for the value of the synapse in which the spike signal is generated is higher than the threshold value, the spike signal is emitted. , A hidden layer that does not emit the spike signal when it is lower than the threshold value; And

An output layer for classifying the grouped data by grouping the exported spike signals and clustering the data to output a result;

Spiking neural network device comprising a.
According to claim 3,

The hidden layer,

Spiking neural network device, characterized in that it consists of a single layer or a multi-layer of fully connected (fully connected).
According to claim 3,

The output layer,

A spiked neural network device characterized in that if there is labeled data among the grouped data, the spike signal is counted and mapped to the corresponding labeling group to be output.
According to claim 1,

The spiking neural network unit,

Spiking neural network device characterized in that the weight value of the synapse is updated by using a time difference between a spike signal emitted by a pre neuron and a spike signal emitted by a post neuron.
The method of claim 6,

The spiking neural network unit,

If the difference between the time at which the pre-neuron sends a spike signal and the time at which the post-neuron sends a spike signal is greater than 0, the weight is increased by LTP (Long Term Potentiation), and when the difference is less than 0, LTD (Long Term Digression) Spiking neural network device, characterized in that.
A communication unit performing communication with an external device;

A sensor unit measuring at least one sensing data among environment information, image information, detection information, and recognition information using a plurality of sensors; And

It receives data received from the communication unit and sensing data measured from the sensor unit in multiple channels, and learns and calculates processing using a spiking neural network device based on neuromorphic technology based on the received multi-channel data. Includes a control unit for performing;

The spiking neural network device,

A pre-processor for receiving the multi-channel data, filtering and normalizing the received data into multi-dimensional data, and extracting feature points of the converted multi-dimensional data; And

Designed to generate a spike signal according to signal strength by allocating one neuron for each extracted multidimensional data of the feature points, connecting each neuron, allocating one synapse between the connected neurons, and the spike signal among the synapses A spiked neural network unit that compares the sum value of the generated synapse value and a preset threshold value, sends out the spike signal, groups the spike signals, classifies them, and clusters them;

Intelligent device comprising a.