KR20200094354A - Method for generating spiking neural network based on burst spikes and inference apparatus based on spiking neural network - Google Patents

Abstract

A spike-based spiking neural network generation method comprises the following steps of: mapping, by a computer device, the weights of a trained artificial neural network model to a spiking neural network (SNN); and optimizing, by the computer device, the SNN to which the weights are mapped. In the optimizing step, a threshold value for determining firing is adjusted according to the state of neurons of the SNN.

Description

METHOD FOR GENERATING SPIKING NEURAL NETWORK BASED ON BURST SPIKES AND INFERENCE APPARATUS BASED ON SPIKING NEURAL NETWORK}

The technique described below relates to a method of building a spiking neural network model.

Recently, the AI (Artificial Intelligence) field is drawing attention as a next-generation promising technology. In particular, deep learning has gained a lot of attention as it has made great achievements in areas such as image recognition. Most of the deep learning learning and inference process consists of a number of CPUs (Central Processing Units) and GPUs, and is performed in a large-scale server computing environment with a lot of computing resources and power. The large amount of computing resources and power consumption demands are obstacles to applying the deep learning model to mobile and embedded environments. In order to solve this problem, various studies have been conducted on a method of quantizing and pruning, which is a method of applying a mobile environment by reducing a deep learning model.

Furthermore, a neuromorphic chip has been developed that consists of a SNN (spiking neural network) and can operate at low power. SNN fires only when the membrane potential of the neuron is higher than the threshold voltage, and it is an event-based because it transmits information between synapses through ignited spikes. event-driven), enabling low-power operation compared to other artificial neural networks.

United States Patent Publication US 2015-0235124

SNN does not have many applications. This is because learning SNN is difficult. In SNN, neurons and synapses cannot be differentiated, so it is impossible to learn SNN using gradient descent and error back-propagation. The most widely known SNN learning method is STDP (Spiking Timing Dependent Plasticity). However, studies on deep SNN using STDP have not been conducted. There has been a study that applied a learning method for DNN to SNN.

The technique described below is intended to provide a method of learning the SNN by applying the learning result of the DNN to the SNN. The technique described below is intended to provide a method for optimizing SNN to which DNN learning results are applied. The technique described below is intended to provide a neural coding technique to which a burst spike method is applied.

The burst spike-based spiking neural network generating method includes mapping a weight of an artificial neural network model trained by a computer device to a spiking neural network (SNN), and the computer device optimizing the SNN to which the weight is mapped. The optimizing step adjusts a threshold value for determining utterance according to the neuron state of the SNN.

The spiking neural network-based reasoning apparatus adjusts a threshold value for the neuron according to a reference value for an input device receiving input data, and a neuron whose state changes according to an input value, and adjusts a weight for the neuron according to the reference value It includes a storage device for storing a spiking neural network (SNN) using neural coding, and a computing device for performing inference by inputting the input data to the SNN.

The technique described below constructs an SNN to which a burst spike based neural coding is applied. Therefore, the technique described below can provide a neural network model capable of fast and accurate reasoning compared to the conventional technique while maintaining the advantages of SNN.

1 is an example of an SNN learning method using DNN.
Fig. 2 is an example of the coding rate.
3 is an example of burst spiking.
4 is an example of a probability that burst spiking occurs according to a threshold value.
5 is an example of an SNN learning process using burst spikes.
6 is an example of an SNN optimization process based on burst spikes.
7 is an example of an inference device based on the SNN model.

The technique described below may be applied to various changes and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the techniques described below to specific embodiments, and should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the techniques described below.

Terms such as first, second, A, B, etc. can be used to describe various components, but the components are not limited by the above terms, and only for distinguishing one component from other components Used only. For example, the first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the technology described below. The term and/or includes a combination of a plurality of related described items or any one of a plurality of related described items.

In the terminology used herein, a singular expression should be understood to include a plurality of expressions unless the context clearly interprets otherwise, and terms such as “comprises” describe features, numbers, steps, actions, and components described. It is to be understood that it means that a part or a combination thereof is present, and does not exclude the presence or addition possibility of one or more other features or numbers, step operation components, parts or combinations thereof.

Prior to the detailed description of the drawings, it is intended to clarify that the division of components in this specification is only divided by the main functions of each component. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each subdivided function. In addition, each of the components to be described below may additionally perform some or all of the functions in charge of other components in addition to the main functions in charge thereof, and some of the main functions in charge of each of the components are different. Needless to say, it may also be carried out in a dedicated manner.

In addition, in performing the method or the method of operation, each process constituting the method may occur differently from the specified order unless a specific order is explicitly stated in the context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The techniques described below relate to techniques for building artificial neural networks. It will be described below that the computer device learns and builds an artificial neural network. The computer device may provide a certain service using the learned artificial neural network. The computer device means a device that processes a certain operation according to a certain program or source code. Computer devices include servers, personal PCs, smart devices or chipsets dedicated to image processing. Depending on the application field, the computer device may be an image processing device.

First, SNN will be briefly described.

The SNN consists of neurons and synapses that generate spike trains. Neurons transmit information to other neurons through a series of spikes, the spike train. Among various neural models, the IF (Integrate-and-Fire) model is the most widely used. The IF neural model incorporates post-synaptic potential (PSP) into the membrane potential (V _mem ). PSP integrated in the j-th neuron in the l-th layer at time t may be expressed as Equation 1 below. In the following description, the same parameter or symbol is used with the same meaning in the equation.

는 PSP의 합이다.

는 l-1번째 계층에 i번째 뉴런과 l번째 계층에 있는 j 번째 뉴런 사이의 시냅스 가중치이다.

는 l-1번째 계층에 i번째 뉴런에 있는 스파이크이다.

는 편향(bias)이다.In Equation 1

Is the sum of the PSP.

Is the synaptic weight between the i-th neuron in the l-1th layer and the j-th neuron in the l-th layer.

Is the spike in the i-th neuron in the l-1th layer.

Is bias.

를 초과하면, 뉴런은 스파이크를 생성한다. 시간 t에서 l번째 계층에 있는 i 번째 뉴런의 스파이크 생성 과정은 아래의 수학식 2와 같이 표현될 수 있다.Threshold with constant membrane potential

When exceeded, the neurons produce spikes. The spike generation process of the i-th neuron in the l-th layer at time t may be expressed as Equation 2 below.

는 스파이크 출력을 표현하는 단위 계단 함수(unit step function)이다. z는 PSP의 합계이고, V_th는 임계값이다. In Equation 2

Is a unit step function representing the spike output. z is the sum of PSP and V _th is the threshold.

If the IF neuron fires the spike, the membrane potential is set to the resting potential (V _rest ) and maintains the previous potential only before the spike fires. The dynamic change of the membrane potential can be expressed as Equation 3 below.

SNN transmits information through a spike train. SNN is an energy efficient model because information is transmitted only when an event called a spike occurs.

SNN learning

Describes the learning method for SNN. SNNs learned by STDP or error back propagation do not have the same performance as DNNs. Recently, SNN learning method using DNN has been studied.

1 is an example of an SNN learning method 100 using DNN. The computer device constructs a DNN corresponding to the SNN, and first learns the DNN (110). DNN having the same (or similar) structure to the SNN is trained by the error back propagation method. The computer device applies the weight of the learned DNN to the SNN (120). The computer device applies a weight for each layer node (neuron) of the DNN to each layer node of the corresponding SNN. That is, it can be said that the computer device maps the corresponding weight in the DNN to the SNN. By performing this process, a learned SNN is prepared. Thereafter, the computer device may make certain inferences (130) using the learned SNN and provide related services.

SNN is structurally different from DNN. Therefore, if the learning result of the DNN is applied to the SNN as it is, some performance decreases. There have been various studies to solve this. For example, there have been studies in which weights are adjusted or normalized according to SNN. Various models for weight control have been studied.

A recent study proposed a new normalization model. The new normalization model implements reset-by-subtraction in a subtractive way to prevent performance degradation due to the initialization of the membrane potential. In this case, the rest potential V _rest may be expressed as Equation 4 below.

Also, in order to reduce information loss between neurons, the sum of PSPs can be expressed as Equation 5 below. Equation 5 represents the integrated PSP in the j-th neuron in the l-th layer at time t, as in Equation 1.

In the case of using such a model, the performance was similar to that of DNN for MNIST and CIFAR-10 data sets. The technique described below is also assumed to use a learning method that applies the learning result of DNN to SNN.

Neural coding

Neural coding is an encoding scheme that transmits information from one neuron to another. Various types of neural coding have been studied.

Typically, rate coding is used. Fig. 2 is an example of the coding rate. In FIG. 2, region A represents a spike train, region B represents a PSP (post-synaptic potential), and region C represents an inter-spike interval histogram (ISIH). ISIH is a histogram of the spacing between spikes. Flash rate coding is based on spike firing rate. Flash rate coding is determined by the number of spikes occurring over a period of time. Therefore, coding of the speech rate is inefficient in terms of transmission capacity and transmission speed of information. For example, 256 (=2 ⁸ ) operations are required to transmit 8-bit information. Referring to FIG. 2, it can be seen that the PSP having the same size is induced for each spike. Eventually, applying the ignition rate coding to the deep SNN brings a lot of latency and high energy consumption.

3 is an example of burst spiking. Burst spiking has a wide variety of spike patterns. The spike pattern may include spike groups having a short inter-spike interval (ISI). Burst spiking can carry more information than a typical spike train. For example, as shown in FIG. 3C, burst spiking may induce a membrane potential that increases information transmission efficiency. If such burst spiking can be applied to a deep SNN, it will be possible to reduce inference time and improve performance.

The technique described below applies a burst spike to a deep SNN to provide a model capable of efficient and fast reasoning. This is to apply the burst spike, which is known to be efficient for information transmission in the conventional biological field, to artificial intelligence models.

In order to functionally implement burst spiking, a burst function as in Equation 6 below may be defined.

는 시간 t에서 l번째 계층에 있는 뉴런 i에 대한 버스트 함수이다. β는 버스트 상수(burst constant)이다.

는 수학식 2에서 설명한 바와 같이 시간 t-1에서 생성된 스파이크를 의미한다.

Is the burst function for neuron i in the l-th layer at time t. β is a burst constant.

Is a spike generated at time t-1 as described in equation (2).

To obtain an efficient burst spike, when a burst function is applied to Equation 5 described above, a threshold may be expressed as Equation 7 below.

는 시간 t에서 l번째 계층에 i번째 뉴런에 대한 임계값이다.

는 뉴런에 설정된 초기 임계값이다.

Is the threshold for the i-th neuron in the l-th layer at time t.

Is the initial threshold set in the neuron.

From Equation 5 and Equation 7, a weight may be determined as shown in Equation 8 below. Equation 8 below is a synaptic weight between the i th neuron in the l-1 th layer and the j th neuron in the l th layer at time t.

는 각 뉴런에 초기 설정된 가중치를 의미한다. 즉, DNN의 가중치가 매핑되어 설정된 값이다.The weight of Equation 8 can be interpreted as a synaptic potentiation based on burst spikes. Through this, SNN can efficiently transmit information. In Equation 8

Denotes a weight initially set for each neuron. That is, the weight of the DNN is mapped and set.

4 is an example of a probability that burst spiking occurs according to a threshold value. FIG. 4 shows the number of burst spikes generated according to different thresholds V _th and the length of the burst. It can be seen that the burst spike increases as the threshold decreases, and long bursts frequently appear.

Neural coding using such a burst spike is called burst coding. Burst coding can dynamically change the size of a synapse according to the state of a neuron. The amount of information to be transmitted can be determined by the cumulative film potential and the number of consecutive spikes. When the spikes occur continuously, the synapse is strengthened through the burst function, and when the spikes do not occur, the synaptic intensity is restored.

5 is an example of an SNN learning process 200 using burst spikes. The computer device constructs a DNN corresponding to the SNN, and first learns the DNN (210). DNN having the same (or similar) structure to the SNN is trained by the error back propagation method. The computer device applies the weight of the learned DNN to the SNN (220). The computer device maps the weights for each layer node (neuron) of the DNN to each layer node of the corresponding SNN. The computer device may optimize the SNN structure by applying a burst spike to the weighted SNN (230). Thereafter, the computer device may make certain inferences using the optimized SNN (240). Structural optimization 230 using burst spikes may be performed in the inference process for specific input data.

6 is an example of an SNN optimization process 300 based on burst spikes. 6 corresponds to step 230 in FIG. 5. 6 basically assumes a state in which the SNN is set as the weight of the learned DNN. For example, if the SNN is intended for image classification, a training image for the optimization process can be used. The computer device inputs a training image to the SNN, and determines whether the final result is correct while adjusting the threshold and weight of the corresponding neuron according to the state of each New Year. The accuracy can also be determined by comparing the DNN's preliminary inference result with the SNN's result. It is assumed that the SNN operates in a feed forward manner like a general DNN.

The computer device may first assign an index to each layer for optimization. The computer device initially sets 310 the hierarchical index. The computer device inputs the training image to the SNN, and each layer of the SNN receives a feed forward input from a node (neuron) of the previous layer according to the structure of the model (320). The neuron receiving the input value may change its state (status update). As described above, the neuron receives a potential (PSP) after synapse and fires when the combined value exceeds a certain threshold. Accordingly, the computer device determines whether the neuron changes state (whether or not speech is in progress) according to the current input value (330).

If the current neuron is in a state in which it is required to fire (YES in 330), the computer device generates a spike and initializes the neuron state (341). Neuronal state initialization means the neuron's membrane potential initialization. The computer device updates the threshold according to the state of the neuron (342). For example, the computer device may reduce the threshold value for the current neuron having a spike by a reference value. The degree of reducing the threshold (reference value) may vary depending on the application type of the SNN or the performance of the system on which the SNN is built. The reference value may be determined according to the burst function in Equation 6 above. In connection, the computer device may perform a synaptic connection enhancement to a current neuron (343). Threshold change 342 and synaptic connection enhancement 343 may be implemented through the burst function described above. Therefore, the current ignited neuron has a constant threshold (usually a reduced threshold) and a weight change. The specific process follows Equations 6 to 8 described above.

The computer device initiates a synaptic connection (350) if the current neuron is not in a firing state. This process is to reduce the weight to the value set in the initial learning process (the weight of the DNN is mapped) when no spike occurs. Therefore, it can be said that the synaptic enhancement in which the threshold value is changed and spikes are generated according to the burst function is activated only for a certain period (short term).

The computer device determines whether the current layer is the last layer (360). If it is not the last layer (NO in 360), the layer index is increased to move to the next layer (370). The next layer is also checked for ignition of each neuron, and if necessary, strengthens synaptic connections to adjust weights.

If the current layer is the last layer (YES in 360), the computer device determines whether the accuracy of the inference result of the SNN is greater than the reference value (380). The computer device can determine whether the accuracy is greater than the reference value by comparing the inferred result of the learned DNN. For example, if the inference result of the learned DNN and the inference result of the SNN are the same, the computer device may determine that the accuracy is greater than the reference value. Here, the reference value can be said to be a target value for optimization of the SNN. If the accuracy is less than the reference value, the computer device may repeat the above-described optimization process (NO in 380). Although not illustrated in FIG. 6, the computer device may end the optimization process if the optimization process is repeated a predetermined number of times in order not to repeat the optimization process indefinitely. Thereafter, the computer device may provide a certain reasoning and service using the optimized SNN.

Neurons are known to use different neurocoding depending on their role and location in the brain. Based on these models, a single coding method may not be used, and different coding strategies may be used according to the characteristics of the SNN layer.

The SNN can be divided into an input layer and a hidden layer. Taking the image classification as an example, the input layer converts the continuous values of the input image into discrete spikes. In image classification, the input value has a limited and static value. For example, phase coding may be appropriate for the input layer. Phase coding is a type of dynamic coding. Meanwhile, burst coding can dynamically determine a transmission rate. Therefore, burst coding may be more appropriate for the hidden layer than the input layer. According to the use of SNN, different types of neural coding techniques can bring higher efficiency. In the sense of using multiple neural coding techniques, this can be referred to as hybrid neurocoding.

7 is an example of an inference device 400 based on the SNN model. The SNN-based reasoning device 400 may be a device that performs the SNN optimization process described in FIG. 6. In addition, the SNN-based reasoning device may be a device that provides a certain reasoning service using the constructed SNN. For example, the SNN-based reasoning device 400 may be a device that performs classification on an image. The SNN-based reasoning device 400 may be physically implemented in various forms. For example, the SNN-based reasoning apparatus 400 may take the form of a smart device, a PC, an IoT device, a server of a network, a chipset dedicated to image processing, and the like.

The SNN-based reasoning device 400 includes a storage device 410, a memory 420, a computing device 430, an interface device 440, and a communication device 450.

The storage device 410 stores programs or source codes for SNN learning or optimization. The storage device 410 may store a training image for learning. The storage device 410 may store an initial SNN model and a DNN model, which are learning targets. The storage device 410 may store the trained DNN model and SNN model. Also, the storage device 410 may store a program or source code for an inference method or service using the learned SNN.

The memory 420 may store data, parameters, image information, etc. generated by the SNN-based reasoning device 400 in learning or optimizing the SNN. Also, the memory 420 may store data generated in the inference process using the SNN.

The interface device 440 is a device that receives certain commands and data from the outside. The interface device 440 may receive a training image or an input image from a physically connected input device or an external storage device. The interface device 440 may receive DNN models, SNN models, and various programs or commands for image processing.

The communication device 450 refers to a configuration that receives and transmits certain information through a wired or wireless network. The communication device 450 may receive a training image or an input image from an external object. The communication device 450 may receive an SNN model, a DNN model, an SNN learning program, and an SNN-based application service providing program. Furthermore, the communication device 450 may transmit the inference result using the learned SNN to an external object.

The communication device 450 to the interface device 440 is a device that receives certain data or commands from the outside. The communication device 450 to the interface device 440 may be referred to as an input device.

The computing device 430 may learn the SNN using a program stored in the storage device 410. Also, the computing device 430 may optimize the SNN using a program stored in the storage device 410. The computing device 430 may make certain inferences using the SNN finally provided. The computing device 430 may be a device such as a processor embedded in a processor, an AP, or a program that processes data and processes a certain operation.

Further, the SNN learning method, the learned SNN generation method, and the learned SNN operation method as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided in a non-transitory computer readable medium.

The non-transitory readable medium means a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. Specifically, the various applications or programs described above may be stored and provided in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

The drawings attached to the present embodiment and the present specification merely show a part of the technical spirit included in the above-described technology, and are easily understood by those skilled in the art within the scope of the technical spirit included in the above-described technical specification and drawings. It will be apparent that all of the examples and specific examples that can be inferred are included in the scope of the above-described technology.

400: SNN-based reasoning device
410: storage device
420: memory
430: computing device
440: interface device
450: communication device

Claims (13)

Mapping, by the computer device, the weight of the trained artificial neural network model to a spiking neural network (SNN); And
The computer device comprises optimizing the SNN to which the weight is mapped,
The step of optimizing is a burst spike-based spiking neural network generating method that adjusts a threshold for determining utterance according to the neuron state of the SNN. According to claim 1,
The step of optimizing
Inputting input data for optimization into the SNN;
Generating a spike when a neuron receiving an input value for each neuron of the SNN is a target neuron in a state to ignite;
Adjusting a threshold value for the target neuron; And
Burst spike-based spiking neural network generating method comprising the step of adjusting the weight for the target neuron. According to claim 1,
The SNN uses burst coding for information transmission, and the burst coding is a burst spike-based spiking neural network generating method that performs burst spiking according to a burst function represented by the following equation.

(

Is a burst function for neurons in the l-th layer at time t, β is a constant for bursts,

Is the spike produced at time t-1) According to claim 3,
A burst spike-based spiking neural network generating method in which the threshold value for a neuron to which the burst function is applied is determined by the following equation.

(

Is the threshold for the i-th neuron in the l-th layer at time t,

Is the threshold previously set for the neuron) According to claim 3,
A method for generating a spiking neural network based on burst spikes, wherein the weight of the neuron to which the burst function is applied is determined by the following equation.

(

Is the synaptic weight between the i-th neuron in the l-1th layer and the j-th neuron in the l-th layer at time t,

Is the weight previously set for the neuron) According to claim 1,
The SNN uses neural coding for information transmission, but the neural coding lowers a threshold value for the specific neuron by a reference value according to a reference value when a specific neuron utters according to an input value, and the specific neuron according to the reference value A method for generating a spiking neural network based on a burst spike that adjusts the weight for the. According to claim 1,
The computer device is a burst spike-based spiking neural network generating method that repeats the optimization process until the accuracy of the SNN exceeds a reference value. An input device that receives input data;
A spiking neural network (SNN) using neural coding for adjusting a threshold value for the neuron according to a reference value and a weight for the neuron according to the reference value for a neuron whose state changes according to an input value is stored. Storage device; And
Spiking neural network-based reasoning apparatus including a computing device that performs inference by inputting the input data to the SNN. The method of claim 8,
The SNN is a spiking neural network based reasoning apparatus provided through a process in which the weight of the initial SNN is optimized by using training data for optimization of the initial SNN to which the weight of the DNN learned as the training data is applied. The method of claim 8,
The SNN uses burst coding for information transmission, wherein the burst coding is a spiking neural network based reasoning apparatus performing burst spiking according to a burst function represented by the following equation.

(

Is a burst function for neurons in the l-th layer at time t, β is a constant for bursts,

Is the spike produced at time t-1) The method of claim 10,
Spiking neural network-based reasoning apparatus that the threshold value for a neuron to which the burst function is applied is determined by the following equation.

(

Is the threshold for the i-th neuron in the l-th layer at time t,

Is the threshold previously set for the neuron) The method of claim 10,
A spiking neural network based reasoning device in which the weight of the neuron to which the burst function is applied is determined by the following equation.

(

Is the synaptic weight between the i-th neuron in the l-1th layer and the j-th neuron in the l-th layer at time t,

Is the weight previously set for the neuron) A computer-readable recording medium recording a program for executing the burst spike-based spiking neural network method according to any one of claims 1 to 7 in a computer.

Images