WO2023101255A1

WO2023101255A1 - Neuromorphic device and method for compensating for variation characteristics in neuromorphic device

Info

Publication number: WO2023101255A1
Application number: PCT/KR2022/017718
Authority: WO
Inventors: 박병국; 전보성; 박경철
Original assignee: 서울대학교산학협력단
Priority date: 2021-11-30
Filing date: 2022-11-11
Publication date: 2023-06-08
Also published as: KR102514650B1

Abstract

A method for compensating for variation characteristics in a neuromorphic device according to an aspect of the present invention, includes the steps of: providing a neuromorphic device including a synapse array and a neuron circuit coupled to the synapse array; comparing the number of input spikes input to the synapse array with the number of output spikes output from the neuron circuit according to an input of the input spike; and adjusting a weight of the synapse array according to a comparison result between the number of input spikes and the number of output spikes.

Description

Neuromorphic device and variation characteristic compensation method in neuromorphic device

The present invention relates to a neuromorphic device and a method for compensating for variation characteristics in a neuromorphic device.

Recently, along with the development of computing technology based on artificial neural networks, research and development on hardware-based neural networks are also being actively conducted.

Neural networks and Spiking Neural Networks (SNNs), which are currently widely studied, originated from imitation of actual biological nervous systems (concepts of memory, learning, and reasoning), but only adopt a similar network structure, signal transmission and It is different from the actual biological nervous system in various aspects, such as information expression method and learning method.

On the other hand, hardware-based SNNs that operate almost the same as real neural networks are rarely used in actual industries because a learning method that outperforms existing neural networks has not yet been developed. However, if synaptic weights are derived using an existing neural network and inference is made using the SNN method, a high-accuracy and ultra-low-power computing system can be implemented, and research on this is being actively conducted.

When the synapse weights learned through the neural network are transcribed into the hardware-based neural network and inferred, almost the same level of performance as the existing neural network can be achieved. However, when implemented in hardware, there may be distribution of characteristics of elements, and variations in characteristics that may occur in the configuration of such elements and circuits inevitably cause a great decrease in inference accuracy.

In order to solve such problems of the prior art, the present invention proposes a new method that can operate in consideration of the variation characteristics of a hardware-based neural network (hereinafter referred to as a neuromorphic device).

An object of the present invention is to provide a neuromorphic device and a method for compensating for a shift characteristic in a neuromorphic device capable of compensating for a shift characteristic of a device through weight adjustment.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for solving the above-described technical problem, a neuromorphic device according to an aspect of the present invention includes a synapse array; a neuronal circuit coupled to the synaptic array; a weight control unit for adjusting the weights stored in the synapse array; and a control unit controlling operations of the synapse array, the neuron circuit, and the weight control unit. At this time, the control unit compares the number of input spikes input to the synapse array with the number of output spikes output from the neuron circuit according to the input of the input spikes, and based on the comparison result of the number of input spikes and the number of output spikes. Accordingly, the weight of the synaptic array is adjusted through the weight control unit.

In addition, a method for compensating for variation characteristics in a neuromorphic device according to another aspect of the present invention includes providing a neuromorphic device including a synapse array and a neuron circuit coupled thereto; comparing the number of input spikes input to the synapse array with the number of output spikes output from the neuron circuit according to the input of the input spike; and adjusting a weight of the synapse array according to a result of comparing the number of input spikes with the number of output spikes.

According to the above-described problem solving means of the present application, in a hardware-based neural network, the inference accuracy deterioration problem that occurs when variation characteristics that can occur in the signal integrator, threshold, or output waveform of a neuron circuit is expressed is solved, and such variation characteristics can be compensated for. In particular, it is possible to adjust the state of a synapse connected between two neurons in an adjacent hidden layer to compensate for variation characteristics, and form weights so that correct weights can be expressed through verification according to inputs and outputs of neurons.

Since the configuration of the present invention can be generally used in all neuron circuits, it can be applied to all systems such as a sensor as well as a spiking neural network and a neural network using a PWM method.

1 is a block diagram showing the configuration of a neuromorphic device according to an embodiment of the present invention.

2 is a flowchart illustrating a method for compensating for variation characteristics in a neuromorphic device according to an embodiment of the present invention.

3 illustrates the configuration of a neuromorphic device according to an embodiment of the present invention.

4 illustrates the concept of comparing the number of spikes according to an embodiment of the present invention.

5 is a diagram for explaining a method for compensating for variation characteristics of a neuromorphic device according to an embodiment of the present invention.

6 illustrates another example to which the method for compensating for variation characteristics according to an embodiment of the present invention may be applied.

7 is a diagram for explaining an effect of a variation characteristic compensation method according to an embodiment of the present invention.

Hereinafter, embodiments of the present application will be described in detail so that those skilled in the art can easily practice with reference to the accompanying drawings. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. And in order to clearly describe the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the present specification, when a part is said to be “connected” to another part, this includes not only the case of being “directly connected” but also the case of being “electrically connected” with another element interposed therebetween. do.

Throughout the present specification, when a member is said to be located “on” another member, this includes not only a case where a member is in contact with another member, but also a case where another member exists between the two members.

The neuromorphic device of the present invention is manufactured to mimic the human brain in hardware using a semiconductor process, and includes a synaptic element corresponding to a synapse of the brain, a neuron circuit corresponding to a neuron, and various peripheral circuits. it means.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

As shown, the neuromorphic device 100 may include a synapse array 110, a neuron circuit 120, a control unit 130, and a weight control unit 140.

The synapse array 110 is implemented to exert the same function as the synapse in the brain, and is typically implemented based on a non-volatile memory device. The synaptic array 110 corresponds to a plurality of synaptic cells, and stores predetermined weights, respectively. For example, the synapse array 110 may include synaptic cells corresponding to the product of the number of front-end neuron circuits coupled to the synapse array 110 and the number of back-end neuron circuits. An operation of storing the weights in the synapse array 110 or a process of reading the stored weights is performed through the same principle as a program operation or a read operation performed in a general non-volatile memory device. Here, the weight means a weight that is multiplied to an input signal in a perceptron structure representing an artificial neural network model, and is additionally defined as a concept including a bias, a special weight having an input of 1.

The neuron circuit 120 may be divided into a front-end neuron circuit or pre-neuron circuit coupled to the front end of the synapse array 110 and a rear-end neuron circuit or post-neuron circuit coupled to the rear end of the synapse array 110 . A typical neuron circuit 120 includes a signal integration unit for integrating a signal transmitted through the immediately preceding synapse, and a comparator for comparing whether or not the integrated signal is greater than or equal to a threshold value, and as a result of the comparison, the threshold value is greater than or equal to When it is, it is configured to output a spike signal according to an ignition operation. In addition, a counter for calculating the number of spike signals may be connected to each neuron circuit 120 . Meanwhile, in relation to the configuration of the signal integrator, an embodiment in which a signal is integrated using a capacitor is generally known.

As such, the synapse array 110 or the neuron circuit 120 has recently undergone evolution in various forms, and since it is basically produced based on a semiconductor process, variation occurs due to the scattering characteristics of the semiconductor device.

In particular, in the case of the neuron circuit 120, since the characteristics of each neuron circuit, such as the value of the capacitor or related circuit element included in the signal integration unit, the threshold value of the comparator, and the output waveform output during firing, are inconsistent, it is necessary to compensate for this. there is. In the present invention, it is intended to compensate for the characteristics due to dispersion occurring in the synapse array or neuron circuit manufactured by the semiconductor process, or various characteristic changes that change over time, and in the present invention, this is a variation characteristic defined as reward.

On the other hand, the neuromorphic device 100 of the present invention has a state in which the neuron circuit 120 is integrated around the synapse array 110, as shown in FIG. 3 to be described later.

The weight adjuster 140 is a peripheral circuit of the synapse array 110 and performs operations for programming weights for the synapse array 110 and reading stored weights. In addition, the weight control unit 140 performs a weight formation operation of newly recording weights in the synapse array and an operation of updating the recorded weights. define. To this end, the weight control unit 140 may include various voltage supply modules for performing an operation such as ISPP (Incremental Step Pulse Program) or ISPE (Incremental Step Pulse Erase) with respect to the synapse array 110. In addition, the weight control unit 140 is configured to perform a program operation of weights or an operation of erasing weights suitable for device characteristics of the synapse array 110 .

The control unit 130 controls the operation of the above synapse array 110, the neuron circuit 120 and the weight control unit 140. In particular, in the present invention, the controller 130 compares the number of input spikes input to the synapse array 110 and the number of output spikes output from the neuron circuit 120 according to the input of the input spikes, and compares the number of input spikes with the number of spikes. According to the comparison result of the number of output spikes, the weight of the synapse array 110 is adjusted through the weight control unit 140 . The controller 130 may collect information on the number of input spikes or the number of output spikes through a counter connected to each neuron circuit 120 .

2 is a flowchart illustrating a method for compensating for variation characteristics in a neuromorphic device according to an embodiment of the present invention, and FIG. 3 illustrates a configuration of a neuromorphic device according to an embodiment of the present invention. 4 illustrates the concept of comparing the number of spikes according to an embodiment of the present invention.

First, a neuromorphic device 100 including a synapse array 110 and a neuron circuit 120 coupled thereto is provided. As shown in FIG. 3, the neuromorphic device of the present invention has a state in which the neuron circuit 120 is coupled together around the synapse array 110. Conventionally, an operation of sensing weights stored in a synapse or recording weights was performed using a synapse array alone. However, in the present invention, a structure in which the synapse array 110 and the neuron circuit 120 are combined is assumed in order to reflect and compensate for all of the mutation characteristics of the synapse array 110 and the neuron circuit 120.

Next, the step of obtaining the number of input spikes input to the synapse array 110 and the number of output spikes output from the neuron circuit 120 according to the input of the input spikes (S210) and comparing them (S220) is performed. do.

As shown in FIG. 4, the number of input spikes (N _in ) is calculated based on the output of the front end neuron circuit coupled to the front end of the synapse array 110, and each spike fired in the front end neuron circuit is a synapse array ( 110) is entered. Then, the number of output spikes (N _out ) is calculated based on the output of the neuron circuit at the rear end coupled to the rear end of the synapse array 110 . For example, when the output generated from the first neuron at the front end of the synapse array 110 is applied in parallel to neurons 1 to 4 at the rear end of the synapse array 110, the number of outputs of the first neuron and the number of outputs of each neuron at the rear end The weight of each synaptic cell can be changed to the target weight by counting the number of outputs. That is, the number of spikes (Nin) of the first neuron of the front end and the number of spikes of the number 1 neuron of the rear end are compared, and the weight of the synaptic cell connected to the number 1 neuron of the front end and the number 1 neuron of the rear end is determined according to the comparison result. Adjust. In this way, the synaptic array includes each synaptic cell connected to a front-end neuron and each back-end neuron, and in the present invention, parallel processing of individually adjusting weights is performed for each synapse cell.

In this way, in the comparing step (S220), the number of input spikes detected by the front-end neuron circuit of the synapse array is compared with the number of output spikes detected by each of the output spikes connected to the front-end neuron circuit in parallel.

Also, comparing the number of input spikes with the number of output spikes ( S220 ) may include calculating an effective weight value obtained by dividing the number of output spikes by the number of input spikes and comparing it with a target weight value. At this time, the effective weight is calculated for each synaptic cell. An effective weight for each synaptic cell is calculated by dividing the number of output spikes output from each trailing neuron circuit by the number of input spikes output from the preceding neuron circuit associated with the corresponding trailing neuron circuit.

Next, according to the comparison result of the number of input spikes and the number of output spikes, weights of synaptic cells included in the synapse array are adjusted (S230).

When the threshold value is 1, the number of input spikes output from the front-end neuron is multiplied by the weight stored in each synapse cell, and this value becomes equal to the number of output spikes output from each back-end neuron. The effective weight is calculated under this premise, and if the effective weight is equal to the target weight or within the allowable error range, it is judged that the effective weight is maintained at an appropriate level.

However, when the effective weight is greater than the target weight by an allowable error or more, that is, when the number of output spikes is counted as greater, the weight of the synapse array 110 is updated in a decreasing direction.

In addition, when the effective weight is smaller than the target weight by an allowable error or more, that is, when the number of output spikes is counted as being smaller, the weights of the synapse array 110 are updated in the direction of increasing them.

At this time, in the step of adjusting the weight, in addition to the operation of updating the weight in the direction of increasing or decreasing the weight, the weight is newly set for the synapse (for example, when the weight is 0) for which no weight has been set. action can be performed.

At this time, based on the difference between the effective weight and the target weight, the amount of increase or decrease of the weight of the synapse array 110 is adjusted. The increase or decrease of the weight can be obtained by setting the weight value to increase or decrease by the difference between the effective weight for the target weight and the current weight value.

Meanwhile, as described above, in order to adjust the weight of the synapse array 110, logic such as ISPP or ISPE may be performed through the weight control unit 140.

In this way, as the weight of each synaptic cell included in the synaptic array is collectively adjusted based on the number of input spikes and the number of output spikes, the variation characteristics of the elements of the synaptic array 110 or the neuron circuit 120 are all With the reflected state, it becomes possible to perform inference.

Meanwhile, the above-described weights have been described on the premise that their values are greater than or equal to 0, but the present invention can also adjust weights having negative values. Negative weights can be based on synapses with positive weights that are connected to the same neuron. After synapses with positive values are formed in this way, negative synapses can be operated in the same way, and effective weights can be calculated conversely through differences in the number of output spikes. According to the effective weight due to dispersion or variation characteristics, the correct negative weight value can be formed using the method proposed above.

In the left and right figures, a case in which the weight value is determined based on the synaptic current (square mark) and a case in which the weight value is determined based on the firing rate (circle mark) are shown.

First, as shown on the left, when the weight is simply determined based on the synaptic current (represented by a square), the signal (charge amount) transmitted to the posterior neuron changes when the width of the spike generated in the anterior neuron changes. As shown in, the effect of substantially increasing the weighted input results.

In contrast, when the synaptic weight is adjusted based on the firing rate proposed by the present invention (marked in a circle), when the width of the front end output waveform increases, the synaptic weight (current) is reduced and the effective weight It is possible to maintain an ideal state in which is constant.

This is equally applicable to the variation of the signal integral component of the posterior neuron and the threshold value. If the effective weight is set to be equal to or sufficiently close to the target weight, even if the signal integral component or threshold value of a particular posterior neuron deviates from the ideal value. It can compensate for the variation characteristics and accurately implement the actual weighted input.

The method of the present invention analyzes the relationship between front and rear neurons and synapses in various network structures such as a Fully-Connected Network (FCN) as well as a Convolutional Neural Network (CNN) as shown in FIG. Through this, weights can be formed to compensate for the variation characteristics. In the present invention, the method for forming variance compensation weights based on utterance rates expresses accurate weights and contributes to the improvement of SNN inference accuracy.

The graph shown in FIG. 7 shows the inference accuracy for the MNIST dataset that occurs when the membrane capacitance, one of the signal integration components of neurons, and the threshold value are not uniform in a spiking neural network having a structure of 1200FC-1200FC-10. It shows the recovery pattern of inference accuracy after compensation for reduction and variation characteristics. It was confirmed that the accuracy was almost perfectly restored to the ideal accuracy in all cases of compensation indicated by the black solid line.

An embodiment of the present invention may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

The above description of the present application is for illustrative purposes, and those skilled in the art will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present application.

Claims

In the neuromorphic device,

synapse array;

a neuronal circuit coupled to the synaptic array;

a weight control unit for adjusting the weights stored in the synapse array;

Including a control unit for controlling the operation of the synapse array, the neuron circuit and the weight control unit,

The control unit compares the number of input spikes input to the synapse array with the number of output spikes output from the neuron circuit according to the input of the input spikes, and according to the comparison result of the number of input spikes and the number of output spikes. , To adjust the weight of the synapse array through the weight control unit, the neuromorphic device.
According to claim 1,

Wherein the control unit compares an effective weight, which is a value obtained by dividing the number of output spikes by the number of input spikes, with a target weight.
According to claim 2,

The control unit parallelly compares the number of input spikes detected in the front-end neuron circuit of the synapse array with the number of output spikes detected in each of the next-stage neuron circuits associated with the front-end neuron circuit, and according to the comparison result , To adjust the weight of each synaptic cell associated with the front-end neuron circuit and the back-end neuron circuit, the neuromorphic device.
A method for compensating for variation characteristics in a neuromorphic device,

providing a neuromorphic device including a synaptic array and a neuronal circuit coupled thereto;

comparing the number of input spikes input to the synapse array with the number of output spikes output from the neuron circuit according to the input of the input spike; and

Adjusting the weight of the synapse array according to a result of comparing the number of input spikes with the number of output spikes.
According to claim 4,

Wherein the comparing step includes comparing an effective weight value obtained by dividing the number of output spikes by the number of input spikes with a target weight value.
According to claim 4,

The comparing step is to compare in parallel the number of input spikes detected in the front-end neuron circuit of the synapse array with the number of output spikes detected in each back-end neuron circuit associated with the front-end neuron circuit,

Adjusting the weight of the synapse array

According to the comparison result, a weight of each synaptic cell associated with the front-end neuron circuit and the back-end neuron circuit is adjusted.