KR102176695B1 - Neural network hardware - Google Patents

Abstract

The present embodiment is a neural network hardware that performs on-chip learning of a neural network. The neural network hardware performs an on-chip learning process to apply a random learning parameter update method when updating a learning parameter.

Description

Neural network hardware {NEURAL NETWORK HARDWARE}

The technology to be described below relates to a neural network hardware acceleration device.

In order to overcome the structural limitations of chips based on the existing von Neumann structure, IC chip developers have developed a neural network based on neurons, which are the basic units that make up the human brain, and synapses that connect these neurons. I have been developing neural network hardware or neuromorphic hardware. Neural networks have overcome the limitations of existing machine learning algorithms and show human-like image, image, pattern learning and cognitive abilities, and are already used in many fields. Numerous companies and researchers have been developing dedicated ASIC chips to perform these neural networks' computation tasks faster with lower power.

In the human brain, neurons connect to form a network, in the process of learning, reasoning, and thinking. Neural networks are formed by imitating their learning activities. Neurons and synapses may be components of a neural network, and synapses are parts that connect neurons and neurons, and the strength and weakness of synaptic connections between neurons is called weight.

Learning a neural network is a process of creating a network, giving data to be trained, and training until the desired data is output. As a result, the process of learning is a process of obtaining weights.

Neural network training and inference are performed using specialized hardware, because performing a learning program (and reasoning program) called a neural network performs numerous parallel operations. In other words, it can be expressed as a process of obtaining an output vector by expressing a neuron and a weighted neuron between the neurons as a weight matrix and multiplying the input vector and the weight matrix, but the number of operations to be performed is large.

The technical task to be achieved by this embodiment is the design of a control controller required to overcome the structural limitations of the existing von Neumann structure-based chips and to make a neural network accelerating chip including two or more neurons per class that operates at higher speed and lower power. Implementation.

One of the technical challenges to be achieved with this example is in the neural network acceleration hardware, when the configuration of the label neuron part is composed of two or more label neurons per class, the neurons of the label neuron part are expressed according to a signal given from the outside. After that, by summing the number of expressed neurons for each class, the class in which the most neurons are expressed is determined as the final inference result, or a hardware control device that presents the inference probability for each class is provided.

In addition, one of the other technical challenges to be achieved with this embodiment is that when inference is performed on a neural network based on a probability model in hardware for accelerating a single neural network based on a pseudorandom number generator, iteratively infers two or more times on the same data. When performing Ensemble inference to derive the final inference result by summing the results of each inference after performing, the seed buffer of the pseudo-random number generator is newly updated for each trial of inference to ensure higher inference accuracy. It is to design and implement a changeable control device.

In addition, one of the other technical challenges to be achieved with this embodiment is to determine whether to apply a random weight update method according to an external signal when updating weights and biases in the learning process in neural network acceleration hardware capable of on-chip learning. When applied, a control device necessary to update weights and biases according to a random weight update method is implemented.

The neural network hardware according to this embodiment: has a plurality of classes, and includes a label neuron unit having two or more label neurons per class, and the expression results of the label neurons are performed multiple times through the neural network. Is selected as a result.

According to the hardware accelerator according to the present embodiment, it is possible to control the hardware that calculates a neural network structure having two or more neurons per class that guarantees higher inference accuracy compared to a conventional neural network structure having one neuron per class. The advantage of being able to implement a capable device is provided.

In addition, according to the hardware accelerator according to the present embodiment, when performing an ensemble inference method that physically aggregates the results of other hardware devices and performs inference through hardware based on a pseudorandom number generator, the seed of the pseudorandom number generator A hardware control device that guarantees higher inference accuracy by changing the buffer to a different value for each trial, two or more trials, or random trials to obtain results similar to those inferred by different hardware. The advantage of being able to implement is provided.

In addition, according to the hardware accelerator according to the present embodiment, when a random number value is sampled and the random number satisfies a predetermined condition, the calculated correction value is applied when updating the weight and bias values, and when the condition is not satisfied. Is a control device that guarantees higher on-chip learning inference accuracy compared to the conventional method by controlling the neural network acceleration hardware so that a random weight update method that does not apply the calculated correction value when updating the weight and bias values is applied. The advantage of being able to implement is provided.

1 is a diagram illustrating a process of inferring a label neuron part of a neural network.
2 is a diagram schematically showing a structure of an acceleration hardware according to an embodiment.
3 is a diagram showing an outline of an accumulation-expression unit according to an embodiment.
4 is a diagram showing an outline of a voting and reasoning unit.
5 is a schematic diagram for explaining the structure of an acceleration hardware according to another embodiment.
6 is a schematic diagram for explaining a voting and reasoning unit according to another embodiment.
7 is a diagram illustrating an overview of a pseudorandom number generator according to another embodiment.
8 is a diagram illustrating a structure of an acceleration hardware according to another embodiment.
9 is a diagram illustrating an outline of a weight correction unit according to another embodiment.
10 is an example of a structure of a device equipped with a neural network.

The technology to be described below may be modified in various ways and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology to be described below with respect to a specific embodiment, and it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the technology described below.

Terms such as 1st, 2nd, A, B, etc. may be used to describe various components, but the components are not limited by the above terms, only for the purpose of distinguishing one component from other components. Is only used. For example, without departing from the scope of the rights of the technology described below, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In terms of the terms used in the present specification, expressions in the singular should be understood as including plural expressions unless clearly interpreted differently in context, and terms such as "includes" are specified features, numbers, steps, actions, and components. It is to be understood that the presence or addition of one or more other features or numbers, step-acting components, parts or combinations thereof is not meant to imply the presence of, parts, or combinations thereof.

Prior to the detailed description of the drawings, it is intended to clarify that the division of the constituent parts in the present specification is merely divided by the main function that each constituent part is responsible for. That is, two or more constituent parts to be described below may be combined into one constituent part, or one constituent part may be divided into two or more according to more subdivided functions. In addition, each of the constituent units to be described below may additionally perform some or all of the functions of other constituent units in addition to its own main function, and some of the main functions of each constituent unit are different. It goes without saying that it may be performed exclusively by.

In addition, in performing the method or operation method, each of the processes constituting the method may occur differently from the specified order unless a specific order is clearly stated in the context. That is, each of the processes may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

Hereinafter, embodiments of neural network hardware including a control device will be described.

First Example

Hereinafter, an embodiment of neural network hardware will be described with reference to FIGS. 1 to 4. As an algorithm for performing inference using a learned neural network, there may be a deterministic algorithm that produces the same output while the same input is provided, and a stochastic algorithm that produces a different output each time. This embodiment is for ensemble inference using a stochastic algorithm. Ensemble reasoning is a reasoning method that stores (votes) the results of multiple times of reasoning, and the result of the final reasoning is based on the result of the vote.

Referring to FIG. 1, a neural network hardware 2 including a control unit 20 according to an embodiment of the present invention receives control signals D1 provided from outside through the control unit 20, and based on these signals. The input data buffer unit 24, the voting and inference unit 224, and the pseudorandom number generator 26 are respectively input data buffer unit control signal D2, voting and reasoning unit control signal D3, and pseudo-random number generator control signal. Control through (D4).

The external control signal D1 may include any one or more of data to be learned, data to be inferred, and one or more control signals, among which, when the hardware 2 performs ensemble inference, the number of inferences is accumulated and the final Information on whether to produce results is included.

The control unit 20 processes the control signal D1 to form an input data buffer unit control signal D2 and provides it to the input data buffer unit 24. In addition, the control unit 20 processes the control signal D1 and provides the input data to the input data buffer unit 24 together with the input data buffer unit control signal D2.

When a single hardware ensemble inference is performed, in the case of the first trial, the input data buffer unit 24 stores the data provided through the input data buffer unit control signal D2 in the input data buffer, and stores the data in the input data buffer, and the accumulation-expression unit 222 To provide.

The input data buffer unit 24 provides the value stored in the input data buffer to the accumulation-expression unit 222 without receiving data to the accumulation-expression unit 222 when performing the inference as many as the number of ensemble inferences determined after the first trial. Works to be performed.

The accumulation-expression unit 222 calculates a partial sum by calculating a product of the input data (input vector) and a part of the weight matrix, and accumulates the calculated partial sum. The accumulated values form a weighted sum.

In addition, the accumulation and expression unit 222 provides the calculated weighted sum as a factor to an activation function, and calculates the activation function to obtain an activation function. The activation value corresponds to the result of expression of the neuron. In an embodiment, the accumulation and expression unit 222 may add the calculated weighted sum and a bias value to provide an activation function as a factor. The bias can have a vector form, and each output neuron currently being calculated can have a different bias value. The bias value can be obtained through a learning process as described later.

When the predetermined number of ensemble inferences is over and a new inference is performed, the control unit initializes the values previously stored in the input data buffer and controls to store the newly received data.

Referring to FIG. 2, the voting and reasoning unit 224 of the hardware 2 receives the voting and reasoning unit control signal D3 formed by the control unit 20 processing the control signal D1, and based on this The voting unit 2242 controls how many trials the result value is accumulated. The voting unit 2242 accumulates and stores the result of the accumulation-expression unit 222 in the expression result buffer until the set number of ensemble inferences is reached, and the next inference is performed without outputting the value to the inference unit 2246. do.

When the determined number of ensemble inferences is reached, the finally accumulated expression result buffer value is sent to the inference unit 2246 to determine the final ensemble inference result, and then the expression result buffer is initialized to prepare for the next ensemble inference. .

3, the pseudo-random number generator 26 of the hardware 2 receives a pseudo-random number generator control signal D4 from an external or a control unit, and the seed buffer unit 262 receives a pseudo-random number according to the number of times that inference is performed. It controls to change the value of the seed buffer, which is the basis of generation. The seed buffer of the seed buffer unit 262 may be changed every trial of the ensemble inference, may be changed every two or more trials, or may be changed irregularly. In general, after the ensemble inference is finished, it returns to the initial seed buffer value, but it may be set to a new seed buffer value.

When performing one inference, we multiply the input by the weight matrix to get the output, which is the value for the output neuron. According to the deterministic algorithm, an output (weighted sum) value obtained by multiplying an input vector and a weight matrix is obtained, which is provided as a factor to an activation function. The output of the activation function is the activation value, and that value is the output value of the neuron.

However, according to the stochastic algorithm, after obtaining an activation value, it is compared with a random number in an appropriate range, and the output value of the neuron is determined as 1 or 0 according to the magnitude relationship between the activation value and the random number value. For example, if the activation value is less than the random number, the final output value of the neuron becomes 0, and if the activation value is greater than the random number value, the final output value of the neuron becomes 1. The reason for using random numbers in this way is that the activation value calculated through the weighted sum value in the stochastic algorithm becomes the probability that the corresponding neuron will be expressed, so it is a method to sample the final output value of the corresponding neuron from the probability. This is because it requires a process of comparing the probability of occurrence and the random number selected. In this way, the stochastic algorithm in which probability intervenes is closer to the real brain than the deterministic algorithm.

When the seed buffer unit 262 sets the value of the seed buffer, the linear feedback shift register (LFSR) and the random number generator 264 generate pseudo-random numbers through this.

Second Example

Hereinafter, an embodiment of neural network hardware will be described with reference to FIGS. 4 to 7. Elements that are the same as or similar to the embodiments described above may not be illustrated in the drawings, and descriptions may be omitted. The present embodiment relates to inference using a neural network, and results obtained by performing a plurality of different inferences are expressed through a plurality of different label neurons.

Referring to FIG. 4, a label neuron part 13 of a neural network includes two or more neurons, and is calculated through a hidden neuron part 11. In one embodiment, the neural network is composed of three layers: an input layer providing an input vector, a label layer providing an output, and a hidden layer between the input layer and the label layer. Can be classified. The hidden layer may be substantially one neuron layer or two or more neuron layers. These layers can be connected by a weight matrix.

The embodiment of FIG. 4 shows the case of having five label neurons per class. In the exemplary embodiment illustrated in FIG. 4, the number of neurons that are finally activated in the label neuron unit 13 may be limited by the number of neurons allocated per class according to an external input signal.

In the example shown in FIG. 4, five labeled neurons are finally expressed, and after that, the number of expressed neurons for each class is summed and the class in which the largest number of neurons is expressed is the result of the final inference on the input data. It shows what was selected as.

In one embodiment, when several label neurons are expressed through one inference, the results may be synthesized and selected as a final inference result. According to another embodiment, a plurality of inference results may be collected and label neurons may be expressed through this. That is, the expression result of the labeled neuron is selected as the result of one or more inferences performed through the neural network.

In another embodiment not shown, the number of neurons that are finally activated in the labeled neuron part 13 may be expressed according to a determined expression condition without limitation, and the final inference result is the number of expressed neurons per each class. It can be converted into inference probability and provided as a final inference result.

In the embodiment illustrated in FIG. 5, the neural network hardware 1 including the controller 10 receives control signals D1 provided from the outside through the controller 10. As described above, the control signal D1 may include one or more control signals along with the input data, and the control signal D1 includes the number of classes of data input to the corresponding hardware 1 and the number of label neurons allocated per class. It may include information on the number and a signal for determining whether to limit the expression number of label neurons.

The control unit 10 controls the accumulation-expression unit 122 by forming and outputting the accumulation-expression unit control signal D2 based on the received control signal D1, and inputs the input data D4 to the input data buffer unit. Provided in (14). Data may be stored in the input data buffer unit 14, and the input data buffer unit 14 provides the stored data D4' to the accumulating unit 1224 of the accumulating-expressing unit 122. The control unit 10 forms a voting and reasoning unit control signal D3 based on the control signal D1, and outputs it to control the voting and reasoning unit 124.

6, the accumulation-expression unit 122 of the hardware 1 receives the accumulation-expression control signal D2, and the expression unit 1226 expresses the value of the label neuron unit 13 Used for The weight storage unit 1222 stores a weight value, which is a weight between neuron layers, and includes memory elements. 6, it is illustrated that the weight storage unit 1222 is included in the accumulation-expression unit 122. In another embodiment, the weight storage unit 122 may exist outside the storage-expression unit 122 and may only provide a value read from the memory device to the storage-expression unit 122.

The storage unit 1224 receives the data D4' stored in the input data buffer unit 14. In addition, the accumulation unit 1224 receives a weight value from the weight storage unit 1222, calculates a partial sum by calculating the weight value from the weight storage unit 1222, and calculates the received data value (D4'), and accumulates the partial sum. (weighted sum) is formed. The expression portion 1226 receives weighted polymerization and determines whether or not the neuronal portion is expressed. In one embodiment, the expression unit 1226 may determine whether to express the neuron unit by adding a bias value to the weighted sum.

Among them, when the accumulation-expression part 122 determines the expression of the label neuron part 13, the total number of label neurons expressed according to the accumulation-expression part control signal D2 is the number of label neurons allocated per class. It serves to express the same as the expression mode of other neurons without limiting or limiting it.

Referring to FIG. 7, the voting and reasoning unit 124 of the hardware 1 receives the voting and reasoning unit control signal D3, and the voting unit 1242 is based on the result of the expression unit 1226. Control to count the number of expressed neurons per class. The voting unit 1242 transmits the count result to the reasoning unit 1246 to determine the final reasoning result or present the reasoning probability for each class.

The voting unit 1242 collects the number of expressed label neurons of each class based on the number of classes received from the voting and inference unit control signal D3 and the number of neurons allocated per class. The inference unit 1246 determines the class with the most votes as the final class inference candidate of the input data based on the result of the voting unit 3110, or determines the number of neurons that have been voted for each class as the inference probability per class. And notified as the final result.

Third Example

Hereinafter, another embodiment of neural network hardware will be described with reference to FIGS. 8 to 9. Elements that are the same as or similar to the embodiments described above may not be illustrated in the drawings, and descriptions may be omitted.

When computing a neural network using hardware, learning parameter values including weights and biases are derived by performing a learning process using a server, and inference is generally performed through a neural network learned using hardware. to be. However, in this embodiment, learning is performed on-chip and inference can be performed in parallel.

Hereinafter, a method of updating a random learning parameter according to the present embodiment will be described. When training a neural network, a weighted sum is obtained by multiplying an input vector and a weight matrix, and a bias value is added to the weighted sum to provide an activation function as a factor. The output of the activation function is an activation value, and neurons provide an output according to the activation value, and the difference between the obtained output and the desired output is an error.

In an embodiment, the bias may have a vector form, and may have a different bias value for each output neuron currently being calculated. The bias value may be calculated through the learning process, through the difference between the currently inferred result value and the ideal correct answer.

Learning of a neural network learns a learning parameter in a direction to reduce this error. As an example, the learning parameter may include a weight and a bias. In the random learning parameter learning according to the present embodiment, weight learning may be performed by adding r·η·ΔW to a target weight value. η means the learning rate, r is the random number, and ΔW is the weight error.

For example, if the random number is 1, η·ΔW is added, and if the random number is 0, η·ΔW is not added. In general, it is known that the higher the learning rate, the faster the learning process proceeds, but the final inference accuracy is low, and the smaller the learning rate, the longer the learning process is, but the final inference accuracy is high. In the present embodiment, multiplying the product of the learning rate and the weight change amount (ΔW) by the random number value is, for example, a value in which the random number value becomes 1 with a probability of 1/2, which is the overall learning rate after multiple times of learning. The effective learning rate (η _eff ) is set to η _eff It plays a role of lowering ≒ 1/2*η. In addition, by using a random number, it can play a role of performing more generalized learning without being biased against some specific learning results, thereby improving the accuracy of final inference.

In the random learning parameter learning according to the present embodiment, the bias learning is similar to the weight learning above, and the difference between the theoretical bias value and the bias value obtained by learning is multiplied by a random number and a learning rate. In addition, it can be done.

Referring to FIG. 8, the neural network hardware 3 including the control device 30 according to an embodiment of the present invention receives control signals D1 provided from the outside through the control unit 30, based on these signals. As a result, the learning parameter correction unit 326 and the random number generator unit 34 are controlled through the learning parameter correction unit control signal D2 and the random number generator control signal D3, respectively. The external control signal D1 includes one or more control signals, among which, when the hardware 1 performs on-chip learning, whether or not the random learning parameter update method is applied. It includes a component to determine and a signal component to determine under what conditions the random number is extracted and applied at the time of application. In addition, the external control signal D1 may include a component of a weight value provided from a memory device (not shown) that stores the weight value. For example, a memory device (not shown) may be included in the same chip as the hardware according to the present embodiment.

The random number generator 34 may be a pseudo random number generator or a true random number generator. The random number generated by the random number generator 34 may be used when the voting and inference unit 324 and the learning parameter correction unit 326 operate.

Briefly describing the entire operation process of the operation unit 32, the accumulation-expression unit 322 performs an operation process of multiplying and accumulating the input data value input from the outside by a weight associated with the data, and adding a bias. The voting and inference unit 324 infers using the calculation result of the accumulation and expression unit 322.

The inference result and the ideal correct answer are compared and the difference is calculated through the error calculation unit 3262 included in the learning parameter correction unit 326. The error calculator 3262 may calculate a difference error ΔW between an ideal weight value and an actual weight value, and a difference error between the ideal bias value and the actual bias value.

The error calculation unit 3262 transfers the calculated weight error ΔW and the error of the bias value to the learning parameter correction value calculation unit 3264. The learning parameter correction value calculation unit 3264 multiplies the provided weight error (ΔW) and bias error by a learning rate (η) and a random number (r) to determine how much to correct the actual weight and bias.

In one embodiment, a learning rate value multiplied by a weight error ΔW and a learning rate value multiplied by a bias error are different from each other. In addition, the random number value multiplied by the weight error ΔW and the random number value multiplied by the bias error are different from each other. In another embodiment, the learning rate value multiplied by the weight error ΔW and the learning rate value multiplied by the bias error are the same. In addition, the random number value multiplied by the weight error ΔW and the random number value multiplied by the bias error are the same.

The control signal D1 largely controls the hardware operation of two parts. The control unit 30 controls the learning parameter correction unit calculation unit 3264 of the learning parameter correction unit 326 through the learning parameter correction unit control signal D2, and also provides a random number generator control signal D3 to generate a random number. Control the base 12.

If a random weight update method is applied during on-chip learning of the hardware 3, the weight correction value calculation unit 3264 calculates a final weight correction value using the random number generated by the random number generator 34.

The method of using the random number may vary depending on the condition of the external control signal D1. For example, a 1-bit random number is generated for each weight, and if the corresponding random number is 0, the final weight correction value is 0, and if the corresponding random number is 1, the final weight correction value calculated through the error value of the error calculation unit 3262 Can be set to

That is, the weight correction value calculated according to the condition of the random number may be used as the final weight correction value to be reflected in the weight update. Depending on the condition of the external control signal (D1), N random numbers of 1 bit are generated for each weight and can be done only when all of the random numbers are 1, or a random number of 2 or more bits is generated and the value of the random number is specified. Only when the value is greater than or equal to the value, the calculated weight correction value can be used as the final weight correction value and reflected in the weight update.

10 is an example of a structure of a device equipped with a neural network. A device equipped with a neural network may be prepared by learning a neural network using training data. In addition, a device equipped with a neural network corresponds to a device that provides a service using a learned neural network. The device described in FIG. 10 may construct a neural network using the above-described process.

10A is an example of a device such as the PC 200 that generates a neural network. The PC 200 receives training data from the training data DB 50. The PC 200 may also receive training data through a physically connected storage medium. The PC 200 may calculate a correction value ΔU of the weight between the label neuron and the hidden neuron using the above-described asymmetric contrast divergence algorithm while generating the neural network. The PC 200 may construct and rebuild hidden neurons using only data neurons.

10(B) is an example of a device such as the server 80 generating a neural network. The server 80 receives training data from the client device 80. Of course, the server 80 may also receive training data from another object connected to the network. The server 80 may calculate a correction value ΔU of the weight between the label neuron and the hidden neuron using the above-described learning algorithm while generating the neural network. The server 80 may build and rebuild hidden neurons using only data neurons.

The neural network may be provided in a circuit or chipset composed of a memory and an operation element. FIG. 10C is an example of a configuration of a device 400 equipped with a neural network without limiting the physical configuration. 10(C) may be a configuration of a PC 200, a server 300, or a circuit board (or chip) mounted with artificial intelligence.

The device 400 equipped with a neural network includes an input device 410, an operation device 420, and a storage device 430. Furthermore, the device 400 equipped with a neural network may further include an output device 440.

The input device 410 receives training data or input data. The input device 410 may be a communication device or an interface device that receives data through a network. Also, the input device 410 may be an interface device that receives data through a wired network. Meanwhile, the input device 410 may receive an external control signal.

The storage device 430 may basically store a neural network model. The storage device 430 may be implemented with various media capable of storing data. The storage device 430 may store an initial neural network before learning, various information and parameters used in a learning process, and store a neural network that has been trained.

The computing device 440 learns and prepares a neural network using training data. The computing device 440 may build data neurons and label neurons based on the training data, and build hidden neurons based on the output values of the data neurons. In addition, the computing device 440 may reconstruct data neurons and label neurons based on output values of hidden neurons. The computing device 440 may learn the neural network according to the above-described process. Accordingly, the computing device 440 may construct and rebuild hidden neurons using only data neurons. The computing device 440 may calculate a correction value ΔU of the weight between the label neuron and the hidden neuron using the above-described asymmetric contrast divergence algorithm while generating the neural network.

The computing device 440 may derive a constant result value by inputting the input data into the learned neural network.

The computing device 440 corresponds to a device that processes data by driving certain commands or programs. The computing device 440 may be implemented with a memory (buffer) that temporarily stores instructions or information and a processor that performs calculation processing. The processor may be implemented as a CPU, AP, or FPGA depending on the type of device.

The output device 440 may be a communication device that transmits necessary data to the outside. The output device 440 may externally transmit a result value derived from the learned neural network. In some cases, the output device 440 may be a device that outputs a neural network learning process or a result value to be derived from the learned neural network on a screen.

In addition, the neural network learning method, the inference method using the neural network, and the neural network operation method as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be provided by being stored in a non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, and ROM.

The present embodiment and the accompanying drawings are merely illustrative of some of the technical ideas included in the above-described technology, and those skilled in the art will be able to easily within the scope of the technical ideas included in the specification and drawings of the above-described technology. It will be apparent that all of the modified examples and specific embodiments that can be inferred are included in the scope of the rights of the above-described technology.

1, 2, 3: hardware 10, 20, 30: control unit
11: hidden neuron part 13: label neuron part
12, 22, 32: operator 34: random number generator
24: input data buffer unit 26: pseudo-random number generator
262: seed buffer unit 264: LFSR and random number generator
122, 222, 322: accumulation-expression unit 1222: weight storage unit
1224: accumulation portion 1226: expression portion
124, 224, 324: Voting and Reasoning Department 1242, 2242: Voting Department
1246, 2246: reasoning unit 326: weight correction unit
3262: error calculation unit 3264: weight correction value calculation unit
200: PC 80: Client device 300: Server 400: Device equipped with ClassRBM
410: input device 420: operation device
430: storage device 440: output device

Claims (5)

Neural network hardware that performs on-chip learning of neural networks,
The neural network hardware performs the on-chip learning process to apply a random learning parameter update method when updating a learning parameter,
The learning parameter includes a weight and a bias,
The learning parameter update
The actual weight correction amount is determined by multiplying the weight error by the weight learning rate and the random number,
Neural network hardware that determines the actual bias correction amount by multiplying the bias error by the bias learning rate and random number.
The method of claim 1,
The random learning parameter update method,
Neural network hardware that applies the calculated training parameter correction value to the actual training parameter update only when the provided random number satisfies a predetermined condition. According to claim 2
The random number generator is any one of a pseudorandom number generator and a pure random number generator,
Neural network hardware that controls the random number generator to generate a random number of 1 bit or more than 2 bits by one or more times each time the random number generator updates one learning parameter according to an external control signal. delete delete

Images