JPH0635881A - Neural network - Google Patents

Abstract

PURPOSE:To minimize the discretization errors of connection strength and to indicate the connection strength with high accuracy. CONSTITUTION:A second arithmetic part 5 calculates the update values of the connection strength Wji based on errors between teacher data from an input part 1 and the arithmetic results of respective nerve cell models by a first arithmetic part 2 calculated by an error calculation part 4. A connection strength setting part 6 makes the update values of the connection strength Wji discrete based on the characteristic of making discrete for which the values between the maximum value and the minimum value in all the update values of the connection strength Wji calculated at the second arithmetic part 5 are made discrete into the prescribed number of gradations. Thus, steps of making discrete are reduced and the discretization errors of the connection strength is reduced without increasing the number of the discretization gradations. Further, the update values of the connection strength are made discrete within the range of the values actually appearing and the connection strength is efficiently indicated with high accuracy.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural circuit device having a limited number of gray levels of coupling strength.

[0002]

2. Description of the Related Art Conventionally, there is a neural circuit device using a nerve cell model composed of a synapse portion 11 and a neuron portion 12 as shown in FIG. And
The neural circuit device is configured in a network form by combining a plurality of nerve cell models so that the output value of the neuron unit 12 in one nerve cell model becomes the input value of the synapse unit 11 in the next nerve cell model. To be done.

Now, it is assumed that the nerve cell model shown in FIG. 4 is the j (j = 1, 2, ..., L) th nerve cell model in the neural circuit device. In FIG. 4, the synapse portion 11 is an expression
The multiplication process and the addition process represented by (1) are performed, and the value obtained by multiplying each of the plurality of input signals X _i (i = 1, 2, ..., N) and the corresponding coupling strength W _ji is further added. The value “net _j ” obtained by the above is output to the neuron unit 12.

[Equation 1] Here, X _i is the output value of the i-th nerve cell model, and W _ji is the coupling strength when the output value from the i-th nerve cell model is input to the j-th nerve cell model.

The neuron section 12 has "A" and "B".
The value input from the synapse unit 11 by performing threshold processing with the sigmoid function represented by the equation (2) where is a positive constant.
Outputs the value "g (net _j )", which is the threshold value for net _j .

[Equation 2]

As described above, the neural circuit device forms a network by combining a plurality of neural cell models, and is configured to execute the above processes in parallel.

Originally, the coupling strength W _ji is a continuous value, but when it is realized on hardware, it is treated as a discretized value. As shown in FIG. 5, the bond strength W
_ji is discretized by the equation (3) using a predetermined value "w ₀ " that does not depend on the nerve cell model. W _ji = n _ji · w ₀ (n _ji = 0,1,2, ..., M-1) (3) Incidentally, the number of gradations in the discretization in FIG. Say) M is "6". Further, a predetermined value w ₀ (hereinafter,
The value of the discretization step) is j = 1, 2, ..., L and i.
= 1, 2, ..., N, the maximum value of the bond strength W _ji "max _ji W
by using the _ji ", represented by the formula _{(4). w 0 = max} ji W ji / (M-1) ... (4)

That is, in FIG. 5, the region between the value "0" and the maximum value "max _ji W _ji " of the coupling strength W _ji is discretized into M stages.

[0008]

However, the above conventional discretization method of the coupling strength W _ji has the following problems. That is, in order to enhance the processing function of the neural circuit device, it is necessary to accurately represent the coupling strength W _ji . However, when the neural circuit device is configured using hardware such as an analog circuit, the number of gradations of the coupling strength W _ji cannot be increased infinitely.

[0009] In discretization methods conventional coupling strength W _ji, bond strength W _ji in the synapses 11 are discretized for each discretization step w ₀ as shown in equation (3), the error due to the discretization the maximum value E can be expressed from a w _0/2 as in equation (5). E = max _ji W _ji / 2 (M-1) (5) That is, when the value of the discrete gradation number M is constant, the maximum value "max _ji W _ji " of the coupling strength W _ji is There is a problem that the larger the discretization error becomes, the larger the discretization error becomes.

Further, the discretization step w ₀ is j = 1,
2, ..., L and bond strength W _{ji at} i = 1,2, ..., N
In order to be uniformly decided only by the maximum value of “max _ji W _ji ”,
, L and i = 1, 2, ..., N, the difference between the maximum value “max _ji W _ji ” and the minimum value “min _ji W _ji ” of the bond strength W _ji is “max. _ji W _ji "if sufficiently smaller than, the value becomes" unnecessary region discretization of the region of interest between 0 "and" min _ji W _ji ". Therefore, the number of gray levels of the coupling strength W _{ji that} can be actually expressed becomes smaller than the discretized gray level number M, the efficiency of the discretization is poor, and the coupling strength W _ji cannot be expressed accurately. In the case of FIG. 5, the number of discrete gradations M is “6”, whereas the number of gradations of the actually expressed coupling strength W _ji (min _ji W _ji ≦ W _ji ≦ max _ji W _ji ) is It is "5" and smaller than the number M of discrete gradations.

Therefore, an object of the present invention is to reduce the discretization error of the coupling strength W _ji without increasing the discretized gradation number M and to accurately represent the coupling strength W _ji. To provide.

[0012]

In order to achieve the above-mentioned object, a neural circuit device according to a first aspect of the present invention calculates the value of the bond strength stored in a bond strength storage unit for the calculation of each nerve cell model forming a network. A first calculation unit that is executed by using the first calculation unit, a second calculation unit that calculates the update value of the bond strength based on the calculation result of the first calculation unit, and an update value of the bond strength that is calculated by the second calculation unit. In a neural circuit device having a bond strength setting unit that updates the value of the bond strength stored in the bond strength storage unit with a value obtained by discretizing the above into a predetermined number of gray levels, Based on the discretization characteristics such that the maximum value and the minimum value of all the bond strength update values calculated by the calculation unit are discretized to the above-described predetermined gradation number, the input bond strength Characterized by discretizing update values That.

In the neural circuit device of the second invention, the calculation of each nerve cell model forming the network is performed by using the value of the bond strength stored in the bond strength storage unit.
An arithmetic unit, a second arithmetic unit that calculates an updated value of the bond strength based on the arithmetic result of the first arithmetic unit, and the second arithmetic unit
In a neural circuit device having a bond strength setting unit for updating the bond strength value stored in the bond strength storage unit with a value obtained by discretizing the update value of the bond strength calculated by the arithmetic unit into a predetermined number of gradations. The coupling strength setting unit is configured to operate in the second
Based on the discretization characteristics such that the interval between the maximum value and the minimum value of the update values of all the connection strengths related to a certain nerve cell model calculated by the arithmetic unit is discretized to the predetermined gradation number. Then, the input update value of the coupling strength relating to the nerve cell model is discretized.

[0014]

According to the first aspect of the invention, the first computing unit performs computation of each nerve cell model forming the network using the value of the binding strength stored in the binding strength storage unit.
Then, based on the calculation result by the first calculation unit, the second calculation unit calculates the updated value of the bond strength.
Then, the discretization is performed such that the coupling strength setting unit discretizes between the maximum value and the minimum value of all the update values of the coupling strength calculated by the second computing unit into the predetermined number of gradations. Based on the characteristics, the updated value of the coupling strength input from the second calculation unit is discretized. Then, the value of the bond strength stored in the bond strength storage unit is updated with the obtained discretized value.

In this way, the updated value of the coupling strength calculated by the second arithmetic unit is discretized into the predetermined number of gradations within the range of the value that actually appears, so that the discretization error is small and the efficiency is high and the precision is high. Discretized in.

In the second invention, the updated value of the bond strength is calculated in the same manner as in the first invention. Then, the bond strength setting unit sets the predetermined number of gray levels between the maximum value and the minimum value of the update values of all the bond strengths calculated for the one nerve cell model by the second computing unit. Based on the discretized characteristics, the update value of the coupling strength related to the nerve cell model input from the second calculation unit is discretized. Then, the value of the bond strength stored in the bond strength storage unit is updated with the obtained discretized value.

In this way, the updated value of the coupling strength for each nerve cell model calculated by the second calculation unit is discretized into the predetermined number of gradations within the range of the value that actually appears,
Efficient and highly accurate discretization with less discretization error.

[0018]

The present invention will be described in detail below with reference to the embodiments shown in the drawings. The present invention is not limited to the following embodiments.

FIG. 1 is a block diagram functionally showing a neural circuit device in each of the following embodiments. In order to simplify the explanation, it is assumed that the neural circuit device in FIG. 1 forms a multi-layer perceptron type network in which a plurality of nerve cell models are arranged in layers, and learning is performed by the error back propagation method.

In FIG. 1, in the normal evaluation, this neural circuit device uses the first operation unit 2 based on the evaluation data input from the input unit 1 to generate synapses in all layered nerve cell models. The calculation of the part 11 and the calculation of the neuron part 12 is performed by the connection strength W _ji stored in the connection strength storage 7 (the output from the i-th nerve cell model of the nerve cell models connected to the j-th nerve cell model). This is carried out by using the bond strength when the value is input to the j-th nerve cell model. Then, the calculation result in the neuron unit 12 of the nerve cell model belonging to the final stage is output from the output unit 3 as an evaluation result.

On the other hand, at the time of learning, based on the learning data input from the input unit 1, the first calculation unit 2
The same calculation as that at the time of evaluation is performed and the calculation result is output to the error calculation unit 4. Further, the teacher data input from the input unit 1 is input to the error calculation unit 4. Then, the error calculation unit 4 calculates the squared error between the calculation result and the teacher data, and inputs the squared error to the second calculation unit 5.

The second arithmetic unit 5 carries out a learning arithmetic operation based on the error backpropagation rule so as to minimize the value of the input squared error (that is, the actual arithmetic result is as much as possible in the teacher data). The updated value of each bond strength W _ji is calculated (as it approaches). The updated value (continuous value) of the bond strength W _ji thus calculated is input to the bond strength setting unit 6 and discretized in the bond strength setting unit 6. Then, the contents of the bond strength storage unit 7 are updated with the bond strength W _ji which is discretized and set by the bond strength setting unit 6.

<First Embodiment> FIG. 2 shows the connection strength W _ji in the synapse portion 11 of the j-th nerve cell model forming the multilayer perceptron type network in the connection strength setting portion 6 in this embodiment. Update value of (below,
Discretization characteristics when discretizing and expressing the coupling strength W _ji ) are shown.

In FIG. 2, the binding strength W _ji is calculated by using predetermined values "w" and "y" that are not dependent on the nerve cell model.
It is discretized according to equation (6). W _ji = n _ji · w + y (n _ji = 0,1,2, ..., M-1) (6) The number M of discrete gradations in FIG. 2 is "6". Further, the value of the predetermined value w (hereinafter referred to as a discretization step) and the predetermined value y are the maximum value of the coupling strength W _ji at j = 1, 2, ..., L and i = 1, 2 ,. Expressions (7) and (8) are expressed using "max _ji W _ji " and the minimum value "min _ji W _ji ". w = (max _ji W _ji −min _ji W _ji ) / (M−1) (7) y = min _ji W _ji (8)

That is, in FIG. 2, j = 1, 2,
, L and i = 1, 2, ..., N, the region between the minimum value “min _ji W _ji ” and the maximum value “max _ji W _ji ” of the coupling strength W _ji is discretized into M stages. In other words, the difference between the maximum value and the minimum value of all the connection strengths given to all the nerve cell models is discretized into M stages.

In the discretization characteristic of the coupling strength W _ji as shown in FIG. 2, since the maximum value E of the error due to discretization is w / 2, it can be expressed as in equation (9). E = (max _ji W _ji −min _ji W _ji ) / 2 (M−1) (9) As a result, the discretization error can be made smaller than the discretization error in the conventional example represented by the above equation (5). Of.

According to the discretization characteristics shown in FIG. 2, j
, L and i = 1, 2, ..., N, the difference between the maximum value “max _ji W _ji ” and the minimum value “min _ji W _ji ” of the bond strength W _ji is “max _ji. Even if it is smaller than "W _ji ", the number of gradations of the bond strength W _{ji that} can be actually expressed is the same as the number of discrete gradations M, so the number of gradations of the bond strength W _{ji that} can be actually expressed decreases. There is nothing to do. Therefore, the bond strength W _ji can be efficiently discretized, and the bond strength W _ji can be represented accurately. In the case of FIG. 2, the number of discretized gradations M and the number of gradations of the expressed coupling strength W _ji (min _ji W _ji ≤W _ji ≤max _ji W _ji ) are both "6".

As described above, in the above embodiment, when the coupling strength setting unit 6 discretizes the updated value of each coupling strength W _ji calculated by the second computing unit 5, j
, L, and i = 1, 2, ..., N, the difference between the maximum value max _ji W _ji and the minimum value min _ji W _ji of the bond strength W _ji
(M-1) The equally divided value w is used as the discretization step. Therefore, the discretization step w is made smaller than in the conventional case, and w /
The maximum value E of the discretization error that can be represented by 2 can be reduced. Furthermore, the number of gray levels of the coupling strength W _{ji that} can be actually expressed becomes the same as the number of discretized gray levels M, which allows efficient discretization. As a result, the bond strength W _ji can be accurately represented.

<Second Embodiment> In the first embodiment, as described above, the maximum value and the minimum value of all the binding strengths given to all the nerve cell models in the multilayer perceptron type network are described. Is discretized into M stages. Therefore, it is very effective when all the binding strengths W _ji given to all the nerve cell models are evenly present within a predetermined range.

However, when only the coupling strength W relating to one nerve cell model is extremely large, the discretization shown in FIG. 2 is used when expressing the coupling strength W _ji relating to most other nerve cell models. An area smaller than the area is sufficient. Therefore, in such a case, the discretization efficiency in expressing the coupling strength W _ji related to most of the nerve cell models becomes very poor.

In this embodiment, in order to efficiently discretize even when only the coupling strength W relating to one certain nerve cell model is extremely large, the discretization step is performed for each nerve cell model. Is set.

FIG. 3 shows the coupling strength W _1i for each nerve cell model.
~ _Shows the discretization characteristics of W _Li . Now, consider the jth neuron model. The bond strength W _{ji in the} j-th nerve cell model is calculated by using the predetermined values “w _j ” and “y _j ” not depending on the nerve cell model determined for each nerve cell model.
Discretized according to (10). W _ji = n _ji · w _j + y _j (n _ji = 0,1, ..., M−1) (10) The number M of discrete gradations in FIG. 3 is “6”. Further, a predetermined value w _j (hereinafter referred to as a discretization step) and y _j
The value of is expressed by equations (11) and (12) using the maximum value “max _i W _ji ” and the minimum value “min _i W _ji ” of the binding strength W _ji at i = 1, 2 ,. expressed. w _j = (max _i W _ji −min _i W _ji ) / (M−1) (11) y _j = min _i W _ji (12)

That is, in FIG. 3, i = 1, 2,
The area between the minimum value "min _i W _ji " and the maximum value "max _i W _ji " of the coupling strength W _{ji at} N is discretized into M stages. In other words, the difference between the maximum value and the minimum value of all the connection strengths W _ji given to the j-th nerve cell model is discretized into M stages.

In the discretization characteristics of the coupling strength W _ji as shown in FIG. 3, the maximum error E _j due to discretization is w _j / 2.
Therefore, it can be expressed as in Expression (13). E _j = (max _i W _ji −min _i W _ji ) / 2 (M−1) (13) As a result, the discretization error is smaller than the discretization error in the conventional example represented by the above equation (5). You can do it.

Further, according to the discretization characteristics shown in FIG. 3, j
The region between the maximum value “max _i W _ji ” and the minimum value “min _i W _ji ” of all the binding strengths W _ji given to the th neuron model is M.
Discretize into stages. Therefore, as shown in FIG. 3 (a), FIG. 3 (b), and FIG. 3 (c), even if there is a difference in the appearance range of the coupling strength between the nerve cell models, the coupling is performed for each nerve cell model. It is optimally discretized according to the intensity range. As a result, as described above, even when only the coupling strength W relating to a certain nerve cell model is extremely large, the discretization step is optimally set according to the coupling strength range for each nerve cell model. , The bond strength W _ji can be efficiently discretized.

Therefore, according to the present embodiment, it is possible to reduce the discretization error as a whole as compared with the case of the first embodiment and further improve the discretization efficiency.

In each of the above embodiments, the network of each nerve cell model constituting the nerve circuit device is described as a multi-layer perceptron type network, but the network structure of the nerve cell model in the nerve circuit device of the present invention is described. Is not limited to this.

[0038]

As is apparent from the above, the neural circuit device of the first invention is such that the updated value of the coupling strength calculated by the second arithmetic unit based on the arithmetic result of each nerve cell model by the first arithmetic unit. Discretization such that the bond strength setting unit discretizes between the maximum value and the minimum value of all the update values of the bond strength calculated by the second calculation unit to the predetermined number of gradations. Since the discretization is performed based on the characteristics, a region smaller than the minimum update value of the coupling strength can be excluded from the discretized region. As a result, the discretization step can be reduced, and the discretization error of the coupling strength can be reduced without increasing the number of discretization gradations.

Further, since the updated value of the bond strength calculated by the second arithmetic unit is discretized into the predetermined number of gradations within the range of the value that actually appears, the bond strength is efficiently and accurately represented. be able to.

The neural circuit device according to the second invention is the first invention.
The updated value of the bond strength for each nerve cell model calculated by the second calculator based on the calculation result of each nerve cell model by the calculator is calculated by the bond strength setting unit as the second value.
Discretized based on the discretization characteristic such that the maximum value and the minimum value of all the update values of the coupling strength related to the nerve cell model calculated by the arithmetic unit are discretized to the predetermined gradation number. Therefore, the region smaller than the minimum update value of all the bond strengths related to the nerve cell model can be excluded from the discretized region. As a result, the discretization step can be set for each nerve cell model according to the appearance range of the update value of the coupling strength related to each nerve cell model, and the entire discretization of the coupling strength can be performed without increasing the number of discrete gradations. The error can be reduced.

Further, the updated value of the bond strength calculated by the second computing unit is discretized into the predetermined number of gradations for each nerve cell model within the range of values that actually appear in each nerve cell model. Therefore, the bond strength can be expressed more efficiently and more accurately.

[Brief description of drawings]

FIG. 1 is a functional block diagram of a neural circuit device according to the present invention.

FIG. 2 is a diagram showing discretization characteristics used in a coupling strength setting unit in the neural circuit device shown in FIG.

3 is a diagram showing a discretization characteristic different from that of FIG. 2 used in the coupling strength setting unit in the neural circuit device shown in FIG.

FIG. 4 is a diagram showing a general nerve cell model.

FIG. 5 is a diagram showing discretization characteristics when discretizing coupling strength in a conventional neural circuit device.

[Explanation of symbols]

1 ... Input unit, 2 ... First arithmetic unit,
4 ... error calculation unit, 5 ... second calculation unit,
6 ... Coupling strength setting unit, 7 ... Coupling strength storing unit, 11 ... Synapse unit, 12 ... Neuron unit.

Claims (2)

[Claims] 1. A first arithmetic unit for performing an arithmetic operation of each nerve cell model forming a network by using a value of a bond strength stored in a bond strength storage unit, and based on a calculation result by the first arithmetic unit. Second for calculating the updated value of the bond strength
An operation unit and a bond strength setting for updating the value of the bond strength stored in the bond strength storage unit with a value obtained by discretizing the update value of the bond strength calculated by the second operation unit into a predetermined gradation number. In the neural circuit device having a section, the coupling strength setting unit separates the maximum value and the minimum value of all the update values of the coupling strength calculated by the second computing unit into the predetermined gradation number. A neural circuit device characterized by discretizing the input update value of the coupling strength based on the discretized characteristic. 2. A first arithmetic unit for executing an arithmetic operation of each nerve cell model forming a network using the value of the bond strength stored in the bond strength storage unit, and based on the calculation result by the first arithmetic unit. Second for calculating the updated value of the bond strength
An operation unit and a bond strength setting for updating the value of the bond strength stored in the bond strength storage unit with a value obtained by discretizing the update value of the bond strength calculated by the second operation unit into a predetermined gradation number. In the neural circuit device having a section, the connection strength setting unit determines that the maximum value and the minimum value of the update values of all the connection strengths related to one certain nerve cell model calculated by the second calculation unit are A neural circuit device characterized by discretizing an input update value of the coupling strength relating to the nerve cell model on the basis of a discretization characteristic that is discretized into a predetermined number of gradations.