JPH02264354A - Data processor - Google Patents

Abstract

PURPOSE:To reduce necessary memory capacity and to shorten a processing time by introducing a degree conception in terms of data and a neural layer, and directly inputting the higher-degree data to the neural layer of a later stage. CONSTITUTION:Neural layers 20 and 30 provided with neurons(21, 22) and 23 in parallel, which output data according to a comparison result between the total sum obtained by multiplying input data A to C by a prescribed weight and a threshold value, are possessed, and the high-order output of the neural layer 20 of an early stage are made into the input data of the neural layer 30 on the next stage. Further the input data are inputted directly to the neural layer 30 on the next stage without passing through the neural layer 20 on the front stage.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a data processing device based on the concept of neural networks.

[Conventional technology]

A neural network in this type of data processing device is constructed in a layered manner by arranging nerve cell models (hereinafter referred to as neurons) 1 shown in FIG. 2 in parallel as shown in FIG. In neuron l, data DII and OX* are input from the outside.

Dl snow,...Dl have weights Wl, W8, W, respectively.
8, . Various methods are possible for this comparison, but for example, when this sum is greater than or equal to the threshold 0, the output data DO becomes "1", and when this sum is smaller than the threshold θ, the output data DO becomes "0". It is determined that

Now, in the neural network shown in Figure 3, if the input data is a bit pattern of n bits, there are 2n types of numerical values expressed by this bit pattern, so in order for the neural network to determine the content of this input data, it needs 2n bits. neurons l are required.

For example, in the case of a 4-bit bit pattern, 16
neurons l are required, and each time the number of bits increases by one, the number of neurons increases by a factor of two.

Furthermore, when a neural network has a multilayer structure, the number of neurons increases as the number of layers (hereinafter referred to as 221 layers) increases, and the number of input paths connected to neurons, that is, the number of synapses also increases. To increase.

[Problem to be solved by the invention]

When the number of synapses is large, weight data or threshold data corresponding to the number must be stored, and a large capacity memory is required for the data processing device as a whole. In addition, the number of multiplications of input data and weights or the number of calculations using a threshold value increases, which slows down the calculation speed. This was done for the purpose of providing equipment.

[Means to solve the problem]

The data processing device according to the present invention introduces the concept of order for data and neural layers, and directly inputs higher order data to the later neural layer.

[Effect]

As a result, the number of neurons and synapses can be reduced, which increases economic efficiency and reduces the required memory capacity and processing time.゛-[Example] Although the processing content actually executed by a neural network is extremely complex, the first embodiment of the present invention will be explained on the assumption that an extremely simple logical operation will be executed, and this example will be explained below. Explain the basic idea of the invention.

Figure 4 (a), (b), and (C) are the results of learning, A and (B).
(10C) This shows a data processing device on which the logical operation in (1) is executed. Fourth
Figure (a) shows that, as explained in the [Prior Art] section, the number 23 of all combinations of three data of AlB and C in the data group IO is added to the neural layer 20 at the first stage.
It shows a configuration in which eight neurons 21 to 28 are provided, and one neuron 31 is provided in the second stage neural layer 30 to calculate the OR (logical sum) of the outputs of the first stage neurons 21 to 2. . In the figure, neurons 21 to 28
The numbers such as roooJ attached above are data A%B, C
These neurons output an output "l" when this bit pattern is input.

In this data processing device, the number of neurons is 9 and the number of synapses is 32, the efficiency of neurons and synapses is extremely low, the required memory capacity is large, and
The processing time is also long.

Figure 4(b) expands equation (1) as follows, and A-B
+ A-C (2) This is an example in which calculations are simplified, and the first neural layer 20 includes neurons 21 for processing the calculations of A-B.
, A-C. The second stage neural layer 30 is a neuron 212
It has a neuron 31 that calculates the OR of two outputs.

In the neuron 21, the weights W, ~W, for the data ASB, , C are, for example, Wl-1, Wt-1, Wz
-0, AW l+BW! Output r when +CWs≧2(3)
Output lJ. Therefore, in this case, the threshold value θ=2. Similarly, in neuron 22, W<-1, Ws
"0.Wb = 1. θ=2. On the other hand, in the neuron 31, W, = 1°W, = t, θ-1.

In the data processing device shown in FIG. 4(b), the number of neurons is 3 and the number of synapses is 8. Compared to the data processing device shown in FIG. 4(a), the neuron efficiency and synaptic efficiency are significantly improved. There is. However, as described below, according to the present invention, these efficiencies are further improved.

FIG. 4(c) shows a first embodiment of the present invention. This configuration has an initial neural layer 20 and a second neural layer 30, and the initial neural layer 20 contains only data B and C. is input, and data A is directly input to the second stage neural layer 30. The first stage neural layer 20 is provided with one neuron 21 for performing the calculation of (B+C), and the second stage neural layer 30 is provided with one neuron 21 for performing the calculation of A・(B10). A neuron 31 is provided. The neuron 21 is set to, for example, the weight W, -ISW,=1, θ-1, and outputs an output of "1" when BWt +CWt≧1(4).On the other hand, the neuron 31 outputs an output of "1".
For example, Wl −1 = w4 = 1, θ = 2,
If the output of the neuron 21 is Yl, then YIW + AW
When 4≧2(5), output “1”. In this example, the number of neurons is 2 and the number of synapses is 4, as shown in FIG. 4(b).
Neuron efficiency and synaptic efficiency are significantly improved compared to the comparative example.

Here, the basic idea of the present invention will be explained.

Paying attention to equation (1) again, data B and C1 are connected by one operator r+(OR)J, and data A,
The result of the operation is connected by the operator "×(AND)". Therefore, data B, C and data A should not be evaluated in the same dimension, but if you dare to evaluate them in the same dimension, This results in a decrease in efficiency as shown in the comparative examples shown in FIGS. 4(a) and 4(b).

Here, it is assumed that the processing content of the neuron is only the evaluation of the following equation (6).

ΣW i A i−〇 (6) (Wi: weight, Ai: input, θ: threshold) Then, it is assumed that the degree of abstraction can be defined for each data, and this degree of abstraction is called "degree."

This order is defined as the order of output data increasing by one order with respect to input data in one neuron.

Further, it is defined that data connected by one operator have the same degree.

According to this definition, in the case of formula (1), data B,
, C are of the same order, and data A is of order 1
higher than rQJ, where the order of data B, , C is rQJ
, the order of data A is rlJ.

Considering the relationship between the order of this data and the hierarchy of neural layers, in Figure 4 (C), neural layer 2
Only zero-order data is input to the neural layer 30, and only first-order data is input to the neural layer 30. Therefore, it is clear that the neural layer 20.30 can be associated with the order of the input data, and hereinafter the same order as the input data will be defined for the neural layer as well.

Then, the neuron efficiency and synaptic efficiency are optimized by classifying the data group to be input to the data processing device according to its order and inputting the data of each order to the neural layer of the corresponding order.

Furthermore, as explained in relation to FIGS. 4(a) and (b), it is also possible to process high-order data by reducing it to a lower order, and in this case, neuron efficiency and synaptic efficiency decrease. . Therefore, the degree of each data should be based on the highest possible degree of that data.

Note that one data may be used for multiple orders, such as (A+B)·B, and in this case, one data may be input to multiple neural layers.

As an example to more clearly explain the degree of abstraction, which is the original meaning of the degree, FIGS. 5(a) to 5(c) show a data processing apparatus for determining the end points of a figure.

There are various ways to consider the end points of a figure, but here,
In the 3×3 composite section (Figure 5(a)),
It is assumed that when any of the patterns shown in FIG. 5(b) (i) to (viii) occurs, the central pixel (pixel E in FIG. 5(a)) is an end point. This end point is determined when one of the pixels A-D and F-1 other than the center pixel has a figure density (
For example, if the center pixel E has a figure density, it is considered an "end point", and otherwise, it is considered "no end point".

The data processing device for this determination is, for example, shown in FIG.
), and this figure shows a second embodiment of the present invention. This data processing device has eight neurons 21 to 2.
8, and a second neural layer 30, which has one neuron 31. Data of pixels A to D, F to Stage neural layer 3
It is entered as 0.

The neurons 21 to 28 of the first stage neural layer respectively determine that only A, only B, only C, etc. are graphic pixels rlJ, and the neuron 31 of the second stage neural layer 30 determines that any of the neurons 21 to 28 is rlJ
and when E is the figure pixel "l", output "
1" is output.

If each pixel data A to D and F to ■ is considered as zero-order data, the output of the neuron 31 becomes second-order data. Therefore, when the input system to the data processing device is an image processing system and the data of each pixel and various feature quantities are input data, if the data of each pixel is considered as zero-order data, the endpoint data is secondary data. It can be seen that it can be treated as 0
The image processing system outputs various data such as the number of groups, large numbers, Euler numbers, texture features, etc.
The order of these data should be considered and input directly to the optimal neural layer.

Next, an example of the configuration of a combinator section in which a data processing apparatus according to the present invention is constructed will be explained based on FIG. 6.

This figure shows a computer section 100 that is used by being connected to an image processing system described in, for example, Japanese Patent Laid-Open No. 63-170784. The processing results are stored in the system memory 1 (11) for output to the outside.
has a central processing unit (CPU) 102, a control data memory 110, an output data generation section 120, a register 103, a shift register 104, a shift register 105, and a multiplexer (MUX) 106 in addition to a system memory IJ 10k. The CPU 102, system memory 1 (11), control data memory 110, and register 103 are interconnected by a bus 107, and the input system 99 is connected not only to the bus 107 but also to the control data memory 110.

The control data memory 110 includes a synapse weight area 111
and an input/output data area 112.

FIG. 4 shows an example where there are two neural layers in addition to the output layer (see FIG. 1(b)), and the synapse weight area 111 contains the synapse weight data in the first neural layer, the synaptic weight data in the second neural layer, This is an area for storing synaptic weight data in the neural layer, synaptic weight data in the output layer, and the like.

The input/output data area 112 contains zero-order input data input to the first neural layer, zero-order output data output from the first neural layer, and input from the first neural layer to the second neural layer. This is an area for storing primary input data, primary output data output from the second neural layer, final output data output from the output layer, and the like.

The output data generation section 120 includes a multiplication circuit 121, an integration circuit 122, and a threshold processing circuit 123, and these circuits are controlled via the register 103.

The multiplication circuit 121 receives input data Ofn and synapse weight data W, and calculates the product of these. Input data DI is sent from control data memory 110 to MUX 10
6 is stored in the shift register 105, and input to the multiplication circuit 121 at a predetermined timing. The synaptic weight data W is transferred from the control data memory 110 to the MU
It is stored in the shift register 104 via X 106, and is input to the multiplication circuit 121 at a predetermined timing.

The integration circuit 122 receives each input data D1. Multiplying circuit 1 for each
The multiplication results of Dl and XW are input from 21, and these multiplication results are integrated. That is, the register of the integration circuit 122 stores the calculation result of Σ(DI i XW i ) for one synapse.

When the sum total of the multiplication results of input data and weights for one synapse is determined, this sum data is input to a threshold value processing circuit 123. The threshold processing circuit 123 stores the data of the threshold θ corresponding to each neuron as a table, for example. In this embodiment, the threshold processing circuit 12
3 compares the total sum data with the threshold value θ according to the following equation (7) and outputs output data DO.

Do−f(Σ(DI i xw i )−θ) (7
) Here, for example, f is a binarization function. That is, the output data 00 becomes "1" when the value obtained by subtracting the threshold value θ from the total data is 0 or more, and conversely becomes "0" when it is smaller than 0.

This output data DO is stored in the input/output layer area 112 of the control data memory 110 via the MUX l06.

FIG. 1(a) shows a third embodiment of the present invention, in which the data processing device has a data group 10 and a neural layer 20.90, and the layer 90 transfers output data to the outside of the data processing device. This is the output layer that outputs to.

The data group 10 is the first group of data 11.12.1
3...and the second group's data 14.15.16.
Consists of... That is, the data of the data group 10 is classified into two types, a first group and a second group, according to their order.

Neural layer 20 is neuron 2L22.23...
・The output layer 90 has neurons 91 and 92
．． It has 93... Each neuron of neural layer 20 is connected to each neuron of output layer 90.

The first group of data IL 12, 13, . . . is input to each neuron of the neural layer 20, and the second group of data 14, 15, 16, .

Each of the two neural layers of the neural layer 2G compares the total sum of each input data multiplied by a weight with a threshold value, and outputs data rl according to the comparison result, as shown in the above equation (7), for example. , or output "0". Each neuron of the output layer 90 corresponds to each neuron of the neural layer 20 and the second group of data 14.15.
Find the sum of 16... multiplied by their respective weights,
According to the comparison result between this sum and the threshold value, data of "1" or rQJ is outputted similarly to the neurons of the neural layer 20.

When this device is applied to an image processing device, the first group of data 11, 12, 13, . . .
0 indicating whether it is "l" (e.g. black) or "0" (e.g. white)
The following data is also the data of the second group 14.15
．． 16... is high-order data indicating the characteristics of the image.

Thus, in the third embodiment, the first group of data, ie, low-order data, is input to the neurons of the neural layer 20, and the second group of data, ie, high-order data, is input to the neurons of the output layer 90. .

Therefore, the neurons in the neural layer 20 perform lower-order processing, for example, processing on the data of the pixel itself, and the neurons in the output layer 90 perform higher-order processing, such as processing on various properties of pixels, etc. .

In this way, in the third embodiment, high-order data is input directly to the output layer 90 and not to the neural layer 20, so the number of synapses, that is, the number of connections between input elements and neurons or between neurons is reduced. However, the number of neurons also decreases. As the number of synapses decreases, the number of operations required for neurons decreases, so
The calculation speed increases and the number of weight data decreases, so
Also, as the number of neurons decreases, the number of threshold values also decreases, which reduces the memory capacity and reduces the number of calculations (thus increasing the calculation speed. Accordingly, high-speed arithmetic processing can be performed with a small memory capacity, and a highly efficient data processing device can be obtained with a simple circuit.

FIG. 1(b) shows a fourth embodiment of the present invention, and the data processing device in this embodiment has three groups of input data lO, a neural layer 20, 30, and an output layer 90.

The input data group 10 is the first group of data 11S 1
2.13... and the second group's data 14.15.
16...and the third group's data 17.18.19
...and is provided. That is, unlike the third embodiment, the input data group is classified into three types. The first neural layer 20 has neurons 21, 22, 23..., and the second neural layer 30 has neurons 31, 22, 23...
．． It has 32.33... The output layer 90 has neurons 9192, 93, . . . . Each number of neurons in the first neural layer 20 is connected to each neuron in the second neural layer 30, and each neuron in the second neural layer 30 is connected to each neuron in the output layer 90, respectively.

The first group of data 11, 12, 13... are sent to each neuron of the first neural layer 20, the second group of data 14, 15, 16... is sent to each neuron of the second neural layer 30, Third group data 17
．． 18, 19, . . . are respectively connected to the output layer 90 and each neuron.

As in the third embodiment, each of the two knee-longs is, for example, (7
), output data "1" or "0" is output according to each input data.

The first group of data 11.12.13... is zero-order data, the second group of data 14.15.16... is first-order data, and the third group of data 17.18.19.
... is secondary data. That is, high-order data is input to the second neural layer 30, and the output layer 9
Further higher-order data is input to 0.

In this third embodiment, as in the third embodiment, the number of synapses is reduced and the number of neurons is also reduced. Therefore, the same effects as in the second embodiment can be obtained.

FIGS. 7(a) and 7(b) show a fifth embodiment of the present invention together with a comparative example, and this example shows a case where input data is processed according to the logical operation (AeB)e (CeD). Here, e indicates "exclusive OR", and A, B, C, and D are digital values of "1" or "0", and the result of this logical operation is also "l".
” or as a digital value of rQJ.

FIG. 7(a) shows a comparative example, and this device has an input data group 10 for passing input data, first and second neural layers 20, 30, and an output layer 90. The input data group 10 consists of input data ASB, C, and D. The first neural layer 20 has four neurons 21.22.23.24 and the second neural layer 30 has four neurons 31.32.33.34. Each data A-D is input to each neuron of the first neural layer 20, and each neuron of the first neural layer 20 is connected to each neuron of the second neural layer 30, respectively. On the other hand, the output layer 90 has one neuron 91, and each neuron of the second neural layer 20 is connected to this neuron 91, respectively.

In the first neural layer 20, each neuron 2
1 has a weight and a threshold by which each input data is multiplied, and according to the above equation (7), output data rlJ is output when the sum of the products of each input data and the weight is greater than or equal to the threshold, and this sum is the threshold. When it is smaller than θ, output data “O” is output.

Similarly, neurons 22, 23, and 24 also output "1" or "0" depending on each input data. Similarly, in the second neural layer 30, each neuron outputs "1" or "0" depending on the input data. Similarly, the neurons 91 of the output layer 90 also respond to the output data from the neurons of the second neural layer 30.
Outputs data of ``1'' or ``0''.

Sate logical operation (AeB) e (([D) (7) The operation result is when data A and B do not match and C and D match, and when A and B match and C and D When there is a mismatch,
It becomes "1". In other cases, it is "0". Therefore, in Figure 7(a), each neuron is
It is composed of sea urchins.

In the first neural layer 20, neurons 21
, 22.23.24 are respectively "(11 XXJ, rlO
xx", rxxol", rXXloJ, outputs "1", and otherwise outputs "0". Here, rXX
J means ignore the data. On the other hand, the second
In the neural layer 30 of , the neurons 31 are
Only the neurons 21 of the first neural layer 20 are "
When outputting ``1'', output ``1'', otherwise output ``0''
Output. Further, when only the neuron 22 of the first neural layer 20 outputs rlJ, the neuron 32 outputs r1. is output, and "0" is output otherwise. Similarly, the neurons 33 are connected to the first neural layer 2
0 outputs "1" when only the neuron 23 outputs "1", and the neuron 34 outputs "1" when only the neuron 24 of the first neural layer 20 outputs "1".
" is output. On the other hand, neurons 91 of the output layer 90
outputs "1" when at least one neuron of the second neural layer 30 outputs rl.

Therefore, if the input data of ASB, C, and D has a bit pattern of rooo 1J, only the neuron 23 in the first neural layer 20 outputs "l", and the other neurons 21, 22, and 24 output "0". Output. As a result, the neuron 33 in the second neural layer 30 outputs "1", and the output layer 90
The neuron 91 outputs "1". Similarly, A,
B, CSD, rootoJ, ro 100. , rl.

00'', rllloJ, rllol,, ``1(11l'')
, ro 111J, the second neural layer 30
, one of the neurons outputs "l", and the neuron 91 of the output layer 90 outputs rlJ.

FIG. 7(b) shows a fifth embodiment of the invention, this device having an input data group 10, eleventh second and third neural layers 20, 30, 40 and an output layer 90. The input data group 10 consists of input data ASB and CSD. The first neural layer 20 has four neurons 21, 22, 23, 24, and the second neural layer 30
has two neurons 31, 32, the third neural layer 40 has two neurons 41, 42, and the output layer 90
has one neuron 91. The data A to D of the input data group are respectively input to each of the two neural layers of the first neural layer 20, and each of the two neural layers of the first neural layer 20 is connected to each of the two neural layers of the second neural layer 30. Ru. Second neural layer 3
Each neuron of the third neural layer 40 is connected to a neuron of the output layer 90, and each neuron of the third neural layer 40 is connected to a neuron of the output layer 90.

Each neuron in each neural layer 20, 30, 40 and output layer 90 outputs rlJ or "O" depending on the input data, as in the case of FIG. 7(a).

In the first neural layer 20, neurons 21
22.23.24 is "0IXXJ, rloxx" when A, B5C5D are expressed as 4-bit patterns, respectively.
, rxxol", rXXloJ, outputs rlJ, and otherwise outputs "0". On the other hand, in the second neural layer 30, the neuron 31 is the same as the neuron 21 or 22 of the first neural layer 20.
When it outputs "1", it outputs "0" otherwise. Further, the neuron 32 outputs rlJ when the neuron 23 or 24 of the first neural layer 20 outputs "l", and outputs "0" at other times. In the third neural layer 40, the neuron 41 outputs "l" when only the neuron 32 of the second neural layer 20 outputs "1", and otherwise outputs "0".
" is output. Further, the neuron 42 outputs "l" when only the neuron 31 of the second neural layer 20 outputs "l", and outputs "0" at other times. On the other hand, the neurons 91 of the output layer 90 indicate that at least one of the neurons of the third neural layer 40 is "l".
”, outputs “1”.

Therefore, if the input data of A, B, , C, D has a bit pattern of "00 (11"), only the neuron 23 in the first neural layer 20 outputs "l",
Other neurons 21, 22, and 24 output "0".

As a result, only the neuron 32 in the second neural layer 30 outputs "l", and the neuron 42 in the third neural layer 40 outputs "l". Therefore, the neuron 91 of the output layer 90 outputs "l". Similarly, A, B, C, D are ro
olo,,roloo",rlooo",rlllOJ
, rl 1(11J, rlollJ, "(In the case of 1111J, one of the neurons in the second neural layer 30 outputs "l", which causes one of the neurons in the third neural layer 40 to output "l". "1
", and the fourth neural layer 40
Either one of the neurons outputs "l". Therefore, the neuron 91 of the output layer 90 outputs "l".

As can be easily understood from FIG. 7(a), in the comparative example, the number of synapses is 36 and the number of neurons is 10. On the other hand, in this example, as understood from FIG. 7(b), the number of synapses is 30 and the number of neurons is 1O.

Shikashite, logical operations (AeB)e (COD)4. :Thus, it is understood that when input data is processed, 36 synapses were required in the comparative example, whereas 30 synapses are required according to the present example.

That is, according to this embodiment, the number of synapses is reduced by about 20%, and the same effects as described in the above embodiments can be obtained. In other words, by increasing the number of neural layers and setting the number of neurons in the neural layer constructed on the output layer 90 side to be less than or equal to the number of neurons in the preceding neural layer, the number of synapses can be reduced, and the number of synapses can be reduced. The memory capacity of the processing device can be reduced and the calculation speed can be improved.

FIGS. 8(a) and (b) show the sixth embodiment of the present invention together with a comparative example. In this example, input data is subjected to a logical operation ((A6)B)e (CeD)) 61)E (8)
Indicates the case where processing is performed according to the following.

FIG. 8(a) shows a comparative example, in which the input data group IO has five data A to E, the first layer 20 has 15 neurons, and the output layer 90 has one neuron. In the comparative example, each term obtained by expanding Equation (8) above is input to each two-neelon of the neural layer 20. Therefore, all data is processed as zero-order data.

On the other hand, FIG. 8(b) shows a sixth embodiment of the present invention, in which this device has an input data group 1o and a first, second, third and fourth neural layer 20°30.40 . 50 and an output layer 90.

The first neural layer 20 and the second neural layer 30 respectively perform (A e B ) and (Ce D ) 2 follow-up processing, and the third neural layer 4° performs (
(A69B) @ (CeD))
and a fourth neural layer 5o and an output layer 9
0, the final result is determined by ((A61)B)@(C6)D))eE.

As can be understood from the comparison between FIG. 8(a) and FIG. 8(b), in the comparative example, the number of synapses is 80 and the number of neurons is 16, whereas in the present example, the number of synapses is 32, and the number of neurons is 10. According to this embodiment, the number of synapses is reduced by about 40%, and the number of neurons is reduced by about 60%. Therefore, in this embodiment as well, effects similar to those described in each of the above embodiments can be obtained.

As can be understood from the above explanation, in the present invention, the layered structure consisting of the input data group, neural layer, and output layer is optimally configured according to the order required for data processing and the order of the input data. Input data needs to be input to an appropriate layer depending on its layer structure. Note that the degree here means the degree of abstraction of data or processing content, as described above.

Now, in order for a neuron to perform the processing described above, the weights must be determined to appropriate values through learning. Therefore, in this embodiment, the weights are changed exponentially over time, as will be described later. There are three methods for modifying the weights, as disclosed in Japanese Patent Application No. 63-297541, and these will be referred to herein as Mode (2), Mode (2), and Mode (2), respectively.

In mode (2), as shown in FIG. 9, the weight of a neuron is modified based on the output of that neuron.

This correction force method is effective when the target value of the output of each layer is clear.

Now, when the neuron produces an output in response to a certain input, and the output matches the target value or is sufficiently close to the target value, the relationship between the input and output at that time should be strengthened. This corresponds to increasing the weight of synapses given significant input. In mode ■, the target value of the output of each neuron is known in advance, so the output of each neuron is compared with the target value, and when the two match or are sufficiently close, for example, in the case of binary input, r1 is input. The weight of synapses that have been synapsed is increased.

In mode (2), as shown in FIG. 10, the weights of the neurons are modified based on the final output evaluation results. This correction method is effective when determining the processing content of the data processing apparatus from a global perspective. As an evaluation method in this mode, evaluation of Hamming distance or Pythagorean distance between the final output of the output layer and the target value, or sensitive evaluation is possible. As a result of this evaluation, if the output is one loss or sufficiently close to the target value, the input/output relationship at that time should be strengthened, and at that time, for example, the weight of each synapse to which "l" is input is increased.

Mode (2) is a weight modification method for a type of learning in which inputs are memorized as they are, and strengthens the relationship between the input and the output that originally occurred for that input. That is, in the configuration of FIG. 9, the weight of the synapse to which rlJ is input in the neuron that outputs "1" in response to that input is increased.

In modifying the weights in this manner, the inventors first assumed that changes in the weights of neurons were changes in membrane potential in living nerve cells. In other words, if the weights are set similar to the membrane potential of biological nerve cells, it is thought that the efficiency of learning in the data processing device will be extremely high, similar to that of biological brain cells. Furthermore, since the weight shows a change similar to the membrane potential, the change can be considered to be expressed by an exponential function, similar to a negative RLC circuit.

Therefore, the weight W is W−±ex as shown in FIG.
It is expressed as p(t) (9). However, t represents the learning time in each neuron, that is, the number of learning times.

In equation (9), when the synapse is excitatory type, the sign is 10, and the weight W starts from 0 and increases rapidly at first, as shown by the solid line I, and the weight changes as time passes from the start of learning. The amount decreases and approaches the maximum value W. On the other hand, if the synapse is an inhibitory type, the sign becomes -, and the weight W starts from 0 and quickly decreases at first, as shown by the solid line J. As time passes from the start of learning, the amount of change becomes smaller (becomes closer to the minimum value W).

Immediately after the start of learning, there is not much data correlation for that synapse, so the weight W at this time is set small;
Thereafter, as the data correlation increases, the weight W rapidly increases, which speeds up the convergence due to learning. On the other hand, if learning has progressed and the weight W has already become large, that synapse has sufficient data correlation in the learning up to that point. Therefore, if the weight W is changed unnecessarily, it will simply cause oscillation and impede the convergence in learning, but since the weight W is set so that it hardly changes, sufficient convergence can be achieved. can get.

Conventionally, inhibitory and excitatory neurons have been considered as the output characteristics of neurons, and in order to optimally place them in a data processing device, a detailed study is required that takes into account the processing content. The connection between the Narayon neuron and the excitatory neuron is complex. However, according to this embodiment, by simply selecting the sign of the weight W as + or - as a characteristic of one synapse, it is possible to arbitrarily realize an inhibitory type or an excitatory type. Therefore, the circuit configuration is simplified and the circuit is free. The degree increases. It has been well known since Rosenblatt (1958
-Year F. Rosenblatt Theperce
ptron: a probabilistic
a+odel forinforsation
storage and organization
in thebrain'', Psychology
cal Review '65: 386-408).

According to this embodiment, the efficiency of learning in the data processing device is improved, and the final output data can be converged and stabilized quickly. Also, as mentioned above, the weight W
Since inhibitory and excitatory properties can be obtained by simply changing the sign of , de.

- Increased degree of freedom in the circuit of the processing equipment.

Note that the temporal change in the weight W does not necessarily need to be determined exactly as an exponential function (for example, it may be approximated by a polygonal line, etc.).

〔Effect of the invention〕

As described above, according to the present invention, the number of synapses or neurons required for a given process can be reduced,
This makes it possible to reduce the memory capacity of the data processing device and improve processing speed. Furthermore, by temporally changing the weights, it is possible to make the data processing device realize learning of brain cells similar to that of a biological system, and improve learning efficiency.

[Brief explanation of drawings]

FIG. 1(a) is a conceptual diagram showing a third embodiment of the present invention, FIG. 1(b) is a conceptual diagram showing a fourth embodiment of the present invention, and FIG. 2 is a general neuron model. Conceptual diagram, Figure 3 is a conceptual diagram showing a general neuron model, Figure 4 (a)
is a conceptual diagram showing a comparative example, Figure 4(b) is a conceptual diagram showing another comparative example, and Figure 4(C) is a conceptual diagram showing another comparative example.
is a conceptual diagram showing the first embodiment of the present invention, FIG. 5(a) is a diagram showing 3x3 convolution in graphic processing, FIG. 5(b) is a diagram showing the state of graphic density of each pixel, FIG. 5(c) is a second embodiment of the present invention, and is a conceptual diagram showing an example of a data processing device that determines endpoints in graphic processing. FIG. FIG. 7(a) is a conceptual diagram showing a comparative example; FIG. 7(b) is a conceptual diagram showing a fifth embodiment of the present invention; FIG. 8(a) is a comparative diagram. A conceptual diagram showing an example. FIG. 8(b) is a conceptual diagram showing the sixth embodiment of the present invention. FIG. 9 is a conceptual diagram of a data processing device that performs mode ■ learning. FIG. 1O is a conceptual diagram showing mode H learning. A conceptual diagram of a data processing device that performs the following. FIG. 11 is a graph showing temporal changes in weights. IO...Data group 11-19, A-■...Data 20.30.40.50,90...Neural layer 21-24.31-34.41,42 91-93...Neuron 90 ...Output layer Figure 2 Figure 1 Figure 3 (i) (iV) (Vii) (ii) (V) (Viii) Figure (iii) (vi) 1υ Figure Figure Figure 3 Procedure amendment ( (Voluntary) May 17, 1999

Claims (15)

[Claims] (1) It has multiple neural layers in which neurons are installed in parallel to output data according to the comparison result of the sum of input data multiplied by a predetermined weight and a threshold value, and the output of one neural layer is sent to the next stage. A data processing device configured to input data to a neural layer, characterized in that the input data is directly input to the second and subsequent neural layers without passing through the preceding neural layer. Data processing equipment. (2) The level of abstraction (
Claim 1 characterized in that data of a higher order (hereinafter referred to as "higher order") is classified according to the order (hereinafter referred to as "order") and is directly input to a later stage neural layer. The data processing device described. (3) Claim 2, characterized in that data of the same classification is directly input to multiple neural layers.
The data processing device described in Section 1. (4) The data processing device according to claim 2, wherein the order of the output data of the neuron is considered to be one order higher than the order of the input data. (5) The data processing device according to claim 2, wherein data pairs connected by one operator are considered to have equal orders. (6) The data processing device according to claim 2, wherein the degree of each data is determined based on the highest possible degree of the data, and the data is classified based on this determination. (7) When the order of the input data input to the first stage neural layer is n_0, the order of the n-th stage (n≧2) neural layer is assumed to be (n_0+n-1) order, and this neural layer has the following: (n_0+n-1) The second claim characterized in that the following data is directly input.
The data processing device described in Section 1. (8) The data processing device according to claim 1, characterized in that the number of neurons in the neural layer decreases as the stage progresses. (9) The data processing device according to claim 1, wherein the final stage neural layer is an output layer that outputs output data to the outside of the data processing device. (10) When there is a neuron whose weight increases depending on certain input data, the data processing according to claim 1 is characterized in that when the input data is repeatedly input, the weight is changed exponentially. Device. (11) The data processing device according to claim 10, wherein the weight increases exponentially. (12) The data processing device according to claim 10, wherein the weight decreases exponentially. (13) The data processing device according to claim 10, wherein the initial value of the weight is zero. (14) The data processing device according to claim 10, wherein the exponential function is an exponential function. (15) The data processing device according to claim 10, wherein the exponential function is an approximation function of the exponential function.