JP2001344589A - Number of ignition distribution control type neural circuit device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a neural circuit device for further accurately reflecting the featured value of an input by realizing an expression configuration that ignited neurons are spatially dispersed more uniformly without being biased to any specific part. SOLUTION: This device is provided with a first neuron layer 101 which receives an input signal, a second neuron layer 102 as an intermediate layer, and a third neuron layer 103 as an output layer. Then, the number of neurons to be simultaneously ignited in the second neuron layer 102 is controlled by a first value designated by a first control means (CYA) 109 corresponding to the second neuron layer, and the number of neurons to be ignited in ignition neuron blocks 104, 105, and 106 in the second neuron layer 102 is controlled by a second value set in a second control means (CYB) 110 corresponding to the second neuron layer. Thus, it is possible to ignite the neurons by spatially dispersing them.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural circuit device used for a learning type pattern recognition device such as character recognition, voice recognition, image recognition and the like, and a robot autonomous operation control device.
In particular, the present invention relates to a firing-number-dispersion control type neural circuit device in which firing neurons are controlled to be spatially dispersed.

[0002]

2. Description of the Related Art Perceptrons, recurrent networks, Hopfield networks, neocognitrons, backpropagation methods, feature map methods, and the like are known as architectures for realizing neural circuit devices using neuron elements having a learning function. Various models have been proposed. Considering that these models are applied to various applications, a considerable number of elements are required, and it is necessary to harden neuron elements to solve the problem of the quantity. Hardening of neuron elements is realized by a method of realizing individual functions using analog circuits, a method of describing functions by programs using existing processors as basic circuits, or a method of digitally configuring by incorporating functional circuits. Have been.

In the former system based on an analog circuit, a neuron element for processing an input signal performs a Gilbert operation of multiplying a plurality of input signals by a weight value for each input signal as a state function of the neuron. This is performed using an analog multiplication circuit or an addition circuit such as an amplifier, and the obtained product-sum output is compared with a threshold value held by each neuron element by a comparison circuit such as a current mirror, and the comparison result is used as a sigmoid function or the like. And output the result. See Carver Mea
d) published in 1989 by "Analog VLSI
and Neural Systems ”(translated by Shiro Usui and Hiroo Yonezu“ Analog VLSI and Neural Circuit Systems ”published in 1993) and“ Neural ”published by the Institute of Electronics, Information and Communication Engineers in 1996 by Atsushi Iwata and Yoshihito Amamiya. Network LSI "and the like.

On the other hand, in the latter system based on a digital processor circuit, each processing such as a neuro-product-sum function, a comparison function, and a function conversion function is described by a program, and is divided and distributed to a plurality of processors to perform parallel processing. Is calculated.

Further, some functions specific to the neuron element, such as the product-sum function and the pulse output function of the neuron element, are realized by an analog circuit, the obtained analog output is subjected to analog-to-digital conversion, and the remaining functions are converted to digital. An eclectic method of processing is also being attempted.

From the viewpoint of practical application, an unsupervised learning architecture model that operates by learning a task is preferable because it can be adaptively operated for various specific applications. The architectural model of the unsupervised learning method is based on the activity states of the neuron elements stored in the neural network, that is, based on the information expression form of the neural network. Divided into The local expression is described by a single winner logic, and has the advantage that self-organizing learning is easy, but has a problem that it is difficult to increase the scale and that additional learning is difficult. Distributed representation has also been confirmed in neurophysiology, with efficiency, generalization ability, scalability,
It has the advantage of high robustness and easy application to specific applications. However, in the conventional distributed expression model, it is necessary to normalize the weight value using a complicated procedure in order to prevent the divergence or disappearance of the weight value. Sometimes, the design of the model is complicated because a plurality of local representations are combined, and it is difficult to expand the model later.
It was pointed out that it did not go beyond the mere superposition of local expressions.

[0007] In contrast to such a conventional distributed expression model, a more efficient distributed expression form is based on a multi-layered neural network with high efficiency and scalability. Japanese Unexamined Patent Publication No. Hei.
The present inventor of the present application has proposed the present invention in 1-338844.

FIG. 8 shows a firing-rate control type neural circuit device described in JP-A-11-338844. The firing number control type neural circuit device includes an input layer LI for receiving an input signal, an output layer LN for outputting a final output, and a single intermediate layer L in FIG.
H and a three-layer hierarchical structure composed of H.

Although the number of neurons in the input layer LI depends on the number of input signals, in FIG. 8, six neurons I0 to I5 receive input signals XX0 to XX5, respectively. Each of the neurons I0 to I5 is composed of all the neurons H0 to H5 in the intermediate layer LH.
The output is given to H5. The number of neurons in the intermediate layer LH depends on the number of categories to be recognized. Here, six neurons H0 to H5 are used. Each of the neurons H0 to H5 receives the output of the neuron I0 to I5 of the input layer LI at the previous stage and supplies the output to all the neurons of the output layer LN at the next stage. The number of neurons in the output layer LN depends on the number of categories to be recognized. Here, six neurons N0 to N5 are used. Each neuron of the output layer LN is connected to the preceding intermediate layer LH
, And outputs final outputs YY0 to YY5.

The neurons in the input layer LI are driven by the power supply VI, the neurons in the intermediate layer LH are driven by the power supply VH, and the neurons in the output layer LN are driven by the power supply VN.

The control means C1 determines whether the number of neurons firing among the neurons of the intermediate layer LH, that is, the number of neurons whose sum of weighted inputs exceeds a threshold value and becomes active, is determined by a specified value or a range of values. Thus, the activity of the neurons H0 to H5 is controlled. Similarly, control means C
Numeral 2 controls the activities of the neurons N0 to N5 so that the number of neurons firing and activating among the neurons in the output layer LN becomes a specified value or a range of values. When the number of active neurons is insufficient, the control means C1 and C2 perform an excitatory connection to each neuron and continuously increase the input level to the neurons until reaching the desired number of activities, or By lowering the threshold value at which each neuron outputs until it reaches, the input level to the neuron is equivalently increased.

In this firing number control type neural circuit device, the threshold level is adjusted by changing the weight coefficient in accordance with the input, and the firing number of the neuron that is simultaneously excited for each layer or over a plurality of layers is determined. By controlling, distributed representation can be easily realized without performing complicated system design.

[0013]

As described above, the firing number control type neural circuit device described in Japanese Patent Application Laid-Open No. H11-338844 has a problem that the calculation at the time of learning of the previous distributed expression model is complicated, and that the neural network device at the time of designing is difficult. It gives a good solution to the problem of complexity of model configuration and subsequent scalability, and the problem of superposition of local expressions.

However, Japanese Patent Application Laid-Open No. 11-338844
In the firing rate control type neural circuit device described above, the number of active neurons is comprehensively specified for each layer, so that firing neurons may be condensed in one place. Had not been. For this reason, when an expression form that is spatially and uniformly distributed, which is essential for distinguishing and accurately recognizing various kinds of recognition targets, cannot be realized, that is, the form is biased to one or several places, and each feature of the recognition target is In some cases, the intended expression form distributed to the user may not be realized.

An object of the present invention is to provide an image forming apparatus as disclosed in JP-A-11-33884.
In addition to inheriting the advantage of the firing number control type neural circuit device described in No. 4, the expression form in which the firing neurons are more evenly distributed without being spatially biased to a specific portion, and more accurately reflect the input feature amount. An object of the present invention is to provide a neural circuit device.

Another object of the present invention is to provide a neural circuit device whose circuit configuration can be adaptively expanded for various applications, which can be easily integrated into a LSI, and whose configuration is simple.

[0017]

According to a first aspect of the present invention, there is provided a neural network device for controlling the number of fires distributed, comprising a plurality of neurons forming a plurality of neuron layers, First control means for controlling the number of neurons activated and activated at the same time to be restricted depending on two or more first specified values or value ranges for each neuron layer; Is controlled such that the number of neurons contained in the firing neuron cluster consisting of neurons that are spatially adjacent and fire and are active at the same time is restricted depending on two or more second specified values or value ranges. And second control means.

A neural network device according to a second aspect of the present invention is a neural network device in which a plurality of neurons form a plurality of neuron layers. First control means for controlling each of the neuron layers so as to be constrained depending on two or more first specified values or value ranges, and at least one or more fired and active elements in the neuron layer And a winner neuron around the central neuron so that the number of firing neurons that simultaneously activate and extract the firing neuron mass consisting of the firing neurons and the designated firing neuron mass is 2 or more. And the second control means for performing learning control on neurons spatially adjacent to the winner neuron.

In the first or second invention, at least one neuron layer of the plurality of neuron layers may be composed of a plurality of blocks each having a plurality of neurons, and further comprises this neuron layer. Even if the plurality of blocks are controlled by a common first control unit in which only a first specified value or a range of values is set, a first specified value corresponding to each block is set. Alternatively, a range of values may be set. Further, even if the plurality of blocks constituting the neuron layer are controlled by a common second control means in which only a second specified value or a range of values or a specified shape is set, The second specified value or range of values or the specified shape may be independently controlled by a plurality of second control means set in correspondence with the block.

The plurality of blocks constituting the neuron layer may be provided in a form having no spatial overlap, or may be provided in a form having a spatially overlapping area between the blocks.

Further, at least two neuron layers constituting a continuous stage among the plurality of neuron layers are composed of a plurality of blocks each having a plurality of neurons, and each of the neurons in each block in the preceding stage is in the subsequent stage. It may be connected to almost all neurons of each block. Even if each block does not have a spatial overlap, even if each block of the preceding neuron layer has a spatially overlapping area between the blocks, each block of the subsequent neuron layer Even if the blocks do not have a spatial overlap, each block of the subsequent neuron layer may have a spatially overlapping region between the blocks.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be specifically described below with reference to the drawings showing the embodiments.

(First Embodiment) FIG. 1 is a diagram showing a first embodiment of a firing-number-dispersion control type neural circuit device according to the first embodiment of the present invention. The firing-number-dispersion control type neural circuit device according to the first embodiment includes a first neuron layer 101 receiving an input signal, a second neuron layer 102 as an intermediate layer,
The third neuron layer 103 as an output layer, the first control means (CYA) 109 for the second neuron layer, and the second control means (CYB) 11 for the second neuron layer
0 and the first control means (CZ
A) 111 and second control means (CZB) 112 for the third neuron layer.

In the first neuron layer 101, n neurons X _{1 to} X _n are arranged in an array, and in the second neuron layer 102, m neurons Y _{1 to} Y _m are similarly arranged in an array. , the p number of neurons Z ₁ to Z _p are arranged similarly to the array in the third neuron layer 103. Although not shown to avoid complication, the output of the n neurons X ₁ to X _n of the first neuron layer 101, each of the m second neuron layer 102 of neurons Y ₁ to Y _m are inputted to all, as well as the output of the m neurons Y ₁ to Y _m of the second neuron layer 102,
Each is input to all of the _p neurons Z _{1 to} Z _p of the third neuron layer 103.

In the first embodiment, the number of neurons that can be fired simultaneously in the second neuron layer 102 is CYA1.
The range of the first value NY1 designated as the value 09 is set to 10 to 20, and similarly, the first value designated to the CZA 111 as the number of neurons capable of firing simultaneously in the third neuron layer 103 is set.
The description will be made assuming that the range of the value NZ1 is 10 to 20.

When the learning is completed in the first embodiment, the state is as follows. When a certain input is shown in the first neuron layer 101, in the second neuron layer 102, the number yo1 of neurons fires in the region of the firing neuron mass 104, and the number yo2 of neurons fires in the region of the firing neuron mass 105, In the region of the firing neuron mass 106, the number yo3 of neurons fire. The output in the state of the second neuron layer 102 is connected to the third neuron layer 103. In the third neuron layer 103, the number zo1 of neurons is fired in the region of the firing neuron mass 107 and the number of neurons in the region of the firing neuron mass 108. The zo2 neuron fires.

Assuming that the range of the second value NY2 set in the CYB 110 as the number of neurons to fire in each of the firing neuron blocks 104, 105, and 106 of the second neuron layer 102 is 3 to 5, the firing neuron blocks 104, 105 The number of firing neurons 105 and 106 depends on the designation of the second value NY2, for example, yo1 = 3, yo2 =
5, yo3 = 4, and the total number of firing neurons ys in the second neuron layer 102 is equal to the second neuron layer 10
The first value NY1 is in the range of 10 to 20. ys = yo1 + yo2 + yo3 = 12 Similarly, the CZB 112 in the third neuron layer 103
Assuming that the range of the second value NZ2 to be set is 4 to 9, the number of neurons firing in the third neuron layer 103 depends on the second value NZ2, for example, zo1 = 6
Zo2 = 8, and the total number of firing neurons zs in the third neuron layer 103 is equal to the third neuron layer 10
The first value NZ1 of 3 is in the range of 10 to 20. z
s = zo1 + zo2 = 14 Next, the second neuron layer 1
The CYA 109 which is the first control means of the C.02 will be described in detail. FIG. 2 is a diagram illustrating the configuration of the CYA 109. The CYA 109 has an activity detection unit 201 that counts the number of fired neurons in the second neuron layer 102 and an activity adjustment unit 204 that adjusts the activity of the neurons in the second neuron layer 102 based on the activity detection result. I have.

The activity detecting section 201 has two firing number counting neurons, and one of them has a minimum value detecting neuron (DMI).
N) 202, one is a maximum value detection neuron (DMAX)
Then, the firing of the neurons in the second neuron layer 102 is controlled using the outputs of both detection neurons. That is,
When the first value NY1 is set in the range of 10 to 20, the DMIN 202 receives signals from all the neurons in the second neuron layer 102 and fires when less than 10 of them are firing. . Also, DMAX2
Numeral 03 receives signals from all neurons in the second neuron layer 102 and fires when more than 20 of them are firing. When the DMIN 202 is firing, the activity adjusting unit 204 sends the second neuron layer 102
Learning to increase the input weight value of all neurons in
When the DMAX 203 is firing, the activity adjustment unit 20
4 learns to reduce the input weight values of all neurons in the second neuron layer 102. Second neuron layer 10
By repeating learning until the number of firing neurons in 2 falls within the range of 10 to 20, the number of firing neurons can be kept within the range of the first value NY1.

The configuration and operation of the first control means CYA 109 of the second neuron layer 102 have been described above.
First control means CZA111 of third neuron layer 103
The same applies to.

Next, the CYB 110 as the second control means of the second neuron layer 102 will be described. CYB1
10, first, each firing neuron mass 104, 105,
The number of firing neurons at 106, 107 and 108 is detected. After the output of the firing neuron is displayed as a binary map on a plane, the binary data is logically processed to extract a region where the firing neurons are close to each other. The number of firing neurons is counted for each extracted region, and CY
Until the number falls within the range of the second value NY2 specified in B110, the input weight value of each neuron in the extracted region is increased or decreased to adjust the threshold level equivalently.
The same learning is performed by updating the weight values of the neurons in the region obtained by expanding the extracted region to the periphery by an equal width and expanding the region.

The second control means CYB110 of the second neuron layer 102 has been described.
The same applies to the second control means CZB112 of No. 03.

(Second Embodiment) In the first embodiment, the CYB 110 only controls the number of firing neurons in the firing neuron block to be in the range of the specified second value NY2. However, the firing neurons in a certain firing neuron mass do not usually have a neatly formed shape, but have a shape such as a partial chip or a thread. In addition, learning by the CYB 110 that limits the number of neurons to be fired as a firing neuron block to a second specified value range, and C that controls the number of neurons fired in the entire second neuron layer 102 are performed.
In some cases, it is not easy to match learning with BA109.

FIG. 3 is a diagram showing a second example of another neural circuit in which the firing number distribution control according to the second embodiment is performed. First
In order to obtain consistency between the designated value and the second designated value, the following simple algorithm is introduced as a second embodiment. Since the shape of the firing neuron mass is preferably a square shape or a circular shape, the square shape is simply used here as the shape of the designated firing neuron mass in which the number of neurons firing simultaneously is two or more.

In FIG. 3, compared to FIG. 1 of the first embodiment, the second control means of the second neuron layer 102 of the second embodiment is changed to CYB 310,
4, 305, and 306 are all transformed into a square firing neuron mass, and the third neuron layer 10
3 is that the second control means is changed to CYB 312, and the firing neuron masses 307 and 308 are all transformed into square firing neuron masses. The total number of firing neurons ys in the second neuron layer 102
And the total number of firing neurons zs in the third neuron layer 103 is ys = yo1 + yo2 + yo3 = 12, zs = zo1 + zo2 = 13, and ys is a range 1 of the first value NY1 of the CYA 109.
0-20, zs is the first value NZ of CZA111
1 is within the range of 10 to 20.

The operation algorithm of the CYB 310 according to the second embodiment will be described using the second neuron layer 102 as an example.
First, the CYB 310 determines, as a central neuron, a firing neuron that produces the largest output in a collapsed firing neuron cluster such as the firing neuron cluster in FIG. 1 and has a target shape around the central neuron. Define a square. A neuron located at the apex of the determined square is extracted as a winner neuron, and reinforcement learning for increasing the input weight value is performed. Together with this reinforcement learning, the Hebbian learning rule is applied to neurons in the immediate vicinity of the winner neuron. Perform emphasis learning by Subsequently, side suppression learning is also performed on the neuron immediately outside the neuron on which the emphasis learning is performed in the immediate vicinity of the winner neuron by the Hebbian learning rule, and the input weight value reduction learning is performed. The selection of the neuron in the immediate vicinity and the outer neuron in the immediate vicinity is performed on the basis of the distance between the neurons, and when the distance between the neurons is equal, the one with the larger output is selected as the nearest neuron. Since it is possible to increase or decrease the number of firing neurons in each firing neuron block in the process of changing the shape of the firing neuron block into a square by learning, the first specified value and the second specified value are It can be easily matched.

Specifically, the emphasis learning and the side suppression learning can be easily realized by multiplying the Mexican hat type value shown in FIG. In the reinforcement and emphasis learning based on the Hebbian learning rule, the weight value is increased only for the synapse receiving the input signal, and the learning of feature extraction is further strengthened. Enhancement learning function representing the lateral inhibition learning, the product of the exponential and trigonometric functions, more easily realized by transform of a function obtained by shifting the origin of the function to positive and negative, for example, f ₁ shown in FIG. 4 (b) (X), can be realized by a DOG function given by f ₂ (x).

The function for determining whether each neuron fires may be a conventional sigmoid type nonlinear function, and the neurons in each layer are fully connected to the neurons in the preceding layer.

(Third Embodiment) FIG. 5 is a block diagram showing a configuration of a neural network device according to a third embodiment of the present invention.
FIG. In the third embodiment, neurons included in the same neuron layer are divided into two or more blocks.

In FIG. 5, the second neuron layer is composed of three blocks 423, 424, and 425. The arrangement of neurons in the first neuron layer 421 and the third neuron layer 422 is the same as in the first embodiment of FIG.

The block 423 of the second neuron layer includes m
The neurons Y _{11 to} Y _1m are arranged, and the block 424
, M neurons Y _{21 to} Y _2m are arranged, and the block 425 includes m neurons Y _{31 to} Y _3m arranged in an array. Blocks 423, 424, 42
The inputs of the five neurons are all connected in parallel to all the neurons X _{1 to} X _n of the first neuron layer 421 at the preceding stage.

In the third embodiment, when the learning is completed, the state is as follows. An input is applied to the first neuron layer 42
1, in the block 423 of the second neuron layer, the number yo 11
Are fired, the number yo21 of neurons are fired in the area of the fired neuron mass 432 in the block 424, and the fired neuron mass 433 in the block 424 is also fired.
The number yo22 of neurons fires in the region of.

Blocks 423 and 42 of the second neuron layer
The output of 4,425 is input to all neurons of the third neuron layer 422. In the third neuron layer 422,
In the region of the firing neuron mass 434, the number zo1 neurons fire, and in the region of the firing neuron mass 435, the number zo2 neurons fire. Firing neuron mass 431, of each block of the second neuron layer
The second value NY2 designating the number of neurons to fire in 432, 433, 434, and 435 is set to 3 to 5 as in the first embodiment. The CYB 402, which is the second control means for the entire second neuron layer, commonly controls the blocks 423, 424, and 425 depending on the second value NY2. Further, the CYA 401, which is the first control means for the entire second neuron layer, controls the blocks 423, 424, and 425 commonly depending on the second value NY1. For example, yo11 = 3, yo21 = 5, yo22
= 4, the blocks 423, 4 of the second neuron layer
The total number of firing neurons ys at 24 and 425 is given by the following equation and is within the range of the first specified value. ys = yo11 + yo21 + yo22 = 12 The CZA 403 performs the first operation on the third neuron layer 422.
The CZB 404 is a second control unit for the third neuron layer 422. The detection and control of the number of firing neurons in each firing neuron mass 431, 432, 433, 434, 435 is the same as in the first embodiment. In the third embodiment, as in the first embodiment, the learning is performed by the Hebb learning rule for the nearest neuron to the winner neuron and the same for the neurons outside the winner neuron. Perform side suppression learning.

In the third embodiment, as in the second embodiment, reinforcement and emphasis learning based on the Hebbian learning rule are performed, and at the same time, the number of neurons to be fired is 2 or more. The shape of the neuron mass can be transformed into a square shape or the like.

(Fourth Embodiment) FIG. 6 is a block diagram showing a fourth embodiment of the configuration of the fired-number-dispersion control type neural circuit device according to the fourth embodiment.
FIG. In order to avoid complication of the drawing, FIG. 6 shows only a portion related to the second neuron layer, but the first neuron layer and the third neuron layer are the same as those in FIG. In the fourth embodiment, a neuron included in the same neuron layer is divided into two or more blocks, and the first specified value or range and the second specified value or range are stored in the second neuron. It can be provided independently for each block of the neuron layer.

In the fourth embodiment shown in FIG.
43 is the first value NY1a set for this block
Is controlled by the CYAa 405 so that the number of firing neurons in the block 443 becomes 6 to 15. The block 444 is controlled by the CYAb 407 based on the first value NY1b set for this block so that the number of firing neurons in the block 444 becomes 8 to 20. Also, the block 445 is based on the first value NY1c set for this block, and
In step 09, the number of firing neurons in the block 445 is controlled to be 9 to 22.

Similarly, for the second set value, the block 443 includes the second value NY set for this block.
Block 443 by CYBa 406 based on 2a
Is controlled so that the number of firing neurons included in the firing neuron block in the block becomes 3 to 5. Block 444 is
Based on the second value NY2b set for this block, the CYBb 408 controls the number of firing neurons included in the firing neuron cluster in the block 444 to be 3 to 5. Further, the block 445 performs CYB based on the second value NY2c set for this block.
By c410, the number of firing neurons included in the firing neuron block in the block 445 is controlled to be 3 to 5.

When the learning is completed in the fourth embodiment, the state is as follows. When an input is presented to the first neuron layer (not shown), the second neuron layer block 44
3, the number yo in the area of the firing neuron mass 451
Eleven neurons fire and a firing neuron mass 452
The number yo12 of neurons fire in the area of 、, and the number yo2 in the area of the firing neuron mass 453 in the block 444.
One neuron fires, the number yo22 of neurons fires in the area of the firing neuron mass 454, and the number yo in the area of the firing neuron mass 455 in the block 445 as well.
31 neurons fire and 32 yo neurons fire in the area of the firing neuron mass 456. The output of each neuron of the blocks 443, 444 and 445 of the second neuron layer is input to all the neurons of the third neuron layer (not shown).

Firing neuron masses 451, 452, 453, 454, 4 in respective blocks of the second neuron layer
The number of neurons firing at 55 and 456 is, for example, yo11 = 3 and yo12 = 4 depending on the second value NY2a of the block 443.
Yo2 = 5, yo2 depending on the second value NY2b of 4
2 = 4, and yo31 = 5 and yo32 = 4 depending on the second value NY2c of the block 445.

The total number of firing neurons ys1 in the block 443 of the second neuron layer is ys1 = yo11 + yo12 = 7, and the first value NY1a specified in the block 443 is
In the range of 6 to 15 pieces.

Similarly, the total number of firing neurons ys2 in the block 444 is ys2 = yo21 + yo22 = 9, and the first value NY1b specified in the block 444
In the range of 8 to 20.

Similarly, the total number of firing neurons ys3 in the block 445 is ys3 = yo31 + yo32 = 9, and the first value NY1c specified in the block 445 is
In the range of 9 to 22.

Each firing neuron mass 451, 452, 45
The detection and control of the number of firing neurons in 3,454,455,456 are the same as in the first embodiment. In the fourth embodiment, as in the first embodiment, learning is performed by the Hebb learning rule for the nearest neuron to the winner neuron, and the same for the neurons outside the winner neuron. Perform side suppression learning.

In the fourth embodiment, as in the second embodiment, reinforcement and emphasis learning based on the Hebbian learning rule are performed, and at the same time, the number of neurons to be fired is 2 or more. The shape of the neuron mass can be transformed into a square shape or the like.

If the first values NY1a, NY1b, NY1c specified for the blocks 443, 444, 445 are the same, the first control means C
YAa405, CYAb407 and CYAc409 are shown in FIG.
May be combined into one first control unit as in the case of CYA 401 in this case. In this case, only the second specified value or range is independent for each block of the second neuron layer. Can be set to

The second values NY2a, NY2b, NY2 specified for the blocks 443, 444, 445
When c is the same, CYBa 406, CYBb 408, and CYBc 410, which are the first control means, respectively.
May be shared as one CYB 402 in FIG. 5 to be integrated into one second control unit.
Can be set independently for each block of the second neuron layer.

(Fifth Embodiment) FIG. 7 is a diagram showing a fifth embodiment of the configuration of the neural network device of the distributed firing control type according to the fifth embodiment.
FIG. In the fifth embodiment, the neuron layer included in the layer is divided into two or more blocks, and the neuron layer preceding the neuron layer is divided into two partial regions. ing.

In FIG. 7, the second neuron layer is divided into a block 463 and a block 464, and the first control means CYA411 and the second control means CYB4
12 is controlled. The configuration is the same as that of the second neuron layer in FIG. 5 except that the number of blocks is 2.

The input layer of the first neuron layer is a block 4
The block 460 includes neurons X _{11 to} X _1n arranged in a matrix,
Block 461 includes neurons X ₂₁ -X _2n also arranged in a matrix. The block 460 and the block 461 may be partially overlapped with each other in the boundary area 465.

Each neuron in the block 463 of the second neuron layer is fully coupled to each neuron in the block 460 of the first neuron layer to receive an input, and each neuron in the block 464 of the second neuron layer is The input is fully coupled with each neuron in the block 461 of one neuron layer and receives an input. Therefore, the neurons of each block of the second neuron layer are connected to different input layer neurons except for the boundary region 465 of the first neuron layer which is the input layer.

The number and arrangement of neurons in the third neuron layer 462 are the same as those in the first embodiment, and CYA4 which is the first control means for the second neuron layer is used.
CZA 413 as the first control means for the first designated value NY1 set to 11 and the third neuron layer
, The first specified value NZ1 is set to 10
Up to 20 pieces.

In the fifth embodiment, when the learning is completed, the state is as follows. When an input is shown in blocks 460 and 461 of the first neuron layer,
In the block 463 of the neuron layer, the number yo11 of neurons fires in the region of the firing neuron mass 471, the number yo21 of neurons fires in the region of the firing neuron mass 472 in the block 464, and the number yo22 in the region of the firing neuron mass 473. Neurons fire. The outputs of the blocks 463 and 464 of the second neuron layer are connected to all the neurons of the third neuron layer 462. In the third neuron layer 462, the number zo1 of neurons fires in the region of the firing neuron mass 474, and the number zo in the region of the firing neuron mass 475.
Two neurons fire.

The number of neurons that fire in the firing neuron masses 471, 472, and 473 in each block of the second neuron layer is specified by a second designated CYB 412, which is the second control means for the second neuron layer. The value NY2 and the second specified value N to be set in the second control means CZB414 for the third neuron layer
Although it is determined depending on Z2, it is assumed that both NY2 and NZ2 are set to 3 to 5.

The number of neurons firing in the second neuron layer is, for example, yo11 = 3, yo21 = 5, yo22
Assuming that = 4, the total number of firing neurons ys in the second neuron layer is ys = yo11 + yo21 + yo22 = 12, which is within the range of the first specified value NY1.

Each firing neuron mass 471, 472, 47
The detection and control of the number of firing neurons in 3,474,475 are the same as in the first embodiment. In the fifth embodiment, as in the first embodiment, the learning is performed by the Hebb learning rule for the nearest neuron to the winner neuron, and the same for the neurons outside the winner neuron. Perform side suppression learning.

In the fifth embodiment, in the same manner as in the second embodiment, reinforcement and emphasis learning based on the Hebbian learning rule are performed, and at the same time, the number of neurons to be fired is two or more designated firings. The shape of the neuron mass can be transformed into a square shape or the like.

It is to be noted that, based on the configuration described in the first to fifth embodiments, a configuration in which each neuron layer is divided into a plurality of neurons, a first specified value or a range of values is specified. In various configurations, such as constraining across layers, constraining a second specified value or range of values across layers, and controlling and learning the firing neuron mass in different ways. Clearly, it can be extended.

[0067]

According to the present invention, it is possible to reduce the complicated calculation at the time of learning, to solve the problem of the complexity of the model configuration design and the subsequent expandability, and to solve the problem that it is merely a combination of local expressions. JP-A-11-33884
In addition to inheriting the basic characteristics of the firing rate control type neural circuit device described in No. 4, the expression form in which the firing neurons are more evenly distributed without being spatially biased to a specific portion can be obtained. The quantity can be grasped and recognized more accurately.

Further, by setting the number of neuron activities in the competing layer to two or more, the number of recognizable categories is increased, and the required number of neurons is prevented from diverging. By reducing the number of neurons and controlling the number of neurons to be fired to a value or range of values designated for each neuron layer, the circuit constituting the neural circuit device can be simplified. Further, since the circuit configuration can be adaptively expanded for various applications, integration can be easily performed.

[Brief description of the drawings]

FIG. 1 is a diagram showing a first example of a firing-number-dispersion control type neural circuit device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a first control unit of a neuron layer.

FIG. 3 is a diagram showing a second example of the firing-number-dispersion control type neural circuit device according to the second embodiment.

FIG. 4A is an example of a Mexican hat type weight value, and FIG. 4B is an example of a DOG function.

FIG. 5 is a diagram of a third example showing the configuration of the firing number distribution control type neural circuit device according to the third embodiment.

FIG. 6 is a diagram of a fourth example showing the configuration of the firing number distribution control type neural circuit device according to the fourth embodiment.

FIG. 7 is a diagram of a fifth example showing the configuration of the firing number distribution control type neural circuit device according to the fifth embodiment.

FIG. 8 is a firing number distribution control type neural circuit device described in JP-A-11-338844.

[Explanation of symbols]

101,421 first neuron layer 102 second neuron layer 103,422,462 third neuron layer 104,105,106,431,432,433,4
51, 452, 453, 454, 455, 456, 47
1,472,473 Firing neuron mass (of the second neuron layer) 107,108,434,435,474,475
Firing neuron mass (of the third neuron layer) 109,401,405,407,409,411
First control means (for the second neuron layer) 110, 310, 402, 406, 408, 410, 4
12 Second control means (for the second neuron layer) 111, 403, 413 First control means (for the third neuron layer) 112, 312, 404, 414 Second control means (for the third neuron layer) 423 , 424, 425, 443, 444, 445, 4
63,464 (constituting the second neuron layer) blocks 460,461 (constituting the first neuron layer) LI input layer LH intermediate layer LN output layer C1, C2 control means

Claims (22)

[Claims] 1. A neural circuit device comprising a plurality of neurons forming a plurality of neuron layers, wherein the number of neurons activated and activated simultaneously is at least two for each neuron layer. First control means for performing control so as to be constrained depending on a value or a range of values, and a firing neuron cluster comprising neurons which are spatially adjacent to each other in the neuron layer and fire and activate simultaneously. And a second control means for controlling the number of neurons to be restricted depending on two or more second specified values or a range of values. 2. A neural circuit device comprising a plurality of neurons forming a plurality of neuron layers, wherein the number of neurons activated and activated simultaneously is two or more for each neuron layer. First control means for performing control so as to be constrained depending on a value or a range of values, a firing neuron cluster comprising at least one firing neuron which fires and activates in the neuron layer, and the firing neuron cluster A winner neuron is set around the central neuron so that the number of firing neurons simultaneously active is 2 or more and the designated firing neuron mass is formed, and the winner neuron is spatially adjacent to the winner neuron. And a second control means for performing learning control on the neuron. 3. The neural network device according to claim 1, wherein at least one of the plurality of neuron layers comprises a plurality of blocks each having a plurality of neurons. 4. The firing according to claim 3, wherein said plurality of blocks constituting the neuron layer are controlled by a common first control means in which only one first specified value or value range is set. A number-distributed control type neural circuit device. 5. The plurality of blocks constituting the neuron layer are independently controlled by a plurality of first control means in which a first specified value or a range of values is set corresponding to each block. 4. The neural circuit device according to claim 3, wherein the number of firings is controlled. 6. The plurality of blocks constituting a neuron layer are controlled by a common second control means in which only one second specified value or range of values or a specified shape is set. Item 3. The number-of-fires distributed control type neural circuit device according to item 3. 7. A plurality of second control means, wherein the plurality of blocks constituting the neuron layer have a second specified value or a range of values or a specified shape corresponding to each block. 4. The neural network device according to claim 3, wherein the neural network device is independently controlled by: 8. The firing number distribution according to claim 3, wherein each neuron included in the plurality of blocks constituting the neuron layer is connected to substantially all neurons included in a neuron layer preceding the neuron layer. Control type neural circuit device. 9. The firing number distribution according to claim 3, wherein each neuron included in the plurality of blocks constituting the neuron layer is connected to substantially all neurons included in a neuron layer subsequent to the neuron layer. Control type neural circuit device. 10. The fire-number-dispersion control type neural circuit device according to claim 3, wherein the plurality of blocks constituting the neuron layer are provided in a form having no spatial overlap. 11. The neural network device according to claim 3, wherein the plurality of blocks constituting the neuron layer are provided in a form having spatially overlapping regions between the blocks. 12. At least two neuron layers constituting a continuous stage among the plurality of neuron layers are composed of a plurality of blocks each having a plurality of neurons, and each of the neurons of each block of the preceding stage is arranged in the subsequent stage. 3. The neural network device according to claim 1, wherein the neural network device is connected to substantially all neurons of each block. 13. The control circuit according to claim 12, wherein each block of the preceding neuron layer is provided in a form having no spatial overlap. 14. The control circuit according to claim 12, wherein each block of the preceding neuron layer is provided in a form having a spatially overlapping area between the blocks. 15. The neural network device according to claim 12, wherein each block of the subsequent neuron layer is provided in a form having no spatial overlap. 16. The control circuit according to claim 12, wherein each block of the subsequent neuron layer is provided in a form having a spatially overlapping area between the blocks. 17. The first control means promotes the activity when the number of neurons active in the neuron layer is small with respect to the first set value or range of values, and promotes the activity when the number is large. Claim 1 or 2 suppresses activity
The number-of-fires distributed control type neural circuit device as described in the above. 18. The activity control system according to claim 1, wherein the first control means detects the number of activated neurons in each neuron layer, and promotes the activity when the number of neurons is small based on the detection result of the activated number detector. 3. The neural network device according to claim 1, further comprising: an activity adjusting unit that suppresses the activity when the number of fires is large. 19. A minimum value detection neuron which activates by detecting that the number of firing neurons is less than a minimum value of a range specified in a neuron layer, and an activity number detection unit which detects that the number of firing neurons is less than a minimum value of a range specified in the neuron layer. A maximum value detection neuron that detects and activates when the maximum value is exceeded, and when the minimum value detection neuron is in an active state, the activity of all neurons in the neuron layer is promoted, and the maximum value detection neuron is activated. 19. The neural network device according to claim 18, which acts to suppress the activity of all the neurons in the neuron layer when the value is in. 20. The method according to claim 19, wherein the first specified value is the second specified value.
Is greater than or equal to twice the specified value of
2. The control apparatus according to claim 1, wherein the maximum value in the specified value range is at least twice the minimum value in the second specified value range. 21. The neural network device according to claim 1, wherein the second specified value or range of values can be individually set for each of the firing neuron masses. 22. An apparatus according to claim 21, wherein the designated shape of the firing neuron mass can be individually set for each of the firing neuron masses.