JPH0567066A - Learning method and signal processor for neural network - Google Patents

Abstract

PURPOSE:To improve a learning capability by changing the arrangement sequence of one bit of each bit column of each connection coefficient of an excitability/suppressiveness relatively to the other bit, in every cycle of a learning. CONSTITUTION:The bit at a position indicated by a first pointer 89 is taken out from a memory 81 which stores the bit column of the positive connection coefficient. The bit at the position indicated by a second pointer 90 is taken out from a memory 82 which stores the bit column of the negative connection coefficient. The bit of the new positive connection coefficient is calculated by a new connection coefficient calculating circuit 88 by using those bits, and then the bit is stored at the bit position indicated by the second pointer 90 in the memory 81. In the same way, the bit of the new negative connection coefficient is calculated by the new connection coefficient calculating circuit 88, and then the bit is stored at the bit position indicated by the second pointer 90 in the memory 82. Thus, the arrangement sequence of each bit is relatively changed in every cycle of the learning.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network learning method and a signal processing device for a neurocomputer which is a parallel distributed information processing device aiming at artificially realizing the information processing function of a nerve cell network. ..

[0002]

2. Description of the Related Art The aim of parallel processing of information is to imitate the function of nerve cells (neurons), which are the basic units of information processing in the living body, and to use this "nerve cell mimicking element" as a network to process information in parallel. This is a so-called neural network. Although it is easy to perform character recognition, associative memory, motion control, etc. in a living body, there are many things that conventional Neumann computers cannot easily achieve. There have been many attempts to solve these problems by imitating a neural system of a living body, in particular, a function peculiar to the living body, that is, parallel processing, self-learning, etc. by a neural network.

First, a conventional neural network model will be described. FIG. 33 is a diagram showing a certain nerve cell unit A, and FIG. 34 is a network thereof. A ₁ , A ₂ , and A ₃ each represent a nerve cell unit. One nerve cell unit combines with many other nerve cell units to receive a signal, process it, and output it. In the case of FIG. 34, the network is hierarchical,
The nerve cell unit A ₂ receives a signal from the nerve cell unit A _{1 in the} previous layer (left side) and outputs it to the nerve cell unit A ₃ in the next layer (right side).

A more detailed description will be given. First, in the nerve cell unit A of FIG. 33, the degree of coupling between another nerve cell unit and its own unit is called a coupling coefficient, and the coupling coefficient of the i-th nerve cell unit and the j-th nerve cell unit. The coupling coefficient is generally represented by T _ij . Now, let's say that my nerve cell unit is the jth unit, and i
Assuming that the output of the nth nerve cell unit is y _i , T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes the input to its own unit. As described above, since one nerve cell unit is connected to many nerve cell units, the result of adding T _ij y _i for these units is Σ
T _ij y _i are the inputs to our neuronal unit in the network. This is called the internal potential and is represented by u _j .

[0005]

[Equation 1]

Then, the input is subjected to a non-linear process to obtain the output of the nerve cell unit. The function used at this time is called a nerve cell response function, and the sigmoid function as shown in equation (2) and FIG. 35 is used as the nonlinear function.

[0007]

[Equation 2]

When such a nerve cell unit is constructed in a network as shown in FIG. 34, each coupling coefficient T
By providing _ij and sequentially calculating equations (1) and (2), parallel processing of information becomes possible and a final output is obtained.

An example of such a network realized by an electric circuit is shown in FIG. This is disclosed in Japanese Patent Application Laid-Open No. 62-295188, and basically, a plurality of amplifiers 1 having an S-shaped transfer function, and the output of each amplifier 1 is input to the input of the amplifier of another layer. A resistive feedback network 2 is provided which is connected as shown by the dashed line. The input side of each amplifier 1 has a C connected by a grounded capacitor and a grounded resistor.
The R time constant circuits 3 are individually connected. Then, the input current _{I 1, I 2, ~,} I n is supplied to the input of the amplifier 1, the output is obtained from a set of these amplifiers 1 output voltage.

Here, the signal strength of the input and output to the network is represented by a voltage, and the strength of the coupling between the nerve cell units is determined by the resistance 4 (in the resistive feedback network 2) connecting the input / output lines between the cells. Is represented by the resistance value of each of the amplifiers 1, and the nerve cell response function is represented by the transfer function of each amplifier 1. That is, in FIG. 36, the plurality of amplifiers 1 have an inverting output and a non-inverting output, and the input of each amplifier 1 has a CR time constant circuit 3 as an input current supply means. A feedback network 2 connects the output of each of the amplifiers 1 to the input by a resistor 4 (T _ij ) which is a value of 1 or a preselected second value. Resistor 4 represents the transconductance between the i-th amplifier output and the j-th amplifier input and is chosen to create multiple minima that the network will balance to minimize the energy function with multiple minima. I am trying to In addition, the coupling between nerve cells has excitability and inhibitory property, and is mathematically represented by the sign of the coupling coefficient, but it is difficult to realize the sign with a constant on the circuit. Then, by dividing the output of the amplifier 1 into two and inverting one output, positive and negative 2
It is realized by generating two signals and selecting them appropriately. An amplifier is used as the one corresponding to the sigmoid function shown in FIG.

However, such an analog circuit system has the following problems. The signal strength is represented by an analog value such as electric potential or current,
When the internal calculation is also performed in an analog manner, its value changes due to temperature characteristics, drift immediately after power-on, and the like. Since it is a network, a large number of elements are required, but it is difficult to make the respective characteristics uniform. When the accuracy or stability of one element becomes a problem, it may cause a new problem when it is used as a network, and the behavior when viewed in the whole network cannot be predicted. Since the coupling coefficient T _ij is fixed and the value learned in advance by another method such as simulation is used, self-learning cannot be performed.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. 37 to 39. FIG. 37 shows a circuit configuration of a single nerve cell, in which each synapse circuit 6 is connected to a cell body circuit 8 via a dendrite circuit 7. FIG. 38 shows an example of the configuration of the synapse circuit 6 in which the input pulse f is multiplied by a factor a through the coefficient circuit 9.
A rate multiplier 10 to which a value obtained by multiplying (a multiplication factor of 1 or 2 by the feedback signal) is input is provided, and the rate multiplier 10 is connected to a synapse weight register 11 that stores a weighting value w. . Further, FIG. 39 shows a configuration example of the cell body circuit 8, and the control circuit 1
2, an up / down counter 13, a rate multiplier 14 and a gate 15 are sequentially connected, and an up / down memory 16 is further provided.

In this case, the input and output of the nerve cell unit is represented by a pulse train, and the pulse density represents the amount of signal.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal into the clock of the rate multiplier 14 and inputting the coupling coefficient into the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _{ij of the} backpropagation model equation.
It corresponds to the part of y _i . Next, the Σ portion of ΣT _ij y _i is realized by the OR circuit shown by the dendrite circuit 7. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and OR is taken for each group. An output is obtained by inputting these two outputs to the up side and down side of the counter 13 and counting. This output is 2
Since it is a decimal number, the rate multiplier 14 is used again to convert it into a pulse density. A neural network can be realized by making this unit a network. The learning is realized by inputting the final output to an external computer, performing numerical calculation inside the computer, and writing the result in the memory 16 of the coupling coefficient. Therefore, there is no self-learning function. Also, the circuit configuration is complicated because the pulse density signal is once converted into a numerical value (binary number) using a counter and then converted into the pulse density again.

As described above, in the case of the conventional technique, the analog circuit system is not reliable in operation, and the learning method by numerical calculation is also complicated in calculation, and is not suitable for hardware implementation. The circuit configuration is complicated. Further, there is a drawback that self-learning cannot be performed on hardware.

In order to solve such a defect, a digital neuron model with a self-learning function is disclosed in Japanese Patent Application No.
No. 4,12,448 and Japanese Patent Application No. 3-29342 have been proposed by the present applicant. A basic idea of a neural network having a neuron element structure using such a digital logic circuit having a self-learning function according to the already proposed method and various structural examples thereof will be described with reference to FIGS. The neural network that is the premise of the proposed example is a digital circuit equipped with a self-learning circuit having a coupling coefficient variable circuit and a coupling coefficient generating circuit that generates a variable coupling coefficient value of the coupling coefficient variable circuit based on an error signal for a teacher signal. It is configured by connecting a signal processing means composed of a plurality of nerve cell mimicking elements by a logic circuit in a mesh.

First, the neural network in the proposed example is implemented by hardware by a digital configuration, but the basic idea is to input / output signals, intermediate signals,
The coupling coefficient, the teacher signal, etc. are all represented by a binary pulse train of "0" and "1". The amount of signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed in the memory. Learning is realized by rewriting this pulse train. For learning, an error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1". It was done like this.

This idea will be described below. First,
Regarding signal processing by a digital logic circuit, signal processing in the forward process will be described. FIG. 5 shows a portion corresponding to one nerve cell unit (nerve cell mimicking element) 20, and the entire neural network is of a hierarchical type as shown in FIG. 6, for example. Input and output are all
The one that is binarized into “1” and “0” and synchronized is used. The intensity of the input signal y _i is represented by a pulse density, and is represented by the number of states of “1” within a certain fixed time as in the pulse train shown in FIG. 7, for example. That is, the example of FIG.
6, the signal is 4 "1" in 6 sync pulses,
There are two "0" s. At this time, it is desirable that the arrangement of "1" and "0" is random.

On the other hand, the coupling coefficient T _ij indicating the degree of coupling between the nerve cell units 20 is similarly expressed by pulse density,
The pulse train of "0" and "1" is prepared in advance in the memory. The example of FIG. 8 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of "1" and "0" is random.

Then, this pulse train is sequentially read from the memory in accordance with the synchronous clock, and as shown in FIG.
The ND gate 21 takes the logical product of the input signal pulse train (y _i ∩ T _ij ). This is used as an input to the nerve cell j. In the case of the above example, the input signal is “10110
When "1" is input, a pulse train is called from the memory in synchronism with this, and "101000" as shown in FIG. 9 is obtained by sequentially performing AND, which means that the input y _i is the coupling coefficient T _ij. And the pulse density is 2/6
It shows that it becomes.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal train is, the more "1"
The more random the arrangement of "0" and "0", the closer to the product of numerical values it has. Not random means
It means that "1" (or "0") is dense (close). When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no data to be read, the head of the data may be returned to and the reading may be repeated.

Since one nerve cell unit 20 has multiple inputs, there are also many "ANDs of the input signal and the coupling coefficient" described above, and then the OR circuit 22 takes the logical sum of these. Since the inputs are synchronized, for example, the first data is "101000" and the second data is "01000".
In the case of “0”, the OR of both is “111000”. If multiple inputs are simultaneously calculated and output, for example, when the number of inputs is m, the result is as shown in FIG. This corresponds to the sum calculation and the non-linear function (sigmoid function) in the analog calculation.

When the pulse density is low, the pulse density of the OR of the pulse density approximately matches the sum of the respective pulse densities. As the pulse density increases, the OR circuit 22
Since the output of is gradually saturated, it does not match the sum of pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, the coupling has excitability and inhibitory ability. In the case of numerical calculation, it is represented by the sign of the coupling coefficient. In the case of an analog circuit, when T _ij is negative (inhibitory coupling), an amplifier is used. It is used to invert the output and _connect it to another nerve cell unit with a resistance value corresponding to T _ij . In this respect, in the present embodiment of the digital system, first, each coupling is divided into two groups of excitatory coupling and inhibitory coupling according to the positive or negative of T _ij , and then “the pulse train of the input signal and the coupling coefficient is divided. The OR of “AND” is calculated for each group. And when only the output of the excitatory coupling group is "1",
When "1" is output and only the output of the inhibitory binding group is "1", "0" is output. When both are "1" or "0", either "1" or "0" may be output, or "1" may be output with a probability of about 1/2. Here, the output "1" is output only when the output of the excitatory coupling group is "1" and the output of the inhibitory coupling group is "0". In order to realize this function, the AND of the output of the inhibitory coupling group and the output of the excitatory coupling group may be ANDed. That is,
As shown in FIG. Expressed as a logical expression,
It is shown by the equations (3) to (5).

[0024]

[Equation 3]

The network of nerve cell units 20 is
Hierarchical type similar to backpropagation (ie, Figure 6)
And If the entire network is synchronized, each layer can be calculated by the functions described above.

On the other hand, each connection is divided into two groups, an excitatory connection and an inhibitory connection, _depending on whether T _ij is positive or negative.
OR between "AND of input signal and pulse train of coupling coefficient"
Is calculated for each group, and then the output of the excitatory coupling group is “0” and the output of the inhibitory coupling group is “1”.
In the case of outputting an output other than in the case of, the (NOT of output of inhibitory coupling group) and (output of excitatory coupling group) may be ORed.

Next, the signal calculation processing in learning (back propagation) will be described. Basically, the error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method of c.

First, the error signal in the final layer will be described as a. The error signal in each nerve cell unit is calculated in the final layer, and the coupling coefficient relating to that nerve cell unit is changed based on the error signal. The calculation method of the error signal for that is described. Here, the "error signal" is defined as follows. When the error is expressed numerically, it is generally +,-
However, in the case of pulse density, both positive and negative cannot be expressed at the same time.
Express the error signal using two types: the signal representing the component. That is, the error signal of the jth nerve cell unit is shown in FIG. In other words, the + component of the error signal is the difference between the teacher signal pulse and the output pulse (1,
0) or (0, 1) is a pulse existing on the teacher signal side, while the − component is a pulse existing on the output side as well. In other words, if the error signal + pulse is added to the output pulse and the error signal−pulse is removed, it becomes a teacher pulse. That is, these positive and negative error signals δ _{j (+)} ,
When δ _{j (-)} is expressed by a logical expression, it becomes the expressions (6) and (7), respectively. In the formula, EXOR represents an exclusive OR. The coupling coefficient is changed based on such an error signal pulse as described later.

[0029]

[Equation 4]

Next, a method of obtaining an error signal in the intermediate layer as b will be described. First, the above-mentioned error signal is back-propagated to change not only the coupling coefficient between the final layer and the layer immediately before it but also the coupling coefficient of the layer before that. Therefore, it is necessary to calculate the error signal in each nerve cell unit in the middle layer. Signals were propagated from the neuron unit with an intermediate layer to each neuron unit in the next layer, which is exactly the reverse of the procedure of collecting error signals in each neuron unit in the next layer. And use it as its own error signal. This can be performed in the same manner as the above-described arithmetic expressions (3) to (5) and the cases shown in FIGS. 7 to 11 in the nerve cell unit. However, the difference from the above-mentioned processing in the nerve cell unit is that y is one signal, while δ has two signals as positive and negative signals, and both signals are considered. It is necessary. Therefore, it is necessary to divide into four cases depending on whether the coupling coefficient T is positive or negative and the error signal δ is positive or negative.

First, the case of excitatory coupling will be described. In this case, for a nerve cell unit having an intermediate layer, the error signal + at the k-th nerve cell unit of the next layer (final layer in FIG. 6), the nerve cell unit and self (intermediate layer in FIG. 6) J-th neuronal cell unit) with the coupling coefficient AND (δ _{k (+)} ∩ T _jk ) is obtained for each neuron cell unit, and the OR between them is taken {∪ (δ _{k (+)} ∩ T _jk )}. This is the error signal δ _{j (+)} of this layer. That is, assuming that the number of nerve cell units in the layer one layer ahead is n, the result is as shown in FIG.

Similarly, the error signal δ of this layer is obtained by ANDing the error signal − and the coupling coefficient in the nerve cell unit of the layer one layer ahead and further ORing them.
_{j (-)} . That is, it becomes as shown in FIG.

Next, the case of inhibitory binding will be described. In this case, the AND of the error signal in the nerve cell unit of the layer one layer ahead and the coupling coefficient of the nerve cell unit and the self is taken, and further the OR between them is taken. This is the error signal δ _{j (+) of} this layer. That is, it becomes as shown in FIG.

Similarly, the AND of the error signal + which is one ahead and the coupling coefficient are ANDed, and the OR of these is taken to obtain the error signal δ _{j (-) of} this layer in the same manner. That is, it becomes as shown in FIG.

Since some nerve cell units are excitatoryly coupled to other nerve cell units and some are inhibitoryly coupled, the error signal δ obtained as shown in FIG. 13 is used. _{j (+)} and the error signal δ obtained as shown in FIG.
OR with _{j (+)} and use it as the error signal δ _{j (+) of} its own nerve cell unit. Similarly, the error signal [delta] _j determined as in FIG. 14 _(-) and the error signal [delta] _j determined as in FIG. 16 _(-)
Is taken as the error signal δ _{j (-) of} its own nerve cell unit.

The above is summarized as shown in equation (8).

[0037]

[Equation 5]

Further, a function corresponding to the learning rate (learning constant) may be provided. When the rate is 1 or less in the numerical calculation, the learning ability is further enhanced. This can be realized by thinning out the pulse train in the pulse train calculation. Here, a counter-like concept is adopted, and the ones shown in FIGS. 17 and 18 are adopted. For example, at the learning rate η = 0.5, every other pulse train of the original signal is thinned out, but even if the pulses of the original signal are not evenly spaced, the original pulse train can be thinned out. 17 and 18, in the case of η = 0.5, every other pulse is thinned out, in the case of η = 0.33, every two pulses are left, and in the case of η = 0.67, two pulses are left. It indicates that one thinning is performed every other time.

In this way, the function of the learning rate is provided by thinning out the error signal. Such thinning out of the error signal can be easily realized by logically operating the output of a counter commercially available or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

By the way, it is not always necessary to multiply the error signal by a learning constant. For example, it may be used only for the calculation for obtaining the coupling coefficient described below. Further, the learning constant for propagating the error signal in the opposite direction may be different from the learning constant used in the calculation for obtaining the coupling coefficient. This means that it is possible to individually set the characteristics of the nerve cell units provided with the network, and it is possible to construct an extremely versatile system. Therefore, it is possible to appropriately adjust the performance of the network.

Further, as c, a method of changing each coupling coefficient by such an error signal will be described. The signal flowing through the line (see FIG. 6) to which the coupling coefficient to be changed belongs and the error signal are ANDed (δ _j ∩ y _i ). However, in this embodiment, the error signal has two signals of + and −, and therefore, the error signals are respectively calculated and obtained as shown in FIGS. The two signals thus obtained are respectively denoted by ΔT
_{Let ij (+)} and ΔT _{ij (-)} .

Next, based on this ΔT _ij , a new T
_ij is obtained. Since this T _ij is an absolute value component, it is classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased with respect to the original T _ij , and ΔT
Reduce the components of _{ij (-)} . That is, it becomes as shown in FIG. On the contrary, in the case of the suppressive property, the component of ΔT _{ij (+)} is reduced and the component of ΔT _{ij (−)} is increased with respect to the original T _ij . That is, FIG.
As shown in 2.

The network is calculated based on the above learning rule.

Next, an actual circuit configuration based on the above algorithm will be described. 23 to 25 show examples of the circuit configuration, the configuration of the entire network is similar to that of FIG. FIG. 23 shows a circuit corresponding to a line (connection) in FIG. 6, and FIG. 24 shows a circuit corresponding to a circle (each nerve cell unit 20) in FIG. Further, FIG. 25 shows a circuit of a portion for obtaining an error signal in the final layer from the output of the final layer and the teacher signal. 23 to 25.
A digital neural network capable of self-learning can be realized by forming the network of the three circuits as shown in FIG.

First, FIG. 23 will be described. In the figure, 25 is an input signal to the nerve cell unit and corresponds to FIG. 7. The value of the coupling coefficient as shown in FIG. 8 is stored in the shift register 26. The shift register 26 has an outlet 26a and an inlet 26b, but may have the same function as a normal shift register, for example, R
It may be a combination of an AM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are ANDed by a logic circuit 28 having an AND gate 27 and performing the processing shown in FIG. The output of the logic circuit 28 must be divided into groups depending on whether the coupling is excitatory or inhibitory. However, it is better to prepare the outputs 29 and 30 for each group in advance and switch which one is output. It is highly versatile. Therefore, here, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31, and the switching gate circuit 32 switches using the information. The switching gate circuit 32 includes two AND gates 32a and 32b and an inverter 32c interposed at one input.

Further, as shown in FIG. 24, gate circuits 33a and 33b having a plurality of OR gate configurations for performing respective input processes (corresponding to FIG. 10) are provided. Furthermore, as shown in the figure, the excitatory coupling group shown in FIG. 11 is “1”,
A gate circuit 34 including an AND gate 34a and an inverter 34b that outputs an output "1" only when the inhibitory coupling group is "0" is provided.

Next, the error signal will be described. The error signal in the final layer is generated by the logic circuit 35 based on the combination of AND and exclusive OR shown in FIG. 25, which corresponds to equations (6) and (7). That is, the error signals 38 and 39 are produced by the output 36 from the final layer and the teacher signal 37. The processing as shown in FIGS. 13 to 16 for calculating the error signal in the intermediate layer is performed by the gate circuit 42 having the AND gate configuration shown in FIG. 23, and the output 4 according to + and − is output.
3,44 is obtained. As described above, it is necessary to classify the connection depending on whether it is excitatory or inhibitory. In this case, the information on excitatory or inhibitory stored in the memory 31 and the +,-signals 45 and 46 of the error signal are used. AND, OR depending on
This is performed by the gate circuit 47 having a gate configuration. The calculation formula (8) for collecting the error signals is performed by the gate circuit 48 having the OR gate structure shown in FIG. Further, the processing of FIGS. 17 and 18 corresponding to the learning rate is performed by the frequency dividing circuit 49 shown in FIG. Finally, a portion for calculating a new coupling coefficient from the error signal, that is, a portion corresponding to the processing of FIGS. 19 to 22 is performed by the gate circuit 50 having the AND, inverter and OR gate configurations shown in FIG. The contents of the shift register 26, that is, the value of the coupling coefficient is rewritten. This gate circuit 50 also needs to be classified depending on the excitability and inhibitory property of the coupling, but the gate circuit 47 performs this.

The grouping method and the output determining method shown in FIGS. 23 and 24 are extracted and shown in FIG.
become that way. That is, the input stage is not divided into groups, but a shift register 26 _{ij, which} is a memory storing coupling coefficients, is individually provided for each input 25 _ij , and the logical result by the AND gate 27 _ij is divided into the grouping memory 31.
The OR gate 33a is divided into two groups via the switching circuit 32 in accordance with
The OR gate 33b side obtains a logical sum, and the OR gate 33b side obtains a logical sum in the case of an inhibitory coupling group. After this, the gate circuit 3
The output is determined by the logical product process of 4.

Note that such a grouping method of excitatory coupling and inhibitory coupling may be configured as shown in FIG. 27, for example. That is, at the input stage, a group of excitatory coupling and a group of inhibitory coupling are grouped in advance, and a coupling coefficient T _ij is stored for each input 25 _ij . Is provided with a shift register 51. After that, the same processing may be performed for each group through the OR gates 33a and 33b. 52 is an AND gate.

FIG. 28 shows the gate circuit 34.
As shown in, the OR gate 34c may be used in place of the AND gate 34a to perform the OR operation.

Further, as shown in FIG. 29, a learning constant setting means 62 for arbitrarily and variably setting the learning constant used in the coupling coefficient variable circuit from the outside may be provided. That is, the learning constant (learning rate) used during learning shown in
Variable to obtain a network circuit with performance suitable for the application. The function of is added.

First, the learning constant setting means 62 is shown in FIG.
The counter 63, which is provided in place of the frequency dividing circuit 49 shown in FIG.
It is composed of OR gates 64 to 67 and one AND gate 68 which logically operate the output of the above to process the learning constant. More specifically, when all the switches Sa to Sd provided on the input sides of the OR gates 64 to 67 connected to the binary outputs A to D of the counter 63 are set to the H level side, η = 1.0.
When all the switches Sa to Sd are set to the L level side, η = 1/16. Therefore, if the number of switches on the H level side is N, then η = (2 to the Nth power) / 16. Therefore, the learning constant can be arbitrarily set by using the switch (or an external signal in place of the switch). When the pulse density is used as the clock input of the counter 63, the AND gate 69 may be appropriately provided for the input of the error signal. The learning constant setting means 62 is not limited to such a circuit configuration. Further, by providing a plurality of such learning constant setting means 62 or appropriately controlling by an external signal, the value of the learning constant used for the calculation of the coupling coefficient and the value of the learning constant used for the back propagation of the error signal. It is also possible to make different.

Further, it may be configured as shown in FIGS. That is, as described above, two types of coupling coefficients, excitatory and inhibitory, are prepared in the basic idea shown in to, and the calculation result for the input signal is the result of using each coupling coefficient. Increase the flexibility of the network by making a majority decision based on the ratio. The function of is added.

First, one nerve cell unit has two coupling coefficients of excitability and inhibitory property. The output result of "AND of input signal and coupling coefficient" is different from that of excitatory coupling. The processing is carried out at a ratio to the case of inhibitory binding. Here, processing with a ratio means the number of cases where the output result obtained by using the excitatory coupling coefficient is “1” and the inhibitory coupling coefficient for a plurality of input signals that are calculated in synchronization. Is compared with the number when the output result obtained by using is 1, and when the latter is greater than the former, "0" is output, and in other cases, the neuron unit outputs Means that. Alternatively, when both are the same, “0” may be output.

30 and 31 show examples of circuit configurations for this purpose. First, one memory for each input 25, specifically, the shift register 70a,
70b is provided. These shift registers 70
Like the shift register 26, a and 70b have a data outlet and a data inlet, but one shift register 70a stores the excitatory coupling coefficient and the other shift register 70b stores the inhibitory coupling coefficient. It was done. The contents sequentially read by the reading means (not shown) from these shift registers 70a and 70b are input 25.
Along with this, they are input to the corresponding AND gates 71a and 71b and the logical product is obtained. There are two kinds of such logical results: excitatory and inhibitory, but here
It is input to the majority decision circuit 72 and the output is determined. That is,
The digital signal of the operation group using the excitatory coupling coefficient based on the shift register 70a is added by the amplifier 73a. Similarly, the digital signal of the operation group using the excitatory coupling coefficient based on the shift register 70b is processed by the amplifier 73b. The addition processing is performed, and the size of both is decided by the comparator 74 by majority. The majority circuit 72
Is not limited to the illustrated example, and may be a general majority circuit.

Here, the grouping method shown in FIG. 24 is extracted and shown as shown in FIG. That is, the coupling coefficient of excitatory coupling and inhibitory coupling is stored for each input 1
A set of memories (shift register) 70 is prepared, and processing is performed until a logical product is obtained for each group divided by the set of memories.

In the example shown in FIG. 32, instead of the majority decision circuit 72, OR gates 33a and 33b and below for obtaining the logical sum for each group are shown as in the case of FIG. 26 and FIG. The gate circuit 34 in this case may also be configured as shown in FIG. 27 or 28.

By the way, in FIG. 32, since each input 25 has one set of shift registers 70a and 70b, rewriting of the coupling coefficient by the self-learning function is also performed for each shift register 70a and 70b. Therefore, in FIG.
As shown in 0, a self-learning circuit 75 that performs the processing of FIGS. 13 to 16 and the equation (8) for calculating a new coupling coefficient using the + and − error signals is provided, and each shift register It is connected to the data entrance side of 70a and 70b. According to this method, the connection of the nerve cell units is not limited to excitatory or inhibitory, so that the network has flexibility and has versatility in actual applications.

The frequency dividing circuit 49 in the case of FIG. 31 may also be replaced with the learning constant setting means 62 as shown in FIG.

The output decision system by the majority decision circuit 72 is not limited to the system having two memories (shift registers 70a and 70b) for each input as shown in FIG. 30, but one for each input. The same can be applied to the one having the memory 26. That is, instead of the combination of FIG. 23 and FIG. 24, the combination of FIG. 23 and FIG. 31 may be used.

[0061]

However, in the pulse density type hierarchical neural network as in the above-mentioned proposed example, the excitatory coupling coefficient (positive coupling coefficient) and the inhibitory coupling coefficient (negative coupling coefficient) in the learning process. The logical operation including both the coupling coefficient) is performed, and in order for these to exert the intended effect, the AND operation has a function of producing the product of the pulse densities, and the OR operation shows the sum of the pulse densities. It is necessary to show the action to make. For that purpose, it is necessary that the bit arrangement of the bit string of the positive coupling coefficient and the bit arrangement of the bit string of the negative coupling coefficient have no correlation. In fact
If the bit arrangements of the bit string A having the pulse density a and the bit string B having the pulse density b are irrelevant to each other (that is, the positions in which the bits in the bit string A are turned on and the bits in the bit string B are turned on). If A∪B is executed many times, the resulting pulse density will be a value close to the sum of a and b on average, and A∩B will be repeated many times. When executed, the resulting pulse density has a value close to the product of a and b on average.

However, if there is a correlation between the bit arrangements of the bit strings A and B, for example, A = “01010101” and B
When the on-bit positions of both of them tend to overlap with each other, as in “= 00010001”, the pulse density of A∪B is not the sum of a and b but a value close to a, and the pulse density of A∩B is a. It becomes a value close to b instead of the product of and b. Therefore, when the correlation of the on-bit position between the bit string of the positive coupling coefficient and the bit string of the negative coupling coefficient becomes large, the learning ability is deteriorated.

As described above, when there are two types of coupling coefficients, positive and negative, for each coupling, the bit arrangement of the bit string of the positive coupling coefficient and the bit arrangement of the bit string of the negative coupling coefficient are correlated. It is possible for learning to proceed in the direction of. For example, as learning progresses, the number of combinations in which the on-bit position of the bit string of the positive coupling coefficient and the on-bit position of the bit string of the negative coupling coefficient match, or conversely, the positive coupling coefficient This is because it is possible that the number of couplings in which the on-bit position of the bit string of 1 and the off-bit position of the bit string of the negative coupling coefficient match is increased.

In such a case, the progress of learning may stop before the learning reaches a desired level. Further, the learning effect is lost if the bit string of the coupling coefficient after learning is slightly shifted for some reason.

[0065]

According to a first aspect of the present invention, a neural network is constructed by hierarchically connecting a plurality of neurons which process an input signal of bit string representation by a digital logic operation and output an output signal of bit string representation. And the weighting data for weighting the input signal when one neuron receives the output signal from another neuron as an input signal is used as the coupling coefficient of the bit string representation, and the excitatory coupling coefficient and the inhibitory property for each coupling. In the neural network learning method for changing the coupling coefficient value stored in the neuron so that the output value from the neural network approaches a desired value. One of a bit string of excitatory coupling coefficient and a bit string of inhibitory coupling coefficient for each cycle The arrangement order of the bits of the bit Retsunai was so changed relative to the arrangement order of the bits of the other bit Retsunai.

According to the second aspect of the present invention, as a signal processing device using such a learning method, a plurality of neurons for processing an input signal represented by a bit string by a digital logic circuit and outputting an output signal represented by the bit string are provided. A neural network is formed by connecting hierarchically, and weighting data for weighting an input signal when a neuron receives an output signal from another neuron as an input signal is used as a coupling coefficient of a bit string representation for each coupling. A coefficient is provided for storing two kinds of excitatory coupling coefficient and inhibitory coupling coefficient, and a coefficient for varying the coupling coefficient value stored in the neuron in order to bring the output value from the neural network close to a desired value. A variable circuit is provided, and the bit string of the excitatory coupling coefficient and the suppression coefficient are provided in the coefficient variable circuit for each cycle of the learning. Provided bit rearrangements means for changing relative either the arrangement order of the bits of one bit Retsunai with respect to the arrangement order of the bits of the other bit Retsunai of the bit string of the coupling coefficient of sex.

[0067]

According to the learning method of the first aspect of the invention or the signal processing apparatus of the second aspect, the bit string of the excitatory coupling coefficient is compared with the bit string of the inhibitory coupling coefficient for each learning cycle. Since it is shifted by 1 bit or more relatively, the bit arrangement correlation between the bit string of the excitatory coupling coefficient and the bit string of the inhibitory coupling coefficient is canceled, and the learning coefficient before the desired coupling coefficient value is reached. It is possible to prevent the progress from being stopped, and the learning effect of the neural network becomes highly resistant to fluctuations in the bit arrangement of the coupling coefficient after learning, and the learning ability is improved.

[0068]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to FIGS. The pulse density type hierarchical neural network or the neuron configuration is based on the previously proposed example and the like, and the same parts as those shown in FIGS. 5 to 32 are denoted by the same reference numerals. In this embodiment, as a method for dealing with excitatory coupling and inhibitory coupling, the excitatory coupling coefficient and the inhibitory coupling coefficient are stored in the memory 70 for each coupling as shown in FIG.
_{It is} based on the method of _aij and 70 _bij .

In this embodiment, learning proceeds in a direction in which the bit arrangement of the bit string of the positive coupling coefficient and the bit arrangement of the bit string of the negative coupling coefficient have a correlation, whereby the learning of the pulse density type hierarchical neural network progresses. The learning effect has a strong tolerance to fluctuations in the on-bit position of the coupling coefficient after learning, and the learning ability is enhanced.

Basically, in the learning process,
For each cycle, shift the bit string of the positive coupling coefficient by 1 bit or more with respect to the bit string of the negative coupling coefficient, or conversely, shift the bit string of the negative coupling coefficient by 1 bit or more with respect to the bit string of the positive coupling coefficient. Thus, it is possible to prevent the bit arrangement of the bit string of the positive coupling coefficient and the bit arrangement of the bit string of the negative coupling coefficient from proceeding in a direction having a correlation.

That is, there are several possible methods for preventing the progress of the correlation between bit strings.
In the present embodiment, as described above, the bit arrangement order of the bit string of the positive coupling coefficient and the bit string of the negative coupling coefficient is relatively shifted for each cycle of learning, for example, relatively shifted by 1 bit or more, which is extremely simple. This is realized by any method. In this case, the bit strings of the positive and negative coupling coefficients may be shifted from each other by 1 bit or more at the time when all the bits of the bit string of the coupling coefficient in the memory containing the coupling coefficient have been updated. If the bit of the positive or negative coupling coefficient is shifted by 1 bit or more every time the bit of 1 is processed, the procedure becomes simpler and the circuit therefor becomes simpler.

Next, a specific learning method according to this embodiment will be described. First, using the nth bit of the bit string of the positive combination coefficient and the (n + 1) th bit of the bit string of the negative combination coefficient, the bit of the new positive combination coefficient is calculated by the logical operation as described in the first embodiment. And the value of the new negative coupling coefficient bit. Or, conversely, by using the (n + 1) th bit of the bit string of the positive combination coefficient and the nth bit of the bit string of the negative combination coefficient, a new positive combination is performed by the logical operation as described in the first embodiment. Compute the bit value of the coefficient and the bit value of the new negative combination coefficient. When storing these new coupling factors in memory,
The value of the bit of the new positive combination coefficient is stored in the nth bit position of the bit string of the positive combination coefficient, and the value of the bit of the new negative combination coefficient is stored in the nth bit position of the bit string of the negative combination coefficient To do. Such processing is performed by n = 1, 2,
3, and so on, in the direction in which the value of n increases, and when the processing of all the bits of the bit string of the coupling coefficient is completed, the processing returns to the beginning and the same processing is repeated.

Alternatively, not limited to the above method, for example, by using the nth bit of the bit string of the positive combination coefficient and the nth bit of the bit string of the negative combination coefficient, a new positive value is obtained by the logical operation as described above. Compute the bit value of the coupling coefficient and the new bit value of the negative coupling coefficient. When storing these calculated new combination coefficients in memory, the value of the bit of the new positive combination coefficient is stored in the nth bit position of the bit string of the positive combination coefficient, but the bit of the new negative combination coefficient is stored. The value of is stored in the (n-1) th bit position of the bit string of the negative combination coefficient. Or, conversely, the value of the bit of the new positive combination coefficient is stored in the (n-1) th bit position of the bit string of the positive combination coefficient, but the value of the bit of the new negative combination coefficient is the bit string of the negative combination coefficient. It is stored in the n-th bit position of. Such a process is repeated in the direction in which the value of n increases, such as n = 1, 2, 3, ..., When all the bits of the bit string of the coupling coefficient have been processed, the process returns to the beginning and the same process is performed. May be repeated.

According to such a learning method, the bits of the positive and negative combination coefficients that are updated at the same time are not used at the same time when they are used in the next learning cycle. Therefore, the bit arrangement of the bit string of the positive combination coefficient is set. It is possible to prevent the learning from proceeding in the direction in which the bit arrangement of the bit string of the negative coupling coefficient and the bit arrangement of the negative coupling coefficient have a correlation. In the above description, the bit string is relatively shifted by 1 bit, but the bit string is not limited to 1 bit, and the point is that it may be 1 bit or more.

Further, the shift of the bit string does not have to be a constant value. That is, where n and i are integers, the nth bit of the bit string of the positive combination coefficient and the nth bit of the bit string of the negative combination coefficient
When calculating the value of the bit of the new coupling coefficient using the + i-th bit and n of the bit string of the negative coupling coefficient
When calculating the value of the bit of the new combination coefficient using the n-th bit and the n + i-th bit of the bit string of the positive combination coefficient, or when calculating the value of the n-th bit of the bit string of the positive combination coefficient and the negative combination coefficient. When the value of the bit of the new coupling coefficient calculated using the n-th bit of the bit string is stored in the ni position of the positive or negative coupling coefficient, if i does not become a constant value and n changes, May also change. However, in this case, it is necessary to prevent overlap between n + i and n-i. That is, for two values, n ₁ and n ₂ ,
It should be ensured that n ₁ + i ₁ and n ₂ + i ₂ do not have the same value, or that n ₁ -i ₁ and n ₂ -i ₂ do not have the same value. In addition, as another method, the point is that the bit arrangement of the bit strings of the positive and negative coupling coefficients does not have correlation, so that one of the bit strings of the positive and negative coupling coefficients is not shifted with respect to the other bit string. Alternatively, only one of the bit strings may be rearranged. For example,
Each time the learning progresses by one cycle, the order of the bits of the bit string of the positive or negative coupling coefficient is reversed, or the bit string is divided into the first half and the second half, and the first half and the second half of these are combined. By replacing and, the bits of the bit string of either the positive or negative coupling coefficient may be rearranged.

Such a learning method is realized by the circuits illustrated in FIGS. In these illustrated examples, the constituent elements are common. First, a memory 81 corresponding to the memory 70 _aij and storing a bit string of positive coupling coefficients
And a memory 82 storing a bit string of negative coupling coefficients corresponding to the memory 70 _bij . Bit storage circuits 83, 84 and bit extraction circuits 85, 86 are attached to the memories 81, 82, respectively. On the output side of the bit extraction circuits 85 and 86, together with the output calculation circuit 87,
The new coupling coefficient calculation circuit 88 is connected, and the output side of the new coupling coefficient calculation circuit 88 is fed back to the memories 81 and 82 through the bit storage circuits 83 and 84, respectively, and can be updated and rewritten. The output calculation circuit 87 has, for example, the circuit configuration shown in FIG. 32, and the new coupling coefficient calculation circuit 88 has, for example, the circuit configuration shown in FIG. 24 (however, in FIG. Through Figure 4
Memory 81, 82 shown outside circuits 87, 88 in FIG.
_Are included in each circuit as shown by memories 70 _aij and 70 _bij ).

Further, a first pointer 89 and a second pointer 90 for specifying the bit position on the bit string in the memories 81 and 82 are provided. Here, the value of the first pointer 89 (bit pointing position) is set to be 1 or more larger than the value of the second pointer 90 (bit pointing position), and the positive and negative coupling coefficients are each 1 bit, totaling 2 bits. Each time the updating is completed, the values of these pointers 89 and 90 are incremented by one. 1 to 4, the positions of these pointers 89 and 90 and the connection relationship are different. These pointers 89 and 90 serve as bit rearrangement means.

First, in the method of FIG. 1, the bit at the position indicated by the first pointer 89 is fetched from the memory 81 storing the bit string of the positive coupling coefficient, and the bit string of the negative coupling coefficient is extracted. The bit at the position indicated by the second pointer 90 is fetched from the stored memory 82. The new coupling coefficient calculation circuit 88 is used by using these extracted bits.
After calculating the bit of the new positive coupling coefficient with, the bit is stored in the memory 81 at the bit position pointed to by the second pointer 90, and the new coupling coefficient calculating circuit 88 is also stored.
After calculating the bit of the new negative coupling coefficient by, the bit is stored in the bit position designated by the second pointer 90 in the memory 82.

In the system shown in FIG. 2, the second pointer 90 is read from the memory 81 storing the bit string of the positive combination coefficient.
Of the position pointed to by the first pointer 89 from the memory 82 storing the bit string of the negative coupling coefficient. After calculating the new positive and negative coupling coefficient bits by the new coupling coefficient calculation circuit 88 using these extracted bits,
The bit is stored in the memory 81, 82 at the bit position pointed to by the second pointer 90.

In the system of FIG. 3, the bit at the position indicated by the first pointer 89 is fetched from the memory 81 storing the bit string of the positive combination coefficient and the bit string of the negative combination coefficient is obtained. First among the stored memories 82
The bit at the position indicated by the pointer 89 is taken out. A new coupling coefficient calculation circuit 8 using these extracted bits
After calculating the bit of the new positive coupling coefficient by 8, the bit is stored in the memory 81 at the bit position pointed to by the first pointer 89, while the new coupling coefficient calculating circuit 8
After calculating the bit of the new negative coupling coefficient by 8, the bit is stored in the memory 82 at the bit position indicated by the second pointer 90.

In the method of FIG. 4, the bit at the position pointed to by the first pointer 89 is taken out from each of the memories 81 and 82 storing the bit strings of positive and negative coupling coefficients,
After calculating the bit of the new positive coupling coefficient by the new coupling coefficient calculation circuit 88 using these extracted bits,
The bit is stored in the memory 81 at the bit position pointed to by the second pointer 90, while the new negative coupling coefficient bit is calculated by the new negative coupling coefficient calculation circuit 88.
The bit is stored in the memory 82 at the bit position indicated by the first pointer 89.

Next, a learning example according to the method of this embodiment will be described by a concrete experimental example. Here, in the hierarchical neural network which has the function of the present embodiment and has three layers, and each layer is composed of three neurons, a set of three input signals and a teacher signal Input signal Teacher signal (1 , 0,0) → (1,0,0) (0,1,0) → (0,1,0) (0,0,1) → (0,0,1) After being presented 2000 times for learning, these three types of input signals are presented 1000 times each and the number of ON bits in the bit array of the positive and negative coupling coefficients is not changed for each presentation. When the output signal was sampled while randomly changing only the bit positions that were turned on, for all three types of input signals,
The output value distribution in which the mode value is a value close to each teacher signal value is obtained.

By the way, as a comparative example, in the same neural network as the above experimental example except that the function of this embodiment is not provided, that is, the bit string of the coupling coefficient is not shifted in the learning process, the same learning as the above experimental example is performed. After that, the same output signal as in the above experimental example was sampled. As a result, the output value from the neuron whose output value should be 1 and the output value from the neuron whose output value should be 0 were almost the same average value. The output value distribution has a spread and the learning effect is lost.

[0084]

Since the present invention is configured as described above, according to the learning method of the first aspect of the invention or the signal processing apparatus of the second aspect, excitatory coupling is performed every learning cycle. Since the arrangement order of the bits in one of the bit string of the coefficient and the bit string of the inhibitory combination coefficient is changed relative to the arrangement order of the bits in the other bit string, excitability It is possible to eliminate the correlation of the bit arrangement between the bit string of the coupling coefficient of and the bit string of the inhibitory coupling coefficient, and thus it is possible to prevent the learning from stopping before reaching the desired level, and It is possible to improve learning ability by making the learning effect highly resistant to fluctuations in the bit arrangement of the coupling coefficient after learning.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram.

FIG. 3 is a block diagram.

FIG. 4 is a block diagram.

FIG. 5 is a logic circuit diagram for performing basic signal processing in an already proposed example.

FIG. 6 is a schematic diagram showing a network configuration example.

FIG. 7 is a timing chart showing an example of logical operation.

FIG. 8 is a timing chart showing an example of logical operation.

FIG. 9 is a timing chart showing an example of logical operation.

FIG. 10 is a timing chart showing an example of logical operation.

FIG. 11 is a timing chart showing an example of logical operation.

FIG. 12 is a timing chart showing an example of logical operation.

FIG. 13 is a timing chart showing an example of logical operation.

FIG. 14 is a timing chart showing an example of logical operation.

FIG. 15 is a timing chart showing an example of logical operation.

FIG. 16 is a timing chart showing an example of logical operation.

FIG. 17 is a timing chart showing an example of logical operation.

FIG. 18 is a timing chart showing an example of logical operation.

FIG. 19 is a timing chart showing an example of logical operation.

FIG. 20 is a timing chart showing an example of logical operation.

FIG. 21 is a timing chart showing an example of logical operation.

FIG. 22 is a timing chart showing an example of logical operation.

FIG. 23 is a logic circuit diagram showing a configuration example of each unit.

FIG. 24 is a logic circuit diagram showing a configuration example of each unit.

FIG. 25 is a logic circuit diagram showing a configuration example of each unit.

FIG. 26 is a logic circuit diagram showing a configuration example of each unit.

FIG. 27 is a logic circuit diagram showing a modified example.

FIG. 28 is a logic circuit diagram showing a modified example.

FIG. 29 is a circuit diagram showing a different configuration example.

FIG. 30 is a circuit diagram showing a further different configuration example.

FIG. 31 is a circuit diagram.

FIG. 32 is a circuit diagram.

FIG. 33 is a conceptual diagram showing one unit configuration showing a conventional example.

FIG. 34 is a conceptual diagram of the neural network configuration.

FIG. 35 is a graph showing a sigmoid function.

FIG. 36 is a circuit diagram showing a specific configuration of one unit.

FIG. 37 is a block diagram showing a digital configuration example.

FIG. 38 is a circuit diagram of a part thereof.

FIG. 39 is a different partial circuit diagram.

[Explanation of symbols]

 54-69 coefficient variable circuit 71, 72 memory 89, 90 bit rearrangement means

Claims (2)

[Claims] 1. A neural network is formed by hierarchically connecting a plurality of neurons that output bit-string representation output signals by processing a bit-string representation input signal by digital logic operation, and one neuron from another neuron is formed. The weight data for weighting the input signal when receiving the output signal as the input signal is stored as the coupling coefficient of the bit string representation, and two types of the coupling coefficient of excitability and the coupling coefficient of inhibition are stored for each coupling. In a learning method of a neural network in which a coupling coefficient value stored in the neuron is changed in order to bring an output value from the neural network close to a desired value, a bit string of excitatory coupling coefficient in each learning cycle. And the bit string of the inhibitory coupling coefficient, the bit arrangement order of the bit in one bit string is A learning method for a neural network, characterized in that the bit arrangement order is changed relative to the bit arrangement order. 2. A neural network is formed by hierarchically connecting a plurality of neurons each of which outputs a bit string representation output signal by processing a bit string representation input signal by a digital logic circuit, and one neuron from another neuron is formed. A memory for storing two kinds of excitatory coupling coefficient and inhibitory coupling coefficient for each coupling as weighting data for weighting the input signal when receiving the output signal as the input signal is used as the coupling coefficient of the bit string representation. A coefficient variable circuit for varying the coupling coefficient value stored in the neuron in order to bring the output value from the neural network close to a desired value is provided, and in the coefficient variable circuit, for each cycle of the learning, The arrangement order of bits in one of the bit string of excitatory coupling coefficient and the bit string of inhibitory coupling coefficient is set to other. A signal processing device comprising: a bit rearrangement unit that relatively changes the arrangement order of bits in the other bit string.