JPH04246766A - Neural network - Google Patents

Abstract

PURPOSE:To set the complex number as a processing object by a simple circuit configuration by providing an arithmetic means for executing an arithmetic processing related to the complex number instead of a neuron. CONSTITUTION:A neural network 1 inputs a complex number being a component of a complex number pattern of an inputted recognition object to neurons 11-1n. The neurons 11-1n transmit the inputted complex number and a real part of a parameter managed by each one to an arithmetic means 2. The arithmetic means 2 calculates a complex number outputted to the neuron of the downstream side from an imaginary part of the stored parameter, the real part of the parameter obtained from each neuron, and a complex pattern. Subsequently, a result of calculation is transmitted to the neurons 11-1n. The neurons 11-1n output the complex number obtained from the arithmetic means 2 in accordance with coupling between the neurons to other neuron.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to neural networks.

0002

2. Description of the Related Art A neural network is a circuit in which neurons having a plurality of learning functions are connected in a network manner for numerical calculation and pattern recognition. Each neuron receives signals output from other neurons, substitutes numerical information indicated by the input signal into a predetermined arithmetic expression within the neuron, and outputs the result of the arithmetic expression as a signal. The arithmetic expression is composed of a variable part and a constant, and numerical information indicated by the input signal is substituted into the variable part. The constant part is used for weighting coefficients and threshold values and is determined in advance by learning. Specifically, some arbitrary pattern information is given to the neural network in advance, and the value of the constant part in the arithmetic expression of each neuron is determined by learning so that the output signal pattern of the neural network becomes a desired one.

[0003] As such neural networks, a hierarchical neural network in which neurons are connected in a hierarchical manner and a mutually coupled neural network in which specific neural networks are interconnected are known.

Conventionally, in this type of neural network, the signal patterns processed were made up of real numbers.

For example, regarding hierarchical neural networks, Rumelhart, D. E. ;Para
llel Distributed Processes
sing, Chap. 8, Vol. 1.The MI
T Press, 1986. (Supervised by Shunichi Amari: P
DP Model: Exploration of Cognitive Science and Neuronal Networks, Part 8
Chapter, Sangyo Tosho, 1989); Hopfield, J. J. :N
ural network and phys
ical systems with emerge
nt collective computation
al abilities, Proc. Natl. A
cad. Sci. USA, vol. 79, 1982. There is.

[0006]

[Problems to be Solved by the Invention] However, in conventional neural networks, the operations performed by neurons are based on real numbers, so complex number operations (also called vector operations), that is, two Neurons cannot perform arithmetic processing on input patterns consisting of more than one type of numerical information. As a result, there has been a problem that conventional neural networks cannot process pattern information represented by complex numbers.

SUMMARY OF THE INVENTION In view of the above, an object of the present invention is to provide a neural network that can process complex numbers with a simple circuit configuration.

[0008]

[Means for Solving the Problems] In order to achieve the above object, the present invention provides a neural network in which a plurality of neurons having a learning function are connected in a network manner, when an input pattern is represented by a complex number. , the apparatus is characterized in that, in place of each of the neurons, an arithmetic means for performing arithmetic processing on complex numbers is provided.

The present invention provides a neural network in which a plurality of neurons having a learning function are connected like a network, in which each of the neurons has a separation function that separates a complex number inputted to the neuron in the form of a signal into a real part and an imaginary part. means, a plurality of arithmetic processing means for performing arithmetic processing according to predetermined arithmetic expressions for the separated real part and imaginary part, and converting the result of the arithmetic processing performed by the plurality of arithmetic processing means into a complex number. It is characterized by comprising a synthesis means for synthesizing.

0010

In the present invention, complex number operations are executed by a dedicated arithmetic processing device such as a computer instead of each neuron so that a conventional neural network can be used. Furthermore, when performing complex number operations, we focused on the fact that the real and imaginary parts can be separated and executed separately. and the imaginary part, separate calculations are performed in each calculation means, and the results of the calculations are combined.

[0011]

Embodiments Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows the basic configuration of an embodiment of the present invention.

In FIG. 1, neural network 1
Inside, a plurality of neurons 11 to 1n are connected like a network. Each neuron can be a conventional neuron with a learning function. However, since a conventional neuron can only perform calculations on real numbers, in this embodiment, the calculation means 2 capable of calculating complex numbers executes the calculation processing on complex numbers that the neuron should perform.

The operation of the embodiment of the present invention will be explained below with reference to the flowchart shown in FIG.

First, in step 21, one learning pattern is input to the neural network 1. Here, the learning pattern consists of an input learning pattern and an output learning pattern, and the output learning pattern is
When an input learning pattern is input to the neural network 1, it represents a pattern that is desired to be output from the neural network 1. It is assumed that the input learning pattern and the output learning pattern are complex number patterns. A complex number pattern is a pattern made of complex numbers, and a pattern is a vector whose constituent elements are numerical values. For example, (a1...an) is a complex number pattern (however, ai is a complex number). Furthermore, in step 21, the neural network 1 inputs complex numbers that are constituent elements of the input learning pattern to each neuron 11, . . . , neuron 1n, and stores the output learning pattern. Next, in step 22, each neuron 11
,..., the neurons 1n transmit the input complex numbers and the real parts of weighting coefficients and threshold parameters managed by each neuron to the calculation means 2. At this time, the calculation means 2 predetermines the complex numbers output by each neuron 11,..., neuron 1n from the imaginary part of the stored parameter, the complex number obtained from each neuron 11,..., neuron 1n, and the real part of the parameter. , and is transmitted to each neuron 11, . . . , neuron 1n. Each neuron 11, . . . , neuron 1n outputs the complex number obtained from the calculation means 2 to other neurons 11, . The neurons on the downstream side transmit the complex numbers input from the neurons on the upstream side and the real parts of the parameters managed by each neuron to the calculation means 2. The calculation means 2 calculates the complex numbers output by each neuron 11,..., neuron 1n from the imaginary part of the stored parameter, the complex number obtained from each neuron 11,..., neuron 1n, and the real part of the parameter, and further calculates the complex number output by each neuron 11,..., neuron 1n. tell each neuron. This interaction between neurons is repeated multiple times (k times in this case). Next, in step 23, if it is determined that there are no more learning patterns to be input to the neural network 1, the process proceeds to step 24, and if there are learning patterns, the process returns to step 21. In step 24,
When the output values of all neurons are determined, the neural network 1 transmits the stored output learning pattern and the output values of each neuron 11, . . . , neuron 1n to the calculation means 2. The calculation means 2 calculates the error between the obtained output learning pattern and the output value of each neuron 11, . . . , neuron 1n, and transmits it to the neural network 1. At this time, the neural network 1 determines that learning is complete when the obtained error is smaller than a predetermined threshold. If learning is completed, proceed to step 26. If the learning is not completed, in step 25, the neural network 1 transmits the error and the real part of the parameters stored in each neuron 11, . . . , neuron 1n to the calculation means 2. At this time, the calculation means 2 uses the obtained error, the real part of the parameter stored in each neuron 11,..., neuron 1n, and the imaginary part of the parameter stored in the calculation means 2 itself to create a new value for the parameter. is determined, the imaginary part of the parameter stored in the calculation means 2 itself is updated, and the new value of the real part of the parameter is sent to each neuron 11.
,..., is transmitted to neuron 1n. Each neuron 11,...
, neuron 1n updates the real part of the parameter with the obtained new value of the real part of the parameter. After that, the process returns to step 21.

In step 26, one complex pattern, which is not necessarily the same as the input learning pattern, is input to the neural network 1. At this time, the neural network 1 converts the complex numbers that are the constituent elements of the input complex number pattern to be recognized into the neurons 11,...,
Input to neuron 1n. Next, at step 27, the neurons 11, . The calculation means 2 calculates a complex number to be output to the neurons on the downstream side from the stored imaginary part of the parameter, the real part of the parameter, and the complex number pattern, and transmits the calculation result to each neuron on the downstream side. By repeating this process, the neural network 1 outputs an output value as if the neuron were performing complex number calculations. Next, in step 28, if there is no other complex number pattern to be input to the neural network 1, the process ends, and if there is another complex number pattern, the process returns to step 26.

A specific example will be shown below. Here, a neural network shown in FIG. 3 is used. The neural network shown in FIG. 3 is a limited version of the neural network shown in FIG. 1, and is a so-called hierarchical neural network. Connections between neurons may be unidirectional, or there may be no connections at all. This means that information flows in one direction, or that there is no information flow in both directions. This can be thought of as follows. In other words, there is no flow of information,
This corresponds to the fact that the calculation means 2 gives 0 as the output value according to the connections between the neurons. As a result, information flows in one direction or in both directions. Also, complex number patterns input to the neural network 31 (including input learning patterns)
Assume that all except one element are 0 (for example, (a0...0)). It is assumed that the non-zero complex number is input to the neuron 311. Further, although the complex number pattern output by the neural network 31 is generally made up of a plurality of complex numbers, here, we will focus only on the complex number corresponding to the neuron 31n. Accordingly, the neural network 31 inputs one complex number and outputs one complex number. Also, in the explanation using Figure 2 above, the exchange between neurons was repeated k times to obtain the output value of the neural network, but
We'll do it twice here. In order to make a comparison with a conventional neural network, a neural network 41 shown in FIG. 4 is used. The neural network 41 shown in FIG. 4 is a conventional neural network that does not have arithmetic means and handles real number patterns, and can be considered to be a two-input, two-output neural network. That is, neuron 411 and neuron 412 are input neurons, and neuron 41 (n-1) and neuron 41n are output neurons. Let's input the real part of one complex number to neuron 411, input its imaginary part to neuron 412, output the real part of one complex number from neuron 41(n-1), and output its imaginary part from neuron 41n. That is. Generally, a complex number is 2
It represents a certain point on the dimensional x-y plane. In other words,
The real part of a complex number corresponds to the x-axis coordinate, and the imaginary part corresponds to the y-axis coordinate. Therefore, the complex numbers representing the black circles (input learning pattern) and white circles (output learning pattern) in FIG. 5 were defined as one learning pattern. In addition, in FIG.
Re represents the real part and Im represents the imaginary part, and the same applies to Re and Im in FIGS. 6 and 7. For example, assume that the point 0.0+1.0i is an input learning pattern, and the corresponding output learning pattern is -1.0+0.0i. The point obtained by rotating 0.0+1.0i 90 degrees in the positive direction (counterclockwise) around the origin is -1.0+0.0i
corresponds to In other words, this learning pattern represents the rotation of a point around the origin. A set of such points is 1
This means that you should have one ready. Furthermore, complex numbers representing the black triangular points in FIG. 5 were given as input patterns. Prepare seven such points. Such learning patterns and input patterns are shown in Figures 3 and 4.
The points shown in the black squares and white squares in FIG. 5 are the results obtained by applying the signals to the neural network shown in FIG. 2 and processing them according to the flowchart shown in FIG. The black square points are the output patterns of the conventional neural network, and the white square points are the output patterns of the present neural network. Incidentally, the points indicated by white triangles are points obtained by rotating the input pattern (black triangular points) by 90 degrees in the positive direction around the origin. The white square points are the points obtained by rotating the input learning pattern (black triangular points) approximately 90 degrees in the positive direction around the origin.
This is not the case with the black square points, which are close to the output learning pattern (white circle points). In other words, this neural network can memorize rotation angles, which is impossible with conventional neural networks. Similarly, FIG. 6 shows the results of 1/2 reduction of the points with respect to the distance from the origin. For example, the learning pattern is such that an input learning pattern 1.0+1.0i corresponds to an output learning pattern 0.5+0.5i. FIG. 7 shows the results regarding parallel movement. For example, the learning pattern is such that an input learning pattern of 0.0+1.0i corresponds to an output learning pattern of 0.5+0.5i. Both clearly demonstrate that this neural network can do things that were impossible with conventional neural networks.

In order to make the operation of such a neural network easier to understand, an example of calculation of complex numbers executed in the neural network shown in FIG. 8 will be explained.

The real parts of the parameters that each neuron currently has at the learning stage are as follows.

[0020] First neuron: has no parameters. Second neuron: has no parameters. Third neuron: Re[w31]=0.3, Re[w32]
=0.7, Re[θ3]=0.2 Fourth neuron: Re[v43]=0.4, Re[γ4]=0.
9 5th neuron: Re[v53]=0.6,
Re[γ5]=0.8 The imaginary parts of the parameters stored in the calculation means are as follows.

Im[w31]=0.2, Im[w32]=0
．． 4, Im[θ3]=0.9
...Im[v43 corresponding to the parameter of the third neuron
]=0.7, Im[γ4 ]=0.1
…
Im[ corresponding to the parameters of the fourth neuron
v53]=0.3, Im[γ5]=0.6

…corresponding to the parameters of the fifth neuron, where:
Re[z] and Im[z] represent the real part and imaginary part of the complex number Z, respectively. vij and wij represent so-called connection weights between neurons, and γi and θi represent so-called threshold values possessed by each neuron.

In this state, the learning pattern {(0.1
+0.5i, 1.0i), (1.0, 0.2+0.7i
)} is input to the neural network. The neural network remembers the output learning pattern. On the other hand, the input learning pattern is input to the first and second neurons. In other words, 0.1+0.5i is for the first neuron,
1.0i is input to the second neuron. The first neuron uses the input complex number 0.1+0.5i and the real part of the parameter that the first neuron has (in this case, it does not have it, but it does not matter if it has an appropriate parameter) as a calculation means. tell. The calculation means is the given complex number 0
．． The output value of the first neuron is 0.1+0 from 1+0.5i, the real part of the parameter, and the imaginary part of the parameter (in this case, you don't have it, but it doesn't matter if you have an appropriate value).
．． 5i (0.1+0.5i is the complex number itself input to the first neuron. In other words, the first neuron outputs the input complex number as it is, but if we express it as passing through a calculation means, It will look like this). 0.1+0.5i is transmitted to the first neuron, and the first neuron outputs 0.1+0.5i toward the third neuron (the connection between neurons is only toward the third neuron, so it is transmitted to other neurons. (no output). Similarly to the above, the second neuron outputs the input complex number 1.0i as is only to the third neuron. The complex numbers 0.1+0.5i and 1.0i will be input to the third neuron. The third neuron is
These 0.1+0.5i and 1.0i and the real parts Re[w31], Re[w32], and Re[θ3] of the parameters are transmitted to the calculation means. The calculation means calculates the imaginary parts Im[w31], Im[w32], I of the parameters managed by the management means.
Using m[θ3] (only the one corresponding to the third neuron), the complex number output by the third neuron is determined according to the following formula.

[0023]

[Math 1]

[0024]

[Formula 2] z=(Re[w31]+iIm[
w31]) (0.1+0.5i)
+(Re[w32]+iIm[w32])1.
0i + (Re[θ3]
+iIm[θ3]) =w3
1(0.1+0.5i)+w321.0i+θ3 When specifically calculated according to Equations 1 and 2, z=-0.
27+1.77i, so f(z)=0.43+0.85i.

This 0.43+0.85i is transmitted to the third neuron. The third neuron outputs this complex number according to the connections between neurons. In this case, the connections between neurons are only to the 4th and 5th neurons, so there is no connection between the 4th and 5th neurons.
．． Outputs 43+0.85i. For this reason, the fourth and fifth
The complex number 0.43+0.85i will be input to the neuron.

The fourth neuron receives the input complex number 0.4
3+0.85i and the real part of the parameter Re[v43], R
e[γ4 ] is transmitted to the calculation means. The calculation means calculates the imaginary parts of the stored parameters Im[v43], Im[γ4
] to determine the complex number output by the fourth neuron according to Equation 1. In this case, the number corresponding to number 2 is

[
0027

[Formula 3] z=(Re[v43]+iIm[
v43]) (0.43+0.85i)
+(Re[γ4]+iIm[γ4])
=v43(0.43+0.8
5i)+γ4. Specifically calculated, z=0.48+0.74
i, so f(z)=0.62+0.68i.

The calculation means transmits 0.62+0.68i to the fourth neuron, and the fourth neuron outputs 0.6
It will have 2+0.68i. The fifth neuron is also processed in the same manner as described above. In other words, the input complex number 0.43+0.85i and the real part Re[v53
], Re[γ5 ] to the calculation means. The calculation means are
The imaginary parts of the stored parameters Im[v53], Im[
γ5] to determine the complex number output by the fifth neuron according to Equation 1. In this case, the corresponding number 2 is

[0029]

[Formula 4] z=(Re[v53]+iIm[
v53]) (0.43+0.85i)
+(Re[γ5]+iIm[γ5])
=v53(0.43+0.8
5) i+γ5. Specifically calculated, z=0.8+1.24i
Therefore, f(z)=0.69+0.76i.

[0030] The calculation means calculates 0.69+0.76i as the fifth
The fifth neuron outputs 0 as the output value.
．． It will have 69+0.76i.

From the above, the output values of the fourth and fifth neurons are 0.62+0.68i and 0.69+0, respectively.
．． It became 76i. Therefore, the neural network uses these two complex numbers and the output learning pattern (1.0 0.
2+0.7i) to the calculation means. The calculation means calculates the error 1 between the output learning pattern and the two complex numbers according to the following equation.

[0032]

[Equation 5] δk = Tk - Ok (k = 4, 5) However,
k is the neuron number, Tk is the corresponding element of the output learning pattern for neuron k, and Ok is the output value of neuron k. Specifically, δ4 = 0.38-0.68i,
δ5=-0.49-0.06i. Next, the error 2 is determined according to the following formula.

[0033]

[Math 6]

When specifically calculated, E=0.84. This error 1 and error 2 are transmitted to the neural network. Neural network has an error of 2 E=0.
84, and determines whether learning is complete based on whether or not the value is greater than a threshold value of 0.5, for example. In this case, 0.84>
Since it is 0.5, it is determined that learning is not completed. At this time, the neural network has an error of 1 (δ4 and δ5)
and transmits all the real parts of the parameters possessed by the third to fifth neurons to the calculation means. The calculation means determines the new value of the parameter according to the following formula.

[0035]

[Formula 7] vkj (new) = vkj (o
ld) +ΔHj(γk(new)−γk(old))

[0036]

[Formula 8] γ(new) = γ(old) +ε{Re[δk
](1-Re[Ok])Re[Ok]
+iIm[δk] (1-Im[Ok])I
m[Ok]}

[0037]

[Formula 9] wji(new) = wji(o
ld) +ΔIi (θj(new)−θj(old)
)

[0038]

[Math. 10]

Here, Δ is the conjugate complex number of the complex number following this symbol, ε is a sufficiently small appropriate integer (for example, ε=0.5),
Ii is the input to neuron i (i=1, 2), Hj is the output value of neuron j (j=3), and Ok is the output value of neuron k (k=4, 5).

The calculation means updates the imaginary part value with the imaginary part value of the new parameter obtained by equations 7 to 10, and transmits the real part to the neural network. The neural network updates the real part of the parameter of each neuron with the given value. After that, the learning pattern input at the beginning is input into the neural network, and the same process is repeated. Then, when learning for that learning pattern is completed, the same process is performed for other learning patterns.

Next, a neural network capable of processing complex number information in a second embodiment will be explained.

In this example, the neurons connected to the network are themselves capable of performing complex number operations. The circuit configuration of this neuron is shown in FIG. In FIG. 9, a separation circuit 1A separates a real part and an imaginary part from a complex signal of an upstream neuron or an input pattern. Arithmetic circuit 1
B executes a predetermined arithmetic expression (the arithmetic expression related to the separated real part in Equations 1 and 2 of the first embodiment) on the real part separated in the separation circuit 1A. The arithmetic circuit 1C executes an arithmetic expression related to the separated imaginary part in a predetermined arithmetic expression (Equations 1 and 2 in the first embodiment) for the imaginary part separated in the separation circuit 1A. Synthesis circuit 1
D creates a complex number by combining the calculation results of the calculation circuits 1B and 1C, and outputs a signal. With such a configuration, complex number calculations for complex numbers made up of two types of input numerical information are divided and performed by the arithmetic circuits 1B and 1C.

When performing learning processing, the arithmetic circuits 1B,
Variable setting of the parameters on the formula of 1C is the same as in the first embodiment. Incidentally, a computer processing procedure when such a neuron is realized by a computer is shown in FIG. 10 for reference.

In addition to this embodiment, the following examples can be given.

1) The neural networks of the first and second embodiments can be implemented using either analog circuits or digital circuits. Further, in the case of the first embodiment, it is possible to connect a conventional analog-configured neural network with arithmetic means such as a computer that performs digital processing. In this case, it goes without saying that analog-to-digital conversion is performed when exchanging information between the neural network and the calculation means.

2) When representing a complex number with an analog signal,
For example, it is conceivable that the voltage value of an analog signal represents a real number, and the current value represents an imaginary number. Further, when a complex number is represented by a digital signal, it is preferable to allocate a plurality of digital signal lines exclusively for the real part and the imaginary part.

3) As information processing performed using complex numbers in a neural network, in addition to arithmetic processing such as vector rotation processing as in this embodiment, it is also possible to perform vector pattern recognition.

[0048]

[Effects of the Invention] As explained above, according to the present invention, since complex numbers can be processed, vector calculations and pattern recognition related to vectors, which could not be realized with conventional devices, are possible, and the neural This has the effect of expanding the field of use of the network.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the basic configuration of an embodiment of the present invention.

FIG. 2 is a flowchart showing the processing procedure of the neural network according to the embodiment of the present invention.

FIG. 3 is a configuration diagram showing an example of neuron connections in the embodiment of the present invention.

FIG. 4 is a configuration diagram showing an example of neuron connections in the embodiment of the present invention.

FIG. 5 is an explanatory diagram showing the relationship between input patterns and output patterns of the neural network according to the embodiment of the present invention.

FIG. 6 is an explanatory diagram showing the relationship between input patterns and output patterns of the neural network according to the embodiment of the present invention.

FIG. 7 is an explanatory diagram showing the relationship between input patterns and output patterns of the neural network according to the embodiment of the present invention.

FIG. 8 is a configuration diagram showing an example of neuron connections in the embodiment of the present invention.

FIG. 9 is a block diagram showing the circuit configuration of a neuron in a second embodiment of the present invention.

FIG. 10 is a flowchart showing a processing procedure for implementing the arithmetic processing performed by the neuron in FIG. 9 on a computer.

[Explanation of symbols]

1, 31, 41 Neural network 1A Separation circuit 1B Arithmetic circuit 1C Arithmetic circuit 1D Synthesizing circuit 11-1n Neuron 311-31n Neuron 411-41n Neuron

Claims (2)

[Claims] 1. In a neural network in which a plurality of neurons having a learning function are connected in a network manner, when an input pattern is represented by a complex number, an arithmetic means for performing arithmetic processing on the complex number in place of each of the neurons. A neural network characterized by the following. 2. In a neural network in which a plurality of neurons having a learning function are connected in a network manner, each of the neurons is provided with separating means for separating a complex number inputted to the neuron in the form of a signal into a real part and an imaginary part. , a plurality of arithmetic processing means that perform arithmetic processing according to predetermined arithmetic expressions for the separated real part and imaginary part, respectively, and combine the results of the arithmetic processing performed by the plurality of arithmetic processing means into a complex number. A neural network characterized by comprising a synthesis means.